Compare commits
8 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 4c8700e5a4 | |||
| 648fcb5d85 | |||
| 61043dea1b | |||
| 44a7049860 | |||
| 7dfd3cdc6c | |||
| c026c8930b | |||
| 32edb19c96 | |||
| ab1289a9d9 |
@@ -1,11 +1,11 @@
|
||||
{
|
||||
"phase": 0,
|
||||
"phase": 4,
|
||||
"stage": "complete",
|
||||
"milestone": "v0.3",
|
||||
"phase_role": "pre_execution",
|
||||
"phase_role": "execution",
|
||||
"project": "atelier",
|
||||
"attempts": 0,
|
||||
"updated_at": "2026-08-05T03:17:00Z",
|
||||
"phase_tag": "v0.2.0",
|
||||
"release_id": 468
|
||||
"updated_at": "2026-08-05T03:55:00Z",
|
||||
"phase_tag": "v0.2.4",
|
||||
"release_id": 472
|
||||
}
|
||||
@@ -85,10 +85,10 @@ NFR milestone: no separate minor tag. The final patch (v0.1.5) IS the v0.2 deliv
|
||||
| Phase | Name | Type | Status | Key Deliverables |
|
||||
|-------|------|------|--------|------------------|
|
||||
| 0 | Pre-Execution | docs | complete | Spec, clarify, research, ideate, plan, PERSONAS.md (extends platform-engineer, adds ml-engineer) — shipped v0.2.0 |
|
||||
| 1 | GitOps + Operators Domain | docs | pending | domains/gitops-operators/{first-principles, argocd, flux, operators, progressive-delivery}.md |
|
||||
| 2 | AI/ML Domain | docs | pending | domains/ai-ml/{first-principles, data-versioning, model-evaluation, serving, monitoring-drift}.md |
|
||||
| 3 | i18n + Compliance Domains | docs | pending | domains/i18n/{first-principles, locale-resources, formatting, rtl-bidi, testing-i18n}.md, domains/compliance/{first-principles, audit-logs, data-retention, policy-as-code, evidence}.md |
|
||||
| 4 | Matrix + Review Integration | docs | pending | matrix/principles-matrix.md (+40 mappings), matrix/domain-coverage.md (incl. C-rule coverage table update), review/{agent-checklist, peer-review-checklist, anti-patterns}.md, MANIFEST.md (+ examples/ listing per ATELIER-91) |
|
||||
| 1 | GitOps + Operators Domain | docs | complete | domains/gitops-operators/{first-principles, argocd, flux, operators, progressive-delivery}.md — shipped v0.2.1 |
|
||||
| 2 | AI/ML Domain | docs | complete | domains/ai-ml/{first-principles, data-versioning, model-evaluation, serving, monitoring-drift}.md — shipped v0.2.2 |
|
||||
| 3 | i18n + Compliance Domains | docs | complete | domains/i18n/{first-principles, locale-resources, formatting, rtl-bidi, testing-i18n}.md, domains/compliance/{first-principles, audit-logs, data-retention, policy-as-code, evidence}.md — shipped v0.2.3 |
|
||||
| 4 | Matrix + Review Integration | docs | complete | matrix/principles-matrix.md (+40 mappings), matrix/domain-coverage.md (incl. C-rule coverage table update), review/{agent-checklist, peer-review-checklist, anti-patterns}.md, MANIFEST.md (+ examples/ listing per ATELIER-91) — shipped v0.2.4 |
|
||||
| 5 | Examples + Cross-Links | docs | pending | examples/good + examples/bad for 4 domains, cross-links to devops/security/observability/data/k8s/iac |
|
||||
| 6 | Final Review + Ship | docs | pending | Review passed, audit clean, milestone merged to main, tag v0.2.6 |
|
||||
|
||||
|
||||
+35
-5
@@ -36,13 +36,43 @@
|
||||
| DevOps | ✓ | ci-cd, environments |
|
||||
| Infrastructure as Code | ✓ | terraform, opentofu, state, modules |
|
||||
| Kubernetes | ✓ | workloads, networking, storage, rbac, helm, kustomize |
|
||||
| GitOps + Operators | ✓ | argocd, flux, operators, progressive-delivery |
|
||||
| AI / ML | ✓ | data-versioning, model-evaluation, serving, monitoring-drift |
|
||||
| i18n | ✓ | locale-resources, formatting, rtl-bidi, testing-i18n |
|
||||
| Compliance | ✓ | audit-logs, data-retention, policy-as-code, evidence |
|
||||
|
||||
## Examples
|
||||
|
||||
> Examples are illustrative markdown with fenced code only (no standalone runtime artifacts per D-020 / D-025). The `examples/` directory listing closes the v0.2 ESC-002 drift (IDEATE-17, ATELIER-91). P5 authored the v0.3 examples and promoted all entries from `pending` to `✓` (verified — every listed file exists).
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `examples/good/` | ✓ | Good-example directory — 8 examples (v0.1 + v0.2 + v0.3) |
|
||||
| `examples/bad/` | ✓ | Bad-example directory — 7 examples (v0.1 + v0.2 + v0.3) |
|
||||
| `examples/good/api-endpoint.md` | ✓ | v0.1 example — good REST endpoint |
|
||||
| `examples/good/react-component.md` | ✓ | v0.1 example — good React component |
|
||||
| `examples/good/db-schema.md` | ✓ | v0.1 example — good DB schema |
|
||||
| `examples/good/error-handler.md` | ✓ | v0.1 example — good error handler |
|
||||
| `examples/bad/god-object.md` | ✓ | v0.1 example — bad god object |
|
||||
| `examples/bad/silent-error.md` | ✓ | v0.1 example — bad silent error |
|
||||
| `examples/bad/leaky-abstraction.md` | ✓ | v0.1 example — bad leaky abstraction |
|
||||
| `examples/good/terraform-module.md` | ✓ | v0.2 example — good IaC module |
|
||||
| `examples/good/k8s-deployment.md` | ✓ | v0.2 example — good k8s deployment |
|
||||
| `examples/bad/terraform-unlocked-state.md` | ✓ | v0.2 example — bad unlocked state |
|
||||
| `examples/bad/k8s-bare-pod-no-resources.md` | ✓ | v0.2 example — bad bare pod |
|
||||
| `examples/good/gitops-pr.md` | ✓ | v0.3 example — good GitOps PR |
|
||||
| `examples/good/ai-ml-reproducibility.md` | ✓ | v0.3 example — good reproducible training run |
|
||||
| `examples/bad/i18n-string-concat.md` | ✓ | v0.3 example — bad i18n string concat |
|
||||
| `examples/bad/compliance-audit-log.md` | ✓ | v0.3 example — bad audit log (P1 + P9 breaches) |
|
||||
|
||||
> **Note:** The `examples/` section was established in P4 with entries pre-listed as `pending P5`. P5 authored the 4 v0.3 examples and promoted all entries to `✓` after verifying every listed file exists on disk. The manifest remains authoritative — unlisted = not part of framework.
|
||||
|
||||
## Cross-Cutting
|
||||
|
||||
| Document | Purpose |
|
||||
|-----------------------------------|----------------------------------|
|
||||
| `matrix/principles-matrix.md` | Maps domain → core principles (13 domains, 130 P-rules post-v0.2) |
|
||||
| `matrix/domain-coverage.md` | Maps core → domains; per-domain coverage |
|
||||
| `review/agent-checklist.md` | Pre-completion agent checklist (incl. IaC + k8s triggers) |
|
||||
| `review/peer-review-checklist.md` | Human peer-review checklist (incl. IaC + k8s sections) |
|
||||
| `review/anti-patterns.md` | Catalog of violations (incl. IaC + k8s + chaos anti-patterns) |
|
||||
| `matrix/principles-matrix.md` | Maps domain → core principles (17 domains, 170 P-rules post-v0.3) |
|
||||
| `matrix/domain-coverage.md` | Maps core → domains; per-domain coverage (incl. v0.3 Core Principle Coverage) |
|
||||
| `review/agent-checklist.md` | Pre-completion agent checklist (incl. IaC + k8s + gitops + ai-ml + i18n + compliance triggers) |
|
||||
| `review/peer-review-checklist.md` | Human peer-review checklist (incl. IaC + k8s + gitops + ai-ml + i18n + compliance sections) |
|
||||
| `review/anti-patterns.md` | Catalog of violations (incl. IaC + k8s + gitops + ai-ml + i18n + compliance + v0.3 chaos anti-patterns) |
|
||||
@@ -0,0 +1,90 @@
|
||||
# Data Versioning — Derived Rules
|
||||
|
||||
> Derives from `domains/ai-ml/first-principles.md`. Covers P2 (Data is
|
||||
> Versioned, Not Just Code) and P3 (Lineage is Traceable End-to-End).
|
||||
> Referenced by `serving.md` and `monitoring-drift.md`. Scope per
|
||||
> D-023: engineering discipline of versioning data, not dataset
|
||||
> content design.
|
||||
|
||||
## Why Data Versioning (P2 Data is Versioned, Not Just Code)
|
||||
|
||||
- `git` versions code well and data badly. Datasets do not fit in
|
||||
git, and a dataset is not recovered from a commit hash.
|
||||
- A model trained on "the data" is a model trained on an unknown
|
||||
input — a C1 (Correctness) violation. The dataset is a build
|
||||
input; it is named, hashed, and recoverable the way any build
|
||||
input is.
|
||||
- Data versioning is the ML analogue of `domains/data/migrations.md`:
|
||||
the schema and contents of the data evolve, every evolution is a
|
||||
versioned migration, and every model points at a specific version.
|
||||
|
||||
## Dataset Hashing and Lineage (P3 Lineage Traceable End-to-End)
|
||||
|
||||
- Every dataset version has a content hash (not a filename or a
|
||||
timestamp). The hash is the identity. A model's lineage record
|
||||
names the dataset hash it was trained on; a serving prediction
|
||||
names the model digest it came from.
|
||||
- Lineage is a graph: prediction → model → training run → dataset →
|
||||
source(s). Any edge missing is an orphan (`domains/observability/logging.md`
|
||||
for the structured-log angle on lineage events).
|
||||
- The lineage record is append-only. Editing it to "fix" a broken
|
||||
trace is the same class of violation as editing an audit log.
|
||||
|
||||
## Train/Val/Test Split Versioning (P2, P4 Eval Defined Before Training)
|
||||
|
||||
- Splits are versioned with the dataset, not derived ad-hoc per run.
|
||||
A split is a deterministic function of (dataset version, split
|
||||
config, random seed). Two runs on the same pinned inputs produce
|
||||
the same splits.
|
||||
- The eval split is held out and never touched by training. A "held
|
||||
out" set that leaked into training is a P4 (Evaluation Defined
|
||||
Before Training) violation, not just a P2 violation — the eval
|
||||
gate is measuring the training set, not the model.
|
||||
- Cross `domains/data/schema-design.md` for the eval input contract:
|
||||
the schema of the eval set is part of the versioned artifact.
|
||||
|
||||
## Tool Comparison (IDEATE-22, D-040)
|
||||
|
||||
| Tool | Versioning Model | Lineage | Best For | Notes |
|
||||
|------|------------------|---------|----------|-------|
|
||||
| DVC | Git-like pointers to content-addressed object store; `.dvc` files in git track data versions | Pipeline DAG in `dvc.yaml`; reproducibility via `dvc repro` | Teams already on git; file/directory datasets; ML pipelines | Treats data like code; shares git's history model. Object store is pluggable (S3, GCS, Azure, SSH) |
|
||||
| Delta Lake | Table format with transaction log (ACID) + time travel via versioned commits; schema enforcement | Time travel queries; lineage via table history + catalog | Large tabular data; lakehouse; streaming + batch on the same table | Not a pipeline tool — pairs with Spark/Trino/Flink. Brings DB guarantees to object storage |
|
||||
| LakeFS | Git-like operations (branch, commit, merge) over object storage itself | Branch model gives isolated, reproducible data branches | Data engineering teams; branch-per-experiment; CI over data | Not a table format — versions objects. Composes with Delta/Iceberg on top |
|
||||
|
||||
- Pick one primary versioning model per platform. Mixing DVC's
|
||||
pointer model with Delta's transaction-log model fragments
|
||||
operational knowledge (C4 Locality).
|
||||
- All three satisfy P2; the choice is which fits the data shape and
|
||||
the team's existing tooling. None is advocated over the others.
|
||||
|
||||
## Reproducibility Contract (P1 Reproducibility is the First Class)
|
||||
|
||||
A reproducible training run records, in one versioned place:
|
||||
|
||||
```
|
||||
run_id: 2026-08-05T09:12:00Z#run-42
|
||||
dataset: s3://ml-data/train@sha256:7f3a...e21
|
||||
splits: dvc.yaml@commit a1b2c4d
|
||||
code: git@a1b2c4d
|
||||
config: configs/train.yaml@commit a1b2c4d
|
||||
environment: ghcr.io/org/train-img@sha256:9c2d...f88
|
||||
eval_spec: configs/eval.yaml@commit a1b2c4d
|
||||
model_digest: registry/model@sha256:b5e1...aa0
|
||||
```
|
||||
|
||||
- Lose any line and the run is anecdote, not evidence.
|
||||
- The record is the lineage root: a prediction cites the
|
||||
`model_digest`, which cites the `run_id`, which cites everything
|
||||
above. This is how P3 (Lineage Traceable End-to-End) is satisfied
|
||||
in practice.
|
||||
|
||||
## What Violates Data Versioning Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| Dataset referenced by `s3://bucket/latest/` | P2 Data is Versioned, Not Just Code |
|
||||
| Splits regenerated with an unpinned seed per run | P2, P4 Evaluation Defined Before Training |
|
||||
| A production model with no dataset hash in its lineage | P3 Lineage Traceable End-to-End |
|
||||
| Editing a lineage record to "clean up" a broken trace | P3 Lineage Traceable End-to-End |
|
||||
| Eval split reachable from the training data path | P4 Evaluation Defined Before Training |
|
||||
| Two platforms versioning the same data with different models | C4 Locality |
|
||||
@@ -0,0 +1,154 @@
|
||||
# AI / ML — First Principles
|
||||
|
||||
> Scope per D-023: this domain covers ML **engineering discipline** —
|
||||
> data versioning, evaluation methodology, serving patterns, and drift
|
||||
> detection. It does **not** cover algorithm design, model architecture
|
||||
> selection, hyperparameter tuning, or model-family comparison. Those
|
||||
> are research choices, not engineering principles, and they have no
|
||||
> derivation in the core C-rules.
|
||||
|
||||
## 1. The Principles
|
||||
|
||||
### P1. Reproducibility is the First Class
|
||||
Every training run is reproducible from pinned data + code + config +
|
||||
environment. An unreproducible run is an unreviewable run: you cannot
|
||||
decide whether a result is correct if you cannot recreate it.
|
||||
Reproducibility is the ML analogue of `domains/devops/P1
|
||||
Reproducibility` and inherits its non-negotiable status. Lose any one
|
||||
of data, code, config, or environment pinning, and the run is
|
||||
anecdote, not evidence.
|
||||
|
||||
### P2. Data is Versioned, Not Just Code
|
||||
Datasets, features, and train/val/test splits are first-class
|
||||
versioned artifacts with content hashes and lineage. `git` alone is
|
||||
insufficient — datasets do not fit in git, and a dataset is not a
|
||||
commit hash. A model trained on "the data" is a model trained on an
|
||||
unknown input, which is a correctness violation. Version data the way
|
||||
you version code: pinned, named, and recoverable.
|
||||
|
||||
### P3. Lineage is Traceable End-to-End
|
||||
Any deployed prediction traces back through model → training run →
|
||||
dataset → source. No orphan models. A model in production with no
|
||||
lineage is a correctness defect: you cannot reason about its failure
|
||||
modes, you cannot roll it back to a known-good dataset, and you cannot
|
||||
tell whether drift is in the model or in the data that built it.
|
||||
Lineage is the audit trail of ML (`domains/observability/logging.md`).
|
||||
|
||||
### P4. Evaluation is Defined Before Training
|
||||
Metrics, splits, and acceptance thresholds are declared a priori, in
|
||||
code, before the model is trained. Cherry-picking metrics post-hoc is
|
||||
a correctness violation: the evaluation is no longer measuring the
|
||||
model, it is rationalizing it. The eval spec is a contract — it is
|
||||
reviewable, it is versioned, and it is the gate the model must pass
|
||||
before it leaves the experiment. This is the ML angle on C2 Clarity:
|
||||
the intent of the model is obvious to its reader because the eval
|
||||
declared it first.
|
||||
|
||||
### P5. Models are Versioned Artifacts
|
||||
A model is a pinned, immutable, registry-tracked artifact with a
|
||||
unique identifier. Never "the latest." A serving endpoint that pulls
|
||||
"latest" is serving an unknown model — its behavior is undefined, its
|
||||
rollback is impossible, and its lineage is broken. The model registry
|
||||
is to models what a container registry is to images
|
||||
(`domains/devops/P7 Immutability`): immutable, addressed by digest,
|
||||
promoted by stage.
|
||||
|
||||
### P6. Serving is Observable
|
||||
Inference latency, throughput, input distributions, and prediction
|
||||
confidence are first-class signals. Silent serving is a bug. A model
|
||||
in production that emits no metrics is a model you cannot operate: you
|
||||
cannot see latency regressions, you cannot see input drift, you cannot
|
||||
see a failing downstream consumer. Observability is designed in, not
|
||||
bolted on (`domains/observability/metrics.md`).
|
||||
|
||||
### P7. Drift is Expected and Detected
|
||||
Data drift, concept drift, and prediction drift are monitored as a
|
||||
matter of course. A drift signal is an incident, not a curiosity. ML
|
||||
systems decay without code changes — the world changes under the
|
||||
model — so "no code changed" is not a defense against a serving
|
||||
regression. Detecting drift is the ML-specific form of C7
|
||||
Observability: you cannot fix a model you cannot see degrading.
|
||||
|
||||
### P8. Inference Inputs are Validated
|
||||
The model's input contract — schema, value ranges, types, and
|
||||
categorical domains — is enforced at the serving boundary.
|
||||
Out-of-contract inputs are rejected, not silently scored. Scoring an
|
||||
out-of-contract input is a correctness violation: the model's output
|
||||
is undefined for inputs outside its training distribution, and
|
||||
returning a number for it is lying to the caller. This is the ML angle
|
||||
on `domains/security/input-validation.md` and inherits C1's
|
||||
non-tradeable status.
|
||||
|
||||
### P9. Pipelines Compose, Notebooks Don't
|
||||
Training and serving flows are composable pipelines with explicit
|
||||
steps, named inputs, named outputs, and contracts between stages.
|
||||
Notebooks are for exploration, not production. A notebook in the
|
||||
serving path is a correctness defect: its state is implicit, its
|
||||
order is human-dependent, and its reproducibility is whatever the last
|
||||
operator remembered. Compose pipelines; keep notebooks in the lab.
|
||||
|
||||
### P10. Rollback Includes the Model
|
||||
A serving rollback restores the prior model artifact, not just the
|
||||
prior code. Promotion is reversible at the model layer. A rollback
|
||||
that redeploys old code but keeps the new model has not rolled back —
|
||||
the model was the thing that regressed. The rollback path must name
|
||||
the prior model digest, the prior dataset version, and the prior eval
|
||||
that cleared it. This is the ML angle on `domains/devops/P4 Rollback
|
||||
First` and `domains/kubernetes/P10 Roll Forward, Roll Back`.
|
||||
|
||||
## 2. Core Principle Trace
|
||||
|
||||
Each AI/ML P-rule derives from one or more core C-rules (C1–C8). The
|
||||
matrix extension lands in P4 of the v0.3 plan; the traces below are
|
||||
authoritative.
|
||||
|
||||
| P-rule | Core | Why |
|
||||
|--------|------|-----|
|
||||
| P1 Reproducibility is the First Class | C1, C5 | Correctness of results; reversibility of runs |
|
||||
| P2 Data is Versioned, Not Just Code | C5, C7 | Reversibility of datasets; observability of data lineage |
|
||||
| P3 Lineage is Traceable End-to-End | C7, C1 | Observability of provenance; correctness of attribution |
|
||||
| P4 Evaluation is Defined Before Training | C1, C2 | Correctness of the eval gate; clarity of a-priori intent |
|
||||
| P5 Models are Versioned Artifacts | C5, C6 | Reversibility of model identity; composability of registry stages |
|
||||
| P6 Serving is Observable | C7 | Observability of inference |
|
||||
| P7 Drift is Expected and Detected | C7, C1 | Observability of degradation; correctness of detection |
|
||||
| P8 Inference Inputs are Validated | C1 | Correctness of the serving boundary (security subset) |
|
||||
| P9 Pipelines Compose, Notebooks Don't | C6, C2 | Composability of stages; clarity of explicit contracts |
|
||||
| P10 Rollback Includes the Model | C5 | Reversibility at the model layer |
|
||||
|
||||
## 3. What Violates These Principles
|
||||
|
||||
| Violation | Principle Breached |
|
||||
|-----------|-------------------|
|
||||
| A training run that cannot be replayed from pinned inputs | P1 Reproducibility is the First Class |
|
||||
| A dataset referenced by a mutable path, not a hash | P2 Data is Versioned, Not Just Code |
|
||||
| A production model with no record of its training data | P3 Lineage is Traceable End-to-End |
|
||||
| Metrics chosen after seeing the results | P4 Evaluation is Defined Before Training |
|
||||
| A serving endpoint that pulls `latest` from the registry | P5 Models are Versioned Artifacts |
|
||||
| A model in production with no latency or throughput metrics | P6 Serving is Observable |
|
||||
| A serving regression dismissed as "no code changed" | P7 Drift is Expected and Detected |
|
||||
| An input with an out-of-range feature scored silently | P8 Inference Inputs are Validated |
|
||||
| A notebook in the serving or training pipeline path | P9 Pipelines Compose, Notebooks Don't |
|
||||
| A rollback that restores code but keeps the regressed model | P10 Rollback Includes the Model |
|
||||
|
||||
## 4. Relationship to Other Domains
|
||||
|
||||
AI/ML is the engineering-discipline layer for model-bearing systems.
|
||||
It borrows the reproducibility, immutability, rollback, and
|
||||
observability disciplines of `domains/devops/` and applies them to
|
||||
the data → model → serving lifecycle. Cross-links are one-directional
|
||||
(per D-026 extended):
|
||||
|
||||
- `domains/devops/P1 Reproducibility` ← P1
|
||||
- `domains/devops/P4 Rollback First` ← P10
|
||||
- `domains/devops/P5 Progressive Delivery` ← P10 (model canary)
|
||||
- `domains/devops/P7 Immutability` ← P5 (model images)
|
||||
- `domains/data/migrations.md` ← P2 (data versioning ↔ migration discipline)
|
||||
- `domains/data/schema-design.md` ← P8 (inference input contract)
|
||||
- `domains/observability/metrics.md` ← P6, P7
|
||||
- `domains/observability/logging.md` ← P3 (lineage)
|
||||
- `domains/security/input-validation.md` ← P8
|
||||
- `domains/security/secrets.md` ← P8 (serving credentials)
|
||||
- `domains/performance/backend.md` ← P6 (serving latency)
|
||||
- `domains/kubernetes/workloads.md` ← P9 (serving on k8s)
|
||||
- `domains/testing/first-principles.md` ← P4 (eval as a gate)
|
||||
- `domains/gitops-operators/first-principles.md` ← P10 (model rollback in a GitOps loop)
|
||||
@@ -0,0 +1,94 @@
|
||||
# Model Evaluation — Derived Rules
|
||||
|
||||
> Derives from `domains/ai-ml/first-principles.md`. Covers P4
|
||||
> (Evaluation is Defined Before Training) and the eval-as-a-gate
|
||||
> discipline. Referenced by `serving.md` (promotion gate) and
|
||||
> `monitoring-drift.md` (online eval). Scope per D-023: evaluation
|
||||
> methodology, not metric math or model-family benchmarks.
|
||||
|
||||
## Evaluation is a Gate, Not a Report (P4 Evaluation Defined Before Training)
|
||||
|
||||
- The eval spec — metrics, splits, thresholds, and pass/fail
|
||||
criteria — is declared in code **before** the model is trained.
|
||||
It is versioned with the data and the code; it is reviewable; it
|
||||
is the contract the model must satisfy to leave the experiment.
|
||||
- Cherry-picking metrics after seeing results is a correctness
|
||||
violation: the eval is no longer measuring the model, it is
|
||||
rationalizing it. The a-priori spec is what makes the eval
|
||||
trustworthy.
|
||||
- This is the ML angle on `domains/testing/first-principles.md` P1
|
||||
(Tests as Specification): the eval declares the model's contract,
|
||||
the model does not declare its own success.
|
||||
|
||||
## The Eval Input Contract (P8 Inference Inputs are Validated, cross `domains/data/schema-design.md`)
|
||||
|
||||
- The eval set has a schema: feature names, types, ranges, and
|
||||
categorical domains. That schema is the same schema the serving
|
||||
boundary enforces (`serving.md`, `domains/security/input-validation.md`).
|
||||
- An eval set whose schema drifted from the serving schema is
|
||||
measuring a different model than the one in production. Schema
|
||||
parity is part of the versioned eval artifact.
|
||||
- Cross `domains/data/schema-design.md`: the eval input contract is
|
||||
a schema-design problem, versioned and reviewed like any schema.
|
||||
|
||||
## Holdout Integrity (P4, P2 Data is Versioned)
|
||||
|
||||
- The held-out eval set is never touched by training, feature
|
||||
selection, or threshold tuning. A "held out" set that influenced
|
||||
any training decision is not held out — it is a third training
|
||||
set, and the eval is measuring memorization.
|
||||
- Splits are versioned with the dataset (`data-versioning.md`).
|
||||
Recreating splits ad-hoc per run breaks comparability across runs.
|
||||
- Reusing a held-out set across many model iterations leaks it
|
||||
incrementally. Rotate or re-split on a cadence; record the
|
||||
rotation in lineage.
|
||||
|
||||
## Offline vs Online Evaluation (P6 Serving is Observable)
|
||||
|
||||
- **Offline eval** runs before promotion: held-out data, pinned
|
||||
model, declared metrics, pass/fail gate. It answers "should this
|
||||
model ship?"
|
||||
- **Online eval** runs after promotion, on live traffic: shadow
|
||||
scoring, A/B, canary metrics. It answers "is this model behaving
|
||||
in production?" It is the bridge to `monitoring-drift.md`.
|
||||
- A model that passed offline and regressed online is not a
|
||||
contradiction — it is a signal that the offline distribution
|
||||
differs from the live one (a P7 drift signal). Both eval layers
|
||||
are required; neither substitutes for the other.
|
||||
|
||||
## Bias and Fairness Checks (Engineering Angle, P4)
|
||||
|
||||
- Bias/fairness checks are part of the a-priori eval spec, not an
|
||||
afterthought. They are metrics with thresholds, declared before
|
||||
training, gated the same as any metric.
|
||||
- This doc covers the **engineering** discipline: the checks are
|
||||
versioned, gated, and recorded in lineage. The choice of which
|
||||
fairness metrics and what thresholds are policy decisions, not
|
||||
engineering principles, and are out of scope here (D-023).
|
||||
|
||||
## Eval-as-a-Gate in the Pipeline (P9 Pipelines Compose)
|
||||
|
||||
- The eval is a pipeline stage with a contract: input = model
|
||||
digest + eval dataset version; output = pass/fail + metric
|
||||
report. It composes with the training stage and the promotion
|
||||
stage.
|
||||
- A promotion that bypasses the eval stage is a P4 violation,
|
||||
regardless of who approved it. The gate is in the pipeline, not
|
||||
in a human sign-off sheet.
|
||||
|
||||
```
|
||||
train -> eval(gate) -> register(promote) -> serve
|
||||
|
|
||||
+-- fail -> abort, no promote
|
||||
```
|
||||
|
||||
## What Violates Evaluation Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| Metrics chosen after seeing the scores | P4 Evaluation Defined Before Training |
|
||||
| Held-out set used in feature selection or threshold tuning | P4, P2 |
|
||||
| Eval schema differs from serving schema | P8 Inference Inputs are Validated |
|
||||
| Promotion by human approval, bypassing the eval stage | P4, P9 Pipelines Compose |
|
||||
| A "passing" model with no online eval in production | P6 Serving is Observable |
|
||||
| Fairness checks added after a model shipped | P4 Evaluation Defined Before Training |
|
||||
@@ -0,0 +1,88 @@
|
||||
# Monitoring & Drift — Derived Rules
|
||||
|
||||
> Derives from `domains/ai-ml/first-principles.md`. Covers P7 (Drift
|
||||
> is Expected and Detected) and the online half of P6 (Serving is
|
||||
> Observable). Referenced by `serving.md` (online eval) and
|
||||
> `model-evaluation.md` (online layer). Scope per D-023: drift
|
||||
> detection methodology, not model retraining architecture.
|
||||
|
||||
## Drift is Expected and Detected (P7 Drift is Expected and Detected)
|
||||
|
||||
- ML systems decay without code changes. The world changes under
|
||||
the model: user behavior shifts, input pipelines change,
|
||||
upstream schemas evolve. "No code changed" is not a defense
|
||||
against a serving regression.
|
||||
- A drift signal is an incident, not a curiosity. It triggers an
|
||||
alert, an investigation, and a decision (retrain, roll back, or
|
||||
accept with a recorded justification). Silent drift is the same
|
||||
class of bug as silent serving (P6).
|
||||
- Cross `domains/observability/metrics.md` for the alerting
|
||||
primitives and `domains/observability/logging.md` for the
|
||||
structured events a drift signal emits.
|
||||
|
||||
## The Three Drift Types (IDEATE-30, D-048)
|
||||
|
||||
| Drift Type | What Changes | Detection Signal | Source of Truth |
|
||||
|------------|--------------|------------------|-----------------|
|
||||
| **Data drift** (input drift) | The distribution of inputs at serving time diverges from the distribution the model was trained on | Statistical distance between the live input distribution and the pinned training-set distribution (e.g., PSI, KL, KS test). Alert on threshold breach | Training dataset hash (`data-versioning.md`) + live input metrics |
|
||||
| **Concept drift** | The relationship between inputs and the target changes — the same input now maps to a different correct output | Ground-truth lag: compare delayed labels against predictions on the same inputs. Rising error rate against a stable input distribution signals concept, not data, drift | Delayed-label feedback stream + prediction log |
|
||||
| **Prediction drift** (output drift) | The distribution of the model's predictions shifts, with no change to inputs | Statistical distance between the live prediction distribution and a pinned baseline prediction distribution. Independent of inputs — catches model-internal regressions and upstream silent changes | Prediction log + baseline prediction snapshot |
|
||||
|
||||
- The three signals are distinct and non-substitutable. Data drift
|
||||
catches the input changing; concept drift catches the world
|
||||
changing; prediction drift catches the model's behavior changing.
|
||||
A monitoring setup with only one is blind to two classes of
|
||||
regression.
|
||||
- Evidently AI and Great Expectations are the canonical tooling:
|
||||
Evidently for drift/statistical reports, Great Expectations for
|
||||
data-quality/contract checks at the pipeline boundary. Both
|
||||
produce the metrics that feed `domains/observability/metrics.md`.
|
||||
|
||||
## Detection Signals in Practice
|
||||
|
||||
- **Data drift** compares live inputs to the **pinned training
|
||||
distribution** — not to "yesterday's inputs." Without a pinned
|
||||
baseline, drift is measured against a moving target and is
|
||||
meaningless. Cross `data-versioning.md` for how the baseline is
|
||||
pinned.
|
||||
- **Concept drift** requires ground truth, which is often delayed
|
||||
(days/weeks). The detection signal is the gap between
|
||||
prediction-time confidence and delayed-label error. A rising
|
||||
error against stable inputs is the signature.
|
||||
- **Prediction drift** needs no ground truth and no input
|
||||
comparison — it watches the model's own output distribution. It
|
||||
is the cheapest signal and the first to fire; it is also the
|
||||
least specific (any of the three drifts can move predictions).
|
||||
|
||||
## Alerting and Retraining Triggers (P7, P10 Rollback Includes the Model)
|
||||
|
||||
- A drift alert is an incident. It does not auto-trigger retraining
|
||||
unsupervised — auto-retraining on drift can lock in a bad
|
||||
distribution. The alert triggers a human decision: investigate,
|
||||
retrain, roll back, or accept.
|
||||
- Retraining is a new training run (`first-principles.md` P1): it
|
||||
produces a new model digest, passes the eval gate
|
||||
(`model-evaluation.md`), and is promoted through the registry
|
||||
(`serving.md`). The prior model stays rollbackable (P10).
|
||||
- Cross `domains/observability/metrics.md` for the alert-rule
|
||||
pattern: threshold + window + severity, routed to the same
|
||||
on-call path as any production incident.
|
||||
|
||||
## Online Evaluation Bridge (P6 Serving is Observable)
|
||||
|
||||
- Online eval (`model-evaluation.md`) is the live counterpart to
|
||||
drift monitoring: shadow scores and A/B canaries measure a
|
||||
candidate model against the incumbent, while drift monitoring
|
||||
measures the incumbent against its own baseline. Both feed the
|
||||
same metrics pipeline.
|
||||
|
||||
## What Violates Monitoring Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| Only one drift type monitored | P7 Drift is Expected and Detected |
|
||||
| Drift baseline is "yesterday's inputs," not pinned training data | P7, P2 Data is Versioned |
|
||||
| Drift alert that auto-retrains without a human gate | P7, P1 Reproducibility |
|
||||
| A serving regression dismissed as "no code changed" | P7 Drift is Expected and Detected |
|
||||
| Concept-drift check with no delayed-label feedback path | P7 Drift is Expected and Detected |
|
||||
| Prediction-distribution change with no alert | P6 Serving is Observable, P7 |
|
||||
@@ -0,0 +1,88 @@
|
||||
# Serving — Derived Rules
|
||||
|
||||
> Derives from `domains/ai-ml/first-principles.md`. Covers P5 (Models
|
||||
> are Versioned Artifacts), P6 (Serving is Observable), P8 (Inference
|
||||
> Inputs are Validated), and P10 (Rollback Includes the Model).
|
||||
> Referenced by `monitoring-drift.md` (online signals) and
|
||||
> `model-evaluation.md` (promotion gate). Scope per D-023: serving
|
||||
> patterns, not model architectures.
|
||||
|
||||
## The Model is an Addressed Artifact (P5 Models are Versioned Artifacts)
|
||||
|
||||
- A serving endpoint pulls a model by digest, never by `latest`. A
|
||||
model pulled by `latest` is an unknown model — its behavior is
|
||||
undefined and its rollback is impossible.
|
||||
- The model registry is to models what a container registry is to
|
||||
images (`domains/devops/P7 Immutability`): immutable, addressed by
|
||||
digest, promoted by stage (staging → prod). Promotion is a
|
||||
registry operation, not a file copy.
|
||||
- A serving rollout names the model digest in its manifest. The
|
||||
digest is part of the deploy's lineage (`data-versioning.md`).
|
||||
|
||||
## Inference Inputs are Validated (P8 Inference Inputs are Validated)
|
||||
|
||||
- The model's input contract — schema, types, ranges, categorical
|
||||
domains — is enforced at the serving boundary, before the model
|
||||
sees the input. Out-of-contract inputs are rejected with a
|
||||
defined error, not silently scored.
|
||||
- Scoring an out-of-contract input is a C1 (Correctness) violation:
|
||||
the model's output is undefined outside its training
|
||||
distribution, and returning a number for it is lying to the
|
||||
caller.
|
||||
- This is the ML angle on `domains/security/input-validation.md`:
|
||||
the validation lives at the boundary, the model is downstream of
|
||||
it, and the contract is versioned with the model.
|
||||
|
||||
## Serving is Observable (P6 Serving is Observable)
|
||||
|
||||
- Every inference path emits: request latency, throughput, input
|
||||
distribution summaries, prediction confidence, and error counts.
|
||||
Silent serving is a bug.
|
||||
- Cross `domains/observability/metrics.md` for the metrics
|
||||
primitives (histograms, counters, gauges) and
|
||||
`domains/observability/tracing.md` for the request-level trace
|
||||
that ties an input to a prediction.
|
||||
- Latency SLAs are enforced via `domains/performance/backend.md`
|
||||
disciplines: budget the inference path, measure the tail (p99),
|
||||
alert on budget breach.
|
||||
|
||||
## Serving Patterns (P9 Pipelines Compose)
|
||||
|
||||
| Pattern | When | Notes |
|
||||
|---------|------|-------|
|
||||
| Inference as a service | Default; model behind an HTTP/gRPC endpoint | KServe, Seldon Core, BentoML. Scales with traffic; model is a deployable, addressable artifact |
|
||||
| Batch inference | Offline scoring of large datasets | No latency SLA; throughput-bound. Same model digest, same input contract |
|
||||
| Embedded / in-process | Latency-critical, single-tenant | Model linked into the app. Trades observability for latency — only when the SLA demands it |
|
||||
|
||||
- Canarying a model is a serving pattern, not a deployment pattern:
|
||||
shift a fraction of traffic to the new model digest, measure
|
||||
online eval (`model-evaluation.md`), abort to the prior digest on
|
||||
regression. This is `domains/devops/P5 Progressive Delivery`
|
||||
applied at the model layer.
|
||||
- Rollback restores the prior model digest (P10 Rollback Includes
|
||||
the Model). A rollback that redeploys old code but keeps the new
|
||||
model has not rolled back. Cross `domains/gitops-operators/first-principles.md`
|
||||
for the GitOps reconciliation loop that drives model rollouts.
|
||||
|
||||
## Tool Landscape (KServe / Seldon Core / BentoML)
|
||||
|
||||
| Tool | Model Packaging | Deployment Surface | Notes |
|
||||
|------|-----------------|--------------------|-------|
|
||||
| KServe | InferenceService CRD; runtime predictors (v2, HuggingFace, PMML, custom) | Kubernetes-native; CRD-driven | Cross `domains/kubernetes/workloads.md`. Brings the k8s reconciliation model to serving |
|
||||
| Seldon Core | SeldonDeployment CRD; graph of predictors | Kubernetes-native; CRD-driven | Emphasizes inference graphs (fan-out, ensemble) as CRD structure |
|
||||
| BentoML | Bento (model + runtime + deps packaged); Yatai registry | Kubernetes or bare container | Focuses on packaging + registry; the Bento is the versioned artifact (P5) |
|
||||
|
||||
- All three satisfy P5/P6/P8 when wired correctly; the choice is
|
||||
packaging model and deployment surface, not correctness.
|
||||
- None is advocated over the others.
|
||||
|
||||
## What Violates Serving Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| Endpoint pulls `latest` from the registry | P5 Models are Versioned Artifacts |
|
||||
| Out-of-range input scored silently | P8 Inference Inputs are Validated |
|
||||
| Serving path emits no latency or throughput metrics | P6 Serving is Observable |
|
||||
| Rollback redeploys code but keeps the regressed model | P10 Rollback Includes the Model |
|
||||
| A notebook in the serving path | P9 Pipelines Compose, Notebooks Don't |
|
||||
| Canary with no abort-to-prior-digest path | P10, `domains/devops/P5 Progressive Delivery` |
|
||||
@@ -0,0 +1,165 @@
|
||||
# Audit Logs — Derived Rules
|
||||
|
||||
> Derives from `domains/compliance/first-principles.md`. Covers P1
|
||||
> (Audit Logs are Append-Only), P2 (Every Significant Action is
|
||||
> Logged), P7 (Identity is Attributable), P9 (Secrets Redacted in
|
||||
> Audit), and P10 (Compliance Posture Observable). Referenced by
|
||||
> `data-retention.md` (retention applies to audit logs themselves)
|
||||
> and `evidence.md` (audit logs are evidence).
|
||||
|
||||
## Audit Logs are Append-Only (P1 Audit Logs are Append-Only)
|
||||
|
||||
- An audit record is immutable once written. The storage substrate
|
||||
enforces this; policy alone does not. Write-once, append-only
|
||||
sinks (WORM buckets, immutable log streams, hash-chained ledgers)
|
||||
are the mechanism.
|
||||
- Deletion or mutation of an audit record is itself an auditable
|
||||
incident. The tampering is the signal, not just the underlying
|
||||
event. A system that allows `DELETE FROM audit_log` is a system
|
||||
whose audit log is a draft.
|
||||
- The append-only guarantee is testable: attempt to write, then
|
||||
attempt to overwrite, then attempt to delete. If the overwrite or
|
||||
delete succeeds, the guarantee is absent and the design is a
|
||||
violation.
|
||||
|
||||
## Structured Audit Events (P2 Every Significant Action is Logged)
|
||||
|
||||
- The set of auditable actions is defined a priori, in code, before
|
||||
the action ships. The catalog is versioned and reviewed. An
|
||||
auditable action with no log line is a violation, not a gap to
|
||||
backfill later.
|
||||
- Audit events are structured (JSON / protobuf / a typed schema),
|
||||
not prose. A prose log line ("user logged in") is unqueryable and
|
||||
unaggregatable; a structured event is both. The event schema is
|
||||
the contract between the producer and the audit pipeline.
|
||||
|
||||
```
|
||||
{
|
||||
"timestamp": "2024-11-07T15:03:22Z",
|
||||
"event": "auth.login",
|
||||
"actor": { "kind": "user", "id": "u_8f3a", "session": "s_12b9" },
|
||||
"action": "succeeded",
|
||||
"target": { "kind": "service", "id": "billing-api" },
|
||||
"source": { "ip": "203.0.113.42", "region": "us-east-1" },
|
||||
"request_id": "req_91c2",
|
||||
"version": "audit-schema/v2"
|
||||
}
|
||||
```
|
||||
|
||||
- The catalog of significant actions typically includes:
|
||||
authentication (success and failure), authorization decisions
|
||||
(allow and deny), data access (read, write, delete), configuration
|
||||
changes, policy changes, retention executions, and admin
|
||||
operations. The exact set is declared per system; the discipline
|
||||
is that it is declared.
|
||||
|
||||
## Cloud Audit Log Conventions (Prior Art, Abstracted)
|
||||
|
||||
- AWS CloudTrail, Google Cloud Audit Logs, and Azure Activity Log
|
||||
share a common shape: immutable, time-ordered, queryable, with
|
||||
actor / action / target / source / result fields. Atelier's
|
||||
audit-logs doc adopts the shape, not the vendor.
|
||||
- The shape is the contract; the sink is the implementation. A
|
||||
self-hosted audit log that follows the same shape composes with
|
||||
the same tooling (SIEM, query engines, evidence exporters) as the
|
||||
cloud vendors'.
|
||||
|
||||
## Queryability (P10 Compliance Posture Observable)
|
||||
|
||||
- An audit log that cannot be queried is an audit log that cannot be
|
||||
used. Queryability is a first-class design goal: the event schema
|
||||
is typed, fields are indexed, and the common queries (who acted on
|
||||
what when, what failed, what was denied) are cheap.
|
||||
- "Who did X between T1 and T2" must be a single query, not a
|
||||
forensics project. If the query requires a custom script per
|
||||
investigation, the audit log is structured for storage, not for
|
||||
use — a C7 (Observability) violation.
|
||||
|
||||
## Identity is Attributable (P7 Identity is Attributable)
|
||||
|
||||
- Every audit event records the authenticated principal that acted —
|
||||
not a shared account, not a generic service, not "admin." The
|
||||
actor field is populated at the time of the action from the
|
||||
authenticated session, not resolved after the fact.
|
||||
- A shared account in the actor field breaks accountability: an
|
||||
event attributed to `svc-deploy` could be any of ten engineers.
|
||||
This is the compliance angle on `domains/security/authorization.md`
|
||||
and `domains/kubernetes/rbac.md`: bind actions to unique
|
||||
principals, not to roles many can assume.
|
||||
- Machine-to-machine actions record the workload identity (a service
|
||||
account, a signed instance identity), not a human — but the
|
||||
identity is still unique and attributable to a deployable unit.
|
||||
|
||||
## Redaction at the Boundary (P9 Secrets Redacted in Audit)
|
||||
|
||||
- Audit logs must not leak secrets, credentials, tokens, or PII.
|
||||
Redaction is structural: applied at the logging boundary, before
|
||||
the record is written to the append-only sink — not opportunistic
|
||||
scrubbing after the fact. Once a secret is in an append-only log,
|
||||
the remediation is expensive (rotate, rewrite access scope), so
|
||||
redaction-at-source is the only sound position.
|
||||
- The redaction policy is itself auditable: which fields are
|
||||
redacted, by what rule, in which event type. A redaction rule
|
||||
that lives in someone's head is a P9 violation waiting to happen.
|
||||
|
||||
```
|
||||
// before redaction (DO NOT LOG)
|
||||
{
|
||||
"event": "config.read",
|
||||
"target": { "kind": "secret", "id": "db-password" },
|
||||
"value": "p@ssw0rd-plaintext-leaked" // VIOLATION
|
||||
}
|
||||
// after structural redaction
|
||||
{
|
||||
"event": "config.read",
|
||||
"target": { "kind": "secret", "id": "db-password" },
|
||||
"value": "[REDACTED:secret]",
|
||||
"redaction": "secret-value-policy/v1"
|
||||
}
|
||||
```
|
||||
|
||||
- Never log request bodies, response bodies, headers like
|
||||
`Authorization`, or environment variables that may carry secrets.
|
||||
Log the *fact* of the action, not the *content* of the secret.
|
||||
|
||||
## Retention of Audit Logs Themselves
|
||||
|
||||
- Audit logs are subject to retention policy (cross `data-
|
||||
retention.md`), but the floor is set by the accountability need,
|
||||
not by storage economy. An audit log deleted before its retention
|
||||
period is a P1 violation dressed as a P3 action.
|
||||
- The retention rule for audit logs is itself logged (meta-audit):
|
||||
when an audit log segment ages out and is deleted, the deletion is
|
||||
recorded in a higher-tier audit log with the rule that authorized
|
||||
it. The chain is observable end to end.
|
||||
|
||||
## What Violates Audit-Log Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| Audit log on a mutable filesystem with no write-once protection | P1 Audit Logs are Append-Only |
|
||||
| `DELETE FROM audit_log WHERE timestamp < ...` as routine cleanup | P1 Audit Logs are Append-Only |
|
||||
| An auth-success event with no audit record | P2 Every Significant Action is Logged |
|
||||
| A prose log line ("user did a thing") instead of a structured event | P2 Every Significant Action is Logged |
|
||||
| A shared `admin` account as the actor in audit events | P7 Identity is Attributable |
|
||||
| An `Authorization: Bearer <token>` header logged in plaintext | P9 Secrets and Sensitive Data are Redacted in Audit |
|
||||
| A redaction rule applied inconsistently across event types | P9 Secrets and Sensitive Data are Redacted in Audit |
|
||||
| "Who did X?" requires a custom forensics script per investigation | P10 Compliance Posture is Observable |
|
||||
| An audit log segment deleted with no meta-audit record | P1 Audit Logs are Append-Only |
|
||||
|
||||
## Relationship to Other Domains
|
||||
|
||||
- `domains/observability/logging.md` — audit logs are structured
|
||||
logging with an append-only guarantee; the logging primitives
|
||||
(levels, structured fields, correlation IDs) compose here.
|
||||
- `domains/security/authorization.md` — the actor in an audit event
|
||||
is the principal the authorization layer authenticated.
|
||||
- `domains/security/secrets.md` — redaction at the logging boundary
|
||||
is the audit-side complement of secret management.
|
||||
- `domains/compliance/data-retention.md` — retention policy applies
|
||||
to audit logs; the audit log's own deletion is meta-audited.
|
||||
- `domains/compliance/evidence.md` — audit logs are a primary
|
||||
evidence artifact; the append-only guarantee is what makes them
|
||||
admissible.
|
||||
- `domains/kubernetes/rbac.md` — workload identity in audit events
|
||||
derives from the RBAC principal that acted.
|
||||
@@ -0,0 +1,159 @@
|
||||
# Data Retention — Derived Rules
|
||||
|
||||
> Derives from `domains/compliance/first-principles.md`. Covers P3
|
||||
> (Retention is Policy, Not Storage) and the data-shape angle on P8
|
||||
> (Subject Access is Honored). Referenced by `audit-logs.md`
|
||||
> (retention applies to audit logs) and `evidence.md` (evidence has
|
||||
> a retention lifecycle). Framework-agnostic per D-024 — no
|
||||
> regulation-specific retention periods.
|
||||
|
||||
## Retention is Policy, Not Storage (P3 Retention is Policy, Not Storage)
|
||||
|
||||
- Data lifetime is declared and enforced as policy, in code — not
|
||||
left to the storage layer's defaults. The policy names what data
|
||||
class is retained for how long, what action fires at end-of-life
|
||||
(delete, archive, anonymize), and what exception path exists (a
|
||||
legal hold suspends deletion).
|
||||
- Deletion at end-of-life is a feature, not a failure. A system that
|
||||
cannot delete on schedule is a system that over-retains, which is
|
||||
the symmetric violation of a system that under-retains. Both are
|
||||
P3 violations; the policy is the arbiter.
|
||||
- "We kept it because the bucket was cheap" is a violation. "We
|
||||
deleted it because the policy said to" is correct. Cost does not
|
||||
override policy; policy is the contract.
|
||||
|
||||
## Retention Policy as Code
|
||||
|
||||
- Retention rules live as code: lifecycle rules on the storage
|
||||
layer, scheduled deletion jobs, tiered storage transitions, and
|
||||
anonymization transforms. The code is versioned, reviewed, and
|
||||
auditable. A retention rule in a spreadsheet is a wishlist; the
|
||||
same rule in a reviewed, deployable lifecycle policy is a control.
|
||||
|
||||
```
|
||||
// illustrative lifecycle policy (abstracted, no vendor DSL)
|
||||
// object-storage lifecycle
|
||||
{
|
||||
"rules": [
|
||||
{
|
||||
"name": "user-events-90d",
|
||||
"match": { "prefix": "events/" },
|
||||
"transitions": [
|
||||
{ "after": "30d", "to": "tier-cold" },
|
||||
{ "after": "90d", "action": "delete" }
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "audit-log-7y",
|
||||
"match": { "prefix": "audit/" },
|
||||
"transitions": [
|
||||
{ "after": "365d", "to": "tier-archive" },
|
||||
{ "after": "2555d", "action": "delete" }
|
||||
],
|
||||
"legal_hold": "suspends-action"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- The retention policy is itself auditable: which rule fired when,
|
||||
against which objects, with what result. The deletion events are
|
||||
logged (`audit-logs.md`) — deletion is a significant action.
|
||||
|
||||
## Retention vs. Backup — The Distinction
|
||||
|
||||
- A **backup** is a recovery mechanism: it exists to restore data
|
||||
after loss. A **retention rule** is a deletion mechanism: it
|
||||
exists to remove data at end-of-life. Conflating them produces
|
||||
data that survives both the deletion policy and the disaster —
|
||||
which is the opposite of compliance.
|
||||
- A backup is governed by a recovery-point / recovery-time objective;
|
||||
a retention rule is governed by a lifetime. They are independent
|
||||
contracts. A backup that is also the retention store is a store
|
||||
where nothing is ever deleted, which is a P3 violation.
|
||||
- A legal hold suspends retention deletion for a defined data set
|
||||
(e.g. data under investigation). The hold is itself a policy
|
||||
action, auditable and time-bounded, not a manual override.
|
||||
|
||||
## Retention is Distinct per Data Class
|
||||
|
||||
- Different data classes have different lifetimes. The retention
|
||||
policy enumerates the classes and their rules; it does not apply
|
||||
one number to everything. Typical classes (the names are
|
||||
abstract; the periods are policy decisions, not regulation-
|
||||
specific):
|
||||
- **Audit logs** — long, often multi-year, governed by
|
||||
accountability needs (`audit-logs.md`).
|
||||
- **User-generated content** — tied to the user's account
|
||||
lifetime; deletion follows account deletion (cross P8 Subject
|
||||
Access).
|
||||
- **Telemetry / metrics** — short, governed by observability need
|
||||
(`domains/observability/metrics.md`); high-resolution data ages
|
||||
to downsampled aggregates.
|
||||
- **Evidence artifacts** — tied to the audit cycle
|
||||
(`evidence.md`); the cycle ends, the evidence ages out.
|
||||
- A single retention rule for "all data" is a C3 (Simplicity)
|
||||
violation of the wrong kind: it is simpler than the requirement
|
||||
allows.
|
||||
|
||||
## Subject Access is Honored (P8 Subject Access is Honored)
|
||||
|
||||
- Data-subject rights — access (what do we have on this subject),
|
||||
export (in a portable form), deletion (and prove it), correction
|
||||
— are operations with defined contracts and audit trails, not
|
||||
ad-hoc tickets. The system implements them as first-class
|
||||
operations; a subject-access request that requires a forensics
|
||||
team is a correctness defect.
|
||||
- Retention and subject access interact at deletion: a subject
|
||||
deletion request fires the deletion policy for that subject's
|
||||
data, the deletion is audited, and the proof of deletion is
|
||||
returned to the subject (and recorded). A subject deletion that
|
||||
skips the audit is a P8 violation dressed as a P3 success.
|
||||
- Cross `domains/data/schema-design.md`: subject access is only
|
||||
computable if the schema tags which records belong to which
|
||||
subject. A schema with no subject linkage cannot honor a subject
|
||||
request — it cannot find the data to delete.
|
||||
|
||||
## Retention Migration Discipline
|
||||
|
||||
- Retention rules change. When the policy changes (a class's
|
||||
lifetime shortens or lengthens), the change is a migration: the
|
||||
new rule applies to data ingested after the cutover, and a
|
||||
backfill applies the new rule to existing data where applicable.
|
||||
Cross `domains/data/migrations.md` for the schema-lifecycle
|
||||
discipline this mirrors.
|
||||
- A retention rule change that is not versioned, not reviewed, and
|
||||
not backfilled is a P3 violation: the policy is not actually the
|
||||
policy if the storage layer does not reflect it.
|
||||
|
||||
## What Violates Retention Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| Data kept indefinitely because "storage is cheap" | P3 Retention is Policy, Not Storage |
|
||||
| A retention rule in a spreadsheet, not in code | P3 Retention is Policy, Not Storage |
|
||||
| A backup bucket used as the retention store (nothing ever deletes) | P3 Retention is Policy, Not Storage |
|
||||
| A single retention period applied to all data classes | P3 Retention is Policy, Not Storage |
|
||||
| A subject deletion with no audit record of the deletion | P8 Subject Access is Honored |
|
||||
| A subject-access request that requires a forensics team | P8 Subject Access is Honored |
|
||||
| A schema with no subject linkage (cannot find data to delete) | P8 Subject Access is Honored |
|
||||
| A legal hold applied ad hoc, not as a policy action | P3 Retention is Policy, Not Storage |
|
||||
| A retention rule change with no backfill to existing data | P3 Retention is Policy, Not Storage |
|
||||
|
||||
## Relationship to Other Domains
|
||||
|
||||
- `domains/data/schema-design.md` — retention requires the schema
|
||||
to tag data class and subject linkage; subject access is only
|
||||
computable over a schema that supports it.
|
||||
- `domains/data/migrations.md` — retention rule changes are
|
||||
migrations; the discipline (version, review, backfill) mirrors
|
||||
schema migrations.
|
||||
- `domains/compliance/audit-logs.md` — audit logs have their own
|
||||
retention floor; deletion of an audit segment is meta-audited.
|
||||
- `domains/compliance/evidence.md` — evidence artifacts have a
|
||||
retention lifecycle tied to the audit cycle.
|
||||
- `domains/observability/metrics.md` — telemetry retention is
|
||||
governed by observability need; high-res data ages to aggregates.
|
||||
- `domains/security/secrets.md` — secrets have a retention lifecycle
|
||||
tied to rotation; a secret past its rotation date is overdue, not
|
||||
retained.
|
||||
@@ -0,0 +1,175 @@
|
||||
# Evidence — Derived Rules
|
||||
|
||||
> Derives from `domains/compliance/first-principles.md`. Covers P6
|
||||
> (Evidence is Collected Continuously), P5 (Policy is a Gate, so
|
||||
> decisions are evidence), P7 (Identity Attributable, so evidence
|
||||
> has provenance), and P10 (Posture Observable, so evidence is
|
||||
> queryable). Referenced by `audit-logs.md` (logs are evidence)
|
||||
> and `data-retention.md` (evidence has a lifecycle).
|
||||
|
||||
## Evidence is Collected Continuously (P6 Evidence is Collected Continuously)
|
||||
|
||||
- Evidence of compliance — logs, configs, scans, attestations,
|
||||
policy decisions, access reviews — is gathered as a byproduct of
|
||||
operation, not assembled manually at audit time. The audit-time
|
||||
scramble is the anti-pattern: it is expensive, it is incomplete,
|
||||
and it produces evidence that is reconstructed rather than
|
||||
recorded.
|
||||
- Continuous evidence collection means the audit packet is a query
|
||||
over already-collected artifacts, not a forensic reconstruction.
|
||||
The auditor asks "show me the access reviews for Q3" and the
|
||||
answer is a query against the evidence store, not a six-week
|
||||
project.
|
||||
- This is the compliance angle on `domains/observability/tracing.md`
|
||||
for distributed evidence (a trace spans the request that produced
|
||||
the evidence) and `domains/observability/metrics.md` for posture
|
||||
signals (a metric is a continuous evidence stream).
|
||||
|
||||
## Evidence is a Byproduct, Not a Deliverable
|
||||
|
||||
- Evidence collected as a byproduct is trustworthy: it records what
|
||||
happened, when it happened, recorded by the system that did it.
|
||||
Evidence assembled at audit time is less trustworthy: it records
|
||||
what someone remembered to write down, when they wrote it, after
|
||||
the fact.
|
||||
- The mechanism: every significant action (`audit-logs.md`) emits
|
||||
its record to an evidence store; every policy decision
|
||||
(`policy-as-code.md`) emits its decision; every deployment emits
|
||||
its signed attestation; every access review emits its result. The
|
||||
store is append-only (`audit-logs.md` P1), queryable (P10), and
|
||||
retention-bound (`data-retention.md`).
|
||||
|
||||
## Provenance and Identity (P7 Identity is Attributable)
|
||||
|
||||
- Evidence has provenance: which system produced it, when, from what
|
||||
input. An evidence artifact with no provenance is anecdote, not
|
||||
evidence — it cannot be attributed to a source, so it cannot be
|
||||
trusted.
|
||||
- Provenance includes the identity of the producer (a workload
|
||||
identity, a service account) and the chain of custody (who has
|
||||
had access to the artifact since it was produced). Cross
|
||||
`domains/security/authorization.md`: the producer's identity is
|
||||
authenticated, not assumed.
|
||||
|
||||
## Signed Attestations (IDEATE-29)
|
||||
|
||||
- A signed attestation is evidence with a cryptographic signature
|
||||
binding the artifact to its producer. The signature is the
|
||||
provenance: it can be verified independently of the producer, and
|
||||
it cannot be forged without the producer's key. Cross
|
||||
`domains/security/supply-chain.md` for the supply-chain angle.
|
||||
- Cosign (Sigstore) and in-toto are the canonical patterns: a
|
||||
builder signs an artifact (container image, deployable, evidence
|
||||
bundle) at production time; a verifier checks the signature at
|
||||
consumption time. The signature is the evidence that the artifact
|
||||
came from where it claims to have come from.
|
||||
|
||||
- **Illustrative signed attestation (Cosign / Sigstore format, NOT
|
||||
a real signature — illustrative only, no live keys):**
|
||||
|
||||
```
|
||||
// Cosign attest — bind an attestation to an image digest
|
||||
// (illustrative; not a real signature)
|
||||
$ cosign attest --type spdxjson \
|
||||
--predicate sbom.spdx.json \
|
||||
my-registry/app@sha256:5a3e1c...f9b2
|
||||
|
||||
// The attestation is stored as a signature in the registry,
|
||||
// bound to the image digest. The payload is a DSSE envelope:
|
||||
|
||||
{
|
||||
"payloadType": "application/vnd.in-toto+json",
|
||||
"payload": "eyJfdHlwZSI6ImF0dGVzdGF0aW9uIn0...",
|
||||
"signatures": [
|
||||
{
|
||||
"sig": "MEUCIQDx...illustrative-base64-signature...==",
|
||||
"keyid": "cosign-key-2024-q4"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
// The decoded payload (an in-toto statement binding the
|
||||
// attestation to the image digest):
|
||||
{
|
||||
"_type": "https://in-toto.io/Statement/v0.1",
|
||||
"predicateType": "https://spdx.dev/Document",
|
||||
"subject": [
|
||||
{
|
||||
"name": "my-registry/app",
|
||||
"digest": { "sha256": "5a3e1c...f9b2" }
|
||||
}
|
||||
],
|
||||
"predicate": {
|
||||
"SPDXID": "SPDXRef-DOCUMENT",
|
||||
"creationInfo": {
|
||||
"created": "2024-11-07T15:03:22Z",
|
||||
"creators": ["Tool: atelier-build-pipeline"]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Verification (independent of the producer):
|
||||
$ cosign verify-attestation --type spdxjson \
|
||||
--certificate-identity-regexp '.*atelier-build.*' \
|
||||
my-registry/app@sha256:5a3e1c...f9b2
|
||||
// Verification succeeded for: my-registry/app@sha256:5a3e1c...f9b2
|
||||
// SBOM attestation found for subject
|
||||
```
|
||||
|
||||
- The attestation is illustrative — the signatures and digests are
|
||||
not real. The shape (DSSE envelope, in-toto statement, subject +
|
||||
predicate, verify-by-identity) is what evidence-as-attestation
|
||||
looks like. A real attestation carries a real signature from a
|
||||
real key held by the builder.
|
||||
|
||||
## Audit-Ready Export
|
||||
|
||||
- The evidence store is queryable at any time, not only at audit
|
||||
time. The audit packet is a query (a date range, a data class, a
|
||||
subject) over the store; the export is a dump of the matching
|
||||
artifacts with their provenance and signatures.
|
||||
- An audit-ready export that requires six weeks of forensics is a
|
||||
P6 violation dressed as a success: the evidence was not collected
|
||||
continuously, it was reconstructed. The export should be a query
|
||||
that runs in minutes, not a project that runs for weeks.
|
||||
|
||||
## Evidence Lifecycle
|
||||
|
||||
- Evidence has a retention lifecycle (`data-retention.md`): an
|
||||
evidence artifact is retained for the audit cycle it supports,
|
||||
then ages out. The retention rule for evidence is itself audited
|
||||
(deletion of evidence is a meta-audited action, like deletion of
|
||||
audit logs).
|
||||
- A legal hold suspends evidence deletion for a defined set — the
|
||||
same mechanism as audit-log holds.
|
||||
|
||||
## What Violates Evidence Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| Evidence assembled by hand the week before an audit | P6 Evidence is Collected Continuously |
|
||||
| An evidence artifact with no provenance (no producer, no timestamp) | P7 Identity is Attributable |
|
||||
| An audit packet that requires six weeks of forensics to produce | P6 Evidence is Collected Continuously |
|
||||
| An attestation with no signature (provenance asserted, not proven) | P7 Identity is Attributable |
|
||||
| Evidence store not queryable between audits | P10 Compliance Posture is Observable |
|
||||
| Evidence deleted before its retention period with no meta-audit | P6 Evidence is Collected Continuously |
|
||||
| Policy decisions not recorded as evidence | P5 Policy is Evaluated as a Gate |
|
||||
| A deployment with no signed attestation of its build provenance | P7 Identity is Attributable |
|
||||
|
||||
## Relationship to Other Domains
|
||||
|
||||
- `domains/security/supply-chain.md` — signed attestations are the
|
||||
supply-chain integrity primitive; evidence.md is the compliance
|
||||
consumer of the same artifact.
|
||||
- `domains/observability/metrics.md` — posture metrics are a
|
||||
continuous evidence stream.
|
||||
- `domains/observability/tracing.md` — distributed traces provide
|
||||
evidence that spans a request across services.
|
||||
- `domains/compliance/audit-logs.md` — audit logs are a primary
|
||||
evidence artifact; the append-only guarantee is what makes them
|
||||
admissible.
|
||||
- `domains/compliance/policy-as-code.md` — policy decisions are
|
||||
evidence of enforcement; the policy code itself is evidence of
|
||||
the rule.
|
||||
- `domains/compliance/data-retention.md` — evidence has a retention
|
||||
lifecycle tied to the audit cycle.
|
||||
@@ -0,0 +1,184 @@
|
||||
# Compliance — First Principles
|
||||
|
||||
> Framework-agnostic per D-024. These principles derive from core
|
||||
> Security (a subset of C1 Correctness), Observability, and
|
||||
> Reversibility. They apply across regulations — NIST CSF, SOC 2,
|
||||
> GDPR, CCPA, HIPAA, PCI-DSS, ISO 27001 — without prescribing any
|
||||
> regulation-specific implementation. Regulation names appear here
|
||||
> only as examples of what the principles support; the principles
|
||||
> themselves are engineering rules, not legal controls.
|
||||
|
||||
## 1. The Principles
|
||||
|
||||
### P1. Audit Logs are Append-Only
|
||||
Audit records are immutable once written. Deletion or mutation of an
|
||||
audit record is itself an auditable incident — the tampering is the
|
||||
signal, not just the underlying event. An audit log that can be edited
|
||||
is not an audit log; it is a draft. Append-only is enforced
|
||||
structurally (write-once storage, immutable buckets, hash-chained
|
||||
records), not by policy alone. This is the compliance angle on
|
||||
`domains/observability/logging.md`: structured logs that cannot be
|
||||
rewritten are the substrate of accountability.
|
||||
|
||||
### P2. Every Significant Action is Logged
|
||||
The set of auditable actions is defined a priori, in code, before the
|
||||
action ships — not retrofitted after an incident. Authentication
|
||||
changes, authorization decisions, data access, configuration changes,
|
||||
policy changes, and deletions are all significant. "We forgot to log
|
||||
it" is a violation, not an excuse. The auditable-action catalog is
|
||||
itself versioned and reviewed. A significant action with no log line
|
||||
is a C7 (Observability) defect and a C1 (Correctness) defect: the
|
||||
system's behavior is invisible, and accountability is impossible.
|
||||
|
||||
### P3. Retention is Policy, Not Storage
|
||||
Data lifetime is declared and enforced as policy, not left to the
|
||||
storage layer's defaults. Deletion at end-of-life is a feature, not a
|
||||
failure. Retention rules live as code (lifecycle rules, scheduled
|
||||
deletion jobs, tiered storage transitions), they are reviewed, and
|
||||
they are auditable. "We kept it because the bucket was cheap" is a
|
||||
violation; "we deleted it because the policy said to" is correct.
|
||||
Retention is distinct from backup: a backup is a recovery mechanism,
|
||||
a retention rule is a deletion mechanism. Keeping them conflated
|
||||
produces data that survives both the deletion policy and the
|
||||
disaster — which is the opposite of compliance. Cross
|
||||
`domains/data/migrations.md` for the schema-lifecycle discipline.
|
||||
|
||||
### P4. Policy is Code
|
||||
Compliance policy is expressed in versioned, reviewable, testable
|
||||
code (OPA / Rego, AWS Cedar, HashiCorp Sentinel, Kyverno) — not in
|
||||
spreadsheets, prose documents, or tribal knowledge. Policy in a
|
||||
spreadsheet is untestable, unreviewable, and undeployable; it is a
|
||||
wishlist, not a control. Policy-as-code inherits the disciplines of
|
||||
`domains/infrastructure-as-code/P1 Declarative Intent`: declarative
|
||||
intent, version control, review before merge, plan before apply. A
|
||||
compliance rule that is not executable is a rule that cannot be
|
||||
enforced, which is a rule that does not exist.
|
||||
|
||||
### P5. Policy is Evaluated as a Gate
|
||||
Policy violations block before the action, not after the audit.
|
||||
Enforcement happens at admission time (kubernetes admission), at
|
||||
pipeline time (CI/CD gates), and at provisioning time (IaC plan
|
||||
gates) — before the non-compliant state is realized. Detecting a
|
||||
violation after it ships is detection, not enforcement. A policy that
|
||||
is "logged but not blocked" is a postcard, not a gate. This is the
|
||||
compliance angle on C5 (Reversibility): a blocked action is
|
||||
reversible by construction; a shipped violation requires remediation,
|
||||
which is more expensive than prevention.
|
||||
|
||||
### P6. Evidence is Collected Continuously
|
||||
Evidence of compliance — logs, configs, scans, attestations, policy
|
||||
decisions, access reviews — is gathered as a byproduct of operation,
|
||||
not assembled manually at audit time. The audit-time scramble is the
|
||||
anti-pattern: it is expensive, it is incomplete, and it produces
|
||||
evidence that is reconstructed rather than recorded. Continuous
|
||||
evidence collection means the audit packet is a query over
|
||||
already-collected artifacts, not a forensic reconstruction. This is
|
||||
the compliance angle on `domains/observability/tracing.md` for
|
||||
distributed evidence and `domains/observability/metrics.md` for
|
||||
posture signals.
|
||||
|
||||
### P7. Identity is Attributable
|
||||
Every logged action traces to an authenticated, non-shared principal.
|
||||
Shared accounts, generic service identities, and "admin" as an actor
|
||||
are violations: an action with no attributable human or workload is
|
||||
an action with no accountability. Identity is recorded in the audit
|
||||
record at the time of the action, not resolved after the fact. This
|
||||
is the compliance angle on `domains/security/authorization.md` and
|
||||
`domains/kubernetes/rbac.md`: the audit subject must be the principal
|
||||
that acted, not a role that many can assume.
|
||||
|
||||
### P8. Subject Access is Honored
|
||||
Data-subject rights — access, export, deletion, correction — are
|
||||
operations with defined contracts and audit trails, not ad-hoc
|
||||
tickets. The system can answer "what do we have on this subject,"
|
||||
"export it in a portable form," and "delete it and prove the
|
||||
deletion" as first-class operations. These are not features bolted on
|
||||
at the end; they are contracts the data layer implements from the
|
||||
start. A subject-access request that requires a forensics team is a
|
||||
correctness defect: the system does not know what it holds. Cross
|
||||
`domains/data/schema-design.md` for the data shapes that make
|
||||
subject access computable.
|
||||
|
||||
### P9. Secrets and Sensitive Data are Redacted in Audit
|
||||
Audit logs themselves must not leak secrets, credentials, PII, or
|
||||
other sensitive data. Redaction is structural — applied at the
|
||||
logging boundary, before the record is written — not opportunistic
|
||||
scrubbing after the fact. A secret that appears in an audit log is a
|
||||
C1 (Correctness) violation (the log is now a secret store) and a
|
||||
security violation (`domains/security/secrets.md`). The redaction
|
||||
policy is itself auditable: which fields are redacted, by what rule,
|
||||
in which log stream. Once a secret is in an append-only log, the
|
||||
remediation is expensive — rotate the secret and rewrite the log's
|
||||
access scope — so redaction-at-source is the only sound position.
|
||||
|
||||
### P10. Compliance Posture is Observable
|
||||
The system reports its own compliance state: drift from policy, open
|
||||
violations, retention status, evidence freshness, policy-evaluation
|
||||
counts. Silent non-compliance is the bug. A compliance posture
|
||||
metric is a first-class signal (`domains/observability/metrics.md`),
|
||||
alertable, and dashboarded. "We didn't know we were non-compliant"
|
||||
is not a defense; it is a C7 (Observability) defect. The posture is
|
||||
queryable at any time, not only at audit time. This is the compliance
|
||||
angle on `domains/infrastructure-as-code/P3 State is Truth`: the
|
||||
compliance state is a versioned, queryable truth, not a vibe.
|
||||
|
||||
## 2. Core Principle Trace
|
||||
|
||||
Each compliance P-rule derives from one or more core C-rules (C1–C8).
|
||||
The matrix extension lands in P4 of the v0.3 plan; the traces below
|
||||
are authoritative.
|
||||
|
||||
| P-rule | Core | Why |
|
||||
|--------|------|-----|
|
||||
| P1 Audit Logs are Append-Only | C1, C5 | Correctness of the record; reversibility of tamper detection |
|
||||
| P2 Every Significant Action is Logged | C7, C1 | Observability of behavior; correctness of a-priori audit scope |
|
||||
| P3 Retention is Policy, Not Storage | C5, C8 | Reversibility of data lifetime; economy of storage as policy |
|
||||
| P4 Policy is Code | C6, C2 | Composability of versioned policy; clarity of executable intent |
|
||||
| P5 Policy is Evaluated as a Gate | C1, C5 | Correctness of pre-action enforcement; reversibility of blocked actions |
|
||||
| P6 Evidence is Collected Continuously | C7, C3 | Observability of compliance state; simplicity of audit-by-query |
|
||||
| P7 Identity is Attributable | C1, C7 | Correctness of accountability (security subset); observability of who acted |
|
||||
| P8 Subject Access is Honored | C1, C5 | Correctness of the data-subject contract; reversibility of deletion |
|
||||
| P9 Secrets and Sensitive Data are Redacted in Audit | C1, C3 | Correctness of not leaking (security subset); simplicity of structural redaction |
|
||||
| P10 Compliance Posture is Observable | C7, C1 | Observability of posture; correctness of self-reported state |
|
||||
|
||||
## 3. What Violates These Principles
|
||||
|
||||
| Violation | Principle Breached |
|
||||
|-----------|-------------------|
|
||||
| An audit log stored on a mutable filesystem with no write-once protection | P1 Audit Logs are Append-Only |
|
||||
| A `DELETE` on an audit record to "clean up a typo" | P1 Audit Logs are Append-Only |
|
||||
| An auth change with no audit log line | P2 Every Significant Action is Logged |
|
||||
| "We'll add logging after we ship the feature" | P2 Every Significant Action is Logged |
|
||||
| Data kept indefinitely because "the bucket is cheap" | P3 Retention is Policy, Not Storage |
|
||||
| A retention rule in a spreadsheet, not in code | P4 Policy is Code |
|
||||
| A policy that logs violations but does not block the action | P5 Policy is Evaluated as a Gate |
|
||||
| Evidence assembled by hand the week before an audit | P6 Evidence is Collected Continuously |
|
||||
| A shared `admin` account as the audit actor | P7 Identity is Attributable |
|
||||
| A subject-access request that requires a forensics team | P8 Subject Access is Honored |
|
||||
| A secret visible in an audit log entry | P9 Secrets and Sensitive Data are Redacted in Audit |
|
||||
| No dashboard for compliance posture between audits | P10 Compliance Posture is Observable |
|
||||
|
||||
## 4. Relationship to Other Domains
|
||||
|
||||
Compliance is the accountability layer that crosses
|
||||
`domains/security/` (it audits security actions),
|
||||
`domains/observability/` (audit logs are structured logging; posture
|
||||
is metrics; evidence is traces), `domains/data/` (retention and
|
||||
subject access are data-layer contracts), and
|
||||
`domains/infrastructure-as-code/` (policy-as-code parallels
|
||||
declarative IaC; compliance state parallels state-as-truth). Cross-
|
||||
links are one-directional (per D-026 extended):
|
||||
|
||||
- `domains/security/authorization.md` ← P7 (attributable identity)
|
||||
- `domains/security/secrets.md` ← P9 (redaction)
|
||||
- `domains/security/supply-chain.md` ← P6 (signed attestations as evidence)
|
||||
- `domains/observability/logging.md` ← P1, P2 (audit logs = structured logging)
|
||||
- `domains/observability/metrics.md` ← P10 (compliance posture metrics)
|
||||
- `domains/observability/tracing.md` ← P6 (evidence from distributed traces)
|
||||
- `domains/data/schema-design.md` ← P3, P8 (retention and subject-access shapes)
|
||||
- `domains/data/migrations.md` ← P3 (retention migration discipline)
|
||||
- `domains/infrastructure-as-code/P1 Declarative Intent` ← P4 (policy-as-code)
|
||||
- `domains/infrastructure-as-code/P3 State is Truth` ← P10 (compliance posture truth)
|
||||
- `domains/kubernetes/rbac.md` ← P7 (audit subject identity)
|
||||
- `domains/devops/ci-cd.md` ← P5 (policy as a pipeline gate)
|
||||
- `domains/devops/first-principles.md` ← P4 (policy as configuration-as-code)
|
||||
@@ -0,0 +1,140 @@
|
||||
# Policy as Code — Derived Rules
|
||||
|
||||
> Derives from `domains/compliance/first-principles.md`. Covers P4
|
||||
> (Policy is Code) and P5 (Policy is Evaluated as a Gate). Referenced
|
||||
> by `audit-logs.md` (policy decisions are audited) and `evidence.md`
|
||||
> (policy decisions are evidence). Framework-agnostic per D-024.
|
||||
|
||||
## Policy is Code (P4 Policy is Code)
|
||||
|
||||
- Compliance policy is expressed in versioned, reviewable, testable
|
||||
code — not in spreadsheets, prose documents, or tribal knowledge.
|
||||
Policy in a spreadsheet is untestable, unreviewable, and
|
||||
undeployable; it is a wishlist, not a control.
|
||||
- Policy-as-code inherits the disciplines of
|
||||
`domains/infrastructure-as-code/P1 Declarative Intent`: declarative
|
||||
intent, version control, review before merge, plan before apply.
|
||||
A compliance rule that is not executable is a rule that cannot be
|
||||
enforced, which is a rule that does not exist.
|
||||
- Policy code is tested like any other code: unit tests for the rule
|
||||
logic (given an input, the rule allows or denies as expected),
|
||||
integration tests for the gate (the rule fires at the right point
|
||||
in the pipeline), and versioning for the policy itself (a policy
|
||||
change is a reviewed, merged, deployed change).
|
||||
|
||||
## Policy is Evaluated as a Gate (P5 Policy is Evaluated as a Gate)
|
||||
|
||||
- Policy violations block **before** the action, not after the
|
||||
audit. Enforcement happens at:
|
||||
- **Admission time** — a kubernetes admission webhook denies a
|
||||
non-compliant resource before it is created
|
||||
(`domains/kubernetes/rbac.md`).
|
||||
- **Pipeline time** — a CI/CD gate denies a non-compliant change
|
||||
before it merges (`domains/devops/ci-cd.md`).
|
||||
- **Provisioning time** — an IaC plan gate denies a non-compliant
|
||||
resource before `apply` (`domains/infrastructure-as-code/`).
|
||||
- A policy that logs violations but does not block the action is a
|
||||
postcard, not a gate. Detection is not enforcement. A logged
|
||||
violation that the actor could ignore is a P5 violation — the
|
||||
policy exists, but the system is not compliant by construction.
|
||||
- The gate is the contract. The policy author writes the rule; the
|
||||
gate operator wires the rule into the enforcement point; the
|
||||
auditor verifies the gate fired. All three are auditable
|
||||
(`audit-logs.md`).
|
||||
|
||||
## Engine Comparison (IDEATE-23)
|
||||
|
||||
| Engine | Policy Language | Evaluation Gate | Ecosystem | Notes |
|
||||
|--------|-----------------|-----------------|-----------|-------|
|
||||
| **OPA / Rego** | Rego (declarative, set-based, Datalog-inspired) | CI/CD, k8s admission (Gatekeeper), HTTP API, IaC plan (Terraform Sentinel-style), service mesh | Broadest ecosystem; CNCF graduated; library of reusable bundles | General-purpose; the default choice when the gate location varies |
|
||||
| **AWS Cedar** | Cedar (declarative, authorization-focused, schema-typed) | k8s admission (via Cedar-agent), application authorization, AVP (Verified Permissions) | AWS-native; tight schema typing; separates policy from entities | Authorization-focused; strong where the policy is "who can do what on which resource" |
|
||||
| **HashiCorp Sentinel** | Sentinel (declarative, restricted, policy-focused) | Terraform / TFE plan gate, Nomad, Vault | HashiCorp ecosystem; embedded in Terraform Enterprise / HCP | IaC-plan-gate native; the enforcement point is the `plan` output |
|
||||
| **Kyverno** | Kyverno (YAML-declarative, k8s-native, no new DSL) | k8s admission (native), cluster-wide policy reports | Kubernetes-native; no separate language — policy is a CRD | k8s-cluster-gate native; the choice when the gate is admission and the team prefers YAML over a DSL |
|
||||
|
||||
- None is advocated over the others. The choice is (a) where the
|
||||
gate fires, (b) the team's tolerance for a new policy language,
|
||||
and (c) ecosystem fit. All four satisfy P4/P5 when wired
|
||||
correctly.
|
||||
- A gate is a gate regardless of engine: the rule is declarative,
|
||||
the evaluation is pre-action, and the decision is allow-or-deny.
|
||||
The engine difference is language and enforcement-point fit, not
|
||||
correctness.
|
||||
|
||||
## Policy Testing
|
||||
|
||||
- Policy code is unit-tested like any other code. A test asserts
|
||||
that a given input produces the expected decision (allow / deny /
|
||||
warn). The test is versioned with the policy; a policy change
|
||||
with no test change is a red flag.
|
||||
|
||||
```
|
||||
// illustrative Rego policy + test
|
||||
// policy: deny containers running as root
|
||||
package k8s.admission
|
||||
|
||||
deny[msg] {
|
||||
input.kind == "Pod"
|
||||
c := input.spec.containers[_]
|
||||
not c.securityContext.runAsNonRoot
|
||||
msg := sprintf("container %s must set runAsNonRoot", [c.name])
|
||||
}
|
||||
|
||||
// test (Rego unit test)
|
||||
package k8s.admission
|
||||
|
||||
test_deny_root_container {
|
||||
some msg in deny with input as {
|
||||
"kind": "Pod",
|
||||
"spec": { "containers": [ { "name": "app", "securityContext": {} } ] }
|
||||
}
|
||||
msg == "container app must set runAsNonRoot"
|
||||
}
|
||||
```
|
||||
|
||||
- Integration tests assert the gate fires: a non-compliant resource
|
||||
submitted to the admission endpoint is denied; a compliant one is
|
||||
allowed. The integration test runs against the real gate, not a
|
||||
mock, because the gate wiring is half the contract.
|
||||
|
||||
## Policy Versioning
|
||||
|
||||
- Policy is versioned in git. A policy change is a reviewed, merged,
|
||||
deployed change — the same discipline as application code. A
|
||||
policy that is edited in production without review is a P4
|
||||
violation: the policy is code, but it is being treated as config.
|
||||
- A policy change can break existing workloads (a new deny rule
|
||||
blocks a previously-allowed resource). The rollout is staged:
|
||||
warn-only mode first (log violations, do not block), then enforce
|
||||
mode after the violation count is zero. This is the policy
|
||||
analogue of `domains/devops/P5 Progressive Delivery`.
|
||||
|
||||
## What Violates Policy-as-Code Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| A compliance rule in a spreadsheet | P4 Policy is Code |
|
||||
| A policy that logs violations but does not block the action | P5 Policy is Evaluated as a Gate |
|
||||
| A policy edited in production without review | P4 Policy is Code |
|
||||
| A policy with no unit tests for the rule logic | P4 Policy is Code |
|
||||
| A gate wired with a mock instead of the real engine | P5 Policy is Evaluated as a Gate |
|
||||
| A new deny rule enforced without a warn-only rollout | P5 Policy is Evaluated as a Gate |
|
||||
| A policy in prose ("the team should not use root containers") | P4 Policy is Code |
|
||||
| A policy decision with no audit record | P5 Policy is Evaluated as a Gate |
|
||||
|
||||
## Relationship to Other Domains
|
||||
|
||||
- `domains/infrastructure-as-code/first-principles.md` — policy-as-
|
||||
code inherits declarative intent, versioning, and plan-before-
|
||||
apply from IaC.
|
||||
- `domains/kubernetes/rbac.md` — k8s admission is a primary
|
||||
enforcement gate; Kyverno and OPA Gatekeeper wire into it.
|
||||
- `domains/devops/ci-cd.md` — CI/CD is a pipeline-time enforcement
|
||||
gate; a policy step blocks a non-compliant change before merge.
|
||||
- `domains/compliance/audit-logs.md` — every policy decision (allow
|
||||
/ deny) is an audited significant action.
|
||||
- `domains/compliance/evidence.md` — policy decisions and the
|
||||
policy code itself are evidence of enforcement posture.
|
||||
- `domains/security/authorization.md` — Cedar's authorization-
|
||||
focused policy overlaps with authz; the split is that authz is
|
||||
the runtime decision, policy-as-code is the reviewed rule that
|
||||
drives it.
|
||||
@@ -0,0 +1,173 @@
|
||||
# Internationalization (i18n) — First Principles
|
||||
|
||||
> Grounded in Unicode ICU + CLDR, W3C i18n WG, BCP 47 / RFC 5646,
|
||||
> ICU MessageFormat / FormatJS / i18next / Mozilla Fluent, the
|
||||
> JavaScript `Intl` API, and WCAG 2.1 AA. The developer's language is
|
||||
> one locale among many, not the neutral form.
|
||||
|
||||
## 1. The Principles
|
||||
|
||||
### P1. Source Language is a Locale, Not the Default
|
||||
The developer's own language is one locale among many — it is not the
|
||||
"neutral" or "unlocalized" form of the product. Strings are extracted
|
||||
from day one, addressed by key, and routed through a locale resource
|
||||
layer even when only one locale is populated. Treating the source
|
||||
language as the default produces hidden concatenations, hardcoded
|
||||
grammar assumptions, and a translation debt that compounds until the
|
||||
first second locale arrives — at which point the fix is a rewrite, not
|
||||
a patch. The source locale is `en-US` (or whatever the team writes in);
|
||||
it is not `null`. This is the i18n angle on `domains/uiux/copywriting.md`:
|
||||
copy lives in resources, not in code.
|
||||
|
||||
### P2. Locale Identifiers are Standardized
|
||||
Use BCP 47 language tags (`en-US`, `ar-EG`, `zh-Hans-CN`, `pt-BR`).
|
||||
No ad-hoc locale codes, no two-letter-only hacks, no invented keys.
|
||||
The tag carries language, script (when needed), and region (when
|
||||
needed); it is the contract between the resource layer, the
|
||||
formatting layer, and the runtime. A locale identifier that is not
|
||||
BCP 47 is a key that cannot be resolved by any standard tool, which
|
||||
is a correctness violation. Cross `domains/data/schema-design.md`:
|
||||
locale identifiers are a data shape with a defined vocabulary.
|
||||
|
||||
### P3. Resources are External, Not Inline
|
||||
User-facing strings live in locale resource files (`.po`, JSON,
|
||||
Fluent `.ftl`, ICU Resource Bundle), never concatenated inline in
|
||||
code. Inline strings are invisible to the translation pipeline,
|
||||
unversionable as a unit, and untestable for completeness. String
|
||||
concatenation in code (`"Welcome, " + name + "!"`) is the cardinal
|
||||
violation: it bakes in source-language grammar and breaks for every
|
||||
locale with different word order. Resources are the boundary; code
|
||||
addresses strings by key, the resource layer resolves the key to the
|
||||
locale. This is the i18n angle on C4 Locality: strings and their
|
||||
locale-specific consequences live together in the resource, not
|
||||
scattered across code.
|
||||
|
||||
### P4. Plural and Gender are Parameterized
|
||||
Plural forms, gender, and select are expressed with ICU MessageFormat
|
||||
(or an equivalent parameterized formatter), never with `if (n == 1)`
|
||||
branching in code. Plural rules are locale-specific — English has
|
||||
one/other, Arabic has six categories, Russian has three — and a
|
||||
hand-rolled branch encodes exactly one locale's rules while pretending
|
||||
to be universal. The formatter is the contract; the resource carries
|
||||
the variants; the code passes the count and lets the formatter choose.
|
||||
A `if (n == 1)` plural is a C1 (Correctness) violation masquerading as
|
||||
a shortcut.
|
||||
|
||||
### P5. Formatting is Locale-Aware
|
||||
Dates, times, numbers, currencies, units, and relative time are
|
||||
formatted via ICU / CLDR / the JavaScript `Intl` API — never
|
||||
hand-rolled. A hand-rolled date formatter encodes one locale's
|
||||
conventions and silently produces wrong output for every other locale
|
||||
(mm/dd/yyyy vs dd/mm/yyyy is the canonical failure). CLDR is the
|
||||
source of truth for locale data; `Intl` is the runtime that exposes
|
||||
it. Formatting correctness is observable: a misformatted date is a
|
||||
wrong answer in the user's locale, even if it is "right" in the
|
||||
developer's. Cross `domains/api/error-responses.md` for localized
|
||||
error messages at API boundaries.
|
||||
|
||||
### P6. Text Direction is a Layout Primitive
|
||||
RTL and bidi are first-class layout concerns, not a CSS afterthought.
|
||||
Logical CSS properties (`margin-inline-start`, `padding-block-end`,
|
||||
`inset-inline-end`) over physical (`margin-left`, `padding-top`). The
|
||||
`dir` attribute is set on the document and on subtrees; the bidi
|
||||
algorithm (UAX #9) handles inline reordering. A layout that assumes
|
||||
LTR is a layout that is wrong for `ar`, `he`, `fa`, `ur`, and any
|
||||
RTL-mixed context. Text direction is not a skin — it is a structural
|
||||
property of the layout, and fixing it late is a rewrite. This is the
|
||||
i18n angle on `domains/uiux/accessibility.md`: RTL support is an
|
||||
accessibility concern for non-Latin-script users.
|
||||
|
||||
### P7. Layout Accommodates Expansion
|
||||
Translated text expands and contracts — German is ~30% longer than
|
||||
English, Japanese often shorter, RTL mirroring shifts every visual
|
||||
anchor. Layouts are flexible: no fixed pixel widths for text, no
|
||||
truncation without an ellipsis-and-title strategy, no
|
||||
`white-space: nowrap` on translatable strings. A layout that breaks
|
||||
on a 30% expansion is a layout that is wrong for most of the world's
|
||||
locales. Designing for the worst-case expansion up front is cheaper
|
||||
than reworking every screen when the first long-form locale ships.
|
||||
|
||||
### P8. Pseudo-Locales Test Early
|
||||
Test with pseudo-locales (accented, lengthened, RTL-mirrored, brack-
|
||||
enclosed) before real translations arrive. A pseudo-locale run
|
||||
surfaces hardcoded strings, layout overflow, broken concatenation,
|
||||
and LTR assumptions while the fix is still cheap — the translator
|
||||
hasn't been paid yet, and the string freeze hasn't happened. Finding
|
||||
these bugs after real translation is a C5 (Reversibility) violation:
|
||||
the cost of undoing is now a re-translation. Cross
|
||||
`domains/testing/fixtures.md` and `domains/testing/pyramid.md` for
|
||||
where pseudo-locales sit in the testing pyramid.
|
||||
|
||||
### P9. Images and Icons are Cultural
|
||||
Icons, colors, gestures, and imagery are locale-sensitive. A
|
||||
mailbox icon means "email" in the US and "mail" in Japan — but a
|
||||
green checkmark means "correct" in the West and "incorrect" in some
|
||||
East Asian contexts. A thumbs-up is positive in much of the world
|
||||
and an insult in parts of the Middle East. Avoid locale-bound symbols
|
||||
as universal; parameterize imagery per locale where the symbol is not
|
||||
globally neutral. Icons are not a universal language; they are a
|
||||
locale with a picture. This is a C2 (Clarity) concern: an icon whose
|
||||
meaning changes by locale is unclear to the reader it was not drawn
|
||||
for.
|
||||
|
||||
### P10. Translation is Reversible and Versioned
|
||||
Resource files are versioned alongside code; a bad translation is a
|
||||
rollback, not a hot-patch. Every locale resource has a history
|
||||
(what shipped when), a provenance (which translator / which service),
|
||||
and a rollback path. A translation that breaks the UI is reverted to
|
||||
the prior resource version, the same way a code regression is
|
||||
reverted to the prior commit. Translations without version history
|
||||
are anecdote, not artifact — you cannot tell what changed, when, or
|
||||
why. This is the i18n angle on C5 Reversibility applied to the
|
||||
resource layer.
|
||||
|
||||
## 2. Core Principle Trace
|
||||
|
||||
Each i18n P-rule derives from one or more core C-rules (C1–C8). The
|
||||
matrix extension lands in P4 of the v0.3 plan; the traces below are
|
||||
authoritative.
|
||||
|
||||
| P-rule | Core | Why |
|
||||
|--------|------|-----|
|
||||
| P1 Source Language is a Locale, Not the Default | C2, C1 | Clarity of locale intent; correctness of treating source as one-of-many |
|
||||
| P2 Locale Identifiers are Standardized | C2, C6 | Clarity of a standard vocabulary; composability with standard tools |
|
||||
| P3 Resources are External, Not Inline | C4, C6 | Locality of strings and their locale consequences; composability of the resource layer |
|
||||
| P4 Plural and Gender are Parameterized | C1, C6 | Correctness of locale-specific plural rules; composability of the formatter contract |
|
||||
| P5 Formatting is Locale-Aware | C1, C7 | Correctness of formatted output; observability of format correctness |
|
||||
| P6 Text Direction is a Layout Primitive | C1, C4 | Correctness of layout for RTL; locality of direction with the text it governs |
|
||||
| P7 Layout Accommodates Expansion | C8, C3 | Economy of rework; simplicity of flexible layouts over per-locale overrides |
|
||||
| P8 Pseudo-Locales Test Early | C7, C5 | Observability of i18n defects early; reversibility of fixing before translation |
|
||||
| P9 Images and Icons are Cultural | C1, C2 | Correctness of locale-appropriate symbols; clarity of meaning across locales |
|
||||
| P10 Translation is Reversible and Versioned | C5 | Reversibility of the resource layer |
|
||||
|
||||
## 3. What Violates These Principles
|
||||
|
||||
| Violation | Principle Breached |
|
||||
|-----------|-------------------|
|
||||
| A user-facing string hardcoded in source | P3 Resources are External, Not Inline |
|
||||
| `"Welcome, " + name + "!"` string concatenation | P3 Resources are External, Not Inline |
|
||||
| `if (n == 1) { return "item"; } else { return "items"; }` | P4 Plural and Gender are Parameterized |
|
||||
| A locale code like `en_us` or `english` instead of `en-US` | P2 Locale Identifiers are Standardized |
|
||||
| A hand-rolled date formatter (`getMonth() + 1 + "/" + getDay()`) | P5 Formatting is Locale-Aware |
|
||||
| `margin-left: 10px` on a translatable layout | P6 Text Direction is a Layout Primitive |
|
||||
| A fixed-width text container that overflows on German | P7 Layout Accommodates Expansion |
|
||||
| First i18n test runs against real translations, not pseudo-locales | P8 Pseudo-Locales Test Early |
|
||||
| A thumbs-up icon shipped as universally positive | P9 Images and Icons are Cultural |
|
||||
| Resource files with no git history or no rollback path | P10 Translation is Reversible and Versioned |
|
||||
| The source language treated as the "unlocalized" default | P1 Source Language is a Locale, Not the Default |
|
||||
|
||||
## 4. Relationship to Other Domains
|
||||
|
||||
i18n is the locale-awareness layer that `domains/uiux/` consumes and
|
||||
that `domains/api/` surfaces at boundaries. It borrows the testing
|
||||
discipline of `domains/testing/` and the data-shape discipline of
|
||||
`domains/data/`. Cross-links are one-directional (per D-026 extended):
|
||||
|
||||
- `domains/uiux/copywriting.md` ← P1, P3 (strings live in resources)
|
||||
- `domains/uiux/accessibility.md` ← P6 (RTL is an a11y concern for non-Latin users)
|
||||
- `domains/uiux/components.md` ← P6, P7 (layout primitives that survive direction and expansion)
|
||||
- `domains/api/error-responses.md` ← P5 (localized error messages)
|
||||
- `domains/data/schema-design.md` ← P2, P3 (locale data shapes)
|
||||
- `domains/testing/fixtures.md` ← P8 (pseudo-locale fixtures)
|
||||
- `domains/testing/pyramid.md` ← P8 (pseudo-locale tier mapping)
|
||||
- `domains/testing/first-principles.md` ← P8 (testing discipline for locale)
|
||||
@@ -0,0 +1,141 @@
|
||||
# Formatting — Derived Rules
|
||||
|
||||
> Derives from `domains/i18n/first-principles.md`. Covers P2 (Locale
|
||||
> Identifiers Standardized), P4 (Plural/Gender Parameterized), and P5
|
||||
> (Formatting is Locale-Aware). Referenced by `locale-resources.md`
|
||||
> (the formatter resolves the messages) and `testing-i18n.md` (the
|
||||
> formatted output is what snapshots assert).
|
||||
|
||||
## Formatting is Locale-Aware (P5 Formatting is Locale-Aware)
|
||||
|
||||
- Dates, times, numbers, currencies, units, and relative time are
|
||||
formatted via ICU / CLDR / the JavaScript `Intl` API — never
|
||||
hand-rolled. CLDR is the source of truth for locale data; `Intl`
|
||||
is the runtime that exposes it.
|
||||
- A hand-rolled formatter encodes one locale's conventions and
|
||||
silently produces wrong output for every other locale. The
|
||||
canonical failure is date format: `mm/dd/yyyy` (US) vs
|
||||
`dd/mm/yyyy` (most of the world) vs `yyyy-mm-dd` (ISO, sortable).
|
||||
Picking one and calling it done is a correctness violation in
|
||||
every locale it is wrong for.
|
||||
|
||||
## BCP 47 Tags Drive Formatting (P2 Locale Identifiers Standardized)
|
||||
|
||||
- Every formatter takes a BCP 47 locale tag. The tag is the contract
|
||||
between the resource layer and the formatting layer: the same tag
|
||||
that selects the resource selects the formatter.
|
||||
- A locale tag that is not BCP 47 cannot be resolved by `Intl`, ICU,
|
||||
or CLDR — the formatter returns the runtime default, which is the
|
||||
developer's locale, not the user's. This is why P2 is a
|
||||
prerequisite of P5: you cannot format for a locale you cannot name.
|
||||
|
||||
## The Intl Surface (ICU/CLDR in the Browser and Node)
|
||||
|
||||
| API | Formats | Notes |
|
||||
|-----|---------|-------|
|
||||
| `Intl.DateTimeFormat` | Dates, times, date+time, time zones | Calendar (`buddhist`, `hebrew`, `islamic`), numbering system (`arab`, `hanidec`) via locale tag extensions |
|
||||
| `Intl.NumberFormat` | Numbers, currencies, units, percent | Notation (`compact`, `scientific`), grouping, sign display |
|
||||
| `Intl.RelativeTimeFormat` | "3 days ago", "in 2 months" | Locale-specific phrasing; numeric vs auto |
|
||||
| `Intl.PluralRules` | Plural category for a count | `one`, `few`, `many`, `other`, `zero`, `two` per CLDR — the engine ICU MessageFormat uses |
|
||||
| `Intl.ListFormat` | "a, b, and c" | Conjunction / disjunction / unit lists, locale-specific separators |
|
||||
| `Intl.Collator` | Locale-aware string sorting | Strength (`base`, `accent`, `case`); numeric collation |
|
||||
|
||||
- All of these are built on ICU/CLDR; they are the runtime baseline.
|
||||
Use them. A `moment.js`-style hand-rolled format string
|
||||
(`"MM/DD/YYYY"`) is a relic of the pre-`Intl` era and a P5
|
||||
violation in any locale-aware code path.
|
||||
|
||||
## Dates and Times
|
||||
|
||||
```
|
||||
// Correct — Intl, locale-aware
|
||||
new Intl.DateTimeFormat("ar-EG", {
|
||||
dateStyle: "full",
|
||||
timeStyle: "short",
|
||||
}).format(new Date());
|
||||
// "الأربعاء، ٧ نوفمبر ٢٠٢٤، ٣:١٥ م"
|
||||
|
||||
// Wrong — hand-rolled, source-locale only
|
||||
const d = new Date();
|
||||
const s = (d.getMonth() + 1) + "/" + d.getDate() + "/" + d.getFullYear();
|
||||
// "11/7/2024" — meaningless in most locales
|
||||
```
|
||||
|
||||
- Time zones are not locales. A locale tells you *how to format* a
|
||||
timestamp; a time zone tells you *what instant* it refers to. Do
|
||||
not derive one from the other (`ar-EG` is not a time zone).
|
||||
Format with the user's locale; render in the user's time zone;
|
||||
store in UTC.
|
||||
|
||||
## Numbers, Currencies, Units
|
||||
|
||||
```
|
||||
new Intl.NumberFormat("de-DE", { style: "currency", currency: "EUR" })
|
||||
.format(1234.56); // "1.234,56 €"
|
||||
|
||||
new Intl.NumberFormat("ar-EG", { style: "currency", currency: "EGP" })
|
||||
.format(1234.56); // "١٬٢٣٤٫٥٦ ج.م."
|
||||
|
||||
new Intl.NumberFormat("en-US", { style: "unit", unit: "kilometer-per-hour" })
|
||||
.format(100); // "100 km/h"
|
||||
```
|
||||
|
||||
- The currency code (`EUR`, `EGP`, `USD`) is ISO 4217; the locale
|
||||
determines the symbol, grouping, and placement. A hand-rolled
|
||||
`"$" + amount` is wrong for `de-DE` (symbol, grouping, placement
|
||||
all differ).
|
||||
|
||||
## Plural Rules (P4 Plural/Gender Parameterized)
|
||||
|
||||
- `Intl.PluralRules` returns the CLDR plural category for a count in
|
||||
a given locale. ICU MessageFormat uses this category to select the
|
||||
variant from the resource (`locale-resources.md`).
|
||||
- Never branch on the raw count in code. The count goes to the
|
||||
formatter; the formatter consults `PluralRules` for the locale;
|
||||
the resource carries the variant for that category.
|
||||
|
||||
```
|
||||
// ICU MessageFormat (FormatJS)
|
||||
new Intl.MessageFormat(
|
||||
"{count, plural, one {# item} other {# items}}",
|
||||
"en-US"
|
||||
).format({ count: 1 }); // "1 item"
|
||||
|
||||
// ar-EG — six categories; the code is identical, only the
|
||||
// resource differs.
|
||||
```
|
||||
|
||||
## Gender and Select
|
||||
|
||||
- ICU MessageFormat also supports `{gender, select, male {...} female {...} other {...}}`
|
||||
for gendered agreement and `{case, select, ...}` for general
|
||||
disjunction. These live in the resource, not in code branches.
|
||||
- A `switch (gender)` in code that picks a string is the same
|
||||
violation as `if (n == 1)`: it encodes one locale's grammar in
|
||||
code and breaks for every locale with different agreement rules.
|
||||
|
||||
## What Violates Formatting Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| `getMonth() + 1 + "/" + getDay()` hand-rolled date | P5 Formatting is Locale-Aware |
|
||||
| `"$" + amount` hand-rolled currency | P5 Formatting is Locale-Aware |
|
||||
| `if (n === 1) "item" else "items"` plural branch | P4 Plural and Gender are Parameterized |
|
||||
| `moment("MM/DD/YYYY")` format string in locale-aware code | P5 Formatting is Locale-Aware |
|
||||
| Deriving time zone from locale tag | P2 Locale Identifiers are Standardized |
|
||||
| A non-BCP-47 tag passed to `Intl` (silently falls back) | P2 Locale Identifiers are Standardized |
|
||||
| `switch (gender)` selecting strings in code | P4 Plural and Gender are Parameterized |
|
||||
| Storing timestamps in local time, not UTC | P5 Formatting is Locale-Aware |
|
||||
|
||||
## Relationship to Other Domains
|
||||
|
||||
- `domains/api/error-responses.md` — API error messages are
|
||||
formatted for the requesting locale; the error code is stable, the
|
||||
message is locale-formatted.
|
||||
- `domains/data/schema-design.md` — locale identifiers, currency
|
||||
codes, and time zones are data contracts; treat them as schema
|
||||
(`en-US`, `EUR`, `UTC`), not free text.
|
||||
- `domains/i18n/locale-resources.md` — the resource layer carries
|
||||
the parameterized messages this formatter resolves.
|
||||
- `domains/testing/fixtures.md` — formatted output per locale is the
|
||||
fixture; snapshot tests assert against it.
|
||||
@@ -0,0 +1,138 @@
|
||||
# Locale Resources — Derived Rules
|
||||
|
||||
> Derives from `domains/i18n/first-principles.md`. Covers P1 (Source
|
||||
> Language is a Locale), P2 (Locale Identifiers Standardized), P3
|
||||
> (Resources External, Not Inline), P4 (Plural/Gender Parameterized),
|
||||
> and P10 (Translation Reversible and Versioned). Referenced by
|
||||
> `formatting.md` (strings the formatter resolves) and
|
||||
> `rtl-bidi.md` (the `dir` the resource layer carries).
|
||||
|
||||
## Resources are the Boundary (P3 Resources are External, Not Inline)
|
||||
|
||||
- User-facing strings live in locale resource files, addressed by
|
||||
key. Code references a key; the resource layer resolves the key to
|
||||
the active locale. The source language is itself a locale
|
||||
(`en-US`), not a fallback baked into code.
|
||||
- String concatenation in code (`"Welcome, " + name + "!"`) is the
|
||||
cardinal violation: it bakes in source-language word order and
|
||||
breaks for every locale with different grammar. Replace every
|
||||
concatenation with a parameterized message:
|
||||
`t("welcome", { name })`.
|
||||
- The resource is the single place a string lives. Editing a string
|
||||
in code instead of the resource is a locality violation (C4): the
|
||||
string and its locale consequences now live apart.
|
||||
|
||||
## Resource File Formats
|
||||
|
||||
| Format | Shape | When | Notes |
|
||||
|--------|-------|------|-------|
|
||||
| `.po` / `.pot` | gettext; msgid → msgstr, plural headers | Server-side, GNU ecosystem, PHP/Python/C | Mature tooling (`xgettext`, `msgmerge`); supports plural categories via header |
|
||||
| JSON (flat or namespaced) | `{ "key": "value" }` per locale | JS/web, i18next, FormatJS | Simple, machine-readable, but no native plural support — wrap with ICU MessageFormat |
|
||||
| Fluent `.ftl` | Mozilla FTL; asymmetric, resolver-driven | Browser-grade l10n, asymmetric translations | One message can resolve differently per locale without code changes; supports attributes, selectors |
|
||||
| ICU Resource Bundle | ICU binary/text resources | ICU-native, JVM, C++ | Tightest integration with ICU formatting/CLDR; steeper tooling |
|
||||
|
||||
- None is advocated over the others. The choice is ecosystem fit,
|
||||
not correctness. All four satisfy P3/P4 when used as the boundary.
|
||||
- A custom format (a hand-rolled `.csv` of strings) is a violation:
|
||||
it is unsupported by standard tooling, has no plural grammar, and
|
||||
cannot compose with `formatting.md`'s ICU layer.
|
||||
|
||||
## Key Naming and Namespaces (P2 Locale Identifiers Standardized)
|
||||
|
||||
- Locale identifiers are BCP 47 tags (`en-US`, `ar-EG`, `zh-Hans-CN`).
|
||||
No ad-hoc codes. The resource file is named for its locale:
|
||||
`en-US.json`, `ar-EG.po`, `ftl/ar-EG/main.ftl`.
|
||||
- Message keys are stable, semantic, and structured — not prose.
|
||||
`checkout.cart.item_count` not `"You have 3 items in your cart"`.
|
||||
A key that is the source string (`t("You have items")`) breaks the
|
||||
moment the source copy is edited; the key must outlive the copy.
|
||||
- Namespaces segment by surface (`checkout.*`, `errors.*`, `onboarding.*`)
|
||||
so that a locale can be loaded incrementally and so that key
|
||||
collisions across surfaces are impossible. A flat namespace with
|
||||
thousands of keys is a C2 (Clarity) violation waiting to happen.
|
||||
|
||||
## Fallback Chains
|
||||
|
||||
- The fallback chain is explicit: requested locale → language-only
|
||||
(`en` from `en-GB`) → default locale → key itself (last resort).
|
||||
The default locale is declared once, not re-derived in every call
|
||||
site.
|
||||
- A missing key in the requested locale falling back silently to the
|
||||
source locale is a P3 violation: the user is silently shown the
|
||||
developer's locale, which is not the locale they asked for. Missing
|
||||
keys must be observable (see `testing-i18n.md`).
|
||||
- Fallback is a property of the resource layer, not of individual
|
||||
components. A component that re-implements fallback is duplicating
|
||||
a contract (C6 Composability violation).
|
||||
|
||||
## Plural and Gender in Resources (P4 Plural/Gender Parameterized)
|
||||
|
||||
- Plural variants live in the resource, selected by the formatter,
|
||||
parameterized by the count. The code passes the count; the resource
|
||||
carries the variants; the formatter picks the right one per the
|
||||
locale's CLDR plural rules.
|
||||
|
||||
```
|
||||
// JSON + ICU MessageFormat (FormatJS / i18next)
|
||||
{
|
||||
"cart.item_count": "{count, plural, one {# item} other {# items}}"
|
||||
}
|
||||
// ar-EG.json — six plural categories per CLDR
|
||||
{
|
||||
"cart.item_count": "{count, plural, zero {لا عناصر} one {عنصر واحد} two {عنصران} few {# عناصر} many {# عنصرًا} other {# عنصر}}"
|
||||
}
|
||||
```
|
||||
|
||||
- `if (n == 1)` branching in code is a violation regardless of
|
||||
language. Arabic has six plural categories; Russian has three;
|
||||
English has two. A two-branch `if` encodes exactly one locale's
|
||||
rules and is wrong for every other.
|
||||
|
||||
## Extraction Tooling (P1, P3)
|
||||
|
||||
- Strings are extracted mechanically (e.g. `xgettext`, `i18next-
|
||||
parser`, FormatJS babel plugin), not by hand-tagging. Mechanical
|
||||
extraction produces a `.pot` template that translators work from;
|
||||
the template is regenerated on every build.
|
||||
- A string that cannot be extracted (built at runtime from
|
||||
fragments) is a P3 violation: it is invisible to the pipeline. If
|
||||
the extractor cannot see it, neither can the translator.
|
||||
- The extracted template is versioned (`P10`): the diff between
|
||||
templates is the change in translatable surface. A template that
|
||||
is not committed is a contract that is not reviewable.
|
||||
|
||||
## Versioning and Rollback (P10 Translation Reversible and Versioned)
|
||||
|
||||
- Resource files are committed to git alongside code. A bad
|
||||
translation is a `git revert` of the resource, not a hot-patch over
|
||||
the translator's work. Every locale resource has history,
|
||||
provenance (which translator / which service produced which
|
||||
commit), and a rollback path.
|
||||
- A locale resource that is generated by a translation service and
|
||||
committed without review is a P10 violation: the resource is
|
||||
versioned but the provenance is opaque. Review the diff the same
|
||||
way you review a code diff.
|
||||
|
||||
## What Violates Locale-Resource Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| `t("You have " + n + " items")` concatenation | P3 Resources are External, Not Inline |
|
||||
| A custom `.csv` string store instead of a standard format | P3 Resources are External, Not Inline |
|
||||
| Locale file named `english.json` not `en-US.json` | P2 Locale Identifiers are Standardized |
|
||||
| `if (n == 1) { t("item") } else { t("items") }` in code | P4 Plural and Gender are Parameterized |
|
||||
| A key equal to the source string (`t("Welcome back")`) | P2 / P10 — keys must outlive copy |
|
||||
| Silent fallback to the source locale with no signal | P3 Resources are External, Not Inline |
|
||||
| A runtime-built string the extractor cannot see | P3 Resources are External, Not Inline |
|
||||
| Resource files committed by a bot with no human review | P10 Translation is Reversible and Versioned |
|
||||
|
||||
## Relationship to Other Domains
|
||||
|
||||
- `domains/uiux/copywriting.md` — copy lives in resources; UI
|
||||
microcopy is the source content the resource layer carries.
|
||||
- `domains/api/error-responses.md` — API error messages are locale-
|
||||
resource keys resolved at the boundary, not inline strings.
|
||||
- `domains/data/schema-design.md` — locale identifiers and resource
|
||||
shapes are a data contract; treat them as schema.
|
||||
- `domains/i18n/formatting.md` — the formatter resolves the
|
||||
parameterized message this layer produces.
|
||||
@@ -0,0 +1,117 @@
|
||||
# RTL and Bidi — Derived Rules
|
||||
|
||||
> Derives from `domains/i18n/first-principles.md`. Covers P6 (Text
|
||||
> Direction is a Layout Primitive) and P7 (Layout Accommodates
|
||||
> Expansion). Referenced by `testing-i18n.md` (RTL coverage is an
|
||||
> e2e tier). Grounded in W3C i18n bidi authoring, UAX #9, and
|
||||
> `domains/uiux/accessibility.md`.
|
||||
|
||||
## Text Direction is a Layout Primitive (P6 Text Direction is a Layout Primitive)
|
||||
|
||||
- RTL and bidi are first-class layout concerns, not a CSS
|
||||
afterthought. The layout is designed for both directions from the
|
||||
first commit, not retrofitted when an RTL locale ships.
|
||||
- Logical CSS properties over physical properties, always. The
|
||||
browser resolves logical → physical from the `dir` attribute; the
|
||||
code never has to.
|
||||
|
||||
| Physical (LTR-only) | Logical (dir-aware) | Resolves to in RTL |
|
||||
|---------------------|---------------------|--------------------|
|
||||
| `margin-left` | `margin-inline-start` | `margin-right` |
|
||||
| `margin-right` | `margin-inline-end` | `margin-left` |
|
||||
| `padding-left` | `padding-inline-start` | `padding-right` |
|
||||
| `left: 0` | `inset-inline-start: 0` | `right: 0` |
|
||||
| `text-align: left` | `text-align: start` | `text-align: right` |
|
||||
| `float: left` | use flexbox/grid + `inline-start` where supported | mirrored |
|
||||
|
||||
- The `dir` attribute is set on the document root (`<html dir="rtl">`)
|
||||
and on subtrees whose direction differs from the document
|
||||
(`<span dir="ltr">` for an embedded Latin run). `dir` is the
|
||||
contract the bidi algorithm (UAX #9) reads; do not fake direction
|
||||
with `text-align` alone.
|
||||
|
||||
## The Bidi Algorithm (UAX #9)
|
||||
|
||||
- The Unicode bidi algorithm resolves inline reordering of mixed-
|
||||
direction runs. The browser applies it; the author's job is to
|
||||
mark direction correctly, not to reorder by hand.
|
||||
- A string like `"The price is 15 USD"` in an RTL context renders
|
||||
with the Latin run `"15 USD"` in LTR within the RTL line — the
|
||||
algorithm handles it *if* the container's `dir` is set. Without
|
||||
`dir`, numbers and Latin fragments drift to the wrong edge.
|
||||
- `dir="auto"` on a container infers direction from the first strong
|
||||
directional character of its content — useful for user-generated
|
||||
content whose direction is unknown. `dir="auto"` is not a
|
||||
replacement for `dir="rtl"` on a known-RTL document.
|
||||
|
||||
## Mirroring (Icons, Controls, Diagrams)
|
||||
|
||||
- Direction-aware icons mirror in RTL: a "back" arrow pointing left
|
||||
in LTR points right in RTL. A "refresh" circular arrow does not
|
||||
mirror. The rule: icons that imply direction mirror; icons that
|
||||
imply time or rotation do not.
|
||||
- Use `[dir="rtl"]` selectors or logical icon variants — never
|
||||
`transform: scaleX(-1)` as a one-off hack scattered across
|
||||
components. Centralize the mirroring rule (a token, a component
|
||||
prop) so it is auditable.
|
||||
- Numbers do not mirror. `"15 USD"` in an RTL line is still
|
||||
`"15 USD"` left-to-right inside the bidi run; mirroring it to
|
||||
`"DSU 51"` is a correctness violation.
|
||||
- Diagrams and flowcharts: a left-to-right process flow in LTR is a
|
||||
right-to-left flow in RTL. Decide per diagram whether the flow
|
||||
mirrors (most do) or is direction-neutral (some scientific
|
||||
schematics).
|
||||
|
||||
## Layout Accommodates Expansion (P7 Layout Accommodates Expansion)
|
||||
|
||||
- Translated text expands. German is ~30% longer than English;
|
||||
Japanese is often shorter but taller; RTL mirroring shifts every
|
||||
visual anchor. Layouts are flexible:
|
||||
- No fixed pixel widths on translatable text containers.
|
||||
- No `white-space: nowrap` on translatable strings.
|
||||
- No `text-overflow: ellipsis` without a `title` carrying the full
|
||||
string.
|
||||
- Buttons sized to fit their longest locale variant, not the
|
||||
source.
|
||||
- A layout that breaks at +30% width is a layout that is wrong for
|
||||
most of the world's locales. Designing for the worst case up front
|
||||
is cheaper than reworking every screen when the first long-form
|
||||
locale ships.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
| Pitfall | Why it breaks | Fix |
|
||||
|---------|---------------|-----|
|
||||
| `margin-left` everywhere | In RTL the start is the right; `margin-left` leaves the right side unstyled | `margin-inline-start` |
|
||||
| `text-align: left` for "default" alignment | In RTL the default is right; `left` pins content to the wrong edge | `text-align: start` |
|
||||
| Icons hardcoded to LTR orientation | "Back" arrow points the wrong way in RTL | Mirror direction-implying icons via `[dir="rtl"]` |
|
||||
| Numbers mirrored with the layout | Numbers are LTR inside RTL; mirroring produces garbage | Leave number runs LTR; the bidi algorithm handles embedding |
|
||||
| Fixed `width: 120px` on a button | German button label overflows and truncates | `min-width` + `max-width` + flex; let content size |
|
||||
| `position: absolute; left: 0` | Pins to the physical left in both directions | `inset-inline-start: 0` |
|
||||
| Fake direction with `text-align` only | The bidi algorithm reads `dir`, not `text-align`; mixed runs reorder wrong | Set `dir` on the container |
|
||||
|
||||
## What Violates RTL/Bidi Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| A layout with no `dir` attribute, assuming LTR | P6 Text Direction is a Layout Primitive |
|
||||
| `margin-left` / `left: 0` / `text-align: left` throughout | P6 Text Direction is a Layout Primitive |
|
||||
| A "back" arrow that points left in the RTL build | P6 Text Direction is a Layout Primitive |
|
||||
| Numbers mirrored to read right-to-left | P6 Text Direction is a Layout Primitive |
|
||||
| `width: 100px` on a text container that overflows in German | P7 Layout Accommodates Expansion |
|
||||
| `white-space: nowrap` on a translated label | P7 Layout Accommodates Expansion |
|
||||
| `dir` faked with `text-align` and no `dir` attribute | P6 Text Direction is a Layout Primitive |
|
||||
| No RTL build until the first RTL locale ships | P6 Text Direction is a Layout Primitive |
|
||||
|
||||
## Relationship to Other Domains
|
||||
|
||||
- `domains/uiux/accessibility.md` — RTL support is an accessibility
|
||||
concern for non-Latin-script users; WCAG 2.1 AA requires that
|
||||
direction be set correctly.
|
||||
- `domains/uiux/components.md` — components are built with logical
|
||||
properties so they survive direction and expansion without per-
|
||||
locale overrides.
|
||||
- `domains/i18n/testing-i18n.md` — RTL coverage is an e2e-tier
|
||||
test; pseudo-locale mirroring surfaces direction bugs early.
|
||||
- `domains/i18n/locale-resources.md` — the `dir` is part of the
|
||||
locale's metadata, carried alongside the resource bundle.
|
||||
@@ -0,0 +1,142 @@
|
||||
# Testing i18n — Derived Rules
|
||||
|
||||
> Derives from `domains/i18n/first-principles.md`. Covers P8
|
||||
> (Pseudo-Locales Test Early) and the testing-discipline angle on
|
||||
> P3 (Resources External), P5 (Formatting Locale-Aware), P6 (Text
|
||||
> Direction), and P10 (Translation Versioned). Referenced by
|
||||
> `locale-resources.md` (missing-key detection) and `rtl-bidi.md`
|
||||
> (RTL coverage tier).
|
||||
|
||||
## Pseudo-Locales Test Early (P8 Pseudo-Locales Test Early)
|
||||
|
||||
- A pseudo-locale is a synthetic locale that transforms the source
|
||||
strings to surface i18n defects before real translations arrive.
|
||||
Three transforms cover the three defect classes:
|
||||
|
||||
| Pseudo-locale | Transform | Surfaces |
|
||||
|---------------|-----------|----------|
|
||||
| `en-XA` (accented) | `Wêlcômê tô thê çhêckôût` | Strings not extracted (raw source appears), encoding bugs |
|
||||
| `en-XB` (lengthened / "long") | `Wᴇʟᴄᴏᴍᴇ ᴛᴏ ᴛʜᴇ ᴄʜᴇᴄᴋᴏᴜᴛ──────` (~30% longer, bracketed) | Layout overflow, fixed widths, truncation |
|
||||
| `en-XC` (RTL-mirrored) | Source rendered with `dir="rtl"` and a Latin-in-RTL run | LTR-only layout assumptions, physical CSS properties |
|
||||
|
||||
- Pseudo-locale tests are cheap: they run against source strings, no
|
||||
translator involved, no string freeze required. A failing pseudo-
|
||||
locale run is a bug found at the cheapest possible point in the
|
||||
pipeline. Finding the same bug after real translation is a C5
|
||||
(Reversibility) violation: the fix now costs a re-translation.
|
||||
|
||||
## Pseudo-Locale → Testing Pyramid Mapping (IDEATE-28)
|
||||
|
||||
- The testing pyramid (`domains/testing/pyramid.md`) has three tiers;
|
||||
i18n tests map to each tier with a distinct signal. The mapping is
|
||||
deliberate: each tier catches a different class of defect, and
|
||||
skipping a tier leaves a blind spot.
|
||||
|
||||
| Pyramid Tier | i18n Test | Defect Caught | Tooling Shape |
|
||||
|--------------|-----------|---------------|---------------|
|
||||
| **Unit** | Missing-key detection | A key referenced in code but absent from the resource bundle; a key present in the source locale but missing from a target locale | Static scan over the resource bundle + code AST; runs per file, no runtime |
|
||||
| **Integration** | Snapshot per locale | Formatted output for a fixture input differs across locales in a way that breaks the contract (wrong plural, wrong date, overflow) | Render a known fixture through the formatter per locale; snapshot-diff against the recorded baseline |
|
||||
| **e2e** | RTL coverage | The app renders and is navigable in `dir="rtl"`; no layout overflow, no off-screen controls, no LTR-pinned anchors | Browser-driven run against the `en-XC` pseudo-locale (or a real RTL locale); assert on layout, not just text |
|
||||
|
||||
- Unit is the broad base (fast, runs on every commit), e2e is the
|
||||
narrow top (slow, runs on PR merge). Integration sits between.
|
||||
This mirrors `domains/testing/pyramid.md` exactly — i18n is not a
|
||||
special case; it is a domain that uses the same tiers.
|
||||
|
||||
## Unit Tier — Missing-Key Detection (P3 Resources External)
|
||||
|
||||
- A static scan compares the set of keys referenced in code against
|
||||
the keys present in each locale bundle. A key in code but not in
|
||||
`en-US` is a P3 violation (the string is not in the resource
|
||||
layer). A key in `en-US` but not in `ar-EG` is a coverage gap —
|
||||
the missing-key scan flags it before the locale ships.
|
||||
- Missing keys fail the build, not the runtime. A missing key that
|
||||
surfaces only when a user switches locale is a defect found in
|
||||
production, which is the most expensive place to find it.
|
||||
|
||||
```
|
||||
// tool output (illustrative)
|
||||
// missing-key scan
|
||||
[FAIL] ar-EG: key "checkout.cart.item_count" referenced in code,
|
||||
absent from ar-EG.json
|
||||
[FAIL] en-US: key "checkout.cart.total" referenced in Checkout.tsx:42,
|
||||
absent from en-US.json (not extracted)
|
||||
[PASS] en-US, ar-EG, de-DE, zh-Hans-CN: all other keys present
|
||||
```
|
||||
|
||||
## Integration Tier — Snapshot per Locale (P5 Formatting Locale-Aware)
|
||||
|
||||
- For a fixed fixture input, render the formatted output per locale
|
||||
and snapshot it. A change in the snapshot is either an intended
|
||||
change (new CLDR data, new copy) or a regression.
|
||||
- The snapshot is per locale, not per format string. The same
|
||||
fixture (`{ count: 1, currency: "EUR", date: 2024-11-07 }`)
|
||||
produces different snapshots for `en-US`, `de-DE`, `ar-EG` — and
|
||||
that difference is the assertion. A locale whose snapshot matches
|
||||
the source locale's is a red flag: the formatter is not actually
|
||||
locale-aware.
|
||||
|
||||
```
|
||||
// snapshot — checkout.cart (fixture: count=1, currency=EUR, date=2024-11-07)
|
||||
// en-US
|
||||
"1 item · €1,234.56 · 11/7/2024"
|
||||
// de-DE
|
||||
"1 Artikel · 1.234,56 € · 07.11.2024"
|
||||
// ar-EG
|
||||
"عنصر واحد · ١٬٢٣٤٫٥٦ € · ٧/١١/٢٠٢٤"
|
||||
```
|
||||
|
||||
- Snapshots are reviewed, not rubber-stamped. A snapshot diff that
|
||||
changes the plural form for `ar-EG` is either a CLDR update (verify)
|
||||
or a regression (revert).
|
||||
|
||||
## e2e Tier — RTL Coverage (P6 Text Direction is a Layout Primitive)
|
||||
|
||||
- A browser-driven run against `en-XC` (or a real RTL locale like
|
||||
`ar-EG`) asserts that the app is navigable in RTL: no overflow, no
|
||||
off-screen controls, no LTR-pinned anchors. The assertion is on
|
||||
layout, not on text — text correctness is the integration tier's
|
||||
job.
|
||||
- RTL e2e is the narrow top of the i18n pyramid: it is slow, it
|
||||
requires a browser, and it catches the defects the lower tiers
|
||||
cannot (the interaction of `dir` with the real layout engine). It
|
||||
runs on PR merge, not on every commit.
|
||||
|
||||
## Snapshot Discipline (P10 Translation Reversible and Versioned)
|
||||
|
||||
- Snapshots are versioned in git. A snapshot that changes because of
|
||||
a real translation update is a committed diff, reviewed like a
|
||||
code change. A snapshot that changes because of a regression is a
|
||||
`git revert`.
|
||||
- A snapshot that is regenerated and committed without review is a
|
||||
P10 violation: the snapshot is versioned but the provenance is
|
||||
opaque. The same discipline applies to snapshots as to resources
|
||||
(`locale-resources.md`).
|
||||
|
||||
## What Violates i18n Testing Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| First i18n test runs against real translations, not pseudo-locales | P8 Pseudo-Locales Test Early |
|
||||
| No missing-key scan — gaps surface only at runtime in production | P3 Resources are External, Not Inline |
|
||||
| Snapshot per locale that matches the source locale's snapshot | P5 Formatting is Locale-Aware |
|
||||
| No RTL e2e — "we'll test RTL when we ship an RTL locale" | P6 Text Direction is a Layout Primitive |
|
||||
| Snapshots regenerated and committed without review | P10 Translation is Reversible and Versioned |
|
||||
| i18n tests only at e2e (no unit/integration tier) | pyramid inversion — `domains/testing/pyramid.md` |
|
||||
| Pseudo-locale run skipped because "it's not a real locale" | P8 Pseudo-Locales Test Early |
|
||||
|
||||
## Relationship to Other Domains
|
||||
|
||||
- `domains/testing/pyramid.md` — the pseudo-locale → pyramid mapping
|
||||
mirrors this domain's unit / integration / e2e tiers exactly.
|
||||
- `domains/testing/fixtures.md` — locale fixtures (a fixed input
|
||||
rendered per locale) are the fixture shape for the integration
|
||||
tier.
|
||||
- `domains/i18n/locale-resources.md` — missing-key detection is the
|
||||
unit-tier scan over the resource bundle this doc defines.
|
||||
- `domains/i18n/formatting.md` — the integration-tier snapshot
|
||||
asserts against the formatter's output.
|
||||
- `domains/i18n/rtl-bidi.md` — the e2e tier exercises the layout
|
||||
rules this doc establishes.
|
||||
- `domains/uiux/accessibility.md` — RTL coverage is an a11y
|
||||
concern; an untested RTL build is an untested a11y surface.
|
||||
@@ -0,0 +1,222 @@
|
||||
# Bad Example: Compliance Audit Log (Two Breaches)
|
||||
|
||||
> An audit logging implementation that violates **two** Atelier
|
||||
> compliance principles in one example (per IDEATE-26, D-044):
|
||||
> **P1** (Audit Logs are Append-Only) — a mutable audit log with
|
||||
> routine `DELETE`/`UPDATE` "cleanup" — and **P9** (Secrets and
|
||||
> Sensitive Data are Redacted in Audit) — a database password leaked
|
||||
> into an audit record. Each violation is cited, then fixed.
|
||||
|
||||
## The Code
|
||||
|
||||
```python
|
||||
# audit_log.py — the audit sink, stored in a mutable Postgres table
|
||||
import psycopg2, datetime
|
||||
|
||||
# P1 VIOLATION: the audit log is a regular mutable table. There is no
|
||||
# write-once protection, no immutable bucket, no hash-chaining.
|
||||
# Any DB user with UPDATE/DELETE can rewrite history.
|
||||
CREATE_TABLE = """
|
||||
CREATE TABLE audit_log (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
timestamp TIMESTAMPTZ NOT NULL,
|
||||
event TEXT NOT NULL,
|
||||
actor TEXT NOT NULL,
|
||||
target TEXT,
|
||||
payload JSONB,
|
||||
request_id TEXT
|
||||
);
|
||||
-- No row-level immutability. No trigger preventing UPDATE/DELETE.
|
||||
"""
|
||||
|
||||
def write_event(event, actor, target=None, payload=None, request_id=None):
|
||||
conn = psycopg2.connect(os.environ["DATABASE_URL"])
|
||||
conn.execute(
|
||||
"INSERT INTO audit_log (timestamp, event, actor, target, payload, request_id) "
|
||||
"VALUES (%s, %s, %s, %s, %s, %s)",
|
||||
(datetime.datetime.utcnow(), event, actor, target,
|
||||
json.dumps(payload), request_id),
|
||||
)
|
||||
|
||||
# P1 VIOLATION (continued): "cleanup" that mutates the audit log.
|
||||
# A routine job deletes records older than 30 days to "save space"
|
||||
# and updates records to "fix typos in the actor field."
|
||||
def cleanup_audit_log():
|
||||
conn = psycopg2.connect(os.environ["DATABASE_URL"])
|
||||
# DELETE — an audit record is destroyed. This is tampering,
|
||||
# dressed as housekeeping.
|
||||
conn.execute("DELETE FROM audit_log WHERE timestamp < NOW() - INTERVAL '30 days'")
|
||||
# UPDATE — an audit record is rewritten. The "fix" is the
|
||||
# violation; the original actor is lost.
|
||||
conn.execute("UPDATE audit_log SET actor = 'admin' WHERE actor LIKE 'svc-%'")
|
||||
```
|
||||
|
||||
```python
|
||||
# The call site that leaks a secret into the audit log.
|
||||
def read_config(key):
|
||||
# ... fetches a secret from the secrets manager ...
|
||||
value = secrets_manager.get(key) # e.g. the raw DB password
|
||||
# P9 VIOLATION: the raw secret value is written into the audit
|
||||
# payload. The append-only log is now a secret store.
|
||||
write_event(
|
||||
event="config.read",
|
||||
actor="api-server",
|
||||
target={"kind": "secret", "id": key},
|
||||
payload={"value": value}, # <- the secret, in plaintext
|
||||
request_id=req.id,
|
||||
)
|
||||
return value
|
||||
```
|
||||
|
||||
The resulting audit record:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": 48213,
|
||||
"timestamp": "2026-08-05T09:12:03Z",
|
||||
"event": "config.read",
|
||||
"actor": "api-server",
|
||||
"target": {"kind": "secret", "id": "db-password"},
|
||||
"payload": {"value": "p@ssw0rd-sup3r-s3cr3t-plaintext"},
|
||||
"request_id": "req_91c2"
|
||||
}
|
||||
```
|
||||
|
||||
A week later, the `cleanup_audit_log` job `DELETE`s this record (it
|
||||
is older than 30 days in the team's "retention" — which is actually a
|
||||
storage-economy decision, not a policy), and `UPDATE`s every
|
||||
`svc-*` actor to `admin`. The secret was in the log for a week,
|
||||
readable by anyone with `SELECT` on the table; now the record of it
|
||||
having been there is gone.
|
||||
|
||||
## What Makes It Bad
|
||||
|
||||
### Breach 1 — Mutable Audit Log (Compliance P1 Audit Logs are Append-Only)
|
||||
- The audit log is a regular mutable Postgres table. `DELETE FROM
|
||||
audit_log WHERE timestamp < ...` and `UPDATE audit_log SET actor =
|
||||
...` both succeed. The log is a draft, not a record.
|
||||
- Routine `DELETE` as "cleanup" is the cardinal P1 violation: the
|
||||
deletion of an audit record is itself an auditable incident, not a
|
||||
housekeeping task. "We deleted old records to save space" is a P3
|
||||
(Retention is Policy, Not Storage) violation *and* a P1 violation —
|
||||
the retention decision is driven by storage cost, and the
|
||||
mechanism is tampering.
|
||||
- The `UPDATE` that rewrites `svc-deploy` → `admin` destroys
|
||||
attribution (a P7 violation stacked on the P1 violation): the
|
||||
original actor is lost, and the replacement (`admin`) is a shared
|
||||
identity that could be any of ten engineers.
|
||||
- **Fix:** the audit sink is append-only *by construction*, not by
|
||||
policy. Write-once storage (WORM bucket, immutable log stream,
|
||||
hash-chained ledger) enforces immutability at the substrate.
|
||||
Retention is a declared policy with a meta-audit of deletions; a
|
||||
human does not run ad-hoc `DELETE` jobs.
|
||||
|
||||
```python
|
||||
# Fix: write to an append-only sink (illustrative — S3 Object Lock,
|
||||
# WORM bucket, or a hash-chained ledger). The API has no update /
|
||||
# delete path; the storage refuses mutation.
|
||||
def write_event(event, actor, target=None, payload=None, request_id=None):
|
||||
record = {
|
||||
"timestamp": datetime.datetime.now(datetime.timezone.utc).isoformat(),
|
||||
"event": event,
|
||||
"actor": actor, # the authenticated principal, not "admin"
|
||||
"target": target,
|
||||
"payload": redact(payload), # see Breach 2 fix
|
||||
"request_id": request_id,
|
||||
"prev_hash": last_hash(), # hash-chaining: tampering breaks the chain
|
||||
}
|
||||
record["hash"] = sha256(canonical_json(record))
|
||||
append_only_sink.write(record) # WORM storage; no update/delete API exists
|
||||
|
||||
# Fix: retention is a declared, reviewed policy — not an ad-hoc DELETE.
|
||||
# When an audit segment ages out, the deletion is itself meta-audited
|
||||
# in a higher-tier log with the rule that authorized it.
|
||||
# (See domains/compliance/data-retention.md and audit-logs.md.)
|
||||
```
|
||||
- See `domains/compliance/audit-logs.md` (Audit Logs are Append-Only)
|
||||
and `domains/compliance/first-principles.md` P1.
|
||||
|
||||
### Breach 2 — Secret Leaked in Audit Log (Compliance P9 Secrets and Sensitive Data are Redacted in Audit)
|
||||
- `payload={"value": value}` writes the raw DB password into the
|
||||
audit record. The append-only log is now a secret store: anyone
|
||||
with `SELECT` on `audit_log` can read production credentials. The
|
||||
log is harder to secure than the secrets manager it read from.
|
||||
- Once the secret is in an append-only log, the remediation is
|
||||
expensive — rotate the secret *and* rewrite the log's access scope
|
||||
(you cannot edit the record; it is append-only). Redaction must
|
||||
happen *at the logging boundary, before the record is written*, not
|
||||
by opportunistic scrubbing after the fact.
|
||||
- The redaction policy here is "nothing" — there is no rule for
|
||||
which fields are redacted, by what mechanism, in which event type.
|
||||
A redaction rule that lives in no one's head and no code is a P9
|
||||
violation waiting to happen (and it happened).
|
||||
- **Fix:** redaction is structural, applied at the logging boundary
|
||||
before the record reaches the append-only sink. The policy is
|
||||
itself auditable (which fields, by what rule, in which event).
|
||||
|
||||
```python
|
||||
# Fix: redaction at the boundary. Log the FACT of the action
|
||||
# (a secret was read), never the CONTENT of the secret.
|
||||
REDACTED_FIELDS = {"value", "token", "password", "authorization", "secret"}
|
||||
|
||||
def redact(payload):
|
||||
if not isinstance(payload, dict):
|
||||
return "[REDACTED:non-object]"
|
||||
out = {}
|
||||
for k, v in payload.items():
|
||||
if k.lower() in REDACTED_FIELDS or "secret" in k.lower():
|
||||
out[k] = "[REDACTED:secret]"
|
||||
else:
|
||||
out[k] = v
|
||||
out["_redaction"] = "secret-value-policy/v1" # the rule is auditable
|
||||
return out
|
||||
|
||||
# The fixed audit record:
|
||||
# {
|
||||
# "event": "config.read",
|
||||
# "actor": "api-server", # the authenticated principal
|
||||
# "target": {"kind": "secret", "id": "db-password"},
|
||||
# "payload": {"value": "[REDACTED:secret]"},
|
||||
# "_redaction": "secret-value-policy/v1",
|
||||
# "request_id": "req_91c2"
|
||||
# }
|
||||
# The fact of the read is logged; the secret never enters the log.
|
||||
```
|
||||
- See `domains/compliance/audit-logs.md` (Redaction at the Boundary)
|
||||
and `domains/compliance/first-principles.md` P9. Cross
|
||||
`domains/security/secrets.md` — the audit-side redaction is the
|
||||
complement of secret management.
|
||||
|
||||
## The Cascade (Two Breaches Compound)
|
||||
|
||||
The two violations compound destructively. The secret enters the
|
||||
mutable log (P9 breach), where it sits readable by any `SELECT`-holder
|
||||
for a week. Then the `cleanup` job `DELETE`s the record (P1 breach) —
|
||||
destroying the evidence that the secret was ever logged, while the
|
||||
secret itself has already been exposed to every reader of the table.
|
||||
The `UPDATE` that rewrites `svc-deploy` → `admin` (a P7 attribution
|
||||
breach stacked on the P1 breach) means that even if a copy of the
|
||||
record survived, the actor who triggered the secret read is no longer
|
||||
identifiable. The team cannot answer "who read the DB password and
|
||||
when" — the log that would answer it was mutated, and the secret it
|
||||
leaked is now in the wild. This is the worst-case interaction of P1
|
||||
and P9: a secret leak with no attributable actor and no surviving
|
||||
record.
|
||||
|
||||
## Cross-Domain Links
|
||||
|
||||
- `domains/compliance/audit-logs.md` — the append-only guarantee and
|
||||
the redaction-at-boundary rule this code violates.
|
||||
- `domains/compliance/first-principles.md` — P1 (Append-Only) and P9
|
||||
(Redacted) are the two breached principles; P7 (Attributable) is
|
||||
breached by the `UPDATE` rewrite.
|
||||
- `domains/compliance/evidence.md` — an audit log that can be
|
||||
`DELETE`d is not admissible evidence; the append-only guarantee is
|
||||
what makes it admissible.
|
||||
- `domains/compliance/data-retention.md` — retention is a declared
|
||||
policy with meta-audited deletions, not an ad-hoc `DELETE` job.
|
||||
- `domains/security/secrets.md` — redaction at the logging boundary
|
||||
is the audit-side complement of secret management.
|
||||
- `domains/observability/logging.md` — audit logs are structured
|
||||
logging with an append-only guarantee; the logging primitives
|
||||
compose here.
|
||||
@@ -0,0 +1,175 @@
|
||||
# Bad Example: i18n String Concatenation
|
||||
|
||||
> A checkout component that violates Atelier's i18n principles. Each
|
||||
> violation is cited, then fixed.
|
||||
|
||||
## The Code
|
||||
|
||||
```typescript
|
||||
// Checkout.tsx — the cardinal i18n violation
|
||||
function CartSummary({ itemCount, name, total, currency, date }) {
|
||||
// P3 VIOLATION: inline string concatenation. The source-language
|
||||
// word order ("Welcome, {name}! You have {n} items") is baked into
|
||||
// code. Every locale with different word order is broken.
|
||||
const welcome = "Welcome, " + name + "!";
|
||||
|
||||
// P4 VIOLATION: hand-rolled plural branching. `if (n === 1)` encodes
|
||||
// exactly English's one/other rule. Arabic (six categories), Russian
|
||||
// (three), Polish (three) are all wrong.
|
||||
const items =
|
||||
itemCount === 1 ? "1 item" : itemCount + " items";
|
||||
|
||||
// P5 VIOLATION: hand-rolled currency + date formatting. "$" + total
|
||||
// is wrong for de-DE (symbol, grouping, placement). The date
|
||||
// `getMonth() + 1 + "/" + getDay()` is US-only (mm/dd/yyyy).
|
||||
const price = "$" + total.toFixed(2);
|
||||
const d = new Date(date);
|
||||
const dateStr = (d.getMonth() + 1) + "/" + d.getDate() + "/" + d.getFullYear();
|
||||
|
||||
return (
|
||||
<div>
|
||||
<h1>{welcome}</h1>
|
||||
<p>{items} · {price} · {dateStr}</p>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
```typescript
|
||||
// The "resource" file — a custom CSV the team hand-rolled.
|
||||
// locale,en_us
|
||||
// welcome_prefix,Welcome,
|
||||
// item_singular,item
|
||||
// item_plural,items
|
||||
//
|
||||
// This is a P3 violation on its own: a custom format no standard
|
||||
// tool (xgettext, i18next, FormatJS) can extract from or compose with.
|
||||
```
|
||||
|
||||
The team runs their first i18n test against real Arabic translations —
|
||||
after the string freeze, after the translator was paid. The Arabic
|
||||
build renders `"Welcome, محمد!"` with the name on the wrong side of
|
||||
the comma, `"1 items"` for a single item (Arabic has six plural
|
||||
categories, not two), and the price as `"$1,234.56"` (Arabic-Egypt
|
||||
formats as `"١٬٢٣٤٫٥٦ ج.م."`). Every screen is a rewrite, not a patch.
|
||||
|
||||
## What Makes It Bad
|
||||
|
||||
### Inline String Concatenation (i18n P3 Resources are External, Not Inline)
|
||||
- `"Welcome, " + name + "!"` bakes English word order into code. In
|
||||
Japanese the name comes first (`ようこそ、محمدさん!`); in Arabic the
|
||||
structure differs again. The concatenation is invisible to the
|
||||
extraction pipeline (`xgettext`, `i18next-parser`) — the translator
|
||||
never sees it as a unit, and the string cannot be versioned or
|
||||
rolled back as a whole.
|
||||
- The custom `.csv` "resource" store is a second P3 violation: no
|
||||
standard tool reads it, it carries no plural grammar, and it cannot
|
||||
compose with the ICU formatting layer.
|
||||
- **Fix:** strings live in a standard locale resource file, addressed
|
||||
by key. Code calls `t("welcome", { name })`; the resource carries
|
||||
the parameterized message.
|
||||
|
||||
```json
|
||||
// en-US.json (ICU MessageFormat)
|
||||
{
|
||||
"checkout.welcome": "Welcome, {name}!",
|
||||
"checkout.cart.summary": "{count, plural, one {# item} other {# items}} · {price} · {date}"
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
// ar-EG.json — six plural categories per CLDR; the code is identical
|
||||
{
|
||||
"checkout.welcome": "أهلاً بك، {name}!",
|
||||
"checkout.cart.summary": "{count, plural, zero {لا عناصر} one {عنصر واحد} two {عنصران} few {# عناصر} many {# عنصرًا} other {# عنصر}} · {price} · {date}"
|
||||
}
|
||||
```
|
||||
- See `domains/i18n/locale-resources.md` (Resources are the Boundary)
|
||||
and `domains/i18n/first-principles.md` P3.
|
||||
|
||||
### Hand-Rolled Plural Branching (i18n P4 Plural and Gender are Parameterized)
|
||||
- `itemCount === 1 ? "1 item" : itemCount + " items"` encodes
|
||||
English's one/other rule and nothing else. Arabic has six
|
||||
categories (zero, one, two, few, many, other); Russian has three
|
||||
(one, few, many); Polish has three with different boundaries. A
|
||||
two-branch `if` is a C1 (Correctness) violation masquerading as a
|
||||
shortcut — it returns a wrong answer for every non-English locale.
|
||||
- **Fix:** the count goes to ICU MessageFormat; the formatter
|
||||
consults `Intl.PluralRules` for the active locale; the resource
|
||||
carries the variant for that category. The code passes the count,
|
||||
nothing more.
|
||||
|
||||
```typescript
|
||||
// The code passes the count; the resource + formatter pick the form.
|
||||
t("checkout.cart.summary", { count: itemCount, price, date });
|
||||
// Intl.PluralRules("ar-EG").select(1) === "one" -> "عنصر واحد"
|
||||
// Intl.PluralRules("ar-EG").select(2) === "two" -> "عنصران"
|
||||
// Intl.PluralRules("ar-EG").select(5) === "few" -> "٥ عناصر"
|
||||
```
|
||||
- See `domains/i18n/locale-resources.md` (Plural and Gender in
|
||||
Resources) and `domains/i18n/formatting.md` (Plural Rules).
|
||||
|
||||
### Hand-Rolled Currency and Date Formatting (i18n P5 Formatting is Locale-Aware)
|
||||
- `"$" + total.toFixed(2)` hardcodes the US dollar symbol, US
|
||||
grouping (`,`), and US placement (symbol before the number). In
|
||||
`de-DE` the euro formats as `"1.234,56 €"` (symbol after, dot
|
||||
grouping). In `ar-EG` the pound formats as `"١٬٢٣٤٫٥٦ ج.م."`
|
||||
(Arabic-Indic digits, different grouping).
|
||||
- `(d.getMonth() + 1) + "/" + d.getDate() + "/" + d.getFullYear()`
|
||||
produces `11/7/2024` — US `mm/dd/yyyy`. Most of the world reads
|
||||
`dd/mm/yyyy`; ISO is `yyyy-mm-dd`. A hand-rolled date formatter
|
||||
encodes one locale's convention and silently produces wrong output
|
||||
for every other.
|
||||
- **Fix:** `Intl.NumberFormat` and `Intl.DateTimeFormat` with a BCP
|
||||
47 locale tag. CLDR is the source of truth; `Intl` is the runtime.
|
||||
|
||||
```typescript
|
||||
new Intl.NumberFormat("ar-EG", { style: "currency", currency: "EGP" })
|
||||
.format(1234.56); // "١٬٢٣٤٫٥٦ ج.م."
|
||||
new Intl.DateTimeFormat("ar-EG", { dateStyle: "medium" })
|
||||
.format(new Date(date)); // "٧ نوفمبر ٢٠٢٤"
|
||||
```
|
||||
- See `domains/i18n/formatting.md` (the Intl surface, dates, numbers,
|
||||
currencies) and `domains/i18n/first-principles.md` P5.
|
||||
|
||||
### Source Language Treated as the Default (i18n P1 Source Language is a Locale)
|
||||
- The component has no resource layer at all for the source locale —
|
||||
English is "just the strings in the code." When the first second
|
||||
locale arrives, the fix is a rewrite (extract every string,
|
||||
restructure every concatenation), not a patch. The source language
|
||||
is `en-US`, a locale among many — it is not `null`.
|
||||
- **Fix:** extract source strings into `en-US.json` from day one,
|
||||
even before a second locale exists. The resource layer is the
|
||||
boundary from the first commit.
|
||||
- See `domains/i18n/first-principles.md` P1 and
|
||||
`domains/uiux/copywriting.md`.
|
||||
|
||||
## The Cascade
|
||||
|
||||
The violations compound. Inline concatenation makes strings invisible
|
||||
to the extraction pipeline, so the translator never receives them as
|
||||
units — they reconstruct them by reading the code. Hand-rolled
|
||||
plurals return wrong answers for every non-English locale, so the
|
||||
Arabic build ships `"1 items"` for a single item. Hand-rolled
|
||||
formatting produces US-shaped output everywhere, so the price and
|
||||
date are wrong for `de-DE`, `ar-EG`, `zh-Hans-CN`, and every other
|
||||
locale. And because the first i18n test ran against real translations
|
||||
(a P8 violation — pseudo-locales should have surfaced all of this
|
||||
while the fix was still cheap), the defects are found after the
|
||||
string freeze, after the translator was paid, and after the release
|
||||
date was promised. The fix is now a re-translation and a re-release,
|
||||
not a commit.
|
||||
|
||||
## Cross-Domain Links
|
||||
|
||||
- `domains/i18n/locale-resources.md` — the resource layer this code
|
||||
lacks; the standard formats (`.po`, JSON, Fluent, ICU Resource
|
||||
Bundle) it should have used.
|
||||
- `domains/i18n/formatting.md` — the `Intl`/ICU/CLDR formatting this
|
||||
code should call instead of hand-rolling.
|
||||
- `domains/i18n/first-principles.md` — P3, P4, P5, and P8 (pseudo-
|
||||
locales test early).
|
||||
- `domains/uiux/copywriting.md` — copy lives in resources, not in
|
||||
code.
|
||||
- `domains/api/error-responses.md` — the same parameterized-message
|
||||
discipline applies to localized API errors.
|
||||
@@ -0,0 +1,226 @@
|
||||
# Good Example: AI/ML Reproducible Training Run
|
||||
|
||||
> A training run that follows Atelier's AI/ML principles. Each aspect
|
||||
> cites the principle it satisfies. Scope per D-023: this is
|
||||
> engineering discipline (reproducibility, versioning, lineage,
|
||||
> serving), **not** algorithm or model design — no architecture
|
||||
> choice, hyperparameter tuning, or model-family comparison appears
|
||||
> here.
|
||||
|
||||
## The Run
|
||||
|
||||
A training run `2026-08-05T09:12:00Z#run-42` produces model
|
||||
`registry/payments-fraud@sha256:b5e1...aa0`. Every input that shaped
|
||||
the model is pinned, named, and recoverable; the eval was declared
|
||||
before training; the model is an addressed artifact in a registry;
|
||||
the rollback path names the prior model and the prior dataset.
|
||||
|
||||
### The Reproducibility Contract
|
||||
|
||||
```yaml
|
||||
# lineage/run-42.yaml — the lineage root, committed alongside the code
|
||||
run_id: 2026-08-05T09:12:00Z#run-42
|
||||
dataset: s3://ml-data/train@sha256:7f3a...e21
|
||||
splits: dvc.yaml@commit a1b2c4d
|
||||
code: git@a1b2c4d
|
||||
config: configs/train.yaml@commit a1b2c4d
|
||||
environment: ghcr.io/org/train-img@sha256:9c2d...f88
|
||||
eval_spec: configs/eval.yaml@commit a1b2c4d
|
||||
model_digest: registry/payments-fraud@sha256:b5e1...aa0
|
||||
status: passed # eval gate passed -> eligible for promotion
|
||||
```
|
||||
|
||||
- Lose any line and the run is anecdote, not evidence. The record is
|
||||
the lineage root: a prediction cites the `model_digest`, which
|
||||
cites this `run_id`, which cites everything above.
|
||||
|
||||
### Data is Versioned (DVC, content-hashed)
|
||||
|
||||
```ini
|
||||
# dvc.yaml — the split config is versioned in git, the data in the
|
||||
# content-addressed object store. Both are pinned by commit + hash.
|
||||
stages:
|
||||
prepare:
|
||||
cmd: python src/prepare.py --input data/raw --out data/splits
|
||||
deps:
|
||||
- data/raw
|
||||
- src/prepare.py
|
||||
outs:
|
||||
- data/splits/train.parquet
|
||||
- data/splits/val.parquet
|
||||
- data/splits/test.parquet
|
||||
# The dataset hash (sha256:7f3a...e21) is recorded in the lineage
|
||||
# contract above. "s3://ml-data/latest" would be a P2 violation.
|
||||
```
|
||||
|
||||
```bash
|
||||
# The dataset is pinned by content hash, not by a mutable path.
|
||||
$ dvc get s3://ml-data/train --rev sha256:7f3a...e21
|
||||
# The split is a deterministic function of (dataset version, split
|
||||
# config, random seed). Two runs on the same pinned inputs produce
|
||||
# the same splits.
|
||||
```
|
||||
|
||||
### Code and Config are Versioned (git)
|
||||
|
||||
```yaml
|
||||
# configs/train.yaml@commit a1b2c4d — versioned with the code
|
||||
# (No algorithm/hyperparameter content is illustrated here — this is
|
||||
# the engineering discipline of pinning the config, not the model
|
||||
# design inside it. Per D-023, algorithm choice is out of scope.)
|
||||
seed: 42
|
||||
splits:
|
||||
train: data/splits/train.parquet
|
||||
val: data/splits/val.parquet
|
||||
test: data/splits/test.parquet # held out, never touched by training
|
||||
```
|
||||
|
||||
### Environment is Pinned (container digest)
|
||||
|
||||
```dockerfile
|
||||
# The training environment is an image addressed by digest, not :latest.
|
||||
# ghcr.io/org/train-img@sha256:9c2d...f88
|
||||
FROM python:3.11-slim
|
||||
# dependencies pinned in requirements.txt with hashes
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
```
|
||||
|
||||
```text
|
||||
# requirements.txt — pinned + hash-pinned (pip-compile / pip-audit)
|
||||
dvc==3.50.2 \
|
||||
--hash=sha256:1c8a...e7
|
||||
mlflow==2.16.0 \
|
||||
--hash=sha256:9b2f...a1
|
||||
# No unpinned ranges. A rerun pulls the exact same wheels.
|
||||
```
|
||||
|
||||
### Evaluation is Defined Before Training (P4)
|
||||
|
||||
```yaml
|
||||
# configs/eval.yaml@commit a1b2c4d — committed BEFORE training runs.
|
||||
# The metrics, splits, and pass/fail thresholds are a-priori; they
|
||||
# are the contract the model must satisfy to leave the experiment.
|
||||
metrics:
|
||||
- name: precision_at_threshold
|
||||
threshold: ">= 0.92"
|
||||
- name: recall_at_threshold
|
||||
threshold: ">= 0.85"
|
||||
- name: false_positive_rate
|
||||
threshold: "<= 0.03"
|
||||
split: data/splits/test.parquet # held out, never in training
|
||||
gate: all_metrics_pass # AND of all thresholds; no cherry-pick
|
||||
# The eval schema equals the serving input contract (serving.md P8):
|
||||
# feature names, types, ranges match the production boundary exactly.
|
||||
```
|
||||
|
||||
- Metrics chosen after seeing scores would be a P4 violation: the eval
|
||||
would be rationalizing, not measuring. See
|
||||
`domains/ai-ml/model-evaluation.md`.
|
||||
|
||||
### The Model is a Versioned Artifact (MLflow registry)
|
||||
|
||||
```bash
|
||||
# After the eval gate passes, the model is registered as an immutable
|
||||
# artifact addressed by digest, then promoted by stage.
|
||||
$ mlflow models register \
|
||||
--name payments-fraud \
|
||||
--model-uri runs:/run-42/model \
|
||||
--description "run-42, dataset sha256:7f3a...e21, eval passed"
|
||||
# registry/payments-fraud@sha256:b5e1...aa0
|
||||
# Stages: None -> Staging -> Production. Promotion is a registry
|
||||
# operation, not a file copy. Never "latest".
|
||||
```
|
||||
|
||||
### The Pipeline Composes (P9)
|
||||
|
||||
```text
|
||||
# The training flow is a pipeline with explicit stages and contracts,
|
||||
# not a notebook. Each stage has named inputs and named outputs.
|
||||
prepare(dataset@hash) -> split(dvc.yaml) -> train(config, env@digest)
|
||||
-> eval(eval.yaml, test@hash) -> [gate: pass] -> register(model@digest)
|
||||
|
|
||||
+-> [gate: fail] -> abort, no promote
|
||||
# A notebook in this path would be a P9 violation: implicit state,
|
||||
# human-dependent order, unreproducible.
|
||||
```
|
||||
|
||||
## What Makes It Good
|
||||
|
||||
### Reproducibility is First Class (AI/ML P1, C1, C5)
|
||||
- data + code + config + environment are all pinned. A second
|
||||
engineer on a second laptop checks out commit `a1b2c4d`, pulls the
|
||||
dataset by hash, pulls the image by digest, and reproduces the run
|
||||
bit-for-bit. The run is reviewable because it is recreatable.
|
||||
- See `domains/ai-ml/first-principles.md` P1 and
|
||||
`domains/devops/first-principles.md` P1 Reproducibility.
|
||||
|
||||
### Data is Versioned, Not Just Code (AI/ML P2, C5, C7)
|
||||
- The dataset is `s3://ml-data/train@sha256:7f3a...e21`, not
|
||||
`s3://ml-data/latest`. A model trained on "the data" is a model
|
||||
trained on an unknown input — a C1 violation. DVC pins the data the
|
||||
way git pins the code.
|
||||
- See `domains/ai-ml/data-versioning.md` (dataset hashing, the DVC /
|
||||
Delta Lake / LakeFS comparison) and `domains/data/migrations.md`.
|
||||
|
||||
### Lineage is Traceable End-to-End (AI/ML P3, C7, C1)
|
||||
- prediction → model → run-42 → dataset → source. Every edge is
|
||||
named; no orphan model. A serving regression traces back to the
|
||||
exact dataset and code that built the model, which is how drift is
|
||||
diagnosed (data drift vs concept drift vs prediction drift).
|
||||
- See `domains/ai-ml/data-versioning.md` (lineage record) and
|
||||
`domains/observability/logging.md`.
|
||||
|
||||
### Evaluation Defined Before Training (AI/ML P4, C1, C2)
|
||||
- `eval.yaml` was committed before `train` ran. The gate is
|
||||
`all_metrics_pass`; a failing metric aborts promotion. Cherry-
|
||||
picking a metric post-hoc is a correctness violation — the eval
|
||||
would no longer measure the model.
|
||||
- See `domains/ai-ml/model-evaluation.md` (eval-as-a-gate) and
|
||||
`domains/testing/first-principles.md` (tests as specification).
|
||||
|
||||
### Models are Versioned Artifacts (AI/ML P5, C5, C6)
|
||||
- The model is `registry/payments-fraud@sha256:b5e1...aa0`, promoted
|
||||
Staging → Production. A serving endpoint that pulled `latest` would
|
||||
be serving an unknown model with no rollback. The registry is to
|
||||
models what a container registry is to images.
|
||||
- See `domains/ai-ml/serving.md` (the model is an addressed artifact)
|
||||
and `domains/devops/first-principles.md` P7 Immutability.
|
||||
|
||||
### Rollback Includes the Model (AI/ML P10, C5)
|
||||
- If production regresses, the rollback restores the prior model
|
||||
digest `registry/payments-fraud@sha256:a1c4...f09` AND the prior
|
||||
serving code. A rollback that redeploys old code but keeps the new
|
||||
model has not rolled back — the model was the thing that regressed.
|
||||
- See `domains/ai-ml/serving.md` (Rollback Includes the Model) and
|
||||
`domains/devops/first-principles.md` P4 Rollback First.
|
||||
|
||||
## What This Example Does NOT Do (And Why That's Good)
|
||||
|
||||
- Does **not** reference the dataset by a mutable path —
|
||||
`s3://ml-data/latest` would be a P2 violation.
|
||||
- Does **not** choose metrics after seeing scores — that is a P4
|
||||
violation (rationalizing, not measuring).
|
||||
- Does **not** pull `latest` from the model registry — that is a P5
|
||||
violation (unknown model, no rollback).
|
||||
- Does **not** contain algorithm/architecture/hyperparameter content
|
||||
— per D-023, those are research choices, not engineering
|
||||
principles, and have no derivation in the core C-rules.
|
||||
- Does **not** run from a notebook — a notebook in the pipeline path
|
||||
is a P9 violation (implicit state, unreproducible).
|
||||
|
||||
## Cross-Domain Links
|
||||
|
||||
- `domains/ai-ml/data-versioning.md` — the DVC pinning, the lineage
|
||||
record, the tool comparison (DVC / Delta Lake / LakeFS).
|
||||
- `domains/ai-ml/serving.md` — the model is promoted as an addressed
|
||||
artifact; the serving boundary validates inputs against the same
|
||||
schema as the eval.
|
||||
- `domains/ai-ml/model-evaluation.md` — the eval-as-a-gate that this
|
||||
run must pass before promotion.
|
||||
- `domains/devops/first-principles.md` P1 Reproducibility — the
|
||||
non-negotiable this run inherits.
|
||||
- `domains/data/migrations.md` — data versioning parallels schema
|
||||
migration discipline.
|
||||
- `domains/observability/logging.md` — the lineage record is a
|
||||
structured, append-only log of provenance.
|
||||
@@ -0,0 +1,197 @@
|
||||
# Good Example: GitOps Pull Request
|
||||
|
||||
> A pull request that changes ArgoCD Application manifests following
|
||||
> Atelier's GitOps + Operators principles. Each aspect cites the
|
||||
> principle it satisfies.
|
||||
|
||||
## The PR
|
||||
|
||||
A PR titled `promote payments-api 1.2.3 to prod` opened against the
|
||||
GitOps repo `platform/gitops`. It changes the `targetRevision` of the
|
||||
payments-api Application from `1.2.2` to `1.2.3`, adds a sync-wave
|
||||
annotation to a new migration Job, and tightens the AppProject's
|
||||
destination allow-list. CI runs plan/diff; nothing pushes to the
|
||||
cluster.
|
||||
|
||||
### The Commit
|
||||
|
||||
```yaml
|
||||
# manifests/prod/payments-api.yaml — the only file changed
|
||||
apiVersion: argoproj.io/v1alpha1
|
||||
kind: Application
|
||||
metadata:
|
||||
name: payments-api
|
||||
namespace: argocd
|
||||
finalizers:
|
||||
- resources-finalizer.argocd.argoproj.io
|
||||
spec:
|
||||
source:
|
||||
repoURL: https://git.example.com/platform/payments
|
||||
targetRevision: 1.2.3 # was 1.2.2 — pinned, not latest
|
||||
path: manifests/prod
|
||||
destination:
|
||||
server: https://kubernetes.default.svc
|
||||
namespace: payments
|
||||
syncPolicy:
|
||||
automated:
|
||||
prune: true
|
||||
selfHeal: true
|
||||
syncOptions:
|
||||
- CreateNamespace=false
|
||||
- PrunePropagationPolicy=foreground
|
||||
```
|
||||
|
||||
```yaml
|
||||
# manifests/prod/payments-db-migration.yaml — new file, wave-ordered
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: payments-db-migrate-1.2.3
|
||||
namespace: payments
|
||||
annotations:
|
||||
argocd.argoproj.io/sync-wave: "-1" # PreSync: run before the app
|
||||
spec:
|
||||
backoffLimit: 0
|
||||
ttlSecondsAfterFinished: 86400
|
||||
template:
|
||||
spec:
|
||||
restartPolicy: OnFailure
|
||||
serviceAccountName: payments-migrator
|
||||
containers:
|
||||
- name: migrate
|
||||
image: registry.example.com/payments/migrate:1.2.3
|
||||
args: ["up", "--to", "1.2.3"]
|
||||
```
|
||||
|
||||
```yaml
|
||||
# projects/payments.yaml — AppProject tightened in the same PR
|
||||
apiVersion: argoproj.io/v1alpha1
|
||||
kind: AppProject
|
||||
metadata:
|
||||
name: payments
|
||||
namespace: argocd
|
||||
spec:
|
||||
sourceRepos:
|
||||
- https://git.example.com/platform/payments
|
||||
destinations:
|
||||
- server: https://kubernetes.default.svc
|
||||
namespace: payments # prod-app can only deploy to payments ns
|
||||
clusterResourceWhitelist:
|
||||
- group: ""
|
||||
kind: Namespace # allowed to create its own namespace
|
||||
roles:
|
||||
- name: payments-team
|
||||
policies:
|
||||
- p, proj:payments:payments-team, applications, sync, payments/*, allow
|
||||
```
|
||||
|
||||
### The CI Pipeline (runs on the PR, before merge)
|
||||
|
||||
```text
|
||||
# .github/workflows/gitops-plan.yml (illustrative steps)
|
||||
- name: validate manifests
|
||||
run: argocd app manifests manifests/prod/ | kubeconform -strict
|
||||
|
||||
- name: diff against live cluster (read-only, no apply)
|
||||
run: argocd app diff payments-api --server $ARGOCD_SERVER --auth-token $READ_ONLY_TOKEN
|
||||
# CI holds a READ-ONLY ArgoCD token. It never holds kubectl rights.
|
||||
# A non-empty diff is the PR's proposed change, rendered for review.
|
||||
|
||||
- name: opa gate (admission policy pre-check)
|
||||
run: opa eval -i manifests/prod/ -d policies/ "data.k8s.admission.deny"
|
||||
# Policy violations fail the PR before merge, not after deploy.
|
||||
```
|
||||
|
||||
## What Makes It Good
|
||||
|
||||
### Git is the Source of Truth (GitOps P1, C1 Correctness)
|
||||
- The promotion is a commit. The cluster's desired state is a
|
||||
derivative of this repo; the repo is the authority. If the change is
|
||||
wrong, `git revert` is the rollback — the recovery path is the
|
||||
history.
|
||||
- See `domains/gitops-operators/first-principles.md` P1 and
|
||||
`domains/gitops-operators/argocd.md` (Application CRD).
|
||||
|
||||
### Pull, Don't Push (GitOps P3, C4 Locality)
|
||||
- CI holds a **read-only** ArgoCD token for `app diff`. It holds no
|
||||
`kubectl` rights against the production cluster. The cluster's
|
||||
ArgoCD controller pulls the merged commit; nothing pushes to the
|
||||
cluster. A compromised CI token can read, not deploy.
|
||||
- See `domains/gitops-operators/argocd.md` (RBAC and SSO) and
|
||||
`domains/gitops-operators/flux.md` for the same pull boundary from
|
||||
the Flux side.
|
||||
|
||||
### State is Immutable and Versioned (GitOps P5, C5 Reversibility)
|
||||
- `targetRevision: 1.2.3` — the Application pins a specific chart
|
||||
revision, not `latest`. The commit that changed it is a permanent
|
||||
record; `git revert` restores `1.2.2` and ArgoCD's `selfHeal`
|
||||
converges the cluster back. No force-push; history is the audit
|
||||
trail.
|
||||
- See `domains/gitops-operators/first-principles.md` P5 and
|
||||
`domains/infrastructure-as-code/state.md` (State is Truth).
|
||||
|
||||
### Sync Waves Order Correctness (GitOps P4, C1)
|
||||
- The migration Job carries `argocd.argoproj.io/sync-wave: "-1"` so
|
||||
it runs in `PreSync` before the payments-api Deployment that
|
||||
depends on the new schema. Wave ordering is a correctness
|
||||
mechanism, not performance — the app starting before its migration
|
||||
is a correctness bug.
|
||||
- See `domains/gitops-operators/argocd.md` (Sync Waves and Hooks).
|
||||
|
||||
### Reconcile, Don't Mutate by Hand (GitOps P8)
|
||||
- `selfHeal: true` + `prune: true` means a hand-edited drift on a
|
||||
managed resource is overwritten on the next loop. The fix for drift
|
||||
is a new commit, not `kubectl edit`. The PR author does not SSH into
|
||||
the cluster to "fix" anything.
|
||||
- See `domains/gitops-operators/argocd.md` (Diff and Drift) and
|
||||
`domains/gitops-operators/first-principles.md` P8.
|
||||
|
||||
### Least Privilege Reconciliation (GitOps P10, C8 Economy)
|
||||
- The AppProject `payments` restricts the Application to the
|
||||
`payments` namespace and the `payments` repo. The controller's
|
||||
ServiceAccount (not shown) is bound to a namespace-scoped Role, not
|
||||
`cluster-admin`. The PR *tightens* the allow-list — least privilege
|
||||
is a direction, not a one-time setting.
|
||||
- See `domains/gitops-operators/argocd.md` (RBAC and SSO) and
|
||||
`domains/kubernetes/rbac.md`.
|
||||
|
||||
### Policy is a Gate (Compliance P5, cross-link)
|
||||
- The `opa eval` step runs the admission policy against the proposed
|
||||
manifests before merge. A violation fails the PR; the non-compliant
|
||||
state is never realized. Detection is not enforcement; this is
|
||||
enforcement.
|
||||
- See `domains/compliance/policy-as-code.md` and
|
||||
`domains/devops/ci-cd.md`.
|
||||
|
||||
### Failure is Observable (GitOps P9)
|
||||
- A sync failure or health degradation on `payments-api` emits
|
||||
ArgoCD status (`Degraded` / `OutOfSync`) and a notification. Silent
|
||||
drift is the bug; this PR does not disable notifications.
|
||||
- See `domains/gitops-operators/argocd.md` (Health and Status) and
|
||||
`domains/observability/metrics.md`.
|
||||
|
||||
## What This PR Does NOT Do (And Why That's Good)
|
||||
|
||||
- Does **not** run `kubectl apply` from CI — that is the push pattern,
|
||||
a P3 violation (see `examples/bad/` for the anti-pattern).
|
||||
- Does **not** use `argocd app set` as the steady state — the change
|
||||
is in git, not in an imperative command's history.
|
||||
- Does **not** store raw Secrets in the GitOps repo — secrets arrive
|
||||
via Sealed Secrets / SOPS / External Secrets, encrypted in git.
|
||||
- Does **not** float `targetRevision: latest` — the Application pins
|
||||
a version; "latest" is an unknown model of the system.
|
||||
|
||||
## Cross-Domain Links
|
||||
|
||||
- `domains/gitops-operators/argocd.md` — the Application CRD, sync
|
||||
waves, RBAC/AppProjects, and the pull model.
|
||||
- `domains/gitops-operators/flux.md` — the same PR pattern from the
|
||||
Flux side (Kustomization CRD, per-cluster autonomy).
|
||||
- `domains/kubernetes/workloads.md` — the Deployment/Job the
|
||||
Application reconciles.
|
||||
- `domains/kubernetes/rbac.md` — the ServiceAccount + Role the
|
||||
controller and the migration Job run as.
|
||||
- `domains/compliance/policy-as-code.md` — the OPA gate is a
|
||||
compliance-as-a-gate enforcement point.
|
||||
- `domains/devops/P4 Rollback First` — `git revert` is the rollback;
|
||||
`selfHeal` is the convergence.
|
||||
+25
-10
@@ -6,14 +6,14 @@
|
||||
|
||||
| Core Principle | Domains that derive from it | Count |
|
||||
|----------------|---------------------------|-------|
|
||||
| C1 Correctness | All 13 (v0.1: 11; v0.2: infrastructure-as-code, kubernetes) | Universal |
|
||||
| C2 Clarity | v0.1: uiux, api, data, testing, observability, errors, documentation, devops; v0.2: infrastructure-as-code, kubernetes | 10 |
|
||||
| C3 Simplicity | v0.1: security, data, testing, performance, documentation, concurrency, devops | 7 |
|
||||
| C4 Locality | v0.1: testing, concurrency; v0.2: infrastructure-as-code, kubernetes | 4 |
|
||||
| C5 Reversibility | v0.1: api, data, uiux, concurrency, devops; v0.2: infrastructure-as-code, kubernetes | 7 |
|
||||
| C6 Composability | v0.1: api, security, observability, errors, documentation, concurrency; v0.2: infrastructure-as-code, kubernetes | 8 |
|
||||
| C7 Observability | v0.1: api, data, testing, performance, observability, errors, devops; v0.2: infrastructure-as-code, kubernetes | 9 |
|
||||
| C8 Economy | v0.1: security, testing, performance, observability, concurrency; v0.2: kubernetes | 6 |
|
||||
| C1 Correctness | All 17 (v0.1: 11; v0.2: infrastructure-as-code, kubernetes; v0.3: gitops-operators, ai-ml, i18n, compliance) | Universal |
|
||||
| C2 Clarity | v0.1: uiux, api, data, testing, observability, errors, documentation, devops; v0.2: infrastructure-as-code, kubernetes; v0.3: gitops-operators, ai-ml, i18n, compliance | 14 |
|
||||
| C3 Simplicity | v0.1: security, data, testing, performance, documentation, concurrency, devops; v0.2: infrastructure-as-code; v0.3: gitops-operators, i18n, compliance | 11 |
|
||||
| C4 Locality | v0.1: testing, concurrency; v0.2: infrastructure-as-code, kubernetes; v0.3: gitops-operators, i18n | 6 |
|
||||
| C5 Reversibility | v0.1: api, data, uiux, concurrency, devops; v0.2: infrastructure-as-code, kubernetes; v0.3: gitops-operators, ai-ml, i18n, compliance | 11 |
|
||||
| C6 Composability | v0.1: api, security, observability, errors, documentation, concurrency; v0.2: infrastructure-as-code, kubernetes; v0.3: gitops-operators, ai-ml, i18n, compliance | 12 |
|
||||
| C7 Observability | v0.1: api, data, testing, performance, observability, errors, devops; v0.2: infrastructure-as-code, kubernetes; v0.3: gitops-operators, ai-ml, i18n, compliance | 13 |
|
||||
| C8 Economy | v0.1: security, testing, performance, observability, concurrency; v0.2: kubernetes; v0.3: gitops-operators, i18n, compliance | 9 |
|
||||
|
||||
## Interpretation
|
||||
|
||||
@@ -39,6 +39,10 @@
|
||||
| DevOps | C1, C2, C3, C5, C7 | Reproducibility + rollback |
|
||||
| Infrastructure as Code | C1, C2, C3, C4, C5, C6, C7 | Declarative + state + composition; broadest derivation alongside Concurrency |
|
||||
| Kubernetes | C1, C2, C4, C5, C6, C7, C8 | Declarative + reversibility + economy; broad derivation (7 C-rules) |
|
||||
| GitOps + Operators | C1, C2, C3, C4, C5, C6, C7, C8 | Source-of-truth + reconciliation + pull-locality + least privilege; broadest derivation (8 C-rules, tied with i18n) |
|
||||
| AI / ML | C1, C2, C5, C6, C7 | Reproducibility + lineage + serving observability |
|
||||
| i18n | C1, C2, C3, C4, C5, C6, C7, C8 | Locale + formatting + direction + reversibility; broadest derivation (8 C-rules) |
|
||||
| Compliance | C1, C2, C3, C5, C6, C7, C8 | Audit + policy-as-code + retention + posture |
|
||||
|
||||
## v0.2 Domain Coverage (per IDEATE-03 schema)
|
||||
|
||||
@@ -47,10 +51,21 @@
|
||||
| Infrastructure as Code | 10 | 4 (terraform, opentofu, state, modules) | ✓ | complete |
|
||||
| Kubernetes | 10 | 6 (workloads, networking, storage, rbac, helm, kustomize) | ✓ | complete |
|
||||
|
||||
## v0.3 Domain Coverage (per IDEATE-03 schema)
|
||||
|
||||
| Domain | P-count | Derived-doc-count | Manifest-listed | Status |
|
||||
|--------|---------|-------------------|-----------------|--------|
|
||||
| GitOps + Operators | 10 | 4 (argocd, flux, operators, progressive-delivery) | ✓ | complete |
|
||||
| AI / ML | 10 | 4 (data-versioning, model-evaluation, serving, monitoring-drift) | ✓ | complete |
|
||||
| i18n | 10 | 4 (locale-resources, formatting, rtl-bidi, testing-i18n) | ✓ | complete |
|
||||
| Compliance | 10 | 4 (audit-logs, data-retention, policy-as-code, evidence) | ✓ | complete |
|
||||
|
||||
## Gaps and Notes
|
||||
|
||||
- No domain derives from only one C-rule. The minimum is 4 (UI/UX: C1, C2, C3, C5, C7 — actually 5). Every domain is multi-rooted.
|
||||
- **Concurrency**, **Infrastructure as Code**, and **Kubernetes** are tied for the broadest derivation (7 C-rules each) — these domains touch the most core concerns.
|
||||
- **Concurrency**, **Infrastructure as Code**, and **Kubernetes** are tied for the broadest derivation among v0.1/v0.2 platform domains (7 C-rules each) — these domains touch the most core concerns.
|
||||
- **GitOps + Operators** and **i18n** are tied for the single broadest-derivation domain overall (8 C-rules each: C1–C8). GitOps adds C4 (pull-credential locality) alongside its source-of-truth/reconciliation/least-privilege derivation; i18n touches correctness, clarity, simplicity, locality, reversibility, composability, observability, and economy (expansion accommodation). This is consistent with both domains' cross-cutting nature.
|
||||
- **UI/UX** and **API** are the most user-facing; they emphasize C2 (Clarity) heavily.
|
||||
- **Security** is the only domain with explicit non-tradeable declarations; this promotes 8 of its rules to C1-equivalent per `core/conflict-resolution.md` §6.
|
||||
- **v0.2 expansion:** C4 (Locality) grew from 2 to 4 domains (added infrastructure-as-code state locality, kubernetes namespace blast-radius). C6 (Composability) grew from 6 to 8. The two new domains are broad-derivation domains (7 C-rules each), consistent with Concurrency's breadth.
|
||||
- **v0.2 expansion:** C4 (Locality) grew from 2 to 4 domains (added infrastructure-as-code state locality, kubernetes namespace blast-radius). C6 (Composability) grew from 6 to 8. The two new domains are broad-derivation domains (7 C-rules each), consistent with Concurrency's breadth.
|
||||
- **v0.3 expansion:** C3 (Simplicity) grew from 7 to 11 (added gitops-operators declarative simplicity, i18n flexible layout, compliance structural redaction). C4 (Locality) grew from 4 to 6 (added gitops-operators pull-credential locality, i18n resource/text-direction locality). C5 (Reversibility) grew from 7 to 11 (added all four v0.3 domains — gitops history, ai-ml reproducibility, i18n translation versioning, compliance append-only/retention). C6 (Composability) grew from 8 to 12. C7 (Observability) grew from 9 to 13 (added all four v0.3 domains — reconciliation, drift detection, format correctness, posture). C2 (Clarity) grew from 10 to 14. The v0.3 expansion broadens every non-universal C-rule's coverage, confirming the four new domains are cross-cutting and well-rooted.
|
||||
@@ -205,8 +205,68 @@ C5=Reversibility · C6=Composability · C7=Observability · C8=Economy
|
||||
| P9 Config and Secrets Sep | C2 | Clarity of configuration |
|
||||
| P10 Roll Forward, Roll Back | C5 | Reversibility of deploys |
|
||||
|
||||
## Coverage Summary (post-v0.2)
|
||||
## GitOps + Operators
|
||||
|
||||
- 13 domains (11 v0.1 + 2 v0.2: infrastructure-as-code, kubernetes)
|
||||
- 130 domain principles total (110 v0.1 + 20 v0.2)
|
||||
| GitOps Principle | Core | Why |
|
||||
|---------------------------|------|---------------------------------------|
|
||||
| P1 Git is the Source of Truth | C1, C5 | Correctness of source; reversibility via history |
|
||||
| P2 Declarative Over Imperative | C2, C3 | Clarity of intent; simplicity of expression |
|
||||
| P3 Pull, Don't Push | C1, C4 | Correctness via security; locality of credentials |
|
||||
| P4 Continuous Reconciliation | C7, C1 | Observability of drift; correctness of converge loop |
|
||||
| P5 State is Immutable and Versioned | C5 | Reversibility through history |
|
||||
| P6 Operators Encode Domain Knowledge | C6, C2 | Composability of expertise; clarity of operations |
|
||||
| P7 Progressive Delivery is Reversible | C5, C1 | Reversibility of promotion; correctness of abort |
|
||||
| P8 Reconcile, Don't Mutate by Hand | C1, C7 | Correctness of source of truth; observability of drift |
|
||||
| P9 Failure is Observable and Surfaced | C7 | Observability of sync/rollout health |
|
||||
| P10 Least Privilege Reconciliation | C1, C8 | Correctness via security; economy of trust |
|
||||
|
||||
## AI / ML
|
||||
|
||||
| AI/ML Principle | Core | Why |
|
||||
|---------------------------|------|---------------------------------------|
|
||||
| P1 Reproducibility is the First Class | C1, C5 | Correctness of runs; reversibility of reproduction |
|
||||
| P2 Data is Versioned, Not Just Code | C5, C7 | Reversibility of data; observability of dataset lineage |
|
||||
| P3 Lineage is Traceable End-to-End | C7, C1 | Observability of predictions; correctness of provenance |
|
||||
| P4 Evaluation is Defined Before Training | C1, C2 | Correctness of metrics; clarity of thresholds |
|
||||
| P5 Models are Versioned Artifacts | C5, C6 | Reversibility of model rollbacks; composability of registry |
|
||||
| P6 Serving is Observable | C7 | Observability of inference |
|
||||
| P7 Drift is Expected and Detected | C7, C1 | Observability of drift; correctness of detection |
|
||||
| P8 Inference Inputs are Validated | C1 | Correctness at serving boundary |
|
||||
| P9 Pipelines Compose, Notebooks Don't | C6, C2 | Composability of steps; clarity of contracts |
|
||||
| P10 Rollback Includes the Model | C5 | Reversibility at the model layer |
|
||||
|
||||
## i18n
|
||||
|
||||
| i18n Principle | Core | Why |
|
||||
|---------------------------|------|---------------------------------------|
|
||||
| P1 Source Language is a Locale, Not the Default | C2, C1 | Clarity; correctness of localization model |
|
||||
| P2 Locale Identifiers are Standardized | C2, C6 | Clarity; composability of BCP 47 |
|
||||
| P3 Resources are External, Not Inline | C4, C6 | Locality of strings; composability of resources |
|
||||
| P4 Plural and Gender are Parameterized | C1, C6 | Correctness across locales; composability of message format |
|
||||
| P5 Formatting is Locale-Aware | C1, C7 | Correctness of formats; observability of format correctness |
|
||||
| P6 Text Direction is a Layout Primitive | C1, C4 | Correctness of RTL/bidi; locality of direction |
|
||||
| P7 Layout Accommodates Expansion | C8, C3 | Economy of rework; simplicity of flexible layout |
|
||||
| P8 Pseudo-Locales Test Early | C7, C5 | Observability of bugs early; reversibility of finding late |
|
||||
| P9 Images and Icons are Cultural | C1, C2 | Correctness; clarity of cultural meaning |
|
||||
| P10 Translation is Reversible and Versioned | C5 | Reversibility of localization changes |
|
||||
|
||||
## Compliance
|
||||
|
||||
| Compliance Principle | Core | Why |
|
||||
|---------------------------|------|---------------------------------------|
|
||||
| P1 Audit Logs are Append-Only | C1, C5 | Correctness of audit; reversibility of immutable record |
|
||||
| P2 Every Significant Action is Logged | C7, C1 | Observability of actions; correctness of audit set |
|
||||
| P3 Retention is Policy, Not Storage | C5, C8 | Reversibility of lifecycle; economy of storage |
|
||||
| P4 Policy is Code | C6, C2 | Composability of policy; clarity of rules |
|
||||
| P5 Policy is Evaluated as a Gate | C1, C5 | Correctness of enforcement; reversibility of block |
|
||||
| P6 Evidence is Collected Continuously | C7, C3 | Observability of posture; simplicity of audit |
|
||||
| P7 Identity is Attributable | C1, C7 | Correctness of attribution; observability of subject |
|
||||
| P8 Subject Access is Honored | C1, C5 | Correctness of rights; reversibility of deletion/export |
|
||||
| P9 Secrets and Sensitive Data are Redacted in Audit | C1, C3 | Correctness via security; simplicity of structural redaction |
|
||||
| P10 Compliance Posture is Observable | C7, C1 | Observability of compliance; correctness of posture |
|
||||
|
||||
## Coverage Summary (post-v0.3)
|
||||
|
||||
- 17 domains (11 v0.1 + 2 v0.2: infrastructure-as-code, kubernetes; 4 v0.3: gitops-operators, ai-ml, i18n, compliance)
|
||||
- 170 domain principles total (110 v0.1 + 20 v0.2 + 40 v0.3)
|
||||
- Every domain P-rule traces to ≥1 core C-rule (C1–C8). No orphans.
|
||||
@@ -147,6 +147,56 @@ If the task touches a domain, run that domain's checklist:
|
||||
- [ ] Rollout history retained; rollback tested (P10)
|
||||
- [ ] Namespaces used to bound blast radius; not `default` in prod (P6)
|
||||
|
||||
### If GitOps + Operators (see `domains/gitops-operators/`)
|
||||
- [ ] Desired state lives in git, not in the cluster (P1)
|
||||
- [ ] Configuration is declarative, not imperative scripts (P2)
|
||||
- [ ] Reconciliation is pull-based; no external push credentials into the cluster (P3)
|
||||
- [ ] Reconciliation loop runs continuously; drift auto-corrected (P4)
|
||||
- [ ] Every change is a commit; history is the audit/rollback path (P5)
|
||||
- [ ] Operational knowledge encoded as CRDs/controllers, not runbooks (P6)
|
||||
- [ ] Progressive delivery (canary/blue-green) has a tested abort/rollback path (P7)
|
||||
- [ ] No manual `kubectl apply`/`kubectl edit` on GitOps-managed resources (P8)
|
||||
- [ ] Sync failures, health degradation, and rollout stalls emit status + notifications (P9)
|
||||
- [ ] Controller credentials scoped to reconciled namespaces/resources; no cluster-admin GitOps robot (P10)
|
||||
|
||||
### If AI / ML (see `domains/ai-ml/`)
|
||||
- [ ] Scope check: this is engineering discipline (data versioning, evaluation, serving, drift), NOT algorithm/model design (D-023) — reject algorithm-design content
|
||||
- [ ] Every training run is reproducible from pinned data + code + config + environment (P1)
|
||||
- [ ] Datasets, features, and splits are versioned artifacts with lineage; `git` alone is insufficient (P2)
|
||||
- [ ] Any deployed prediction traces back through model → training run → dataset → source (P3)
|
||||
- [ ] Metrics, splits, and thresholds declared a priori; no post-hoc metric cherry-picking (P4)
|
||||
- [ ] Models are pinned, immutable, registry-tracked artifacts; never "the latest" (P5)
|
||||
- [ ] Inference latency, throughput, input distributions, and prediction confidence are observed (P6)
|
||||
- [ ] Data drift, concept drift, and prediction drift are monitored; a drift signal is an incident (P7)
|
||||
- [ ] Inference inputs validated against the model's contract (schema, ranges, types); out-of-contract rejected (P8)
|
||||
- [ ] Training/serving flows are composable pipelines with explicit steps; notebooks not in production (P9)
|
||||
- [ ] Serving rollback restores the prior model artifact, not just the prior code (P10)
|
||||
|
||||
### If i18n (see `domains/i18n/`)
|
||||
- [ ] Source language treated as one locale among many, not the "neutral" default (P1)
|
||||
- [ ] Locale identifiers use BCP 47 tags; no ad-hoc locale codes (P2)
|
||||
- [ ] User-facing strings in locale resource files, not concatenated inline in code (P3)
|
||||
- [ ] Plural/gender/select use ICU MessageFormat (or equivalent); no `if (n == 1)` branching (P4)
|
||||
- [ ] Dates, times, numbers, currencies, units via ICU/CLDR/`Intl`; no hand-rolled formatters (P5)
|
||||
- [ ] RTL/bidi is a first-class layout concern; logical CSS properties (`start`/`end`) over physical (`left`/`right`) (P6)
|
||||
- [ ] Layouts accommodate translation expansion; no fixed pixel widths for text (P7)
|
||||
- [ ] Pseudo-locales (accented, lengthened, RTL-mirrored) used to test before real translations arrive (P8)
|
||||
- [ ] Icons, colors, and imagery reviewed for locale-sensitivity; no locale-bound symbols treated as universal (P9)
|
||||
- [ ] Resource files versioned; a bad translation is a rollback, not a hot-patch (P10)
|
||||
|
||||
### If Compliance (see `domains/compliance/`)
|
||||
- [ ] Scope check: framework-agnostic — no regulation-specific (GDPR/HIPAA/SOC2/PCI) content (D-024)
|
||||
- [ ] Audit records are immutable once written; deletion/mutation is itself an auditable incident (P1)
|
||||
- [ ] The set of auditable actions is defined a priori; "we forgot to log it" is a violation (P2)
|
||||
- [ ] Data lifetime is declared and enforced as policy; deletion at end-of-life is a feature (P3)
|
||||
- [ ] Compliance policy expressed in versioned, reviewable, testable code (OPA/Cedar/Kyverno/Sentinel), not spreadsheets/prose (P4)
|
||||
- [ ] Policy violations block before the action (admission/CI/CD-time), not after the audit (P5)
|
||||
- [ ] Evidence gathered as a byproduct of operation, not assembled manually at audit time (P6)
|
||||
- [ ] Every logged action traces to an authenticated principal; no shared/generic identities (P7)
|
||||
- [ ] Data-subject rights (access, export, deletion) are operations with defined contracts and audit trails (P8)
|
||||
- [ ] Audit logs do not leak secrets; redaction is structural, not opportunistic (P9)
|
||||
- [ ] System reports its own compliance state (drift from policy, open violations, retention status) (P10)
|
||||
|
||||
## Final Gate
|
||||
|
||||
- [ ] Have I read the relevant domain's first-principles?
|
||||
|
||||
+62
-1
@@ -163,4 +163,65 @@ When you see a pattern listed here, it is a defect. Cite the principle it violat
|
||||
|--------------|----------|-----|
|
||||
| Orphaned P-rule (a domain principle with no matrix row) | matrix completeness, C6 | Breaks the conflict-resolution arbiter; the rule has no core trace |
|
||||
| Deployable example artifact (standalone `.tf`/`.yaml` under `examples/`) | PROJECT.md "no runtime code", D-025 | Violates the docs-only contract; examples must be `.md` with fenced code |
|
||||
| Unlisted v0.2 doc (new doc not added to MANIFEST) | manifest rule | Not part of the framework by definition |
|
||||
| Unlisted v0.2 doc (new doc not added to MANIFEST) | manifest rule | Not part of the framework by definition |
|
||||
|
||||
## v0.3 Chaos Anti-Patterns (from IDEATE-20, IDEATE-24, IDEATE-25, IDEATE-27)
|
||||
|
||||
These are named, cross-cutting violations specific to the v0.3 domains. Reject on sight.
|
||||
|
||||
| Anti-Pattern | Breaches | Why |
|
||||
|--------------|----------|-----|
|
||||
| GitOps push-pattern (external CI pushes manifests to the cluster instead of an in-cluster agent pulling from git) | gitops P3 Pull, Don't Push; C1, C4 | Inverts the source-of-truth flow; requires push credentials into the cluster; breaks the reconciliation model (IDEATE-24, D-042) |
|
||||
| i18n LTR-only assumption (layout assumes left-to-right; no `dir` attribute, physical CSS properties only) | i18n P6 Text Direction is a Layout Primitive; C1, C4 | Disqualifying for RTL/Bidi users; locale-correctness violation (IDEATE-25, D-043) |
|
||||
| AI/ML orphan-model (a deployed prediction endpoint whose model has no lineage trace — no record of training run, dataset, or version) | ai-ml P3 Lineage is Traceable End-to-End; C7, C1 | Unreviewable, unrollbackable; the model is an unattributed artifact (IDEATE-27, D-045) |
|
||||
| Compliance mutable audit log (audit records can be edited or deleted by an operator) | compliance P1 Audit Logs are Append-Only; C1, C5 | Destroys the audit trail; the audit log's value is immutability — mutation is itself an incident |
|
||||
|
||||
### v0.3 Deployable Artifact Types (IDEATE-20, D-020)
|
||||
|
||||
The following standalone file types are forbidden under `examples/` and elsewhere in the framework. Examples are `.md` files with fenced code only.
|
||||
|
||||
| Forbidden standalone artifact | Belongs in | Why |
|
||||
|-------------------------------|-----------|-----|
|
||||
| `.po` / `.pot` resource files | fenced code in `examples/good/`/`examples/bad/*.md` | Runtime localization artifact; violates docs-only contract |
|
||||
| `.rego` / `.cedar` / `.sentinel` policy files | fenced code in `examples/*.md` | Runtime policy artifact; violates docs-only contract |
|
||||
| Model artifacts (`.pkl`, `.onnx`, `.pt`, `.h5`, `.safetensors`) | fenced code + prose in `examples/*.md` | Runtime model artifact; violates docs-only contract |
|
||||
| Signed manifests as standalone files (`.sig`, `.att`, `.intoto.jsonl`) | fenced code in `examples/*.md` | Runtime attestation artifact; violates docs-only contract |
|
||||
| Standalone `.yaml` / `.tf` / `.sh` | fenced code in `examples/*.md` | (Carried forward from v0.2) Runtime deployable artifact |
|
||||
|
||||
## v0.3 Domain-Specific Anti-Patterns
|
||||
|
||||
### GitOps + Operators
|
||||
|
||||
| Anti-Pattern | Breaches | Why |
|
||||
|--------------|----------|-----|
|
||||
| Push-based deploy (external CI `kubectl apply` into the cluster) | P3 Pull, Don't Push | Inverts the model; requires push credentials; bypasses reconciliation |
|
||||
| Manual `kubectl apply`/`kubectl edit` on a GitOps-managed resource | P8 Reconcile, Don't Mutate by Hand | Unreconciled drift; the next loop overwrites it — silent and unattributed |
|
||||
| `cluster-admin` GitOps robot (controller bound to cluster-admin) | P10 Least Privilege Reconciliation | Overbroad grant; blast radius = entire cluster |
|
||||
| No sync-failure notification (silent drift on health degradation) | P9 Failure is Observable and Surfaced | Silent drift is the bug the loop was supposed to surface |
|
||||
|
||||
### AI / ML
|
||||
|
||||
| Anti-Pattern | Breaches | Why |
|
||||
|--------------|----------|-----|
|
||||
| Unreproducible training run (unpinned data, code, config, or environment) | P1 Reproducibility is the First Class | Unreviewable; cannot debug, cannot rollback |
|
||||
| "Use the latest model" (serving points at `model:latest` instead of a pinned version) | P5 Models are Versioned Artifacts | Unversioned drift; rollback undefined |
|
||||
| Notebook in production (training/serving flow is a Jupyter notebook) | P9 Pipelines Compose, Notebooks Don't | No contracts, no composition, no reproducibility |
|
||||
| Orphan model (deployed prediction with no lineage trace) | P3 Lineage is Traceable End-to-End | Unattributed artifact; cannot trace to data/code (IDEATE-27) |
|
||||
|
||||
### i18n
|
||||
|
||||
| Anti-Pattern | Breaches | Why |
|
||||
|--------------|----------|-----|
|
||||
| Inline string concatenation (`"Hello, " + name + "!"` in code) | P3 Resources are External, Not Inline | Not extractable; breaks translations; word-order differs per locale |
|
||||
| `if (n == 1)` plural branching (hand-rolled plural logic) | P4 Plural and Gender are Parameterized | Wrong for Arabic, Russian, Polish; ICU MessageFormat handles plurals |
|
||||
| LTR-only layout (no `dir` attribute, physical CSS `left`/`right`) | P6 Text Direction is a Layout Primitive | Disqualifying for RTL/Bidi (IDEATE-25) |
|
||||
| Hand-rolled date/number formatter (`new Date().toString()`, manual string formatting) | P5 Formatting is Locale-Aware | Locale-incorrect; ignores ICU/CLDR |
|
||||
|
||||
### Compliance
|
||||
|
||||
| Anti-Pattern | Breaches | Why |
|
||||
|--------------|----------|-----|
|
||||
| Mutable audit log (operator can `UPDATE`/`DELETE` audit records) | P1 Audit Logs are Append-Only | Destroys the audit trail; mutation is itself an incident |
|
||||
| Shared/generic identity in audit (`admin` or `system` as the actor for all actions) | P7 Identity is Attributable | No attribution; no accountability; cannot investigate |
|
||||
| Secret leaked in audit log (request body or token captured in an audit event) | P9 Secrets and Sensitive Data are Redacted in Audit | Audit log becomes a secret exfiltration channel |
|
||||
| Manual evidence assembly at audit time (scramble to collect logs/scans/attestations on demand) | P6 Evidence is Collected Continuously | Audit-unready; evidence gathered under pressure is incomplete and unreliable |
|
||||
@@ -79,6 +79,56 @@ Run the relevant domain section from `agent-checklist.md` (UI/UX, API, Security,
|
||||
- [ ] Are ConfigMaps and Secrets separate?
|
||||
- [ ] Is the rollback path tested, not assumed?
|
||||
|
||||
### If GitOps + Operators
|
||||
- [ ] Is desired state sourced from git, not from the cluster?
|
||||
- [ ] Is configuration declarative, not imperative scripts?
|
||||
- [ ] Is reconciliation pull-based (no external push credentials into the cluster)?
|
||||
- [ ] Does the reconciliation loop run continuously and auto-correct drift?
|
||||
- [ ] Is every change a commit, with history as the audit/rollback path?
|
||||
- [ ] Is operational knowledge encoded as CRDs/controllers, not runbooks humans must remember?
|
||||
- [ ] Does progressive delivery (canary/blue-green) have a tested abort/rollback path?
|
||||
- [ ] Are there manual `kubectl apply`/`kubectl edit` on GitOps-managed resources? (flag as incident)
|
||||
- [ ] Do sync failures, health degradation, and rollout stalls emit status + notifications?
|
||||
- [ ] Are controller credentials scoped to reconciled namespaces/resources (no cluster-admin GitOps robot)?
|
||||
|
||||
### If AI / ML
|
||||
- [ ] Scope check: is this engineering discipline (data versioning, evaluation, serving, drift), NOT algorithm/model design? (D-023 — reject algorithm-design content)
|
||||
- [ ] Is every training run reproducible from pinned data + code + config + environment?
|
||||
- [ ] Are datasets, features, and splits versioned artifacts with lineage (not just `git`)?
|
||||
- [ ] Can any deployed prediction trace back through model → training run → dataset → source?
|
||||
- [ ] Are metrics, splits, and thresholds declared a priori (no post-hoc metric cherry-picking)?
|
||||
- [ ] Are models pinned, immutable, registry-tracked artifacts (never "the latest")?
|
||||
- [ ] Is inference observable (latency, throughput, input distributions, prediction confidence)?
|
||||
- [ ] Are data drift, concept drift, and prediction drift monitored (drift signal = incident)?
|
||||
- [ ] Are inference inputs validated against the model's contract (schema, ranges, types)?
|
||||
- [ ] Are training/serving flows composable pipelines (not notebooks in production)?
|
||||
- [ ] Does serving rollback restore the prior model artifact, not just the prior code?
|
||||
|
||||
### If i18n
|
||||
- [ ] Is the source language treated as one locale among many, not the "neutral" default?
|
||||
- [ ] Do locale identifiers use BCP 47 tags (no ad-hoc locale codes)?
|
||||
- [ ] Are user-facing strings in locale resource files (not concatenated inline in code)?
|
||||
- [ ] Do plural/gender/select use ICU MessageFormat (no `if (n == 1)` branching)?
|
||||
- [ ] Are dates, times, numbers, currencies, units formatted via ICU/CLDR/`Intl` (no hand-rolled formatters)?
|
||||
- [ ] Is RTL/bidi a first-class layout concern (logical CSS properties over physical)?
|
||||
- [ ] Do layouts accommodate translation expansion (no fixed pixel widths for text)?
|
||||
- [ ] Are pseudo-locales used to test before real translations arrive?
|
||||
- [ ] Are icons, colors, and imagery reviewed for locale-sensitivity?
|
||||
- [ ] Are resource files versioned (bad translation = rollback, not hot-patch)?
|
||||
|
||||
### If Compliance
|
||||
- [ ] Scope check: is this framework-agnostic (no regulation-specific GDPR/HIPAA/SOC2/PCI content)? (D-024)
|
||||
- [ ] Are audit records immutable once written (deletion/mutation is itself an auditable incident)?
|
||||
- [ ] Is the set of auditable actions defined a priori ("we forgot to log it" is a violation)?
|
||||
- [ ] Is data lifetime declared and enforced as policy (deletion at end-of-life is a feature)?
|
||||
- [ ] Is compliance policy expressed in versioned, reviewable, testable code (not spreadsheets/prose)?
|
||||
- [ ] Do policy violations block before the action (admission/CI/CD-time, not after the audit)?
|
||||
- [ ] Is evidence gathered as a byproduct of operation (not assembled manually at audit time)?
|
||||
- [ ] Does every logged action trace to an authenticated principal (no shared/generic identities)?
|
||||
- [ ] Are data-subject rights (access, export, deletion) operations with defined contracts and audit trails?
|
||||
- [ ] Do audit logs avoid leaking secrets (redaction is structural, not opportunistic)?
|
||||
- [ ] Does the system report its own compliance state (drift from policy, open violations, retention status)?
|
||||
|
||||
## Review Etiquette
|
||||
|
||||
- **Comment, don't command.** "This could be X" not "Change this to X."
|
||||
|
||||
Reference in New Issue
Block a user