From 4c8700e5a4db682ab293259a497448036019fecf Mon Sep 17 00:00:00 2001 From: Jon Chery Date: Wed, 5 Aug 2026 03:40:37 +0000 Subject: [PATCH] docs(P5): complete examples + cross-links phase ---ci--- project: atelier phase: 5 milestone: v0.3 status: complete requirements: covered: [ATELIER-86, ATELIER-87, ATELIER-88] partial: [] ---/ci--- --- MANIFEST.md | 38 ++--- examples/bad/compliance-audit-log.md | 222 ++++++++++++++++++++++++ examples/bad/i18n-string-concat.md | 175 +++++++++++++++++++ examples/good/ai-ml-reproducibility.md | 226 +++++++++++++++++++++++++ examples/good/gitops-pr.md | 197 +++++++++++++++++++++ 5 files changed, 839 insertions(+), 19 deletions(-) create mode 100644 examples/bad/compliance-audit-log.md create mode 100644 examples/bad/i18n-string-concat.md create mode 100644 examples/good/ai-ml-reproducibility.md create mode 100644 examples/good/gitops-pr.md diff --git a/MANIFEST.md b/MANIFEST.md index 93a1556..f98fb09 100644 --- a/MANIFEST.md +++ b/MANIFEST.md @@ -43,29 +43,29 @@ ## Examples -> Examples are illustrative markdown with fenced code only (no standalone runtime artifacts per D-020 / D-025). The `examples/` directory listing closes the v0.2 ESC-002 drift (IDEATE-17, ATELIER-91). P5 examples populate this section. +> Examples are illustrative markdown with fenced code only (no standalone runtime artifacts per D-020 / D-025). The `examples/` directory listing closes the v0.2 ESC-002 drift (IDEATE-17, ATELIER-91). P5 authored the v0.3 examples and promoted all entries from `pending` to `✓` (verified — every listed file exists). | Path | Status | Notes | |------|--------|-------| -| `examples/good/` | (pending P5) | Good-example directory — to be populated by v0.3 P5 | -| `examples/bad/` | (pending P5) | Bad-example directory — to be populated by v0.3 P5 | -| `examples/good/api-endpoint.md` | (pending P5) | v0.1 example — to be listed when P5 back-fills | -| `examples/good/react-component.md` | (pending P5) | v0.1 example — to be listed when P5 back-fills | -| `examples/good/db-schema.md` | (pending P5) | v0.1 example — to be listed when P5 back-fills | -| `examples/good/error-handler.md` | (pending P5) | v0.1 example — to be listed when P5 back-fills | -| `examples/bad/god-object.md` | (pending P5) | v0.1 example — to be listed when P5 back-fills | -| `examples/bad/silent-error.md` | (pending P5) | v0.1 example — to be listed when P5 back-fills | -| `examples/bad/leaky-abstraction.md` | (pending P5) | v0.1 example — to be listed when P5 back-fills | -| `examples/good/terraform-module.md` | (pending P5) | v0.2 example — to be listed when P5 back-fills | -| `examples/good/k8s-deployment.md` | (pending P5) | v0.2 example — to be listed when P5 back-fills | -| `examples/bad/terraform-unlocked-state.md` | (pending P5) | v0.2 example — to be listed when P5 back-fills | -| `examples/bad/k8s-bare-pod-no-resources.md` | (pending P5) | v0.2 example — to be listed when P5 back-fills | -| `examples/good/gitops-pr.md` | (pending P5) | v0.3 example — to be authored in P5 | -| `examples/good/ai-ml-reproducibility.md` | (pending P5) | v0.3 example — to be authored in P5 | -| `examples/bad/i18n-string-concat.md` | (pending P5) | v0.3 example — to be authored in P5 | -| `examples/bad/compliance-audit-log.md` | (pending P5) | v0.3 example — to be authored in P5 | +| `examples/good/` | ✓ | Good-example directory — 8 examples (v0.1 + v0.2 + v0.3) | +| `examples/bad/` | ✓ | Bad-example directory — 7 examples (v0.1 + v0.2 + v0.3) | +| `examples/good/api-endpoint.md` | ✓ | v0.1 example — good REST endpoint | +| `examples/good/react-component.md` | ✓ | v0.1 example — good React component | +| `examples/good/db-schema.md` | ✓ | v0.1 example — good DB schema | +| `examples/good/error-handler.md` | ✓ | v0.1 example — good error handler | +| `examples/bad/god-object.md` | ✓ | v0.1 example — bad god object | +| `examples/bad/silent-error.md` | ✓ | v0.1 example — bad silent error | +| `examples/bad/leaky-abstraction.md` | ✓ | v0.1 example — bad leaky abstraction | +| `examples/good/terraform-module.md` | ✓ | v0.2 example — good IaC module | +| `examples/good/k8s-deployment.md` | ✓ | v0.2 example — good k8s deployment | +| `examples/bad/terraform-unlocked-state.md` | ✓ | v0.2 example — bad unlocked state | +| `examples/bad/k8s-bare-pod-no-resources.md` | ✓ | v0.2 example — bad bare pod | +| `examples/good/gitops-pr.md` | ✓ | v0.3 example — good GitOps PR | +| `examples/good/ai-ml-reproducibility.md` | ✓ | v0.3 example — good reproducible training run | +| `examples/bad/i18n-string-concat.md` | ✓ | v0.3 example — bad i18n string concat | +| `examples/bad/compliance-audit-log.md` | ✓ | v0.3 example — bad audit log (P1 + P9 breaches) | -> **Note:** The `examples/` section structure is established here (P4) so P5 can populate it. Files marked "pending P5" do not yet exist; they will be authored in Phase 5 and promoted from `pending` to `✓` upon completion. Listing them here as `pending` makes the manifest authoritative about what *will* exist and prevents drift. +> **Note:** The `examples/` section was established in P4 with entries pre-listed as `pending P5`. P5 authored the 4 v0.3 examples and promoted all entries to `✓` after verifying every listed file exists on disk. The manifest remains authoritative — unlisted = not part of framework. ## Cross-Cutting diff --git a/examples/bad/compliance-audit-log.md b/examples/bad/compliance-audit-log.md new file mode 100644 index 0000000..ab48cf0 --- /dev/null +++ b/examples/bad/compliance-audit-log.md @@ -0,0 +1,222 @@ +# Bad Example: Compliance Audit Log (Two Breaches) + +> An audit logging implementation that violates **two** Atelier +> compliance principles in one example (per IDEATE-26, D-044): +> **P1** (Audit Logs are Append-Only) — a mutable audit log with +> routine `DELETE`/`UPDATE` "cleanup" — and **P9** (Secrets and +> Sensitive Data are Redacted in Audit) — a database password leaked +> into an audit record. Each violation is cited, then fixed. + +## The Code + +```python +# audit_log.py — the audit sink, stored in a mutable Postgres table +import psycopg2, datetime + +# P1 VIOLATION: the audit log is a regular mutable table. There is no +# write-once protection, no immutable bucket, no hash-chaining. +# Any DB user with UPDATE/DELETE can rewrite history. +CREATE_TABLE = """ +CREATE TABLE audit_log ( + id BIGSERIAL PRIMARY KEY, + timestamp TIMESTAMPTZ NOT NULL, + event TEXT NOT NULL, + actor TEXT NOT NULL, + target TEXT, + payload JSONB, + request_id TEXT +); +-- No row-level immutability. No trigger preventing UPDATE/DELETE. +""" + +def write_event(event, actor, target=None, payload=None, request_id=None): + conn = psycopg2.connect(os.environ["DATABASE_URL"]) + conn.execute( + "INSERT INTO audit_log (timestamp, event, actor, target, payload, request_id) " + "VALUES (%s, %s, %s, %s, %s, %s)", + (datetime.datetime.utcnow(), event, actor, target, + json.dumps(payload), request_id), + ) + +# P1 VIOLATION (continued): "cleanup" that mutates the audit log. +# A routine job deletes records older than 30 days to "save space" +# and updates records to "fix typos in the actor field." +def cleanup_audit_log(): + conn = psycopg2.connect(os.environ["DATABASE_URL"]) + # DELETE — an audit record is destroyed. This is tampering, + # dressed as housekeeping. + conn.execute("DELETE FROM audit_log WHERE timestamp < NOW() - INTERVAL '30 days'") + # UPDATE — an audit record is rewritten. The "fix" is the + # violation; the original actor is lost. + conn.execute("UPDATE audit_log SET actor = 'admin' WHERE actor LIKE 'svc-%'") +``` + +```python +# The call site that leaks a secret into the audit log. +def read_config(key): + # ... fetches a secret from the secrets manager ... + value = secrets_manager.get(key) # e.g. the raw DB password + # P9 VIOLATION: the raw secret value is written into the audit + # payload. The append-only log is now a secret store. + write_event( + event="config.read", + actor="api-server", + target={"kind": "secret", "id": key}, + payload={"value": value}, # <- the secret, in plaintext + request_id=req.id, + ) + return value +``` + +The resulting audit record: + +```json +{ + "id": 48213, + "timestamp": "2026-08-05T09:12:03Z", + "event": "config.read", + "actor": "api-server", + "target": {"kind": "secret", "id": "db-password"}, + "payload": {"value": "p@ssw0rd-sup3r-s3cr3t-plaintext"}, + "request_id": "req_91c2" +} +``` + +A week later, the `cleanup_audit_log` job `DELETE`s this record (it +is older than 30 days in the team's "retention" — which is actually a +storage-economy decision, not a policy), and `UPDATE`s every +`svc-*` actor to `admin`. The secret was in the log for a week, +readable by anyone with `SELECT` on the table; now the record of it +having been there is gone. + +## What Makes It Bad + +### Breach 1 — Mutable Audit Log (Compliance P1 Audit Logs are Append-Only) +- The audit log is a regular mutable Postgres table. `DELETE FROM + audit_log WHERE timestamp < ...` and `UPDATE audit_log SET actor = + ...` both succeed. The log is a draft, not a record. +- Routine `DELETE` as "cleanup" is the cardinal P1 violation: the + deletion of an audit record is itself an auditable incident, not a + housekeeping task. "We deleted old records to save space" is a P3 + (Retention is Policy, Not Storage) violation *and* a P1 violation — + the retention decision is driven by storage cost, and the + mechanism is tampering. +- The `UPDATE` that rewrites `svc-deploy` → `admin` destroys + attribution (a P7 violation stacked on the P1 violation): the + original actor is lost, and the replacement (`admin`) is a shared + identity that could be any of ten engineers. +- **Fix:** the audit sink is append-only *by construction*, not by + policy. Write-once storage (WORM bucket, immutable log stream, + hash-chained ledger) enforces immutability at the substrate. + Retention is a declared policy with a meta-audit of deletions; a + human does not run ad-hoc `DELETE` jobs. + + ```python + # Fix: write to an append-only sink (illustrative — S3 Object Lock, + # WORM bucket, or a hash-chained ledger). The API has no update / + # delete path; the storage refuses mutation. + def write_event(event, actor, target=None, payload=None, request_id=None): + record = { + "timestamp": datetime.datetime.now(datetime.timezone.utc).isoformat(), + "event": event, + "actor": actor, # the authenticated principal, not "admin" + "target": target, + "payload": redact(payload), # see Breach 2 fix + "request_id": request_id, + "prev_hash": last_hash(), # hash-chaining: tampering breaks the chain + } + record["hash"] = sha256(canonical_json(record)) + append_only_sink.write(record) # WORM storage; no update/delete API exists + + # Fix: retention is a declared, reviewed policy — not an ad-hoc DELETE. + # When an audit segment ages out, the deletion is itself meta-audited + # in a higher-tier log with the rule that authorized it. + # (See domains/compliance/data-retention.md and audit-logs.md.) + ``` +- See `domains/compliance/audit-logs.md` (Audit Logs are Append-Only) + and `domains/compliance/first-principles.md` P1. + +### Breach 2 — Secret Leaked in Audit Log (Compliance P9 Secrets and Sensitive Data are Redacted in Audit) +- `payload={"value": value}` writes the raw DB password into the + audit record. The append-only log is now a secret store: anyone + with `SELECT` on `audit_log` can read production credentials. The + log is harder to secure than the secrets manager it read from. +- Once the secret is in an append-only log, the remediation is + expensive — rotate the secret *and* rewrite the log's access scope + (you cannot edit the record; it is append-only). Redaction must + happen *at the logging boundary, before the record is written*, not + by opportunistic scrubbing after the fact. +- The redaction policy here is "nothing" — there is no rule for + which fields are redacted, by what mechanism, in which event type. + A redaction rule that lives in no one's head and no code is a P9 + violation waiting to happen (and it happened). +- **Fix:** redaction is structural, applied at the logging boundary + before the record reaches the append-only sink. The policy is + itself auditable (which fields, by what rule, in which event). + + ```python + # Fix: redaction at the boundary. Log the FACT of the action + # (a secret was read), never the CONTENT of the secret. + REDACTED_FIELDS = {"value", "token", "password", "authorization", "secret"} + + def redact(payload): + if not isinstance(payload, dict): + return "[REDACTED:non-object]" + out = {} + for k, v in payload.items(): + if k.lower() in REDACTED_FIELDS or "secret" in k.lower(): + out[k] = "[REDACTED:secret]" + else: + out[k] = v + out["_redaction"] = "secret-value-policy/v1" # the rule is auditable + return out + + # The fixed audit record: + # { + # "event": "config.read", + # "actor": "api-server", # the authenticated principal + # "target": {"kind": "secret", "id": "db-password"}, + # "payload": {"value": "[REDACTED:secret]"}, + # "_redaction": "secret-value-policy/v1", + # "request_id": "req_91c2" + # } + # The fact of the read is logged; the secret never enters the log. + ``` +- See `domains/compliance/audit-logs.md` (Redaction at the Boundary) + and `domains/compliance/first-principles.md` P9. Cross + `domains/security/secrets.md` — the audit-side redaction is the + complement of secret management. + +## The Cascade (Two Breaches Compound) + +The two violations compound destructively. The secret enters the +mutable log (P9 breach), where it sits readable by any `SELECT`-holder +for a week. Then the `cleanup` job `DELETE`s the record (P1 breach) — +destroying the evidence that the secret was ever logged, while the +secret itself has already been exposed to every reader of the table. +The `UPDATE` that rewrites `svc-deploy` → `admin` (a P7 attribution +breach stacked on the P1 breach) means that even if a copy of the +record survived, the actor who triggered the secret read is no longer +identifiable. The team cannot answer "who read the DB password and +when" — the log that would answer it was mutated, and the secret it +leaked is now in the wild. This is the worst-case interaction of P1 +and P9: a secret leak with no attributable actor and no surviving +record. + +## Cross-Domain Links + +- `domains/compliance/audit-logs.md` — the append-only guarantee and + the redaction-at-boundary rule this code violates. +- `domains/compliance/first-principles.md` — P1 (Append-Only) and P9 + (Redacted) are the two breached principles; P7 (Attributable) is + breached by the `UPDATE` rewrite. +- `domains/compliance/evidence.md` — an audit log that can be + `DELETE`d is not admissible evidence; the append-only guarantee is + what makes it admissible. +- `domains/compliance/data-retention.md` — retention is a declared + policy with meta-audited deletions, not an ad-hoc `DELETE` job. +- `domains/security/secrets.md` — redaction at the logging boundary + is the audit-side complement of secret management. +- `domains/observability/logging.md` — audit logs are structured + logging with an append-only guarantee; the logging primitives + compose here. \ No newline at end of file diff --git a/examples/bad/i18n-string-concat.md b/examples/bad/i18n-string-concat.md new file mode 100644 index 0000000..1e497b2 --- /dev/null +++ b/examples/bad/i18n-string-concat.md @@ -0,0 +1,175 @@ +# Bad Example: i18n String Concatenation + +> A checkout component that violates Atelier's i18n principles. Each +> violation is cited, then fixed. + +## The Code + +```typescript +// Checkout.tsx — the cardinal i18n violation +function CartSummary({ itemCount, name, total, currency, date }) { + // P3 VIOLATION: inline string concatenation. The source-language + // word order ("Welcome, {name}! You have {n} items") is baked into + // code. Every locale with different word order is broken. + const welcome = "Welcome, " + name + "!"; + + // P4 VIOLATION: hand-rolled plural branching. `if (n === 1)` encodes + // exactly English's one/other rule. Arabic (six categories), Russian + // (three), Polish (three) are all wrong. + const items = + itemCount === 1 ? "1 item" : itemCount + " items"; + + // P5 VIOLATION: hand-rolled currency + date formatting. "$" + total + // is wrong for de-DE (symbol, grouping, placement). The date + // `getMonth() + 1 + "/" + getDay()` is US-only (mm/dd/yyyy). + const price = "$" + total.toFixed(2); + const d = new Date(date); + const dateStr = (d.getMonth() + 1) + "/" + d.getDate() + "/" + d.getFullYear(); + + return ( +
+

{welcome}

+

{items} · {price} · {dateStr}

+
+ ); +} +``` + +```typescript +// The "resource" file — a custom CSV the team hand-rolled. +// locale,en_us +// welcome_prefix,Welcome, +// item_singular,item +// item_plural,items +// +// This is a P3 violation on its own: a custom format no standard +// tool (xgettext, i18next, FormatJS) can extract from or compose with. +``` + +The team runs their first i18n test against real Arabic translations — +after the string freeze, after the translator was paid. The Arabic +build renders `"Welcome, محمد!"` with the name on the wrong side of +the comma, `"1 items"` for a single item (Arabic has six plural +categories, not two), and the price as `"$1,234.56"` (Arabic-Egypt +formats as `"١٬٢٣٤٫٥٦ ج.م.‏"`). Every screen is a rewrite, not a patch. + +## What Makes It Bad + +### Inline String Concatenation (i18n P3 Resources are External, Not Inline) +- `"Welcome, " + name + "!"` bakes English word order into code. In + Japanese the name comes first (`ようこそ、محمدさん！`); in Arabic the + structure differs again. The concatenation is invisible to the + extraction pipeline (`xgettext`, `i18next-parser`) — the translator + never sees it as a unit, and the string cannot be versioned or + rolled back as a whole. +- The custom `.csv` "resource" store is a second P3 violation: no + standard tool reads it, it carries no plural grammar, and it cannot + compose with the ICU formatting layer. +- **Fix:** strings live in a standard locale resource file, addressed + by key. Code calls `t("welcome", { name })`; the resource carries + the parameterized message. + + ```json + // en-US.json (ICU MessageFormat) + { + "checkout.welcome": "Welcome, {name}!", + "checkout.cart.summary": "{count, plural, one {# item} other {# items}} · {price} · {date}" + } + ``` + + ```json + // ar-EG.json — six plural categories per CLDR; the code is identical + { + "checkout.welcome": "أهلاً بك، {name}!", + "checkout.cart.summary": "{count, plural, zero {لا عناصر} one {عنصر واحد} two {عنصران} few {# عناصر} many {# عنصرًا} other {# عنصر}} · {price} · {date}" + } + ``` +- See `domains/i18n/locale-resources.md` (Resources are the Boundary) + and `domains/i18n/first-principles.md` P3. + +### Hand-Rolled Plural Branching (i18n P4 Plural and Gender are Parameterized) +- `itemCount === 1 ? "1 item" : itemCount + " items"` encodes + English's one/other rule and nothing else. Arabic has six + categories (zero, one, two, few, many, other); Russian has three + (one, few, many); Polish has three with different boundaries. A + two-branch `if` is a C1 (Correctness) violation masquerading as a + shortcut — it returns a wrong answer for every non-English locale. +- **Fix:** the count goes to ICU MessageFormat; the formatter + consults `Intl.PluralRules` for the active locale; the resource + carries the variant for that category. The code passes the count, + nothing more. + + ```typescript + // The code passes the count; the resource + formatter pick the form. + t("checkout.cart.summary", { count: itemCount, price, date }); + // Intl.PluralRules("ar-EG").select(1) === "one" -> "عنصر واحد" + // Intl.PluralRules("ar-EG").select(2) === "two" -> "عنصران" + // Intl.PluralRules("ar-EG").select(5) === "few" -> "٥ عناصر" + ``` +- See `domains/i18n/locale-resources.md` (Plural and Gender in + Resources) and `domains/i18n/formatting.md` (Plural Rules). + +### Hand-Rolled Currency and Date Formatting (i18n P5 Formatting is Locale-Aware) +- `"$" + total.toFixed(2)` hardcodes the US dollar symbol, US + grouping (`,`), and US placement (symbol before the number). In + `de-DE` the euro formats as `"1.234,56 €"` (symbol after, dot + grouping). In `ar-EG` the pound formats as `"١٬٢٣٤٫٥٦ ج.م.‏"` + (Arabic-Indic digits, different grouping). +- `(d.getMonth() + 1) + "/" + d.getDate() + "/" + d.getFullYear()` + produces `11/7/2024` — US `mm/dd/yyyy`. Most of the world reads + `dd/mm/yyyy`; ISO is `yyyy-mm-dd`. A hand-rolled date formatter + encodes one locale's convention and silently produces wrong output + for every other. +- **Fix:** `Intl.NumberFormat` and `Intl.DateTimeFormat` with a BCP + 47 locale tag. CLDR is the source of truth; `Intl` is the runtime. + + ```typescript + new Intl.NumberFormat("ar-EG", { style: "currency", currency: "EGP" }) + .format(1234.56); // "١٬٢٣٤٫٥٦ ج.م.‏" + new Intl.DateTimeFormat("ar-EG", { dateStyle: "medium" }) + .format(new Date(date)); // "٧ نوفمبر ٢٠٢٤" + ``` +- See `domains/i18n/formatting.md` (the Intl surface, dates, numbers, + currencies) and `domains/i18n/first-principles.md` P5. + +### Source Language Treated as the Default (i18n P1 Source Language is a Locale) +- The component has no resource layer at all for the source locale — + English is "just the strings in the code." When the first second + locale arrives, the fix is a rewrite (extract every string, + restructure every concatenation), not a patch. The source language + is `en-US`, a locale among many — it is not `null`. +- **Fix:** extract source strings into `en-US.json` from day one, + even before a second locale exists. The resource layer is the + boundary from the first commit. +- See `domains/i18n/first-principles.md` P1 and + `domains/uiux/copywriting.md`. + +## The Cascade + +The violations compound. Inline concatenation makes strings invisible +to the extraction pipeline, so the translator never receives them as +units — they reconstruct them by reading the code. Hand-rolled +plurals return wrong answers for every non-English locale, so the +Arabic build ships `"1 items"` for a single item. Hand-rolled +formatting produces US-shaped output everywhere, so the price and +date are wrong for `de-DE`, `ar-EG`, `zh-Hans-CN`, and every other +locale. And because the first i18n test ran against real translations +(a P8 violation — pseudo-locales should have surfaced all of this +while the fix was still cheap), the defects are found after the +string freeze, after the translator was paid, and after the release +date was promised. The fix is now a re-translation and a re-release, +not a commit. + +## Cross-Domain Links + +- `domains/i18n/locale-resources.md` — the resource layer this code + lacks; the standard formats (`.po`, JSON, Fluent, ICU Resource + Bundle) it should have used. +- `domains/i18n/formatting.md` — the `Intl`/ICU/CLDR formatting this + code should call instead of hand-rolling. +- `domains/i18n/first-principles.md` — P3, P4, P5, and P8 (pseudo- + locales test early). +- `domains/uiux/copywriting.md` — copy lives in resources, not in + code. +- `domains/api/error-responses.md` — the same parameterized-message + discipline applies to localized API errors. \ No newline at end of file diff --git a/examples/good/ai-ml-reproducibility.md b/examples/good/ai-ml-reproducibility.md new file mode 100644 index 0000000..fa08125 --- /dev/null +++ b/examples/good/ai-ml-reproducibility.md @@ -0,0 +1,226 @@ +# Good Example: AI/ML Reproducible Training Run + +> A training run that follows Atelier's AI/ML principles. Each aspect +> cites the principle it satisfies. Scope per D-023: this is +> engineering discipline (reproducibility, versioning, lineage, +> serving), **not** algorithm or model design — no architecture +> choice, hyperparameter tuning, or model-family comparison appears +> here. + +## The Run + +A training run `2026-08-05T09:12:00Z#run-42` produces model +`registry/payments-fraud@sha256:b5e1...aa0`. Every input that shaped +the model is pinned, named, and recoverable; the eval was declared +before training; the model is an addressed artifact in a registry; +the rollback path names the prior model and the prior dataset. + +### The Reproducibility Contract + +```yaml +# lineage/run-42.yaml — the lineage root, committed alongside the code +run_id: 2026-08-05T09:12:00Z#run-42 +dataset: s3://ml-data/train@sha256:7f3a...e21 +splits: dvc.yaml@commit a1b2c4d +code: git@a1b2c4d +config: configs/train.yaml@commit a1b2c4d +environment: ghcr.io/org/train-img@sha256:9c2d...f88 +eval_spec: configs/eval.yaml@commit a1b2c4d +model_digest: registry/payments-fraud@sha256:b5e1...aa0 +status: passed # eval gate passed -> eligible for promotion +``` + +- Lose any line and the run is anecdote, not evidence. The record is + the lineage root: a prediction cites the `model_digest`, which + cites this `run_id`, which cites everything above. + +### Data is Versioned (DVC, content-hashed) + +```ini +# dvc.yaml — the split config is versioned in git, the data in the +# content-addressed object store. Both are pinned by commit + hash. +stages: + prepare: + cmd: python src/prepare.py --input data/raw --out data/splits + deps: + - data/raw + - src/prepare.py + outs: + - data/splits/train.parquet + - data/splits/val.parquet + - data/splits/test.parquet + # The dataset hash (sha256:7f3a...e21) is recorded in the lineage + # contract above. "s3://ml-data/latest" would be a P2 violation. +``` + +```bash +# The dataset is pinned by content hash, not by a mutable path. +$ dvc get s3://ml-data/train --rev sha256:7f3a...e21 +# The split is a deterministic function of (dataset version, split +# config, random seed). Two runs on the same pinned inputs produce +# the same splits. +``` + +### Code and Config are Versioned (git) + +```yaml +# configs/train.yaml@commit a1b2c4d — versioned with the code +# (No algorithm/hyperparameter content is illustrated here — this is +# the engineering discipline of pinning the config, not the model +# design inside it. Per D-023, algorithm choice is out of scope.) +seed: 42 +splits: + train: data/splits/train.parquet + val: data/splits/val.parquet + test: data/splits/test.parquet # held out, never touched by training +``` + +### Environment is Pinned (container digest) + +```dockerfile +# The training environment is an image addressed by digest, not :latest. +# ghcr.io/org/train-img@sha256:9c2d...f88 +FROM python:3.11-slim +# dependencies pinned in requirements.txt with hashes +COPY requirements.txt . +RUN pip install --no-cache-dir -r requirements.txt +``` + +```text +# requirements.txt — pinned + hash-pinned (pip-compile / pip-audit) +dvc==3.50.2 \ + --hash=sha256:1c8a...e7 +mlflow==2.16.0 \ + --hash=sha256:9b2f...a1 +# No unpinned ranges. A rerun pulls the exact same wheels. +``` + +### Evaluation is Defined Before Training (P4) + +```yaml +# configs/eval.yaml@commit a1b2c4d — committed BEFORE training runs. +# The metrics, splits, and pass/fail thresholds are a-priori; they +# are the contract the model must satisfy to leave the experiment. +metrics: + - name: precision_at_threshold + threshold: ">= 0.92" + - name: recall_at_threshold + threshold: ">= 0.85" + - name: false_positive_rate + threshold: "<= 0.03" +split: data/splits/test.parquet # held out, never in training +gate: all_metrics_pass # AND of all thresholds; no cherry-pick +# The eval schema equals the serving input contract (serving.md P8): +# feature names, types, ranges match the production boundary exactly. +``` + +- Metrics chosen after seeing scores would be a P4 violation: the eval + would be rationalizing, not measuring. See + `domains/ai-ml/model-evaluation.md`. + +### The Model is a Versioned Artifact (MLflow registry) + +```bash +# After the eval gate passes, the model is registered as an immutable +# artifact addressed by digest, then promoted by stage. +$ mlflow models register \ + --name payments-fraud \ + --model-uri runs:/run-42/model \ + --description "run-42, dataset sha256:7f3a...e21, eval passed" +# registry/payments-fraud@sha256:b5e1...aa0 +# Stages: None -> Staging -> Production. Promotion is a registry +# operation, not a file copy. Never "latest". +``` + +### The Pipeline Composes (P9) + +```text +# The training flow is a pipeline with explicit stages and contracts, +# not a notebook. Each stage has named inputs and named outputs. +prepare(dataset@hash) -> split(dvc.yaml) -> train(config, env@digest) + -> eval(eval.yaml, test@hash) -> [gate: pass] -> register(model@digest) + | + +-> [gate: fail] -> abort, no promote +# A notebook in this path would be a P9 violation: implicit state, +# human-dependent order, unreproducible. +``` + +## What Makes It Good + +### Reproducibility is First Class (AI/ML P1, C1, C5) +- data + code + config + environment are all pinned. A second + engineer on a second laptop checks out commit `a1b2c4d`, pulls the + dataset by hash, pulls the image by digest, and reproduces the run + bit-for-bit. The run is reviewable because it is recreatable. +- See `domains/ai-ml/first-principles.md` P1 and + `domains/devops/first-principles.md` P1 Reproducibility. + +### Data is Versioned, Not Just Code (AI/ML P2, C5, C7) +- The dataset is `s3://ml-data/train@sha256:7f3a...e21`, not + `s3://ml-data/latest`. A model trained on "the data" is a model + trained on an unknown input — a C1 violation. DVC pins the data the + way git pins the code. +- See `domains/ai-ml/data-versioning.md` (dataset hashing, the DVC / + Delta Lake / LakeFS comparison) and `domains/data/migrations.md`. + +### Lineage is Traceable End-to-End (AI/ML P3, C7, C1) +- prediction → model → run-42 → dataset → source. Every edge is + named; no orphan model. A serving regression traces back to the + exact dataset and code that built the model, which is how drift is + diagnosed (data drift vs concept drift vs prediction drift). +- See `domains/ai-ml/data-versioning.md` (lineage record) and + `domains/observability/logging.md`. + +### Evaluation Defined Before Training (AI/ML P4, C1, C2) +- `eval.yaml` was committed before `train` ran. The gate is + `all_metrics_pass`; a failing metric aborts promotion. Cherry- + picking a metric post-hoc is a correctness violation — the eval + would no longer measure the model. +- See `domains/ai-ml/model-evaluation.md` (eval-as-a-gate) and + `domains/testing/first-principles.md` (tests as specification). + +### Models are Versioned Artifacts (AI/ML P5, C5, C6) +- The model is `registry/payments-fraud@sha256:b5e1...aa0`, promoted + Staging → Production. A serving endpoint that pulled `latest` would + be serving an unknown model with no rollback. The registry is to + models what a container registry is to images. +- See `domains/ai-ml/serving.md` (the model is an addressed artifact) + and `domains/devops/first-principles.md` P7 Immutability. + +### Rollback Includes the Model (AI/ML P10, C5) +- If production regresses, the rollback restores the prior model + digest `registry/payments-fraud@sha256:a1c4...f09` AND the prior + serving code. A rollback that redeploys old code but keeps the new + model has not rolled back — the model was the thing that regressed. +- See `domains/ai-ml/serving.md` (Rollback Includes the Model) and + `domains/devops/first-principles.md` P4 Rollback First. + +## What This Example Does NOT Do (And Why That's Good) + +- Does **not** reference the dataset by a mutable path — + `s3://ml-data/latest` would be a P2 violation. +- Does **not** choose metrics after seeing scores — that is a P4 + violation (rationalizing, not measuring). +- Does **not** pull `latest` from the model registry — that is a P5 + violation (unknown model, no rollback). +- Does **not** contain algorithm/architecture/hyperparameter content + — per D-023, those are research choices, not engineering + principles, and have no derivation in the core C-rules. +- Does **not** run from a notebook — a notebook in the pipeline path + is a P9 violation (implicit state, unreproducible). + +## Cross-Domain Links + +- `domains/ai-ml/data-versioning.md` — the DVC pinning, the lineage + record, the tool comparison (DVC / Delta Lake / LakeFS). +- `domains/ai-ml/serving.md` — the model is promoted as an addressed + artifact; the serving boundary validates inputs against the same + schema as the eval. +- `domains/ai-ml/model-evaluation.md` — the eval-as-a-gate that this + run must pass before promotion. +- `domains/devops/first-principles.md` P1 Reproducibility — the + non-negotiable this run inherits. +- `domains/data/migrations.md` — data versioning parallels schema + migration discipline. +- `domains/observability/logging.md` — the lineage record is a + structured, append-only log of provenance. \ No newline at end of file diff --git a/examples/good/gitops-pr.md b/examples/good/gitops-pr.md new file mode 100644 index 0000000..4fdffc8 --- /dev/null +++ b/examples/good/gitops-pr.md @@ -0,0 +1,197 @@ +# Good Example: GitOps Pull Request + +> A pull request that changes ArgoCD Application manifests following +> Atelier's GitOps + Operators principles. Each aspect cites the +> principle it satisfies. + +## The PR + +A PR titled `promote payments-api 1.2.3 to prod` opened against the +GitOps repo `platform/gitops`. It changes the `targetRevision` of the +payments-api Application from `1.2.2` to `1.2.3`, adds a sync-wave +annotation to a new migration Job, and tightens the AppProject's +destination allow-list. CI runs plan/diff; nothing pushes to the +cluster. + +### The Commit + +```yaml +# manifests/prod/payments-api.yaml — the only file changed +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: payments-api + namespace: argocd + finalizers: + - resources-finalizer.argocd.argoproj.io +spec: + source: + repoURL: https://git.example.com/platform/payments + targetRevision: 1.2.3 # was 1.2.2 — pinned, not latest + path: manifests/prod + destination: + server: https://kubernetes.default.svc + namespace: payments + syncPolicy: + automated: + prune: true + selfHeal: true + syncOptions: + - CreateNamespace=false + - PrunePropagationPolicy=foreground +``` + +```yaml +# manifests/prod/payments-db-migration.yaml — new file, wave-ordered +apiVersion: batch/v1 +kind: Job +metadata: + name: payments-db-migrate-1.2.3 + namespace: payments + annotations: + argocd.argoproj.io/sync-wave: "-1" # PreSync: run before the app +spec: + backoffLimit: 0 + ttlSecondsAfterFinished: 86400 + template: + spec: + restartPolicy: OnFailure + serviceAccountName: payments-migrator + containers: + - name: migrate + image: registry.example.com/payments/migrate:1.2.3 + args: ["up", "--to", "1.2.3"] +``` + +```yaml +# projects/payments.yaml — AppProject tightened in the same PR +apiVersion: argoproj.io/v1alpha1 +kind: AppProject +metadata: + name: payments + namespace: argocd +spec: + sourceRepos: + - https://git.example.com/platform/payments + destinations: + - server: https://kubernetes.default.svc + namespace: payments # prod-app can only deploy to payments ns + clusterResourceWhitelist: + - group: "" + kind: Namespace # allowed to create its own namespace + roles: + - name: payments-team + policies: + - p, proj:payments:payments-team, applications, sync, payments/*, allow +``` + +### The CI Pipeline (runs on the PR, before merge) + +```text +# .github/workflows/gitops-plan.yml (illustrative steps) +- name: validate manifests + run: argocd app manifests manifests/prod/ | kubeconform -strict + +- name: diff against live cluster (read-only, no apply) + run: argocd app diff payments-api --server $ARGOCD_SERVER --auth-token $READ_ONLY_TOKEN + # CI holds a READ-ONLY ArgoCD token. It never holds kubectl rights. + # A non-empty diff is the PR's proposed change, rendered for review. + +- name: opa gate (admission policy pre-check) + run: opa eval -i manifests/prod/ -d policies/ "data.k8s.admission.deny" + # Policy violations fail the PR before merge, not after deploy. +``` + +## What Makes It Good + +### Git is the Source of Truth (GitOps P1, C1 Correctness) +- The promotion is a commit. The cluster's desired state is a + derivative of this repo; the repo is the authority. If the change is + wrong, `git revert` is the rollback — the recovery path is the + history. +- See `domains/gitops-operators/first-principles.md` P1 and + `domains/gitops-operators/argocd.md` (Application CRD). + +### Pull, Don't Push (GitOps P3, C4 Locality) +- CI holds a **read-only** ArgoCD token for `app diff`. It holds no + `kubectl` rights against the production cluster. The cluster's + ArgoCD controller pulls the merged commit; nothing pushes to the + cluster. A compromised CI token can read, not deploy. +- See `domains/gitops-operators/argocd.md` (RBAC and SSO) and + `domains/gitops-operators/flux.md` for the same pull boundary from + the Flux side. + +### State is Immutable and Versioned (GitOps P5, C5 Reversibility) +- `targetRevision: 1.2.3` — the Application pins a specific chart + revision, not `latest`. The commit that changed it is a permanent + record; `git revert` restores `1.2.2` and ArgoCD's `selfHeal` + converges the cluster back. No force-push; history is the audit + trail. +- See `domains/gitops-operators/first-principles.md` P5 and + `domains/infrastructure-as-code/state.md` (State is Truth). + +### Sync Waves Order Correctness (GitOps P4, C1) +- The migration Job carries `argocd.argoproj.io/sync-wave: "-1"` so + it runs in `PreSync` before the payments-api Deployment that + depends on the new schema. Wave ordering is a correctness + mechanism, not performance — the app starting before its migration + is a correctness bug. +- See `domains/gitops-operators/argocd.md` (Sync Waves and Hooks). + +### Reconcile, Don't Mutate by Hand (GitOps P8) +- `selfHeal: true` + `prune: true` means a hand-edited drift on a + managed resource is overwritten on the next loop. The fix for drift + is a new commit, not `kubectl edit`. The PR author does not SSH into + the cluster to "fix" anything. +- See `domains/gitops-operators/argocd.md` (Diff and Drift) and + `domains/gitops-operators/first-principles.md` P8. + +### Least Privilege Reconciliation (GitOps P10, C8 Economy) +- The AppProject `payments` restricts the Application to the + `payments` namespace and the `payments` repo. The controller's + ServiceAccount (not shown) is bound to a namespace-scoped Role, not + `cluster-admin`. The PR *tightens* the allow-list — least privilege + is a direction, not a one-time setting. +- See `domains/gitops-operators/argocd.md` (RBAC and SSO) and + `domains/kubernetes/rbac.md`. + +### Policy is a Gate (Compliance P5, cross-link) +- The `opa eval` step runs the admission policy against the proposed + manifests before merge. A violation fails the PR; the non-compliant + state is never realized. Detection is not enforcement; this is + enforcement. +- See `domains/compliance/policy-as-code.md` and + `domains/devops/ci-cd.md`. + +### Failure is Observable (GitOps P9) +- A sync failure or health degradation on `payments-api` emits + ArgoCD status (`Degraded` / `OutOfSync`) and a notification. Silent + drift is the bug; this PR does not disable notifications. +- See `domains/gitops-operators/argocd.md` (Health and Status) and + `domains/observability/metrics.md`. + +## What This PR Does NOT Do (And Why That's Good) + +- Does **not** run `kubectl apply` from CI — that is the push pattern, + a P3 violation (see `examples/bad/` for the anti-pattern). +- Does **not** use `argocd app set` as the steady state — the change + is in git, not in an imperative command's history. +- Does **not** store raw Secrets in the GitOps repo — secrets arrive + via Sealed Secrets / SOPS / External Secrets, encrypted in git. +- Does **not** float `targetRevision: latest` — the Application pins + a version; "latest" is an unknown model of the system. + +## Cross-Domain Links + +- `domains/gitops-operators/argocd.md` — the Application CRD, sync + waves, RBAC/AppProjects, and the pull model. +- `domains/gitops-operators/flux.md` — the same PR pattern from the + Flux side (Kustomization CRD, per-cluster autonomy). +- `domains/kubernetes/workloads.md` — the Deployment/Job the + Application reconciles. +- `domains/kubernetes/rbac.md` — the ServiceAccount + Role the + controller and the migration Job run as. +- `domains/compliance/policy-as-code.md` — the OPA gate is a + compliance-as-a-gate enforcement point. +- `domains/devops/P4 Rollback First` — `git revert` is the rollback; + `selfHeal` is the convergence. \ No newline at end of file