Compare commits
9 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 61043dea1b | |||
| 44a7049860 | |||
| 7dfd3cdc6c | |||
| c026c8930b | |||
| 32edb19c96 | |||
| ab1289a9d9 | |||
| 47674969a1 | |||
| ce36db0579 | |||
| b7da50f56e |
@@ -1,15 +1,11 @@
|
||||
{
|
||||
"phase": 4,
|
||||
"phase": 3,
|
||||
"stage": "complete",
|
||||
"milestone": "v0.4",
|
||||
"milestone": "v0.3",
|
||||
"phase_role": "execution",
|
||||
"project": "atelier",
|
||||
"attempts": 0,
|
||||
"updated_at": "2026-08-05T07:00:00Z",
|
||||
"milestone_complete": false,
|
||||
"milestone_branch": "milestone/v0.4-edge-quantum-langs",
|
||||
"phase_branch": "phase/04-matrix-review-manifest",
|
||||
"tag_base": "v0.3",
|
||||
"phase_tag": "v0.3.4",
|
||||
"release_id": 486
|
||||
"updated_at": "2026-08-05T03:45:00Z",
|
||||
"phase_tag": "v0.2.3",
|
||||
"release_id": 471
|
||||
}
|
||||
@@ -16,18 +16,8 @@ atelier/
|
||||
│ ├── gitops-operators/ # v0.3: GitOps + Operators (argocd, flux, operators, progressive-delivery)
|
||||
│ ├── ai-ml/ # v0.3: ML engineering (data-versioning, model-evaluation, serving, monitoring-drift)
|
||||
│ ├── i18n/ # v0.3: internationalization (locale-resources, formatting, rtl-bidi, testing-i18n)
|
||||
│ ├── compliance/ # v0.3: compliance/audit (audit-logs, data-retention, policy-as-code, evidence)
|
||||
│ ├── edge/ # v0.4: edge computing (cdn, offline-first, iot, sync)
|
||||
│ └── messaging/ # v0.4: cross-process messaging (queues, pubsub, streams, delivery-semantics)
|
||||
│ └── compliance/ # v0.3: compliance/audit (audit-logs, data-retention, policy-as-code, evidence)
|
||||
├── languages/ # Language-specific application of domains
|
||||
│ ├── typescript.md # v0.1 first-principles stub
|
||||
│ ├── python.md # v0.1 first-principles stub
|
||||
│ ├── go.md # v0.1 first-principles stub
|
||||
│ ├── rust.md # v0.1 first-principles stub
|
||||
│ ├── ts-*.md # v0.4: 4 derived docs (ts-types, ts-tooling, ts-async, ts-testing)
|
||||
│ ├── py-*.md # v0.4: 4 derived docs (py-types, py-tooling, py-async, py-testing)
|
||||
│ ├── go-*.md # v0.4: 4 derived docs (go-types, go-tooling, go-concurrency, go-testing)
|
||||
│ └── rs-*.md # v0.4: 4 derived docs (rs-ownership, rs-tooling, rs-async, rs-testing)
|
||||
├── review/ # Evaluation checklists and anti-patterns
|
||||
├── matrix/ # Cross-reference: domain ↔ core
|
||||
└── examples/ # Worked examples (good + bad)
|
||||
@@ -116,28 +106,4 @@ From the v0.3 ideation stage (IDEATE-17..30), the following architectural refine
|
||||
- **Core Principle Coverage table (IDEATE-19 → ATELIER-81):** `matrix/domain-coverage.md` contains two tables — the per-domain row schema table (covered by v0.2 IDEATE-03) AND the "Core Principle Coverage" table mapping C1–C8 → domains. Both must be extended for the 4 new domains; the C-rule counts shift (e.g., C4 Locality adds i18n + gitops; C5 Reversibility adds ai-ml + compliance + gitops + i18n).
|
||||
- **Cross-link type unchanged:** v0.3 introduces no new cross-link type. All cross-links remain one-directional outward from new domains to existing (D-033). No back-link edits to v0.1/v0.2 content.
|
||||
|
||||
See `.ciagent/atelier/RESEARCH.md` "v0.3 Research" for the full prior-art survey and principle inventory rationale, and `.ciagent/atelier/PERSONAS.md` for the v0.3 persona roster (5 active: lead-developer, tech-writer, domain-expert + 2 phase-specific platform-engineer, ml-engineer; 3 default personas deactivated).
|
||||
|
||||
## v0.4 Domain Additions
|
||||
|
||||
Two new top-level domains and a language-derived extension expand the tree under the same hierarchy rules. All follow the v0.1/v0.2/v0.3 contract: docs-only markdown with illustrative code fences (no runtime/deployable artifacts per D-020). Total matrix grows from 170 → 190 domain principles across 17 → 19 domains.
|
||||
|
||||
- **`edge/`** — edge computing domain. Owns the proximity/location/constraint angle (D-061): compute near the user or data source, offline as a first-class state, constrained resources declared, sync conflicts bounded, partition-survivable telemetry. Boundary vs `performance/` (generic measurement/optimization), `observability/` (generic structured telemetry), and `concurrency/` (in-process primitives): edge owns only the proximity/location/disconnection concerns that arise at the network edge — performance's locality is algorithmic, edge's locality is geographic (C4). 10 P-rules, broad derivation (7 C-rules: C1, C3, C4, C5, C6, C7, C8). Depends on `core/`. Cross-links are one-directional outward (D-026 extended): `performance/frontend` (P6 geographic cache invalidation vs generic caching), `performance/P4` (P3 constrained-device vs generic resource bounds), `observability/metrics` + `observability/logging` (P10 local-first telemetry), `concurrency/patterns` (P5 offline write-queue analog), `security/secrets` (P9 device identity), `security/input-validation` (P6 cache poisoning), `data/migrations` (P4 schema migration under sync). Edge↔messaging intra-v0.4 cross-links resolve bidirectionally in P5 (ATELIER-114, IDEATE-40): `edge/iot.md` → `messaging/queues` (MQTT QoS parallels to delivery semantics). Derived docs: `cdn.md`, `offline-first.md`, `iot.md`, `sync.md`.
|
||||
|
||||
- **`messaging/`** — cross-process messaging domain. Owns the cross-process/network-failure-model angle (D-062): messages as contracts, explicit delivery semantics, idempotent consumers, bounded backpressure, observable lag. Boundary vs `concurrency/` (in-process/crash-failure-model): messaging's queue fails by network partition/broker restart/consumer crash-and-retry; concurrency's queue fails by OOM or thread crash. The discriminator is the failure model (D-062). 10 P-rules, broad derivation (7 C-rules: C1, C2, C4, C5, C6, C7, C8). Depends on `core/`. Cross-links are one-directional outward (D-026 extended): `concurrency/patterns` (P6 broker-backed bounded queue vs in-process Pattern 5; P3 cross-process idempotent retry vs in-process retry), `observability/metrics` + `observability/tracing` (P10 broker signals), `data/schema-design` (P1/P9 message schema evolution), `errors/patterns` (P5 errors as data for poison messages), `edge/iot` (P4 — bidirectional resolution of the MQTT QoS cross-link per IDEATE-40). Derived docs: `queues.md`, `pubsub.md`, `streams.md`, `delivery-semantics.md`.
|
||||
|
||||
### v0.4 Language-Derived Extension
|
||||
|
||||
The `languages/` tree extends from a first-principles-only stub (v0.1) to a full derived-doc tree (v0.4). Each of the 4 v0.1 stubs (`typescript.md`, `python.md`, `go.md`, `rust.md`) is extended with a pointer to its 4 derived docs (16 total). Language-derived docs **apply, not derive** domain principles (D-063): they introduce **no new P-rules** and **no new matrix rows** (D-066). Every section header names the traced core C-rule(s) AND the traced domain P-rule(s) inline (e.g., `## Nominal vs Structural Typing (C1 Correctness, Data P7 Type Fidelity, API P1 Contract Fidelity)`). The first-principles stubs retain their existing section structure — **no P1–P10 numbering** is added to languages (D-063); languages are an application lens, not a principle source, preserving the `core > domains > languages` ordering ("conflicts go UP, never sideways").
|
||||
|
||||
The `matrix/principles-matrix.md` gains **no language rows** — it stays at 190 P-rules / 19 domains (D-066). Language-derived-doc traceability is captured in a new `## Languages Coverage` sub-table in `matrix/domain-coverage.md` (exactly 16 rows, one per derived doc) — the verifiable aggregate; inline section-headers are the per-doc detail. Every derived doc traces to ≥1 domain P-rule (D-066, no orphans).
|
||||
|
||||
Derived docs by language:
|
||||
- **TypeScript** — `ts-types.md`, `ts-tooling.md`, `ts-async.md`, `ts-testing.md`
|
||||
- **Python** — `py-types.md`, `py-tooling.md`, `py-async.md`, `py-testing.md`
|
||||
- **Go** — `go-types.md`, `go-tooling.md`, `go-concurrency.md`, `go-testing.md`
|
||||
- **Rust** — `rs-ownership.md`, `rs-tooling.md`, `rs-async.md`, `rs-testing.md`
|
||||
|
||||
The manifest (`MANIFEST.md`) is extended with a new `## Languages` section (D-065) listing the 4 v0.1 stubs (closing pre-existing drift parallel to the v0.2 ESC-002 / ATELIER-91 precedent) and the 16 new v0.4 derived docs. The `review/` docs (agent-checklist, peer-review-checklist, anti-patterns) gain edge, messaging, and language-derived trigger sections. Examples (P5) are illustrative markdown with fenced code only — no standalone `.ts`/`.py`/`.go`/`.rs` runtime artifacts (D-020 docs-only contract, extended in v0.4 to language-derived-doc artifact types).
|
||||
|
||||
See `.ciagent/atelier/RESEARCH-v0.4.md` for the full prior-art survey (Akamai/Cloudflare, Kafka/RabbitMQ/NATS, Jepsen, Reactive Streams, CloudEvents, CRDTs, PouchDB, MQTT/CoAP) and `.ciagent/atelier/PERSONAS.md` for the v0.4 persona roster (5 active: lead-developer, tech-writer, domain-expert + 2 phase-specific edge-engineer, languages-engineer).
|
||||
See `.ciagent/atelier/RESEARCH.md` "v0.3 Research" for the full prior-art survey and principle inventory rationale, and `.ciagent/atelier/PERSONAS.md` for the v0.3 persona roster (5 active: lead-developer, tech-writer, domain-expert + 2 phase-specific platform-engineer, ml-engineer; 3 default personas deactivated).
|
||||
@@ -1,100 +0,0 @@
|
||||
# Atelier v0.4 — Clarify Stage
|
||||
|
||||
> Generated during v0.4 P0 CLARIFY stage. Autonomy level: `full` (defaults auto-accepted).
|
||||
> All decisions below are above the 0.60 confidence threshold → auto-decided, no HITL.
|
||||
> Decisions continue from D-060 (last v0.3 decision). Scope: NEW v0.4 work only (edge, messaging, language-derived docs). Prior milestone decisions (D-001..D-060) are not re-clarified.
|
||||
|
||||
## Ambiguities Resolved: 9
|
||||
|
||||
### 1. Edge domain P-rules — boundary vs performance/concurrency/observability (avoid duplication)
|
||||
- **Question:** The edge domain is latency-bound, resource-constrained, and geo-distributed — concerns that overlap with the existing performance domain (caching P5, lazy P6, async P7), concurrency domain (bounded queues P9, immutability P1), and observability domain (structured logs, correlation). How do edge's 10 P-rules distinguish themselves without duplicating existing domain P-rules?
|
||||
- **Default resolution:** Edge owns the **proximity/location angle**: compute placed near the user or data source, location-aware routing, resource constraints of edge nodes, and offline-capability. The boundary is: performance owns *generic measurement & optimization*; concurrency owns *in-process synchronization primitives*; observability owns *generic logging/metrics/tracing*; edge owns *the placement, constraint, and disconnection concerns that only arise at the network edge*. Edge P-rules trace heavily to C4 Locality (the defining edge trait — compute near data/user) and C8 Economy (constrained devices), plus C5 Reversibility (offline/sync reversibility), C7 Observability (geo-distributed visibility), C1 Correctness (eventual-consistency correctness under partition). Edge is a broad-derivation domain (6–8 C-rules, parallel to i18n/gitops). Cross-links to performance/concurrency/observability are one-directional outward (per D-026 extended); no back-link edits.
|
||||
- **Rationale:** Every prior domain owned a distinct "angle" on shared core principles (k8s P8 storage-explicit vs iac P3 state-is-truth — same C-rules, different scope per D-026/RESEARCH). Edge's unique contribution is *where compute runs*, which no existing domain addresses. The C4 Locality emphasis is the discriminator: performance's locality is algorithmic (data near compute); edge's locality is geographic (compute near user). Confirmed by RESEARCH.md pattern: "each domain owns its own principle angle."
|
||||
- **Confidence:** 0.88
|
||||
- **Alternatives rejected:** (a) Edge as a derived doc under performance — rejected: geo-distribution + offline + IoT constrained-device concerns warrant a full 10-P-rule set, not a single derived doc (parallel to D-020 k8s-vs-devops reasoning); (b) Edge reusing performance P5/P6/P7 verbatim — rejected: would orphan edge rules from their actual derivation (C4 Locality, C8 Economy) and violate the "no duplicate rows" matrix invariant.
|
||||
- **Decision:** D-061
|
||||
|
||||
### 2. Messaging domain boundary vs concurrency/patterns.md (in-process vs cross-system)
|
||||
- **Question:** `domains/concurrency/patterns.md` already covers "Message Passing" (Pattern 1), "Bounded Queue with Backpressure" (Pattern 5), and "Actor Model" (Pattern 3). The new messaging domain covers queues, pub/sub, streams, and delivery semantics. Where is the boundary — does messaging duplicate concurrency's queue/message-passing patterns?
|
||||
- **Default resolution:** Concurrency owns **in-process** synchronization primitives (threads, locks, channels, async within one program). Messaging owns **cross-process / cross-system** asynchronous communication (brokers: Kafka, RabbitMQ, SQS, NATS, Pulsar). The discriminator: messaging concerns arise *only when producer and consumer are separate systems* — delivery semantics (at-least-once/at-most-once/exactly-once), ordering across partitions, idempotency across retries, dead-letter queues, replay from offsets, consumer-group coordination. Concurrency's "bounded queue" is the in-process analog; messaging's "bounded queue" is the broker-backed analog with network-failure semantics. Messaging P-rules trace to C1 Correctness (delivery guarantees), C5 Reversibility (replay/rollback), C7 Observability (lag, DLQ visibility), C6 Composability (pub/sub decoupling). Cross-link concurrency/patterns.md ← messaging (one-directional); messaging references concurrency as the in-process foundation but owns the distributed-delivery layer above it.
|
||||
- **Rationale:** The v0.2 RESEARCH established the precedent: gitops-operators is "the deployment-automation layer above kubernetes/infrastructure-as-code — borrows their model, adds the git-as-source-of-truth layer." Messaging is the same: the async-delivery layer above concurrency — borrows the bounded-queue/backpressure model, adds cross-system delivery/ordering/idempotency. Concurrency/patterns.md Pattern 5 explicitly scopes itself to in-process ("producer is blocked or signaled"); messaging extends this to network-partition scenarios. No duplication because the *failure model* differs (crash vs network failure).
|
||||
- **Confidence:** 0.86
|
||||
- **Alternatives rejected:** (a) Messaging as a derived doc under concurrency — rejected: delivery semantics, stream processing, and broker comparison warrant a full 10-P-rule domain (parallel to D-021 gitops-unification reasoning: splitting would fragment P-rules); (b) Merge messaging P-rules into concurrency — rejected: would bloat concurrency to 20 rules and conflate in-process vs distributed concerns, breaking the "10 P-rules per domain" invariant (D-018/D-030); (c) No cross-link to concurrency — rejected: the bounded-queue/backpressure lineage should be explicit (one outbound cross-link per derived doc per ATELIER-114).
|
||||
- **Decision:** D-062
|
||||
|
||||
### 3. Language-derived docs structure — no new P-rules, section-header trace pattern (confirm D-059)
|
||||
- **Question:** PROJECT.md states language-derived docs "trace to existing domain P-rules, not new matrix rows" (the v0.4 Cross-Domain Integration bullet). But how exactly is a language-derived doc structured? The existing `languages/*.md` stubs use section headers like `## Type System (C1 Correctness, Data P7 Type Fidelity)` — they have NO numbered P1–P10 rules. Do the 16 new derived docs (ts-types, ts-tooling, etc.) introduce their own P-rules, or follow the same section-header-trace pattern with no new P-rule numbering?
|
||||
- **Default resolution:** Language-derived docs introduce **NO new P-rules**. Each derived doc follows the existing stub structure: section headers that name the traced core C-rule(s) and domain P-rule(s) inline (e.g., `## Strict Null Checks (C1 Correctness, Errors P1 Errors are Data)`), with bullet-point guidance and fenced code. The 4 derived docs per language each focus on one facet (types, tooling, async/concurrency, testing) and trace to the relevant existing domain P-rules. The first-principles stub for each language (typescript.md, etc.) is *extended* with a pointer to its derived docs but retains its existing section structure — it does NOT gain P1–P10 numbering. Matrix row count stays at 190 (170 + 20 for edge+messaging only).
|
||||
- **Rationale:** ARCHITECTURE.md dependency direction states `languages/` depend on `domains/` ("language rules apply domain rules") — languages are an *application lens*, not a principle source. The existing stubs confirm this: they trace to `Data P7`, `Errors P1`, `Concurrency P7`, `Observability P1`, `DevOps P2` — never inventing new P-rules. D-059 (referenced in PROJECT.md v0.4 section) already fixed the matrix treatment; this decision confirms the *document structure* that implements it. Adding P1–P10 to languages would break the hierarchy (languages would become peer domains, violating "conflicts go UP, never sideways" and the core>domains>languages ordering).
|
||||
- **Confidence:** 0.92
|
||||
- **Alternatives rejected:** (a) Language-derived docs get their own P1–P10 per language (40 new P-rules) — rejected: violates ARCHITECTURE.md hierarchy (languages apply, not derive) and D-059's matrix ruling; would also push matrix to 230 rows, breaking the stated 190 total; (b) Hybrid: first-principles stub gains P-rules, derived docs don't — rejected: inconsistent within a language and still introduces new matrix rows contrary to PROJECT.md; (c) Derived docs are pure code examples with no principle traces — rejected: would orphan them from the matrix/traceability contract and lose the "why over what" tech-writer constraint.
|
||||
- **Decision:** D-063
|
||||
|
||||
### 4. Language-derived doc file naming — ts-/py-/go-/rs- prefixes confirmed
|
||||
- **Question:** PROJECT.md v0.4 uses short prefixes for derived doc filenames: `ts-types.md`, `py-types.md`, `go-types.md`, `rs-ownership.md` — but the parent first-principles files use full names (`typescript.md`, `python.md`, `go.md`, `rust.md`). The prefix for Rust is `rs-` (not `rust-`), and Go uses `go-` (matching its parent). Is the ts-/py-/go-/rs- prefix scheme correct, or should it be revisited for consistency (e.g., typescript-types, or rust-ownership)?
|
||||
- **Default resolution:** Confirm the **ts-/py-/go-/rs-** prefix scheme as specified in PROJECT.md. `ts-` (TypeScript), `py-` (Python), `go-` (Go), `rs-` (Rust). These match the dominant ecosystem conventions: `ts` is the universal TypeScript abbreviation (tsconfig, ts-node, .ts extension); `py` is the universal Python abbreviation (.py, pyproject, py.test); `go` matches the language's own binary/module naming; `rs-` is the established Rust ecosystem prefix (crate names, file conventions). The full parent filename + short-prefix-derived-doc pattern mirrors how domains work (`kubernetes/` parent + `workloads.md` derived — the derived doc doesn't repeat the full domain name).
|
||||
- **Rationale:** Consistency with domain-derived-doc naming: derived docs use short, distinct slugs (kubernetes/workloads.md, not kubernetes-kubernetes-workloads.md). The short prefix disambiguates the language while keeping filenames compact. `rs-` over `rust-` follows the broader Rust ecosystem (the `rs-` prefix is more common in registry/package naming than `rust-`, which can collide with the `rust` toolchain). `go-` is correct because Go's ecosystem uses `go` universally and the parent file is already `go.md`.
|
||||
- **Confidence:** 0.84
|
||||
- **Alternatives rejected:** (a) Full-name prefixes (typescript-types.md, python-types.md, rust-ownership.md) — rejected: verbose, inconsistent with domain-derived-doc compactness, and `rust-` collides with toolchain naming conventions; (b) Single-letter or extension-based (ts/types.md subdirectory per language) — rejected: changes the flat `languages/` structure established in v0.1 (MANIFEST lists `languages/typescript.md`, not `languages/typescript/first-principles.md`); introduces directory nesting the v0.1 manifest schema doesn't support; (c) Normalize Rust to `rust-` for readability — rejected: breaks ecosystem convention alignment; `rs-` is the recognized Rust short prefix.
|
||||
- **Decision:** D-064
|
||||
|
||||
### 5. Language-derived docs in MANIFEST — new languages/ section (closes pre-existing drift)
|
||||
- **Question:** The current `MANIFEST.md` (post-v0.3) has sections for Core, Domains, Examples, and Cross-Cutting — but **no `languages/` section at all**, despite `languages/{typescript,python,go,rust}.md` existing since v0.1 (ATELIER-31, P6). This is pre-existing manifest drift (parallel to the v0.2 ESC-002 note where `examples/` was unlisted). Where do the 16 new language-derived docs (and the 4 existing stubs) go in the manifest?
|
||||
- **Default resolution:** Add a **new top-level `## Languages` section** to `MANIFEST.md` in P4, parallel to the `## Domains` and `## Examples` sections. The section lists each language (TypeScript, Python, Go, Rust) with its first-principles stub (✓) and its 4 derived docs. This *both* lists the 16 new v0.4 derived docs AND closes the pre-existing drift by listing the 4 v0.1 stubs that were never added. The section appears after `## Domains` and before `## Examples` (matching the reading order: core → domains → languages). The manifest's Cross-Cutting summary line updates: "19 domains, 190 P-rules post-v0.4" and notes that languages apply (not derive) domain rules.
|
||||
- **Rationale:** The manifest is authoritative — "unlisted = not part of the framework" (PROJECT.md constraint, MANIFEST.md header). The v0.2 audit escalation ESC-002 established the precedent: when a directory exists but is unlisted, the next milestone that touches it adds the listing (IDEATE-17 → ATELIER-91 closed examples/ drift in v0.3). v0.4 touches `languages/` for the first time since v0.1, so it closes the languages/ drift on the same principle. Placing the section between Domains and Examples matches `core/reading-order.md` (languages are read after domains, before examples) and the ARCHITECTURE.md dependency direction (languages depend on domains). This is a single P4 task, not split across phases.
|
||||
- **Confidence:** 0.90
|
||||
- **Alternatives rejected:** (a) List language-derived docs under the existing `## Domains` section — rejected: languages are not domains (ARCHITECTURE.md distinguishes the two levels; mixing them breaks the hierarchy and the matrix row-count invariant); (b) Defer the 4 v0.1 stubs to a future milestone, list only the 16 new docs — rejected: would leave the pre-existing drift half-closed and is inconsistent with the ESC-002 precedent (close the whole directory when you touch it); (c) Put languages/ at the end after Cross-Cutting — rejected: violates reading order (languages are consumed before examples/matrix reference, per core/reading-order.md).
|
||||
- **Decision:** D-065
|
||||
|
||||
### 6. Matrix treatment of language-derived docs — NO new matrix rows; traceability via domain-coverage sub-table + inline traces
|
||||
- **Question:** D-059 (PROJECT.md) says language-derived docs "trace to existing domain P-rules, not new matrix rows." But traceability must be *captured somewhere* — where? Does `matrix/principles-matrix.md` gain a languages section (with non-numbered trace rows)? Does `matrix/domain-coverage.md` gain a languages sub-table? How is "this language-derived doc traces to these domain P-rules" recorded verifiably?
|
||||
- **Default resolution:** `matrix/principles-matrix.md` gains **NO language rows** — it stays at 190 P-rules (170 + 20 for edge+messaging), 19 domains. `matrix/domain-coverage.md` gains a **new `## Languages Coverage` sub-table** (per ATELIER-107) with schema: `Language | Derived Doc | Traced Domain P-rules | Traced Core C-rules`. Each of the 16 derived docs gets one row listing the domain P-rules its section-headers trace to (e.g., `TypeScript | ts-types.md | Data P7 Type Fidelity, Errors P1, C1 | C1, C6`). Traceability is ALSO captured inline in each derived doc's section headers (the existing stub pattern: `## Section (C-rule, Domain P-rule)`). The domain-coverage sub-table is the verifiable aggregate; the inline headers are the per-doc detail. The agent-checklist (ATELIER-108) gains a language-derived-docs check: "every derived-doc section header names ≥1 traced domain P-rule."
|
||||
- **Rationale:** The principles-matrix is the domain→core join table; languages are not domains and have no P-rules to map (D-063), so adding rows would either be empty or fake. The domain-coverage sub-table is the right home: it already holds the per-domain coverage schema (IDEATE-03) and the C-rule→domains table (IDEATE-19) — a languages sub-table is the natural extension, parallel in shape. This gives reviewers a single place to verify "no language-derived doc is orphaned from its traced domain P-rules" without scanning all 16 files. Mirrors how v0.3's C-rule coverage table (IDEATE-19) was the complement to the per-domain row schema.
|
||||
- **Confidence:** 0.87
|
||||
- **Alternatives rejected:** (a) Add a `## Languages` section to principles-matrix.md with one row per derived doc → C-rule — rejected: the matrix maps P-rules, and languages have no P-rules (D-063); this would conflate the join table's semantics and inflate the "190 P-rules" count that PROJECT.md/ATELIER-106 explicitly state; (b) Traceability only inline in section headers, no aggregate table — rejected: not verifiable in one place; reviewers would have to open 16 files to confirm no orphans, violating the domain-expert persona's "no orphans in matrix" constraint; (c) Add languages as a 20th "domain" row in the per-domain coverage table — rejected: languages are not domains (ARCHITECTURE.md hierarchy); would corrupt the domain count invariant.
|
||||
- **Decision:** D-066
|
||||
|
||||
### 7. Phase-specific personas — edge-engineer (P1) + languages-engineer (P3); messaging (P2) by tech-writer + domain-expert with edge-engineer consultation
|
||||
- **Question:** ROADMAP v0.4 names "edge-engineer + languages-engineer phase-specific personas." That is only TWO named personas for three execution phases (P1 edge, P2 messaging, P3 languages). Does the messaging domain (P2) get its own phase-specific persona (e.g., messaging-engineer), or is it covered by existing/adjacent personas? If the latter, who owns messaging P2 authorship and with what constraints?
|
||||
- **Default resolution:** Add **two** phase-specific personas: `edge-engineer` (active P1, removed after milestone) and `languages-engineer` (active P3, removed after milestone). The **messaging domain (P2) is authored by tech-writer + domain-expert** (the permanent roster), with edge-engineer consulted for the distributed-systems/async overlap (D-062 boundary). edge-engineer constraints: ["latency is a correctness constraint", "compute placement is a design decision", "offline is a first-class state", "trace to core", "10 P-rules per domain", "no runtime code", "resource constraints are explicit", "sync conflict is a correctness failure"]. languages-engineer constraints: ["apply, don't derive (D-063)", "trace every section to an existing domain P-rule", "language idioms serve principles, not the reverse", "no new P-rules", "no new matrix rows (D-066)", "fenced code is illustrative only"]. Messaging P2 gets a task must-have baked into the plan: "P1/P2 demonstrated edge + messaging boundary (D-062); delivery-semantics comparison table required (D-069)" so the edge-engineer's distributed-systems lens survives the P1→P2 persona transition (parallel to D-051's ml-engineer-constraints-baked-into-task pattern from v0.3).
|
||||
- **Rationale:** The ROADMAP explicitly names only 2 personas — this is the authoritative signal (the ROADMAP is finalized in PLAN and pre-named personas per PROJECT.md v0.4 section + ROADMAP P0 row). v0.3 set the precedent (D-022): smaller-surface or within-competence domains don't get new personas (i18n + compliance in v0.3 P3 used tech-writer + domain-expert, no new personas). Messaging, while specialized, is within domain-expert's cross-cutting competence (delivery semantics, idempotency, ordering are correctness/observability derivations the domain-expert already validates). Limiting to 2 phase-specific personas holds the active roster to 5 (3 permanent + 2 phase-specific), matching v0.3's peak and avoiding the "persona explosion" risk flagged in v0.3 RESEARCH. The edge-engineer consultation for P2 ensures the D-062 in-process-vs-cross-system boundary is respected without a 3rd persona.
|
||||
- **Confidence:** 0.80
|
||||
- **Alternatives rejected:** (a) Three phase-specific personas (edge-engineer P1, messaging-engineer P2, languages-engineer P3) — rejected: ROADMAP names only 2; would push roster to 6 active (3+3), exceeding v0.3's 5-persona peak and raising the explosion risk; messaging's specialty, while real, is less novel than ai-ml's (which warranted ml-engineer per D-019/D-020); (b) One persona (edge-engineer) covers P1+P2, languages-engineer P3 — rejected: edge (latency/geo/resource) and messaging (delivery/ordering/idempotency) are distinct specialties; forcing one persona blurs the D-062 boundary and dilutes edge-specific constraints; (c) No new personas, all three phases by tech-writer + domain-expert — rejected: edge's latency-as-correctness and compute-placement concerns, and languages' apply-don't-derive discipline, benefit from specialist constraints the generalist personas lack (v0.2 D-014 / v0.3 D-019 precedent).
|
||||
- **Decision:** D-067
|
||||
|
||||
### 8. Examples — confirm 2 good + 2 bad pre-named set; bad examples cite breached P-rules
|
||||
- **Question:** PROJECT.md / ATELIER-112/113 pre-name the 4 v0.4 examples: `good/edge-offline-sync.md`, `good/messaging-idempotent-consumer.md`, `bad/edge-sync-loop.md`, `bad/messaging-shared-subscription.md`. Are these the right picks (vs alternatives like good/cdn-cache-invalidation or bad/messaging-blocking-consumer), and what breach semantics must the bad examples follow?
|
||||
- **Default resolution:** Confirm the pre-named set as specified. **Good:** `edge-offline-sync.md` (demonstrates offline-first + sync conflict resolution via CRDT or LWW — the highest-illustration-value edge case, exercising P1 offline + the sync.md decision matrix), `messaging-idempotent-consumer.md` (demonstrates exactly-once-via-idempotency-keys + dedup — the highest-illustration-value messaging case, exercising delivery-semantics + idempotency). **Bad:** `edge-sync-loop.md` (a sync conflict that oscillates/loops forever — the chaos anti-pattern pre-specified in ATELIER-110), `messaging-shared-subscription.md` (a shared subscription causing duplicate processing / fan-out duplication — the chaos anti-pattern pre-specified in ATELIER-110). Each bad example must cite the **specific P-rule breached** (per D-025 / IDEATE-07 named-bad-example convention carried forward v0.1→v0.2→v0.3). `edge-sync-loop.md` breaches the edge sync P-rule (exact P# TBD in P1, but it is the "sync conflicts are bounded, not infinite" rule); `messaging-shared-subscription.md` breaches the messaging subscription P-rule (exact P# TBD in P2, but it is the "consumer identity is per-consumer, not shared" rule). Both bad examples are single-breach (unlike v0.3's two-breach compliance example D-044 — edge/messaging breaches are cleaner as single-rule violations).
|
||||
- **Rationale:** The pre-named set covers one good + one bad per new domain — balanced, parallel to v0.2 (terraform + k8s) and v0.3 (gitops + ai-ml good; i18n + compliance bad). D-025 (v0.2) established 2-good + 2-bad for 2-domain milestones; v0.4 follows the same ratio. The chaos anti-patterns (sync loop, shared subscription) are pre-specified in ATELIER-110 because they are the highest-illustration-value *fundamental* violations — a sync loop is to edge what push-deploy was to GitOps (IDEATE-24); a shared subscription is to messaging what an orphan model was to ai-ml (IDEATE-27). Alternatives (cdn-cache-invalidation good, blocking-consumer bad) are lower-value: cache invalidation is a performance-domain concern (cross-link only), and blocking-consumer overlaps concurrency/patterns.md (D-062 boundary). Single-breach is correct because edge/messaging P-rules are individually clear; the two-breach pattern (D-044) was specific to compliance where two breaches (append-only + redaction) compound the highest-stakes domain.
|
||||
- **Confidence:** 0.85
|
||||
- **Alternatives rejected:** (a) `good/cdn-cache-invalidation.md` instead of edge-offline-sync — rejected: cache invalidation is primarily a performance/P5 concern; offline-sync exercises more edge-unique P-rules (offline state, sync, conflict); (b) `bad/messaging-blocking-consumer.md` instead of shared-subscription — rejected: blocking-consumer is a concurrency/patterns.md concern (D-062 in-process boundary); shared-subscription is a cross-system messaging-unique failure; (c) Two-breach bad examples (per D-044 compliance pattern) — rejected: D-044 was domain-specific to compliance's compounded stakes; edge/messaging single-rule breaches are clearer and match v0.1/v0.2 bad-example convention; (d) 4 good + 4 bad (one per derived doc) — rejected: would unbalance P5 (D-025 reasoning: 2+2 covers highest-illustration-value; remaining coverage via cross-links + anti-patterns).
|
||||
- **Decision:** D-068
|
||||
|
||||
### 9. Derived-doc decision/comparison tables — confirm queues/sync matrices + add streams platform comparison
|
||||
- **Question:** v0.2 established the decision-matrix pattern (Helm vs Kustomize in ATELIER-46/47), v0.3 transferred it to 3 derived docs (ArgoCD vs Flux, DVC vs Delta Lake vs LakeFS, OPA vs Cedar vs Kyverno vs Sentinel). v0.4 REQUIREMENTS pre-specify two: `queues.md` needs an at-least-once/at-most-once/exactly-once comparison (ATELIER-98), and `sync.md` needs a CRDT/last-write-win decision matrix (ATELIER-96). But `streams.md` (ATELIER-100) and `delivery-semantics.md` (ATELIER-101) have no comparison table specified. Should they also get one, following the established pattern?
|
||||
- **Default resolution:** Confirm the two pre-specified matrices (queues delivery-semantics comparison in queues.md; CRDT-vs-LWW decision matrix in sync.md). Add **two more** to maintain pattern parity (every v0.4 derived doc that compares distinct tools/models gets a decision/comparison table, per the v0.2→v0.3 transfer convention): `streams.md` gets a **stream-platform comparison table** (Kafka vs Kinesis vs Pulsar/NATS JetStream) covering ordering guarantees, partitioning model, replay/retention, and use-case fit; `delivery-semantics.md` gets a **dead-letter strategy comparison table** (retry-count-limit vs TTL-with-backoff vs poison-queue vs DLQ+alert) covering when each applies, failure visibility, and operational cost. Additionally, `cdn.md` (ATELIER-93) already has a pre-specified "edge-cache vs origin decision matrix" — that is confirmed (the cache-hit/miss/invalidation tradeoff). Total: 4 decision/comparison tables across the 9 new derived docs (cdn, queues, sync, streams, delivery-semantics = 5 candidate docs; 4 get tables — delivery-semantics gets a strategy comparison rather than a tool-vs-tool matrix since it's a pattern comparison, not a product comparison).
|
||||
- **Rationale:** The v0.2→v0.3 transfer pattern (REQUIREMENTS v0.3 "Within-Project Pattern Transfer" table, IDEATE-21/22/23) established that derived docs comparing distinct tools/models MUST include a decision/comparison table — this is now a project convention, not a one-off. streams.md compares Kafka/Kinesis/Pulsar (3+ distinct platforms with different ordering/replay models) — directly parallel to v0.3's data-versioning tool comparison (DVC/Delta Lake/LakeFS, IDEATE-22). delivery-semantics.md compares dead-letter strategies (distinct failure-handling models) — parallel to v0.2's Helm-vs-Kustomize "when X fits" guidance. Omitting tables from streams/delivery-semantics would be an inconsistency reviewers should flag (the v0.3 transfer-table verification was a P4 review check). cdn.md's edge-cache-vs-origin matrix is already required (ATELIER-93) and is the cache-placement analog of Helm-vs-Kustomize.
|
||||
- **Confidence:** 0.82
|
||||
- **Alternatives rejected:** (a) Only the 2 pre-specified tables (queues, sync), none for streams/delivery-semantics — rejected: breaks the v0.2→v0.3 transfer convention; streams.md comparing 3 platforms without a table would be the first multi-tool derived doc since v0.2 to omit one; (b) Tables in all 5 candidate docs including a tool-vs-tool table in delivery-semantics — rejected: delivery-semantics compares *strategies/patterns*, not products; forcing a product table (e.g., "Kafka DLQ vs RabbitMQ DLQ") would be misleading since DLQ semantics are cross-platform; a strategy comparison is the correct shape; (c) Defer streams/delivery-semantics tables to v0.5 — rejected: pattern-parity should land in the milestone that introduces the docs; deferral creates immediate drift (the v0.3 transfer table explicitly verifies within-milestone parity).
|
||||
- **Decision:** D-069
|
||||
|
||||
## Decisions Log
|
||||
|
||||
| ID | Decision | Confidence |
|
||||
|----|----------|------------|
|
||||
| D-061 | Edge domain owns the proximity/location angle (C4 Locality, C8 Economy heavy); performance/concurrency/observability own generic primitives; cross-links one-directional; edge is a broad-derivation domain (6–8 C-rules) | 0.88 |
|
||||
| D-062 | Messaging owns cross-process/cross-system async (brokers, delivery semantics); concurrency owns in-process primitives; boundary is the failure model (network vs crash); cross-link concurrency ← messaging | 0.86 |
|
||||
| D-063 | Language-derived docs introduce NO new P-rules; follow existing section-header-trace pattern (C-rule + Domain P-rule inline); first-principles stubs extended but retain structure, no P1–P10 numbering | 0.92 |
|
||||
| D-064 | Confirm ts-/py-/go-/rs- prefix scheme for language-derived doc filenames (ecosystem convention alignment; mirrors domain-derived-doc compactness) | 0.84 |
|
||||
| D-065 | Add new top-level `## Languages` section to MANIFEST.md in P4 (between Domains and Examples); lists 4 stubs + 16 derived docs; closes pre-existing languages/ drift parallel to ESC-002 | 0.90 |
|
||||
| D-066 | principles-matrix.md gains NO language rows (stays 190/19 domains); domain-coverage.md gains a `## Languages Coverage` sub-table (Language → Derived Doc → Traced Domain P-rules → Traced C-rules); inline section-headers are per-doc detail | 0.87 |
|
||||
| D-067 | Two phase-specific personas: edge-engineer (P1) + languages-engineer (P3); messaging (P2) by tech-writer + domain-expert with edge-engineer consultation; constraints baked into P2 task must-have (D-051 pattern) | 0.80 |
|
||||
| D-068 | Confirm pre-named examples (good: edge-offline-sync, messaging-idempotent-consumer; bad: edge-sync-loop, messaging-shared-subscription); bad examples cite specific breached P-rule; single-breach (not D-044 two-breach) | 0.85 |
|
||||
| D-069 | 4 decision/comparison tables across v0.4 derived docs: cdn (edge-cache vs origin), queues (delivery semantics), sync (CRDT vs LWW), streams (Kafka vs Kinesis vs Pulsar), delivery-semantics (dead-letter strategy comparison) — pattern parity with v0.2/v0.3 | 0.82 |
|
||||
|
||||
## Summary
|
||||
- 9 ambiguities identified and auto-resolved at full autonomy
|
||||
- Average confidence: 0.86
|
||||
- All resolutions grounded in existing project conventions (D-001..D-060, prior domain structure, core C-rules, ARCHITECTURE.md hierarchy, RESEARCH.md prior-art pattern, ESC-002/IDEATE-17 manifest-drift precedent, IDEATE-07/D-025 named-bad-example convention, v0.2→v0.3 decision-matrix transfer pattern, D-051 constraints-baked-into-task persona pattern)
|
||||
- No decision fell below the 0.60 autonomy threshold → no HITL escalation triggered
|
||||
- Lowest-confidence decision (D-067, 0.80): messaging P2 persona choice — resolved by deferring to the ROADMAP's explicit 2-persona naming and the D-022 within-competence precedent, with edge-engineer consultation to preserve the D-062 boundary
|
||||
@@ -1,193 +0,0 @@
|
||||
# Atelier — Grill (Adversarial Red-Team Review)
|
||||
|
||||
> Pre-execution gate for milestone v0.4 (Edge + Messaging + Language-Derived Docs).
|
||||
> Default assumption: the project is unfeasible, over-scoped, and too costly. Not convinced until evidence forces it.
|
||||
> Mode: full autonomy. Auto-resolve at confidence ≥ 0.60; escalate only < 0.60 that cannot be auto-resolved.
|
||||
|
||||
---
|
||||
|
||||
## v0.4 Grill — 2026-08-05
|
||||
|
||||
**Milestone:** v0.4 — Edge + Messaging + Language-Derived Docs
|
||||
**Phase:** 0 (Pre-Execution, GRILL stage)
|
||||
**Grill scope:** all 9 axes + meta
|
||||
**Prior grill runs:** v0.3 (GRILL.md — PROCEED @ 0.80). v0.1/v0.2 P0 stages did not include a GRILL stage.
|
||||
**Autonomy:** full (binding verdicts; no HITL questions)
|
||||
**Axes:** 9 review axes (Feasibility, Scope, Cost/Risk, Vertical-slice integrity, Traceability, Decision quality, Persona coverage, Test/Verify, Cross-link/matrix integrity)
|
||||
|
||||
### Verdict: **CONDITIONAL PROCEED** (overall confidence 0.83)
|
||||
|
||||
The plan is the 4th consecutive NFR milestone using the same proven docs-only contract (v0.1 35 reqs → v0.2 24 reqs → v0.3 32 reqs → v0.4 26 reqs). All three prior milestones shipped clean with identical structure. The 26-req scope is *smaller* than v0.3's 32 reqs, and per-phase doc counts (P1:5, P2:5, P3:20, P4:7, P5:5, P6:2) are within demonstrated throughput. The two new domains (edge, messaging) have well-reasoned boundaries against existing domains (D-061 proximity angle; D-062 cross-process failure model) and the 16-doc language extension is the largest single phase (P3) but introduces **zero** matrix rows (D-063/D-066), so it does not stress the traceability invariant.
|
||||
|
||||
The verdict is **CONDITIONAL** rather than clean PROCEED because the red-team verification surfaced **one false premise** baked into the plan and the audit: IDEATE-34/ATELIER-106 refinement asserts a "pre-existing matrix discrepancy (171 vs stated 170)" that must be reconciled before appending the 20 v0.4 rows. Direct verification of `matrix/principles-matrix.md` shows the matrix contains **exactly 170 P-rows** (17 domains × 10 rows each), matching the invariant — the discrepancy does not exist. The reconciliation task (04-01-01) and audit test (06-01-02) are built on a wrong premise. This is non-blocking (the verification step is harmless — it will simply confirm 170) but the task wording is misleading and could cause an executor to "fix" a non-existent problem (e.g., deleting a valid row to hit a phantom 171→170). Two binding conditions below fix this before ship.
|
||||
|
||||
---
|
||||
|
||||
### Axis 1 — Feasibility
|
||||
|
||||
| # | Forcing Question | Evidence | Answer | Confidence |
|
||||
|---|-----------------|----------|--------|------------|
|
||||
| 1.1 | Can P1–P6 actually be delivered with the proposed resources/personas? | PLAN-v0.4.md: 46 tasks across 6 phases. PERSONAS.md: 3 permanent + 2 phase-specific personas, peak 5 active (matches v0.3). Per-phase doc counts: P1=5, P2=5, P3=20, P4=7, P5=5, P6=2. v0.1 P3 shipped 27 derived docs in one phase (RESEARCH-v0.4 §Language-Derived). | Feasible. v0.1 P3 demonstrated 27 derived docs in a single phase; v0.4 P3's 20 language files (4 stubs + 16 derived) is below that watermark, and language-derived docs have a *simpler* structure (section-header-trace, no P1–P10 derivation) than domain derived docs. The 5-persona peak matches v0.3 which shipped clean. | 0.85 |
|
||||
| 1.2 | Are edge/messaging P-rules novel enough to not duplicate existing domains (per D-061/D-062)? | RESEARCH-v0.4 §Edge boundary table: edge P6 (geographic/partition cache invalidation) vs `performance/P5 Caching with Intent` (generic C3,C6) — distinct angle. §Messaging boundary table: messaging P6 (broker-backed backpressure, network failure) vs `concurrency/P9 Bounded Queues` (in-process, OOM failure) — distinct failure model. `concurrency/patterns.md` Pattern 5 explicitly scopes to "producer is blocked or signaled" (in-process); messaging extends to network-partition regime. | Novel. The discriminator is sound: edge owns *where compute runs* (geographic locality, C4); performance owns *generic measurement*; concurrency owns *in-process primitives*; messaging owns *cross-process/network-failure-model*. No verbatim duplication. Each domain owns a distinct angle on shared C-rules — the same pattern that distinguished k8s P8 from iac P3 in v0.2 (D-021 precedent). | 0.86 |
|
||||
| 1.3 | Is the 16-doc language-derived extension (P3) achievable in one phase? | RESEARCH-v0.4 §Language-Derived: 16 derived docs follow the existing stub's section-header-trace pattern (confirmed in `languages/typescript.md`: `## Type System (C1 Correctness, Data P7 Type Fidelity)`). PLAN-v0.4 P3 Wave 2: 16 tasks at ≤5 concurrent (5-then-5-then-5-then-1, A-008 @ 0.90). D-063: no P1–P10 derivation (apply, don't derive) — lower per-doc effort than domain derived docs. | Achievable. Language-derived docs are structurally simpler than domain derived docs (no first-principles derivation, no matrix rows, no decision matrices — they apply existing P-rules). v0.1 P3 produced 27 derived docs with heavier structure. The 16-doc load is within demonstrated throughput and below v0.1's high-water mark. | 0.83 |
|
||||
|
||||
**Axis 1 confidence: 0.85.** No challenges.
|
||||
|
||||
### Axis 2 — Scope
|
||||
|
||||
| # | Forcing Question | Evidence | Answer | Confidence |
|
||||
|---|-----------------|----------|--------|------------|
|
||||
| 2.1 | Is v0.4 over-scoped (2 new domains + 16 language docs + matrix + review + examples + cross-links)? Should it split into v0.4 + v0.5? What would you cut? | PROJECT.md v0.4 section: 2 domains (10 P-rules each = 20 matrix rows), 16 language-derived docs (0 matrix rows), matrix/review/examples/manifest extensions. REQUIREMENTS ATELIER-92..117 = 26 reqs. v0.3 had 32 reqs (4 domains, 40 matrix rows) and shipped clean. | **Not over-scoped — smaller than v0.3.** 26 reqs vs v0.3's 32. The 16 language-derived docs are the volume driver but carry zero matrix-row cost (D-063/D-066), so the traceability surface grows by only 20 rows (edge + messaging), the smallest matrix extension since v0.2. Cutting to v0.4a (2 domains) + v0.4b (languages) would double release overhead with no load benefit — P3's 16 docs are simpler per-doc than domain docs. The current single-milestone plan is the right call. | 0.85 |
|
||||
| 2.2 | What would you cut if forced? | (adversarial) | The simplest 80% version: ship edge + messaging domains + matrix/review (drop the 16 language-derived docs to v0.5). This halves the doc count. **But:** the language-derived extension closes pre-existing MANIFEST drift (languages/ unlisted since v0.1, per D-065/IDEATE-31) and is the v0.3 ROADMAP "Future Milestones" candidate explicitly assigned to v0.4. Deferring it re-opens the 2x-deferral zombie risk flagged in v0.3 G-001. The 16 docs are low-cost (apply, don't derive) — keeping them is correct. | 0.82 |
|
||||
| 2.3 | Who owns the requirements, and are they frozen? | REQUIREMENTS.md v0.4 reqs ATELIER-92..117 (all pending). Ideation log IDEATE-31..43: 13 accepted, 0 deferred, 0 rejected. Traceability matrix maps phases→reqs. | Frozen post-ideation. 26 reqs across 6 phases. 13 accepted refinements baked into task must-haves (no moving targets). The ideation stage closed with 0 deferred. | 0.88 |
|
||||
| 2.4 | What is explicitly out of scope? | PROJECT.md lines 49–55: tooling/linters, translation, agent adapters, per-domain release artifacts, runtime code. D-020: docs-only. D-063: no new P-rules for languages. | Explicit and enumerated: no runtime code, no tooling, no translation, no new core principles, no new matrix rows for languages, no standalone `.ts`/`.py`/`.go`/`.rs` artifacts (D-020 + ATELIER-110 artifact-types extension). The scope boundary is answerable. | 0.88 |
|
||||
| 2.5 | Are there hidden requirements disclosed late? | IDEATE-32 → ATELIER-117 (ARCHITECTURE.md v0.4 update — architecture-tree drift, detected during ideation). Verified: `.ciagent/atelier/ARCHITECTURE.md` component tree stops at v0.3 (lines 14–19 list only through compliance). | ATELIER-117 is a *detected* hidden requirement, surfaced and added during ideation (not late in delivery). The architecture-tree drift is real (ARCHITECTURE.md has no edge/messaging/languages-derived entries). IDEATE-32 caught it before execution — the correct time to surface it. No undisclosed hidden requirements remain. | 0.85 |
|
||||
|
||||
**Axis 2 confidence: 0.86.** No challenges.
|
||||
|
||||
### Axis 3 — Cost/Risk
|
||||
|
||||
| # | Forcing Question | Evidence | Answer | Confidence |
|
||||
|---|-----------------|----------|--------|------------|
|
||||
| 3.1 | 46 tasks across 6 phases — what's the riskiest task? | PLAN-v0.4 task inventory: P1 01-01-01 (edge first-principles), P4 04-01-01 (matrix extension), P4 04-01-02 (domain-coverage with Languages sub-table), P3 03-02a..d (16 derived docs). | The riskiest task is **04-01-01 (matrix extension + IDEATE-34 reconciliation)** — but the risk is not the work (appending 20 rows is mechanical); the risk is the **false premise** that the matrix has 171 rows vs stated 170. Direct verification shows the matrix has **exactly 170 rows** (17 × 10). The reconciliation step is built on a wrong premise. **Mitigation:** see binding decision G-014 — the task must verify 170 exists and append 20 to reach 190; it must NOT "fix" a phantom 171→170. | 0.80 |
|
||||
| 3.2 | What's the cost of a phase failing mid-way? | NFR milestone, docs-only. All phases produce markdown. No runtime, no infra, no migration. | Cost of a phase failing = re-authoring markdown. P3 (largest) failing mid-way wastes the 16 derived-doc tokens but the work is recoverable (edit + re-run). No cascading infrastructure cost, no data loss, no deployment rollback. Rework is bounded and cheap — the same property that made v0.1/v0.2/v0.3 low-risk. | 0.90 |
|
||||
| 3.3 | Predictable cost drivers not in the original budget? | PROJECT.md: no runtime code, no infrastructure, no licensing, no external services. RESEARCH-v0.4 §Risks: 10 risks, all content/traceability (no external cost drivers). | None. Docs-only = no licensing, no infra, no security review fees, no support contracts. Token cost is proportional to markdown volume, scoped by REQ count (26). The only cost driver is markdown authoring, which is bounded. | 0.88 |
|
||||
| 3.4 | Single project-killing risk? | RESEARCH-v0.4 §Risks: orphaned P-rules, edge/messaging duplication, language-derived new-P-rule violation, MANIFEST drift, runtime artifacts, persona explosion, matrix row-count, decision-matrix omission. | The single risk that would undermine the framework's core value: **language-derived docs introducing new P-rules** (D-063 violation) or **orphaned from domain P-rules** (D-066 violation). Either breaks the traceable hierarchy (the unique value). **Mitigation:** languages-engineer persona constraint "apply, don't derive" + agent-checklist language-derived scope check (ATELIER-108) + P3 Verify (no new P-rules, every section traces ≥1 domain P-rule) + P4 Languages Coverage sub-table (exactly 16 rows, IDEATE-42) + P6 domain-expert verification. Well-mitigated with multiple layers. | 0.84 |
|
||||
|
||||
**Axis 3 confidence: 0.86.** Challenge: IDEATE-34 false premise (G-014). Auto-resolved with binding condition.
|
||||
|
||||
### Axis 4 — Vertical-slice integrity
|
||||
|
||||
| # | Forcing Question | Evidence | Answer | Confidence |
|
||||
|---|-----------------|----------|--------|------------|
|
||||
| 4.1 | Is each phase independently shippable per D-053? | PLAN-v0.4 invariants: P1 ships edge self-consistent (matrix in P4); P2 ships messaging self-consistent; P3 ships 16 language docs self-consistent; P4 closes traceability + manifest + architecture; P5 closes examples + cross-links; P6 ships release. Tag sequence v0.3.1→v0.3.6. | Each phase is shippable. P1/P2 ship domains whose P-rules trace to core internally (matrix extension is traceability closure, not internal-consistency). P3 ships language docs whose section-headers trace to existing domain P-rules (the Languages Coverage sub-table that aggregates these lands in P4 — but each derived doc is self-consistent because it traces to already-shipped domain P-rules). P4 closes manifest + matrix + architecture. P5 closes examples + cross-links. This mirrors v0.3's D-053 pattern that shipped clean. | 0.84 |
|
||||
| 4.2 | P1 ships edge domain before matrix lands in P4 — acceptable? | D-053 (v0.3): "matrix rows land in P4 — acceptable because the domain is self-consistent; matrix extension is the traceability closure, not a blocker." v0.2/v0.3 both shipped domain phases before P4 matrix extension. | Acceptable. The edge domain's 10 P-rules each name their C-rule derivation *inline* in `first-principles.md` (per PLAN 01-01-01 must-have). The matrix row is the *aggregate* trace; the inline trace is the *per-rule* detail. A domain with correct inline traces is self-consistent even before the matrix row exists. The matrix in P4 is the verifiable closure, not the source of truth. Precedent: v0.2/v0.3 both did this. | 0.86 |
|
||||
| 4.3 | P3 ships language docs that trace to domain P-rules — traceable without the matrix update? | D-063/D-066: language-derived docs use section-header-trace pattern (C-rule + Domain P-rule inline). Languages Coverage sub-table lands in P4. `languages/typescript.md` confirms pattern: `## Type System (C1 Correctness, Data P7 Type Fidelity)`. | Traceable without the matrix update. Each language-derived doc's section headers name the traced domain P-rules inline — the traceability is in the doc itself, not the matrix. The Languages Coverage sub-table (P4) is the *aggregate verifiable view*, parallel to how the domain→core matrix is the aggregate view of inline domain traces. A language-derived doc with correct inline section-header traces is self-consistent even before the sub-table exists. | 0.85 |
|
||||
| 4.4 | Edge↔messaging cross-links dangle at P1 tag — acceptable? | PLAN 01-02-03 (edge/iot.md): "cross-link `messaging/queues` ... this link dangles until P2, acceptable per D-053, verified bidirectional in P5 ATELIER-114 per IDEATE-40". A-010 @ 0.86. | Acceptable but a genuine temporary broken-link state. edge/iot.md (P1) cross-links messaging/queues.md (P2) — at the v0.3.1 tag, that link 404s. This is acceptable per D-053 (cross-links close in P5) and is the same pattern as v0.3 intra-milestone links. **Mitigation:** P5 cross-link audit (05-02-01) explicitly verifies edge↔messaging links resolve in BOTH directions before P6 ship (IDEATE-40). The temporary dangle is a documented, time-bounded state, not a silent defect. | 0.82 |
|
||||
|
||||
**Axis 4 confidence: 0.84.** No blocking challenges.
|
||||
|
||||
### Axis 5 — Traceability
|
||||
|
||||
| # | Forcing Question | Evidence | Answer | Confidence |
|
||||
|---|-----------------|----------|--------|------------|
|
||||
| 5.1 | Do all 20 proposed new P-rules trace to ≥1 core C-rule without duplicating existing domain P-rules? | RESEARCH-v0.4 §Edge: P1→C4,C1; P2→C1,C5; P3→C8,C1; P4→C1,C5; P5→C1; P6→C1,C3; P7→C1,C5; P8→C4,C6; P9→C1,C8; P10→C7,C5. §Messaging: P1→C1,C2; P2→C1,C2; P3→C1; P4→C1,C2; P5→C1,C5; P6→C1,C8; P7→C4,C6; P8→C5,C7; P9→C5,C6; P10→C7,C1. All 20 map to ≥1 C-rule. Edge derives 7 C-rules; messaging derives 7 C-rules. `core/first-principles.md` confirms C1–C8 definitions. | All 20 trace to ≥1 C-rule. None duplicates an existing domain P-rule verbatim: edge P6 (geographic cache invalidation, C1/C3) ≠ performance P5 (generic caching, C3/C6) — different angle + different C-rule mix. messaging P6 (broker backpressure, C1/C8) ≠ concurrency P9 (in-process bounded queue, C1/C8) — same C-rules but different failure model (network vs OOM), per D-062. The matrix invariant (every P-rule → ≥1 C-rule, no orphans) holds. | 0.87 |
|
||||
| 5.2 | RESEARCH-v0.4 against `core/first-principles.md` — derivations valid? | C1 Correctness (non-negotiable), C4 Locality (decisions near consequences), C5 Reversibility, C8 Economy (constrained resources). Edge P1 "Proximity is the Design Driver" → C4 Locality (compute near user/data) — valid geographic-locality derivation. Edge P3 "Resources are Constrained" → C8 Economy — valid. Messaging P7 "Partitioning is Intentional" → C4 Locality (ordering locality) — valid. | Derivations are valid. Edge's C4 emphasis (geographic locality) is a sound reading of "decisions and consequences live near each other" — geographic proximity is a locality concern. Messaging's C4 (partition-key = ordering locality) is sound. No forced/contrived derivations detected. The C1 emphasis (correctness under partition/network-failure) is correct — partition tolerance is a correctness property, not a performance preference (edge P1 explicitly states this). | 0.85 |
|
||||
| 5.3 | RESEARCH-v0.4 against existing domains (concurrency/patterns.md for messaging; performance/observability for edge) — no duplication? | `concurrency/patterns.md` Pattern 1 (Message Passing, in-process channels), Pattern 5 (Bounded Queue, in-process "producer is blocked or signaled"). `performance` P5 Caching (generic C3,C6). RESEARCH-v0.4 §Messaging boundary: messaging owns cross-process broker-backed analog with network-failure semantics. §Edge boundary: edge owns geographic/partition-aware cache invalidation. | No duplication. The failure-model discriminator (D-062) is the cleanest boundary: concurrency fails by OOM/crash (in-process); messaging fails by network partition/broker-restart (cross-system). Edge's cache angle (P6, partition-aware invalidation, C1/C3) is distinct from performance's generic cache (P5, C3/C6). Cross-links are one-directional outward (D-026 extended) — no back-link edits to v0.1/v0.2/v0.3 content. | 0.86 |
|
||||
| 5.4 | Will the matrix stay at exactly 190 P-rules post-v0.4 (D-059)? | Direct verification: `matrix/principles-matrix.md` currently has exactly **170 P-rows** (17 domains × 10, per awk count). RESEARCH-v0.4 §Matrix: +20 rows (10 edge + 10 messaging). | **CRITICAL FINDING:** The matrix currently has exactly 170 rows, matching the invariant. IDEATE-34/ATELIER-106 refinement claims a "171 vs stated 170 discrepancy" that must be reconciled — **this discrepancy does not exist**. Post-v0.4 will be 170 + 20 = 190, exactly as planned. The reconciliation step (PLAN 04-01-01, 06-01-02) is built on a false premise. See binding decision G-014. | 0.88 |
|
||||
|
||||
**Axis 5 confidence: 0.86.** Challenge: IDEATE-34 false premise (G-014) — auto-resolved with binding condition; does not block but must be corrected in task wording.
|
||||
|
||||
### Axis 6 — Decision quality
|
||||
|
||||
| # | Forcing Question | Evidence | Answer | Confidence |
|
||||
|---|-----------------|----------|--------|------------|
|
||||
| 6.1 | Are D-054..D-075 well-grounded? | CLARIFY-v0.4.md D-061..D-069 (avg confidence 0.86, range 0.80–0.92). PLAN-v0.4.md D-070..D-073 (range 0.83–0.88). All above 0.80 except D-067 (0.80, messaging persona). PROJECT.md v0.4 section encodes D-054..D-060 (SPECIFY). | Well-grounded. 15 decisions, average 0.85. All ≥0.80. The lowest is D-067 (0.80 — messaging P2 persona choice) which is explicitly justified by deferring to ROADMAP's 2-persona naming + D-022 within-competence precedent + edge-engineer consultation baked into P2 task must-have (D-051 pattern). No decision is below the 0.80 "well-grounded" bar. | 0.85 |
|
||||
| 6.2 | Any low-confidence (<0.80) decisions that should be revisited? | CLARIFY-v0.4.md: D-067 @ 0.80 is the floor. PLAN-v0.4 assumptions A-006..A-010 (0.82–0.95). | None below 0.80. D-067 @ 0.80 is the lowest decision; it is justified by the ROADMAP's explicit 2-persona naming (the authoritative signal) and the D-022 precedent. Revisiting it would require a 3rd phase-specific persona (messaging-engineer), which the RESEARCH explicitly rejected to avoid persona-explosion (roster would hit 6, exceeding v0.3's 5 peak). The constraint-baked-into-task mitigation (PLAN 02-01-01 must-have: "P1/P2 demonstrated edge + messaging boundary (D-062)") preserves the distributed-systems lens without the persona. No revisit needed. | 0.83 |
|
||||
| 6.3 | Any decisions contradict prior decisions (D-001..D-053)? | D-053 (vertical-slice integrity) — honored (PLAN invariants line 7). D-020 (docs-only) — honored (PLAN invariants line 13, no `.ts`/`.py`/`.go`/`.rs` artifacts). D-026 (one-directional cross-links) — honored (PLAN 05-02-01 "no back-link edits to v0.1/v0.2/v0.3 content"). D-025 (2+2 examples) — honored (D-068 confirms). | No contradictions. D-053/D-020/D-026/D-025 are all explicitly carried forward. D-063 (no new P-rules for languages) is consistent with ARCHITECTURE.md hierarchy (`languages/` depend on `domains/`). D-066 (no language matrix rows) preserves D-059's 190 total. The v0.4 decisions extend the v0.1→v0.2→v0.3 convention chain without breaking any prior commitment. | 0.88 |
|
||||
| 6.4 | Does IDEATE-34 (matrix discrepancy) contradict the actual matrix state? | IDEATE-34 refinement (REQUIREMENTS ATELIER-106 note): "The current matrix has 171 P-rows vs the stated 170 invariant — P4 must reconcile this pre-existing discrepancy." Direct verification: matrix has exactly 170 rows. | **Yes — IDEATE-34 is based on a false premise.** The matrix has 170 rows, not 171. The "reconcile 171→170" instruction in PLAN 04-01-01 and audit 06-01-02 is unfounded. This is a decision-quality defect in the ideation stage: IDEATE-34 (confidence 0.82) asserted a discrepancy without verifying the actual count. **Not a contradiction of a prior decision, but a false factual claim.** See binding decision G-014 — the reconciliation step must be reframed as "verify 170 exists, append 20, confirm 190" — NOT "find and fix a phantom 171st row." | 0.80 |
|
||||
|
||||
**Axis 6 confidence: 0.84.** Challenge: IDEATE-34 false premise (G-014).
|
||||
|
||||
### Axis 7 — Persona coverage
|
||||
|
||||
| # | Forcing Question | Evidence | Answer | Confidence |
|
||||
|---|-----------------|----------|--------|------------|
|
||||
| 7.1 | Do edge-engineer + languages-engineer + tech-writer + domain-expert + lead-developer cover all P1–P6 authoring needs? | PERSONAS.md: 3 permanent (lead-developer, tech-writer, domain-expert) + 2 phase-specific (edge-engineer P1, languages-engineer P3). PLAN persona assignments: P1 edge-engineer; P2 tech-writer + domain-expert (edge-engineer consult); P3 languages-engineer; P4 domain-expert/tech-writer/lead-developer; P5 edge-engineer + tech-writer + domain-expert; P6 all 5 review. | Coverage is complete. P1 edge (edge-engineer specialist). P2 messaging (tech-writer + domain-expert generalists with edge-engineer consultation for D-062 boundary — baked into task 02-01-01 must-have per D-051 pattern). P3 languages (languages-engineer specialist). P4 matrix/review/manifest/architecture (domain-expert/tech-writer/lead-developer split by artifact type). P5 examples (edge-engineer for edge examples, tech-writer for messaging). P6 all 5 review then remove 2. No phase lacks an assigned author. | 0.84 |
|
||||
| 7.2 | Is messaging P2 under-resourced (no phase-specific persona per D-067)? | D-067 @ 0.80: messaging P2 authored by tech-writer + domain-expert with edge-engineer consultation. CLARIFY-v0.4 §7: rejected messaging-engineer persona (roster would hit 6, exceeding v0.3's 5 peak). PLAN 02-01-01 must-have: "edge-engineer consultation baked in (D-067, D-051 pattern): P1/P2 demonstrated edge + messaging boundary (D-062)." | Under-resourced *relative to P1/P3* (which have specialists) but **adequately resourced** for the work. Messaging's specialty (delivery semantics, idempotency, ordering) is within domain-expert's cross-cutting competence (correctness/observability derivations the domain-expert already validates). The edge-engineer consultation + task must-have preserves the D-062 boundary. RESEARCH-v0.4 §Messaging provides thorough prior-art (Kafka, RabbitMQ, NATS, Jepsen, Reactive Streams, CloudEvents). The risk is content-quality (specialist depth) not traceability (domain-expert validates that). Mitigation is adequate per the v0.3 G-009 precedent (i18n/compliance shipped without specialists). | 0.78 |
|
||||
| 7.3 | Is there a key-person dependency? | PERSONAS.md: edge-engineer + languages-engineer are phase-specific, removed post-v0.4. D-051 pattern (constraints baked into task must-haves) survives persona removal. | No key-person risk beyond the v0.3-bounded pattern. Personas are constraint sets, not humans. The languages-engineer's "apply, don't derive" constraint (D-063) is baked into P3 task must-haves AND the agent-checklist (ATELIER-108) AND the P3 Verify block. The edge-engineer's "proximity is correctness" constraint (D-061) is baked into P1 task 01-01-01 must-have. If either persona "fails," the constraints survive in the task specs and review checks. | 0.82 |
|
||||
|
||||
**Axis 7 confidence: 0.81.** Challenge: messaging P2 under-resourcing (G-015) — auto-resolved; bounded by RESEARCH prior-art + domain-expert validation, parallel to v0.3 G-009.
|
||||
|
||||
### Axis 8 — Test/Verify
|
||||
|
||||
| # | Forcing Question | Evidence | Answer | Confidence |
|
||||
|---|-----------------|----------|--------|------------|
|
||||
| 8.1 | Are the per-phase Verify blocks in PLAN-v0.4.md sufficient to catch a broken phase before ship? | PLAN-v0.4 each phase has a Verify block: P1 (structural 5 files, behavioral P1–P10 trace, security P9/P6, quality cross-links + decision matrices + D-061 boundary). P2 (similar + edge↔messaging link resolution post-P2). P3 (20 files, no new P-rules, no P1–P10 on stubs, no runtime artifacts, IDEATE-40 cross-links). P4 (matrix 190, coverage sub-table 16 rows, ARCHITECTURE.md component tree, review parity, MANIFEST authoritative). P5 (4 examples, single-breach citation, bidirectional cross-links). P6 (26 reqs covered, reconstruction, matrix 190, MANIFEST, ARCHITECTURE, audit clean). | Sufficient. Each Verify block has structural + behavioral + security + quality checks tied to specific decisions (D-061/D-062/D-063/D-066/D-068/D-069/IDEATE-40/IDEATE-42). The P4 Verify is the strongest: matrix row-count (190), Languages Coverage sub-table (exactly 16 rows — catches orphan language docs), MANIFEST reconstruction (catches unlisted docs), ARCHITECTURE.md component-tree test (catches architecture-tree drift). P6 audit adds the reconstruction + matrix-count + MANIFEST reconstruction triple-test. | 0.86 |
|
||||
| 8.2 | What would slip through? | (adversarial) | (a) **Content correctness** — Verify blocks check structure + traceability + cross-links, not technical accuracy of, e.g., the CRDT-vs-LWW decision matrix in sync.md or the Kafka rebalance-strategy enumeration in streams.md. A technically wrong but well-structured doc passes Verify. Mitigation: P6 domain-expert + edge-engineer/languages-engineer review (D-052), but this is one layer, not redundant. (b) **The IDEATE-34 false premise** — the P4/P6 "reconcile 171→170" check is unverifiable against a non-existent discrepancy; an executor could misinterpret it as "delete a row" (harmful) or "confirm 170" (correct). G-014 fixes the wording. (c) **Edge↔messaging dangling links at P1 tag** — structurally acceptable (D-053) but a real broken-link state until P2; only caught bidirectionally in P5, not at the P1/P2 tags. | 0.78 |
|
||||
| 8.3 | Is there a working definition of done? | PLAN-v0.4 P6 Verify: 26 reqs covered, reconstruction test passes, matrix exactly 190 (reconciled), MANIFEST reconstruction (incl. Languages section), ARCHITECTURE.md component-tree test, audit clean, tag v0.3.6, branches deleted, personas removed (roster=3). | DoD is concrete and testable: 26 reqs covered, matrix = 190 rows (10 × 19), MANIFEST reconstruction passes (incl. Languages), tag v0.3.6 on main. Not "whatever the demo shows." The matrix-count test (190) and MANIFEST reconstruction are mechanical, automatable checks. | 0.88 |
|
||||
|
||||
**Axis 8 confidence: 0.84.** Challenges: content-correctness gap (inherent to docs projects, bounded by P6 review), IDEATE-34 wording (G-014), dangling links (G-016). All auto-resolved.
|
||||
|
||||
### Axis 9 — Cross-link / matrix integrity
|
||||
|
||||
| # | Forcing Question | Evidence | Answer | Confidence |
|
||||
|---|-----------------|----------|--------|------------|
|
||||
| 9.1 | Will the matrix stay at exactly 190 P-rules post-v0.4 (D-059)? | Direct verification: matrix currently has exactly 170 rows. RESEARCH-v0.4 §Matrix: +20 (10 edge + 10 messaging). Languages add 0 (D-063/D-066). | Yes — 170 + 20 = 190. The invariant holds. The IDEATE-34 "171 vs 170" claim is false; see G-014. Post-v0.4 the matrix will have exactly 190 rows (10 per domain × 19), the smallest extension since v0.2. | 0.88 |
|
||||
| 9.2 | Will the languages/ sub-table in domain-coverage.md capture all 16 language-derived docs? | ATELIER-107 (IDEATE-42): Languages Coverage sub-table with exactly 16 rows. RESEARCH-v0.4 §Language-Derived sketches the 16-row sub-table (TS quartet + PY quartet + GO quartet + RS quartet). PLAN 04-01-02 must-have: "EXACTLY 16 rows (one per language-derived doc)". P4 Verify: "Languages Coverage sub-table has exactly 16 rows". P6 06-01-01: "domain-expert verifies ... 16 Languages Coverage sub-table rows match the derived docs' actual section-header traces (IDEATE-42)". | Yes — the sub-table is specified at exactly 16 rows, verifiable as a P4 review check, and cross-checked in P6 against the actual section-header traces. The 16 = 4 languages × 4 derived docs. No orphans (every row traced to ≥1 domain P-rule per D-066). The sub-table is the verifiable aggregate; inline section-headers are per-doc detail. | 0.86 |
|
||||
| 9.3 | Will the bidirectional edge↔messaging cross-links actually be verified (IDEATE-43/IDEATE-40)? | PLAN 05-02-01 (cross-link audit): "(b) edge↔messaging intra-v0.4 links resolve in BOTH directions — every edge→messaging and messaging→edge link resolves by the P6 ship". P5 Verify: "edge↔messaging intra-v0.4 links resolve bidirectionally (IDEATE-40)". RESEARCH-v0.4 §Review: edge/iot.md → messaging/queues.md (MQTT QoS parallels). | Yes — the P5 cross-link audit (05-02-01) explicitly verifies bidirectional edge↔messaging resolution, AND language-derived docs link to both new (edge/messaging) and existing domains where relevant (e.g., ts-async → messaging/delivery-semantics; go-concurrency → concurrency/patterns + messaging/queues). The audit is a dedicated Wave 2 task after all docs exist, so links are resolvable. The one gap: at the P1 tag (v0.3.1), edge→messaging links dangle (messaging authored in P2) — acceptable per D-053, time-bounded, verified in P5. | 0.84 |
|
||||
| 9.4 | MANIFEST integrity — will the Languages section close the pre-existing drift? | MANIFEST.md verified: currently has NO `## Languages` section (sections: Reading Order, Core, Domains, Examples, Cross-Cutting). `languages/{typescript,python,go,rust}.md` exist since v0.1 but are unlisted. D-065/IDEATE-31: add `## Languages` between `## Domains` and `## Examples`. PLAN 04-03-01 must-have: lists 4 stubs + 16 derived docs + Cross-Cutting = "19 domains, 190 P-rules post-v0.4". | Yes — the Languages section is specified to list 4 v0.1 stubs (closing pre-existing drift parallel to ESC-002/ATELIER-91) + 16 new derived docs, positioned per reading order (core→domains→languages→examples). P4 Verify: "MANIFEST is authoritative — every v0.4 file listed, languages/ listed closing pre-existing drift". P6 MANIFEST reconstruction test catches any unlisted doc. | 0.87 |
|
||||
|
||||
**Axis 9 confidence: 0.86.** Challenge: dangling links at P1 tag (G-016) — auto-resolved; time-bounded + P5-verified.
|
||||
|
||||
### Meta — Closing Review
|
||||
|
||||
| # | Forcing Question | Evidence | Answer | Confidence |
|
||||
|---|-----------------|----------|--------|------------|
|
||||
| M.1 | If you were the auditor, what would you flag? | (adversarial) | **The IDEATE-34 false premise** — the plan and audit instruct executors to "reconcile a pre-existing 171-vs-170 matrix discrepancy" that does not exist (verified: exactly 170 rows). This is the single most material finding: a verification gate built on a wrong fact. An executor could (a) waste effort hunting a phantom row, or (b) worse, delete a valid row to "fix" 171→170, breaking the invariant. G-014 fixes the wording to "verify 170, append 20, confirm 190." Secondary: content-correctness is one-layer-validated (P6 review only) — inherent to docs projects, bounded. | 0.82 |
|
||||
| M.2 | What is the project NOT doing that it should? | PROJECT.md out-of-scope: tooling/linters. | Not doing: automated P-rule→C-rule trace validation (manual domain-expert verification only). At 190 P-rules + 16 language-derived traces, manual verification is approaching the limit where tooling would be warranted — but remains feasible for this milestone. Explicitly out of scope; not a v0.4 blocker. The Languages Coverage sub-table (16 rows) actually *enables* future tooling by making language traces machine-readable. | 0.82 |
|
||||
| M.3 | Simplest 80% version? | (adversarial) | Ship edge + messaging domains + matrix/review/manifest/architecture (drop the 16 language-derived docs to v0.5). This cuts P3 entirely (20 files → 0) and reduces the milestone to ~10 reqs. **But:** the language-derived extension closes pre-existing MANIFEST drift (D-065) and is the v0.3 ROADMAP "Future Milestones" candidate explicitly assigned to v0.4. Deferring re-opens the 2x-deferral zombie risk (v0.3 G-001). The 16 docs are low-cost (apply-don't-derive, no matrix rows). Keeping them is the right scope call. | 0.83 |
|
||||
| M.4 | What must be true in 90 days for success, and is it true today? | (adversarial) | Must be true: (a) 20 new P-rules trace cleanly to C-rules — RESEARCH-v0.4 pre-maps them, P4 verifies. (b) 16 language-derived docs each trace to ≥1 domain P-rule (no orphans) — P3 Verify + P4 sub-table (16 rows) + P6 cross-check. (c) Content is technically correct (CRDT/LWW, Kafka rebalance, MQTT QoS, delivery semantics) — RESEARCH grounds it, P6 validates. (d) Matrix = exactly 190 — verified today at 170, +20 planned. (a)(b)(d) have explicit verification gates with correct premises (post-G-014). (c) depends on execution quality. All achievable. | 0.83 |
|
||||
|
||||
**Meta confidence: 0.83.**
|
||||
|
||||
---
|
||||
|
||||
## Summary Table
|
||||
|
||||
| # | Axis | Verdict | Confidence |
|
||||
|---|------|---------|------------|
|
||||
| 1 | Feasibility | PASS | 0.85 |
|
||||
| 2 | Scope | PASS | 0.86 |
|
||||
| 3 | Cost/Risk | CONDITIONAL (G-014: fix IDEATE-34 wording) | 0.86 |
|
||||
| 4 | Vertical-slice integrity | PASS | 0.84 |
|
||||
| 5 | Traceability | CONDITIONAL (G-014: false premise in reconciliation step) | 0.86 |
|
||||
| 6 | Decision quality | CONDITIONAL (G-014: IDEATE-34 factual defect) | 0.84 |
|
||||
| 7 | Persona coverage | PASS (G-015 noted, bounded) | 0.81 |
|
||||
| 8 | Test/Verify | CONDITIONAL (G-014: audit test premise; G-016: dangling links acknowledged) | 0.84 |
|
||||
| 9 | Cross-link / matrix integrity | PASS | 0.86 |
|
||||
| — | Meta | PASS | 0.83 |
|
||||
| **—** | **Overall** | **CONDITIONAL PROCEED** | **0.83** |
|
||||
|
||||
---
|
||||
|
||||
## Binding Decisions
|
||||
|
||||
| ID | Decision | Rationale | Confidence | Alternatives |
|
||||
|----|----------|-----------|------------|--------------|
|
||||
| G-013 | Proceed with v0.4 scope as planned (2 domains + 16 language-derived docs); do NOT split into v0.4/v0.5 | 26 reqs < v0.3's 32 reqs; language-derived docs carry 0 matrix-row cost (D-063/D-066); v0.1 P3 demonstrated 27 derived docs in one phase; deferral re-opens 2x-zombie risk (v0.3 G-001); the 16 docs are the v0.3 ROADMAP "Future Milestones" candidate explicitly assigned to v0.4 | 0.85 | Split into v0.4a (domains) + v0.4b (languages) — rejected: double release overhead, no load benefit; Drop languages to v0.5 — rejected: 2x deferral zombie risk |
|
||||
| G-014 | **CONDITION — fix IDEATE-34 false premise before P4 executes.** The matrix currently has exactly 170 P-rows (verified: 17 domains × 10). There is NO "171 vs 170 discrepancy." PLAN task 04-01-01 and audit 06-01-02 must be reworded from "reconcile pre-existing 171→170 discrepancy" to "verify matrix has exactly 170 rows before appending 20; confirm post-append count = 190." The executor must NOT hunt for or delete a phantom 171st row. | Direct verification of `matrix/principles-matrix.md`: `awk '/^## /{...} /^\| P[0-9]/{count++}'` returns 170 total, 15 lines per section header (10 P-rows + header rows), 17 domain sections. IDEATE-34 (confidence 0.82) asserted the discrepancy without verifying the actual count — a factual defect in the ideation stage. The reconciliation instruction is unfounded and risks an executor breaking the invariant by deleting a valid row. | 0.88 | Leave the wording as-is — rejected: an executor could delete a valid row to "fix" a non-existent discrepancy; Remove the reconciliation step entirely — rejected: keeping a verify-170 step is harmless and good practice, just reworded |
|
||||
| G-015 | Proceed with tech-writer + domain-expert for messaging P2 without a messaging-engineer persona (confirms D-067) | Messaging's specialty (delivery semantics, idempotency, ordering) is within domain-expert's cross-cutting competence; edge-engineer consultation + task must-have (02-01-01) preserves D-062 boundary; RESEARCH-v0.4 §Messaging provides thorough prior-art (Kafka/RabbitMQ/NATS/Jepsen/Reactive Streams/CloudEvents); v0.3 G-009 precedent (i18n/compliance shipped without specialists); adding a 3rd phase-specific persona would push roster to 6, exceeding v0.3's 5 peak (persona-explosion risk). Assumption A-011 logged: messaging content correctness depends on RESEARCH prior-art + P6 review. | 0.78 | Add messaging-engineer persona — rejected: persona explosion (roster 6 > v0.3 peak 5); Defer messaging to v0.5 — rejected: scope-creep deferral |
|
||||
| G-016 | The edge↔messaging dangling cross-link at P1 tag (edge/iot.md → messaging/queues.md) is acceptable per D-053; the P5 audit (05-02-01) MUST verify bidirectional resolution before P6 ship (confirms IDEATE-40) | D-053 established that cross-links close in P5, not at the domain-phase tag; v0.3 had the same intra-milestone dangling-link pattern; the dangle is time-bounded (resolves at P2 tag v0.3.2); P5 audit explicitly checks both directions (IDEATE-40). Not a silent defect — a documented, verified state. | 0.82 | Block P1 ship until messaging exists — rejected: breaks vertical-slice integrity (D-053), serializes independent work; Remove the edge→messaging link from P1 — rejected: loses the MQTT QoS / delivery-semantics parallel, a high-value cross-link |
|
||||
| G-017 | CONFIRM D-063/D-066: language-derived docs introduce NO new P-rules and NO new matrix rows; the Languages Coverage sub-table (exactly 16 rows) is the verifiable aggregate | Direct verification of `languages/typescript.md` confirms the section-header-trace pattern (`## Type System (C1 Correctness, Data P7 Type Fidelity)` — no P1–P10 numbering). ARCHITECTURE.md confirms hierarchy (`languages/` depend on `domains/`). Adding P-rules to languages would break the hierarchy and the 190-row invariant. The 16-row sub-table is the right traceability home (parallel to the per-domain row schema). | 0.87 | Add P1–P10 to languages — rejected: breaks hierarchy + matrix invariant; Add language rows to principles-matrix.md — rejected: languages have no P-rules to map (D-063) |
|
||||
| G-018 | CONFIRM D-065/IDEATE-31: MANIFEST gains a `## Languages` section closing pre-existing drift since v0.1 (parallel to ESC-002/ATELIER-91) | MANIFEST.md verified: NO `## Languages` section exists despite `languages/{typescript,python,go,rust}.md` existing since v0.1. This is pre-existing drift on the exact ESC-002 precedent (examples/ was unlisted, closed in v0.3). v0.4 touches `languages/` for the first time since v0.1, so it closes the drift. | 0.87 | Defer stubs to a future milestone, list only 16 new docs — rejected: half-closes the drift, inconsistent with ESC-002 precedent; List languages under `## Domains` — rejected: languages are not domains (ARCHITECTURE.md hierarchy) |
|
||||
|
||||
### Escalations
|
||||
|
||||
**None.** All challenges auto-resolved at confidence ≥ 0.78 (range: 0.78–0.88). At full autonomy, the IDEATE-34 false premise (G-014) is the lowest-confidence finding but is auto-resolvable by rewording the task (not an escalation — it does not require a human decision, just a factual correction). No axis scored below 0.60 on any forcing question.
|
||||
|
||||
### Assumptions Logged (this grill)
|
||||
|
||||
| # | Assumption | Confidence |
|
||||
|---|-----------|------------|
|
||||
| A-011 | Messaging P2 content correctness (delivery-semantics, Kafka rebalance, MQTT QoS) depends on RESEARCH-v0.4 prior-art quality + P6 domain-expert/edge-engineer review, not on a specialist messaging-engineer persona. If P6 surfaces fundamental content errors, the remedy is P2 rework (bounded — markdown edits). Parallel to v0.3 A-006 (i18n/compliance without specialists). | 0.78 |
|
||||
|
||||
### Conditions for SHIP (CONDITIONAL verdict)
|
||||
|
||||
The phase-0 plan is sound and P1 may proceed immediately. The following condition MUST be addressed before P4 executes (it does not block P1/P2/P3):
|
||||
|
||||
1. **G-014 (mandatory):** Reword the IDEATE-34/ATELIER-106 reconciliation instructions in PLAN-v0.4 task 04-01-01 and audit task 06-01-02. The matrix currently has exactly 170 rows — there is no 171-vs-170 discrepancy. The task must say "verify matrix has exactly 170 rows; append 20 (10 edge + 10 messaging); confirm post-v0.4 count = 190" — NOT "reconcile a pre-existing 171→170 discrepancy." This is a factual correction to prevent an executor from hunting for or deleting a phantom row. The orchestrator should apply this rewording before P4 (or the P4 executor should interpret "reconcile" as "verify 170 exists" rather than "find a missing row").
|
||||
|
||||
No other conditions block ship. The plan is the 4th consecutive NFR milestone using the same proven structure; all prior milestones shipped clean.
|
||||
|
||||
---
|
||||
|
||||
## Overall Verdict
|
||||
|
||||
**CONDITIONAL PROCEED** at overall confidence **0.83**.
|
||||
|
||||
The v0.4 phase-0 plan (spec → clarify → research → ideate → plan → grill) is sound: the 26-req scope is smaller than v0.3's 32, the two new domains have well-reasoned boundaries (D-061 proximity angle; D-062 cross-process failure model), the 16-doc language extension carries zero matrix-row cost, all 20 new P-rules trace to ≥1 C-rule, and the verification gates (matrix=190, Languages sub-table=16 rows, MANIFEST reconstruction, ARCHITECTURE.md component-tree test, bidirectional edge↔messaging cross-link audit) are concrete and testable.
|
||||
|
||||
The single CONDITIONAL is the **IDEATE-34 false premise** (G-014): the plan instructs executors to "reconcile a pre-existing 171-vs-170 matrix discrepancy" that direct verification shows does not exist (the matrix has exactly 170 rows). This is a factual defect in the ideation stage, not a structural flaw. It is non-blocking for P1/P2/P3 (which don't touch the matrix count) and is fixed by rewording the P4/P6 task instructions to "verify 170, append 20, confirm 190" — preventing an executor from breaking the invariant by deleting a valid row to fix a phantom discrepancy.
|
||||
|
||||
**P1 may proceed immediately.** The G-014 rewording must land before P4 executes. No escalation to human required (confidence 0.83 ≥ 0.60).
|
||||
@@ -1,154 +0,0 @@
|
||||
# Atelier v0.4 — Ideate Stage
|
||||
|
||||
> Generated during v0.4 P0 IDEATE stage. Autonomy level: `full`
|
||||
> (decision_confidence_threshold = 0.60 → auto-accept ≥ 0.6, auto-reject
|
||||
> below). Interactive mode used internally but auto-decided; no HITL.
|
||||
> Ideas continue from IDEATE-30 (last v0.3 idea). Tier 3 (cross-project)
|
||||
> skipped silently — only one project in registry (per workflow error
|
||||
> recovery). All ideas are within v0.4 scope (ATELIER-92..116 + one new
|
||||
> req ATELIER-117).
|
||||
|
||||
## Scope
|
||||
|
||||
Refine the v0.4 requirements (ATELIER-92..116) and the RESEARCH-v0.4.md
|
||||
proposed content, parallel to the v0.2 (IDEATE-01..16) and v0.3
|
||||
(IDEATE-17..30) ideation logs. Sources: git-native pattern mining
|
||||
(lessons, decisions D-001..D-072, ESC-001/002, compound solutions),
|
||||
coverage-gap analysis (pending v0.4 reqs), verification-layer inversion,
|
||||
architectural-drift detection (MANIFEST vs actual files, ARCHITECTURE.md
|
||||
vs directory tree, matrix row-count invariant), spec-driven improvement,
|
||||
backend-enriched best-practice/chaos ideation for edge/messaging/
|
||||
languages.
|
||||
|
||||
## Counts
|
||||
|
||||
- **Generated:** 13 ideas (mechanical: 8, backend-enriched: 5)
|
||||
- **Accepted:** 13 (all v0.4-scope, confidence ≥ 0.79, above 0.6 autonomy threshold → auto-accepted)
|
||||
- **Skipped:** 0
|
||||
- **New requirements added:** 1 (ATELIER-117 — ARCHITECTURE.md v0.4 update, from IDEATE-32)
|
||||
- **Refinements to existing reqs:** 12 (IDEATE-31, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43)
|
||||
- **By tier:** mechanical 8, backend-enriched 5, cross-project 0 (skipped silently — single-project registry)
|
||||
- **By category:** drift 3, coverage 3, spec 2, quality 2, chaos 1, improvement 2
|
||||
|
||||
## Ideas Table
|
||||
|
||||
| IDEATE-ID | Source | Category | Confidence | Decision | Mapped REQ |
|
||||
|-----------|--------|----------|------------|----------|------------|
|
||||
| IDEATE-31 | mechanical (MANIFEST drift, parallel to ESC-002) | drift | 0.86 | accepted → refines | ATELIER-111 (languages/ section must list 4 v0.1 stubs + 16 derived docs) |
|
||||
| IDEATE-32 | mechanical (ARCHITECTURE.md vs directory tree) | drift | 0.88 | accepted → new req | ATELIER-117 (update ARCHITECTURE.md for v0.4: component tree + v0.4 section) |
|
||||
| IDEATE-33 | mechanical (IDEATE-19 pattern transfer) | coverage | 0.90 | accepted → refines | ATELIER-107 (Core Principle Coverage C1–C8 → domains table update for edge + messaging) |
|
||||
| IDEATE-34 | mechanical (matrix row-count invariant) | drift | 0.82 | accepted → refines | ATELIER-106 (verify post-v0.4 = exactly 190; verify current = exactly 170 per grill G-014 — the "171 vs 170" claim was a false premise, corrected) |
|
||||
| IDEATE-35 | mechanical (D-069 vs req-row text) | spec | 0.89 | accepted → refines | ATELIER-100, ATELIER-101 (req text must reference streams + dead-letter comparison tables) |
|
||||
| IDEATE-36 | mechanical (RESEARCH anti-patterns vs ATELIER-110 text) | quality | 0.85 | accepted → refines | ATELIER-110 (language-derived-doc anti-patterns + .ts/.py/.go/.rs artifact-type extension) |
|
||||
| IDEATE-37 | backend-enriched (chaos anti-pattern enumeration) | chaos | 0.86 | accepted → refines | ATELIER-110 (include all 6 RESEARCH chaos anti-patterns: edge sync-loop/TTL-less/blocking-call; messaging shared-sub/blocking-consumer/unbounded-retry) |
|
||||
| IDEATE-38 | backend-enriched (IDEATE-28 pattern transfer) | improvement | 0.80 | accepted → refines | ATELIER-94 (offline-first.md maps offline write-queue patterns to testing pyramid tiers) |
|
||||
| IDEATE-39 | backend-enriched (IDEATE-29 pattern transfer) | improvement | 0.79 | accepted → refines | ATELIER-101 (delivery-semantics.md fenced idempotency-key dedup-store example, not prose-only) |
|
||||
| IDEATE-40 | backend-enriched (intra-milestone cross-link dependency) | chaos | 0.84 | accepted → refines | ATELIER-114 (P5 cross-link audit must verify edge↔messaging intra-v0.4 links resolve in both directions) |
|
||||
| IDEATE-41 | backend-enriched (IDEATE-30 pattern transfer) | improvement | 0.81 | accepted → refines | ATELIER-100 (streams.md enumerates consumer-group rebalance strategies: sticky/cooperative/eager) |
|
||||
| IDEATE-42 | mechanical (D-066 sub-table verifiability) | coverage | 0.87 | accepted → refines | ATELIER-107 (Languages Coverage sub-table = exactly 16 rows; P4 review check) |
|
||||
| IDEATE-43 | mechanical (IDEATE-18 pattern transfer) | coverage | 0.83 | accepted → refines | ATELIER-111 (MANIFEST Cross-Cutting summary = "19 domains, 190 P-rules post-v0.4") |
|
||||
|
||||
## Refinements Notes (applied to existing reqs at execute time, not changing req rows)
|
||||
|
||||
- **ATELIER-106** (IDEATE-34, corrected by grill G-014): matrix coverage summary must read
|
||||
"post-v0.4: 19 domains, 190 P-rules" in BOTH the summary block and
|
||||
the per-domain section count (IDEATE-18 carried forward). P4 must
|
||||
verify the matrix contains exactly 190 P-rows (10 per domain × 19).
|
||||
The current matrix has **exactly 170 P-rows** (17 domains × 10,
|
||||
verified by grill G-014 direct count) — the IDEATE-34 "171 vs 170"
|
||||
claim was a FALSE PREMISE. P4 must verify 170 exists, append 20 rows,
|
||||
and confirm post-v0.4 count = 190. Do NOT hunt for or delete a phantom
|
||||
171st row.
|
||||
- **ATELIER-107** (IDEATE-33, IDEATE-42): the domain-coverage.md
|
||||
extension must update THREE artifacts: (a) the per-domain row-schema
|
||||
table with edge + messaging rows; (b) the "Core Principle Coverage"
|
||||
table (C1–C8 → domains) for the 2 new domains (C4 Locality grows by
|
||||
edge P1/P8 + messaging P7; C5 Reversibility grows by edge P2/P4/P7/P10
|
||||
+ messaging P5/P8/P9; C7 Observability grows by edge P10 + messaging
|
||||
P8/P10; C8 Economy grows by edge P3/P9 + messaging P6; C2 grows by
|
||||
messaging P1/P2/P4) — parallel to v0.3 IDEATE-19; (c) the new
|
||||
`## Languages Coverage` sub-table with EXACTLY 16 rows (one per
|
||||
language-derived doc), verifiable as a P4 review check (no orphans,
|
||||
every derived doc traced to ≥1 domain P-rule).
|
||||
- **ATELIER-110** (IDEATE-36, IDEATE-37): the anti-patterns extension
|
||||
must include (a) the v0.4 deployable-artifact-types table extended to
|
||||
`.ts`/`.py`/`.go`/`.rs` standalone files (language-derived docs risk,
|
||||
per D-020 docs-only contract); (b) language-derived-doc anti-
|
||||
patterns: introducing a new P-rule (e.g., "TS P1"), orphan section
|
||||
with no domain P-rule trace, runtime standalone artifact, and
|
||||
first-principles stub gaining P1–P10 numbering; (c) all 6 RESEARCH-
|
||||
specified chaos anti-patterns, not just the 4 named in the req-row
|
||||
text — edge: sync conflict loop (P4), TTL-less cache (P6), blocking
|
||||
call on constrained device (P3/P5); messaging: shared subscription
|
||||
(P2/P3), blocking consumer (P6), unbounded retry / no DLQ (P5).
|
||||
- **ATELIER-100** (IDEATE-35, IDEATE-41): streams.md must include (a)
|
||||
the stream-platform comparison table (Kafka vs Kinesis vs
|
||||
Pulsar/NATS JetStream — ordering, partitioning, replay/retention,
|
||||
use-case fit) per D-069, AND (b) an enumeration of consumer-group
|
||||
rebalance strategies (sticky/cooperative vs eager) as a sub-
|
||||
comparison — each with partition-stop-the-world cost and use-case
|
||||
fit. Parallel to v0.3 IDEATE-30 (drift-type enumeration).
|
||||
- **ATELIER-101** (IDEATE-35, IDEATE-39): delivery-semantics.md must
|
||||
include (a) the dead-letter strategy comparison table (retry-count-
|
||||
limit vs TTL-with-backoff vs poison-queue vs DLQ+alert) per D-069,
|
||||
AND (b) a fenced idempotency-key dedup-store example (not prose-
|
||||
only) — parallel to v0.3 IDEATE-29 (fenced signed-attestation
|
||||
example). Prose-only delivery-semantics guidance is weak; a fenced
|
||||
consumer-with-dedup-store demonstrates P3 (idempotent consumers)
|
||||
concretely.
|
||||
- **ATELIER-111** (IDEATE-31, IDEATE-43): the new `## Languages`
|
||||
MANIFEST section must list the 4 v0.1 first-principles stubs
|
||||
(typescript.md, python.md, go.md, rust.md — closing the pre-existing
|
||||
drift since v0.1, parallel to ESC-002/ATELIER-91) AND the 16 new
|
||||
derived docs. The Cross-Cutting summary line must update to exactly
|
||||
"19 domains, 190 P-rules post-v0.4" (IDEATE-18 carried forward).
|
||||
- **ATELIER-114** (IDEATE-40): the P5 cross-link audit must verify
|
||||
edge↔messaging intra-v0.4 links resolve in BOTH directions, not only
|
||||
new→existing. Edge/iot.md (P1) cross-links messaging/queues.md (MQTT
|
||||
QoS parallels delivery semantics) and messaging/iot.md-style
|
||||
references — but messaging/ docs are authored in P2, so at the P1
|
||||
tag (v0.3.1) those links dangle. This is acceptable per vertical-
|
||||
slice integrity (D-053 — matrix/cross-links close in P4/P5), but the
|
||||
P5 audit (ATELIER-114) must explicitly check that every edge→
|
||||
messaging and messaging→edge link resolves by the P6 ship. The
|
||||
audit must also confirm language-derived docs (P3) link to BOTH
|
||||
new (edge/messaging) and existing domains where relevant (e.g.,
|
||||
ts-async.md → messaging/delivery-semantics; go-concurrency.md →
|
||||
concurrency/patterns + messaging/queues).
|
||||
- **ATELIER-94** (IDEATE-38): offline-first.md must map offline write-
|
||||
queue / conflict-detection patterns to the testing pyramid tiers
|
||||
(unit: conflict detection on a merge function; integration:
|
||||
reconnect reconcile against a local store; e2e: partition
|
||||
simulation with a fake network) — parallel to v0.3 IDEATE-28 (i18n
|
||||
pseudo-locale → pyramid). Avoids generic "test offline" guidance.
|
||||
|
||||
## Within-Project Pattern Transfer (v0.1 → v0.2 → v0.3 → v0.4)
|
||||
|
||||
| Prior Lesson | v0.4 Application | Status |
|
||||
|--------------|------------------|--------|
|
||||
| IDEATE-10/11/12 + 21/22/23 (decision/comparison tables in multi-tool derived docs) | D-069 + IDEATE-35 transfer the pattern to streams.md (Kafka/Kinesis/Pulsar) + delivery-semantics.md (dead-letter strategies) | ✓ transferred |
|
||||
| IDEATE-17 (manifest drift closure — examples/ unlisted → ATELIER-91) | IDEATE-31 → ATELIER-111 (languages/ unlisted since v0.1 — closes pre-existing drift on the same ESC-002 precedent) | ✓ transferred |
|
||||
| IDEATE-18 (coverage summary totals must update — string + per-domain count) | IDEATE-43 → ATELIER-111 (Cross-Cutting = "19 domains, 190 P-rules post-v0.4") + IDEATE-34 → ATELIER-106 (summary block) | ✓ transferred |
|
||||
| IDEATE-19 (Core Principle Coverage C1–C8 → domains table update) | IDEATE-33 → ATELIER-107 (C-rule coverage table extended for edge + messaging with the documented C-count shifts) | ✓ transferred |
|
||||
| IDEATE-20 (anti-patterns specificity: domain violations + artifact types) | IDEATE-36 + IDEATE-37 → ATELIER-110 (language-derived anti-patterns + .ts/.py/.go/.rs artifact types + all 6 chaos anti-patterns) | ✓ extended |
|
||||
| IDEATE-07 (named bad examples cite the P-rule breached) | ATELIER-113 + D-068 already cover (single-breach, cite specific P-rule) | ✓ carried forward |
|
||||
| IDEATE-08 (cross-link verification: every new derived doc ≥1 outbound) | ATELIER-114 already covers; IDEATE-40 extends to edge↔messaging intra-milestone links | ✓ extended |
|
||||
| IDEATE-13/14 (chaos: orphan P-rule + deployable artifact) | IDEATE-36 → ATELIER-110 (language orphan-section + standalone .ts/.py/.go/.rs runtime artifact) | ✓ extended |
|
||||
| IDEATE-28 (pseudo-locale testing → testing pyramid tiers) | IDEATE-38 → ATELIER-94 (offline write-queue testing → pyramid tiers) | ✓ transferred |
|
||||
| IDEATE-29 (fenced signed-attestation example, not prose-only) | IDEATE-39 → ATELIER-101 (fenced idempotency-key dedup-store example) | ✓ transferred |
|
||||
| IDEATE-30 (drift-type enumeration: data/concept/prediction + signal each) | IDEATE-41 → ATELIER-100 (rebalance-strategy enumeration: sticky/cooperative/eager + cost each) | ✓ transferred |
|
||||
|
||||
## Summary
|
||||
|
||||
- 13 ideas generated, 13 accepted (confidence ≥ 0.79), 0 skipped.
|
||||
- 1 new requirement: ATELIER-117 (ARCHITECTURE.md v0.4 update —
|
||||
coverage gap; no existing req covers the architecture-tree drift).
|
||||
- 12 refinements baked into ATELIER-94, 100, 101, 106, 107, 110, 111,
|
||||
114 — to be honored by the PLAN stage as task must-haves (parallel
|
||||
to v0.3's Refinements Notes → Task Bake-In Map).
|
||||
- The within-project pattern-transfer convention (decision matrices,
|
||||
manifest-drift closure, coverage-summary totals, C-rule coverage
|
||||
table, chaos anti-patterns, fenced examples, enumerations, pyramid
|
||||
mappings) continues from v0.2→v0.3→v0.4 unchanged.
|
||||
- Tier 3 (cross-project) skipped silently: single-project registry.
|
||||
- No idea fell below the 0.60 autonomy threshold → no HITL escalation.
|
||||
@@ -2,10 +2,7 @@
|
||||
|
||||
> Generated during P0 RESEARCH. Atelier is a documentation-only project;
|
||||
> the default runtime persona roster is deactivated. Custom personas
|
||||
> reflect the docs-as-code nature of the work. Updated per milestone:
|
||||
> v0.4 adds `edge-engineer` (P1) and `languages-engineer` (P3) phase-
|
||||
> specific personas (D-067); messaging (P2) is authored by tech-writer
|
||||
> + domain-expert with edge-engineer consultation.
|
||||
> reflect the docs-as-code nature of the work.
|
||||
|
||||
## Active Personas
|
||||
|
||||
@@ -23,7 +20,7 @@
|
||||
- **frameworks:** []
|
||||
- **constraints:** ["markdown-first", "examples are mandatory", "why over what", "audience awareness"]
|
||||
- **territory:** ["**/*.md", "LICENSE", "CHANGELOG.md", "CONTRIBUTING.md"]
|
||||
- **reason:** Primary author of all framework markdown content. Owns docs-as-code discipline (Documentation P1). Excludes `.ciagent/**` governance files (owned by lead-developer). For v0.4: co-authors the messaging domain (P2) with domain-expert, per D-067.
|
||||
- **reason:** Primary author of all framework markdown content. Owns docs-as-code discipline (Documentation P1). Excludes `.ciagent/**` governance files (owned by lead-developer).
|
||||
|
||||
### domain-expert
|
||||
- **active:** true
|
||||
@@ -31,7 +28,7 @@
|
||||
- **frameworks:** []
|
||||
- **constraints:** ["every principle traces to core", "10 principles per domain", "no orphans in matrix"]
|
||||
- **territory:** ["core/**", "domains/**", "matrix/**", "review/**", "examples/**"]
|
||||
- **reason:** Validates domain principle correctness and traceability to core principles. Ensures the matrix is complete and no domain rule is orphaned. For v0.4: co-authors the messaging domain (P2) with tech-writer, consulting edge-engineer for the D-062 boundary, per D-067.
|
||||
- **reason:** Validates domain principle correctness and traceability to core principles. Ensures the matrix is complete and no domain rule is orphaned.
|
||||
|
||||
## Inactive Personas (from default roster)
|
||||
|
||||
@@ -49,45 +46,25 @@
|
||||
|
||||
## Phase-Specific Personas
|
||||
|
||||
### edge-engineer (v0.4 — REMOVE after milestone completion)
|
||||
### platform-engineer (v0.3 — extended, active for v0.3, removed after milestone completion)
|
||||
|
||||
- **active:** true
|
||||
- **phase_specific:** true
|
||||
- **domain:** edge computing
|
||||
- **frameworks:** []
|
||||
- **constraints:** ["latency-bound first", "resource-constrained examples", "trace to core", "10 P-rules per domain", "no runtime code", "proximity is the distinguishing constraint (D-061)", "offline-capability is non-negotiable"]
|
||||
- **territory:** ["domains/edge/**", "examples/good/edge-offline-sync.md", "examples/bad/edge-sync-loop.md"]
|
||||
- **reason:** Per D-067: edge domain authoring (cdn, offline-first, iot, sync) benefits from a specialist persona with the proximity-is-correctness and offline-is-non-negotiable constraints. The distinguishing angle is proximity/location (D-061) — distinct from performance (generic), concurrency (in-process), and observability (generic telemetry). Active for v0.4 P1 (Edge Domain) only; consulted during P2 (Messaging) for the D-062 in-process-vs-cross-system boundary; removed after v0.4 completes; roster returns to 3 active personas.
|
||||
|
||||
### languages-engineer (v0.4 — REMOVE after milestone completion)
|
||||
|
||||
- **active:** true
|
||||
- **phase_specific:** true
|
||||
- **domain:** language-specific application
|
||||
- **frameworks:** []
|
||||
- **constraints:** ["trace to existing domain P-rules (D-063, D-066) — no new matrix rows", "per-language idioms", "fenced code examples in-language", "10 P-rules per domain does NOT apply to language docs", "manifest gains Languages section (D-065)"]
|
||||
- **territory:** ["languages/**"]
|
||||
- **reason:** Per D-067: language-derived extension authoring (16 derived docs across TypeScript, Python, Go, Rust) benefits from a specialist persona enforcing the apply-don't-derive discipline (D-063) and the no-new-matrix-rows invariant (D-066). Languages are an application lens, not a principle source — the existing stubs' section-header-trace pattern is preserved, no P1–P10 numbering is introduced. Active for v0.4 P3 (Language-Derived Extension) only; removed after v0.4 completes; roster returns to 3 active personas.
|
||||
|
||||
### platform-engineer (v0.3 — REMOVED after milestone completion)
|
||||
|
||||
- **active:** false
|
||||
- **phase_specific:** true
|
||||
- **domain:** infrastructure/platform-automation
|
||||
- **frameworks:** []
|
||||
- **constraints:** ["declarative-first", "stateless examples", "trace to core", "10 P-rules per domain", "no runtime code", "source-of-truth is git", "reconciliation loop is the primitive"]
|
||||
- **territory:** ["domains/gitops-operators/**", "examples/good/gitops-pr.md", "examples/bad/* (gitops-related)"]
|
||||
- **reason:** Per D-019 / D-027: the v0.2 platform-engineer persona was reused and extended for P1 (gitops-operators). Removed after v0.3 completed; roster returns to 3 active personas. The v0.3 gitops-operators content remains owned by tech-writer + domain-expert for cross-link maintenance. v0.4: still inactive.
|
||||
- **reason:** Per D-019 / D-027: the v0.2 platform-engineer persona is reused and extended for P1 (gitops-operators), because GitOps/Operators/Progressive Delivery build directly on the k8s + IaC declarative-reconciliation model the persona already embodies. Removed after v0.3 completes; roster returns to 3 active personas. The v0.2 IaC/k8s content remains owned by tech-writer + domain-expert for cross-link maintenance.
|
||||
|
||||
### ml-engineer (v0.3 — REMOVED after milestone completion)
|
||||
### ml-engineer (v0.3 — phase-specific, removed after milestone completion)
|
||||
|
||||
- **active:** false
|
||||
- **active:** true
|
||||
- **phase_specific:** true
|
||||
- **domain:** machine-learning engineering
|
||||
- **frameworks:** []
|
||||
- **constraints:** ["reproducibility is non-negotiable", "data lineage is traceable", "trace to core", "10 P-rules per domain", "no runtime code", "engineering discipline not algorithm design (D-023)", "examples are illustrative markdown only"]
|
||||
- **territory:** ["domains/ai-ml/**", "examples/good/ai-ml-reproducibility.md"]
|
||||
- **reason:** Per D-019 / D-020: AI/ML domain authoring (data versioning, model evaluation, serving, monitoring/drift) benefited from a specialist persona with reproducibility and data-lineage constraints. Scope was engineering discipline, NOT algorithm/model design (D-023). Removed after v0.3 completed; roster returns to 3 active personas. v0.4: still inactive.
|
||||
- **reason:** Per D-019 / D-020: AI/ML domain authoring (data versioning, model evaluation, serving, monitoring/drift) benefits from a specialist persona with reproducibility and data-lineage constraints the existing tech-writer persona lacks. Scope is engineering discipline, NOT algorithm/model design (D-023). Removed after v0.3 completes; roster returns to 3 active personas.
|
||||
|
||||
### platform-engineer (v0.2 — REMOVED after milestone completion)
|
||||
|
||||
@@ -97,7 +74,7 @@
|
||||
- **frameworks:** []
|
||||
- **constraints:** ["declarative-first", "stateless examples", "trace to core", "10 P-rules per domain", "no runtime code"]
|
||||
- **territory:** ["domains/infrastructure-as-code/**", "domains/kubernetes/**", "examples/good/terraform-module.md", "examples/good/k8s-deployment.md", "examples/bad/*"]
|
||||
- **reason:** Specialist authoring for IaC/k8s domain content (terraform, opentofu, state, modules, k8s workloads/networking/storage/rbac/helm/kustomize) where the existing tech-writer persona lacks the domain expertise. Was active for v0.2 P1–P4 only; removed after milestone v0.2 completed (per D-027, D-014). Roster returns to 3 active personas. v0.4: still inactive.
|
||||
- **reason:** Specialist authoring for IaC/k8s domain content (terraform, opentofu, state, modules, k8s workloads/networking/storage/rbac/helm/kustomize) where the existing tech-writer persona lacks the domain expertise. Was active for v0.2 P1–P4 only; removed after milestone v0.2 completed (per D-027, D-014). Roster returns to 3 active personas.
|
||||
|
||||
## Territory Enforcement
|
||||
|
||||
@@ -105,11 +82,9 @@ Mode: `warn` (per config.json `personas.territory_enforcement`).
|
||||
|
||||
At `warn`, territory violations are logged but not blocked. This is appropriate for a docs project where tech-writer may touch `.ciagent/` files incidentally (e.g., updating ROADMAP status). Strict mode would be appropriate once territories stabilize.
|
||||
|
||||
## v0.4 Persona Roster Summary (active during milestone)
|
||||
## v0.3 Persona Roster Summary
|
||||
|
||||
Active personas (3, span full milestone): lead-developer, tech-writer, domain-expert.
|
||||
Phase-specific personas (2, active during milestone): edge-engineer (P1 Edge Domain), languages-engineer (P3 Language-Derived Extension) — both removed after v0.4 completion per D-067.
|
||||
Messaging (P2) is authored by tech-writer + domain-expert (the permanent roster) with edge-engineer consultation for the distributed-systems/async overlap (D-062 boundary), per D-067. The edge-engineer's distributed-systems lens survives the P1→P2 persona transition via a task must-have baked into the plan (the D-051 constraints-baked-into-task pattern from v0.3): "P1/P2 demonstrated edge + messaging boundary (D-062); delivery-semantics comparison table required (D-069)."
|
||||
Inactive default personas (3, unchanged): data-engineer, backend-engineer, frontend-engineer.
|
||||
Historical phase-specific personas (3, inactive): platform-engineer (v0.2), platform-engineer (v0.3 extended), ml-engineer (v0.3) — retained as historical record; "v0.4: still inactive" appended to their reasons.
|
||||
Peak active roster during v0.4: 5 (3 permanent + 2 phase-specific), matching v0.3's peak and avoiding persona-explosion risk. Roster returns to 3 active personas after v0.4 completion.
|
||||
Active personas for v0.3 (5): lead-developer, tech-writer, domain-expert (span full milestone), platform-engineer (phase-specific, P1 gitops-operators), ml-engineer (phase-specific, P2 ai-ml).
|
||||
- **Phase assignment:** P1 GitOps/Operators → platform-engineer; P2 AI/ML → ml-engineer; P3 i18n + Compliance → tech-writer + domain-expert (D-022); P4–P5 matrix/review/examples → tech-writer + domain-expert + lead-developer.
|
||||
Inactive personas (3, unchanged): data-engineer, backend-engineer, frontend-engineer.
|
||||
Post-v0.3: platform-engineer + ml-engineer removed; roster returns to 3 active personas (lead-developer, tech-writer, domain-expert).
|
||||
@@ -1,285 +0,0 @@
|
||||
# Atelier — Plan (v0.4)
|
||||
|
||||
> Vertical-slice plans with wave ordering for milestone v0.4 (Edge + Messaging + Language-Derived Docs). Plans reference REQ-IDs from `.ciagent/atelier/REQUIREMENTS.md` (ATELIER-92..117). NFR milestone — all phases produce docs; no `feat` code. Per `parallelization.max_concurrent_agents = 5`, wave parallelism is capped at 5 concurrent tasks; waves larger than 5 are split into sub-waves. Grounded in `RESEARCH-v0.4.md` (proposed P-rules, derived-doc outlines, anti-patterns, examples, matrix rows) and decisions D-054..D-073.
|
||||
|
||||
## Plan-Level Invariants
|
||||
|
||||
- **Vertical-slice integrity (D-053 carried forward):** each phase is independently shippable. P1 ships the edge domain self-consistent (matrix rows land in P4 — the domain is internally consistent; matrix extension is traceability closure, not a blocker). P2 ships messaging self-consistent. P3 ships the 16 language-derived docs self-consistent. P4 closes traceability + manifest + architecture. P5 closes examples + cross-links. P6 ships the release. Edge↔messaging intra-milestone cross-links may dangle at the P1 tag (v0.3.1) because messaging is authored in P2; this is acceptable per D-053 — the P5 audit (ATELIER-114) verifies both directions resolve by the P6 ship (IDEATE-40).
|
||||
- **Tag sequence:** v0.3.0 (P0) → v0.3.1 (P1) → v0.3.2 (P2) → v0.3.3 (P3) → v0.3.4 (P4) → v0.3.5 (P5) → v0.3.6 (P6 = milestone release). NFR milestone: no separate minor tag; the final patch (v0.3.6) IS the v0.4 deliverable.
|
||||
- **Persona lifecycle:** edge-engineer + languages-engineer active P1–P5, review their authored content in P6 Wave 1 (D-052 precedent), removed in P6 Wave 2 (roster returns to 3 active). Messaging (P2) authored by tech-writer + domain-expert with edge-engineer consultation (D-067); the edge-engineer's distributed-systems lens survives the P1→P2 transition via a task must-have baked into P2 tasks (D-051 constraints-baked-into-task pattern). Peak active roster during v0.4: 5 (3 permanent + 2 phase-specific).
|
||||
- **Refinements are execute-time must-haves:** all 12 IDEATE-v0.4.md refinements (IDEATE-31, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43) and the refinement notes in REQUIREMENTS.md are baked into the relevant task Must-have columns — they are NOT separate tasks. See the "v0.4 Ideation Refinements → Task Bake-In Map" at the end of this file.
|
||||
- **Cross-link rule (IDEATE-08 carried forward):** every new derived doc has ≥1 outbound cross-link to a MANIFEST-listed doc. The P5 audit (ATELIER-114) verifies this AND edge↔messaging intra-v0.4 links resolve in both directions (IDEATE-40) AND language-derived docs link to both new (edge/messaging) and existing domains where relevant.
|
||||
- **Anti-pattern pre-specification (RESEARCH-v0.4 §Review Integration):** 6 chaos anti-patterns are pre-specified to reject on sight — edge: sync-conflict-loop (P4), TTL-less-cache (P6), blocking-call-on-constrained-device (P3/P5); messaging: shared-subscription (P2/P3), blocking-consumer (P6), unbounded-retry (P5). These land in `review/anti-patterns.md` in P4; the 2 named bad examples that demonstrate them (edge-sync-loop, messaging-shared-subscription) land in P5.
|
||||
- **No runtime code (D-020 carried forward):** all examples and fenced code are illustrative markdown only. The v0.4 deployable-artifact-types table extends to `.ts`/`.py`/`.go`/`.rs` standalone files (forbidden in language-derived docs).
|
||||
|
||||
## Phase 0 — Pre-Execution (COMPLETE)
|
||||
|
||||
Stages: SPECIFY ✓ → CLARIFY ✓ → RESEARCH ✓ → IDEATE ✓ → PLAN ✓ → GRILL → SHIP
|
||||
Branch: `phase/00-pre-execution` (from `milestone/v0.4-edge-quantum-langs`)
|
||||
Tag: v0.3.0
|
||||
|
||||
## Phase 1 — Edge Domain
|
||||
|
||||
**Goal:** Author the `domains/edge/` tree — 10 first principles (P1–P10) plus 4 derived docs (cdn, offline-first, iot, sync). Edge owns the proximity/location angle (D-061): compute near the user/data source, offline as a first-class state, constrained resources declared, sync conflicts bounded, partition-survivable telemetry. Each P-rule derives from core C1–C8 (matrix extension lands in P4).
|
||||
**Branch:** `phase/01-edge` (from `milestone/v0.4-edge-quantum-langs`)
|
||||
**Personas:** edge-engineer (author), domain-expert (validate traceability), tech-writer (style/format)
|
||||
**Tag:** v0.3.1
|
||||
**Requirements:** ATELIER-92, ATELIER-93, ATELIER-94, ATELIER-95, ATELIER-96
|
||||
|
||||
### Wave 1 (sequential — first-principles must exist before derived docs)
|
||||
| Task | File | Persona | REQ-ID | Must-have |
|
||||
|------|------|---------|--------|-----------|
|
||||
| 01-01-01 | `domains/edge/first-principles.md` | edge-engineer | ATELIER-92 | 10 P-rules P1–P10 per RESEARCH-v0.4 §Edge (P1 Proximity is the Design Driver, P2 Offline is a First-Class State, P3 Resources are Constrained and Declared, P4 Sync Conflicts are Bounded Not Infinite, P5 Edge Operations are Idempotent, P6 Cache Invalidation is Explicit, P7 Partial Degradation is Engineered, P8 Geographic Distribution is a First-Class Constraint, P9 Identity is Constrained at the Edge, P10 Edge Observability Survives Partition); each names the core C-rule(s) it derives from; each has definition + "what violates"; broad-derivation domain (7 C-rules: C1, C3, C4, C5, C6, C7, C8); edge-engineer constraints enforced (proximity is the distinguishing constraint per D-061, offline-capability is non-negotiable, latency is a correctness constraint) |
|
||||
|
||||
### Wave 2 (parallel — 4 derived docs, independent; ≤5 concurrent)
|
||||
| Task | File | Persona | REQ-ID | Must-have |
|
||||
|------|------|---------|--------|-----------|
|
||||
| 01-02-01 | `domains/edge/cdn.md` | edge-engineer | ATELIER-93 | CDN/edge-cache placement + invalidation per RESEARCH-v0.4 §Edge (cache key design, TTL vs explicit invalidation, cache-hit/miss/origin-fetch, origin shielding, purge strategies URL vs soft vs surrogate-key, cache poisoning prevention, multi-CDN routing P8); **decision matrix required (D-069): edge-cache vs origin — when to serve from the pop, when to fetch fresh, when to shield the origin**; cross-link `performance/frontend` (generic caching), `security/input-validation` (cache poisoning), `observability/metrics` (cache-hit ratio); ≥1 outbound cross-link to a MANIFEST-listed doc (IDEATE-08) |
|
||||
| 01-02-02 | `domains/edge/offline-first.md` | edge-engineer | ATELIER-94 | Offline as a first-class state (P2), local-first storage (IndexedDB/SQLite/on-device), queue-and-forward for writes, conflict detection on reconnect, UI for offline state, service workers per RESEARCH-v0.4 §Edge; **offline write-queue / conflict-detection patterns mapped to testing pyramid tiers (IDEATE-38, ATELIER-94 refinement): unit (conflict detection on a merge function), integration (reconnect reconcile against a local store), e2e (partition simulation with a fake network)**; fenced code: a service-worker cache strategy + an offline-write-queue sketch; cross-link `concurrency/patterns` (bounded buffer = in-process analog), `observability/logging` (local-first logging); ≥1 outbound cross-link (IDEATE-08) |
|
||||
| 01-02-03 | `domains/edge/iot.md` | edge-engineer | ATELIER-95 | Constrained devices at the edge per RESEARCH-v0.4 §Edge: device resource classes (P3), constrained protocols (MQTT, CoAP), device identity + provisioning (P9), telemetry from devices (P10), command idempotency (P5), partial-degradation when devices drop (P7); fenced code: an MQTT publish/subscribe payload with QoS levels + a device-provisioning manifest; cross-link `security/secrets` (device credentials), `messaging/queues` (MQTT QoS parallels delivery semantics — one-directional outward; this link dangles until P2, acceptable per D-053, verified bidirectional in P5 ATELIER-114 per IDEATE-40); ≥1 outbound cross-link (IDEATE-08) |
|
||||
| 01-02-04 | `domains/edge/sync.md` | edge-engineer | ATELIER-96 | Reconciling divergent state across partitions per RESEARCH-v0.4 §Edge: the sync problem (P4), conflict resolution strategies, CRDTs (state-based + operation-based), last-write-win (LWW) with vector clocks/timestamps, merge semantics, conflict-free vs conflict-tolerant data types; **decision matrix required (D-069): CRDT vs last-write-win — when each applies, the correctness cost, the operational cost**; fenced code: a CRDT register (LWW-element-set) + an LWW-with-vector-clock merge; cross-link `data/migrations` (schema migration under sync), `concurrency/patterns` (immutability aids merge); ≥1 outbound cross-link (IDEATE-08) |
|
||||
|
||||
**Verify (P1):**
|
||||
- Structural: 5 files exist under `domains/edge/`
|
||||
- Behavioral: every P1–P10 in first-principles names ≥1 C-rule (domain-expert sign-off); broad-derivation 7 C-rules (C1, C3, C4, C5, C6, C7, C8) confirmed
|
||||
- Security: P9 (Identity is Constrained at the Edge) + P6 (Cache Invalidation is Explicit) sections present
|
||||
- Quality: each derived doc has ≥1 outbound cross-link to a MANIFEST-listed doc (IDEATE-08); cdn.md carries the edge-cache-vs-origin decision matrix (D-069); sync.md carries the CRDT-vs-LWW decision matrix (D-069); offline-first.md maps offline write-queue to testing pyramid tiers (IDEATE-38); no duplication of performance/concurrency/observability P-rules (D-061 boundary — proximity/location angle only)
|
||||
|
||||
## Phase 2 — Messaging Domain
|
||||
|
||||
**Goal:** Author the `domains/messaging/` tree — 10 first principles (P1–P10) plus 4 derived docs (queues, pubsub, streams, delivery-semantics). Messaging owns the cross-process / network-failure-model angle (D-062): messages as contracts, explicit delivery semantics, idempotent consumers, bounded backpressure, observable lag. Concurrency owns the in-process analog; cross-link is one-directional outward. Each P-rule derives from core C1–C8 (matrix extension lands in P4).
|
||||
**Branch:** `phase/02-messaging` (from `milestone/v0.4-edge-quantum-langs`)
|
||||
**Personas:** tech-writer (author), domain-expert (author + validate traceability), edge-engineer (consult — D-067 distributed-systems/async overlap)
|
||||
**Tag:** v0.3.2
|
||||
**Requirements:** ATELIER-97, ATELIER-98, ATELIER-99, ATELIER-100, ATELIER-101
|
||||
|
||||
### Wave 1 (sequential — first-principles first)
|
||||
| Task | File | Persona | REQ-ID | Must-have |
|
||||
|------|------|---------|--------|-----------|
|
||||
| 02-01-01 | `domains/messaging/first-principles.md` | tech-writer + domain-expert (edge-engineer consult) | ATELIER-97 | 10 P-rules P1–P10 per RESEARCH-v0.4 §Messaging (P1 Messages are Contracts, P2 Ordering is a Property Not an Assumption, P3 Consumers are Idempotent, P4 Delivery Semantics are Explicit, P5 Dead-Letter Handling is Defined, P6 Backpressure is Bounded, P7 Partitioning is Intentional, P8 Replay and Retention are Configured, P9 Schemas Evolve Compatibly, P10 Messaging is Observable); each names the core C-rule(s) it derives from; each has definition + "what violates"; broad-derivation domain (7 C-rules: C1, C2, C4, C5, C6, C7, C8); **edge-engineer consultation baked in (D-067, D-051 pattern): P1/P2 demonstrated edge + messaging boundary (D-062 — cross-process/network-failure-model, not in-process concurrency); no duplication of concurrency P-rules** |
|
||||
|
||||
### Wave 2 (parallel — 4 derived docs, independent; ≤5 concurrent)
|
||||
| Task | File | Persona | REQ-ID | Must-have |
|
||||
|------|------|---------|--------|-----------|
|
||||
| 02-02-01 | `domains/messaging/queues.md` | tech-writer + domain-expert | ATELIER-98 | Point-to-point async delivery via queues per RESEARCH-v0.4 §Messaging: producer/consumer model, ack/nack, visibility timeouts, redelivery, FIFO vs standard queues, prefetch + concurrency, long polling; **comparison table required (D-069): at-least-once vs at-most-once vs exactly-once — semantics, latency cost, implementation cost, when each fits**; fenced code: a producer/consumer pair with idempotency key + a redelivery+DLQ flow; cross-link `concurrency/patterns` (in-process bounded queue — one-directional outward), `observability/metrics` (lag); ≥1 outbound cross-link (IDEATE-08) |
|
||||
| 02-02-02 | `domains/messaging/pubsub.md` | tech-writer + domain-expert | ATELIER-99 | Fan-out via publish/subscribe per RESEARCH-v0.4 §Messaging: topic/subscription model, fan-out semantics, shared vs independent subscriptions (the `messaging-shared-subscription` chaos anti-pattern lives here — pre-specified in P4 ATELIER-110), filtering, ordering across subscriptions; fenced code: a publish + two independent subscriptions; cross-link `messaging/streams` (pub/sub vs stream — durability boundary), `observability/metrics` (per-subscription lag); ≥1 outbound cross-link (IDEATE-08) |
|
||||
| 02-02-03 | `domains/messaging/streams.md` | tech-writer + domain-expert | ATELIER-100 | Durable-log messaging + stream processing per RESEARCH-v0.4 §Messaging: partitioned log model, offsets, consumer groups, replay, retention windows, stream processing (windowing, joins, exactly-once via transactions); **stream-platform comparison table required (D-069): Kafka vs Kinesis vs Pulsar/NATS JetStream — ordering guarantees, partitioning model, replay/retention, use-case fit**; **consumer-group rebalance strategy enumeration (IDEATE-41, ATELIER-100 refinement): sticky/cooperative vs eager — each with partition-stop-the-world cost + use-case fit (parallel to v0.3 IDEATE-30 drift-type enumeration)**; fenced code: a consumer-group reading from offsets + a transactional exactly-once producer; cross-link `messaging/delivery-semantics` (exactly-once via transactions), `data/schema-design` (stream schema), `observability/tracing` (cross-partition traces); ≥1 outbound cross-link (IDEATE-08) |
|
||||
| 02-02-04 | `domains/messaging/delivery-semantics.md` | tech-writer + domain-expert | ATELIER-101 | Correctness properties of delivery per RESEARCH-v0.4 §Messaging: the three semantics (P4), idempotency (P3), ordering (P2), dead-letter strategies (P5), retry budgets + backoff, poison messages; **dead-letter strategy comparison table required (D-069): retry-count-limit vs TTL-with-backoff vs poison-queue vs DLQ+alert — when each applies, failure visibility, operational cost**; **fenced idempotency-key dedup-store example (IDEATE-39, ATELIER-101 refinement): not prose-only — a fenced consumer-with-dedup-store demonstrates P3 concretely (parallel to v0.3 IDEATE-29 fenced signed-attestation)**; fenced code: an idempotent consumer with a dedup store + a DLQ routing rule; cross-link `concurrency/patterns` (retry/backoff in-process analog), `errors/patterns` (errors as data for message failures), `observability/metrics` (DLQ depth as an alert); ≥1 outbound cross-link (IDEATE-08) |
|
||||
|
||||
**Verify (P2):**
|
||||
- Structural: 5 files exist under `domains/messaging/`
|
||||
- Behavioral: every P1–P10 traces to ≥1 C-rule (domain-expert sign-off); broad-derivation 7 C-rules (C1, C2, C4, C5, C6, C7, C8) confirmed
|
||||
- Security: P1 (Messages are Contracts) + P5 (Dead-Letter Handling is Defined) sections present
|
||||
- Quality: each derived doc ≥1 outbound cross-link (IDEATE-08); queues.md carries the delivery-semantics comparison table (D-069); streams.md carries the stream-platform comparison table (D-069) AND the rebalance-strategy enumeration (IDEATE-41); delivery-semantics.md carries the dead-letter strategy comparison table (D-069) AND a fenced idempotency-key dedup-store example (IDEATE-39); no duplication of concurrency/patterns.md P-rules (D-062 boundary — cross-process/network-failure-model only); edge↔messaging cross-links resolve bidirectionally post-P2 (verified in P5 ATELIER-114 per IDEATE-40)
|
||||
|
||||
## Phase 3 — Language-Derived Extension
|
||||
|
||||
**Goal:** Extend each `languages/*.md` from a first-principles-only stub to a full derived-doc tree. 4 languages × (extend stub + 4 derived docs) = 16 new derived docs. Language-derived docs introduce NO new P-rules (D-063) and NO new matrix rows (D-066); each section traces to existing domain P-rules via the section-header-trace pattern. The first-principles stub is extended with a pointer to its derived docs but retains its section structure — no P1–P10 numbering (D-063).
|
||||
**Branch:** `phase/03-languages-derived` (from `milestone/v0.4-edge-quantum-langs`)
|
||||
**Personas:** languages-engineer (author), domain-expert (validate traceability)
|
||||
**Tag:** v0.3.3
|
||||
**Requirements:** ATELIER-102, ATELIER-103, ATELIER-104, ATELIER-105
|
||||
|
||||
### Wave 1 (parallel — 4 first-principles stubs extended, independent; ≤5 concurrent → 4 tasks)
|
||||
| Task | File | Persona | REQ-ID | Must-have |
|
||||
|------|------|---------|--------|-----------|
|
||||
| 03-01-01 | `languages/typescript.md` (extend stub) | languages-engineer | ATELIER-102 | Extend existing first-principles stub with a pointer to its 4 derived docs (ts-types, ts-tooling, ts-async, ts-testing); **retain existing section structure — NO P1–P10 numbering added (D-063)**; existing section-header-trace pattern preserved (e.g., `## Type System (C1 Correctness, Data P7 Type Fidelity)`); languages-engineer constraints enforced (apply don't derive, no new P-rules, no new matrix rows per D-063/D-066) |
|
||||
| 03-01-02 | `languages/python.md` (extend stub) | languages-engineer | ATELIER-103 | Extend existing first-principles stub with a pointer to its 4 derived docs (py-types, py-tooling, py-async, py-testing); retain existing section structure — NO P1–P10 numbering (D-063); section-header-trace pattern preserved |
|
||||
| 03-01-03 | `languages/go.md` (extend stub) | languages-engineer | ATELIER-104 | Extend existing first-principles stub with a pointer to its 4 derived docs (go-types, go-tooling, go-concurrency, go-testing); retain existing section structure — NO P1–P10 numbering (D-063); section-header-trace pattern preserved |
|
||||
| 03-01-04 | `languages/rust.md` (extend stub) | languages-engineer | ATELIER-105 | Extend existing first-principles stub with a pointer to its 4 derived docs (rs-ownership, rs-tooling, rs-async, rs-testing); retain existing section structure — NO P1–P10 numbering (D-063); section-header-trace pattern preserved |
|
||||
|
||||
### Wave 2 (parallel — 16 derived docs, grouped by language; ≤5 concurrent → executor schedules 5-then-5-then-5-then-1)
|
||||
|
||||
> **Parallelism note:** 16 independent tasks all depend only on Wave 1. The executor schedules at most 5 concurrently per `parallelization.max_concurrent_agents`. The 2a/2b/2c/2d labels organize by language, not a hard sequencing barrier — all sub-waves are in the same dependency tier.
|
||||
|
||||
**Wave 2a — TypeScript (4 derived docs, ≤5 concurrent)**
|
||||
| Task | File | Persona | REQ-ID | Must-have |
|
||||
|------|------|---------|--------|-----------|
|
||||
| 03-02a-01 | `languages/ts-types.md` | languages-engineer | ATELIER-102 | TypeScript type system per RESEARCH-v0.4 §Language-Derived: nominal vs structural typing (branded types), generics, narrowing, utility types, discriminated unions; each section header names ≥1 traced domain P-rule AND core C-rule inline (e.g., `## Nominal vs Structural Typing (C1 Correctness, Data P7 Type Fidelity, API P1 Contract Fidelity)`); fenced TS code in-language (illustrative only, no standalone .ts — D-020); traces to Data P7, API P1, Errors P1, Concurrency P1; ≥1 outbound cross-link to a traced domain doc (IDEATE-08; per IDEATE-40 also cross-link new edge/messaging where relevant, e.g., ts-types → messaging/delivery-semantics if schema-fidelity applies) |
|
||||
| 03-02a-02 | `languages/ts-tooling.md` | languages-engineer | ATELIER-102 | TypeScript tooling per RESEARCH-v0.4 §Language-Derived: tsc, ESLint, ts-jest, project references, tsconfig discipline; section headers trace to DevOps P2, DevOps P1, Documentation P1; fenced TS code in-language; ≥1 outbound cross-link (IDEATE-08) |
|
||||
| 03-02a-03 | `languages/ts-async.md` | languages-engineer | ATELIER-102 | TypeScript async per RESEARCH-v0.4 §Language-Derived: Promises + AbortSignal, async/await, error handling, cancellation; section headers trace to Concurrency P7, P8, P9, Errors P5; fenced TS code in-language; cross-link `concurrency/patterns` + `messaging/delivery-semantics` (per IDEATE-40 — ts-async → messaging/delivery-semantics where async-retry parallels apply); ≥1 outbound cross-link (IDEATE-08) |
|
||||
| 03-02a-04 | `languages/ts-testing.md` | languages-engineer | ATELIER-102 | TypeScript testing per RESEARCH-v0.4 §Language-Derived: Vitest/Jest, mock discipline, type-level tests; section headers trace to Testing P1, P2, P3, P9; fenced TS code in-language; cross-link `testing/pyramid` + `testing/fixtures`; ≥1 outbound cross-link (IDEATE-08) |
|
||||
|
||||
**Wave 2b — Python (4 derived docs, ≤5 concurrent; runs in parallel with 2a — total 8 tasks, capped at 5 → executor schedules 5 then 3)**
|
||||
| Task | File | Persona | REQ-ID | Must-have |
|
||||
|------|------|---------|--------|-----------|
|
||||
| 03-02b-01 | `languages/py-types.md` | languages-engineer | ATELIER-103 | Python type system per RESEARCH-v0.4 §Language-Derived: type hints + Pydantic, mypy/pyright, gradual typing; section headers trace to Data P7, Errors P1, API P1; fenced Python code in-language (no standalone .py — D-020); ≥1 outbound cross-link (IDEATE-08) |
|
||||
| 03-02b-02 | `languages/py-tooling.md` | languages-engineer | ATELIER-103 | Python tooling per RESEARCH-v0.4 §Language-Derived: ruff, mypy, poetry, uv, virtualenv discipline; section headers trace to DevOps P2, Documentation P1; fenced Python code in-language; ≥1 outbound cross-link (IDEATE-08) |
|
||||
| 03-02b-03 | `languages/py-async.md` | languages-engineer | ATELIER-103 | Python async per RESEARCH-v0.4 §Language-Derived: asyncio, anyio, cancellation, structured concurrency; section headers trace to Concurrency P7, P8, P9; fenced Python code in-language; cross-link `concurrency/patterns` + `messaging/queues` (per IDEATE-40 where applicable); ≥1 outbound cross-link (IDEATE-08) |
|
||||
| 03-02b-04 | `languages/py-testing.md` | languages-engineer | ATELIER-103 | Python testing per RESEARCH-v0.4 §Language-Derived: pytest, factory_boy, fixture discipline, parametrize; section headers trace to Testing P1, P2, P3; fenced Python code in-language; cross-link `testing/pyramid` + `testing/fixtures`; ≥1 outbound cross-link (IDEATE-08) |
|
||||
|
||||
**Wave 2c — Go (4 derived docs, ≤5 concurrent; same dependency tier as 2a/2b/2d)**
|
||||
| Task | File | Persona | REQ-ID | Must-have |
|
||||
|------|------|---------|--------|-----------|
|
||||
| 03-02c-01 | `languages/go-types.md` | languages-engineer | ATELIER-104 | Go type system per RESEARCH-v0.4 §Language-Derived: named types, generics, interfaces, type assertion discipline; section headers trace to Data P7, Errors P1, API P1; fenced Go code in-language (no standalone .go — D-020); ≥1 outbound cross-link (IDEATE-08) |
|
||||
| 03-02c-02 | `languages/go-tooling.md` | languages-engineer | ATELIER-104 | Go tooling per RESEARCH-v0.4 §Language-Derived: go vet, golangci-lint, go test -race, module discipline; section headers trace to DevOps P2, Concurrency P6; fenced Go code in-language; ≥1 outbound cross-link (IDEATE-08) |
|
||||
| 03-02c-03 | `languages/go-concurrency.md` | languages-engineer | ATELIER-104 | Go concurrency per RESEARCH-v0.4 §Language-Derived: goroutines, channels, context, select, sync primitives — Go's distinctive strength earns a dedicated concurrency doc (not go-async); section headers trace to Concurrency P1, P3, P5, P7, P8, P9; fenced Go code in-language; cross-link `concurrency/patterns` + `messaging/queues` (per IDEATE-40 — go-concurrency → concurrency/patterns + messaging/queues where channel/broker analogs apply); ≥1 outbound cross-link (IDEATE-08) |
|
||||
| 03-02c-04 | `languages/go-testing.md` | languages-engineer | ATELIER-104 | Go testing per RESEARCH-v0.4 §Language-Derived: table-driven tests, t.Parallel, t.Cleanup, race detector; section headers trace to Testing P1, P2, P3, P9; fenced Go code in-language; cross-link `testing/pyramid` + `testing/fixtures`; ≥1 outbound cross-link (IDEATE-08) |
|
||||
|
||||
**Wave 2d — Rust (4 derived docs, ≤5 concurrent; same dependency tier as 2a/2b/2c)**
|
||||
| Task | File | Persona | REQ-ID | Must-have |
|
||||
|------|------|---------|--------|-----------|
|
||||
| 03-02d-01 | `languages/rs-ownership.md` | languages-engineer | ATELIER-105 | Rust ownership per RESEARCH-v0.4 §Language-Derived: Send/Sync, lifetimes, borrowing, ownership transfer — Rust's distinctive strength earns a dedicated ownership doc (not rs-types); section headers trace to Concurrency P1, P3, Data P7; fenced Rust code in-language (no standalone .rs — D-020); ≥1 outbound cross-link (IDEATE-08) |
|
||||
| 03-02d-02 | `languages/rs-tooling.md` | languages-engineer | ATELIER-105 | Rust tooling per RESEARCH-v0.4 §Language-Derived: cargo, clippy, fmt, edition discipline; section headers trace to DevOps P2, Documentation P1; fenced Rust code in-language; ≥1 outbound cross-link (IDEATE-08) |
|
||||
| 03-02d-03 | `languages/rs-async.md` | languages-engineer | ATELIER-105 | Rust async per RESEARCH-v0.4 §Language-Derived: tokio, async traits, cancellation, pin; section headers trace to Concurrency P5, P7, P8, P9; fenced Rust code in-language; cross-link `concurrency/patterns` + `messaging/delivery-semantics` (per IDEATE-40 where applicable); ≥1 outbound cross-link (IDEATE-08) |
|
||||
| 03-02d-04 | `languages/rs-testing.md` | languages-engineer | ATELIER-105 | Rust testing per RESEARCH-v0.4 §Language-Derived: #[test], proptest, property testing, mock discipline; section headers trace to Testing P1, P2, P3, P9; fenced Rust code in-language; cross-link `testing/pyramid` + `testing/fixtures`; ≥1 outbound cross-link (IDEATE-08) |
|
||||
|
||||
> **Parallelism note:** Wave 2a + 2b + 2c + 2d together = 16 independent tasks. The executor schedules at most 5 concurrently per `parallelization.max_concurrent_agents`; the remaining 11 run as slots free. The 2a/2b/2c/2d labels are organizational (by language), not a hard sequencing barrier — all sub-waves are in the same dependency tier (all depend only on Wave 1).
|
||||
|
||||
**Verify (P3):**
|
||||
- Structural: 4 stubs extended + 16 new derived docs exist under `languages/` (20 language files total)
|
||||
- Behavioral: every derived-doc section header names ≥1 traced domain P-rule AND core C-rule inline (D-063, D-066); no new P-rules introduced; no P1–P10 numbering on language stubs (D-063); domain-expert verifies no orphan sections (every derived doc traces to ≥1 domain P-rule)
|
||||
- Security: no standalone `.ts`/`.py`/`.go`/`.rs` runtime artifacts (D-020 docs-only contract — fenced code in `.md` only)
|
||||
- Quality: each derived doc ≥1 outbound cross-link to a traced domain doc (IDEATE-08); language-derived docs link to BOTH new (edge/messaging) and existing domains where relevant (IDEATE-40); the 16 derived docs collectively trace to the domains sketched in RESEARCH-v0.4 §Language-Derived (the Languages Coverage sub-table that captures these traces lands in P4 ATELIER-107)
|
||||
|
||||
## Phase 4 — Matrix + Review + Manifest + Architecture Integration
|
||||
|
||||
**Goal:** Extend the matrix (+20 P-rule → C-rule mappings, 10 edge + 10 messaging; verify pre-existing 170 rows per G-014, append 20, confirm post-v0.4 is exactly 190), domain-coverage (per-domain rows + Core Principle Coverage table for 2 new domains + new Languages Coverage sub-table with exactly 16 rows), ARCHITECTURE.md (v0.4 section + component tree for edge/messaging/languages-derived), review docs (agent + peer-review + anti-patterns with all 6 chaos anti-patterns + language-derived anti-patterns + .ts/.py/.go/.rs artifact types), and the manifest (new `## Languages` section closing pre-existing drift + post-v0.4 totals). Closes the traceability loop and makes the manifest + architecture authoritative for v0.4.
|
||||
**Branch:** `phase/04-matrix-review-manifest` (from `milestone/v0.4-edge-quantum-langs`)
|
||||
**Personas:** domain-expert (matrix + coverage + anti-patterns), tech-writer (checklists), lead-developer (manifest + ARCHITECTURE.md — authoritative index + architecture tree)
|
||||
**Tag:** v0.3.4
|
||||
**Requirements:** ATELIER-106, ATELIER-107, ATELIER-117, ATELIER-108, ATELIER-109, ATELIER-110, ATELIER-111
|
||||
|
||||
### Wave 1 (sequential — matrix is the arbiter, must be authoritative first; then coverage; then ARCHITECTURE.md)
|
||||
| Task | File | Persona | REQ-ID | Must-have |
|
||||
|------|------|---------|--------|-----------|
|
||||
| 04-01-01 | `matrix/principles-matrix.md` (extend) | domain-expert | ATELIER-106 | Add "Edge" + "Messaging" sections, 10 rows each, format matching v0.1/v0.2/v0.3 tables (rows per RESEARCH-v0.4 §Matrix Integration); **review check: row count per new domain = 10, each row ≥1 C-rule** (IDEATE-02 carried forward); **verify pre-existing count first (IDEATE-34 corrected per G-014, ATELIER-106 refinement): the current matrix has exactly 170 P-rows (17 domains × 10, verified by grill G-014) — confirm 170 exists BEFORE appending the 20 v0.4 rows; do NOT hunt for or delete a phantom 171st row; post-v0.4 count must be exactly 190 (= 170 + 20)**; update Coverage Summary to "post-v0.4: 19 domains, 190 P-rules" in BOTH the summary block AND the per-domain section count (IDEATE-18, IDEATE-43 carried forward) |
|
||||
| 04-01-02 | `matrix/domain-coverage.md` (extend) | domain-expert | ATELIER-107 | THREE artifacts updated (IDEATE-33, IDEATE-42, ATELIER-107 refinement): (a) per-domain row-schema table — add Edge + Messaging rows (schema: domain, P-count, derived-doc-count, manifest-listed, status per IDEATE-03); (b) Core Principle Coverage table (C1–C8 → domains) — extend for 2 new domains: C4 Locality grows by edge P1/P8 + messaging P7; C5 Reversibility grows by edge P2/P4/P7/P10 + messaging P5/P8/P9; C7 Observability grows by edge P10 + messaging P8/P10; C8 Economy grows by edge P3/P9 + messaging P6; C2 grows by messaging P1/P2/P4 (parallel to v0.3 IDEATE-19); (c) new `## Languages Coverage` sub-table with EXACTLY 16 rows (one per language-derived doc) — schema: Language \| Derived Doc \| Traced Domain P-rules \| Traced Core C-rules; verifiable as a P4 review check (no orphans, every derived doc traced to ≥1 domain P-rule, per D-066) |
|
||||
| 04-01-03 | `ARCHITECTURE.md` (extend) | lead-developer | ATELIER-117 | Extend the component tree with `edge/` + `messaging/` domain entries; add a `## v0.4 Domain Additions` section documenting the two new domains (boundary per D-061/D-062, derived docs, cross-link targets); note the `languages/` derived-doc extension (D-063 apply-don't-derive — no new P-rules, no new matrix rows). Closes architecture-tree drift: the file currently stops at v0.3 with no edge/messaging/languages-derived coverage (IDEATE-32) |
|
||||
|
||||
### Wave 2 (parallel — 3 review docs, independent; ≤5 concurrent)
|
||||
| Task | File | Persona | REQ-ID | Must-have |
|
||||
|------|------|---------|--------|-----------|
|
||||
| 04-02-01 | `review/agent-checklist.md` (extend) | tech-writer | ATELIER-108 | Add 3 trigger sections per RESEARCH-v0.4 §Review Integration: "If Edge (see domains/edge/)" — 10 checks (P1 proximity/latency-as-correctness, P2 offline, P3 constrained resources declared, P4 sync conflicts bounded, P5 idempotent, P6 cache invalidation explicit, P7 partial degradation, P8 geographic distribution, P9 identity constrained, P10 local-first telemetry); "If Messaging (see domains/messaging/)" — 10 checks (P1 messages are contracts, P2 ordering explicit, P3 idempotent consumers, P4 delivery semantics explicit, P5 dead-letter defined, P6 backpressure bounded, P7 partitioning intentional, P8 replay/retention configured, P9 schemas evolve compatibly, P10 observable); "If Language-Derived Docs (see languages/)" — 5 checks (no new P-rules per D-063/D-066; every section header names ≥1 traced domain P-rule AND C-rule inline; fenced code in-language + illustrative only per D-020; first-principles stub retains structure — no P1–P10 numbering; cross-links to traced domain docs present ≥1 outbound per derived doc per ATELIER-114) |
|
||||
| 04-02-02 | `review/peer-review-checklist.md` (extend) | tech-writer | ATELIER-109 | Add 3 peer-review sections mirroring the agent-checklist as reviewer-voice questions, plus 3 scope checks (IDEATE-09 parity carried forward): edge scope check (proximity/location/constraint angle, not generic perf or in-process concurrency — D-061); messaging scope check (cross-process/network-failure-model, not in-process concurrency — D-062); language-derived scope check (apply don't derive — no new P-rules, no new matrix rows — D-063, D-066) |
|
||||
| 04-02-03 | `review/anti-patterns.md` (extend) | domain-expert | ATELIER-110 | Per RESEARCH-v0.4 §Review Integration + IDEATE-36/37 (ATELIER-110 refinement): (a) **v0.4 deployable-artifact-types table extended to `.ts`/`.py`/`.go`/`.rs` standalone files** (language-derived docs risk, per D-020 docs-only contract); (b) **all 6 RESEARCH-specified chaos anti-patterns** (not just the 4 named in the req-row text) — edge: sync conflict loop (P4, C1/C5 — infinite oscillation is a correctness failure/livelock), TTL-less cache (P6, C1/C3 — stale-forever under partition), blocking call on constrained device (P3/P5, C8/C1 — blocks the node, no timeout, retry unsafe without idempotency); messaging: shared subscription (P2/P3, C1 — breaks ordering + dedup), blocking consumer (P6, C1/C8 — blocks the partition, lag grows), unbounded retry / no DLQ (P5, C1/C5 — infinite retry is a livelock); (c) **language-derived-doc anti-patterns**: introducing a new P-rule (e.g., "TS P1" — violates D-063 + ARCHITECTURE.md hierarchy), orphan section with no domain P-rule trace (violates D-066 + matrix completeness), runtime standalone artifact `.ts`/`.py`/`.go`/`.rs` (violates D-020), first-principles stub gaining P1–P10 numbering (violates D-063); (d) **domain-specific anti-patterns** per RESEARCH-v0.4 §Review Integration — edge 10 (central-region-only deployment, crash-on-disconnect, undeclared resource budget, sync oscillation, non-idempotent edge write, TTL-less cache, crash-on-node-failure, random geographic placement, shared edge-device credential, fire-and-forget telemetry), messaging 10 (schemaless message, undocumented "FIFO", non-idempotent consumer, unstated delivery semantic, no DLQ, unbounded retry budget, default partition key, ephemeral broker, silent schema change, silent backlog) |
|
||||
|
||||
### Wave 3 (sequential — MANIFEST after review docs settled; per D-050 precedent anti-patterns + manifest combine under lead-developer so manifest lists file paths after anti-pattern content is settled)
|
||||
| Task | File | Persona | REQ-ID | Must-have |
|
||||
|------|------|---------|--------|-----------|
|
||||
| 04-03-01 | `MANIFEST.md` (extend) | lead-developer | ATELIER-111 | Per D-065 + IDEATE-31/43 (ATELIER-111 refinement): (a) add 2 new domains (edge, messaging) + all 10 derived docs to the Domains table; (b) add a new top-level `## Languages` section between `## Domains` and `## Examples` (matching reading order: core → domains → languages → examples) listing the 4 v0.1 first-principles stubs (typescript.md, python.md, go.md, rust.md — closes pre-existing drift since v0.1, parallel to ESC-002/ATELIER-91) AND the 16 new v0.4 derived docs; section note: "Language docs apply (not derive) domain principles. They introduce no new P-rules and no new matrix rows (D-063, D-066)."; (c) update Cross-Cutting summary line to exactly "19 domains, 190 P-rules post-v0.4" (IDEATE-18, IDEATE-43 carried forward) |
|
||||
|
||||
**Verify (P4):**
|
||||
- Structural: matrix has 19 domain sections (17 v0.1/v0.2/v0.3 + 2 new), exactly 190 P-rules (170 pre-existing verified per G-014 + 20 new; NOT 171 reconciled to 170 — the 171 discrepancy was a false premise corrected by grill G-014); domain-coverage has the per-domain v0.4 table AND the updated C-rule coverage table AND the Languages Coverage sub-table (exactly 16 rows); ARCHITECTURE.md has the v0.4 section + component tree; 3 review docs extended; MANIFEST lists all v0.4 docs + the new Languages section
|
||||
- Behavioral: every new edge + messaging P-rule has a matrix row; domain-expert verifies no orphans (IDEATE-13 carried forward); Coverage Summary reads "19 domains, 190 P-rules" (IDEATE-18/43); Languages Coverage sub-table has exactly 16 rows — every language-derived doc traced to ≥1 domain P-rule (IDEATE-42, D-066); ARCHITECTURE.md component tree matches the directory tree (IDEATE-32)
|
||||
- Security: anti-patterns cover all 6 chaos anti-patterns (edge sync-loop/TTL-less/blocking-call; messaging shared-sub/blocking-consumer/unbounded-retry) + edge P9 (shared credential) + messaging P1 (schemaless) + language-derived runtime-artifact (.ts/.py/.go/.rs)
|
||||
- Quality: MANIFEST is authoritative — every v0.4 file listed, languages/ listed closing pre-existing drift (ATELIER-111 per IDEATE-31); unlisted = not part of framework; agent-checklist + peer-review-checklist have parity across edge + messaging + language-derived (ATELIER-108 ↔ ATELIER-109); matrix row-count test passes (exactly 190, reconciled)
|
||||
|
||||
## Phase 5 — Examples + Cross-Links
|
||||
|
||||
**Goal:** Add 2 good + 2 bad examples (D-068 — pre-named, highest illustration value) and verify cross-domain links from the 2 new domains + 16 language-derived docs to existing ones, AND edge↔messaging intra-v0.4 links resolve in both directions (IDEATE-40). Examples are markdown with fenced code only (no standalone .ts/.py/.go/.rs artifacts — D-020). Bad examples are single-breach citing the specific breached P-rule (D-068).
|
||||
**Branch:** `phase/05-examples-crosslinks` (from `milestone/v0.4-edge-quantum-langs`)
|
||||
**Personas:** edge-engineer (edge examples), tech-writer (messaging examples + cross-link audit), domain-expert (P-rule citation + cross-link validation)
|
||||
**Tag:** v0.3.5
|
||||
**Requirements:** ATELIER-112, ATELIER-113, ATELIER-114
|
||||
|
||||
### Wave 1 (parallel — 4 examples, independent; ≤5 concurrent)
|
||||
| Task | File | Persona | REQ-ID | Must-have |
|
||||
|------|------|---------|--------|-----------|
|
||||
| 05-01-01 | `examples/good/edge-offline-sync.md` | edge-engineer | ATELIER-112 | Good edge example per RESEARCH-v0.4 §Examples: demonstrates offline-first + sync conflict resolution via CRDT or LWW — highest-illustration-value edge case; markdown with fenced code (a CRDT register merge + an offline write queue + a reconnect reconcile); P-rules demonstrated: edge P2 (Offline is a First-Class State), P4 (Sync Conflicts are Bounded), P5 (Idempotent), P10 (local-first telemetry); exercises the `sync.md` CRDT-vs-LWW decision matrix; cross-link to `edge/offline-first.md` + `edge/sync.md`; no runtime artifacts |
|
||||
| 05-01-02 | `examples/good/messaging-idempotent-consumer.md` | tech-writer | ATELIER-112 | Good messaging example per RESEARCH-v0.4 §Examples: demonstrates exactly-once-via-idempotency-keys + dedup — highest-illustration-value messaging case; markdown with fenced code (an idempotent consumer with a dedup store + a DLQ routing rule); P-rules demonstrated: messaging P3 (Consumers are Idempotent), P4 (Delivery Semantics Explicit), P5 (Dead-Letter Handling), P10 (Observable); exercises the `delivery-semantics.md` idempotency + DLQ guidance; cross-link to `messaging/delivery-semantics.md` + `messaging/queues.md`; no runtime artifacts |
|
||||
| 05-01-03 | `examples/bad/edge-sync-loop.md` | edge-engineer | ATELIER-113 | Bad edge example per RESEARCH-v0.4 §Examples + ATELIER-110: a sync conflict that oscillates/loops forever — the pre-specified edge chaos anti-pattern; **single-breach (D-068): cites edge P4 (Sync Conflicts are Bounded, Not Infinite) breached** — prose explains why it is a livelock, not eventual consistency; markdown with fenced code showing the oscillating sync; cross-link to `edge/sync.md` + `review/anti-patterns.md` (the sync-conflict-loop chaos anti-pattern) |
|
||||
| 05-01-04 | `examples/bad/messaging-shared-subscription.md` | tech-writer | ATELIER-113 | Bad messaging example per RESEARCH-v0.4 §Examples + ATELIER-110: a shared subscription causing duplicate processing / fan-out duplication — the pre-specified messaging chaos anti-pattern; **single-breach (D-068): primary breach cited is messaging P2 (Ordering is a Property, Not an Assumption); P3 (Consumers are Idempotent) noted as the compounding consequence — the example remains single-breach in its named violation**; markdown with fenced code showing the shared subscription; cross-link to `messaging/pubsub.md` + `review/anti-patterns.md` (the shared-subscription chaos anti-pattern) |
|
||||
|
||||
### Wave 2 (sequential — cross-link audit after all docs exist)
|
||||
| Task | File | Persona | REQ-ID | Must-have |
|
||||
|------|------|---------|--------|-----------|
|
||||
| 05-02-01 | Cross-link audit (all new derived docs across edge + messaging + languages) | tech-writer | ATELIER-114 | Per IDEATE-08 (carried forward) + IDEATE-40 (ATELIER-114 refinement): (a) every new derived doc (10 domain derived + 16 language-derived = 26) has ≥1 outbound cross-link to a MANIFEST-listed doc; links resolve; (b) **edge↔messaging intra-v0.4 links resolve in BOTH directions** — every edge→messaging and messaging→edge link resolves by the P6 ship (edge/iot.md → messaging/queues.md MQTT QoS parallels; messaging docs that reference edge where relevant); (c) **language-derived docs link to BOTH new (edge/messaging) and existing domains where relevant** (e.g., ts-async.md → messaging/delivery-semantics; go-concurrency.md → concurrency/patterns + messaging/queues); domain-expert validates the cross-link targets are correct (not just present); no back-link edits to v0.1/v0.2/v0.3 content (D-026 extended — one-directional outward) |
|
||||
|
||||
**Verify (P5):**
|
||||
- Structural: 4 new example files exist (all .md)
|
||||
- Behavioral: each bad example cites the P-rule breached (edge: P4 single-breach; messaging: P2 primary + P3 consequence, single-breach named violation per D-068); each good example cites the P-rules it demonstrates
|
||||
- Security: no standalone `.ts`/`.py`/`.go`/`.rs` runtime artifacts (deployable artifact mitigation, IDEATE-36, D-020)
|
||||
- Quality: all cross-links from the 26 new derived docs resolve to MANIFEST-listed docs; edge↔messaging intra-v0.4 links resolve bidirectionally (IDEATE-40); language-derived docs link to both new + existing domains where relevant (IDEATE-40); no back-link edits to v0.1/v0.2/v0.3 content (D-026 extended)
|
||||
|
||||
## Phase 6 — Final Review + Ship (N+1)
|
||||
|
||||
**Goal:** Multi-persona review across all v0.4 phases, audit, milestone ship. P6 IS the v0.4 release (NFR → no separate minor tag; v0.3.6 IS the deliverable). Phase-specific personas (edge-engineer, languages-engineer) review their authored content in Wave 1 (D-052 precedent), then are removed in Wave 2 (roster returns to 3 active).
|
||||
**Branch:** `phase/06-final-review-ship` (from `milestone/v0.4-edge-quantum-langs`)
|
||||
**Personas:** lead-developer (coordinate + ship), domain-expert (review), tech-writer (review), edge-engineer (review edge content, then removed), languages-engineer (review language-derived content, then removed)
|
||||
**Tag:** v0.3.6 (IS the v0.4 milestone release — NFR, no separate minor tag)
|
||||
**Requirements:** ATELIER-115, ATELIER-116
|
||||
|
||||
### Wave 1 — Review (sequential — edge-engineer + languages-engineer review their content per D-052 before removal)
|
||||
| Task | Activity | Persona | REQ-ID | Must-have |
|
||||
|------|----------|---------|--------|-----------|
|
||||
| 06-01-01 | `ciagent-review` — multi-persona review of all v0.4 changes | lead-developer | ATELIER-115 | Auto-apply P0 fixes; flag P1+ for post-hoc; if P1+ found, fix in this phase; **edge-engineer reviews edge domain content (D-061 boundary — proximity/location angle, not generic perf/concurrency)**; **languages-engineer reviews language-derived content (D-063 apply-don't-derive scope check — no new P-rules, no P1–P10 numbering, every section traces to a domain P-rule)**; tech-writer + domain-expert review messaging content (D-062 boundary — cross-process/network-failure-model); domain-expert verifies all 20 new P-rules (10 edge + 10 messaging) trace to ≥1 C-rule (no orphans) AND the 16 Languages Coverage sub-table rows match the derived docs' actual section-header traces (IDEATE-42) |
|
||||
| 06-01-02 | `ciagent-audit` — reconstruction + discipline | lead-developer | ATELIER-115 | git log matches .ciagent/ files; branch hygiene; commit discipline (every commit has `---ci---` block with `project: atelier`); MANIFEST reconstruction test (every listed doc exists, every existing doc is listed — incl. the new Languages section per ATELIER-111); **matrix row-count test: exactly 190 P-rows (10 per domain × 19 — the pre-existing count was exactly 170 per grill G-014 verification; the IDEATE-34 "171-vs-170 discrepancy" was a false premise, corrected — verify 170 + 20 = 190, NOT 171 + 20)**; ARCHITECTURE.md component-tree test (matches directory tree per ATELIER-117/IDEATE-32) |
|
||||
|
||||
### Wave 2 — Ship (sequential — after review; phase-specific personas removed)
|
||||
| Task | Activity | Persona | REQ-ID | Must-have |
|
||||
|------|----------|---------|--------|-----------|
|
||||
| 06-02-01 | `ciagent-ship` — milestone ship | lead-developer | ATELIER-116 | Merge `phase/06` → `milestone/v0.4-edge-quantum-langs` → `main`; tag `v0.3.6`; Gitea release with full milestone summary; delete all v0.4 branches (tags preserve history) |
|
||||
| 06-02-02 | Complete milestone (REQUIREMENTS + ROADMAP + PERSONAS) | lead-developer | ATELIER-116 | Mark all v0.4 requirements (ATELIER-92..117) `covered`; ROADMAP v0.4 → complete; **PERSONAS: remove edge-engineer + languages-engineer (roster returns to 3 active per D-067)**; clear checkpoint |
|
||||
|
||||
**Verify (P6):**
|
||||
- Structural: all 26 v0.4 requirements (ATELIER-92..117) marked covered
|
||||
- Behavioral: reconstruction test passes (git log ↔ .ciagent/); matrix row-count test passes (exactly 190 — reconciled per IDEATE-34); MANIFEST reconstruction test passes (incl. Languages section); ARCHITECTURE.md component-tree test passes (per ATELIER-117/IDEATE-32)
|
||||
- Security: audit clean (no critical issues); no standalone runtime artifacts (`.ts`/`.py`/`.go`/`.rs` forbidden per D-020); no new P-rules in language-derived docs (D-063); edge/messaging boundaries respected (D-061/D-062)
|
||||
- Quality: milestone merged to main, tag v0.3.6 exists, all v0.4 branches deleted; edge-engineer + languages-engineer personas removed (roster = 3 active per D-067)
|
||||
|
||||
## v0.4 Wave Ordering Summary
|
||||
|
||||
| Phase | Waves | Parallelism |
|
||||
|-------|-------|-------------|
|
||||
| P0 | (pre-exec) | Sequential stages (specify→clarify→research→ideate→plan→grill) — COMPLETE |
|
||||
| P1 | 2 | Wave 1 sequential (first-principles), Wave 2 parallel (4 derived docs) |
|
||||
| P2 | 2 | Wave 1 sequential (first-principles), Wave 2 parallel (4 derived docs) |
|
||||
| P3 | 2 | Wave 1 parallel (4 stubs extended), Wave 2 parallel (16 derived docs — 2a TS + 2b Python + 2c Go + 2d Rust, capped at 5 concurrent) |
|
||||
| P4 | 3 | Wave 1 sequential (matrix → coverage → ARCHITECTURE.md), Wave 2 parallel (3 review docs), Wave 3 sequential (MANIFEST after review docs settled) |
|
||||
| P5 | 2 | Wave 1 parallel (4 examples), Wave 2 sequential (cross-link audit) |
|
||||
| P6 | 2 | Wave 1 sequential (review → audit), Wave 2 sequential (ship → complete) |
|
||||
|
||||
## v0.4 Requirements → Phase Mapping
|
||||
|
||||
| Phase | Requirements | Count |
|
||||
|-------|-------------|-------|
|
||||
| 1 | ATELIER-92..96 | 5 |
|
||||
| 2 | ATELIER-97..101 | 5 |
|
||||
| 3 | ATELIER-102..105 | 4 |
|
||||
| 4 | ATELIER-106, 107, 117, 108, 109, 110, 111 | 7 |
|
||||
| 5 | ATELIER-112, 113, 114 | 3 |
|
||||
| 6 | ATELIER-115, 116 | 2 |
|
||||
| **Total** | | **26** |
|
||||
|
||||
## v0.4 Ideation Refinements → Task Bake-In Map
|
||||
|
||||
All 12 accepted ideation refinements (IDEATE-31, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43) + the 1 new requirement (IDEATE-32 → ATELIER-117) are baked into the relevant phase tasks as explicit must-have notes (parallel to v0.3's Refinements Notes → Task Bake-In Map):
|
||||
|
||||
| IDEATE-ID | Refinement | Baked Into Task(s) | How |
|
||||
|-----------|-----------|-------------------|-----|
|
||||
| IDEATE-31 | MANIFEST Languages section lists 4 v0.1 stubs + 16 derived docs (refines ATELIER-111) | 04-03-01 | MANIFEST gains `## Languages` section between `## Domains` and `## Examples`; lists 4 stubs (closes pre-existing drift since v0.1, parallel to ESC-002/ATELIER-91) + 16 derived docs |
|
||||
| IDEATE-32 | ARCHITECTURE.md v0.4 update (new req ATELIER-117) | 04-01-03 | ARCHITECTURE.md gains `## v0.4 Domain Additions` section + component tree entries for edge/messaging/languages-derived; closes architecture-tree drift |
|
||||
| IDEATE-33 | Core Principle Coverage C1–C8 → domains table update for edge + messaging (refines ATELIER-107) | 04-01-02 | domain-coverage.md C-rule coverage table extended (C4 +edge P1/P8 +messaging P7; C5 +edge P2/P4/P7/P10 +messaging P5/P8/P9; C7 +edge P10 +messaging P8/P10; C8 +edge P3/P9 +messaging P6; C2 +messaging P1/P2/P4) |
|
||||
| IDEATE-34 | Matrix post-v0.4 = exactly 190; verify current 170 (corrected per G-014 — the "171 vs 170" claim was a false premise; matrix has exactly 170) (refines ATELIER-106) | 04-01-01, 06-01-02 | P4 verifies 170 exists before appending 20 rows; P6 audit verifies exactly 190 (10 × 19). Do NOT hunt for/delete a phantom 171st row |
|
||||
| IDEATE-35 | streams.md + delivery-semantics.md comparison tables required (refines ATELIER-100, 101) | 02-02-03, 02-02-04 | streams.md stream-platform comparison (Kafka/Kinesis/Pulsar); delivery-semantics.md dead-letter strategy comparison |
|
||||
| IDEATE-36 | Language-derived-doc anti-patterns + .ts/.py/.go/.rs artifact-type extension (refines ATELIER-110) | 04-02-03 | anti-patterns.md gains language-derived anti-patterns (new P-rule, orphan section, runtime artifact, P1–P10 numbering) + v0.4 artifact types table extended to .ts/.py/.go/.rs |
|
||||
| IDEATE-37 | All 6 RESEARCH chaos anti-patterns (refines ATELIER-110) | 04-02-03 | anti-patterns.md includes all 6: edge sync-loop/TTL-less/blocking-call; messaging shared-sub/blocking-consumer/unbounded-retry (not just the 4 named in req-row text) |
|
||||
| IDEATE-38 | offline-first.md maps offline write-queue to testing pyramid tiers (refines ATELIER-94) | 01-02-02 | offline-first.md maps unit (conflict detection on merge function), integration (reconnect reconcile vs local store), e2e (partition simulation with fake network) |
|
||||
| IDEATE-39 | delivery-semantics.md fenced idempotency-key dedup-store example (refines ATELIER-101) | 02-02-04 | delivery-semantics.md includes a fenced consumer-with-dedup-store (not prose-only) — parallel to v0.3 IDEATE-29 signed-attestation |
|
||||
| IDEATE-40 | P5 cross-link audit verifies edge↔messaging intra-v0.4 links both directions (refines ATELIER-114) | 05-02-01 | Cross-link audit checks edge→messaging AND messaging→edge links resolve; language-derived docs link to both new + existing domains where relevant |
|
||||
| IDEATE-41 | streams.md enumerates consumer-group rebalance strategies (refines ATELIER-100) | 02-02-03 | streams.md enumerates sticky/cooperative/eager — each with partition-stop-the-world cost + use-case fit (parallel to v0.3 IDEATE-30 drift-type enumeration) |
|
||||
| IDEATE-42 | Languages Coverage sub-table = exactly 16 rows (refines ATELIER-107) | 04-01-02, 06-01-01 | domain-coverage.md gains `## Languages Coverage` sub-table with exactly 16 rows; P6 review verifies the 16 rows match the derived docs' actual section-header traces |
|
||||
| IDEATE-43 | MANIFEST Cross-Cutting summary = "19 domains, 190 P-rules post-v0.4" (refines ATELIER-111) | 04-03-01 | MANIFEST Cross-Cutting summary line updates to exactly "19 domains, 190 P-rules post-v0.4" (IDEATE-18 carried forward) |
|
||||
|
||||
## v0.4 Decisions Logged (planning stage)
|
||||
|
||||
| ID | Decision | Rationale | Confidence |
|
||||
|----|----------|-----------|------------|
|
||||
| D-070 | P3 Wave 2 splits into 2a/2b/2c/2d labels (by language) but all are the same dependency tier | 16 derived docs are all independent post-Wave-1; the 2a/2b/2c/2d labels organize by language, the executor schedules ≤5 concurrent per config.json. Avoids inventing a false dependency between languages (parallel to v0.3 D-049 i18n/compliance 2a/2b split) | 0.88 |
|
||||
| D-071 | P4 splits into 3 waves: matrix+coverage+ARCHITECTURE (Wave 1 sequential), review docs (Wave 2 parallel), MANIFEST (Wave 3 sequential after review docs settled) | Matrix is the arbiter (must be authoritative first per v0.2/v0.3 precedent); ARCHITECTURE.md follows coverage so the component tree reflects the matrix; review docs are independent of each other; MANIFEST lists file paths and totals, sequenced after anti-patterns content is settled (D-050 precedent — anti-patterns + manifest combine under lead-developer). Net 3 waves, respecting that ARCHITECTURE.md (ATELIER-117) is a new v0.4 deliverable that must follow the matrix/coverage extension | 0.83 |
|
||||
| D-072 | P6 splits into 2 waves: review (Wave 1 — edge-engineer + languages-engineer review their content per D-052 before removal), ship (Wave 2 — tag + merge + remove personas) | Phase-specific personas review their authored content one final time in Wave 1 (D-052 precedent — ensures D-061 edge boundary and D-063 apply-don't-derive are checked by the specialist), then are removed in Wave 2 (roster returns to 3 per D-067). Separating review from ship ensures the specialist review completes before the personas are deactivated | 0.84 |
|
||||
|
||||
## Assumptions Logged
|
||||
|
||||
| # | Assumption | Confidence |
|
||||
|---|-----------|------------|
|
||||
| A-006 | The pre-existing matrix has exactly 170 P-rows (verified by grill G-014 direct count); appending 20 yields exactly 190 with no reconciliation needed. The IDEATE-34 "171-vs-170 discrepancy" was a false premise corrected by the grill (parallel to v0.3's matrix-extension assumption A-002) | 0.88 |
|
||||
| A-007 | The 4 new domain P-rules (edge + messaging) map to existing core C1–C8 without needing new core principles (core is stable at 8 — carried forward from A-002) | 0.95 |
|
||||
| A-008 | Wave 2 of P3 (16 derived docs) can be scheduled by the executor as 5-then-5-then-5-then-1 without a hard sub-wave barrier (carried forward from v0.3 A-003) | 0.90 |
|
||||
| A-009 | The Languages Coverage sub-table's 16 rows can be authored in P4 from the P3 derived docs' actual section-header traces (the RESEARCH-v0.4 sketch is the expected trace set; finalized when the derived docs are authored) | 0.85 |
|
||||
| A-010 | The edge↔messaging intra-milestone cross-links (edge/iot.md → messaging/queues.md) dangle at the P1 tag (v0.3.1) and resolve once P2 ships (v0.3.2); this is acceptable per D-053 vertical-slice integrity and verified bidirectionally in P5 (IDEATE-40) | 0.86 |
|
||||
|
||||
## Milestone Type: NFR
|
||||
|
||||
Per branch-strategy.md, this is an NFR milestone (all phases are docs/chore — no `feat` code). NFR milestones produce progressive patches; the final patch (v0.3.6) IS the v0.4 deliverable. No separate minor tag.
|
||||
@@ -164,11 +164,6 @@ NFR milestone: no separate minor tag. The final patch (v0.2.6) IS the v0.3 deliv
|
||||
| D-046 | i18n testing-i18n.md must map pseudo-locale testing to testing pyramid tiers | Avoids generic "test i18n" guidance; maps to unit (missing-key), integration (snapshot per locale), e2e (RTL coverage) | 0.78 |
|
||||
| D-047 | compliance evidence.md must include a fenced signed-attestation example (Cosign or in-toto) | Prose-only evidence guidance is weak; a fenced example demonstrates the principle concretely (P6 Evidence Collected Continuously) | 0.80 |
|
||||
| D-048 | ai-ml monitoring-drift.md must enumerate 3 drift types (data/concept/prediction) with a detection signal per type | Avoids conflating drift types; each has distinct detection signals and retraining triggers | 0.82 |
|
||||
| D-049 | P3 splits Wave 2 into 2a (i18n) + 2b (compliance) labels but both are the same dependency tier | 8 derived docs are all independent post-Wave-1; 2a/2b labels organize by domain, executor schedules ≤5 concurrent per config.json. Avoids inventing a false dependency between i18n and compliance | 0.88 |
|
||||
| D-050 | P4 Wave 2 runs exactly 5 concurrent tasks (at the max_concurrent_agents cap) | matrix, coverage, agent-checklist, peer-review-checklist, anti-patterns, manifest = 6 extensions, but anti-patterns (04-02-04) and manifest (04-02-05) are combined under lead-developer for manifest to sequence after anti-patterns content is settled. Net 5 concurrent slots | 0.82 |
|
||||
| D-051 | P5 ai-ml-reproducibility.md example authored by tech-writer with ml-engineer consultation (not ml-engineer primary) | ml-engineer is removed after P2 per PERSONAS.md; P5 examples are tech-writer territory. ml-engineer constraints are baked into the task must-have (P1/P2/P3/P5 demonstrated) so the constraint survives the persona | 0.80 |
|
||||
| D-052 | P6 review uses platform-engineer + ml-engineer for content review before removal | Phase-specific personas review their authored content one final time in P6 Wave 1, then are removed in 06-01-04. Ensures D-023 (ai-ml scope) and GitOps correctness are checked by the specialist before the roster returns to 3 | 0.84 |
|
||||
| D-053 | Vertical-slice integrity: each phase is independently shippable | P1 ships gitops-operators domain docs (matrix rows land in P4 — acceptable because the domain is self-consistent; matrix extension is the traceability closure, not a blocker for the domain's internal consistency). P3 ships 2 domains together (D-022). P4 closes traceability + manifest. P5 closes examples + cross-links. P6 ships the release | 0.86 |
|
||||
|
||||
## Cross-Project References
|
||||
|
||||
@@ -177,51 +172,4 @@ None yet. Atelier is a standalone docs framework.
|
||||
## Milestone History
|
||||
|
||||
- **v0.1** — Initial Framework (COMPLETE). 8 core principles, 11 domains, 110 domain principles, full matrix, 4+3 examples, 4 languages. Tag v0.0.7.
|
||||
- **v0.2** — Infrastructure as Code + Kubernetes (COMPLETE). Adds 2 domains (20 new P-rules), matrix/review/examples integration. Tags v0.1.0–v0.1.5; v0.1.5 is the v0.2 release.
|
||||
- **v0.3** — GitOps + Operators + AI/ML + i18n + Compliance (COMPLETE). Adds 4 domains (40 new P-rules), matrix 130→170 across 13→17 domains, review/checklist/anti-patterns integration, 4 new examples, MANIFEST examples/ listing (closes ESC-002). Tags v0.2.0–v0.2.6; v0.2.6 is the v0.3 release.
|
||||
- **v0.4** — Edge + Messaging + Language-Derived Docs (ACTIVE). Adds 2 new top-level domains (edge, messaging — 20 new P-rules), and extends `languages/` with derived docs per language (typescript, python, go, rust). Matrix 170→190 across 17→19 domains; review/checklist/anti-patterns extended; 4 new examples; MANIFEST gains a `languages/` section. Tags v0.3.0–v0.3.N; final patch IS the v0.4 deliverable.
|
||||
|
||||
## v0.4 — Edge + Messaging + Language-Derived Docs
|
||||
|
||||
**Milestone type:** NFR (all phases produce docs — no `feat` runtime code)
|
||||
**Tag line:** v0.3.x (previous minor from v0.4)
|
||||
**Scope:** Extend the domain tree with two new top-level domains covering edge computing and messaging systems, plus extend `languages/` from first-principles-only stubs to full derived-doc trees. All content is docs-only markdown with illustrative code fences; no runtime/deployable artifacts. Closes the "Future Milestones" candidates in the v0.3 ROADMAP (edge domain + language-specific derived docs).
|
||||
|
||||
### New Domains
|
||||
|
||||
- `domains/edge/` — edge computing domain (latency-bound, resource-constrained, geo-distributed)
|
||||
- `first-principles.md` — 10 edge principles (P1–P10)
|
||||
- Derived: `cdn.md`, `offline-first.md`, `iot.md`, `sync.md`
|
||||
- `domains/messaging/` — messaging systems domain (async, queues, streams, pub/sub)
|
||||
- `first-principles.md` — 10 messaging principles (P1–P10)
|
||||
- Derived: `queues.md`, `pubsub.md`, `streams.md`, `delivery-semantics.md`
|
||||
|
||||
### Language-Derived Extension
|
||||
|
||||
Extend each `languages/*.md` from a single first-principles stub to a full derived-doc tree mirroring the domain structure (parallel to how `domains/` have first-principles + derived). Each language gets derived docs for the most-relevant domains (4 derived docs per language = 16 total), traced to that language's P-rules and to core C-rules.
|
||||
|
||||
- `languages/typescript.md` (existing first-principles) + derived: `ts-types.md`, `ts-tooling.md`, `ts-async.md`, `ts-testing.md`
|
||||
- `languages/python.md` + derived: `py-types.md`, `py-tooling.md`, `py-async.md`, `py-testing.md`
|
||||
- `languages/go.md` + derived: `go-types.md`, `go-tooling.md`, `go-concurrency.md`, `go-testing.md`
|
||||
- `languages/rust.md` + derived: `rs-ownership.md`, `rs-tooling.md`, `rs-async.md`, `rs-testing.md`
|
||||
|
||||
### Cross-Domain Integration
|
||||
|
||||
- Extend `matrix/principles-matrix.md` with 20 new P-rules → core C-rule mappings (10 per new domain; the language-derived docs trace to existing domain P-rules, not new matrix rows)
|
||||
- Extend `matrix/domain-coverage.md` with edge + messaging (row schema unchanged); add a `languages/` coverage sub-table listing each language's derived docs and the domains they trace to
|
||||
- Extend `review/agent-checklist.md`, `review/peer-review-checklist.md`, and `review/anti-patterns.md` with edge + messaging sections + a language-derived-docs section
|
||||
- Update `MANIFEST.md` with (a) the 2 new domains, (b) a new `languages/` section listing each language's first-principles + derived docs, (c) post-v0.4 totals (19 domains, 190 P-rules)
|
||||
- New examples (good + bad): `examples/good/edge-offline-sync.md`, `examples/good/messaging-idempotent-consumer.md`, `examples/bad/edge-sync-loop.md`, `examples/bad/messaging-shared-subscription.md`
|
||||
- Cross-links from new domains to existing `devops/`, `observability/`, `data/`, `concurrency/`, `kubernetes/`, `infrastructure-as-code/` domains; cross-links from language-derived docs to their traced domain docs
|
||||
|
||||
### Phase Plan (proposed, finalized in PLAN)
|
||||
|
||||
- P0 Pre-Execution: spec, clarify, research, ideate, plan, grill
|
||||
- P1 Edge domain
|
||||
- P2 Messaging domain
|
||||
- P3 Language-Derived Extension (16 derived docs across 4 languages)
|
||||
- P4 Matrix + Review Integration (20 new mappings, manifest languages/ section, checklist parity)
|
||||
- P5 Examples + Cross-Links
|
||||
- P6 Final Review + Ship (IS the v0.4 release → tag v0.3.6)
|
||||
|
||||
NFR milestone: no separate minor tag. The final patch (v0.3.6) IS the v0.4 deliverable.
|
||||
- **v0.2** — Infrastructure as Code + Kubernetes (COMPLETE). Adds 2 domains (20 new P-rules), matrix/review/examples integration. Tags v0.1.0–v0.1.5; v0.1.5 is the v0.2 release.
|
||||
@@ -133,38 +133,38 @@ All 35 requirements covered. 8 core principles, 11 domains, 110 domain principle
|
||||
|
||||
| REQ-ID | Requirement | Priority | Phase | Status |
|
||||
|--------|-------------|----------|-------|--------|
|
||||
| ATELIER-60 | `domains/gitops-operators/first-principles.md` — 10 GitOps/operator principles (P1–P10) | P0 | 1 | covered |
|
||||
| ATELIER-61 | `domains/gitops-operators/argocd.md` — ArgoCD derived doc | P1 | 1 | covered |
|
||||
| ATELIER-62 | `domains/gitops-operators/flux.md` — Flux derived doc | P1 | 1 | covered |
|
||||
| ATELIER-63 | `domains/gitops-operators/operators.md` — Kubernetes Operators derived doc | P1 | 1 | covered |
|
||||
| ATELIER-64 | `domains/gitops-operators/progressive-delivery.md` — progressive delivery derived doc | P1 | 1 | covered |
|
||||
| ATELIER-65 | `domains/ai-ml/first-principles.md` — 10 AI/ML principles (P1–P10) | P0 | 2 | covered |
|
||||
| ATELIER-66 | `domains/ai-ml/data-versioning.md` — data/model versioning derived doc | P1 | 2 | covered |
|
||||
| ATELIER-67 | `domains/ai-ml/model-evaluation.md` — evaluation derived doc | P1 | 2 | covered |
|
||||
| ATELIER-68 | `domains/ai-ml/serving.md` — model serving derived doc | P1 | 2 | covered |
|
||||
| ATELIER-69 | `domains/ai-ml/monitoring-drift.md` — monitoring/drift derived doc | P1 | 2 | covered |
|
||||
| ATELIER-70 | `domains/i18n/first-principles.md` — 10 i18n principles (P1–P10) | P0 | 3 | covered |
|
||||
| ATELIER-71 | `domains/i18n/locale-resources.md` — locale resource management derived doc | P1 | 3 | covered |
|
||||
| ATELIER-72 | `domains/i18n/formatting.md` — formatting (dates/numbers/units) derived doc | P1 | 3 | covered |
|
||||
| ATELIER-73 | `domains/i18n/rtl-bidi.md` — RTL/bidi layout derived doc | P1 | 3 | covered |
|
||||
| ATELIER-74 | `domains/i18n/testing-i18n.md` — i18n testing derived doc | P1 | 3 | covered |
|
||||
| ATELIER-75 | `domains/compliance/first-principles.md` — 10 compliance principles (P1–P10) | P0 | 3 | covered |
|
||||
| ATELIER-76 | `domains/compliance/audit-logs.md` — audit logging derived doc | P1 | 3 | covered |
|
||||
| ATELIER-77 | `domains/compliance/data-retention.md` — data retention derived doc | P1 | 3 | covered |
|
||||
| ATELIER-78 | `domains/compliance/policy-as-code.md` — policy-as-code derived doc | P1 | 3 | covered |
|
||||
| ATELIER-79 | `domains/compliance/evidence.md` — evidence collection derived doc | P1 | 3 | covered |
|
||||
| ATELIER-80 | Extend `matrix/principles-matrix.md` with 40 new P-rules → core C-rule mappings (10 per new domain; review check: row count per domain = 10, each row ≥1 C-rule) | P0 | 4 | covered |
|
||||
| ATELIER-81 | Extend `matrix/domain-coverage.md` with gitops-operators, ai-ml, i18n, compliance (row schema: domain, P-count, derived-doc-count, manifest-listed, status) | P1 | 4 | covered |
|
||||
| ATELIER-82 | Extend `review/agent-checklist.md` with 4 new domain trigger sections | P1 | 4 | covered |
|
||||
| ATELIER-83 | Extend `review/peer-review-checklist.md` with 4 new domain sections (parity with agent-checklist) | P1 | 4 | covered |
|
||||
| ATELIER-84 | Extend `review/anti-patterns.md` with 4 new domain violations incl. orphaned P-rule + deployable example artifact | P1 | 4 | covered |
|
||||
| ATELIER-85 | Update `MANIFEST.md` to list all new v0.3 documents (manifest authoritative) | P0 | 4 | covered |
|
||||
| ATELIER-86 | `examples/good/gitops-pr.md` + `examples/good/ai-ml-reproducibility.md` — 2 good examples (markdown with fenced code only) | P2 | 5 | covered |
|
||||
| ATELIER-87 | `examples/bad/i18n-string-concat.md` + `examples/bad/compliance-audit-log.md` — 2 named bad examples (each cites the P-rule breached) | P2 | 5 | covered |
|
||||
| ATELIER-88 | Cross-links from new domains to existing devops/security/observability/data/kubernetes/infrastructure-as-code domains (review check: every new derived doc ≥1 outbound cross-link to a MANIFEST-listed doc) | P1 | 5 | covered |
|
||||
| ATELIER-89 | Final review passes (all v0.3 phases reviewed, audit clean) | P0 | 6 | covered |
|
||||
| ATELIER-90 | Milestone v0.3 released (tag v0.2.6, merged to main) | P0 | 6 | covered |
|
||||
| ATELIER-91 | Add `examples/` directory listing to `MANIFEST.md` (pre-existing drift from v0.2 audit escalation ESC-002 note: examples/ unlisted; manifest is authoritative) | P1 | 4 | covered |
|
||||
| ATELIER-60 | `domains/gitops-operators/first-principles.md` — 10 GitOps/operator principles (P1–P10) | P0 | 1 | pending |
|
||||
| ATELIER-61 | `domains/gitops-operators/argocd.md` — ArgoCD derived doc | P1 | 1 | pending |
|
||||
| ATELIER-62 | `domains/gitops-operators/flux.md` — Flux derived doc | P1 | 1 | pending |
|
||||
| ATELIER-63 | `domains/gitops-operators/operators.md` — Kubernetes Operators derived doc | P1 | 1 | pending |
|
||||
| ATELIER-64 | `domains/gitops-operators/progressive-delivery.md` — progressive delivery derived doc | P1 | 1 | pending |
|
||||
| ATELIER-65 | `domains/ai-ml/first-principles.md` — 10 AI/ML principles (P1–P10) | P0 | 2 | pending |
|
||||
| ATELIER-66 | `domains/ai-ml/data-versioning.md` — data/model versioning derived doc | P1 | 2 | pending |
|
||||
| ATELIER-67 | `domains/ai-ml/model-evaluation.md` — evaluation derived doc | P1 | 2 | pending |
|
||||
| ATELIER-68 | `domains/ai-ml/serving.md` — model serving derived doc | P1 | 2 | pending |
|
||||
| ATELIER-69 | `domains/ai-ml/monitoring-drift.md` — monitoring/drift derived doc | P1 | 2 | pending |
|
||||
| ATELIER-70 | `domains/i18n/first-principles.md` — 10 i18n principles (P1–P10) | P0 | 3 | pending |
|
||||
| ATELIER-71 | `domains/i18n/locale-resources.md` — locale resource management derived doc | P1 | 3 | pending |
|
||||
| ATELIER-72 | `domains/i18n/formatting.md` — formatting (dates/numbers/units) derived doc | P1 | 3 | pending |
|
||||
| ATELIER-73 | `domains/i18n/rtl-bidi.md` — RTL/bidi layout derived doc | P1 | 3 | pending |
|
||||
| ATELIER-74 | `domains/i18n/testing-i18n.md` — i18n testing derived doc | P1 | 3 | pending |
|
||||
| ATELIER-75 | `domains/compliance/first-principles.md` — 10 compliance principles (P1–P10) | P0 | 3 | pending |
|
||||
| ATELIER-76 | `domains/compliance/audit-logs.md` — audit logging derived doc | P1 | 3 | pending |
|
||||
| ATELIER-77 | `domains/compliance/data-retention.md` — data retention derived doc | P1 | 3 | pending |
|
||||
| ATELIER-78 | `domains/compliance/policy-as-code.md` — policy-as-code derived doc | P1 | 3 | pending |
|
||||
| ATELIER-79 | `domains/compliance/evidence.md` — evidence collection derived doc | P1 | 3 | pending |
|
||||
| ATELIER-80 | Extend `matrix/principles-matrix.md` with 40 new P-rules → core C-rule mappings (10 per new domain; review check: row count per domain = 10, each row ≥1 C-rule) | P0 | 4 | pending |
|
||||
| ATELIER-81 | Extend `matrix/domain-coverage.md` with gitops-operators, ai-ml, i18n, compliance (row schema: domain, P-count, derived-doc-count, manifest-listed, status) | P1 | 4 | pending |
|
||||
| ATELIER-82 | Extend `review/agent-checklist.md` with 4 new domain trigger sections | P1 | 4 | pending |
|
||||
| ATELIER-83 | Extend `review/peer-review-checklist.md` with 4 new domain sections (parity with agent-checklist) | P1 | 4 | pending |
|
||||
| ATELIER-84 | Extend `review/anti-patterns.md` with 4 new domain violations incl. orphaned P-rule + deployable example artifact | P1 | 4 | pending |
|
||||
| ATELIER-85 | Update `MANIFEST.md` to list all new v0.3 documents (manifest authoritative) | P0 | 4 | pending |
|
||||
| ATELIER-86 | `examples/good/gitops-pr.md` + `examples/good/ai-ml-reproducibility.md` — 2 good examples (markdown with fenced code only) | P2 | 5 | pending |
|
||||
| ATELIER-87 | `examples/bad/i18n-string-concat.md` + `examples/bad/compliance-audit-log.md` — 2 named bad examples (each cites the P-rule breached) | P2 | 5 | pending |
|
||||
| ATELIER-88 | Cross-links from new domains to existing devops/security/observability/data/kubernetes/infrastructure-as-code domains (review check: every new derived doc ≥1 outbound cross-link to a MANIFEST-listed doc) | P1 | 5 | pending |
|
||||
| ATELIER-89 | Final review passes (all v0.3 phases reviewed, audit clean) | P0 | 6 | pending |
|
||||
| ATELIER-90 | Milestone v0.3 released (tag v0.2.6, merged to main) | P0 | 6 | pending |
|
||||
| ATELIER-91 | Add `examples/` directory listing to `MANIFEST.md` (pre-existing drift from v0.2 audit escalation ESC-002 note: examples/ unlisted; manifest is authoritative) | P1 | 4 | pending |
|
||||
|
||||
## v0.3 Traceability Matrix
|
||||
|
||||
@@ -230,166 +230,4 @@ All 35 requirements covered. 8 core principles, 11 domains, 110 domain principle
|
||||
| IDEATE-03 (domain-coverage row schema) | ATELIER-81 + IDEATE-19 extend with C-rule coverage table update | ✓ extended |
|
||||
| IDEATE-07 (named bad examples cite P-rule breached) | ATELIER-87 + IDEATE-26 refine (two-breach example) | ✓ extended |
|
||||
| IDEATE-10/11/12 (decision/comparison tables) | IDEATE-21/22/23 transfer the pattern to 3 v0.3 derived docs | ✓ transferred |
|
||||
| v0.2 audit ESC-002 note (examples/ not in MANIFEST) | IDEATE-17 → ATELIER-91 | ✓ addressed |
|
||||
|
||||
## v0.4 Requirements — Edge + Messaging + Language-Derived Docs
|
||||
|
||||
**Milestone type:** NFR (all phases produce docs)
|
||||
**Tag line:** v0.3.x (previous minor from v0.4)
|
||||
|
||||
| REQ-ID | Requirement | Priority | Phase | Status |
|
||||
|--------|-------------|----------|-------|--------|
|
||||
| ATELIER-92 | `domains/edge/first-principles.md` — 10 edge principles (P1–P10) | P0 | 1 | pending |
|
||||
| ATELIER-93 | `domains/edge/cdn.md` — CDN/edge-cache derived doc (with edge-cache vs origin decision matrix) | P1 | 1 | pending |
|
||||
| ATELIER-94 | `domains/edge/offline-first.md` — offline-first derived doc | P1 | 1 | pending |
|
||||
| ATELIER-95 | `domains/edge/iot.md` — constrained-device/IoT derived doc | P1 | 1 | pending |
|
||||
| ATELIER-96 | `domains/edge/sync.md` — conflict/sync derived doc (CRDT/last-write-win decision matrix) | P1 | 1 | pending |
|
||||
| ATELIER-97 | `domains/messaging/first-principles.md` — 10 messaging principles (P1–P10) | P0 | 2 | pending |
|
||||
| ATELIER-98 | `domains/messaging/queues.md` — queue derived doc (with at-least-once/at-most-once/exactly-once comparison) | P1 | 2 | pending |
|
||||
| ATELIER-99 | `domains/messaging/pubsub.md` — pub/sub derived doc | P1 | 2 | pending |
|
||||
| ATELIER-100 | `domains/messaging/streams.md` — stream processing derived doc | P1 | 2 | pending |
|
||||
| ATELIER-101 | `domains/messaging/delivery-semantics.md` — delivery semantics derived doc (idempotency, ordering, dead-letter) | P1 | 2 | pending |
|
||||
| ATELIER-102 | `languages/typescript.md` extended + `languages/ts-types.md`, `ts-tooling.md`, `ts-async.md`, `ts-testing.md` — 4 TS derived docs | P1 | 3 | pending |
|
||||
| ATELIER-103 | `languages/python.md` extended + `languages/py-types.md`, `py-tooling.md`, `py-async.md`, `py-testing.md` — 4 Python derived docs | P1 | 3 | pending |
|
||||
| ATELIER-104 | `languages/go.md` extended + `languages/go-types.md`, `go-tooling.md`, `go-concurrency.md`, `go-testing.md` — 4 Go derived docs | P1 | 3 | pending |
|
||||
| ATELIER-105 | `languages/rust.md` extended + `languages/rs-ownership.md`, `rs-tooling.md`, `rs-async.md`, `rs-testing.md` — 4 Rust derived docs | P1 | 3 | pending |
|
||||
| ATELIER-106 | Extend `matrix/principles-matrix.md` with 20 new P-rules → core C-rule mappings (10 per new domain; review check: row count per domain = 10, each row ≥1 C-rule) — post-v0.4: 19 domains, 190 P-rules | P0 | 4 | pending |
|
||||
| ATELIER-107 | Extend `matrix/domain-coverage.md` with edge + messaging (per-domain row schema) + add `languages/` coverage sub-table (language → derived docs → traced domains) | P1 | 4 | pending |
|
||||
| ATELIER-108 | Extend `review/agent-checklist.md` with edge + messaging trigger sections + a language-derived-docs section | P1 | 4 | pending |
|
||||
| ATELIER-109 | Extend `review/peer-review-checklist.md` with edge + messaging + language-derived sections (parity with agent-checklist) | P1 | 4 | pending |
|
||||
| ATELIER-110 | Extend `review/anti-patterns.md` with edge + messaging violations incl. orphaned P-rule + deployable example artifact; pre-specify domain chaos anti-patterns (edge: sync conflict loop, TTL-less cache; messaging: shared subscription, blocking consumer) | P1 | 4 | pending |
|
||||
| ATELIER-111 | Update `MANIFEST.md` to list new v0.4 domains + new `languages/` section (first-principles + derived docs per language) + post-v0.4 totals | P0 | 4 | pending |
|
||||
| ATELIER-112 | `examples/good/edge-offline-sync.md` + `examples/good/messaging-idempotent-consumer.md` — 2 good examples (markdown with fenced code only) | P2 | 5 | pending |
|
||||
| ATELIER-113 | `examples/bad/edge-sync-loop.md` + `examples/bad/messaging-shared-subscription.md` — 2 named bad examples (each cites the P-rule breached) | P2 | 5 | pending |
|
||||
| ATELIER-114 | Cross-links from new domains to existing devops/observability/data/concurrency/kubernetes/infrastructure-as-code domains; cross-links from language-derived docs to their traced domain docs (review check: every new derived doc ≥1 outbound cross-link to a MANIFEST-listed doc) | P1 | 5 | pending |
|
||||
| ATELIER-115 | Final review passes (all v0.4 phases reviewed, audit clean) | P0 | 6 | pending |
|
||||
| ATELIER-116 | Milestone v0.4 released (tag v0.3.6, merged to main) | P0 | 6 | pending |
|
||||
| ATELIER-117 | Update `ARCHITECTURE.md` for v0.4 — extend the component tree with `edge/` + `messaging/` domain entries, add a `## v0.4 Domain Additions` section documenting the two new domains (boundary per D-061/D-062, derived docs, cross-link targets), and note the `languages/` derived-doc extension (D-063 apply-don't-derive). Closes architecture-tree drift: the file currently stops at v0.3 with no edge/messaging/languages-derived coverage (IDEATE-32). | P1 | 4 | pending |
|
||||
|
||||
## v0.4 Traceability Matrix
|
||||
|
||||
| Phase | Requirements |
|
||||
|-------|-------------|
|
||||
| 0 (Pre-Execution) | (governance: spec, clarify, research, ideate, plan) |
|
||||
| 1 (Edge Domain) | ATELIER-92..ATELIER-96 |
|
||||
| 2 (Messaging Domain) | ATELIER-97..ATELIER-101 |
|
||||
| 3 (Language-Derived Extension) | ATELIER-102..ATELIER-105 |
|
||||
| 4 (Matrix + Review Integration) | ATELIER-106..ATELIER-111, ATELIER-117 |
|
||||
| 5 (Examples + Cross-Links) | ATELIER-112..ATELIER-114 |
|
||||
| 6 (Final Review + Ship) | ATELIER-115, ATELIER-116 |
|
||||
|
||||
## v0.4 Ideation Log
|
||||
|
||||
**Generated:** 13 ideas (mechanical: 8, backend-enriched: 5)
|
||||
**Accepted:** 13 (all v0.4-scope, confidence ≥ 0.79, above 0.6 autonomy threshold → auto-accepted)
|
||||
**Skipped:** 0
|
||||
**New requirements added:** 1 (ATELIER-117 — ARCHITECTURE.md v0.4 update, from IDEATE-32)
|
||||
**Refinements to existing reqs:** 12 (IDEATE-31, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43)
|
||||
**Deferred to v0.5:** 0
|
||||
**Rejected:** 0
|
||||
|
||||
| IDEATE-ID | Source | Category | Confidence | Decision | Mapped REQ |
|
||||
|-----------|--------|----------|------------|----------|------------|
|
||||
| IDEATE-31 | mechanical (MANIFEST drift, parallel to ESC-002) | drift | 0.86 | accepted → refines | ATELIER-111 (languages/ section must list 4 v0.1 stubs + 16 derived docs) |
|
||||
| IDEATE-32 | mechanical (ARCHITECTURE.md vs directory tree) | drift | 0.88 | accepted → new req | ATELIER-117 (update ARCHITECTURE.md for v0.4: component tree + v0.4 section) |
|
||||
| IDEATE-33 | mechanical (IDEATE-19 pattern transfer) | coverage | 0.90 | accepted → refines | ATELIER-107 (Core Principle Coverage C1–C8 → domains table update for edge + messaging) |
|
||||
| IDEATE-34 | mechanical (matrix row-count invariant) | drift | 0.82 | accepted → refines | ATELIER-106 (verify post-v0.4 = exactly 190; verify current = exactly 170 per grill G-014 — the "171 vs 170" claim was a false premise, corrected) |
|
||||
| IDEATE-35 | mechanical (D-069 vs req-row text) | spec | 0.89 | accepted → refines | ATELIER-100, ATELIER-101 (req text must reference streams + dead-letter comparison tables) |
|
||||
| IDEATE-36 | mechanical (RESEARCH anti-patterns vs ATELIER-110 text) | quality | 0.85 | accepted → refines | ATELIER-110 (language-derived-doc anti-patterns + .ts/.py/.go/.rs artifact-type extension) |
|
||||
| IDEATE-37 | backend-enriched (chaos anti-pattern enumeration) | chaos | 0.86 | accepted → refines | ATELIER-110 (include all 6 RESEARCH chaos anti-patterns: edge sync-loop/TTL-less/blocking-call; messaging shared-sub/blocking-consumer/unbounded-retry) |
|
||||
| IDEATE-38 | backend-enriched (IDEATE-28 pattern transfer) | improvement | 0.80 | accepted → refines | ATELIER-94 (offline-first.md maps offline write-queue patterns to testing pyramid tiers) |
|
||||
| IDEATE-39 | backend-enriched (IDEATE-29 pattern transfer) | improvement | 0.79 | accepted → refines | ATELIER-101 (delivery-semantics.md fenced idempotency-key dedup-store example, not prose-only) |
|
||||
| IDEATE-40 | backend-enriched (intra-milestone cross-link dependency) | chaos | 0.84 | accepted → refines | ATELIER-114 (P5 cross-link audit must verify edge↔messaging intra-v0.4 links resolve in both directions) |
|
||||
| IDEATE-41 | backend-enriched (IDEATE-30 pattern transfer) | improvement | 0.81 | accepted → refines | ATELIER-100 (streams.md enumerates consumer-group rebalance strategies: sticky/cooperative/eager) |
|
||||
| IDEATE-42 | mechanical (D-066 sub-table verifiability) | coverage | 0.87 | accepted → refines | ATELIER-107 (Languages Coverage sub-table = exactly 16 rows; P4 review check) |
|
||||
| IDEATE-43 | mechanical (IDEATE-18 pattern transfer) | coverage | 0.83 | accepted → refines | ATELIER-111 (MANIFEST Cross-Cutting summary = "19 domains, 190 P-rules post-v0.4") |
|
||||
|
||||
### Refinements Notes (applied to existing reqs at execute time, not changing req rows)
|
||||
|
||||
- **ATELIER-106** (IDEATE-34, corrected by grill G-014): matrix coverage summary must read
|
||||
"post-v0.4: 19 domains, 190 P-rules" in BOTH the summary block and
|
||||
the per-domain section count (IDEATE-18 carried forward). P4 must
|
||||
verify the matrix contains exactly 190 P-rows (10 per domain × 19).
|
||||
The current matrix has **exactly 170 P-rows** (17 domains × 10,
|
||||
verified by grill G-014 direct count) — the IDEATE-34 "171 vs 170"
|
||||
claim was a FALSE PREMISE. P4 must verify 170 exists, append 20 rows,
|
||||
and confirm post-v0.4 count = 190. Do NOT hunt for or delete a phantom
|
||||
171st row.
|
||||
- **ATELIER-107** (IDEATE-33, IDEATE-42): the domain-coverage.md
|
||||
extension must update THREE artifacts: (a) the per-domain row-schema
|
||||
table with edge + messaging rows; (b) the "Core Principle Coverage"
|
||||
table (C1–C8 → domains) for the 2 new domains (C4 Locality grows by
|
||||
edge P1/P8 + messaging P7; C5 Reversibility grows by edge P2/P4/P7/P10
|
||||
+ messaging P5/P8/P9; C7 Observability grows by edge P10 + messaging
|
||||
P8/P10; C8 Economy grows by edge P3/P9 + messaging P6; C2 grows by
|
||||
messaging P1/P2/P4) — parallel to v0.3 IDEATE-19; (c) the new
|
||||
`## Languages Coverage` sub-table with EXACTLY 16 rows (one per
|
||||
language-derived doc), verifiable as a P4 review check (no orphans,
|
||||
every derived doc traced to ≥1 domain P-rule).
|
||||
- **ATELIER-110** (IDEATE-36, IDEATE-37): the anti-patterns extension
|
||||
must include (a) the v0.4 deployable-artifact-types table extended to
|
||||
`.ts`/`.py`/`.go`/`.rs` standalone files (language-derived docs risk,
|
||||
per D-020 docs-only contract); (b) language-derived-doc anti-
|
||||
patterns: introducing a new P-rule (e.g., "TS P1"), orphan section
|
||||
with no domain P-rule trace, runtime standalone artifact, and
|
||||
first-principles stub gaining P1–P10 numbering; (c) all 6 RESEARCH-
|
||||
specified chaos anti-patterns, not just the 4 named in the req-row
|
||||
text — edge: sync conflict loop (P4), TTL-less cache (P6), blocking
|
||||
call on constrained device (P3/P5); messaging: shared subscription
|
||||
(P2/P3), blocking consumer (P6), unbounded retry / no DLQ (P5).
|
||||
- **ATELIER-100** (IDEATE-35, IDEATE-41): streams.md must include (a)
|
||||
the stream-platform comparison table (Kafka vs Kinesis vs
|
||||
Pulsar/NATS JetStream — ordering, partitioning, replay/retention,
|
||||
use-case fit) per D-069, AND (b) an enumeration of consumer-group
|
||||
rebalance strategies (sticky/cooperative vs eager) as a sub-
|
||||
comparison — each with partition-stop-the-world cost and use-case
|
||||
fit. Parallel to v0.3 IDEATE-30 (drift-type enumeration).
|
||||
- **ATELIER-101** (IDEATE-35, IDEATE-39): delivery-semantics.md must
|
||||
include (a) the dead-letter strategy comparison table (retry-count-
|
||||
limit vs TTL-with-backoff vs poison-queue vs DLQ+alert) per D-069,
|
||||
AND (b) a fenced idempotency-key dedup-store example (not prose-
|
||||
only) — parallel to v0.3 IDEATE-29 (fenced signed-attestation
|
||||
example). Prose-only delivery-semantics guidance is weak; a fenced
|
||||
consumer-with-dedup-store demonstrates P3 (idempotent consumers)
|
||||
concretely.
|
||||
- **ATELIER-111** (IDEATE-31, IDEATE-43): the new `## Languages`
|
||||
MANIFEST section must list the 4 v0.1 first-principles stubs
|
||||
(typescript.md, python.md, go.md, rust.md — closing the pre-existing
|
||||
drift since v0.1, parallel to ESC-002/ATELIER-91) AND the 16 new
|
||||
derived docs. The Cross-Cutting summary line must update to exactly
|
||||
"19 domains, 190 P-rules post-v0.4" (IDEATE-18 carried forward).
|
||||
- **ATELIER-114** (IDEATE-40): the P5 cross-link audit must verify
|
||||
edge↔messaging intra-v0.4 links resolve in BOTH directions, not only
|
||||
new→existing. Edge/iot.md (P1) cross-links messaging/queues.md (MQTT
|
||||
QoS parallels delivery semantics) and messaging/iot.md-style
|
||||
references — but messaging/ docs are authored in P2, so at the P1
|
||||
tag (v0.3.1) those links dangle. This is acceptable per vertical-
|
||||
slice integrity (D-053 — matrix/cross-links close in P4/P5), but the
|
||||
P5 audit (ATELIER-114) must explicitly check that every edge→
|
||||
messaging and messaging→edge link resolves by the P6 ship. The
|
||||
audit must also confirm language-derived docs (P3) link to BOTH
|
||||
new (edge/messaging) and existing domains where relevant (e.g.,
|
||||
ts-async.md → messaging/delivery-semantics; go-concurrency.md →
|
||||
concurrency/patterns + messaging/queues).
|
||||
- **ATELIER-94** (IDEATE-38): offline-first.md must map offline write-
|
||||
queue / conflict-detection patterns to the testing pyramid tiers
|
||||
(unit: conflict detection on a merge function; integration:
|
||||
reconnect reconcile against a local store; e2e: partition
|
||||
simulation with a fake network) — parallel to v0.3 IDEATE-28 (i18n
|
||||
pseudo-locale → pyramid). Avoids generic "test offline" guidance.
|
||||
|
||||
### Within-Project Pattern Transfer (v0.1 → v0.2 → v0.3 → v0.4) — verified
|
||||
|
||||
| Prior Lesson | v0.4 Application | Status |
|
||||
|--------------|------------------|--------|
|
||||
| IDEATE-10/11/12 + 21/22/23 (decision/comparison tables in multi-tool derived docs) | D-069 + IDEATE-35 transfer the pattern to streams.md (Kafka/Kinesis/Pulsar) + delivery-semantics.md (dead-letter strategies) | ✓ transferred |
|
||||
| IDEATE-17 (manifest drift closure — examples/ unlisted → ATELIER-91) | IDEATE-31 → ATELIER-111 (languages/ unlisted since v0.1 — closes pre-existing drift on the same ESC-002 precedent) | ✓ transferred |
|
||||
| IDEATE-18 (coverage summary totals must update — string + per-domain count) | IDEATE-43 → ATELIER-111 (Cross-Cutting = "19 domains, 190 P-rules post-v0.4") + IDEATE-34 → ATELIER-106 (summary block) | ✓ transferred |
|
||||
| IDEATE-19 (Core Principle Coverage C1–C8 → domains table update) | IDEATE-33 → ATELIER-107 (C-rule coverage table extended for edge + messaging with the documented C-count shifts) | ✓ transferred |
|
||||
| IDEATE-20 (anti-patterns specificity: domain violations + artifact types) | IDEATE-36 + IDEATE-37 → ATELIER-110 (language-derived anti-patterns + .ts/.py/.go/.rs artifact types + all 6 chaos anti-patterns) | ✓ extended |
|
||||
| IDEATE-07 (named bad examples cite the P-rule breached) | ATELIER-113 + D-068 already cover (single-breach, cite specific P-rule) | ✓ carried forward |
|
||||
| IDEATE-08 (cross-link verification: every new derived doc ≥1 outbound) | ATELIER-114 already covers; IDEATE-40 extends to edge↔messaging intra-milestone links | ✓ extended |
|
||||
| IDEATE-13/14 (chaos: orphan P-rule + deployable artifact) | IDEATE-36 → ATELIER-110 (language orphan-section + standalone .ts/.py/.go/.rs runtime artifact) | ✓ extended |
|
||||
| IDEATE-28 (pseudo-locale testing → testing pyramid tiers) | IDEATE-38 → ATELIER-94 (offline write-queue testing → pyramid tiers) | ✓ transferred |
|
||||
| IDEATE-29 (fenced signed-attestation example, not prose-only) | IDEATE-39 → ATELIER-101 (fenced idempotency-key dedup-store example) | ✓ transferred |
|
||||
| IDEATE-30 (drift-type enumeration: data/concept/prediction + signal each) | IDEATE-41 → ATELIER-100 (rebalance-strategy enumeration: sticky/cooperative/eager + cost each) | ✓ transferred |
|
||||
| v0.2 audit ESC-002 note (examples/ not in MANIFEST) | IDEATE-17 → ATELIER-91 | ✓ addressed |
|
||||
File diff suppressed because it is too large
Load Diff
@@ -88,9 +88,9 @@ NFR milestone: no separate minor tag. The final patch (v0.1.5) IS the v0.2 deliv
|
||||
| 1 | GitOps + Operators Domain | docs | complete | domains/gitops-operators/{first-principles, argocd, flux, operators, progressive-delivery}.md — shipped v0.2.1 |
|
||||
| 2 | AI/ML Domain | docs | complete | domains/ai-ml/{first-principles, data-versioning, model-evaluation, serving, monitoring-drift}.md — shipped v0.2.2 |
|
||||
| 3 | i18n + Compliance Domains | docs | complete | domains/i18n/{first-principles, locale-resources, formatting, rtl-bidi, testing-i18n}.md, domains/compliance/{first-principles, audit-logs, data-retention, policy-as-code, evidence}.md — shipped v0.2.3 |
|
||||
| 4 | Matrix + Review Integration | docs | complete | matrix/principles-matrix.md (+40 mappings), matrix/domain-coverage.md (incl. C-rule coverage table update), review/{agent-checklist, peer-review-checklist, anti-patterns}.md, MANIFEST.md (+ examples/ listing per ATELIER-91) — shipped v0.2.4 |
|
||||
| 5 | Examples + Cross-Links | docs | complete | examples/good + examples/bad for 4 domains, cross-links to devops/security/observability/data/k8s/iac — shipped v0.2.5 |
|
||||
| 6 | Final Review + Ship | docs | complete | Review passed, audit clean, milestone merged to main, tag v0.2.6 |
|
||||
| 4 | Matrix + Review Integration | docs | pending | matrix/principles-matrix.md (+40 mappings), matrix/domain-coverage.md (incl. C-rule coverage table update), review/{agent-checklist, peer-review-checklist, anti-patterns}.md, MANIFEST.md (+ examples/ listing per ATELIER-91) |
|
||||
| 5 | Examples + Cross-Links | docs | pending | examples/good + examples/bad for 4 domains, cross-links to devops/security/observability/data/k8s/iac |
|
||||
| 6 | Final Review + Ship | docs | pending | Review passed, audit clean, milestone merged to main, tag v0.2.6 |
|
||||
|
||||
## v0.3 Phase Tag Mapping
|
||||
|
||||
@@ -119,44 +119,7 @@ NFR milestone: no separate minor tag. The final patch (v0.2.6) IS the v0.3 deliv
|
||||
|
||||
## Future Milestones
|
||||
|
||||
- **v0.4** (ACTIVE): `domains/edge/`, `domains/messaging/`, `languages/`-derived docs (typescript/python/go/rust → 4 derived docs each).
|
||||
- **v0.5** (candidates): `domains/quantum/`, tooling adapters (linters), translation/localization of framework docs.
|
||||
|
||||
## Milestone: v0.4 — Edge + Messaging + Language-Derived Docs (ACTIVE)
|
||||
|
||||
**Milestone type:** NFR (all phases produce docs — no `feat` code)
|
||||
**Tag line:** v0.3.x (previous minor from v0.4)
|
||||
**Phases:** P0 (pre-execution) + P1–P5 (execution) + P6 (final review+ship)
|
||||
|
||||
| Phase | Name | Type | Status | Key Deliverables |
|
||||
|-------|------|------|--------|------------------|
|
||||
| 0 | Pre-Execution | docs | complete | Spec, clarify, research, ideate, plan, PERSONAS.md (adds edge-engineer + languages-engineer phase-specific personas) — shipped v0.3.0 |
|
||||
| 1 | Edge Domain | docs | complete | domains/edge/{first-principles, cdn, offline-first, iot, sync}.md — shipped v0.3.1 |
|
||||
| 2 | Messaging Domain | docs | complete | domains/messaging/{first-principles, queues, pubsub, streams, delivery-semantics}.md — shipped v0.3.2 |
|
||||
| 3 | Language-Derived Extension | docs | complete | languages/ × 4 → first-principles + 4 derived docs each (16 derived docs) — shipped v0.3.3 |
|
||||
| 4 | Matrix + Review Integration | docs | complete | matrix/principles-matrix.md (+20 mappings, 170→190), matrix/domain-coverage.md (+ languages/ sub-table), review/{agent-checklist, peer-review-checklist, anti-patterns}.md, MANIFEST.md (languages/ section), ARCHITECTURE.md — shipped v0.3.4 |
|
||||
| 5 | Examples + Cross-Links | docs | pending | examples/good + examples/bad for 2 domains, cross-links to devops/observability/data/concurrency/kubernetes/infrastructure-as-code + language→domain links |
|
||||
| 6 | Final Review + Ship | docs | pending | Review passed, audit clean, milestone merged to main, tag v0.3.6 |
|
||||
|
||||
## v0.4 Phase Tag Mapping
|
||||
|
||||
Per branch-strategy.md, milestone `v0.4` tags run on the `v0.3.x` patch line:
|
||||
|
||||
| Phase | Tag | Notes |
|
||||
|-------|-----|-------|
|
||||
| P0 | v0.3.0 | Pre-execution release |
|
||||
| P1 | v0.3.1 | Edge domain |
|
||||
| P2 | v0.3.2 | Messaging domain |
|
||||
| P3 | v0.3.3 | Language-derived extension |
|
||||
| P4 | v0.3.4 | Matrix + review integration |
|
||||
| P5 | v0.3.5 | Examples + cross-links |
|
||||
| P6 | v0.3.6 | Final review + ship — **IS the v0.4 milestone release** |
|
||||
|
||||
NFR milestone: no separate minor tag. The final patch (v0.3.6) IS the v0.4 deliverable.
|
||||
|
||||
## v0.4 Ideation Outcome
|
||||
|
||||
(Populated during P0 IDEATE stage.)
|
||||
- **v0.4** (candidates): `domains/edge/`, `domains/quantum/`, language-specific derived docs, tooling adapters (linters), translation/localization of framework docs.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
{
|
||||
"slug": "atelier",
|
||||
"name": "Atelier",
|
||||
"milestone": "v0.4",
|
||||
"milestone": "v0.3",
|
||||
"status": "active"
|
||||
}
|
||||
],
|
||||
|
||||
+24
-39
@@ -40,54 +40,39 @@
|
||||
| AI / ML | ✓ | data-versioning, model-evaluation, serving, monitoring-drift |
|
||||
| i18n | ✓ | locale-resources, formatting, rtl-bidi, testing-i18n |
|
||||
| Compliance | ✓ | audit-logs, data-retention, policy-as-code, evidence |
|
||||
| Edge | ✓ | cdn, offline-first, iot, sync |
|
||||
| Messaging | ✓ | queues, pubsub, streams, delivery-semantics |
|
||||
|
||||
## Languages
|
||||
|
||||
> Language docs apply (not derive) domain principles. They introduce
|
||||
> no new P-rules and no new matrix rows (D-063, D-066). First-
|
||||
> principles stubs from v0.1; derived docs added in v0.4.
|
||||
|
||||
| Language | First Principles | Derived Documents |
|
||||
|----------|------------------|-------------------|
|
||||
| TypeScript | ✓ (v0.1 stub) | ts-types, ts-tooling, ts-async, ts-testing |
|
||||
| Python | ✓ (v0.1 stub) | py-types, py-tooling, py-async, py-testing |
|
||||
| Go | ✓ (v0.1 stub) | go-types, go-tooling, go-concurrency, go-testing |
|
||||
| Rust | ✓ (v0.1 stub) | rs-ownership, rs-tooling, rs-async, rs-testing |
|
||||
|
||||
## Examples
|
||||
|
||||
> Examples are illustrative markdown with fenced code only (no standalone runtime artifacts per D-020 / D-025). The `examples/` directory listing closes the v0.2 ESC-002 drift (IDEATE-17, ATELIER-91). P5 authored the v0.3 examples and promoted all entries from `pending` to `✓` (verified — every listed file exists).
|
||||
> Examples are illustrative markdown with fenced code only (no standalone runtime artifacts per D-020 / D-025). The `examples/` directory listing closes the v0.2 ESC-002 drift (IDEATE-17, ATELIER-91). P5 examples populate this section.
|
||||
|
||||
| Path | Status | Notes |
|
||||
|------|--------|-------|
|
||||
| `examples/good/` | ✓ | Good-example directory — 8 examples (v0.1 + v0.2 + v0.3) |
|
||||
| `examples/bad/` | ✓ | Bad-example directory — 7 examples (v0.1 + v0.2 + v0.3) |
|
||||
| `examples/good/api-endpoint.md` | ✓ | v0.1 example — good REST endpoint |
|
||||
| `examples/good/react-component.md` | ✓ | v0.1 example — good React component |
|
||||
| `examples/good/db-schema.md` | ✓ | v0.1 example — good DB schema |
|
||||
| `examples/good/error-handler.md` | ✓ | v0.1 example — good error handler |
|
||||
| `examples/bad/god-object.md` | ✓ | v0.1 example — bad god object |
|
||||
| `examples/bad/silent-error.md` | ✓ | v0.1 example — bad silent error |
|
||||
| `examples/bad/leaky-abstraction.md` | ✓ | v0.1 example — bad leaky abstraction |
|
||||
| `examples/good/terraform-module.md` | ✓ | v0.2 example — good IaC module |
|
||||
| `examples/good/k8s-deployment.md` | ✓ | v0.2 example — good k8s deployment |
|
||||
| `examples/bad/terraform-unlocked-state.md` | ✓ | v0.2 example — bad unlocked state |
|
||||
| `examples/bad/k8s-bare-pod-no-resources.md` | ✓ | v0.2 example — bad bare pod |
|
||||
| `examples/good/gitops-pr.md` | ✓ | v0.3 example — good GitOps PR |
|
||||
| `examples/good/ai-ml-reproducibility.md` | ✓ | v0.3 example — good reproducible training run |
|
||||
| `examples/bad/i18n-string-concat.md` | ✓ | v0.3 example — bad i18n string concat |
|
||||
| `examples/bad/compliance-audit-log.md` | ✓ | v0.3 example — bad audit log (P1 + P9 breaches) |
|
||||
| `examples/good/` | (pending P5) | Good-example directory — to be populated by v0.3 P5 |
|
||||
| `examples/bad/` | (pending P5) | Bad-example directory — to be populated by v0.3 P5 |
|
||||
| `examples/good/api-endpoint.md` | (pending P5) | v0.1 example — to be listed when P5 back-fills |
|
||||
| `examples/good/react-component.md` | (pending P5) | v0.1 example — to be listed when P5 back-fills |
|
||||
| `examples/good/db-schema.md` | (pending P5) | v0.1 example — to be listed when P5 back-fills |
|
||||
| `examples/good/error-handler.md` | (pending P5) | v0.1 example — to be listed when P5 back-fills |
|
||||
| `examples/bad/god-object.md` | (pending P5) | v0.1 example — to be listed when P5 back-fills |
|
||||
| `examples/bad/silent-error.md` | (pending P5) | v0.1 example — to be listed when P5 back-fills |
|
||||
| `examples/bad/leaky-abstraction.md` | (pending P5) | v0.1 example — to be listed when P5 back-fills |
|
||||
| `examples/good/terraform-module.md` | (pending P5) | v0.2 example — to be listed when P5 back-fills |
|
||||
| `examples/good/k8s-deployment.md` | (pending P5) | v0.2 example — to be listed when P5 back-fills |
|
||||
| `examples/bad/terraform-unlocked-state.md` | (pending P5) | v0.2 example — to be listed when P5 back-fills |
|
||||
| `examples/bad/k8s-bare-pod-no-resources.md` | (pending P5) | v0.2 example — to be listed when P5 back-fills |
|
||||
| `examples/good/gitops-pr.md` | (pending P5) | v0.3 example — to be authored in P5 |
|
||||
| `examples/good/ai-ml-reproducibility.md` | (pending P5) | v0.3 example — to be authored in P5 |
|
||||
| `examples/bad/i18n-string-concat.md` | (pending P5) | v0.3 example — to be authored in P5 |
|
||||
| `examples/bad/compliance-audit-log.md` | (pending P5) | v0.3 example — to be authored in P5 |
|
||||
|
||||
> **Note:** The `examples/` section was established in P4 with entries pre-listed as `pending P5`. P5 authored the 4 v0.3 examples and promoted all entries to `✓` after verifying every listed file exists on disk. The manifest remains authoritative — unlisted = not part of framework.
|
||||
> **Note:** The `examples/` section structure is established here (P4) so P5 can populate it. Files marked "pending P5" do not yet exist; they will be authored in Phase 5 and promoted from `pending` to `✓` upon completion. Listing them here as `pending` makes the manifest authoritative about what *will* exist and prevents drift.
|
||||
|
||||
## Cross-Cutting
|
||||
|
||||
| Document | Purpose |
|
||||
|-----------------------------------|----------------------------------|
|
||||
| `matrix/principles-matrix.md` | Maps domain → core principles (19 domains, 190 P-rules post-v0.4) |
|
||||
| `matrix/domain-coverage.md` | Maps core → domains; per-domain coverage (incl. v0.4 Core Principle Coverage + Languages Coverage sub-table) |
|
||||
| `review/agent-checklist.md` | Pre-completion agent checklist (incl. edge + messaging + language-derived triggers) |
|
||||
| `review/peer-review-checklist.md` | Human peer-review checklist (incl. edge + messaging + language-derived sections) |
|
||||
| `review/anti-patterns.md` | Catalog of violations (incl. edge + messaging + language-derived + v0.4 chaos anti-patterns + .ts/.py/.go/.rs artifact types) |
|
||||
| `matrix/principles-matrix.md` | Maps domain → core principles (17 domains, 170 P-rules post-v0.3) |
|
||||
| `matrix/domain-coverage.md` | Maps core → domains; per-domain coverage (incl. v0.3 Core Principle Coverage) |
|
||||
| `review/agent-checklist.md` | Pre-completion agent checklist (incl. IaC + k8s + gitops + ai-ml + i18n + compliance triggers) |
|
||||
| `review/peer-review-checklist.md` | Human peer-review checklist (incl. IaC + k8s + gitops + ai-ml + i18n + compliance sections) |
|
||||
| `review/anti-patterns.md` | Catalog of violations (incl. IaC + k8s + gitops + ai-ml + i18n + compliance + v0.3 chaos anti-patterns) |
|
||||
@@ -1,227 +0,0 @@
|
||||
# CDN — Derived Rules
|
||||
|
||||
> Derives from `domains/edge/first-principles.md`. Applies P1
|
||||
> (Proximity is the Design Driver) and P6 (Cache Invalidation is
|
||||
> Explicit) primarily, with P5 (idempotent cache fill), P8
|
||||
> (geographic distribution), and P9 (identity at the edge). For the
|
||||
> edge-cache-vs-origin decision, see the decision matrix below.
|
||||
> Cross-links `domains/performance/frontend` for generic caching,
|
||||
> `domains/security/input-validation` for cache poisoning, and
|
||||
> `domains/observability/metrics` for cache-hit ratio.
|
||||
|
||||
## What a CDN Is (P1 Proximity is the Design Driver)
|
||||
|
||||
- A content delivery network is a fleet of PoPs (points of presence)
|
||||
placed near users. The PoP serves cached content; the origin is
|
||||
the authoritative source. The CDN's whole purpose is P1: compute
|
||||
(the cache) is placed near the user so the round trip to the origin
|
||||
does not bound latency. Latency is a correctness constraint at the
|
||||
edge (C1), not a performance preference.
|
||||
- The CDN is the canonical edge-cache architecture (Akamai,
|
||||
Cloudflare, Fastly): PoPs near users, origin shielding, cache-key
|
||||
normalization, purge APIs. Atelier derives the
|
||||
placement/invalidation principles, not the vendor config.
|
||||
- The boundary with `domains/performance/frontend` is per D-061:
|
||||
performance owns *generic* caching and optimization (cache what is
|
||||
expensive, stable, read often — `performance/P5 Caching with
|
||||
Intent`); edge owns the *geographic, partition-aware* placement and
|
||||
invalidation angle. A CDN is an edge concern because its defining
|
||||
trait is geographic distribution (P8) and partition-aware
|
||||
invalidation (P6), not measurement.
|
||||
|
||||
## Cache Key Design (P6 Cache Invalidation is Explicit)
|
||||
|
||||
- The cache key is the contract between the URL and the cached
|
||||
representation. A key that varies on the wrong dimensions serves
|
||||
the wrong content; a key that varies on too many dimensions
|
||||
collapses the hit ratio. Key design *is* the invalidation
|
||||
surface: a key that includes a content hash or version segment
|
||||
makes invalidation explicit; a key that ignores `Vary` headers
|
||||
serves stale variants.
|
||||
- Normalize the key: lower-case the host, strip default ports,
|
||||
sort query parameters, ignore tracking parameters. A
|
||||
non-normalized key is a cache-poisoning vector (see
|
||||
`domains/security/input-validation`) and a hit-ratio destroyer
|
||||
(see `domains/observability/metrics`).
|
||||
- A cache with no explicit key strategy is a TTL-less cache under
|
||||
partition (P6 violation): staleness is silent and unbounded.
|
||||
|
||||
```http
|
||||
# Cache key derivation: vary on what changes content, ignore what
|
||||
# does not. The key is the tuple (host, normalized-path, sorted-
|
||||
# query, Vary-headers); the cache entry is the representation + TTL.
|
||||
Cache-Key: example.com /api/v1/products?sort=price®ion=us Vary:Accept-Encoding
|
||||
Cache-Control: public, max-age=60, s-maxage=600, stale-while-revalidate=300
|
||||
Vary: Accept-Encoding
|
||||
```
|
||||
|
||||
- `max-age` bounds the browser cache; `s-maxage` bounds the CDN
|
||||
PoP; `stale-while-revalidate` allows serving stale while
|
||||
refetching. Each is an explicit invalidation strategy (P6).
|
||||
|
||||
## TTL vs Explicit Invalidation (P6, C3 Simplicity)
|
||||
|
||||
- **TTL-based invalidation** (`max-age`, `s-maxage`): the cache entry
|
||||
expires after a duration. Simple, no origin contact required to
|
||||
invalidate, but bounded staleness is the contract — the entry may
|
||||
be stale up to TTL. Fits content where eventual consistency is
|
||||
acceptable (asset fingerprints, lists, derived images).
|
||||
- **Explicit invalidation** (purge, surrogate keys): the operator
|
||||
signals the cache to drop entries. Tighter staleness bounds, but
|
||||
requires the origin or operator to know which entries to purge.
|
||||
Fits content where staleness is a correctness defect (price
|
||||
updates, availability, breaking news).
|
||||
- A TTL-less cache with no explicit invalidation is a P6 violation:
|
||||
stale-forever under partition. Every cache must have one or the
|
||||
other (or both), and the choice is documented per content type.
|
||||
|
||||
## Cache-Hit / Miss / Origin-Fetch (P5, P6)
|
||||
|
||||
- **Hit**: the PoP serves from cache. Latency is PoP-local (P1).
|
||||
- **Miss**: the PoP has no entry; it fetches from the origin (or an
|
||||
origin-shield PoP). The fetch must be idempotent (P5) — a retried
|
||||
miss must not corrupt the cache or double-write side effects.
|
||||
- **Revalidate**: the PoP holds a stale entry and asks the origin
|
||||
(`If-None-Match`, `If-Modified-Since`); a 304 refreshes the TTL
|
||||
without re-fetching the body. Revalidation is the bandwidth-economical
|
||||
middle ground (C8).
|
||||
|
||||
```http
|
||||
# Conditional revalidation — the PoP asks the origin "is this still
|
||||
# current?" The 304 response refreshes the TTL without a body.
|
||||
GET /api/v1/products HTTP/1.1
|
||||
Host: example.com
|
||||
If-None-Match: "etag-7a3f"
|
||||
|
||||
HTTP/1.1 304 Not Modified
|
||||
ETag: "etag-7a3f"
|
||||
Cache-Control: s-maxage=600
|
||||
```
|
||||
|
||||
- A cache-hit ratio that is not measured is a gate on noise — see
|
||||
`domains/observability/metrics` for the SLI/SLO discipline that
|
||||
makes the hit ratio a meaningful signal. A CDN with no hit-ratio
|
||||
metric is operating blind (P10 analog).
|
||||
|
||||
## Origin Shielding (P1, P8, C8 Economy)
|
||||
|
||||
- Origin shielding routes all origin fetches through a single
|
||||
shield PoP (or shield region). The shield absorbs the
|
||||
thundering-herd: 10 000 PoPs missing the same URL fetch the origin
|
||||
once, not 10 000 times. This is C8 Economy (origin bandwidth is
|
||||
bounded) and P1 (the shield is itself a proximity layer for the
|
||||
origin).
|
||||
- Shielding is a geographic decision (P8): the shield sits in a
|
||||
region close to the origin, not close to the user. The shield is
|
||||
the inner ring of the CDN; the user-facing PoPs are the outer ring.
|
||||
- A CDN without origin shielding under a stampede will overload the
|
||||
origin; a shield that is itself partitioned from the origin must
|
||||
degrade gracefully (P7) — serve stale per `stale-while-revalidate`
|
||||
rather than 500.
|
||||
|
||||
## Purge Strategies (P6, C3 Simplicity)
|
||||
|
||||
| Strategy | Granularity | Latency to Invalidate | Cost | Best for |
|
||||
|----------|-------------|-----------------------|------|----------|
|
||||
| URL purge | One URL | Seconds | Low (one entry) | Surgical fixes, single-page corrections |
|
||||
| Soft purge | One URL (mark stale, serve while refetch) | Seconds | Low | High-traffic URLs where a hard purge causes a stampede |
|
||||
| Surrogate-key purge | A tag set (e.g., `product:123`, `category:shoes`) | Seconds | Medium (key indexing) | Related-content invalidation (a product update purges all its category pages) |
|
||||
| Wildcard purge | A path prefix or pattern | Seconds to minutes | High (scan) | Site-wide template changes |
|
||||
| All-cache purge | Everything | Seconds | Very high (origin stampede) | Disaster recovery only; never the steady-state invalidation path |
|
||||
|
||||
- Surrogate-key purge (Fastly, Akamai) is the highest-value
|
||||
strategy: tag cache entries with content keys, then purge by tag.
|
||||
This is explicit invalidation at scale (P6) without the origin
|
||||
stampede of an all-cache purge.
|
||||
- An all-cache purge as the steady-state invalidation path is a P6
|
||||
violation dressed as a feature — it pushes the origin load back to
|
||||
100% miss, defeating the CDN's purpose (P1).
|
||||
|
||||
## Cache Poisoning Prevention (P9, cross-link security/input-validation)
|
||||
|
||||
- A cache poisoned by a crafted request (a URL with a malicious
|
||||
header that gets cached and served to others) is a correctness
|
||||
defect (C1) and a security breach (P9 — the edge node is
|
||||
exploited). Prevent poisoning by:
|
||||
- Normalizing the cache key (strip untrusted query parameters,
|
||||
ignore unknown headers, lower-case the host).
|
||||
- Validating `Vary` against an allow-list; never `Vary: *` on a
|
||||
shared cache (poisonable via header injection).
|
||||
- Treating uncacheable responses (`Set-Cookie`,
|
||||
`Cache-Control: private`) as never-stored.
|
||||
- See `domains/security/input-validation` for the general
|
||||
input-validation discipline the cache key must follow. The cache
|
||||
key is a validation surface; a non-validated key is an attack
|
||||
surface.
|
||||
|
||||
## Multi-CDN Routing (P8 Geographic Distribution)
|
||||
|
||||
- A multi-CDN strategy routes each request to the best PoP across
|
||||
providers (Akamai + Cloudflare + Fastly). Routing is
|
||||
location-aware (P8): latency, cost, and availability vary by
|
||||
region and provider. The DNS layer (or a client-side router)
|
||||
selects the CDN per request.
|
||||
- Multi-CDN is a P8 decision, not a vendor-management decision:
|
||||
geographic distribution is the first-class constraint. A
|
||||
single-CDN deployment routes everything to one provider's PoPs;
|
||||
a multi-CDN deployment routes by region, latency, and cost.
|
||||
- Invalidation across multiple CDNs is harder (P6): each provider
|
||||
has its own purge API and surrogate-key scheme. A multi-CDN purge
|
||||
must fan out to all providers; a purge that reaches only one CDN
|
||||
leaves the others stale. Track purge completion per provider —
|
||||
see `domains/observability/metrics` for the per-CDN hit-ratio and
|
||||
purge-latency signals.
|
||||
|
||||
```http
|
||||
# A CDN config example: cache-control headers + a purge rule.
|
||||
# Origin response: declare the cache contract (P6).
|
||||
HTTP/1.1 200 OK
|
||||
Cache-Control: public, max-age=60, s-maxage=600, stale-while-revalidate=300
|
||||
Surrogate-Key: product:123 category:shoes
|
||||
ETag: "etag-7a3f"
|
||||
Vary: Accept-Encoding
|
||||
|
||||
# Purge rule (Fastly-style surrogate-key): when product 123
|
||||
# updates, purge every cache entry tagged product:123 OR
|
||||
# category:shoes. Explicit, bounded, no origin stampede (P6).
|
||||
POST /service/svc1/purge
|
||||
Surrogate-Key: product:123 category:shoes
|
||||
# Returns: {"status": "ok", "id": "purge-abc"} — poll the purge
|
||||
# status to confirm completion across all PoPs (P8, P10).
|
||||
```
|
||||
|
||||
## Edge-Cache vs Origin — Decision Matrix (D-069)
|
||||
|
||||
| Strategy | When | Latency | Origin Load | Correctness Risk |
|
||||
|----------|------|---------|-------------|------------------|
|
||||
| Serve from PoP (cache hit) | The PoP holds a fresh entry (within TTL or revalidated) | Lowest (PoP-local, P1) | None | Low — bounded by TTL staleness (P6) |
|
||||
| Serve stale while revalidate | The PoP holds a stale entry and `stale-while-revalidate` is set | Low (stale served immediately, refetch in background) | Background refetch (1 per entry) | Medium — stale served up to the revalidate window; acceptable for eventually-consistent content |
|
||||
| Fetch fresh from origin (miss) | The PoP has no entry, or the content is non-cacheable | High (origin round trip) | Full fetch per miss | Low — fresh by construction; the miss is the correctness floor |
|
||||
| Origin-shield fetch | Multiple PoPs miss the same URL; the shield collapses the herd | Medium (PoP → shield → origin) | Bounded to one origin fetch per shield (C8) | Low — shield is the inner ring; staleness bounded by shield TTL |
|
||||
| Purge and serve fresh | Explicit invalidation received (surrogate-key or URL purge) | Medium (purge propagates, then fresh fetch) | Full fetch post-purge | Lowest — explicit invalidation is the tightest staleness bound (P6) |
|
||||
| Serve from origin directly (bypass cache) | Content is non-cacheable (personalized, real-time) | Highest (every request hits origin) | Full fetch per request | Lowest for correctness, highest for origin load — use sparingly |
|
||||
|
||||
- The default is **serve from PoP** when fresh, **fetch fresh from
|
||||
origin** on miss with **origin-shield** to bound origin load, and
|
||||
**purge and serve fresh** when explicit invalidation is required.
|
||||
Bypass-the-cache is for non-cacheable content only — bypassing for
|
||||
cacheable content is a P1 violation (you have defeated the CDN).
|
||||
- The correctness risk column is bounded by the invalidation
|
||||
strategy (P6): every row except "bypass" carries staleness risk
|
||||
that is bounded by TTL or explicit purge. A row with no
|
||||
invalidation strategy is a P6 violation.
|
||||
|
||||
## What Violates CDN Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| TTL-less edge cache under partition (stale-forever, no explicit invalidation) | P6 Cache Invalidation is Explicit |
|
||||
| Cache key that varies on untrusted query parameters (poisonable) | P6, P9 (`domains/security/input-validation`) |
|
||||
| All-cache purge as the steady-state invalidation path (origin stampede) | P6, C8 Economy |
|
||||
| Non-normalized cache key (case-sensitive host, unsorted query) | P6, `domains/security/input-validation` |
|
||||
| Bypass-the-cache for cacheable content | P1 Proximity is the Design Driver (defeats the CDN) |
|
||||
| Multi-CDN with no per-CDN purge completion tracking | P8, P10 (stale cache invisible to the operator) |
|
||||
| Origin fetch that is not idempotent under retry | P5 Edge Operations are Idempotent |
|
||||
| Cache-hit ratio not measured | P10, `domains/observability/metrics` |
|
||||
| Single-CDN deployed where geographic distribution requires multi-CDN | P8 Geographic Distribution |
|
||||
| Shield PoP that 500s instead of serving stale under partition | P7 Partial Degradation is Engineered |
|
||||
@@ -1,218 +0,0 @@
|
||||
# Edge — First Principles
|
||||
|
||||
## 1. The Principles
|
||||
|
||||
### P1. Proximity is the Design Driver
|
||||
Compute, storage, and data are placed near the user or the data
|
||||
source. At the edge, latency is a correctness constraint (C1), not a
|
||||
performance preference — a late answer is a wrong answer when the
|
||||
round trip to a central region exceeds the user's or device's
|
||||
tolerance. This is the geographic expression of `C4 Locality`:
|
||||
performance's locality is algorithmic (data near compute); edge's
|
||||
locality is geographic (compute near user/data source). Placement is
|
||||
a design decision, not an accident of deployment, and it is
|
||||
constrained by `P8 Geographic Distribution`. The proximity angle is
|
||||
the distinguishing trait of the edge domain per D-061: this is what
|
||||
separates edge from `domains/performance/` (which owns *generic*
|
||||
measurement and optimization, not placement).
|
||||
|
||||
### P2. Offline is a First-Class State
|
||||
The system continues to operate when disconnected from the center.
|
||||
Partition is the norm, not the exception; reconciliation happens on
|
||||
reconnect, never assumed to be instant. An app that crashes on
|
||||
disconnect has no offline state and is unengineered. Offline
|
||||
operation derives from `C5 Reversibility` — the disconnected state
|
||||
is reversible back to consistency via reconciliation — and `C1
|
||||
Correctness`, because correctness under partition is the contract,
|
||||
not eventual correctness as a hedge. This is the foundation for
|
||||
`domains/edge/offline-first.md` and the precondition for the
|
||||
bounded-conflict discipline of `P4`.
|
||||
|
||||
### P3. Resources are Constrained and Declared
|
||||
Edge nodes — IoT sensors, gateways, point-of-sale devices, CDN PoP
|
||||
caches, 5G MEC nodes — have bounded CPU, memory, power, and
|
||||
bandwidth. Constraints are declared per node class, never assumed
|
||||
infinite. An undeclared budget is a defect: unbounded growth is a
|
||||
bug, and a constrained device with no budget will OOM or exhaust
|
||||
power. This derives from `C8 Economy` (use no more than the task
|
||||
requires) and `C1 Correctness` (a node that exceeds its bounds has
|
||||
failed). This is the edge-specific angle on `domains/performance/P4
|
||||
Resource Bounds` — performance owns the generic principle; edge owns
|
||||
the constrained-device reality. See `domains/edge/iot.md` for the
|
||||
per-device-class application.
|
||||
|
||||
### P4. Sync Conflicts are Bounded, Not Infinite
|
||||
Divergent state across partitioned nodes converges. Oscillation and
|
||||
infinite sync loops are correctness failures, not eventual
|
||||
consistency. A merge that never terminates is a livelock; a CRDT
|
||||
without merge semantics or an LWW without a monotonic clock can
|
||||
oscillate forever. This derives from `C1 Correctness` (convergence is
|
||||
a correctness contract) and `C5 Reversibility` (divergent state is
|
||||
reversible back to convergence). The bound may be eventual (CRDTs) or
|
||||
arbitrated (LWW with vector clocks), but it must exist. This is the
|
||||
foundation for `domains/edge/sync.md` and the rule the
|
||||
`edge-sync-loop` chaos anti-pattern breaches.
|
||||
|
||||
### P5. Edge Operations are Idempotent
|
||||
Sync, cache fill, and device commands are retried by nature — the
|
||||
network is partition-prone and the operation will be re-attempted.
|
||||
Idempotency keys (or deterministic operations) make retries safe. A
|
||||
non-idempotent edge write retried with side effects doubles the
|
||||
effect; a non-idempotent cache fill under retry corrupts the cache.
|
||||
This derives from `C1 Correctness`: correctness under retry is the
|
||||
contract, not a nice-to-have. This parallels `domains/messaging/P3
|
||||
Consumers are Idempotent` (cross-process delivery) and is the edge's
|
||||
device-and-cache-flavored analog — see `domains/edge/iot.md` for
|
||||
device command idempotency and `domains/edge/cdn.md` for cache-fill
|
||||
idempotency.
|
||||
|
||||
### P6. Cache Invalidation is Explicit
|
||||
Edge caches carry a defined invalidation or TTL strategy. A
|
||||
stale-forever cache under partition is a silent correctness defect;
|
||||
a TTL-less cache with no explicit invalidation is a bug, not a
|
||||
feature. This derives from `C1 Correctness` (cached state must be
|
||||
correct) and `C3 Simplicity` (a defined invalidation strategy is
|
||||
simpler and clearer than ad-hoc staleness). This is distinct from
|
||||
`domains/performance/P5 Caching with Intent`, which owns *generic*
|
||||
caching and optimization; edge owns the *geographic,
|
||||
partition-aware* invalidation angle — when a PoP is partitioned from
|
||||
the origin, the invalidation strategy is the correctness mechanism.
|
||||
See `domains/edge/cdn.md` for purge strategies (URL vs soft vs
|
||||
surrogate-key) and the edge-cache-vs-origin decision matrix.
|
||||
|
||||
### P7. Partial Degradation is Engineered
|
||||
The system degrades gracefully when an edge node or link fails. A
|
||||
partial service is a designed state with a defined contract, not a
|
||||
crash. One node's failure must not collapse the whole fleet; the
|
||||
degraded mode is documented, observable, and recoverable. This
|
||||
derives from `C1 Correctness` (the degraded contract is a
|
||||
correctness bound) and `C5 Reversibility` (recovery from degradation
|
||||
is reversible by construction). A crash-on-node-failure system has
|
||||
no degradation contract — it has an all-or-nothing failure mode that
|
||||
violates the fleet assumption. See `domains/edge/iot.md` for
|
||||
device-drop degradation and `domains/edge/offline-first.md` for
|
||||
partition degradation.
|
||||
|
||||
### P8. Geographic Distribution is a First-Class Constraint
|
||||
The fleet is geo-distributed; routing, fan-out, and data placement
|
||||
are location-aware decisions, not accidents of deployment. The
|
||||
system is many nodes across many locations, not a single deployment.
|
||||
Data residency, regional latency, and PoP selection are engineered,
|
||||
not discovered in production. This derives from `C4 Locality` (the
|
||||
placement of data and compute is a locality decision) and `C6
|
||||
Composability` (the fleet composes from location-aware parts, each
|
||||
with its own contract). This is the structural companion to `P1
|
||||
Proximity`: P1 says *where* compute should be (near the user); P8
|
||||
says the *distribution* of compute across geographies is a
|
||||
first-class constraint. See `domains/edge/cdn.md` for multi-CDN
|
||||
routing.
|
||||
|
||||
### P9. Identity is Constrained at the Edge
|
||||
Edge devices and nodes hold scoped, minimal credentials. No edge
|
||||
node is a cluster-admin-equivalent; device identity is per-device,
|
||||
not shared. One compromise must not equal a fleet compromise. This
|
||||
derives from `C1 Correctness` (security is a subset of correctness —
|
||||
an exploitable edge node does not do what it was supposed to do) and
|
||||
`C8 Economy` of trust (the credential scope is minimal for the task).
|
||||
A shared edge-device credential is the edge analog of a
|
||||
cluster-admin GitOps robot — blast radius is unbounded. See
|
||||
`domains/security/secrets.md` for the general secret-hygiene
|
||||
principles and `domains/edge/iot.md` for device provisioning.
|
||||
|
||||
### P10. Edge Observability Survives Partition
|
||||
Telemetry is local-first: buffered on the node and forwarded on
|
||||
reconnect. Partition does not blind the operator. A fire-and-forget
|
||||
telemetry pipeline loses data when the link drops; a local-first
|
||||
buffer survives. This derives from `C7 Observability` (the fleet's
|
||||
behavior is visible to the operator) and `C5 Reversibility` (the
|
||||
buffered telemetry is reversible back to visibility on reconnect).
|
||||
This is distinct from `domains/observability/P1 Structured by
|
||||
Default`, which owns *generic* structured telemetry; edge owns the
|
||||
*partition-survivable, local-first* angle. See
|
||||
`domains/observability/metrics.md` and
|
||||
`domains/observability/logging.md` for the generic structured-
|
||||
telemetry foundations edge builds on.
|
||||
|
||||
## 2. Core Principle Trace
|
||||
|
||||
Each edge P-rule derives from one or more core C-rules (C1–C8). The
|
||||
matrix extension lands in P4 of the v0.4 plan; the traces below are
|
||||
authoritative. Edge is a broad-derivation domain touching 7 of 8
|
||||
core principles (C1, C3, C4, C5, C6, C7, C8); C2 (Clarity) is not a
|
||||
primary derivation — edge clarity is indirect (a cache with explicit
|
||||
invalidation is clearer than one without, but the primary trace is
|
||||
C1/C3).
|
||||
|
||||
| P-rule | Core | Why |
|
||||
|--------|------|-----|
|
||||
| P1 Proximity is the Design Driver | C4, C1 | Locality of compute near user/data; correctness via latency |
|
||||
| P2 Offline is a First-Class State | C1, C5 | Correctness under partition; reversibility of reconciliation |
|
||||
| P3 Resources are Constrained and Declared | C8, C1 | Economy of constrained nodes; correctness of declared bounds |
|
||||
| P4 Sync Conflicts are Bounded, Not Infinite | C1, C5 | Correctness of convergence; reversibility of divergent state |
|
||||
| P5 Edge Operations are Idempotent | C1 | Correctness under retry |
|
||||
| P6 Cache Invalidation is Explicit | C1, C3 | Correctness of cached state; simplicity of defined invalidation |
|
||||
| P7 Partial Degradation is Engineered | C1, C5 | Correctness of degraded modes; reversibility of recovery |
|
||||
| P8 Geographic Distribution is a First-Class Constraint | C4, C6 | Locality of placement; composability of the fleet |
|
||||
| P9 Identity is Constrained at the Edge | C1, C8 | Correctness via security; economy of trust |
|
||||
| P10 Edge Observability Survives Partition | C7, C5 | Observability of the fleet; reversibility of buffered telemetry |
|
||||
|
||||
## 3. What Violates These Principles
|
||||
|
||||
| Violation | Principle Breached |
|
||||
|-----------|-------------------|
|
||||
| Central-region-only deployment for a latency-bound workload | P1 Proximity is the Design Driver |
|
||||
| App that crashes on disconnect (no offline state) | P2 Offline is a First-Class State |
|
||||
| Undeclared edge-node resource budget (assumes infinite CPU/memory) | P3 Resources are Constrained and Declared |
|
||||
| Sync loop that oscillates forever (CRDT without merge-semantics, LWW without monotonic clock) | P4 Sync Conflicts are Bounded, Not Infinite |
|
||||
| Non-idempotent edge write (cache-fill or device command retried with side effects) | P5 Edge Operations are Idempotent |
|
||||
| TTL-less edge cache under partition (stale-forever, no explicit invalidation) | P6 Cache Invalidation is Explicit |
|
||||
| Crash-on-node-failure (no partial-degradation contract) | P7 Partial Degradation is Engineered |
|
||||
| Random geographic placement (no location-aware routing) | P8 Geographic Distribution is a First-Class Constraint |
|
||||
| Shared edge-device credential (one key for the whole fleet) | P9 Identity is Constrained at the Edge |
|
||||
| Fire-and-forget telemetry (no on-node buffer; data lost on partition) | P10 Edge Observability Survives Partition |
|
||||
| Blocking call on a constrained IoT device with no timeout | P3, P5 (blocks the node; retry unsafe without idempotency) |
|
||||
| Multi-CDN routing with no PoP-selection logic (latency uncontrolled) | P8, P1 (placement not a design decision) |
|
||||
|
||||
## 4. Relationship to Other Domains
|
||||
|
||||
Edge computing is the engineering discipline of placing compute,
|
||||
storage, and data **near the source of generation or consumption**
|
||||
rather than in a centralized cloud. The distinguishing constraints are
|
||||
latency-bound operation, resource-constrained nodes,
|
||||
geo-distribution as a fleet, and partition-prone operation. Edge
|
||||
overlaps three existing domains by *subject* but not by *angle*: per
|
||||
D-061, edge owns the proximity/location/constraint/disconnection
|
||||
concerns that only arise at the network edge. The C4 Locality
|
||||
emphasis is the discriminator: performance's locality is algorithmic
|
||||
(data near compute); edge's locality is geographic (compute near
|
||||
user/data source). Cross-links are one-directional outward (per
|
||||
D-026 extended); no back-link edits to v0.1/v0.2/v0.3 content.
|
||||
|
||||
- `domains/performance/frontend` ← P6 (edge owns geographic,
|
||||
partition-aware cache invalidation; performance owns *generic*
|
||||
caching and measurement — D-061 boundary)
|
||||
- `domains/performance/P4 Resource Bounds` ← P3 (edge owns
|
||||
constrained-device reality; performance owns the generic
|
||||
unbounded-growth-is-a-bug principle)
|
||||
- `domains/observability/metrics` ← P10 (cache-hit ratio, edge
|
||||
telemetry aggregation; edge owns the local-first angle, observability
|
||||
owns generic structured metrics)
|
||||
- `domains/observability/logging` ← P10 (local-first logging buffered
|
||||
on-node and forwarded on reconnect)
|
||||
- `domains/concurrency/patterns` ← P5 (the offline write-queue is
|
||||
the cross-partition analog of the in-process bounded buffer —
|
||||
concurrency owns in-process; edge owns partition-survivable)
|
||||
- `domains/security/secrets` ← P9 (device credentials are scoped,
|
||||
per-device, never shared — edge owns the constrained-identity
|
||||
angle; security owns the general secret hygiene)
|
||||
- `domains/security/input-validation` ← P6 (cache poisoning
|
||||
prevention — edge cache keys are a validation surface)
|
||||
- `domains/data/migrations` ← P4 (schema migration under sync must
|
||||
reconcile across partitioned nodes; data owns the generic migration
|
||||
discipline, edge owns the partitioned-reconcile angle)
|
||||
|
||||
> Note: cross-links to `domains/messaging/` (e.g., MQTT QoS parallels
|
||||
> for delivery semantics) are intentionally omitted here — the
|
||||
> messaging domain is authored in P2. The intra-v0.4 edge↔messaging
|
||||
> links are added in P5 (ATELIER-114 per IDEATE-40) once both
|
||||
> domains exist; the dangling link is acceptable per D-053.
|
||||
@@ -1,258 +0,0 @@
|
||||
# IoT — Derived Rules
|
||||
|
||||
> Derives from `domains/edge/first-principles.md`. Applies P3
|
||||
> (Resources are Constrained and Declared) primarily, with P5
|
||||
> (command idempotency), P7 (partial degradation when devices drop),
|
||||
> P9 (device identity and provisioning), and P10 (telemetry from
|
||||
> devices). Cross-links `domains/security/secrets` for device
|
||||
> credentials and `domains/messaging/queues` for the MQTT QoS
|
||||
> parallels to delivery semantics.
|
||||
|
||||
## What IoT at the Edge Is (P3 Resources are Constrained and Declared)
|
||||
|
||||
- IoT at the edge is the engineering discipline of operating
|
||||
constrained devices — sensors, actuators, gateways, microcontrollers
|
||||
— as first-class participants in a distributed system. The
|
||||
distinguishing constraint is per-device resource bounds (P3): a
|
||||
battery-powered sensor has kilobytes of RAM, a constrained
|
||||
protocol, and a multi-year sleep budget. These constraints are
|
||||
declared per device class, never assumed infinite.
|
||||
- The boundary is per D-061: edge owns the constrained-device
|
||||
reality; performance owns the generic unbounded-growth-is-a-bug
|
||||
principle (`performance/P4 Resource Bounds`); concurrency owns
|
||||
in-process primitives. IoT is an edge concern because its defining
|
||||
traits are constrained resources (P3), geographic distribution as
|
||||
a fleet (P8), partition-prone operation (P2), and device-scoped
|
||||
identity (P9) — concerns that only arise at the network edge.
|
||||
- See `domains/edge/offline-first.md` for the partition-survival
|
||||
discipline that constrained devices depend on, and
|
||||
`domains/edge/sync.md` for the reconciliation of device state
|
||||
across partitions.
|
||||
|
||||
## Device Resource Classes (P3, C8 Economy)
|
||||
|
||||
- A device resource class declares the bounds for a class of
|
||||
devices: CPU (MHz, cores), memory (KB/MB), power (battery mAh,
|
||||
duty-cycle budget), bandwidth (bytes/sec, latency budget), and
|
||||
storage (KB/MB). Every device in the fleet is assigned to a class;
|
||||
every operation is budgeted against its class.
|
||||
- An undeclared budget is a defect (P3 violation): a sensor that
|
||||
sends telemetry every second without a duty-cycle budget exhausts
|
||||
its battery in days, not years. The budget is the correctness
|
||||
bound (C1) and the economy bound (C8).
|
||||
- A device class implies a protocol choice: a class-0 device
|
||||
(constrained sensor, KB RAM) speaks CoAP; a class-1 device
|
||||
(gateway, MB RAM) speaks MQTT; a class-2 device (edge compute
|
||||
node, GB RAM) speaks HTTP. The protocol follows the constraint,
|
||||
not the reverse.
|
||||
|
||||
| Class | RAM | Power | Protocol | Typical role |
|
||||
|-------|-----|-------|----------|--------------|
|
||||
| 0 (constrained sensor) | < 10 KB | Battery, multi-year | CoAP, LoRaWAN | Telemetry only, no inbound commands |
|
||||
| 1 (actuator, gateway) | 10 KB – 1 MB | Battery or wired, weeks-months | MQTT, CoAP | Telemetry + commands, queue-and-forward |
|
||||
| 2 (edge compute) | > 1 MB | Wired, continuous | HTTP, MQTT | Local aggregation, gateway, edge inference |
|
||||
|
||||
## Constrained Protocols — MQTT and CoAP (P3, P5)
|
||||
|
||||
- **MQTT** is the canonical pub/sub protocol for constrained devices.
|
||||
It is lightweight (2-byte header), broker-backed, and provides QoS
|
||||
levels (0, 1, 2) that map to delivery semantics. MQTT is the
|
||||
cross-process analog of message-queue delivery — see
|
||||
`domains/messaging/queues` for the general queue/delivery-semantics
|
||||
discipline; the cross-link is one-directional outward (edge →
|
||||
messaging) per D-062 and D-026 extended.
|
||||
- **CoAP** is the REST analog for constrained devices: UDP-based,
|
||||
low-overhead, with confirmable (CON) and non-confirmable (NON)
|
||||
message types. CoAP fits class-0 devices where TCP is too heavy.
|
||||
- A blocking synchronous call on a constrained device with no
|
||||
timeout is the `blocking-call-on-constrained-device` chaos
|
||||
anti-pattern: it blocks the node, has no timeout (= hang), and
|
||||
retries are unsafe without idempotency (P3 + P5 breach). Every
|
||||
device operation must be async with a timeout, and every retried
|
||||
operation must be idempotent.
|
||||
|
||||
```json
|
||||
// MQTT publish/subscribe payload with QoS levels (P5 idempotency,
|
||||
// P3 constrained protocol).
|
||||
// QoS 0 — at-most-once: fire-and-forget, no ack. For telemetry
|
||||
// where a dropped sample is acceptable (P3 economy of the
|
||||
// constrained link).
|
||||
{
|
||||
"topic": "devices/sensor-7/temperature",
|
||||
"qos": 0,
|
||||
"payload": {
|
||||
"device": "sensor-7",
|
||||
"ts": 1700000000,
|
||||
"value": 21.4,
|
||||
"unit": "C"
|
||||
}
|
||||
}
|
||||
|
||||
// QoS 1 — at-least-once: acked, may duplicate. The consumer must
|
||||
// be idempotent (P5) — dedup by (device, ts) or an idempotency key.
|
||||
{
|
||||
"topic": "devices/actuator-3/command",
|
||||
"qos": 1,
|
||||
"payload": {
|
||||
"device": "actuator-3",
|
||||
"idempotencyKey": "cmd-1700000000-1",
|
||||
"command": "set-point",
|
||||
"value": 22.0
|
||||
}
|
||||
}
|
||||
|
||||
// QoS 2 — exactly-once: four-step handshake, no duplication. The
|
||||
// heaviest QoS; use only where duplicates are intolerable AND the
|
||||
// device has the budget for the handshake (class-1+ only, P3).
|
||||
{
|
||||
"topic": "devices/actuator-3/irreversible-command",
|
||||
"qos": 2,
|
||||
"payload": {
|
||||
"device": "actuator-3",
|
||||
"idempotencyKey": "cmd-1700000000-2",
|
||||
"command": "calibrate"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- The QoS choice is a P5 (idempotency) and P3 (resource) decision:
|
||||
QoS 0 is cheapest (no ack) but lossy; QoS 1 requires consumer
|
||||
idempotency (P5); QoS 2 is exactly-once but costs a four-step
|
||||
handshake on a constrained link. The default for telemetry is QoS
|
||||
0 or 1; the default for commands is QoS 1 with an idempotency key
|
||||
(P5); QoS 2 is reserved for irreversible commands where the
|
||||
device budget permits.
|
||||
|
||||
## Device Identity and Provisioning (P9 Identity is Constrained at the Edge)
|
||||
|
||||
- Every device holds a per-device identity: a unique device ID and a
|
||||
scoped credential (X.509 certificate, API token, or rotating
|
||||
key). No shared fleet credential — one compromise must not equal
|
||||
a fleet compromise (P9). The credential scope is minimal: a
|
||||
device can publish to `devices/<its-id>/+` and subscribe to
|
||||
`devices/<its-id>/commands`, nothing else.
|
||||
- Provisioning is the act of assigning a device identity at
|
||||
enrollment time. The provisioning manifest declares the device,
|
||||
its class, its allowed topics, and its credential. The manifest is
|
||||
the P9 contract — a device operating outside its manifest scope is
|
||||
a violation.
|
||||
- A device that is provisioned with a shared fleet key (the
|
||||
`shared-edge-device-credential` anti-pattern) is a P9 violation:
|
||||
blast radius is unbounded. See `domains/security/secrets` for the
|
||||
general secret-hygiene discipline (per-identity credentials,
|
||||
rotation, minimal scope) that device provisioning builds on.
|
||||
|
||||
```yaml
|
||||
# Device provisioning manifest (P9 per-device identity + scoped
|
||||
# credentials). The manifest is the contract; the device operates
|
||||
# only within its declared scope.
|
||||
device:
|
||||
id: sensor-7
|
||||
class: 0 # P3 resource class
|
||||
model: temp-sensor-v2
|
||||
firmware: 1.4.2
|
||||
identity:
|
||||
cert: "sha256-of-device-cert"
|
||||
credentialScope:
|
||||
publish:
|
||||
- "devices/sensor-7/temperature"
|
||||
- "devices/sensor-7/status"
|
||||
subscribe:
|
||||
- "devices/sensor-7/commands"
|
||||
# No wildcard, no fleet-wide topics (P9).
|
||||
provisioning:
|
||||
enrolledAt: 2024-01-15T00:00:00Z
|
||||
rotatesEvery: 90d
|
||||
# Per-device credential; never shared (P9, domains/security/secrets).
|
||||
```
|
||||
|
||||
## Telemetry from Devices (P10 Edge Observability Survives Partition)
|
||||
|
||||
- Device telemetry is local-first (P10): the device buffers telemetry
|
||||
on-node and forwards on reconnect. A fire-and-forget telemetry
|
||||
pipeline loses data when the link drops; a buffered pipeline
|
||||
survives. The buffer is bounded by the device class (P3): a
|
||||
class-0 sensor buffers minutes of telemetry, not hours.
|
||||
- Telemetry is observable in aggregate: the operator sees the fleet's
|
||||
behavior, not just per-device. A device that has not reported in
|
||||
its expected interval is itself a signal (a dead device, a
|
||||
partitioned device, a drained battery). See
|
||||
`domains/observability/metrics` for the generic structured-metrics
|
||||
discipline; edge owns the partition-survivable, local-first angle.
|
||||
- Telemetry must not be a secrets channel (P9 analog, see
|
||||
`domains/observability/P6 No Secrets in Observability`): device
|
||||
credentials, PII, and personally-identifying location must not
|
||||
enter telemetry payloads.
|
||||
|
||||
## Command Idempotency (P5 Edge Operations are Idempotent)
|
||||
|
||||
- Device commands are retried by nature (the network is
|
||||
partition-prone). Every command carries an idempotency key so a
|
||||
retried command does not double-apply (P5). A `set-point` command
|
||||
retried with the same idempotency key sets the point once, not
|
||||
twice; an `open-valve` command retried is safe because the valve
|
||||
is already open.
|
||||
- Irreversible commands (a calibration burn-in, a firmware flash)
|
||||
require stronger idempotency: the device tracks applied
|
||||
idempotency keys and refuses re-application. A retried irreversible
|
||||
command without idempotency tracking double-applies the effect
|
||||
(P5 violation, possibly a physical-side-effect bug).
|
||||
- The idempotency key is per-command, not per-device. A device that
|
||||
dedups by device ID alone will drop distinct commands issued in
|
||||
the same window. Use `(device, command-id, ts-window)` or a
|
||||
UUID per command.
|
||||
|
||||
## Partial Degradation When Devices Drop (P7 Partial Degradation is Engineered)
|
||||
|
||||
- A fleet degrades when devices drop (battery exhaustion, partition,
|
||||
hardware failure). The system must continue to operate with the
|
||||
remaining devices; a whole-system crash on one device's failure is
|
||||
a P7 violation. The degraded mode is documented: which
|
||||
aggregations are valid with N-1 devices, which alerts fire, which
|
||||
fallbacks engage.
|
||||
- A device that drops is not an incident by itself — fleets expect
|
||||
churn. The operator-facing signal is the *aggregate* health (X%
|
||||
of devices reporting, Y% partitioned for >Z minutes), not the
|
||||
per-device drop. Per-device drop alerts are noise; aggregate
|
||||
degradation alerts are signal (see `domains/observability/metrics`).
|
||||
- A command to a dropped device must time out (P5 — idempotent
|
||||
retry) and degrade (P7 — the fleet continues without that
|
||||
device). A command that blocks forever waiting for a dropped
|
||||
device is the `blocking-call-on-constrained-device` chaos
|
||||
anti-pattern (P3 + P5 breach).
|
||||
|
||||
## Cross-Link to Messaging (P5, cross-link messaging/queues)
|
||||
|
||||
- MQTT QoS 0/1/2 maps to at-most-once / at-least-once / exactly-once
|
||||
delivery semantics — the same three-way tradeoff documented in
|
||||
`domains/messaging/queues`. The cross-link is one-directional
|
||||
outward (edge → messaging) per D-026 extended: edge owns the
|
||||
constrained-device protocol angle; messaging owns the generic
|
||||
cross-process delivery-semantics angle.
|
||||
- This link dangles until P2 (the messaging domain is authored in
|
||||
P2); it is verified bidirectional in P5 (ATELIER-114 per
|
||||
IDEATE-40). Acceptable per D-053 (vertical-slice integrity — P1
|
||||
ships the edge domain self-consistent; the messaging cross-link
|
||||
resolves by the P6 ship).
|
||||
- The parallel: a constrained device's QoS 1 publish is the
|
||||
device-flavored instance of an at-least-once queue delivery — the
|
||||
consumer (the broker or the downstream service) must be
|
||||
idempotent (P5 here, `messaging/P3 Consumers are Idempotent`
|
||||
there). The idempotency discipline is the same; the protocol and
|
||||
failure model differ (constrained-device link vs broker-backed
|
||||
network).
|
||||
|
||||
## What Violates IoT-at-the-Edge Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| Undeclared device resource budget (assumes infinite battery/RAM) | P3 Resources are Constrained and Declared |
|
||||
| Shared fleet credential (one key for all devices) | P9 Identity is Constrained at the Edge |
|
||||
| Non-idempotent device command (retried command doubles the effect) | P5 Edge Operations are Idempotent |
|
||||
| Blocking synchronous call on a constrained device with no timeout | P3, P5 (blocks the node; retry unsafe) |
|
||||
| Fire-and-forget telemetry with no on-device buffer (lost on partition) | P10 Edge Observability Survives Partition |
|
||||
| Whole-system crash on one device's failure (no degradation contract) | P7 Partial Degradation is Engineered |
|
||||
| Device credential scope that includes fleet-wide topics (over-scoped) | P9, `domains/security/secrets` |
|
||||
| QoS 2 used on a class-0 device (no budget for the handshake) | P3 Resources are Constrained and Declared |
|
||||
| Per-device-drop alert (noise; aggregate degradation is the signal) | P7, `domains/observability/metrics` |
|
||||
| Telemetry payload that includes device credentials or PII | P9, `domains/observability/P6 No Secrets in Observability` |
|
||||
@@ -1,354 +0,0 @@
|
||||
# Offline-First — Derived Rules
|
||||
|
||||
> Derives from `domains/edge/first-principles.md`. Applies P2
|
||||
> (Offline is a First-Class State) primarily, with P5 (idempotent
|
||||
> queue-and-forward), P4 (bounded sync conflicts on reconnect), P7
|
||||
> (partial degradation), and P10 (local-first telemetry). Cross-links
|
||||
> `domains/concurrency/patterns` for the in-process bounded-buffer
|
||||
> analog and `domains/observability/logging` for local-first logging.
|
||||
|
||||
## What Offline-First Is (P2 Offline is a First-Class State)
|
||||
|
||||
- Offline-first is the design discipline in which the system
|
||||
continues to operate when disconnected from the center. Partition
|
||||
is the norm, not the exception; reconciliation happens on
|
||||
reconnect. The offline state is engineered, not a degenerate mode
|
||||
the app falls into by accident.
|
||||
- The boundary is per D-061: edge owns the
|
||||
proximity/location/disconnection angle. An offline-first web app
|
||||
is an edge concern because its defining trait is partition-survival
|
||||
(P2), not generic performance. The local-first storage is the
|
||||
edge device's constrained-resource reality (P3).
|
||||
- Offline-first is the precondition for the bounded-conflict
|
||||
discipline of `P4 Sync Conflicts are Bounded, Not Infinite`:
|
||||
without offline operation there is nothing to reconcile; with it,
|
||||
the reconnect reconciliation is the correctness mechanism. See
|
||||
`domains/edge/sync.md` for the conflict-resolution strategies.
|
||||
|
||||
## Local-First Storage (P2, P3)
|
||||
|
||||
- Local-first storage holds the working copy on the device:
|
||||
IndexedDB (browser), SQLite (mobile, embedded), or on-device file
|
||||
storage (desktop, IoT gateway). The local store is the authority
|
||||
while offline; the server is reconciled later, not consulted per
|
||||
read.
|
||||
- The local store is bounded by the device (P3 — Resources are
|
||||
Constrained and Declared). A local store that grows without bound
|
||||
is a defect: declare a budget (e.g., a 50 MB IndexedDB quota, a
|
||||
30-day rolling window), and evict outside the budget deterministically.
|
||||
- The local store is the offline state; without it the app is
|
||||
online-only and crashes on disconnect (P2 violation). The store is
|
||||
the reversibility mechanism (C5): every local write is reversible
|
||||
on reconcile.
|
||||
|
||||
```typescript
|
||||
// Local-first store sketch (IndexedDB). The app reads from the
|
||||
// local store, never the network, while offline. Writes queue
|
||||
// locally and forward on reconnect (P2, P5).
|
||||
const db = await openDB("atelier-offline", 1, {
|
||||
upgrade(db) {
|
||||
const store = db.createObjectStore("pending-writes", {
|
||||
keyPath: "id",
|
||||
});
|
||||
store.createIndex("by-createdAt", "createdAt");
|
||||
},
|
||||
});
|
||||
|
||||
async function readRecord(id: string) {
|
||||
// Read from local store first; the network is a reconcile path,
|
||||
// not the read path.
|
||||
return db.get("pending-writes", id);
|
||||
}
|
||||
```
|
||||
|
||||
## Queue-and-Forward for Writes (P5 Edge Operations are Idempotent)
|
||||
|
||||
- Every write while offline is queued locally and forwarded to the
|
||||
server on reconnect. The queue is the offline write-queue; the
|
||||
forward is the reconcile. Each queued write carries an idempotency
|
||||
key so a retried forward (the network is partition-prone) does not
|
||||
double-apply (P5).
|
||||
- The queue is bounded (P3): a queue that grows without limit on a
|
||||
constrained device will exhaust it. Declare a max-queue-depth and
|
||||
a max-queue-bytes; reject or evict beyond the bound with a defined
|
||||
policy (oldest-first, lowest-priority-first).
|
||||
- The queue is the cross-partition analog of the in-process bounded
|
||||
buffer — see `domains/concurrency/patterns` (bounded buffer,
|
||||
backpressure). Concurrency owns the in-process analog; edge owns
|
||||
the partition-survivable analog. The failure model differs: the
|
||||
in-process buffer fails by OOM; the offline write-queue fails by
|
||||
partition or device loss.
|
||||
|
||||
```typescript
|
||||
// Offline write-queue sketch. Each entry carries an idempotency
|
||||
// key (P5) so a retried forward is safe. The queue is bounded by
|
||||
// maxDepth (P3).
|
||||
interface PendingWrite {
|
||||
id: string; // local id
|
||||
idempotencyKey: string; // server-side dedup key (P5)
|
||||
collection: string;
|
||||
payload: unknown;
|
||||
createdAt: number;
|
||||
}
|
||||
|
||||
const MAX_DEPTH = 1000;
|
||||
|
||||
async function queueWrite(write: Omit<PendingWrite, "id" | "idempotencyKey" | "createdAt">) {
|
||||
const depth = await db.count("pending-writes");
|
||||
if (depth >= MAX_DEPTH) {
|
||||
// P3: bounded queue. Evict the oldest pending write or reject.
|
||||
// Rejecting is correct when the write is higher-priority than
|
||||
// the oldest; evicting is correct when the newest is lowest.
|
||||
throw new Error("offline-queue-full");
|
||||
}
|
||||
const entry: PendingWrite = {
|
||||
...write,
|
||||
id: crypto.randomUUID(),
|
||||
idempotencyKey: `${write.collection}:${crypto.randomUUID()}`,
|
||||
createdAt: Date.now(),
|
||||
};
|
||||
await db.put("pending-writes", entry);
|
||||
// The forward loop picks this up when connectivity returns.
|
||||
}
|
||||
|
||||
async function forwardPendingWrites(server: Server) {
|
||||
const pending = await db.getAllFromIndex("pending-writes", "by-createdAt");
|
||||
for (const write of pending) {
|
||||
// P5: idempotent — the server dedups by idempotencyKey.
|
||||
await server.apply(write, write.idempotencyKey);
|
||||
await db.delete("pending-writes", write.id);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Conflict Detection on Reconnect (P4 Sync Conflicts are Bounded)
|
||||
|
||||
- On reconnect, the queued writes are forwarded; the server may
|
||||
have advanced while the device was offline. A conflict is when the
|
||||
local write and the server state diverge. Conflict detection is
|
||||
the precondition for bounded reconciliation (P4): a write forwarded
|
||||
blindly (last-write-wins with no clock) is a P4 violation waiting
|
||||
to happen.
|
||||
- Conflict resolution strategies (CRDT, LWW with vector clocks,
|
||||
application-specific merge) are the subject of
|
||||
`domains/edge/sync.md` — the CRDT-vs-LWW decision matrix there
|
||||
determines which applies. Offline-first owns the *detection*; sync
|
||||
owns the *resolution*.
|
||||
- A reconnect that detects no conflicts when conflicts exist is a
|
||||
silent correctness defect (C1, P4). Detection must be conservative:
|
||||
when in doubt, flag a conflict and surface it to the merge
|
||||
function or the user.
|
||||
|
||||
## UI for Offline State (P7 Partial Degradation is Engineered)
|
||||
|
||||
- The UI must reflect the offline state visibly: a "you are offline,
|
||||
changes will sync when connected" banner, a pending-writes counter,
|
||||
a last-synced timestamp. A UI that hides the offline state
|
||||
violates P7 — the degraded mode is a designed state with a defined
|
||||
contract, not a silent fall-through.
|
||||
- The UI must function while offline: reads from local-first
|
||||
storage, writes to the queue, navigation that does not require the
|
||||
network. An app that shows a blank screen or a spinner-forever when
|
||||
offline has no offline state (P2 violation) and no degradation
|
||||
contract (P7 violation).
|
||||
- The pending-writes counter is the local-first analog of the
|
||||
messaging consumer-lag metric — see
|
||||
`domains/observability/metrics` for the lag-discipline parallel.
|
||||
|
||||
## Service Workers (P2, P6)
|
||||
|
||||
- A service worker is a client-side proxy that intercepts network
|
||||
requests and serves from a local cache. It is the browser's
|
||||
offline-first primitive: the service worker cache is the
|
||||
offline-capable store for assets; the IndexedDB store is the
|
||||
offline-capable store for data.
|
||||
- The service worker cache is an edge cache (P6 — Cache Invalidation
|
||||
is Explicit): it must carry a TTL or explicit invalidation
|
||||
strategy. A service worker that caches forever and never
|
||||
invalidates is a TTL-less edge cache under partition — a P6
|
||||
violation (stale-forever).
|
||||
- See `domains/edge/cdn.md` for the generic edge-cache invalidation
|
||||
discipline; the service worker is the on-device instance of it.
|
||||
|
||||
```javascript
|
||||
// Service worker cache strategy: stale-while-revalidate for
|
||||
// assets, network-first for data, explicit version-bump for
|
||||
// breaking changes (P6).
|
||||
const CACHE = "atelier-v3"; // bump on deploy to invalidate (P6)
|
||||
const ASSETS = ["/", "/app.js", "/styles.css"];
|
||||
|
||||
self.addEventListener("install", (event) => {
|
||||
event.waitUntil(
|
||||
caches.open(CACHE).then((cache) => cache.addAll(ASSETS))
|
||||
);
|
||||
});
|
||||
|
||||
self.addEventListener("fetch", (event) => {
|
||||
const url = new URL(event.request.url);
|
||||
if (url.pathname.startsWith("/api/")) {
|
||||
// Network-first for data; fall back to cache on partition (P2).
|
||||
event.respondWith(
|
||||
fetch(event.request).catch(() => caches.match(event.request))
|
||||
);
|
||||
} else {
|
||||
// Stale-while-revalidate for assets (P6 explicit invalidation).
|
||||
event.respondWith(
|
||||
caches.open(CACHE).then(async (cache) => {
|
||||
const cached = await cache.match(event.request);
|
||||
const network = fetch(event.request).then((resp) => {
|
||||
cache.put(event.request, resp.clone());
|
||||
return resp;
|
||||
}).catch(() => cached);
|
||||
return cached || network;
|
||||
})
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
self.addEventListener("activate", (event) => {
|
||||
// P6: explicit invalidation. Drop old caches on activate.
|
||||
event.waitUntil(
|
||||
caches.keys().then((keys) =>
|
||||
Promise.all(keys.filter((k) => k !== CACHE).map((k) => caches.delete(k)))
|
||||
)
|
||||
);
|
||||
});
|
||||
```
|
||||
|
||||
## Offline Write-Queue and Conflict Detection Mapped to the Testing Pyramid (IDEATE-38, ATELIER-94)
|
||||
|
||||
The offline write-queue and conflict-detection patterns must be
|
||||
tested at every tier of the testing pyramid. Each tier exercises a
|
||||
different failure mode; skipping a tier leaves a correctness gap
|
||||
(P2, P4 violations that surface only in production partitions).
|
||||
|
||||
| Pyramid Tier | What it exercises | What it proves |
|
||||
|-------------|-------------------|----------------|
|
||||
| **Unit** | Conflict detection on a merge function (pure inputs → expected merge result) | The merge logic is correct in isolation (P4) — given two divergent states, the merge returns the converged state |
|
||||
| **Integration** | Reconnect reconcile against a local store (fake server, real IndexedDB/SQLite) | The queue-and-forward loop drains correctly; the local store and server converge after reconnect (P2, P5) |
|
||||
| **E2e** | Partition simulation with a fake network (the app runs in a browser, the network is cut and restored) | The offline state, UI, and reconcile work end-to-end under partition (P2, P7) |
|
||||
|
||||
- **Unit — conflict detection on a merge function.** The merge
|
||||
function is pure: given two divergent states and a clock, it
|
||||
returns the converged state. Test every merge case (LWW, CRDT
|
||||
register, set union, application-specific three-way merge) as a
|
||||
pure function. This is the cheapest tier and the highest coverage
|
||||
per test — see `domains/testing/pyramid`.
|
||||
|
||||
```typescript
|
||||
// Unit test sketch: conflict detection on a merge function (P4).
|
||||
// The merge function is pure; no network, no store. Test that
|
||||
// divergent states converge and that the merge is bounded (no
|
||||
// oscillation).
|
||||
|
||||
function mergeLWW(local: State, remote: State, clock: Clock): State {
|
||||
// Last-write-wins: the state with the later vector-clock wins.
|
||||
// Returns the converged state (P4).
|
||||
return clock.compare(local.clock, remote.clock) >= 0 ? local : remote;
|
||||
}
|
||||
|
||||
// Unit cases:
|
||||
// - local ahead → local wins
|
||||
// - remote ahead → remote wins
|
||||
// - concurrent (clocks incomparable) → conflict flagged or LWW tiebreak
|
||||
// - identical → no-op convergence (bounded, no oscillation)
|
||||
test("mergeLWW converges when local is ahead", () => {
|
||||
const local = { v: 2, clock: { a: 2 } };
|
||||
const remote = { v: 1, clock: { a: 1 } };
|
||||
expect(mergeLWW(local, remote, { compare: (a, b) => a.a - b.a })).toEqual(local);
|
||||
});
|
||||
```
|
||||
|
||||
- **Integration — reconnect reconcile against a local store.** A
|
||||
fake server stands in for the network; the real IndexedDB (or
|
||||
SQLite) holds the queue. The test fills the queue while offline,
|
||||
reconnects, and asserts the queue drains and the server and local
|
||||
store converge. This exercises the queue-and-forward loop (P5)
|
||||
and the reconcile against real storage.
|
||||
|
||||
```typescript
|
||||
// Integration test sketch: reconnect reconcile against a local
|
||||
// store. A fake server; real IndexedDB. The queue drains; the
|
||||
// server and local store converge after reconnect (P2, P5).
|
||||
|
||||
test("reconnect reconciles pending writes against the server", async () => {
|
||||
const db = await openDB("test-offline", 1, { /* schema */ });
|
||||
const server = new FakeServer();
|
||||
await queueWrite(db, { collection: "docs", payload: { v: 1 } });
|
||||
// Simulate offline: server is unreachable.
|
||||
server.offline();
|
||||
await queueWrite(db, { collection: "docs", payload: { v: 2 } });
|
||||
expect(await db.count("pending-writes")).toBe(2);
|
||||
// Simulate reconnect: server is reachable.
|
||||
server.online();
|
||||
await forwardPendingWrites(server, db);
|
||||
expect(await db.count("pending-writes")).toBe(0);
|
||||
expect(await server.latest("docs")).toEqual({ v: 2 });
|
||||
});
|
||||
```
|
||||
|
||||
- **E2e — partition simulation with a fake network.** The app runs
|
||||
in a real browser; a fake network layer cuts and restores the
|
||||
connection. The test asserts the UI shows the offline state, the
|
||||
writes queue, the reconnect reconciles, and the UI returns to
|
||||
online. This is the highest-fidelity tier and the lowest coverage
|
||||
per test — run a small number of representative scenarios, not a
|
||||
combinatorial matrix.
|
||||
|
||||
```typescript
|
||||
// E2e test sketch: partition simulation with a fake network. The
|
||||
// app runs in a browser; the network is cut and restored. Asserts
|
||||
// the offline UI state, the queue, the reconcile, and the online
|
||||
// recovery (P2, P7).
|
||||
|
||||
test("app survives a network partition and reconciles on reconnect", async () => {
|
||||
await page.goto("https://app.example.com");
|
||||
await page.click("text=Edit document");
|
||||
await page.fill("textarea", "offline edit");
|
||||
// Cut the network.
|
||||
await page.setOffline(true);
|
||||
await page.click("text=Save");
|
||||
await expect(page.locator("text=You are offline")).toBeVisible();
|
||||
await expect(page.locator("text=1 pending change")).toBeVisible();
|
||||
// Restore the network.
|
||||
await page.setOffline(false);
|
||||
await expect(page.locator("text=All changes synced")).toBeVisible();
|
||||
await expect(page.locator("text=0 pending changes")).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
- The three tiers are complementary: unit proves the merge logic,
|
||||
integration proves the reconcile loop, e2e proves the partition
|
||||
behavior. Skipping any tier leaves a correctness gap. See
|
||||
`domains/testing/pyramid` for the pyramid discipline and
|
||||
`domains/testing/fixtures` for the fake-server and fake-network
|
||||
fixture patterns.
|
||||
|
||||
## Observability (P10 Edge Observability Survives Partition)
|
||||
|
||||
- The offline state is itself an observable signal: the
|
||||
pending-writes count, the last-synced timestamp, the
|
||||
reconcile-failure count. A device stuck offline for days with a
|
||||
full queue is an incident; without local-first telemetry it is
|
||||
invisible (P10 violation).
|
||||
- Local-first logging (buffered on-device, forwarded on reconnect)
|
||||
is the offline-first instance of `P10 Edge Observability Survives
|
||||
Partition`. See `domains/observability/logging` for the generic
|
||||
structured-logging discipline the local-first buffer builds on.
|
||||
- A reconcile failure that is not logged locally is a silent defect
|
||||
— the operator cannot debug what they cannot see (C7, P10).
|
||||
|
||||
## What Violates Offline-First Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| App that crashes on disconnect (no offline state) | P2 Offline is a First-Class State |
|
||||
| Unbounded offline write-queue (grows until device OOM) | P3 Resources are Constrained and Declared |
|
||||
| Queued write forwarded without an idempotency key (retry doubles the effect) | P5 Edge Operations are Idempotent |
|
||||
| Reconnect that detects no conflicts when conflicts exist | P4 Sync Conflicts are Bounded, Not Infinite |
|
||||
| UI that hides the offline state (no banner, no pending counter) | P7 Partial Degradation is Engineered |
|
||||
| Service worker cache with no TTL and no explicit invalidation | P6 Cache Invalidation is Explicit |
|
||||
| Reconcile failure with no local log (silent under partition) | P10 Edge Observability Survives Partition |
|
||||
| Merge function that oscillates (no convergence guarantee) | P4, `domains/edge/sync.md` |
|
||||
| Local-first store with no declared budget (grows without bound) | P3, `domains/concurrency/patterns` (bounded buffer analog) |
|
||||
| E2e tests that never simulate a partition (offline path untested) | P2, `domains/testing/pyramid` |
|
||||
@@ -1,315 +0,0 @@
|
||||
# Sync — Derived Rules
|
||||
|
||||
> Derives from `domains/edge/first-principles.md`. Applies P4 (Sync
|
||||
> Conflicts are Bounded, Not Infinite) primarily, with P5
|
||||
> (idempotent merge operations), P2 (offline as the precondition),
|
||||
> and P10 (sync is observable). Cross-links `domains/data/migrations`
|
||||
> for schema migration under sync and `domains/concurrency/patterns`
|
||||
> for the immutability-aid-merge principle.
|
||||
|
||||
## What the Sync Problem Is (P4 Sync Conflicts are Bounded, Not Infinite)
|
||||
|
||||
- Sync is the discipline of reconciling divergent state across
|
||||
partitioned nodes. While partitioned, each node accepts writes
|
||||
independently; on reconnect, the divergent state must converge.
|
||||
The correctness contract is that the merge terminates and
|
||||
converges — oscillation and infinite sync loops are correctness
|
||||
failures, not eventual consistency (P4).
|
||||
- Sync is the edge domain's deepest problem: it is the
|
||||
reconciliation layer above `P2 Offline is a First-Class State`.
|
||||
Without offline operation there is nothing to sync; with it, the
|
||||
reconnect reconciliation is the correctness mechanism. See
|
||||
`domains/edge/offline-first.md` for the offline write-queue that
|
||||
produces the divergent state to be reconciled.
|
||||
- The boundary is per D-061: edge owns the partitioned-reconcile
|
||||
angle; concurrency owns the in-process analog
|
||||
(`concurrency/P1 Immutability by Default` — immutability aids
|
||||
merge); data owns the generic migration discipline
|
||||
(`data/migrations`). Sync is an edge concern because its defining
|
||||
trait is partitioned divergence, a concern that only arises at the
|
||||
network edge.
|
||||
|
||||
## Conflict Resolution Strategies (P4, C1 Correctness, C5 Reversibility)
|
||||
|
||||
- A conflict is when two nodes have divergent state for the same
|
||||
logical entity and no total order determines which is correct.
|
||||
Resolution strategies fall into two families:
|
||||
- **Conflict-free**: the data type guarantees convergence by
|
||||
construction (CRDTs). The merge is deterministic; no conflict
|
||||
surfaces to the user or the application.
|
||||
- **Conflict-tolerant**: the data type can conflict; the
|
||||
resolution policy (last-write-win, three-way merge,
|
||||
application-specific) arbitrates. Conflicts may surface to the
|
||||
user or be silently resolved per a documented policy.
|
||||
- The choice is a P4 decision: conflict-free types guarantee the
|
||||
bound (convergence) but constrain the data model; conflict-tolerant
|
||||
types are flexible but require the resolution policy to be correct
|
||||
and bounded (no oscillation). See the decision matrix below.
|
||||
|
||||
## CRDTs — Conflict-Free Replicated Data Types (P4, C5, C6)
|
||||
|
||||
- A CRDT is a data type whose merge operation is associative,
|
||||
commutative, and idempotent. Given any set of divergent replicas,
|
||||
merging them in any order converges to the same state — the merge
|
||||
is deterministic and terminating (P4 bound). CRDTs derive from C5
|
||||
Reversibility (divergent state reverses to convergence) and C6
|
||||
Composability (CRDTs compose: a CRDT map of CRDT registers is
|
||||
itself a CRDT).
|
||||
- **State-based (CvRDT — convergent):** each replica carries its
|
||||
full state; merge is a least-upper-bound on a semi-lattice. The
|
||||
payload is larger (full state per merge); the merge is simple
|
||||
(one function). Fits small state and unreliable networks.
|
||||
- **Operation-based (CmRDT — commutative):** each replica carries
|
||||
operations; merge is applying the operations in causal order. The
|
||||
payload is smaller (ops, not state); the delivery must be
|
||||
reliable and causally ordered. Fits large state and reliable
|
||||
transport.
|
||||
- The tradeoff: state-based is simpler but heavier; operation-based
|
||||
is lighter but requires causal delivery. Both guarantee
|
||||
convergence (P4); the choice is a C8 Economy decision (bandwidth
|
||||
vs delivery complexity).
|
||||
|
||||
```typescript
|
||||
// CRDT register: LWW-element-set (state-based, CvRDT). The merge
|
||||
// is deterministic — the register with the later timestamp wins.
|
||||
// Convergence is guaranteed (P4); the merge is idempotent (P5).
|
||||
|
||||
interface LWWRegister<T> {
|
||||
value: T;
|
||||
timestamp: number; // monotonic clock; ties broken by node id
|
||||
nodeId: string;
|
||||
}
|
||||
|
||||
function mergeLWWRegister<T>(
|
||||
local: LWWRegister<T>,
|
||||
remote: LWWRegister<T>,
|
||||
): LWWRegister<T> {
|
||||
// The merge is associative, commutative, idempotent (P4, P5).
|
||||
// (local.timestamp, local.nodeId) > (remote.timestamp, remote.nodeId)
|
||||
// is a total order — no oscillation, no infinite loop.
|
||||
if (local.timestamp > remote.timestamp) return local;
|
||||
if (local.timestamp < remote.timestamp) return remote;
|
||||
// Tie: break by node id for a deterministic total order.
|
||||
return local.nodeId > remote.nodeId ? local : remote;
|
||||
}
|
||||
|
||||
// The register is a CRDT: merge(merge(a, b), c) === merge(a, merge(b, c))
|
||||
// for any replicas a, b, c. Convergence is guaranteed (P4).
|
||||
```
|
||||
|
||||
```typescript
|
||||
// CRDT set: add-wins last-write-wins element set (state-based).
|
||||
// Each element carries a timestamp; remove only wins if the
|
||||
// remove-timestamp is later than the add-timestamp. This avoids
|
||||
// the remove-wins-vs-add race (P4) without surfacing a conflict.
|
||||
|
||||
interface AWLWWSet<T> {
|
||||
adds: Map<T, number>; // element -> add-timestamp
|
||||
removes: Map<T, number>; // element -> remove-timestamp
|
||||
}
|
||||
|
||||
function mergeAWLWWSet<T>(a: AWLWWSet<T>, b: AWLWWSet<T>): AWLWWSet<T> {
|
||||
const adds = new Map<T, number>(a.adds);
|
||||
const removes = new Map<T, number>(a.removes);
|
||||
for (const [el, ts] of b.adds) {
|
||||
adds.set(el, Math.max(adds.get(el) ?? 0, ts)); // add-wins union
|
||||
}
|
||||
for (const [el, ts] of b.removes) {
|
||||
removes.set(el, Math.max(removes.get(el) ?? 0, ts));
|
||||
}
|
||||
return { adds, removes };
|
||||
}
|
||||
|
||||
function contains<T>(set: AWLWWSet<T>, el: T): boolean {
|
||||
const addTs = set.adds.get(el) ?? 0;
|
||||
const rmTs = set.removes.get(el) ?? 0;
|
||||
return addTs > rmTs; // add wins on equal timestamp (P4 bounded)
|
||||
}
|
||||
```
|
||||
|
||||
## Last-Write-Win (LWW) with Vector Clocks (P4, C1, C5)
|
||||
|
||||
- LWW is the simplest conflict-tolerant strategy: the write with the
|
||||
latest timestamp wins. It is cheap, but it silently discards
|
||||
concurrent writes — the "lost update" is the correctness cost. LWW
|
||||
is correct only when the timestamp is a total order (a monotonic
|
||||
clock, not wall time), and when lost concurrent writes are
|
||||
acceptable (e.g., caching, presence, ephemeral state).
|
||||
- **Vector clocks** are the timestamp that knows about concurrency.
|
||||
A vector clock records the logical time of each node; two writes
|
||||
are concurrent iff neither vector dominates the other. LWW with
|
||||
vector clocks: a write that is causally later wins; a write that
|
||||
is concurrent conflicts and is resolved by a tiebreak (node id,
|
||||
wall time, or application policy).
|
||||
- The tiebreak is the P4 bound: the conflict must be resolved
|
||||
deterministically (no oscillation) and the resolution must be
|
||||
documented. A tiebreak by wall time alone (no vector clock) is a
|
||||
P4 violation waiting to happen — wall time skews across nodes,
|
||||
and a clock skew can flip the tiebreak, oscillating the merge.
|
||||
|
||||
```typescript
|
||||
// LWW with vector clocks (conflict-tolerant, P4 bounded). The
|
||||
// vector clock records causal order; concurrent writes conflict;
|
||||
// the conflict is tiebroken deterministically (no oscillation).
|
||||
|
||||
type VectorClock = Record<string, number>; // nodeId -> counter
|
||||
|
||||
function compareClock(a: VectorClock, b: VectorClock): "before" | "after" | "equal" | "concurrent" {
|
||||
let aBefore = false, bBefore = false;
|
||||
const keys = new Set([...Object.keys(a), ...Object.keys(b)]);
|
||||
for (const k of keys) {
|
||||
const av = a[k] ?? 0;
|
||||
const bv = b[k] ?? 0;
|
||||
if (av < bv) aBefore = true;
|
||||
if (av > bv) bBefore = true;
|
||||
}
|
||||
if (aBefore && bBefore) return "concurrent";
|
||||
if (aBefore) return "before";
|
||||
if (bBefore) return "after";
|
||||
return "equal";
|
||||
}
|
||||
|
||||
interface LWWVectorState<T> {
|
||||
value: T;
|
||||
clock: VectorClock;
|
||||
writerId: string; // tiebreak: deterministic, no oscillation (P4)
|
||||
}
|
||||
|
||||
function mergeLWWVector<T>(
|
||||
local: LWWVectorState<T>,
|
||||
remote: LWWVectorState<T>,
|
||||
): LWWVectorState<T> {
|
||||
const order = compareClock(local.clock, remote.clock);
|
||||
if (order === "before") return remote; // remote causally later
|
||||
if (order === "after" || order === "equal") return local;
|
||||
// Concurrent: tiebreak by writer id (deterministic, P4 bounded).
|
||||
return local.writerId > remote.writerId ? local : remote;
|
||||
}
|
||||
```
|
||||
|
||||
- The merge is idempotent (P5): merging the same two replicas twice
|
||||
yields the same result. The tiebreak by `writerId` is a total
|
||||
order, so the merge cannot oscillate (P4 bound).
|
||||
- A vector-clock merge that surfaces the concurrent conflict to the
|
||||
application (instead of tiebreaking) is also valid — the
|
||||
application resolves per its own policy. The P4 bound is that the
|
||||
resolution terminates; the policy determines whether the user sees
|
||||
the conflict or the system silences it.
|
||||
|
||||
## Merge Semantics (P4, P5, cross-link concurrency/patterns)
|
||||
|
||||
- The merge function is the heart of sync. Its properties (P4):
|
||||
- **Associative**: `merge(merge(a, b), c) === merge(a, merge(b, c))`.
|
||||
- **Commutative**: `merge(a, b) === merge(b, a)`.
|
||||
- **Idempotent**: `merge(a, a) === a` (P5 — retried merges are safe).
|
||||
- Immutability aids merge: an immutable state representation (the
|
||||
CRDT payload, the LWW register with a clock) makes the merge a
|
||||
pure function of two inputs, with no in-place mutation race. See
|
||||
`domains/concurrency/patterns` (`concurrency/P1 Immutability by
|
||||
Default`) for the in-process immutability principle; sync is the
|
||||
cross-partition instance of it.
|
||||
- A merge that mutates in place is a P5 violation waiting to
|
||||
happen: a retried merge mutates the same state twice, and the
|
||||
result is not idempotent. Always merge into a new state; never
|
||||
mutate the inputs.
|
||||
|
||||
## Conflict-Free vs Conflict-Tolerant Data Types (P4, C3 Simplicity)
|
||||
|
||||
- **Conflict-free (CRDTs):** the data type guarantees convergence.
|
||||
The application never sees a conflict; the merge is deterministic.
|
||||
The cost: the data model is constrained (counters, sets, registers,
|
||||
maps of these). A conflict-free type for arbitrary JSON is hard;
|
||||
a conflict-free type for a counter is a PN-counter.
|
||||
- **Conflict-tolerant (LWW, three-way merge, application policy):**
|
||||
the data type can conflict; the resolution policy arbitrates. The
|
||||
cost: the policy must be correct and bounded (no oscillation), and
|
||||
the conflict may surface to the user. The benefit: any data model
|
||||
can be made conflict-tolerant (just pick a tiebreak).
|
||||
- The choice is the decision matrix below. It is a P4 decision
|
||||
(which bound), a C1 decision (which correctness cost is
|
||||
acceptable), and a C3 decision (which simplicity is affordable).
|
||||
See also `domains/data/migrations` for the schema-evolution angle
|
||||
— a schema change under sync must be compatible with both
|
||||
replicas, or the merge fails on the new shape.
|
||||
|
||||
## Schema Migration Under Sync (P4, cross-link data/migrations)
|
||||
|
||||
- A schema migration under sync is harder than a single-node
|
||||
migration: both replicas must understand the new shape, or the
|
||||
merge fails. The migration must be forward-and-backward compatible
|
||||
across all replicas that may still hold the old shape — see
|
||||
`domains/data/migrations` for the generic compatibility discipline.
|
||||
- A breaking schema change under sync requires a staged migration:
|
||||
deploy the new-shape-aware merge first (it accepts both shapes),
|
||||
then deploy the new shape, then deploy the old-shape-removing
|
||||
merge. A big-bang schema change under sync is a P4 violation: the
|
||||
replicas that have not yet upgraded will fail the merge, and the
|
||||
sync will not converge.
|
||||
- The merge function's version awareness is the P4 bound: the merge
|
||||
must handle every shape version that may exist in the fleet, or
|
||||
reject (and surface) the merge rather than silently corrupting.
|
||||
|
||||
## CRDT vs Last-Write-Win — Decision Matrix (D-069)
|
||||
|
||||
| Strategy | When | Correctness Guarantee | Operational Cost | Failure Mode |
|
||||
|----------|------|------------------------|-------------------|--------------|
|
||||
| CRDT (state-based, CvRDT) | The data model fits a CRDT (counter, set, register, map of these); convergence must be guaranteed without surfacing conflicts; the network is unreliable (full-state merge tolerates dropped ops) | Strong eventual convergence — `merge(a, b) === merge(b, a)` for any replicas (P4 bound by construction) | Medium — full state per merge (bandwidth); semi-lattice merge function per type; CRDT library or hand-rolled | A bug in the merge function = silent divergence (C1); large state = bandwidth cost on constrained links (P3) |
|
||||
| CRDT (operation-based, CmRDT) | The data model fits a CRDT; bandwidth is constrained (ops are smaller than state); the transport is reliable and causally ordered | Strong eventual convergence — same guarantee, smaller payload | High — requires causal delivery (vector clock or broker with ordering); op transform must be idempotent (P5) | Causal-delivery violation = lost ops = divergence; op-transform bug = silent divergence |
|
||||
| Last-Write-Win (LWW) with vector clocks | The data model is arbitrary (any JSON, any record); concurrent writes are acceptable to discard or tiebreak; a total order tiebreak (node id) is acceptable | Bounded convergence — causally-later writes win; concurrent writes are tiebroken deterministically (P4 bound via tiebreak) | Low — simple merge (compare clocks, pick winner); no CRDT library; small payload | Concurrent writes are silently discarded (lost update); tiebreak by wall time = clock-skew oscillation (P4 violation); no vector clock = no concurrent-write detection = silent loss |
|
||||
| LWW with wall-clock timestamp only | The data model is ephemeral (cache, presence); lost updates are acceptable; the clock is roughly synchronized (NTP) | Weak — convergence eventually, but concurrent writes may oscillate with clock skew; no concurrent-write detection | Lowest — one timestamp per write; no clock vector | Clock skew = oscillation (P4 violation); concurrent writes silently lost; not a correctness-safe strategy for durable state |
|
||||
| Three-way merge (application-specific) | The data model is structured (documents, forms); conflicts should surface to the user or a domain-specific resolver; the merge is field-level | Bounded if the merge function is correct (associative, commutative, idempotent — P4, P5); conflicts surface per field | High — application-specific merge function per type; UI for conflict resolution; user-facing conflict surface | Merge-function bug = silent divergence or oscillation; unbounded conflict UI = user fatigue |
|
||||
|
||||
- The default for structured state that must converge silently is a
|
||||
**CRDT** (state-based for unreliable networks, operation-based for
|
||||
bandwidth-constrained reliable transport). The default for
|
||||
arbitrary JSON where lost concurrent updates are acceptable is
|
||||
**LWW with vector clocks** (never wall-clock-only for durable
|
||||
state). The default for user-facing documents where conflicts
|
||||
should surface is **three-way merge** with a documented resolution
|
||||
policy.
|
||||
- The failure-mode column is the P4 check: every row except
|
||||
wall-clock-only LWW carries a bounded failure mode (the bug is in
|
||||
the implementation, not the strategy). Wall-clock-only LWW carries
|
||||
an unbounded failure mode (clock skew = oscillation) and is a P4
|
||||
violation for durable state. Use it only for ephemeral state
|
||||
where lost updates are acceptable.
|
||||
- The choice is a P4 decision (which bound) and a C1 decision
|
||||
(which correctness cost). A CRDT guarantees convergence but
|
||||
constrains the data model; LWW is flexible but discards concurrent
|
||||
writes. Neither is universally correct; the matrix is the
|
||||
decision tool.
|
||||
|
||||
## Observability of Sync (P10 Edge Observability Survives Partition)
|
||||
|
||||
- Sync is itself an observable operation: the merge count, the
|
||||
conflict count, the convergence lag (time from reconnect to
|
||||
convergence), and the divergent-replica count are first-class
|
||||
signals. A sync that runs forever without converging is the
|
||||
`edge-sync-loop` chaos anti-pattern (P4 breach); without
|
||||
observability it is invisible until the user notices the stale
|
||||
state.
|
||||
- A conflict that is silently resolved should be logged (the
|
||||
resolution policy applied, the discarded write's idempotency key,
|
||||
the winning write's clock). A conflict that surfaces to the user
|
||||
should be metricated (the conflict rate, the resolution time).
|
||||
See `domains/observability/metrics` for the generic discipline.
|
||||
- A divergent replica that has not converged after the expected
|
||||
window is an incident; without a metric it is invisible (P10
|
||||
breach). Wire sync convergence to an alert — the
|
||||
`divergent-replica-count` is the sync analog of the messaging
|
||||
`consumer-lag` metric.
|
||||
|
||||
## What Violates Sync Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| Sync loop that oscillates forever (CRDT without merge-semantics, LWW without monotonic clock) | P4 Sync Conflicts are Bounded, Not Infinite |
|
||||
| LWW with wall-clock timestamp only on durable state (clock skew = oscillation) | P4, C1 (no concurrent-write detection) |
|
||||
| Merge function that mutates inputs in place (retried merge is not idempotent) | P5 Edge Operations are Idempotent, `domains/concurrency/patterns` |
|
||||
| Big-bang schema change under sync (replicas fail the merge) | P4, `domains/data/migrations` |
|
||||
| Conflict silently resolved with no log (the policy is invisible) | P10 Edge Observability Survives Partition |
|
||||
| Divergent replica with no convergence-lag metric (invisible stale state) | P10, `domains/observability/metrics` |
|
||||
| Three-way merge with an unbounded conflict UI (user fatigue, no termination) | P4 (the merge must terminate) |
|
||||
| Operation-based CRDT without causal delivery (lost ops = divergence) | P4, C1 (the delivery contract is the bound) |
|
||||
| Merge that surfaces every concurrent conflict to the user (no default policy) | P4, C3 (the default policy is the simplicity bound) |
|
||||
| Sync with no convergence test (the merge is untested under partition) | P4, `domains/testing/pyramid` |
|
||||
@@ -1,379 +0,0 @@
|
||||
# Delivery Semantics — Derived Rules
|
||||
|
||||
> Derives from `domains/messaging/first-principles.md`. Applies P4
|
||||
> (Delivery Semantics are Explicit), P3 (Consumers are Idempotent),
|
||||
> P2 (Ordering is a Property, Not an Assumption), P5 (Dead-Letter
|
||||
> Handling is Defined), and P6 (Backpressure is Bounded) primarily,
|
||||
> with P10 (DLQ depth as an alert). For the dead-letter strategy
|
||||
> decision, see the comparison table below. Includes a fenced
|
||||
> idempotency-key dedup-store example (IDEATE-39). Cross-links
|
||||
> `domains/concurrency/patterns` for the in-process retry/backoff
|
||||
> analog, `domains/errors/patterns` for errors as data for message
|
||||
> failures, and `domains/observability/metrics` for DLQ depth as an
|
||||
> alert.
|
||||
|
||||
## The Three Delivery Semantics (P4 Delivery Semantics are Explicit)
|
||||
|
||||
- **At-most-once**: a message is delivered 0 or 1 times; loss is
|
||||
possible, duplication is not. The producer fires-and-forgets; the
|
||||
broker does not ack; the consumer does not dedup. Lowest latency,
|
||||
lowest implementation cost, lossy. Fits telemetry where a dropped
|
||||
sample is acceptable (MQTT QoS 0, fire-and-forget metrics).
|
||||
- **At-least-once**: a message is delivered 1 or more times;
|
||||
duplication is possible, loss is not. The producer sends and
|
||||
waits for a broker ack; the consumer processes and acks; a crash
|
||||
before the consumer's ack triggers redelivery. The consumer must
|
||||
be idempotent (P3). The default for side-effecting operations
|
||||
(orders, payments, commands). The vast majority of broker-backed
|
||||
queues (SQS standard, RabbitMQ ack, MQTT QoS 1).
|
||||
- **Exactly-once**: a message is delivered exactly 1 time; no
|
||||
loss, no duplication. In practice, this is at-least-once plus
|
||||
idempotency (P3), or a transactional consume-process-produce loop
|
||||
(P4 — see `domains/messaging/streams.md`). Jepsen analyses verify
|
||||
broker claims: "exactly-once" requires independent verification;
|
||||
the durable engineering practice is at-least-once with idempotent
|
||||
consumers (P3), which collapses to exactly-once under correct
|
||||
dedup.
|
||||
- The semantic is declared per channel (P4), not emergent. An
|
||||
unstated semantic is a defect: the consumer guesses, and the
|
||||
guess is wrong under the first failure. See
|
||||
`domains/messaging/queues.md` for the three-semantics comparison
|
||||
table (semantics, latency cost, implementation cost, when each
|
||||
fits).
|
||||
|
||||
## Idempotency (P3 Consumers are Idempotent)
|
||||
|
||||
- Idempotency is the correctness property that makes at-least-once
|
||||
safe. A consumer that processes the same message twice has the
|
||||
same effect as processing it once. The mechanism is the
|
||||
idempotency key: a per-message unique identifier the consumer
|
||||
uses to dedup redeliveries.
|
||||
- The idempotency key is per-message, not per-producer or
|
||||
per-session. A consumer that dedups by producer alone drops
|
||||
distinct messages issued in the same window. Use a UUID per
|
||||
message, or a deterministic key derived from the message content
|
||||
(e.g., `(entity, operation, version)`).
|
||||
- The dedup store is bounded (P6 — Backpressure is Bounded): a
|
||||
dedup store that grows without bound is a memory leak. Use a TTL
|
||||
window longer than the broker's max-redelivery window, or a
|
||||
bounded LRU. The TTL is the P6 bound: a key seen within the TTL
|
||||
is a redelivery; a key older than the TTL is expired (the broker
|
||||
has given up redelivering it).
|
||||
|
||||
## Idempotency-Key Dedup Store (IDEATE-39, P3 — fenced example)
|
||||
|
||||
The dedup store is the concrete mechanism that makes a consumer
|
||||
idempotent under at-least-once delivery. This is the fenced
|
||||
example required by IDEATE-39 (parallel to the v0.3 IDEATE-29
|
||||
signed-attestation fenced example): it is NOT prose-only — the
|
||||
consumer-with-dedup-store demonstrates P3 concretely.
|
||||
|
||||
```python
|
||||
# Idempotent consumer with a dedup store (P3 Consumers are
|
||||
# Idempotent, P6 Backpressure is Bounded — the dedup store is
|
||||
# TTL-bounded). The consumer dedups by idempotency key before
|
||||
# processing; a redelivered message is a no-op, not a double-apply.
|
||||
|
||||
import time
|
||||
|
||||
# P6: the dedup store is bounded by a TTL window. The TTL must
|
||||
# exceed the broker's max-redelivery window; beyond the TTL, the
|
||||
# key is expired (the broker has given up). A dedup store with no
|
||||
# TTL is a memory leak (P6 violation).
|
||||
DEDUP_TTL_SECONDS = 24 * 3600 # longer than max-redelivery window
|
||||
|
||||
class DedupStore:
|
||||
"""A TTL-bounded idempotency-key dedup store (P3, P6).
|
||||
|
||||
seen(key): True if the key was processed within the TTL.
|
||||
mark(key): Record the key as processed (with a timestamp).
|
||||
"""
|
||||
def __init__(self, backend):
|
||||
# backend is a Redis, a DB, or an in-process LRU. The
|
||||
# backend must be shared across consumer instances if the
|
||||
# subscription is shared (P2/P3 — see domains/messaging/
|
||||
# pubsub.md on shared vs independent subscriptions).
|
||||
self.backend = backend
|
||||
|
||||
def seen(self, key: str) -> bool:
|
||||
ts = self.backend.get(key)
|
||||
if ts is None:
|
||||
return False
|
||||
if time.time() - ts > DEDUP_TTL_SECONDS:
|
||||
# P6: expired. The broker has given up redelivering;
|
||||
# this key is no longer a redelivery signal.
|
||||
self.backend.delete(key)
|
||||
return False
|
||||
return True
|
||||
|
||||
def mark(self, key: str):
|
||||
self.backend.set(key, time.time(), ttl=DEDUP_TTL_SECONDS)
|
||||
|
||||
|
||||
# The idempotent consumer: dedup before process, mark after
|
||||
# process, ack after mark. A crash before mark re-processes (the
|
||||
# dedup store does not have the key); a crash before ack
|
||||
# redelivers (the broker did not see the ack) and the dedup store
|
||||
# makes the redelivery a no-op (P3).
|
||||
def consume_idempotent(broker, dedup: DedupStore, process):
|
||||
for message in broker.receive():
|
||||
# P3: dedup BEFORE process. A redelivered message is a
|
||||
# no-op, not a double-apply.
|
||||
if dedup.seen(message["idempotencyKey"]):
|
||||
broker.ack(message) # already processed; skip
|
||||
continue
|
||||
try:
|
||||
process(message["payload"])
|
||||
# P3: mark AFTER process succeeds. A crash between
|
||||
# process and mark re-processes (acceptable: the
|
||||
# process must be idempotent OR the mark must be
|
||||
# transactional with the process — see below).
|
||||
dedup.mark(message["idempotencyKey"])
|
||||
broker.ack(message)
|
||||
except TransientError as exc:
|
||||
# P6: bounded retry with backoff. Nack for redelivery;
|
||||
# the broker redelivers after exponential backoff.
|
||||
broker.nack(message, delay=backoff(message["attempt"]))
|
||||
except PoisonError as exc:
|
||||
# P5: poison message. Route to DLQ, do NOT retry
|
||||
# forever (see the DLQ routing rule below).
|
||||
route_to_dlq(broker, message, exc)
|
||||
broker.ack(message) # remove from the origin queue
|
||||
```
|
||||
|
||||
- The order `process → mark → ack` gives at-least-once with
|
||||
idempotent dedup: a crash before `mark` re-processes (the dedup
|
||||
store does not have the key), and a crash before `ack`
|
||||
redelivers (the dedup store makes the redelivery a no-op). If
|
||||
`process` is not itself idempotent, the `process → mark` window
|
||||
must be transactional (e.g., process and mark in one DB
|
||||
transaction) — otherwise a crash in the window double-applies.
|
||||
- For a non-idempotent `process` (e.g., a payment that must not
|
||||
double-charge), use a transactional dedup: process and mark in
|
||||
one DB transaction, so the mark commits iff the process
|
||||
commits. This is the "exactly-once via idempotency" pattern
|
||||
(P4): at-least-once delivery plus a transactional
|
||||
process-and-mark collapses to exactly-once under correct
|
||||
transactional semantics.
|
||||
|
||||
## Ordering (P2 Ordering is a Property, Not an Assumption)
|
||||
|
||||
- The delivery semantic interacts with ordering (P2). At-least-once
|
||||
with per-partition ordering: a redelivery within a partition
|
||||
preserves order (the redelivered message re-appears in its
|
||||
original position relative to other messages the consumer has
|
||||
not yet seen). At-least-once with no ordering: a redelivery may
|
||||
appear out of order relative to messages delivered after it.
|
||||
- The consumer must not assume an ordering property the broker
|
||||
does not provide (P2). A standard queue delivers per-receive-node
|
||||
arrival order but no global order and no order across redeliveries;
|
||||
a FIFO queue delivers strict per-group order including across
|
||||
redeliveries; a partitioned stream delivers strict per-partition
|
||||
order across redeliveries. Document the property; do not assume
|
||||
it. See `domains/messaging/queues.md` (FIFO vs standard) and
|
||||
`domains/messaging/streams.md` (per-partition order).
|
||||
|
||||
## Dead-Letter Strategies (P5 Dead-Letter Handling is Defined)
|
||||
|
||||
- A poison message (unparseable, repeatedly failing, or exhausting
|
||||
the retry budget) must be routed to a dead-letter queue, not
|
||||
retried forever or silently dropped (P5). The DLQ is observable
|
||||
(P10 — depth is an alert) and drainable (an operator can
|
||||
inspect, replay, or discard with audit).
|
||||
- The dead-letter strategy determines when a message is
|
||||
dead-lettered and what the operator sees. The choice is the
|
||||
decision matrix below (D-069).
|
||||
|
||||
## Dead-Letter Strategy Comparison (D-069)
|
||||
|
||||
| Strategy | When It Applies | Failure Visibility | Operational Cost |
|
||||
|----------|-----------------|-------------------|------------------|
|
||||
| **Retry-count-limit** | A fixed max-redeliveries count (e.g., 5). After N redeliveries, route to DLQ. Simple, predictable. | The redelivery count is visible in the DLQ entry; the operator sees how many times it was retried. | Low — a counter per message; no backoff tuning. Risk: retries fire as fast as the broker redelivers, hammering a downstream that is already failing (P6 — no backoff = no backpressure escape). |
|
||||
| **TTL-with-backoff** | A max time-to-live for redelivery (e.g., 30 minutes) with exponential backoff between retries. After the TTL, route to DLQ. | The TTL and the backoff schedule are visible; the operator sees the retry timeline. | Medium — backoff tuning per message type. Benefit: backoff gives the downstream time to recover (P6 — the retry rate is bounded); fits transient failures (a downstream that is briefly unavailable). |
|
||||
| **Poison-queue** | A separate queue for messages that fail a specific check (unparseable, schema-invalid, unknown type) before any processing retry. Routed immediately, not retried. | The poison queue is a separate signal from the DLQ; the operator sees parse-vs-process failures distinctly. | Low — a routing rule per check. Benefit: distinguishes "never going to succeed" (poison) from "might succeed on retry" (DLQ). Use for unparseable messages that no retry will fix. |
|
||||
| **DLQ + alert** | Any of the above strategies, plus an alert on DLQ depth. The DLQ is observable (P10) — depth, age, and rate are alerted. | Highest — the operator is paged on DLQ growth; the DLQ is a first-class signal, not a graveyard. | Medium — alerting setup per DLQ. This is the P5/P10 floor: a DLQ without an alert is a silent correctness defect (poison messages accumulate invisibly). |
|
||||
|
||||
- The default for transient failures is **TTL-with-backoff + DLQ
|
||||
+ alert**: backoff gives the downstream time to recover (P6),
|
||||
the TTL bounds the retry budget (P5), the DLQ captures the
|
||||
unprocessable, and the alert makes it visible (P10). The default
|
||||
for unparseable messages is **poison-queue + alert**: route
|
||||
immediately, do not retry a message no retry will fix.
|
||||
- **Retry-count-limit alone** (no backoff, no alert) is the
|
||||
`messaging-unbounded-retry` chaos anti-pattern's cousin: it
|
||||
caps the count but hammers the downstream at full retry rate,
|
||||
and the DLQ grows silently if no alert is wired. Always pair
|
||||
a retry budget with backoff (P6) and an alert (P10).
|
||||
- The failure-visibility column is the P10 check: every strategy
|
||||
must surface the failure to the operator. A strategy with no
|
||||
visibility is a P10 violation regardless of its retry semantics.
|
||||
- The operational-cost column is the C8 tradeoff: more visibility
|
||||
and more backoff cost more to set up but pay back in operational
|
||||
stability. The P5/P10 floor is "DLQ + alert"; below that, the
|
||||
strategy is a silent defect waiting to grow.
|
||||
|
||||
## DLQ Routing Rule (P5, P10)
|
||||
|
||||
```python
|
||||
# DLQ routing rule (P5 dead-letter handling, P6 bounded retry with
|
||||
# backoff, P10 DLQ depth alert). Combines TTL-with-backoff for
|
||||
# transient failures and poison-queue for unparseable messages,
|
||||
# with an alert on DLQ depth.
|
||||
|
||||
import json, time
|
||||
|
||||
MAX_RETRY_TTL_SECONDS = 30 * 60 # 30 min total retry window
|
||||
DLQ = "orders-dlq"
|
||||
POISON = "orders-poison" # unparseable; never retried
|
||||
|
||||
def route_to_dlq(broker, message, reason):
|
||||
"""Route a message to the DLQ with audit metadata (P5)."""
|
||||
broker.send(DLQ, body=json.dumps({
|
||||
"original": message,
|
||||
"reason": str(reason),
|
||||
"deadLetteredAt": now_iso(),
|
||||
"redeliveryCount": message.get("attempt", 0),
|
||||
}))
|
||||
# P10: emit a metric so DLQ depth alerts fire. A DLQ that
|
||||
# grows with no alert is a silent correctness defect (P5/P10).
|
||||
metrics.increment("dlq.depth", tags={"queue": "orders"})
|
||||
# The ack removes the message from the origin queue; the DLQ
|
||||
# is the durable record (P5 — observable and drainable).
|
||||
|
||||
def consume_with_dlq(broker, dedup, process):
|
||||
for message in broker.receive():
|
||||
# P1: parse first. An unparseable message is poison —
|
||||
# route immediately, do NOT retry (no retry will fix it).
|
||||
try:
|
||||
payload = json.loads(message["body"])
|
||||
except (ValueError, SchemaError) as exc:
|
||||
broker.send(POISON, body=json.dumps({
|
||||
"original": message["body"],
|
||||
"reason": f"parse-failed: {exc}",
|
||||
"poisonedAt": now_iso(),
|
||||
}))
|
||||
metrics.increment("poison.depth", tags={"queue": "orders"})
|
||||
broker.ack(message) # remove from origin; poison queue holds it
|
||||
continue
|
||||
|
||||
# P3: dedup before process.
|
||||
if dedup.seen(payload["idempotencyKey"]):
|
||||
broker.ack(message); continue
|
||||
|
||||
attempt = payload.get("attempt", 0)
|
||||
first_attempt_ts = payload.get("firstAttemptTs", time.time())
|
||||
|
||||
try:
|
||||
process(payload)
|
||||
dedup.mark(payload["idempotencyKey"])
|
||||
broker.ack(message)
|
||||
except TransientError as exc:
|
||||
# P6: TTL-with-backoff. If the retry window is
|
||||
# exhausted, route to DLQ; otherwise redeliver with
|
||||
# exponential backoff.
|
||||
if time.time() - first_attempt_ts > MAX_RETRY_TTL_SECONDS:
|
||||
route_to_dlq(broker, message, exc) # P5
|
||||
broker.ack(message)
|
||||
else:
|
||||
broker.nack(message, delay=backoff(attempt))
|
||||
except PermanentError as exc:
|
||||
# A permanent error (e.g., a not-found dependency)
|
||||
# does not benefit from retry — route to DLQ now.
|
||||
route_to_dlq(broker, message, exc)
|
||||
broker.ack(message)
|
||||
```
|
||||
|
||||
- The routing rule distinguishes three failure modes: **poison**
|
||||
(unparseable — route immediately, no retry), **transient**
|
||||
(retry with backoff until the TTL, then DLQ), and **permanent**
|
||||
(a retry will not fix it — DLQ now). This distinction is the P5
|
||||
discipline: not every failure is a retry; some are immediate
|
||||
DLQs.
|
||||
- The DLQ entry carries `reason`, `deadLetteredAt`, and
|
||||
`redeliveryCount` — it is auditable (the operator knows why each
|
||||
message was dead-lettered and how many times it was retried).
|
||||
This is the errors-as-data discipline — see
|
||||
`domains/errors/patterns` for the general principle a DLQ entry
|
||||
instantiates.
|
||||
|
||||
## Retry Budgets and Backoff (P6 Backpressure is Bounded)
|
||||
|
||||
- The retry budget is the cap on redelivery: a count, a TTL, or
|
||||
both. After the budget, the message routes to the DLQ (P5). An
|
||||
unbounded retry budget is the `messaging-unbounded-retry` chaos
|
||||
anti-pattern (P5 breach): the consumer never makes progress past
|
||||
the poison message.
|
||||
- **Exponential backoff** spaces retries: 1s, 2s, 4s, 8s, ... with
|
||||
a jitter to avoid thundering-herd synchrony. Backoff gives the
|
||||
downstream time to recover (P6 — the retry rate is bounded,
|
||||
giving the downstream a chance to catch up). A retry with no
|
||||
backoff hammers the downstream at full rate, making the failure
|
||||
worse.
|
||||
- The retry budget × the backoff schedule is the P6 bound: the
|
||||
consumer's retry load is bounded by design, not by luck. See
|
||||
`domains/concurrency/patterns` Pattern 6 (Timeout on Every
|
||||
Block) for the in-process retry/backoff analog; messaging owns
|
||||
the broker-backed instance where the redelivery comes from the
|
||||
broker across a network, not an in-process loop (D-062).
|
||||
|
||||
## Poison Messages (P5, P1)
|
||||
|
||||
- A poison message is one no retry will fix: unparseable (the
|
||||
schema is wrong, P1), unknown type (the consumer does not handle
|
||||
this version, P9), or a permanent failure (a not-found
|
||||
dependency). Retrying a poison message wastes resources and
|
||||
blocks the queue (P6 — the consumer never makes progress).
|
||||
- Poison messages route to the poison queue immediately (no
|
||||
retry), distinct from the DLQ (which holds messages that
|
||||
exhausted their retry budget on transient failures). The
|
||||
distinction is the P5 discipline: a poison queue is for
|
||||
"never going to succeed"; a DLQ is for "might have succeeded
|
||||
but didn't within the budget."
|
||||
- A poison queue without an alert is the same defect as a DLQ
|
||||
without an alert (P10): the operator cannot see the poison
|
||||
accumulating. Wire both to `domains/observability/metrics`.
|
||||
|
||||
## Cross-Link to Concurrency (P6, cross-link concurrency/patterns)
|
||||
|
||||
- The retry/backoff discipline here is the cross-process analog
|
||||
of `domains/concurrency/patterns` Pattern 6 (Timeout on Every
|
||||
Block) for in-process retry. Concurrency owns the in-process
|
||||
analog (a retry loop within one program, with a timeout per
|
||||
attempt); messaging owns the broker-backed instance (the broker
|
||||
redelivers across a network, the consumer applies backoff via
|
||||
nack-with-delay). The failure model differs: in-process retry
|
||||
fails by a thread crash or a timeout; broker-backed retry fails
|
||||
by network partition, broker restart, or consumer crash-and-retry
|
||||
(D-062).
|
||||
- The cross-link is one-directional outward (messaging →
|
||||
concurrency) per D-026 extended: messaging references concurrency
|
||||
as the in-process foundation; concurrency does not back-link to
|
||||
messaging.
|
||||
|
||||
## Cross-Link to Errors (P5, cross-link errors/patterns)
|
||||
|
||||
- A poison message is an errors-as-data instance: the failure is
|
||||
captured as a DLQ entry (with `reason`, `redeliveryCount`,
|
||||
`deadLetteredAt`), not swallowed. See `domains/errors/patterns`
|
||||
for the general errors-as-data discipline a DLQ entry
|
||||
instantiates. The DLQ is the async-messaging instance of an
|
||||
error log — observable, auditable, drainable.
|
||||
- The cross-link is one-directional outward (messaging → errors):
|
||||
messaging references errors for the errors-as-data pattern;
|
||||
errors does not back-link to messaging.
|
||||
|
||||
## What Violates Delivery-Semantics Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| Unstated delivery semantic (at-least-once vs exactly-once guessed) | P4 Delivery Semantics are Explicit |
|
||||
| Non-idempotent consumer under at-least-once delivery | P3 Consumers are Idempotent |
|
||||
| Dedup store with no TTL (memory leak; grows without bound) | P6 Backpressure is Bounded |
|
||||
| Dedup key per-producer (distinct messages in the same window deduped) | P3 Consumers are Idempotent |
|
||||
| Retry with no backoff (hammers the downstream at full rate) | P6 Backpressure is Bounded |
|
||||
| Unbounded retry budget (consumer never progresses past the poison) | P5 Dead-Letter Handling is Defined |
|
||||
| DLQ with no depth alert (poison messages accumulate invisibly) | P10, `domains/observability/metrics` |
|
||||
| Poison message retried forever (no poison queue, no immediate DLQ) | P5 Dead-Letter Handling is Defined |
|
||||
| Process-not-idempotent with non-transactional mark (crash in window double-applies) | P3, P4 |
|
||||
| DLQ entry with no reason/audit metadata (uninspectable failure) | P5, `domains/errors/patterns` |
|
||||
| Ordering assumption the broker does not provide (FIFO assumed on standard queue) | P2 Ordering is a Property, Not an Assumption |
|
||||
@@ -1,307 +0,0 @@
|
||||
# Messaging — First Principles
|
||||
|
||||
## 1. The Principles
|
||||
|
||||
### P1. Messages are Contracts
|
||||
A message has an explicit, versioned schema. Producer and consumer
|
||||
agree on shape before exchange; the schema is the boundary, not a
|
||||
guess. A schemaless message — a free-form JSON blob the consumer
|
||||
parses by hope — is a defect: the consumer breaks silently on the
|
||||
next shape change, and the producer has no contract to evolve
|
||||
against. This derives from `C1 Correctness` (the exchange must
|
||||
carry what the parties agreed to) and `C2 Clarity` (the schema
|
||||
makes the boundary obvious to both sides). This is the
|
||||
cross-process expression of the contract discipline that
|
||||
`domains/api/rest` owns for synchronous request/response: the
|
||||
message schema is to async exchange what the API contract is to
|
||||
sync exchange. It is distinct from
|
||||
`domains/concurrency/patterns` Pattern 1 (Message Passing), which
|
||||
owns the *in-process* channel primitive — here the contract spans
|
||||
separate systems and survives network failure (D-062). See
|
||||
`domains/messaging/queues.md` for the queue-flavored application
|
||||
and `domains/messaging/streams.md` for the durable-log-flavored
|
||||
application.
|
||||
|
||||
### P2. Ordering is a Property, Not an Assumption
|
||||
Ordering guarantees — per-partition strict, global, or none — are
|
||||
explicit and documented. "It's FIFO" is a claim that must be backed
|
||||
by the broker's partitioning contract, not an assumption the
|
||||
consumer makes and the broker may not honor. A standard queue
|
||||
delivers in arrival order per receive-node but offers no global
|
||||
ordering across shards; a FIFO queue delivers strict per-message-
|
||||
group order but at a latency cost; a partitioned stream delivers
|
||||
strict per-partition order but only within a partition. Each is a
|
||||
distinct, declared property. This derives from `C1 Correctness`
|
||||
(order is a correctness property — a consumer that assumes order
|
||||
the broker does not provide is wrong) and `C2 Clarity` (the
|
||||
ordering guarantee is documented, not discovered in production).
|
||||
This is distinct from in-process ordering, which
|
||||
`domains/concurrency/patterns` Pattern 1 owns for channels within
|
||||
one program: messaging ordering survives network failure, broker
|
||||
restart, and consumer crash-and-retry — a stronger failure model
|
||||
than thread-local channels (D-062). The `messaging-shared-
|
||||
subscription` chaos anti-pattern breaches this rule: two consumers
|
||||
sharing one subscription break per-consumer ordering because the
|
||||
broker dispatches each message to an arbitrary consumer. See
|
||||
`domains/messaging/streams.md` for the partition-order contract
|
||||
and `domains/messaging/delivery-semantics.md` for the interaction
|
||||
of ordering with the three delivery semantics.
|
||||
|
||||
### P3. Consumers are Idempotent
|
||||
Delivery is at-least-once by default across the network; a
|
||||
consumer deduplicates via idempotency keys or deterministic
|
||||
processing. "Exactly-once" is idempotency plus at-least-once, not a
|
||||
broker guarantee — Jepsen analyses of Kafka, RabbitMQ, and NATS
|
||||
establish that exactly-once claims require independent
|
||||
verification, and the durable engineering practice is to make
|
||||
consumers idempotent under redelivery. A non-idempotent consumer
|
||||
under at-least-once delivery doubles the effect on every retry; a
|
||||
non-idempotent consumer under a claimed exactly-once broker is a
|
||||
bug waiting for the broker's exactly-once invariant to break. This
|
||||
derives from `C1 Correctness`: correctness under redelivery is the
|
||||
contract, not a nice-to-have. This parallels
|
||||
`domains/edge/P5 Edge Operations are Idempotent` (the
|
||||
cross-partition device-and-cache-flavored analog) and is the
|
||||
cross-process instance of the retry-safety discipline that
|
||||
`domains/concurrency/patterns` Pattern 6 (Timeout on Every Block)
|
||||
implies for in-process retry. It is distinct from in-process
|
||||
retry because the redelivery comes from the broker across a
|
||||
network, not from an in-process loop (D-062). See
|
||||
`domains/messaging/delivery-semantics.md` for the idempotency-key
|
||||
dedup-store pattern.
|
||||
|
||||
### P4. Delivery Semantics are Explicit
|
||||
At-least-once / at-most-once / exactly-once is a declared choice
|
||||
per channel, not an emergent behavior. The tradeoff — latency cost,
|
||||
implementation complexity, operational cost — is made consciously
|
||||
and documented. At-most-once is fire-and-forget (low latency, lossy);
|
||||
at-least-once is acked with possible duplication (the default,
|
||||
requires idempotent consumers per P3); exactly-once is at-least-once
|
||||
plus idempotency or a transactional two-phase commit (highest cost,
|
||||
narrowest fit). An unstated semantic is a defect: the consumer
|
||||
guesses, and the guess is wrong under the first failure. This
|
||||
derives from `C1 Correctness` (the chosen semantic must hold) and
|
||||
`C2 Clarity` (the tradeoff is visible to the reader and the
|
||||
operator). This is the cross-process analog of the explicit-failure-
|
||||
mode discipline that `domains/errors/patterns` owns for synchronous
|
||||
code — messaging makes the delivery-mode choice as explicit as an
|
||||
error-handling choice. See `domains/messaging/queues.md` for the
|
||||
three-semantics comparison table and `domains/messaging/delivery-
|
||||
semantics.md` for the correctness properties of each.
|
||||
|
||||
### P5. Dead-Letter Handling is Defined
|
||||
Poison messages — unparseable, repeatedly failing, or exhausting
|
||||
the retry budget — are routed to a dead-letter queue, not retried
|
||||
forever or silently dropped. The DLQ is observable and drainable: an
|
||||
operator can inspect it, replay from it, or discard with audit. An
|
||||
unbounded retry loop is a livelock: the consumer never makes
|
||||
progress past the poison message. A silent drop is a correctness
|
||||
defect: the message vanished with no record. This derives from `C1
|
||||
Correctness` (poison messages must not livelock the consumer or
|
||||
silently disappear) and `C5 Reversibility` (the DLQ is the
|
||||
reversibility mechanism — a dead-lettered message can be reprocessed
|
||||
after the bug is fixed). This is the cross-process analog of the
|
||||
bounded-error discipline that `domains/errors/patterns` owns for
|
||||
synchronous code: a poison message is an error-as-data instance
|
||||
that must be observable and recoverable, not swallowed. It is
|
||||
distinct from in-process error handling because the failure spans a
|
||||
network and a consumer restart (D-062). See
|
||||
`domains/messaging/delivery-semantics.md` for the dead-letter
|
||||
strategy comparison table and the DLQ routing rule pattern.
|
||||
|
||||
### P6. Backpressure is Bounded
|
||||
A slow consumer cannot unbounded-buffer the broker or the
|
||||
producer. Backpressure is explicit: consumer lag is visible,
|
||||
max-unacked is bounded, the retry budget is capped. A consumer
|
||||
that falls behind without a visible signal is a silent backlog —
|
||||
the operator cannot fix what they cannot see, and the broker's
|
||||
memory grows without bound until it fails. This derives from `C1
|
||||
Correctness` (a backlog that grows until OOM is a correctness
|
||||
failure) and `C8 Economy` (the broker's memory is bounded by
|
||||
design, not by luck). This is distinct from
|
||||
`domains/concurrency/P9 Bounded Queues` and
|
||||
`domains/concurrency/patterns` Pattern 5 (Bounded Queue with
|
||||
Backpressure), which own the *in-process* analog: concurrency's
|
||||
bounded queue fails by OOM or thread crash; messaging's bounded
|
||||
backpressure fails by network partition, broker restart, or
|
||||
consumer crash-and-retry (D-062). The Reactive Streams
|
||||
specification (`request(n)`, `onNext` bounded) is the in-process
|
||||
instance; messaging's broker-backed backpressure is the
|
||||
cross-process instance above it. See `domains/messaging/queues.md`
|
||||
for prefetch and max-unacked and `domains/observability/metrics`
|
||||
for consumer-lag as an alert.
|
||||
|
||||
### P7. Partitioning is Intentional
|
||||
The partition key determines ordering, parallelism, and hotspots.
|
||||
Key choice is a design decision with documented rationale, not a
|
||||
default. A key that hashes unevenly creates a hot partition that
|
||||
limits throughput; a key that does not match the ordering need
|
||||
breaks per-key semantics; a key that is too coarse (one partition
|
||||
for the whole topic) serializes all the traffic. The partition
|
||||
count is a capacity bound: too few partitions cap parallelism, too
|
||||
many partition overhead the broker. This derives from `C4 Locality`
|
||||
(ordering and parallelism are co-located with the partition) and
|
||||
`C6 Composability` (the partition is the unit of parallelism and
|
||||
scaling — consumer groups compose from per-partition workers).
|
||||
This is the cross-process analog of the locality discipline that
|
||||
`domains/performance/` owns for generic data-near-compute
|
||||
optimization: performance's locality is algorithmic (data near
|
||||
compute); messaging's locality is partitional (order and
|
||||
parallelism near the partition). See `domains/messaging/streams.md`
|
||||
for the partitioned-log model and consumer-group rebalance
|
||||
strategies.
|
||||
|
||||
### P8. Replay and Retention are Configured
|
||||
Retention windows and replay-from-offset are explicit. A message
|
||||
is not ephemeral by default; the broker is a durable log, not a
|
||||
pipe. A topic with no retention is a fire-and-forget stream — a
|
||||
consumer that falls behind loses data permanently; a topic with
|
||||
infinite retention is an unbounded log — the broker grows until
|
||||
disk exhaustion. Both are defects: the retention window is a
|
||||
declared bound, and replay-from-offset is the mechanism that makes
|
||||
the log durable (re-consumable) rather than ephemeral. This derives
|
||||
from `C5 Reversibility` (a retained message is reversible — it can
|
||||
be re-consumed; an ephemeral message is not) and `C7 Observability`
|
||||
(the durable log is itself an observable record of what happened —
|
||||
the offset is the position from which to replay). This is the
|
||||
foundation for `domains/messaging/streams.md` and the rule that
|
||||
distinguishes a stream from a queue (a queue deletes on ack; a
|
||||
stream retains for replay). See `domains/messaging/pubsub.md` for
|
||||
the pub/sub-vs-stream durability boundary.
|
||||
|
||||
### P9. Schemas Evolve Compatibly
|
||||
Schema changes are backward- and forward-compatible by
|
||||
construction. Breaking changes are versioned migrations, not
|
||||
silent shape edits. A producer that ships a new field the old
|
||||
consumer ignores is backward-compatible; a consumer that handles a
|
||||
missing field the new producer omits is forward-compatible. A
|
||||
silent schema change — the producer renames a field and the
|
||||
consumer parses `undefined` — is a P1 violation (the contract was
|
||||
broken) compounded here as an evolution defect. This derives from
|
||||
`C5 Reversibility` (a schema change is reversible by versioning —
|
||||
the old shape is still readable) and `C6 Composability` (producers
|
||||
and consumers of different versions compose because the schema
|
||||
evolves compatibly). This parallels `domains/data/migrations`
|
||||
(schema migration for databases) and `domains/api/versioning`
|
||||
(API contract evolution): messaging's schema evolution is the
|
||||
async instance of the same compatibility discipline. See
|
||||
`domains/messaging/streams.md` for the stream-schema-evolution
|
||||
angle.
|
||||
|
||||
### P10. Messaging is Observable
|
||||
Consumer lag, DLQ depth, throughput, and consumer-group health are
|
||||
first-class signals. Silent backlog is a bug, not a feature: a
|
||||
consumer that falls behind with no lag metric is invisible until
|
||||
the downstream effect surfaces — by which time the backlog may be
|
||||
hours or days. A DLQ that grows without an alert is a silent
|
||||
correctness defect: poison messages are accumulating and no one
|
||||
knows. This derives from `C7 Observability` (the broker's behavior
|
||||
is visible to the operator) and `C1 Correctness` (backlog
|
||||
detection is a correctness bound — unbounded lag is a failure).
|
||||
This is distinct from `domains/observability/metrics`, which owns
|
||||
*generic* structured metrics; messaging owns the *broker-specific*
|
||||
signals — lag, DLQ depth, partition imbalance, consumer-group
|
||||
rebalance events. See `domains/observability/metrics` for the
|
||||
generic SLI/SLO discipline and `domains/observability/tracing` for
|
||||
cross-partition traces.
|
||||
|
||||
## 2. Core Principle Trace
|
||||
|
||||
Each messaging P-rule derives from one or more core C-rules
|
||||
(C1–C8). The matrix extension lands in P4 of the v0.4 plan; the
|
||||
traces below are authoritative. Messaging is a broad-derivation
|
||||
domain touching 7 of 8 core principles (C1, C2, C4, C5, C6, C7,
|
||||
C8); C3 (Simplicity) is not a primary derivation — messaging is
|
||||
inherently a tradeoff domain where simplicity yields to the
|
||||
correctness of delivery guarantees (a simpler-than-necessary
|
||||
delivery model does not handle the failure cases, per C3's
|
||||
"simpler than necessary is also a violation").
|
||||
|
||||
| P-rule | Core | Why |
|
||||
|--------|------|-----|
|
||||
| P1 Messages are Contracts | C1, C2 | Correctness of the exchange; clarity of the schema boundary |
|
||||
| P2 Ordering is a Property, Not an Assumption | C1, C2 | Correctness of order; clarity of the guarantee |
|
||||
| P3 Consumers are Idempotent | C1 | Correctness under redelivery |
|
||||
| P4 Delivery Semantics are Explicit | C1, C2 | Correctness of the chosen semantic; clarity of the tradeoff |
|
||||
| P5 Dead-Letter Handling is Defined | C1, C5 | Correctness of poison-message routing; reversibility of reprocessing |
|
||||
| P6 Backpressure is Bounded | C1, C8 | Correctness of bounded backlog; economy of broker memory |
|
||||
| P7 Partitioning is Intentional | C4, C6 | Locality of order; composability of parallelism |
|
||||
| P8 Replay and Retention are Configured | C5, C7 | Reversibility of replay; observability of the durable log |
|
||||
| P9 Schemas Evolve Compatibly | C5, C6 | Reversibility of schema changes; composability of versions |
|
||||
| P10 Messaging is Observable | C7, C1 | Observability of lag/DLQ; correctness of backlog detection |
|
||||
|
||||
## 3. What Violates These Principles
|
||||
|
||||
| Violation | Principle Breached |
|
||||
|-----------|-------------------|
|
||||
| Schemaless message (no versioned contract; consumer parses by guess) | P1 Messages are Contracts |
|
||||
| "It's FIFO" with no documented partition contract | P2 Ordering is a Property, Not an Assumption |
|
||||
| Non-idempotent consumer under at-least-once delivery | P3 Consumers are Idempotent |
|
||||
| Unstated delivery semantic (at-least-once vs exactly-once guessed) | P4 Delivery Semantics are Explicit |
|
||||
| No dead-letter queue (poison message retried forever or silently dropped) | P5 Dead-Letter Handling is Defined |
|
||||
| Unbounded retry budget (no cap; slow consumer stalls the partition) | P6 Backpressure is Bounded |
|
||||
| Default partition key (no rationale; hotspot or wrong-order) | P7 Partitioning is Intentional |
|
||||
| Ephemeral broker (no retention; no replay) | P8 Replay and Retention are Configured |
|
||||
| Silent schema change (producer breaks consumers with no version bump) | P9 Schemas Evolve Compatibly |
|
||||
| Silent backlog (no lag metric; consumer falls behind invisibly) | P10 Messaging is Observable |
|
||||
| Shared subscription (two consumers share one subscription; per-consumer ordering breaks) | P2 Ordering is a Property, Not an Assumption (P3 compounding) |
|
||||
| Blocking consumer (slow downstream call with no timeout; broker redelivers to the stuck consumer) | P6 Backpressure is Bounded |
|
||||
|
||||
## 4. Relationship to Other Domains
|
||||
|
||||
Messaging systems are the engineering discipline of
|
||||
**cross-process, cross-system asynchronous communication via
|
||||
brokers**. Producer and consumer are separate systems; the broker
|
||||
is the intermediary that brokers delivery, ordering, retention,
|
||||
and failure semantics. The distinguishing constraints are a
|
||||
cross-process failure model (network, not crash), explicit
|
||||
delivery semantics, decoupled producer/consumer lifecycle, and
|
||||
replay-and-retention as a durable-log property. Messaging overlaps
|
||||
`domains/concurrency/` by *subject* (messages, queues,
|
||||
backpressure) but not by *failure model*: per D-062, messaging
|
||||
owns the cross-process/network-failure-model angle; concurrency
|
||||
owns the in-process/crash-failure-model angle. The discriminator
|
||||
is the failure model: concurrency's queue fails by OOM or thread
|
||||
crash; messaging's queue fails by network partition, broker
|
||||
restart, or consumer crash-and-retry. Messaging extends
|
||||
concurrency's bounded-queue/backpressure model to the network-
|
||||
partition regime. Cross-links are one-directional outward (per
|
||||
D-026 extended); no back-link edits to v0.1/v0.2/v0.3 content.
|
||||
|
||||
- `domains/concurrency/patterns` ← P6 (the broker-backed bounded
|
||||
queue is the cross-process analog of the in-process bounded
|
||||
buffer — concurrency Pattern 5 owns in-process; messaging owns
|
||||
the network-failure-model instance above it, per D-062)
|
||||
- `domains/concurrency/patterns` ← P3 (idempotent retry is the
|
||||
cross-process analog of in-process retry-safety — the failure
|
||||
model differs: broker redelivery across a network vs in-process
|
||||
loop)
|
||||
- `domains/observability/metrics` ← P10 (consumer lag and DLQ
|
||||
depth as alerts; observability owns the generic SLI/SLO
|
||||
discipline, messaging owns the broker-specific signals)
|
||||
- `domains/observability/tracing` ← P10 (cross-partition traces
|
||||
for stream processing; observability owns the generic tracing
|
||||
discipline, messaging owns the cross-partition propagation)
|
||||
- `domains/data/schema-design` ← P1, P9 (message schema design
|
||||
and evolution; data owns the generic schema discipline,
|
||||
messaging owns the cross-process message-shape instance)
|
||||
- `domains/errors/patterns` ← P5 (errors as data for message
|
||||
failures; a poison message is an error-as-data instance that must
|
||||
be observable and recoverable, not swallowed)
|
||||
- `domains/edge/iot` ← P4 (the edge↔messaging cross-link
|
||||
resolves bidirectionally here: edge/iot.md links outward to
|
||||
messaging/queues for MQTT QoS parallels to delivery semantics;
|
||||
this first-principles doc acknowledges the back-link — the
|
||||
edge/iot.md → messaging/queues link from P1 now resolves because
|
||||
messaging/queues.md exists, completing the bidirectionality per
|
||||
IDEATE-40)
|
||||
|
||||
> Note: the edge/iot.md → messaging/queues cross-link (MQTT QoS
|
||||
> parallels for delivery semantics) was authored in P1 with a
|
||||
> dangling reference; this P2 authorship of messaging/queues.md
|
||||
> resolves it. The bidirectionality is verified in P5
|
||||
> (ATELIER-114 per IDEATE-40). The cross-link is one-directional
|
||||
> outward from edge/iot.md; this first-principles doc
|
||||
> acknowledges the resolution without editing edge/iot.md (per
|
||||
> D-026 extended — no back-link edits to v0.1/v0.2/v0.3 or to
|
||||
> P1-authored edge content).
|
||||
@@ -1,269 +0,0 @@
|
||||
# Pub/Sub — Derived Rules
|
||||
|
||||
> Derives from `domains/messaging/first-principles.md`. Applies P1
|
||||
> (Messages are Contracts), P2 (Ordering is a Property, Not an
|
||||
> Assumption), P3 (Consumers are Idempotent), and P4 (Delivery
|
||||
> Semantics are Explicit) primarily, with P7 (partitioning), P10
|
||||
> (per-subscription lag). The `messaging-shared-subscription` chaos
|
||||
> anti-pattern lives here (pre-specified in P4 ATELIER-110).
|
||||
> Cross-links `domains/messaging/streams` for the pub/sub-vs-stream
|
||||
> durability boundary and `domains/observability/metrics` for
|
||||
> per-subscription lag.
|
||||
|
||||
## What Pub/Sub Is (P1 Messages are Contracts)
|
||||
|
||||
- Pub/sub is the fan-out primitive: a producer publishes a message
|
||||
to a topic; N independent subscriptions each receive a copy. The
|
||||
message has an explicit, versioned schema (P1): the topic's
|
||||
schema is the contract every subscription agrees to before
|
||||
subscribing. A schemaless topic is a defect — every subscriber
|
||||
breaks silently on the next shape change.
|
||||
- The boundary with queues is the fan-out ratio. A queue is
|
||||
point-to-point (one producer, one consumer); pub/sub is
|
||||
one-to-many (one producer, N consumers, each with its own
|
||||
subscription). The boundary with streams is the durability model
|
||||
— see the cross-link below. Pub/sub is an async concern because
|
||||
producer and consumers are separate systems and the failure model
|
||||
is network, not crash (D-062).
|
||||
- See `domains/messaging/queues.md` for the point-to-point variant
|
||||
and `domains/messaging/streams.md` for the durable-log variant.
|
||||
|
||||
## Topic / Subscription Model (P1, P3, P4)
|
||||
|
||||
- A **topic** is the named stream of messages. A **subscription**
|
||||
is a durable cursor over the topic: each subscription receives
|
||||
every message published after it was created (subject to
|
||||
retention and filtering). The subscription is independent — its
|
||||
ack, redelivery, and DLQ are per-subscription, not shared.
|
||||
- Each subscription is a consumer under at-least-once by default
|
||||
(P4): the broker redelivers until the subscription acks, and the
|
||||
subscriber must be idempotent (P3). A subscription with no
|
||||
idempotency dedup duplicates every redelivered message.
|
||||
- The topic's schema evolves compatibly (P9 — Schemas Evolve
|
||||
Compatibly): a new field the old subscriber ignores is
|
||||
backward-compatible; a renamed field the old subscriber parses
|
||||
as `undefined` is a P1 violation.
|
||||
|
||||
```python
|
||||
# Publish + two independent subscriptions (P1 contract, P3
|
||||
# idempotency, P4 at-least-once per subscription). Each
|
||||
# subscription is an independent durable cursor; acking one does
|
||||
# not affect the other.
|
||||
|
||||
import json, uuid
|
||||
|
||||
# --- Publisher ---
|
||||
def publish(topic, event, broker):
|
||||
# P1: versioned schema on the topic. All subscribers must
|
||||
# understand this schema (or a compatible superset — P9).
|
||||
message = {
|
||||
"schema": "user.signed-up.v1",
|
||||
"id": str(uuid.uuid4()),
|
||||
"idempotencyKey": f"user:{event['userId']}:signup",
|
||||
"payload": event,
|
||||
}
|
||||
broker.publish(topic=topic, body=json.dumps(message))
|
||||
|
||||
# --- Subscription A: welcome-email service ---
|
||||
def subscribe_welcome(broker, dedup_store, send_email):
|
||||
sub = broker.subscribe(topic="users", subscription="welcome-email")
|
||||
for message in sub.receive():
|
||||
# P3: idempotent per subscription. A redelivered message is
|
||||
# a no-op for THIS subscription, not for the others.
|
||||
if dedup_store.seen(("welcome", message["idempotencyKey"])):
|
||||
sub.ack(message)
|
||||
continue
|
||||
send_email(message["payload"]["email"], "Welcome!")
|
||||
dedup_store.mark(("welcome", message["idempotencyKey"]))
|
||||
sub.ack(message)
|
||||
|
||||
# --- Subscription B: analytics-ingest service ---
|
||||
def subscribe_analytics(broker, dedup_store, ingest):
|
||||
# Independent subscription: its own cursor, its own dedup,
|
||||
# its own ack. Welcome-email acking does NOT advance this.
|
||||
sub = broker.subscribe(topic="users", subscription="analytics")
|
||||
for message in sub.receive():
|
||||
if dedup_store.seen(("analytics", message["idempotencyKey"])):
|
||||
sub.ack(message)
|
||||
continue
|
||||
ingest(message["payload"])
|
||||
dedup_store.mark(("analytics", message["idempotencyKey"]))
|
||||
sub.ack(message)
|
||||
```
|
||||
|
||||
- The dedup key is scoped per subscription: `(subscription,
|
||||
idempotencyKey)`. A redelivery to subscription A that was already
|
||||
processed by A is a no-op for A; the same message delivered to
|
||||
subscription B is processed by B independently. Scoping the dedup
|
||||
key by subscription prevents one subscription's dedup from
|
||||
masking another's redelivery.
|
||||
|
||||
## Fan-Out Semantics (P4, P7)
|
||||
|
||||
- Fan-out means every subscription receives every published message
|
||||
(subject to filtering — see below). The broker duplicates the
|
||||
message per subscription; each subscription's delivery is
|
||||
independent. The fan-out ratio is the number of subscriptions; the
|
||||
broker's cost scales with fan-out × message size.
|
||||
- Partitioning (P7) applies to topics that are partitioned for
|
||||
throughput: a partitioned topic delivers per-partition order, and
|
||||
each subscription receives from every partition. A subscription
|
||||
that consumes partitions in parallel must handle per-partition
|
||||
ordering and cross-partition non-ordering (P2 — document the
|
||||
property, do not assume global order).
|
||||
- The delivery semantic is per-subscription (P4): subscription A
|
||||
may be at-least-once, subscription B may be at-most-once (for a
|
||||
loss-tolerant analytics feed). The choice is per subscription,
|
||||
declared, not emergent.
|
||||
|
||||
## Shared vs Independent Subscriptions (P2, P3 — the chaos anti-pattern)
|
||||
|
||||
- An **independent subscription** is one durable cursor per
|
||||
consumer group: each subscription receives every message in
|
||||
topic order (per partition, P2) and acks independently. This is
|
||||
the correct default: per-consumer ordering and per-consumer
|
||||
idempotency hold.
|
||||
- A **shared subscription** is one subscription shared by multiple
|
||||
consumers: the broker dispatches each message to an arbitrary
|
||||
consumer in the shared group. This breaks per-consumer ordering
|
||||
(P2 — consumer A sees message 3 before consumer B sees message
|
||||
1) and complicates idempotency (P3 — the dedup state must be
|
||||
shared across consumers, not per-consumer). This is the
|
||||
`messaging-shared-subscription` chaos anti-pattern
|
||||
(pre-specified in P4 ATELIER-110): the primary breach is P2
|
||||
(ordering); P3 (idempotency) is the compounding consequence.
|
||||
- A shared subscription is correct ONLY when the consumers are
|
||||
stateless, the per-message processing is order-independent, and
|
||||
the dedup store is shared (a shared Redis, a shared DB). A shared
|
||||
subscription for order-dependent or per-consumer-stateful
|
||||
processing is the chaos anti-pattern: the broker's arbitrary
|
||||
dispatch breaks the order the consumer assumes.
|
||||
|
||||
```python
|
||||
# The messaging-shared-subscription chaos anti-pattern (P2
|
||||
# ordering breach, P3 idempotency compounding). Two consumers
|
||||
# share one subscription; the broker dispatches each message to
|
||||
# an arbitrary consumer. Per-consumer ordering breaks; dedup
|
||||
# must be shared (and often is not).
|
||||
|
||||
# BAD — shared subscription, per-consumer dedup (chaos):
|
||||
def shared_subscription_bad(broker, send_email):
|
||||
# Both consumers call subscribe with the SAME subscription
|
||||
# name. The broker round-robins; consumer A gets msg 1, msg 3;
|
||||
# consumer B gets msg 2, msg 4. Per-consumer order is broken.
|
||||
# If each consumer has its OWN dedup store, a redelivery to
|
||||
# the OTHER consumer re-processes (P3 breach).
|
||||
sub = broker.subscribe(topic="users", subscription="shared")
|
||||
for message in sub.receive():
|
||||
# Per-consumer dedup — WRONG. A redelivered message may
|
||||
# land on the other consumer, which has not seen it.
|
||||
if local_dedup.seen(message["idempotencyKey"]): # per-consumer
|
||||
sub.ack(message); continue
|
||||
send_email(message["payload"]["email"], "Welcome!")
|
||||
local_dedup.mark(message["idempotencyKey"])
|
||||
sub.ack(message)
|
||||
|
||||
# CORRECT — independent subscriptions (per-consumer ordering,
|
||||
# per-subscription dedup):
|
||||
def independent_subscriptions_good(broker, send_email):
|
||||
sub = broker.subscribe(topic="users", subscription="welcome-email")
|
||||
for message in sub.receive():
|
||||
if dedup_store.seen(("welcome", message["idempotencyKey"])):
|
||||
sub.ack(message); continue
|
||||
send_email(message["payload"]["email"], "Welcome!")
|
||||
dedup_store.mark(("welcome", message["idempotencyKey"]))
|
||||
sub.ack(message)
|
||||
```
|
||||
|
||||
- If a shared subscription is genuinely required (stateless,
|
||||
order-independent, shared dedup), document the choice and the
|
||||
shared-dedup requirement (P2 — the ordering property is "none
|
||||
across consumers"; P3 — the dedup is shared). The default is
|
||||
independent subscriptions; shared is an opt-in for the narrow case.
|
||||
|
||||
## Filtering (P4, C8 Economy)
|
||||
|
||||
- **Subscription filtering** lets a subscription receive only
|
||||
messages matching a filter (e.g., `event.type == "order"`).
|
||||
Filtering at the broker saves bandwidth (C8 — the subscriber
|
||||
does not receive and discard) and reduces subscriber load.
|
||||
- **Server-side filtering** (broker evaluates the filter before
|
||||
delivery) is more efficient than **client-side filtering**
|
||||
(subscriber receives and discards). Server-side filtering is the
|
||||
default where the broker supports it (GCP Pub/Sub, SNS filtering,
|
||||
NATS subject filtering); client-side is the fallback.
|
||||
- A filter that is too broad wastes bandwidth; a filter that is
|
||||
too narrow drops messages the subscriber needed. The filter is
|
||||
a P1 (contract) and P4 (semantic) decision: the subscription's
|
||||
filter is part of its declared contract.
|
||||
|
||||
## Ordering Across Subscriptions (P2)
|
||||
|
||||
- A topic with per-partition ordering delivers per-partition order
|
||||
to each subscription. Across subscriptions, there is no ordering
|
||||
guarantee: subscription A may ack message 3 while subscription B
|
||||
is still on message 1. This is correct and expected — each
|
||||
subscription is independent.
|
||||
- Within a subscription, ordering holds per partition (P2 — the
|
||||
documented property). A subscription that processes partitions
|
||||
in parallel must not assume cross-partition order. A subscription
|
||||
that needs global order must use a single partition (sacrificing
|
||||
parallelism, P7) or an external sequencing mechanism.
|
||||
- The `messaging-shared-subscription` anti-pattern breaks even
|
||||
per-partition order within a subscription: the broker's arbitrary
|
||||
dispatch to consumers in the shared group breaks the per-
|
||||
partition sequence each consumer sees.
|
||||
|
||||
## Pub/Sub vs Stream — The Durability Boundary (cross-link messaging/streams)
|
||||
|
||||
- Pub/sub and streams are both fan-out or one-to-many primitives,
|
||||
but their durability model differs. Pub/sub is a
|
||||
**push-to-subscription** model: each subscription is a cursor,
|
||||
retention is short (the subscription's unacked window), and
|
||||
replay is limited to the unacked messages. A subscription that
|
||||
falls behind beyond the retention window loses messages
|
||||
permanently.
|
||||
- A stream is a **durable-log** model: messages are retained by
|
||||
the log for a configured window (P8 — Replay and Retention are
|
||||
Configured), and any consumer group can replay from any offset
|
||||
within the window. A stream consumer that falls behind can
|
||||
catch up by replaying; a pub/sub subscription that falls behind
|
||||
beyond retention cannot.
|
||||
- The choice is the durability requirement: if the consumer must
|
||||
be able to replay (reprocessing, backfill, new consumer starting
|
||||
from the beginning), use a stream. If the consumer only needs
|
||||
the live feed (and can tolerate loss on a long fall-behind),
|
||||
pub/sub is lighter. See `domains/messaging/streams.md` for the
|
||||
durable-log model, offsets, and consumer groups.
|
||||
|
||||
## Observability — Per-Subscription Lag (P10)
|
||||
|
||||
- Per-subscription lag (messages published minus messages acked
|
||||
for each subscription, or the age of the oldest unacked message
|
||||
per subscription) is the primary pub/sub health signal. Each
|
||||
subscription has its own lag — a fast subscription and a slow
|
||||
subscription on the same topic are independent signals.
|
||||
- A subscription whose lag grows beyond the retention window is a
|
||||
silent data-loss risk: the broker will drop the oldest messages,
|
||||
and the subscription will never see them. Alert on lag relative
|
||||
to retention — lag approaching retention is the loss threshold.
|
||||
- Wire per-subscription lag to `domains/observability/metrics` as
|
||||
an SLI per subscription. A topic with N subscriptions has N lag
|
||||
metrics; a single aggregate hides the slow one. See
|
||||
`domains/observability/metrics` for the generic SLI/SLO
|
||||
discipline.
|
||||
|
||||
## What Violates Pub/Sub Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| Shared subscription for order-dependent processing (broker dispatch breaks per-consumer order) | P2 Ordering is a Property, Not an Assumption |
|
||||
| Shared subscription with per-consumer dedup (redelivery to the other consumer re-processes) | P3 Consumers are Idempotent |
|
||||
| Schemaless topic (no versioned contract; subscribers parse by guess) | P1 Messages are Contracts |
|
||||
| Subscription with no idempotency dedup (redelivered message duplicates the effect) | P3 Consumers are Idempotent |
|
||||
| Subscription whose lag approaches retention (silent data loss) | P10, `domains/observability/metrics` |
|
||||
| Unstated delivery semantic per subscription (at-least-once vs at-most-once guessed) | P4 Delivery Semantics are Explicit |
|
||||
| Filter that is too narrow (drops messages the subscriber needed) | P1, P4 |
|
||||
| Partitioned topic with no documented per-partition ordering contract | P2 Ordering is a Property, Not an Assumption |
|
||||
| No per-subscription lag metric (slow subscription invisible) | P10 Messaging is Observable |
|
||||
| Cross-partition order assumption within a subscription (no global order guarantee) | P2, P7 |
|
||||
@@ -1,318 +0,0 @@
|
||||
# Queues — Derived Rules
|
||||
|
||||
> Derives from `domains/messaging/first-principles.md`. Applies P1
|
||||
> (Messages are Contracts), P3 (Consumers are Idempotent), P4
|
||||
> (Delivery Semantics are Explicit), P5 (Dead-Letter Handling is
|
||||
> Defined), and P6 (Backpressure is Bounded) primarily, with P2
|
||||
> (ordering), P7 (partitioning), and P10 (observable lag). For the
|
||||
> at-least-once / at-most-once / exactly-once decision, see the
|
||||
> comparison table below. Cross-links `domains/concurrency/patterns`
|
||||
> for the in-process bounded-queue analog and
|
||||
> `domains/observability/metrics` for consumer lag.
|
||||
|
||||
## What a Queue Is (P1 Messages are Contracts)
|
||||
|
||||
- A queue is a point-to-point async delivery primitive. A producer
|
||||
enqueues a message; exactly one consumer dequeues and processes
|
||||
it. The message has an explicit, versioned schema (P1): the
|
||||
producer and consumer agree on shape before exchange, and the
|
||||
schema is the boundary — a schemaless message is a defect (the
|
||||
consumer breaks silently on the next shape change).
|
||||
- The boundary is per D-062: messaging owns the cross-process /
|
||||
network-failure-model angle; concurrency owns the in-process
|
||||
analog. A queue is a messaging concern because producer and
|
||||
consumer are separate systems, the broker is the intermediary,
|
||||
and the failure model is network (the message can be lost,
|
||||
duplicated, reordered, or delayed by the broker or the network,
|
||||
not by a thread crash). The in-process bounded buffer
|
||||
(`domains/concurrency/patterns` Pattern 5) is the analog below
|
||||
this boundary — it fails by OOM; a broker-backed queue fails by
|
||||
partition, broker restart, or consumer crash-and-retry.
|
||||
- See `domains/messaging/pubsub.md` for the fan-out (one-to-many)
|
||||
variant and `domains/messaging/streams.md` for the durable-log
|
||||
(replay-from-offset) variant. A queue deletes on ack; a stream
|
||||
retains for replay — the durability boundary is the
|
||||
distinguishing trait.
|
||||
|
||||
## Producer / Consumer Model (P1, P3)
|
||||
|
||||
- The producer enqueues a message with an idempotency key (P3).
|
||||
The consumer dequeues, processes, and acks. If the consumer
|
||||
crashes before acking, the broker redelivers; the idempotency
|
||||
key makes the redelivery safe (the consumer dedups, not the
|
||||
broker).
|
||||
- The idempotency key is per-message, not per-producer or
|
||||
per-session. A consumer that dedups by producer alone will drop
|
||||
distinct messages issued in the same window. Use a UUID per
|
||||
message, or a deterministic key derived from the message content
|
||||
(e.g., `(entity, operation, version)`).
|
||||
|
||||
```python
|
||||
# Producer/consumer pair with idempotency key (P1 contract, P3
|
||||
# idempotency). The producer tags each message with a versioned
|
||||
# schema and a unique idempotency key; the consumer dedups by the
|
||||
# key so a redelivered message is processed once (P3).
|
||||
|
||||
# --- Producer ---
|
||||
import json, uuid
|
||||
|
||||
def enqueue(order, broker):
|
||||
# P1: versioned schema. The message carries its schema version
|
||||
# so the consumer can route by shape (P9 evolution discipline).
|
||||
message = {
|
||||
"schema": "order.created.v1",
|
||||
"id": str(uuid.uuid4()),
|
||||
"idempotencyKey": f"order:{order['id']}:{order['version']}",
|
||||
"payload": order,
|
||||
}
|
||||
broker.send(queue="orders", body=json.dumps(message))
|
||||
# At-least-once by default (P4): the broker acks the send; the
|
||||
# consumer may see this message more than once under retry.
|
||||
|
||||
# --- Consumer ---
|
||||
def consume(broker, dedup_store, process_order):
|
||||
for message in broker.receive(queue="orders"):
|
||||
# P3: idempotent consumer. Dedup by idempotency key before
|
||||
# processing; a redelivered message is a no-op, not a
|
||||
# double-apply.
|
||||
if dedup_store.seen(message["idempotencyKey"]):
|
||||
broker.ack(message) # already processed; skip
|
||||
continue
|
||||
try:
|
||||
process_order(message["payload"])
|
||||
dedup_store.mark(message["idempotencyKey"])
|
||||
broker.ack(message) # success; broker drops it
|
||||
except Exception:
|
||||
broker.nack(message) # redeliver (at-least-once, P4)
|
||||
```
|
||||
|
||||
- The dedup store is bounded (P6 — Backpressure is Bounded): a
|
||||
dedup store that grows without bound is a memory leak. Use a TTL
|
||||
window longer than the broker's max-redelivery window, or a
|
||||
bounded LRU. See `domains/messaging/delivery-semantics.md` for
|
||||
the full idempotency-key dedup-store pattern.
|
||||
|
||||
## Ack / Nack (P4 Delivery Semantics are Explicit)
|
||||
|
||||
- **Ack** tells the broker the message was processed; the broker
|
||||
drops it. **Nack** (negative ack) tells the broker the
|
||||
processing failed; the broker redelivers (at-least-once) or
|
||||
routes to a DLQ (after the retry budget — P5).
|
||||
- A consumer that neither acks nor nacks within the visibility
|
||||
timeout causes the broker to redeliver (the broker assumes the
|
||||
consumer died). This is the at-least-once default: the broker
|
||||
prefers duplication to loss.
|
||||
- The semantic is explicit (P4): at-least-once is the default; the
|
||||
consumer must be idempotent (P3). At-most-once is fire-and-forget
|
||||
(no ack; the broker drops on send) — lossy but lowest latency.
|
||||
Exactly-once is at-least-once plus idempotency, or a
|
||||
transactional two-phase commit — see the comparison table below.
|
||||
|
||||
## Visibility Timeouts and Redelivery (P4, P5)
|
||||
|
||||
- The visibility timeout is the window the broker hides a message
|
||||
after delivery, waiting for the ack. If the consumer does not
|
||||
ack within the window, the broker makes the message visible
|
||||
again and redelivers it (to the same consumer or another). This
|
||||
is the at-least-once mechanism: the broker assumes a
|
||||
no-ack-in-time consumer is dead.
|
||||
- The timeout must be longer than the processing time, or the
|
||||
broker redelivers a message the consumer is still processing —
|
||||
causing duplicate processing (which P3 idempotency makes safe,
|
||||
but which wastes resources). A timeout shorter than processing
|
||||
time is a P6 (backpressure) smell: the consumer is too slow for
|
||||
the configured timeout.
|
||||
- Redelivery has a budget (P5): after N redeliveries or a TTL, the
|
||||
message routes to the DLQ. An unbounded retry budget is the
|
||||
`messaging-unbounded-retry` chaos anti-pattern: the consumer
|
||||
never makes progress past the poison message.
|
||||
|
||||
## FIFO vs Standard Queues (P2 Ordering is a Property, Not an Assumption)
|
||||
|
||||
- A **standard queue** delivers in arrival order per receive-node
|
||||
but offers no global ordering across shards, no per-message-
|
||||
group ordering, and may redeliver out of order under retry. It
|
||||
is the high-throughput default; ordering is *not* guaranteed
|
||||
(P2: the ordering property is "none" — explicitly documented).
|
||||
- A **FIFO queue** delivers strict per-message-group order: all
|
||||
messages with the same group ID are delivered to one consumer
|
||||
in send order. The cost is throughput (FIFO queues cap at lower
|
||||
TPS) and latency (the broker must sequence per group). The
|
||||
ordering property is "per-group strict" — explicitly documented
|
||||
(P2).
|
||||
- The choice is a P2 decision (which ordering guarantee) and a C8
|
||||
decision (throughput cost). A consumer that assumes FIFO on a
|
||||
standard queue is a P2 violation: the broker does not provide
|
||||
the guarantee the consumer assumes. Document the property; do
|
||||
not assume it.
|
||||
|
||||
## Prefetch and Concurrency (P6 Backpressure is Bounded)
|
||||
|
||||
- **Prefetch** (or max-unacked) bounds how many messages the
|
||||
broker delivers to one consumer without an ack. A prefetch of 1
|
||||
is strict stop-and-wait (lowest throughput, tightest backpressure);
|
||||
a prefetch of N allows the consumer to process N in flight
|
||||
(higher throughput, more memory). An unbounded prefetch is a P6
|
||||
violation: the broker floods the consumer's memory.
|
||||
- **Consumer concurrency** is the number of parallel workers
|
||||
processing from the queue. More workers increase throughput up to
|
||||
the downstream's limit; beyond that, the workers stall the
|
||||
downstream (P6 — the backpressure propagates to the
|
||||
downstream, not the broker).
|
||||
- The prefetch × concurrency product is the in-flight cap. Declare
|
||||
it (P6): an undeclared cap is a defect — the consumer either
|
||||
underutilizes the broker (prefetch too low) or OOMs under load
|
||||
(prefetch too high). This is the cross-process analog of
|
||||
`domains/concurrency/patterns` Pattern 5 (bounded queue with
|
||||
backpressure): concurrency owns the in-process analog;
|
||||
messaging owns the broker-backed instance.
|
||||
|
||||
## Long Polling (P6, C8 Economy)
|
||||
|
||||
- Long polling (or `ReceiveMessage` with a wait-time-seconds)
|
||||
holds the receive request open until a message arrives or the
|
||||
wait expires. This reduces empty-receive round trips (C8
|
||||
economy of the constrained link) and reduces latency-to-first-
|
||||
message (the message is delivered when it arrives, not on the
|
||||
next poll cycle).
|
||||
- Long polling is the default for low-throughput queues: short
|
||||
polling burns CPU on empty receives; long polling waits for
|
||||
work. For high-throughput queues, the broker is usually full
|
||||
enough that long polling adds no latency; for low-throughput
|
||||
queues, long polling is the difference between 20ms and 20s
|
||||
latency-to-first-message.
|
||||
|
||||
## Redelivery + DLQ Flow (P5 Dead-Letter Handling is Defined)
|
||||
|
||||
- A poison message (unparseable, repeatedly failing, or exhausting
|
||||
the retry budget) routes to the dead-letter queue. The DLQ is
|
||||
observable (P10 — DLQ depth is an alert) and drainable (an
|
||||
operator can inspect, replay, or discard with audit).
|
||||
- The retry budget is bounded (P6): N redeliveries, or a TTL with
|
||||
exponential backoff. After the budget is exhausted, the message
|
||||
is moved to the DLQ, not retried forever. An unbounded retry is
|
||||
the `messaging-unbounded-retry` chaos anti-pattern (P5 breach).
|
||||
- The DLQ routing rule is a redelivery-count or TTL threshold
|
||||
plus a target queue. See `domains/messaging/delivery-semantics.md`
|
||||
for the dead-letter strategy comparison table.
|
||||
|
||||
```python
|
||||
# Redelivery + DLQ flow (P5 dead-letter handling, P6 bounded
|
||||
# retry budget). The consumer tracks redelivery count; after the
|
||||
# budget, the message routes to the DLQ. The DLQ is observable
|
||||
# (P10 — depth is an alert) and drainable.
|
||||
|
||||
MAX_REDELIVERIES = 5
|
||||
DLQ = "orders-dlq"
|
||||
|
||||
def consume_with_dlq(broker, dedup_store, process_order):
|
||||
for message in broker.receive(queue="orders"):
|
||||
# P3 idempotency: a redelivered, already-processed message
|
||||
# is acked and skipped (not re-processed, not DLQ'd).
|
||||
if dedup_store.seen(message["idempotencyKey"]):
|
||||
broker.ack(message)
|
||||
continue
|
||||
try:
|
||||
process_order(message["payload"])
|
||||
dedup_store.mark(message["idempotencyKey"])
|
||||
broker.ack(message)
|
||||
except Exception as exc:
|
||||
# P5: bounded retry budget. After MAX_REDELIVERIES,
|
||||
# route to DLQ — do NOT retry forever.
|
||||
count = message.get("redeliveryCount", 0) + 1
|
||||
if count >= MAX_REDELIVERIES:
|
||||
broker.send(DLQ, body=json.dumps({
|
||||
"original": message,
|
||||
"reason": str(exc),
|
||||
"deadLetteredAt": now_iso(),
|
||||
"redeliveryCount": count,
|
||||
}))
|
||||
broker.ack(message) # remove from the origin queue
|
||||
# P10: the DLQ depth must alert. A DLQ that grows
|
||||
# with no alert is a silent correctness defect.
|
||||
else:
|
||||
# Nack with backoff: the broker redelivers after a
|
||||
# delay. The backoff caps the retry rate (P6).
|
||||
broker.nack(message, delay=exponential_backoff(count))
|
||||
```
|
||||
|
||||
- The `deadLetteredAt` and `reason` fields make the DLQ entry
|
||||
observable and auditable: an operator inspecting the DLQ sees
|
||||
why each message was dead-lettered and when. See
|
||||
`domains/errors/patterns` for the errors-as-data discipline the
|
||||
DLQ entry follows.
|
||||
|
||||
## Delivery Semantics Comparison (D-069)
|
||||
|
||||
| Semantic | Guarantee | Latency Cost | Implementation Cost | When It Fits |
|
||||
|----------|-----------|-------------|---------------------|--------------|
|
||||
| **At-most-once** | A message is delivered 0 or 1 times; loss is possible, duplication is not | Lowest (no ack; fire-and-forget) | Lowest (no ack, no dedup) | Telemetry where a dropped sample is acceptable; high-throughput metrics; MQTT QoS 0; logs where a lost line is tolerable. Never for billing, orders, or any side-effecting operation. |
|
||||
| **At-least-once** | A message is delivered 1 or more times; duplication is possible, loss is not | Low (one ack round-trip) | Medium (consumer must be idempotent — P3; dedup store required) | The default for side-effecting operations: orders, payments, commands. The consumer dedups via idempotency keys (P3); the broker guarantees delivery. Fits the vast majority of broker-backed queues (SQS standard, RabbitMQ ack, MQTT QoS 1). |
|
||||
| **Exactly-once** | A message is delivered exactly 1 time; no loss, no duplication | Highest (two-phase commit or transactional producer+consumer) | Highest (requires transactions, a transactional producer, and a transactional consumer — or at-least-once plus idempotency, which collapses to at-least-once with dedup) | Rare. Kafka transactions (consume-process-produce in one transaction); MQTT QoS 2 (four-step handshake). In practice, "exactly-once" is usually at-least-once plus idempotency (P3) — the broker does not guarantee it; the consumer enforces it. Jepsen analyses verify broker claims. |
|
||||
|
||||
- The default for side-effecting operations is **at-least-once with
|
||||
idempotent consumers** (P3). At-most-once is for loss-tolerant
|
||||
telemetry. Exactly-once is reserved for the narrow case where
|
||||
the consume-process-produce loop must be transactional (Kafka
|
||||
transactions) — and even then, the consumer should be idempotent
|
||||
as defense-in-depth.
|
||||
- The latency cost column is the C8 tradeoff: at-most-once is
|
||||
cheapest, exactly-once is most expensive. The implementation
|
||||
cost column is the C1/C3 tradeoff: at-most-once is simplest,
|
||||
exactly-once is most complex (and most fragile — a transactional
|
||||
consumer that partially fails is a bug source). The "when it
|
||||
fits" column is the P4 decision: declare the semantic per
|
||||
channel, do not let it emerge.
|
||||
- This is the decision matrix required by D-069 for queues; see
|
||||
`domains/messaging/delivery-semantics.md` for the correctness
|
||||
properties of each semantic and the dead-letter strategy
|
||||
comparison.
|
||||
|
||||
## Observability — Consumer Lag (P10 Messaging is Observable)
|
||||
|
||||
- Consumer lag (messages enqueued minus messages acked, or the
|
||||
age of the oldest unacked message) is the primary queue health
|
||||
signal. A lag that grows without bound is a P6 violation (the
|
||||
consumer is slower than the producer) and a P10 violation if it
|
||||
is not alerted.
|
||||
- DLQ depth is the secondary signal: a DLQ that grows is a P5
|
||||
signal (poison messages are accumulating) and a P10 signal if
|
||||
not alerted. Wire both to `domains/observability/metrics` as
|
||||
SLIs with SLOs (e.g., lag < 1000 messages, DLQ depth < 10).
|
||||
- A queue with no lag metric is operating blind (P10 violation):
|
||||
the operator cannot see the consumer falling behind until the
|
||||
downstream effect surfaces.
|
||||
|
||||
## Cross-Link to Concurrency (P6, cross-link concurrency/patterns)
|
||||
|
||||
- The broker-backed queue is the cross-process analog of the
|
||||
in-process bounded buffer. `domains/concurrency/patterns`
|
||||
Pattern 5 (Bounded Queue with Backpressure) owns the in-process
|
||||
instance ("producer is blocked or signaled" within one program);
|
||||
messaging owns the broker-backed instance above it (the producer
|
||||
is the broker's enqueue, the consumer is the broker's dequeue, the
|
||||
backpressure is the prefetch cap and the lag signal). The
|
||||
failure model differs: in-process fails by OOM; broker-backed
|
||||
fails by network partition, broker restart, or consumer
|
||||
crash-and-retry (D-062).
|
||||
- The cross-link is one-directional outward (messaging →
|
||||
concurrency) per D-026 extended: messaging references concurrency
|
||||
as the in-process foundation; concurrency does not back-link to
|
||||
messaging.
|
||||
|
||||
## What Violates Queue Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| Schemaless message (no versioned contract; consumer parses by guess) | P1 Messages are Contracts |
|
||||
| Non-idempotent consumer under at-least-once delivery (redelivery doubles the effect) | P3 Consumers are Idempotent |
|
||||
| Unstated delivery semantic (at-least-once vs exactly-once guessed) | P4 Delivery Semantics are Explicit |
|
||||
| No DLQ (poison message retried forever or silently dropped) | P5 Dead-Letter Handling is Defined |
|
||||
| Unbounded prefetch (broker floods consumer memory) | P6 Backpressure is Bounded |
|
||||
| Unbounded retry budget (no cap; consumer never progresses past the poison) | P5, P6 |
|
||||
| Consumer that assumes FIFO on a standard queue (ordering not guaranteed) | P2 Ordering is a Property, Not an Assumption |
|
||||
| Visibility timeout shorter than processing time (redeliver while still processing) | P4, P6 |
|
||||
| DLQ with no depth alert (poison messages accumulate invisibly) | P10, `domains/observability/metrics` |
|
||||
| No consumer-lag metric (consumer falls behind invisibly) | P10, `domains/observability/metrics` |
|
||||
| Dedup store that grows without bound (memory leak) | P6 Backpressure is Bounded |
|
||||
| Default prefetch with no rationale (underutilizes or OOMs) | P6, `domains/concurrency/patterns` |
|
||||
@@ -1,329 +0,0 @@
|
||||
# Streams — Derived Rules
|
||||
|
||||
> Derives from `domains/messaging/first-principles.md`. Applies P8
|
||||
> (Replay and Retention are Configured) primarily, with P2
|
||||
> (per-partition ordering), P3 (idempotent consumers), P4 (exactly-
|
||||
> once via transactions), P7 (partitioning), and P10 (observable
|
||||
> consumer-group health). For the Kafka / Kinesis / Pulsar /
|
||||
> NATS JetStream decision, see the stream-platform comparison
|
||||
> table below. For the consumer-group rebalance strategy choice,
|
||||
> see the rebalance enumeration below (IDEATE-41). Cross-links
|
||||
> `domains/messaging/delivery-semantics` for exactly-once via
|
||||
> transactions, `domains/data/schema-design` for stream schema,
|
||||
> and `domains/observability/tracing` for cross-partition traces.
|
||||
|
||||
## What a Stream Is (P8 Replay and Retention are Configured)
|
||||
|
||||
- A stream is a durable-log messaging primitive. Messages are
|
||||
appended to a partitioned, replicated log; consumers read from an
|
||||
offset and advance at their own pace. The log is retained for a
|
||||
configured window (P8) — a stream is a durable log, not a pipe.
|
||||
A consumer that falls behind can catch up by replaying from an
|
||||
earlier offset; a consumer that starts fresh can replay from the
|
||||
beginning (within retention).
|
||||
- The boundary with queues and pub/sub is the durability model. A
|
||||
queue deletes on ack; a pub/sub subscription retains only its
|
||||
unacked window; a stream retains the whole log for the configured
|
||||
retention. This makes a stream replayable (P8 — the
|
||||
reversibility mechanism) and observable as a record (P10 — the
|
||||
log is itself an audit of what happened). See
|
||||
`domains/messaging/pubsub.md` for the pub/sub-vs-stream
|
||||
durability boundary discussion.
|
||||
- The boundary with concurrency is per D-062: messaging owns the
|
||||
cross-process/network-failure-model angle. A stream is a
|
||||
messaging concern because the log spans brokers and consumers
|
||||
across a network, and the failure model is partition, broker
|
||||
restart, or consumer crash-and-retry — not in-process OOM or
|
||||
thread crash.
|
||||
|
||||
## Partitioned Log Model (P2, P7)
|
||||
|
||||
- A stream is partitioned for throughput and parallelism. Each
|
||||
partition is an ordered, append-only log; messages within a
|
||||
partition are strictly ordered (P2 — per-partition strict order
|
||||
is the documented property). Across partitions, there is no
|
||||
ordering guarantee: partition 0 and partition 1 are independent
|
||||
logs.
|
||||
- The partition key (P7 — Partitioning is Intentional) determines
|
||||
which partition a message lands on. A key that hashes evenly
|
||||
spreads load; a key that matches the per-entity ordering need
|
||||
(e.g., `userId` for user events) keeps a user's events on one
|
||||
partition in order; a key that is too coarse (one partition for
|
||||
the whole topic) serializes all traffic.
|
||||
- The partition count is a capacity bound: it caps the parallelism
|
||||
(one consumer per partition per consumer group) and the
|
||||
throughput (each partition has a write-throughput limit). Too
|
||||
few partitions cap parallelism; too many partition overhead the
|
||||
broker (file handles, replication, rebalance cost). The choice
|
||||
is documented (P7), not defaulted.
|
||||
|
||||
## Offsets (P2, P8)
|
||||
|
||||
- An offset is a consumer's position in a partition. The consumer
|
||||
reads from its last committed offset; acking (committing the
|
||||
offset) advances it. A consumer that crashes before committing
|
||||
re-reads from the last committed offset (at-least-once by
|
||||
default, P4) — the consumer must be idempotent (P3).
|
||||
- The offset is per-partition (P2): each partition has its own
|
||||
position, and the consumer commits them independently (or
|
||||
atomically across partitions in a transaction — see below).
|
||||
- Replay (P8) is resetting the offset backward: a consumer can
|
||||
replay from the beginning of retention, from a timestamp, or
|
||||
from a specific offset. This is the durable-log property that
|
||||
distinguishes a stream from a queue.
|
||||
|
||||
## Consumer Groups (P3, P7, P10)
|
||||
|
||||
- A consumer group is a set of consumers sharing the stream's
|
||||
partitions: each partition is assigned to exactly one consumer
|
||||
in the group. The group is the unit of parallelism and the unit
|
||||
of offset tracking. Within a group, each consumer handles its
|
||||
assigned partitions; across groups, each group independently
|
||||
reads the whole stream (the pub/sub fan-out property, per
|
||||
subscription/group).
|
||||
- A consumer in the group is idempotent (P3): under at-least-once
|
||||
(the default), a redelivery after a crash-and-retry re-processes
|
||||
messages. The consumer dedups by idempotency key, or processes
|
||||
deterministically (e.g., a stateful aggregation that overwrites
|
||||
with the latest value).
|
||||
- The consumer group's health is observable (P10): per-partition
|
||||
lag (offset of the consumer vs the log's head), the group's
|
||||
consumption rate, and rebalance events are first-class signals.
|
||||
A group whose lag grows without bound is a P6 (backpressure)
|
||||
smell and a P10 (observability) violation if not alerted.
|
||||
|
||||
```python
|
||||
# Consumer-group reading from offsets (P2 per-partition order,
|
||||
# P3 idempotent under at-least-once, P7 partition assignment,
|
||||
# P8 replay from offset). Each consumer in the group handles its
|
||||
# assigned partitions; the group commits offsets atomically or
|
||||
# per-partition.
|
||||
|
||||
def consume_stream(stream, group, dedup_store, process_event):
|
||||
# Assign partitions to this consumer by the group's
|
||||
# rebalance strategy (see the enumeration below).
|
||||
for partition in stream.assigned_partitions(group, consumer=ME):
|
||||
# Read from the last committed offset (P8 — replay by
|
||||
# resetting this offset).
|
||||
offset = stream.committed_offset(group, partition)
|
||||
for message in stream.read(partition, from_offset=offset):
|
||||
# P3: idempotent under at-least-once. A redelivery
|
||||
# after a crash-and-retry re-processes; dedup by key.
|
||||
if dedup_store.seen(message["idempotencyKey"]):
|
||||
stream.commit(group, partition, message["offset"])
|
||||
continue
|
||||
process_event(message["payload"])
|
||||
dedup_store.mark(message["idempotencyKey"])
|
||||
# Commit the offset to advance (P8 — the position is
|
||||
# the replay pointer).
|
||||
stream.commit(group, partition, message["offset"])
|
||||
```
|
||||
|
||||
- The commit-after-process order gives at-least-once (a crash
|
||||
before commit re-reads); the commit-before-process order gives
|
||||
at-most-once (a crash after commit loses the unprocessed
|
||||
message). The default is at-least-once with idempotent consumers
|
||||
(P3, P4).
|
||||
|
||||
## Consumer-Group Rebalance Strategies (IDEATE-41, ATELIER-100 refinement)
|
||||
|
||||
When a consumer joins or leaves the group, the broker must
|
||||
reassign partitions. The rebalance strategy determines the cost
|
||||
and the use-case fit. This enumeration parallels the v0.3
|
||||
IDEATE-30 drift-type enumeration (each strategy with its
|
||||
stop-the-world cost and use-case fit).
|
||||
|
||||
| Strategy | Mechanism | Partition Stop-the-World Cost | Use-Case Fit |
|
||||
|----------|-----------|-------------------------------|--------------|
|
||||
| **Eager rebalance** (stop-the-world) | Every consumer in the group revokes ALL its partitions, the broker reassigns the full partition set, then consumers resume. Every rebalance pauses the whole group. | High — every partition pauses for every rebalance; the whole group stops processing during the revocation+reassignment window. Throughput drops to zero during rebalance. | Simple brokers, small groups, or rarely-rebalancing groups where the simplicity of full revocation outweighs the pause cost. Kafka's legacy protocol (pre-2.4). Avoid for large groups or frequent scale events. |
|
||||
| **Sticky (incremental cooperative) rebalance** | The broker reassigns only the partitions that must move (the joining/leaving consumer's share); existing partitions stay assigned. The rebalance is incremental and cooperative — no full revocation. | Low — only the moving partitions pause; the rest of the group continues processing. The pause is proportional to the changed partition count, not the total. | The default for large groups, frequent scale events, and rolling deploys. Kafka's CooperativeStickyAssignor (2.4+), Pulsar, NATS JetStream. Prefer for any group where a full stop-the-world on every deploy is unacceptable. |
|
||||
| **Cooperative (no-revoke) rebalance** | A subset of sticky where no partition is revoked unless the consumer leaves; only additions are incremental. The strictest minimization of stop-the-world. | Lowest — only added partitions pause; existing assignments are untouched. | Groups where partition assignment is append-only (consumers join but rarely leave). Useful for long-lived consumers with incremental scaling. |
|
||||
|
||||
- The default for any non-trivial group is **sticky/cooperative**:
|
||||
a rolling deploy that triggers an eager rebalance pauses the
|
||||
whole group on every pod restart, which is unacceptable at
|
||||
scale. The eager strategy is a legacy default that survives
|
||||
because it is simple; prefer sticky where the broker supports
|
||||
it.
|
||||
- The "partition stop-the-world cost" column is the P6
|
||||
(backpressure) angle: a full stop-the-world during rebalance
|
||||
causes lag to spike (the consumer is paused, the producer is
|
||||
not). Sticky rebalance bounds the spike to the moving partitions.
|
||||
- A rebalance that pauses without a lag alert is a P10 violation:
|
||||
the operator cannot see the rebalance-induced lag. Wire rebalance
|
||||
events to `domains/observability/metrics` as an event signal.
|
||||
|
||||
## Replay and Retention Windows (P8, C5 Reversibility)
|
||||
|
||||
- Retention is the configured window the log keeps messages: time-
|
||||
based (e.g., 7 days), size-based (e.g., 10 GB per partition), or
|
||||
compacted (keep the latest value per key — a changelog). A
|
||||
stream with no retention is a pipe, not a log (P8 violation); a
|
||||
stream with infinite retention grows until disk exhaustion (P6
|
||||
violation — backpressure on the broker).
|
||||
- Replay (P8) is resetting a consumer's offset to re-read from
|
||||
within the retention window. Use cases: reprocessing after a
|
||||
consumer bug (replay from the timestamp of the buggy deploy),
|
||||
backfilling a new consumer (replay from the beginning), or
|
||||
reindexing (replay to rebuild a derived store).
|
||||
- Compacted topics (Kafka log-compaction, Pulsar compaction) keep
|
||||
the latest value per key and discard older values for the same
|
||||
key. This turns the log into a changelog — a durable
|
||||
materialized view that replays to the current state. Compaction
|
||||
is a P8 (retention) and C5 (reversibility) mechanism: the log
|
||||
retains the current state per key and is replayable to it.
|
||||
|
||||
## Stream Processing (P2, P3, P4)
|
||||
|
||||
- Stream processing is computing over the stream as it arrives:
|
||||
windowing (tumbling, sliding, session windows), joins (stream-
|
||||
stream, stream-table), aggregations (count, sum, per-key
|
||||
windows), and stateful transformations. The processing is
|
||||
per-partition ordered (P2 — a window over a partition is
|
||||
deterministic; a window across partitions is not unless the
|
||||
window is global).
|
||||
- Stream processing consumers are idempotent (P3): a redelivery
|
||||
after a crash re-processes a window; the aggregation must
|
||||
tolerate re-application (e.g., a sum is idempotent under replay
|
||||
if the window is keyed by offset range, not by wall time).
|
||||
- Exactly-once stream processing (P4) requires transactions: the
|
||||
consume-process-produce loop is one transaction — the input
|
||||
offset commit and the output produce are atomic. See the
|
||||
transactional exactly-once producer below.
|
||||
|
||||
## Exactly-Once via Transactions (P4, P3)
|
||||
|
||||
- Exactly-once stream processing is at-least-once plus a
|
||||
transaction: the consumer commits the input offset and produces
|
||||
the output in one transactional operation. If the consumer
|
||||
crashes mid-transaction, neither the offset commit nor the
|
||||
output produce happens — the consumer re-reads from the last
|
||||
committed offset and re-processes (at-least-once), but the
|
||||
transaction ensures the output is produced exactly once.
|
||||
- This is NOT a broker guarantee of exactly-once delivery; it is
|
||||
at-least-once delivery plus idempotent/transactional processing
|
||||
(P3, P4). Jepsen analyses of Kafka transactions confirm the
|
||||
boundaries: the transaction is atomic within the broker, but
|
||||
the downstream sink must be transactional or idempotent too.
|
||||
- See `domains/messaging/delivery-semantics.md` for the full
|
||||
exactly-once-via-idempotency discussion.
|
||||
|
||||
```python
|
||||
# Transactional exactly-once producer (P4 exactly-once via
|
||||
# transactions, P3 idempotent produce). The consume-process-
|
||||
# produce loop is one transaction: the input offset commit and
|
||||
# the output produce are atomic. A crash mid-transaction rolls
|
||||
# both back; the consumer re-reads and re-processes.
|
||||
|
||||
def consume_transform_produce(stream, group, txn_producer):
|
||||
# Begin a transaction. All produces and the offset commit in
|
||||
# this block are atomic (P4).
|
||||
with txn_producer.transaction() as txn:
|
||||
for partition in stream.assigned_partitions(group, ME):
|
||||
offset = stream.committed_offset(group, partition)
|
||||
for message in stream.read(partition, from_offset=offset):
|
||||
output = transform(message["payload"])
|
||||
# P3: idempotent produce. The txn producer
|
||||
# dedups by an epoch+sequence so a retried
|
||||
# transaction does not double-produce.
|
||||
txn.produce(
|
||||
topic="enriched-events",
|
||||
key=message["key"],
|
||||
value=output,
|
||||
idempotencyKey=message["idempotencyKey"],
|
||||
)
|
||||
# Commit the input offset within the same
|
||||
# transaction (P4 atomicity). A crash before
|
||||
# txn.commit() rolls this back; the consumer
|
||||
# re-reads from the prior offset.
|
||||
txn.commit_offset(group, partition, message["offset"])
|
||||
# txn.commit() makes the produces and the offset commit
|
||||
# visible atomically. A crash before this point aborts
|
||||
# both; a crash after is safe (idempotent produce — P3).
|
||||
```
|
||||
|
||||
- The transactional producer's idempotency (P3) is the defense
|
||||
against a retried transaction: the broker dedups the output by
|
||||
the producer's epoch and sequence so a re-commit does not
|
||||
double-produce. The transaction (P4) is the defense against a
|
||||
partial failure: the offset and the output commit together.
|
||||
|
||||
## Stream Schema (P1, P9, cross-link data/schema-design)
|
||||
|
||||
- A stream's messages carry a versioned schema (P1). The schema
|
||||
evolves compatibly (P9): a new field the old consumer ignores is
|
||||
backward-compatible; a renamed field the old consumer parses as
|
||||
`undefined` is a P1 violation compounded as an evolution defect.
|
||||
- Stream schemas are often registered in a schema registry
|
||||
(Confluent, Apicurio) that enforces compatibility on produce.
|
||||
A producer that tries to publish an incompatible schema is
|
||||
rejected; the registry is the P1/P9 enforcement point.
|
||||
- See `domains/data/schema-design` for the generic schema-design
|
||||
discipline (Avro, Protobuf, JSON Schema); messaging owns the
|
||||
stream-specific instance — the registry, the per-topic
|
||||
compatibility mode, the consumer-side routing by schema
|
||||
version.
|
||||
|
||||
## Stream-Platform Comparison (D-069)
|
||||
|
||||
| Axis | Apache Kafka | AWS Kinesis | Apache Pulsar | NATS JetStream |
|
||||
|------|--------------|-------------|---------------|----------------|
|
||||
| **Ordering** | Per-partition strict (P2); global only via single-partition topic | Per-shard strict; global only via single shard | Per-partition strict; global via single partition; also supports shared (out-of-order) subscriptions | Per-stream strict; per-subject ordering; global via single stream |
|
||||
| **Partitioning model** | Partitions (immutable count post-creation; increase requires recreate); key→partition by hash | Shards (reshardable: split/merge at runtime); key→shard by hash | Partitions (resizable; Pulsar's layered architecture separates compute from storage); key→partition by hash | Streams (subject-based; republish to resize); key→stream by subject |
|
||||
| **Replay / retention** | Time- or size-based retention; compaction (latest-per-key); replay from offset or timestamp | Time-based retention (24h–365d); replay from sequence number or timestamp; no compaction | Time- or size-based; compaction; replay from offset or timestamp; tiered storage (hot/warm/cold) | Time- or size-based; per-stream max-age; replay from sequence; no native compaction |
|
||||
| **Consumer groups** | Group-coordinated; offsets stored in an internal topic; eager (legacy) or sticky/cooperative (2.4+) rebalance | Enhanced fan-out consumers (per-shard HTTP/2 push); KCL for group coordination; no native group rebalance (shard is the unit) | Group-coordinated; shared or failover subscription modes; cooperative rebalance | Per-stream consumers; durable cursors; no native group rebalance (stream is the unit) |
|
||||
| **Exactly-once** | Transactions (KIP-98): consume-process-produce atomic; idempotent producer (KIP-516) | No native exactly-once; at-least-once with consumer-side dedup (P3) | Transactions: produce-ack atomic; idempotent producer | At-least-once by default; dedup window per stream (P3 idempotency) |
|
||||
| **Use-case fit** | High-throughput durable logs, stream processing (Kafka Streams, Flink), event sourcing, multi-consumer replay | AWS-native streaming, log ingestion, simple ETL within AWS; low operational burden | Cloud-native, geo-replication, tiered storage, mixed pub/sub + streaming; multi-tenant | Lightweight, low-latency, edge-friendly; NATS ecosystem; simpler ops than Kafka |
|
||||
| **Watch out for** | Partition count is fixed at creation (resize requires recreate + republish); rebalance cost on large groups; operational complexity | Shard limits per account; no compaction; retention cap at 365 days; AWS lock-in | Two-arch (BookKeeper + Brokers) operational complexity; smaller ecosystem | Smaller ecosystem; no native compaction; fewer stream-processing libraries |
|
||||
|
||||
- The default for high-throughput durable logs with multi-consumer
|
||||
replay is **Kafka**; for AWS-native streaming, **Kinesis**; for
|
||||
geo-replicated multi-tenant or mixed pub/sub + streaming,
|
||||
**Pulsar**; for lightweight low-latency edge-friendly streaming,
|
||||
**NATS JetStream** (which also cross-links `domains/edge/iot`
|
||||
via MQTT parallels — see the edge↔messaging bidirectionality in
|
||||
`domains/messaging/first-principles.md` §4).
|
||||
- The ordering column is the P2 check: every platform provides
|
||||
per-partition/per-shard strict order; none provides global order
|
||||
across partitions except by single-partition. The replay/
|
||||
retention column is the P8 check: every platform retains for a
|
||||
configured window; replay is from offset or timestamp. The
|
||||
exactly-once column is the P4 check: Kafka and Pulsar provide
|
||||
transactions; Kinesis and JetStream rely on at-least-once plus
|
||||
consumer-side idempotency (P3).
|
||||
|
||||
## Cross-Partition Traces (P10, cross-link observability/tracing)
|
||||
|
||||
- A stream-processing pipeline that fans out across partitions
|
||||
must propagate a trace context per event: the trace ID follows
|
||||
the event from source to processed output, even as the event
|
||||
crosses partition boundaries. Without cross-partion traces, a
|
||||
downstream error cannot be traced back to its source event.
|
||||
- See `domains/observability/tracing` for the generic distributed-
|
||||
tracing discipline (trace context propagation, span
|
||||
correlation). Messaging owns the stream-specific instance: the
|
||||
trace context is a message header, the span boundary is the
|
||||
consume-process-produce edge, and the cross-partition
|
||||
correlation is by trace ID (not by partition — partitions are
|
||||
independent logs, P2).
|
||||
- A stream processor with no trace propagation is a P10
|
||||
violation: the operator cannot trace a processed event back to
|
||||
its source. Wire the trace context into every produce and every
|
||||
consume.
|
||||
|
||||
## What Violates Stream Discipline
|
||||
|
||||
| Violation | Principle |
|
||||
|-----------|-----------|
|
||||
| Stream with no retention (pipe, not log; no replay) | P8 Replay and Retention are Configured |
|
||||
| Infinite retention (grows until disk exhaustion) | P8, P6 |
|
||||
| Consumer group with eager rebalance at scale (full stop-the-world per deploy) | P6, IDEATE-41 (use sticky/cooperative) |
|
||||
| Non-idempotent stream consumer under at-least-once (redelivery re-processes the window) | P3 Consumers are Idempotent |
|
||||
| Default partition key (no rationale; hotspot or wrong-order) | P7 Partitioning is Intentional |
|
||||
| Partition count too low (caps parallelism) or too high (overhead) | P7 |
|
||||
| Cross-partition order assumption (no global order guarantee) | P2 Ordering is a Property, Not an Assumption |
|
||||
| Exactly-once claimed without transactional consume-process-produce (P4 violation) | P4 Delivery Semantics are Explicit |
|
||||
| Stream schema with no registry / no compatibility enforcement (silent shape break) | P1, P9, `domains/data/schema-design` |
|
||||
| No per-partition lag metric (consumer falls behind invisibly) | P10, `domains/observability/metrics` |
|
||||
| No cross-partition trace propagation (downstream error untraceable) | P10, `domains/observability/tracing` |
|
||||
| Compacted topic treated as a full log (old values already discarded) | P8, C1 (compaction is a retention mode, not a full log) |
|
||||
@@ -1,222 +0,0 @@
|
||||
# Bad Example: Compliance Audit Log (Two Breaches)
|
||||
|
||||
> An audit logging implementation that violates **two** Atelier
|
||||
> compliance principles in one example (per IDEATE-26, D-044):
|
||||
> **P1** (Audit Logs are Append-Only) — a mutable audit log with
|
||||
> routine `DELETE`/`UPDATE` "cleanup" — and **P9** (Secrets and
|
||||
> Sensitive Data are Redacted in Audit) — a database password leaked
|
||||
> into an audit record. Each violation is cited, then fixed.
|
||||
|
||||
## The Code
|
||||
|
||||
```python
|
||||
# audit_log.py — the audit sink, stored in a mutable Postgres table
|
||||
import psycopg2, datetime
|
||||
|
||||
# P1 VIOLATION: the audit log is a regular mutable table. There is no
|
||||
# write-once protection, no immutable bucket, no hash-chaining.
|
||||
# Any DB user with UPDATE/DELETE can rewrite history.
|
||||
CREATE_TABLE = """
|
||||
CREATE TABLE audit_log (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
timestamp TIMESTAMPTZ NOT NULL,
|
||||
event TEXT NOT NULL,
|
||||
actor TEXT NOT NULL,
|
||||
target TEXT,
|
||||
payload JSONB,
|
||||
request_id TEXT
|
||||
);
|
||||
-- No row-level immutability. No trigger preventing UPDATE/DELETE.
|
||||
"""
|
||||
|
||||
def write_event(event, actor, target=None, payload=None, request_id=None):
|
||||
conn = psycopg2.connect(os.environ["DATABASE_URL"])
|
||||
conn.execute(
|
||||
"INSERT INTO audit_log (timestamp, event, actor, target, payload, request_id) "
|
||||
"VALUES (%s, %s, %s, %s, %s, %s)",
|
||||
(datetime.datetime.utcnow(), event, actor, target,
|
||||
json.dumps(payload), request_id),
|
||||
)
|
||||
|
||||
# P1 VIOLATION (continued): "cleanup" that mutates the audit log.
|
||||
# A routine job deletes records older than 30 days to "save space"
|
||||
# and updates records to "fix typos in the actor field."
|
||||
def cleanup_audit_log():
|
||||
conn = psycopg2.connect(os.environ["DATABASE_URL"])
|
||||
# DELETE — an audit record is destroyed. This is tampering,
|
||||
# dressed as housekeeping.
|
||||
conn.execute("DELETE FROM audit_log WHERE timestamp < NOW() - INTERVAL '30 days'")
|
||||
# UPDATE — an audit record is rewritten. The "fix" is the
|
||||
# violation; the original actor is lost.
|
||||
conn.execute("UPDATE audit_log SET actor = 'admin' WHERE actor LIKE 'svc-%'")
|
||||
```
|
||||
|
||||
```python
|
||||
# The call site that leaks a secret into the audit log.
|
||||
def read_config(key):
|
||||
# ... fetches a secret from the secrets manager ...
|
||||
value = secrets_manager.get(key) # e.g. the raw DB password
|
||||
# P9 VIOLATION: the raw secret value is written into the audit
|
||||
# payload. The append-only log is now a secret store.
|
||||
write_event(
|
||||
event="config.read",
|
||||
actor="api-server",
|
||||
target={"kind": "secret", "id": key},
|
||||
payload={"value": value}, # <- the secret, in plaintext
|
||||
request_id=req.id,
|
||||
)
|
||||
return value
|
||||
```
|
||||
|
||||
The resulting audit record:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": 48213,
|
||||
"timestamp": "2026-08-05T09:12:03Z",
|
||||
"event": "config.read",
|
||||
"actor": "api-server",
|
||||
"target": {"kind": "secret", "id": "db-password"},
|
||||
"payload": {"value": "p@ssw0rd-sup3r-s3cr3t-plaintext"},
|
||||
"request_id": "req_91c2"
|
||||
}
|
||||
```
|
||||
|
||||
A week later, the `cleanup_audit_log` job `DELETE`s this record (it
|
||||
is older than 30 days in the team's "retention" — which is actually a
|
||||
storage-economy decision, not a policy), and `UPDATE`s every
|
||||
`svc-*` actor to `admin`. The secret was in the log for a week,
|
||||
readable by anyone with `SELECT` on the table; now the record of it
|
||||
having been there is gone.
|
||||
|
||||
## What Makes It Bad
|
||||
|
||||
### Breach 1 — Mutable Audit Log (Compliance P1 Audit Logs are Append-Only)
|
||||
- The audit log is a regular mutable Postgres table. `DELETE FROM
|
||||
audit_log WHERE timestamp < ...` and `UPDATE audit_log SET actor =
|
||||
...` both succeed. The log is a draft, not a record.
|
||||
- Routine `DELETE` as "cleanup" is the cardinal P1 violation: the
|
||||
deletion of an audit record is itself an auditable incident, not a
|
||||
housekeeping task. "We deleted old records to save space" is a P3
|
||||
(Retention is Policy, Not Storage) violation *and* a P1 violation —
|
||||
the retention decision is driven by storage cost, and the
|
||||
mechanism is tampering.
|
||||
- The `UPDATE` that rewrites `svc-deploy` → `admin` destroys
|
||||
attribution (a P7 violation stacked on the P1 violation): the
|
||||
original actor is lost, and the replacement (`admin`) is a shared
|
||||
identity that could be any of ten engineers.
|
||||
- **Fix:** the audit sink is append-only *by construction*, not by
|
||||
policy. Write-once storage (WORM bucket, immutable log stream,
|
||||
hash-chained ledger) enforces immutability at the substrate.
|
||||
Retention is a declared policy with a meta-audit of deletions; a
|
||||
human does not run ad-hoc `DELETE` jobs.
|
||||
|
||||
```python
|
||||
# Fix: write to an append-only sink (illustrative — S3 Object Lock,
|
||||
# WORM bucket, or a hash-chained ledger). The API has no update /
|
||||
# delete path; the storage refuses mutation.
|
||||
def write_event(event, actor, target=None, payload=None, request_id=None):
|
||||
record = {
|
||||
"timestamp": datetime.datetime.now(datetime.timezone.utc).isoformat(),
|
||||
"event": event,
|
||||
"actor": actor, # the authenticated principal, not "admin"
|
||||
"target": target,
|
||||
"payload": redact(payload), # see Breach 2 fix
|
||||
"request_id": request_id,
|
||||
"prev_hash": last_hash(), # hash-chaining: tampering breaks the chain
|
||||
}
|
||||
record["hash"] = sha256(canonical_json(record))
|
||||
append_only_sink.write(record) # WORM storage; no update/delete API exists
|
||||
|
||||
# Fix: retention is a declared, reviewed policy — not an ad-hoc DELETE.
|
||||
# When an audit segment ages out, the deletion is itself meta-audited
|
||||
# in a higher-tier log with the rule that authorized it.
|
||||
# (See domains/compliance/data-retention.md and audit-logs.md.)
|
||||
```
|
||||
- See `domains/compliance/audit-logs.md` (Audit Logs are Append-Only)
|
||||
and `domains/compliance/first-principles.md` P1.
|
||||
|
||||
### Breach 2 — Secret Leaked in Audit Log (Compliance P9 Secrets and Sensitive Data are Redacted in Audit)
|
||||
- `payload={"value": value}` writes the raw DB password into the
|
||||
audit record. The append-only log is now a secret store: anyone
|
||||
with `SELECT` on `audit_log` can read production credentials. The
|
||||
log is harder to secure than the secrets manager it read from.
|
||||
- Once the secret is in an append-only log, the remediation is
|
||||
expensive — rotate the secret *and* rewrite the log's access scope
|
||||
(you cannot edit the record; it is append-only). Redaction must
|
||||
happen *at the logging boundary, before the record is written*, not
|
||||
by opportunistic scrubbing after the fact.
|
||||
- The redaction policy here is "nothing" — there is no rule for
|
||||
which fields are redacted, by what mechanism, in which event type.
|
||||
A redaction rule that lives in no one's head and no code is a P9
|
||||
violation waiting to happen (and it happened).
|
||||
- **Fix:** redaction is structural, applied at the logging boundary
|
||||
before the record reaches the append-only sink. The policy is
|
||||
itself auditable (which fields, by what rule, in which event).
|
||||
|
||||
```python
|
||||
# Fix: redaction at the boundary. Log the FACT of the action
|
||||
# (a secret was read), never the CONTENT of the secret.
|
||||
REDACTED_FIELDS = {"value", "token", "password", "authorization", "secret"}
|
||||
|
||||
def redact(payload):
|
||||
if not isinstance(payload, dict):
|
||||
return "[REDACTED:non-object]"
|
||||
out = {}
|
||||
for k, v in payload.items():
|
||||
if k.lower() in REDACTED_FIELDS or "secret" in k.lower():
|
||||
out[k] = "[REDACTED:secret]"
|
||||
else:
|
||||
out[k] = v
|
||||
out["_redaction"] = "secret-value-policy/v1" # the rule is auditable
|
||||
return out
|
||||
|
||||
# The fixed audit record:
|
||||
# {
|
||||
# "event": "config.read",
|
||||
# "actor": "api-server", # the authenticated principal
|
||||
# "target": {"kind": "secret", "id": "db-password"},
|
||||
# "payload": {"value": "[REDACTED:secret]"},
|
||||
# "_redaction": "secret-value-policy/v1",
|
||||
# "request_id": "req_91c2"
|
||||
# }
|
||||
# The fact of the read is logged; the secret never enters the log.
|
||||
```
|
||||
- See `domains/compliance/audit-logs.md` (Redaction at the Boundary)
|
||||
and `domains/compliance/first-principles.md` P9. Cross
|
||||
`domains/security/secrets.md` — the audit-side redaction is the
|
||||
complement of secret management.
|
||||
|
||||
## The Cascade (Two Breaches Compound)
|
||||
|
||||
The two violations compound destructively. The secret enters the
|
||||
mutable log (P9 breach), where it sits readable by any `SELECT`-holder
|
||||
for a week. Then the `cleanup` job `DELETE`s the record (P1 breach) —
|
||||
destroying the evidence that the secret was ever logged, while the
|
||||
secret itself has already been exposed to every reader of the table.
|
||||
The `UPDATE` that rewrites `svc-deploy` → `admin` (a P7 attribution
|
||||
breach stacked on the P1 breach) means that even if a copy of the
|
||||
record survived, the actor who triggered the secret read is no longer
|
||||
identifiable. The team cannot answer "who read the DB password and
|
||||
when" — the log that would answer it was mutated, and the secret it
|
||||
leaked is now in the wild. This is the worst-case interaction of P1
|
||||
and P9: a secret leak with no attributable actor and no surviving
|
||||
record.
|
||||
|
||||
## Cross-Domain Links
|
||||
|
||||
- `domains/compliance/audit-logs.md` — the append-only guarantee and
|
||||
the redaction-at-boundary rule this code violates.
|
||||
- `domains/compliance/first-principles.md` — P1 (Append-Only) and P9
|
||||
(Redacted) are the two breached principles; P7 (Attributable) is
|
||||
breached by the `UPDATE` rewrite.
|
||||
- `domains/compliance/evidence.md` — an audit log that can be
|
||||
`DELETE`d is not admissible evidence; the append-only guarantee is
|
||||
what makes it admissible.
|
||||
- `domains/compliance/data-retention.md` — retention is a declared
|
||||
policy with meta-audited deletions, not an ad-hoc `DELETE` job.
|
||||
- `domains/security/secrets.md` — redaction at the logging boundary
|
||||
is the audit-side complement of secret management.
|
||||
- `domains/observability/logging.md` — audit logs are structured
|
||||
logging with an append-only guarantee; the logging primitives
|
||||
compose here.
|
||||
@@ -1,158 +0,0 @@
|
||||
# Bad Example: Edge Sync Loop (Livelock)
|
||||
|
||||
> A two-node sync that oscillates divergent state forever. This is
|
||||
> the pre-specified `edge-sync-loop` chaos anti-pattern (ATELIER-110).
|
||||
> **Single-breach per D-068:** the principle breached is **Edge P4
|
||||
> (Sync Conflicts are Bounded, Not Infinite)**. The prose explains
|
||||
> why this is a livelock, not eventual consistency.
|
||||
|
||||
## The Code
|
||||
|
||||
```typescript
|
||||
// Two edge nodes (a tablet and a dispatcher workstation) sync a
|
||||
// work-order status. The merge uses wall-clock timestamps with no
|
||||
// vector clock and no deterministic tiebreak. A clock skew flips
|
||||
// the winner on every merge pass; the state oscillates forever.
|
||||
|
||||
interface WorkOrderState {
|
||||
orderId: string;
|
||||
status: string;
|
||||
wallClock: number; // P4 VIOLATION: wall time, not monotonic
|
||||
nodeId: string;
|
||||
}
|
||||
|
||||
// P4 VIOLATION: the merge picks the later wall-clock write as the
|
||||
// winner. Wall time skews across nodes; a skew of even a few
|
||||
// milliseconds flips the winner. With no vector clock, concurrent
|
||||
// writes are not detected; with no deterministic tiebreak, equal
|
||||
// timestamps are resolved by whichever node's clock is ahead.
|
||||
function mergeBad(local: WorkOrderState, remote: WorkOrderState): WorkOrderState {
|
||||
// No vector clock. No monotonic logical clock. No deterministic
|
||||
// tiebreak by nodeId. This is wall-clock-only LWW on durable
|
||||
// state — the row in the CRDT-vs-LWW decision matrix that is a
|
||||
// P4 violation for anything but ephemeral state.
|
||||
if (local.wallClock >= remote.wallClock) return local;
|
||||
return remote;
|
||||
}
|
||||
|
||||
// The sync loop: each node merges the other's state, writes the
|
||||
// result, and the next sync pass flips it back. The loop never
|
||||
// terminates.
|
||||
async function syncLoop(node: "tablet" | "dispatcher", peer: WorkOrderState) {
|
||||
let local = store.get(peer.orderId);
|
||||
while (true) {
|
||||
const merged = mergeBad(local, peer);
|
||||
store.set(merged.orderId, merged);
|
||||
await pushToPeer(merged); // peer receives, merges, pushes back
|
||||
peer = await pullFromPeer(); // peer's clock is now ahead — flips winner
|
||||
local = store.get(peer.orderId); // local re-merges; flips again
|
||||
// The loop runs forever. Convergence is never reached. This is
|
||||
// a livelock: the system is making progress (each pass writes)
|
||||
// but the state never converges.
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The tablet sets `status: "completed"` at wall time `10:00:00.500`.
|
||||
The dispatcher's clock is 50ms ahead; it sets `status: "reassigned"`
|
||||
at wall time `10:00:00.550`. The first merge: dispatcher wins
|
||||
(`10:00:00.550 > 10:00:00.500`). The tablet receives `reassigned`,
|
||||
but its clock drifts ahead by 100ms during the next sync pass; it
|
||||
writes `completed` at `10:00:00.650`. The second merge: tablet
|
||||
wins. The dispatcher's clock drifts ahead again; it writes
|
||||
`reassigned` at `10:00:00.750`. The third merge: dispatcher wins.
|
||||
The state flips between `completed` and `reassigned` on every sync
|
||||
pass. The loop runs forever.
|
||||
|
||||
## Why It Violates
|
||||
|
||||
### Sync Conflicts are Bounded, Not Infinite (Edge P4, C1, C5)
|
||||
|
||||
- **The breach:** the merge has no convergence bound. The CRDT-vs-
|
||||
LWW decision matrix in `domains/edge/sync.md` is explicit:
|
||||
wall-clock-only LWW on durable state carries an **unbounded
|
||||
failure mode** — clock skew = oscillation — and is a P4
|
||||
violation for anything but ephemeral state. This code is the
|
||||
matrix's failure column made real.
|
||||
- **Why it is a livelock, not eventual consistency:** eventual
|
||||
consistency guarantees that, in the absence of new writes, all
|
||||
replicas eventually converge. This system never converges even
|
||||
with no new writes: the clock skew alone drives the oscillation.
|
||||
Each sync pass writes (so the system is "busy"), but the state
|
||||
never settles — the definition of a livelock. The P4 contract is
|
||||
that the merge terminates and converges; this merge does neither.
|
||||
- The missing pieces, per the decision matrix:
|
||||
- **No monotonic logical clock** (a hybrid logical clock or
|
||||
vector clock) — wall time skews, and the skew is unbounded.
|
||||
- **No vector clock** — concurrent writes are not detected, so
|
||||
the conflict is invisible; the merge silently flips instead of
|
||||
surfacing.
|
||||
- **No deterministic tiebreak** (e.g., `nodeId`) on equal
|
||||
timestamps — equal wall times are resolved by whichever node's
|
||||
clock is ahead, which is not a stable property.
|
||||
- The P4 bound requires that the merge function be associative,
|
||||
commutative, and idempotent, terminating in one pass. A
|
||||
wall-clock merge with clock skew satisfies none of these: it is
|
||||
not associative (order of merges flips the winner), not
|
||||
idempotent (a re-merge after a clock drift flips the result),
|
||||
and not terminating (the loop runs forever).
|
||||
|
||||
## The Fix
|
||||
|
||||
```typescript
|
||||
// Fix: replace wall-clock LWW with a CRDT register (state-based
|
||||
// CvRDT) using a monotonic logical clock and a deterministic
|
||||
// tiebreak by nodeId. The merge is now a total order — no
|
||||
// oscillation (P4 bounded). See the CRDT-vs-LWW decision matrix:
|
||||
// for a register on an unreliable network, a state-based CRDT is
|
||||
// the correct row.
|
||||
|
||||
interface LWWRegister<T> {
|
||||
value: T;
|
||||
timestamp: number; // monotonic logical clock (HLC), not wall time
|
||||
nodeId: string; // deterministic tiebreak
|
||||
}
|
||||
|
||||
function mergeLWWRegister<T>(
|
||||
local: LWWRegister<T>,
|
||||
remote: LWWRegister<T>,
|
||||
): LWWRegister<T> {
|
||||
// Associative, commutative, idempotent (P4 bound, P5 idempotent).
|
||||
// (timestamp, nodeId) is a total order — convergence in one pass.
|
||||
if (local.timestamp > remote.timestamp) return local;
|
||||
if (local.timestamp < remote.timestamp) return remote;
|
||||
return local.nodeId > remote.nodeId ? local : remote;
|
||||
}
|
||||
|
||||
// The sync is now one pass: merge, write, done. No loop, no
|
||||
// oscillation. A re-merge of the same two replicas yields the same
|
||||
// result (P5 idempotent), so a retried sync is safe.
|
||||
async function syncOnce(local: LWWRegister<unknown>, remote: LWWRegister<unknown>) {
|
||||
const converged = mergeLWWRegister(local, remote);
|
||||
store.set(converged);
|
||||
await pushToPeer(converged);
|
||||
// Done. No while(true). The merge terminates.
|
||||
}
|
||||
```
|
||||
|
||||
The fix selects the CRDT row of the decision matrix (a register on
|
||||
an unreliable network): a monotonic logical clock (HLC) eliminates
|
||||
clock skew; a deterministic tiebreak by `nodeId` eliminates the
|
||||
equal-timestamp flip. The merge is now a total order that converges
|
||||
in one pass — the P4 bound. See `examples/good/edge-offline-sync.md`
|
||||
for the full good-example version of this pattern.
|
||||
|
||||
## Cross-Domain Links
|
||||
|
||||
- `domains/edge/sync.md` — the CRDT-vs-LWW decision matrix; the
|
||||
wall-clock-only LWW row is the failure mode this example
|
||||
instantiates; the CRDT row is the fix.
|
||||
- `domains/edge/first-principles.md` — P4 (Sync Conflicts are
|
||||
Bounded, Not Infinite) is the principle breached.
|
||||
- `review/anti-patterns.md` — the `edge-sync-loop` chaos
|
||||
anti-pattern (edge P4, C1, C5 — infinite oscillation is a
|
||||
correctness failure, not eventual consistency; the sync is a
|
||||
livelock).
|
||||
- `examples/good/edge-offline-sync.md` — the good-example version
|
||||
of this pattern: a CRDT register merge that converges in one
|
||||
pass.
|
||||
@@ -1,175 +0,0 @@
|
||||
# Bad Example: i18n String Concatenation
|
||||
|
||||
> A checkout component that violates Atelier's i18n principles. Each
|
||||
> violation is cited, then fixed.
|
||||
|
||||
## The Code
|
||||
|
||||
```typescript
|
||||
// Checkout.tsx — the cardinal i18n violation
|
||||
function CartSummary({ itemCount, name, total, currency, date }) {
|
||||
// P3 VIOLATION: inline string concatenation. The source-language
|
||||
// word order ("Welcome, {name}! You have {n} items") is baked into
|
||||
// code. Every locale with different word order is broken.
|
||||
const welcome = "Welcome, " + name + "!";
|
||||
|
||||
// P4 VIOLATION: hand-rolled plural branching. `if (n === 1)` encodes
|
||||
// exactly English's one/other rule. Arabic (six categories), Russian
|
||||
// (three), Polish (three) are all wrong.
|
||||
const items =
|
||||
itemCount === 1 ? "1 item" : itemCount + " items";
|
||||
|
||||
// P5 VIOLATION: hand-rolled currency + date formatting. "$" + total
|
||||
// is wrong for de-DE (symbol, grouping, placement). The date
|
||||
// `getMonth() + 1 + "/" + getDay()` is US-only (mm/dd/yyyy).
|
||||
const price = "$" + total.toFixed(2);
|
||||
const d = new Date(date);
|
||||
const dateStr = (d.getMonth() + 1) + "/" + d.getDate() + "/" + d.getFullYear();
|
||||
|
||||
return (
|
||||
<div>
|
||||
<h1>{welcome}</h1>
|
||||
<p>{items} · {price} · {dateStr}</p>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
```typescript
|
||||
// The "resource" file — a custom CSV the team hand-rolled.
|
||||
// locale,en_us
|
||||
// welcome_prefix,Welcome,
|
||||
// item_singular,item
|
||||
// item_plural,items
|
||||
//
|
||||
// This is a P3 violation on its own: a custom format no standard
|
||||
// tool (xgettext, i18next, FormatJS) can extract from or compose with.
|
||||
```
|
||||
|
||||
The team runs their first i18n test against real Arabic translations —
|
||||
after the string freeze, after the translator was paid. The Arabic
|
||||
build renders `"Welcome, محمد!"` with the name on the wrong side of
|
||||
the comma, `"1 items"` for a single item (Arabic has six plural
|
||||
categories, not two), and the price as `"$1,234.56"` (Arabic-Egypt
|
||||
formats as `"١٬٢٣٤٫٥٦ ج.م."`). Every screen is a rewrite, not a patch.
|
||||
|
||||
## What Makes It Bad
|
||||
|
||||
### Inline String Concatenation (i18n P3 Resources are External, Not Inline)
|
||||
- `"Welcome, " + name + "!"` bakes English word order into code. In
|
||||
Japanese the name comes first (`ようこそ、محمدさん!`); in Arabic the
|
||||
structure differs again. The concatenation is invisible to the
|
||||
extraction pipeline (`xgettext`, `i18next-parser`) — the translator
|
||||
never sees it as a unit, and the string cannot be versioned or
|
||||
rolled back as a whole.
|
||||
- The custom `.csv` "resource" store is a second P3 violation: no
|
||||
standard tool reads it, it carries no plural grammar, and it cannot
|
||||
compose with the ICU formatting layer.
|
||||
- **Fix:** strings live in a standard locale resource file, addressed
|
||||
by key. Code calls `t("welcome", { name })`; the resource carries
|
||||
the parameterized message.
|
||||
|
||||
```json
|
||||
// en-US.json (ICU MessageFormat)
|
||||
{
|
||||
"checkout.welcome": "Welcome, {name}!",
|
||||
"checkout.cart.summary": "{count, plural, one {# item} other {# items}} · {price} · {date}"
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
// ar-EG.json — six plural categories per CLDR; the code is identical
|
||||
{
|
||||
"checkout.welcome": "أهلاً بك، {name}!",
|
||||
"checkout.cart.summary": "{count, plural, zero {لا عناصر} one {عنصر واحد} two {عنصران} few {# عناصر} many {# عنصرًا} other {# عنصر}} · {price} · {date}"
|
||||
}
|
||||
```
|
||||
- See `domains/i18n/locale-resources.md` (Resources are the Boundary)
|
||||
and `domains/i18n/first-principles.md` P3.
|
||||
|
||||
### Hand-Rolled Plural Branching (i18n P4 Plural and Gender are Parameterized)
|
||||
- `itemCount === 1 ? "1 item" : itemCount + " items"` encodes
|
||||
English's one/other rule and nothing else. Arabic has six
|
||||
categories (zero, one, two, few, many, other); Russian has three
|
||||
(one, few, many); Polish has three with different boundaries. A
|
||||
two-branch `if` is a C1 (Correctness) violation masquerading as a
|
||||
shortcut — it returns a wrong answer for every non-English locale.
|
||||
- **Fix:** the count goes to ICU MessageFormat; the formatter
|
||||
consults `Intl.PluralRules` for the active locale; the resource
|
||||
carries the variant for that category. The code passes the count,
|
||||
nothing more.
|
||||
|
||||
```typescript
|
||||
// The code passes the count; the resource + formatter pick the form.
|
||||
t("checkout.cart.summary", { count: itemCount, price, date });
|
||||
// Intl.PluralRules("ar-EG").select(1) === "one" -> "عنصر واحد"
|
||||
// Intl.PluralRules("ar-EG").select(2) === "two" -> "عنصران"
|
||||
// Intl.PluralRules("ar-EG").select(5) === "few" -> "٥ عناصر"
|
||||
```
|
||||
- See `domains/i18n/locale-resources.md` (Plural and Gender in
|
||||
Resources) and `domains/i18n/formatting.md` (Plural Rules).
|
||||
|
||||
### Hand-Rolled Currency and Date Formatting (i18n P5 Formatting is Locale-Aware)
|
||||
- `"$" + total.toFixed(2)` hardcodes the US dollar symbol, US
|
||||
grouping (`,`), and US placement (symbol before the number). In
|
||||
`de-DE` the euro formats as `"1.234,56 €"` (symbol after, dot
|
||||
grouping). In `ar-EG` the pound formats as `"١٬٢٣٤٫٥٦ ج.م."`
|
||||
(Arabic-Indic digits, different grouping).
|
||||
- `(d.getMonth() + 1) + "/" + d.getDate() + "/" + d.getFullYear()`
|
||||
produces `11/7/2024` — US `mm/dd/yyyy`. Most of the world reads
|
||||
`dd/mm/yyyy`; ISO is `yyyy-mm-dd`. A hand-rolled date formatter
|
||||
encodes one locale's convention and silently produces wrong output
|
||||
for every other.
|
||||
- **Fix:** `Intl.NumberFormat` and `Intl.DateTimeFormat` with a BCP
|
||||
47 locale tag. CLDR is the source of truth; `Intl` is the runtime.
|
||||
|
||||
```typescript
|
||||
new Intl.NumberFormat("ar-EG", { style: "currency", currency: "EGP" })
|
||||
.format(1234.56); // "١٬٢٣٤٫٥٦ ج.م."
|
||||
new Intl.DateTimeFormat("ar-EG", { dateStyle: "medium" })
|
||||
.format(new Date(date)); // "٧ نوفمبر ٢٠٢٤"
|
||||
```
|
||||
- See `domains/i18n/formatting.md` (the Intl surface, dates, numbers,
|
||||
currencies) and `domains/i18n/first-principles.md` P5.
|
||||
|
||||
### Source Language Treated as the Default (i18n P1 Source Language is a Locale)
|
||||
- The component has no resource layer at all for the source locale —
|
||||
English is "just the strings in the code." When the first second
|
||||
locale arrives, the fix is a rewrite (extract every string,
|
||||
restructure every concatenation), not a patch. The source language
|
||||
is `en-US`, a locale among many — it is not `null`.
|
||||
- **Fix:** extract source strings into `en-US.json` from day one,
|
||||
even before a second locale exists. The resource layer is the
|
||||
boundary from the first commit.
|
||||
- See `domains/i18n/first-principles.md` P1 and
|
||||
`domains/uiux/copywriting.md`.
|
||||
|
||||
## The Cascade
|
||||
|
||||
The violations compound. Inline concatenation makes strings invisible
|
||||
to the extraction pipeline, so the translator never receives them as
|
||||
units — they reconstruct them by reading the code. Hand-rolled
|
||||
plurals return wrong answers for every non-English locale, so the
|
||||
Arabic build ships `"1 items"` for a single item. Hand-rolled
|
||||
formatting produces US-shaped output everywhere, so the price and
|
||||
date are wrong for `de-DE`, `ar-EG`, `zh-Hans-CN`, and every other
|
||||
locale. And because the first i18n test ran against real translations
|
||||
(a P8 violation — pseudo-locales should have surfaced all of this
|
||||
while the fix was still cheap), the defects are found after the
|
||||
string freeze, after the translator was paid, and after the release
|
||||
date was promised. The fix is now a re-translation and a re-release,
|
||||
not a commit.
|
||||
|
||||
## Cross-Domain Links
|
||||
|
||||
- `domains/i18n/locale-resources.md` — the resource layer this code
|
||||
lacks; the standard formats (`.po`, JSON, Fluent, ICU Resource
|
||||
Bundle) it should have used.
|
||||
- `domains/i18n/formatting.md` — the `Intl`/ICU/CLDR formatting this
|
||||
code should call instead of hand-rolling.
|
||||
- `domains/i18n/first-principles.md` — P3, P4, P5, and P8 (pseudo-
|
||||
locales test early).
|
||||
- `domains/uiux/copywriting.md` — copy lives in resources, not in
|
||||
code.
|
||||
- `domains/api/error-responses.md` — the same parameterized-message
|
||||
discipline applies to localized API errors.
|
||||
@@ -1,188 +0,0 @@
|
||||
# Bad Example: Messaging Shared Subscription
|
||||
|
||||
> Two consumers share one subscription; the broker dispatches each
|
||||
> message to an arbitrary consumer. Per-consumer ordering breaks;
|
||||
> per-consumer dedup is wrong. This is the pre-specified
|
||||
> `messaging-shared-subscription` chaos anti-pattern (ATELIER-110).
|
||||
> **Single-breach per D-068:** the primary principle breached is
|
||||
> **Messaging P2 (Ordering is a Property, Not an Assumption)**. P3
|
||||
> (Consumers are Idempotent) is noted as the compounding
|
||||
> consequence — the example remains single-breach in its named
|
||||
> violation.
|
||||
|
||||
## The Code
|
||||
|
||||
```python
|
||||
# Two email workers share ONE subscription on the "users" topic.
|
||||
# The broker round-robins: worker A gets msg 1, msg 3; worker B
|
||||
# gets msg 2, msg 4. Per-consumer order is broken (P2 breach).
|
||||
# Each worker has its OWN dedup store — a redelivery to the other
|
||||
# worker re-processes (P3 compounding).
|
||||
|
||||
import json
|
||||
|
||||
def shared_subscription_bad(broker, send_email, worker_id: str):
|
||||
# P2 VIOLATION: both workers call subscribe with the SAME
|
||||
# subscription name. The broker dispatches each message to an
|
||||
# arbitrary worker in the shared group. Worker A sees msg 3
|
||||
# before worker B sees msg 1; per-worker order is broken.
|
||||
sub = broker.subscribe(topic="users", subscription="welcome-shared")
|
||||
|
||||
# P3 compounding: per-worker dedup. A redelivered message may
|
||||
# land on the OTHER worker, which has not seen it, so the
|
||||
# per-worker dedup store does not catch it — the message is
|
||||
# processed twice across the two workers.
|
||||
local_dedup = DedupStore(backend=redis_for(worker_id))
|
||||
|
||||
for message in sub.receive():
|
||||
payload = json.loads(message["body"])
|
||||
if local_dedup.seen(payload["idempotencyKey"]):
|
||||
sub.ack(message); continue
|
||||
send_email(payload["email"], "Welcome!")
|
||||
local_dedup.mark(payload["idempotencyKey"])
|
||||
sub.ack(message)
|
||||
```
|
||||
|
||||
```python
|
||||
# The two workers are launched with the same subscription name.
|
||||
# Worker A and Worker B both call subscribe("users", "welcome-shared").
|
||||
# The broker sees one shared subscription; it round-robins.
|
||||
|
||||
def start_workers(broker):
|
||||
# Worker A
|
||||
spawn(shared_subscription_bad, broker, send_email, worker_id="A")
|
||||
# Worker B — same subscription name, same topic
|
||||
spawn(shared_subscription_bad, broker, send_email, worker_id="B")
|
||||
# The broker dispatches: A gets msg 1, B gets msg 2, A gets msg 3,
|
||||
# B gets msg 4. If worker A is slow, B may get msg 2 and msg 4
|
||||
# before A acks msg 1. Per-worker order is broken (P2).
|
||||
```
|
||||
|
||||
The scenario: a user signs up, then immediately updates their
|
||||
email preference (two messages in topic order: `user.signed-up.v1`,
|
||||
`user.preference-updated.v1`). The broker dispatches `signed-up` to
|
||||
worker A and `preference-updated` to worker B. Worker B sends the
|
||||
preference email before worker A sends the welcome email — the
|
||||
user sees the preference confirmation before the welcome. Then the
|
||||
broker redelivers `signed-up` (worker A's ack was slow); this time
|
||||
it dispatches to worker B. Worker B's local dedup store has never
|
||||
seen `signed-up` (it was processed by A), so B sends the welcome
|
||||
email again. The user receives two welcome emails. Per-consumer
|
||||
ordering broke (P2); per-consumer dedup did not catch the
|
||||
cross-worker redelivery (P3 compounding).
|
||||
|
||||
## Why It Violates
|
||||
|
||||
### Ordering is a Property, Not an Assumption (Messaging P2, C1, C2)
|
||||
|
||||
- **The breach (primary):** the two workers share one subscription,
|
||||
and the broker dispatches each message to an arbitrary worker in
|
||||
the shared group. Per-consumer ordering is broken: worker A sees
|
||||
`preference-updated` (msg 2) before it sees `signed-up` (msg 1)
|
||||
if the broker round-robins them to different workers. The
|
||||
workers' code assumes topic order, but the shared subscription
|
||||
provides no per-consumer order guarantee — the broker's dispatch
|
||||
is arbitrary. P2 requires that the ordering property be explicit
|
||||
and documented; here it is assumed (topic order) but not provided
|
||||
(arbitrary dispatch). The assumption is wrong.
|
||||
- The P2 contract is that "it's FIFO" is a claim backed by the
|
||||
broker's partitioning contract, not an assumption the consumer
|
||||
makes. A shared subscription's contract is "no per-consumer
|
||||
order"; the workers' code assumes the opposite. See
|
||||
`domains/messaging/pubsub.md` (Shared vs Independent
|
||||
Subscriptions) and `domains/messaging/first-principles.md` P2.
|
||||
|
||||
### Consumers are Idempotent — compounding consequence (Messaging P3, C1)
|
||||
|
||||
- **The compounding consequence (not the named breach):** each
|
||||
worker has its OWN dedup store. A redelivered message that lands
|
||||
on the *other* worker is not in that worker's dedup store, so it
|
||||
is processed again. The dedup is per-worker, but the subscription
|
||||
is shared — the dedup must be shared across workers to be
|
||||
correct under a shared subscription. The per-worker dedup store
|
||||
is wrong for a shared subscription; a shared dedup store (a
|
||||
shared Redis, a shared DB) is required.
|
||||
- Per D-068, the example remains single-breach in its named
|
||||
violation: P2 is the primary breach (the shared subscription
|
||||
breaks ordering); P3 is the compounding consequence (the
|
||||
per-worker dedup is wrong *because* the subscription is shared).
|
||||
If the subscription were independent, per-worker dedup would be
|
||||
correct. The shared subscription is the root cause; P3 is the
|
||||
downstream effect.
|
||||
|
||||
## The Fix
|
||||
|
||||
```python
|
||||
# Fix 1 (default): independent subscriptions. Each consumer gets
|
||||
# its own durable cursor; per-consumer order holds (P2); per-
|
||||
# consumer dedup is correct (P3). This is the default per
|
||||
# domains/messaging/pubsub.md.
|
||||
|
||||
def independent_subscriptions_good(broker, send_email, worker_id: str):
|
||||
# Each worker has its OWN subscription name. The broker
|
||||
# delivers every message to every subscription in topic order.
|
||||
sub = broker.subscribe(
|
||||
topic="users",
|
||||
subscription=f"welcome-{worker_id}", # per-consumer
|
||||
)
|
||||
# Per-consumer dedup is now correct: a redelivery to THIS
|
||||
# worker is caught by THIS worker's dedup store.
|
||||
dedup = DedupStore(backend=redis_for(worker_id))
|
||||
for message in sub.receive():
|
||||
payload = json.loads(message["body"])
|
||||
if dedup.seen(payload["idempotencyKey"]):
|
||||
sub.ack(message); continue
|
||||
send_email(payload["email"], "Welcome!")
|
||||
dedup.mark(payload["idempotencyKey"])
|
||||
sub.ack(message)
|
||||
```
|
||||
|
||||
```python
|
||||
# Fix 2 (if a shared subscription is genuinely required): the
|
||||
# consumers must be stateless, the processing order-independent,
|
||||
# AND the dedup store must be SHARED across workers. Document the
|
||||
# ordering property as "none across consumers" (P2 — the property
|
||||
# is explicit, not assumed) and use a shared dedup backend (P3).
|
||||
|
||||
def shared_subscription_stateless(broker, send_email):
|
||||
# P2: document the ordering property. A shared subscription
|
||||
# provides NO per-consumer order; processing must be order-
|
||||
# independent. Do not assume topic order.
|
||||
sub = broker.subscribe(topic="users", subscription="welcome-shared")
|
||||
# P3: SHARED dedup. A redelivery to any worker is caught by the
|
||||
# shared store.
|
||||
shared_dedup = DedupStore(backend=shared_redis)
|
||||
for message in sub.receive():
|
||||
payload = json.loads(message["body"])
|
||||
if shared_dedup.seen(payload["idempotencyKey"]):
|
||||
sub.ack(message); continue
|
||||
send_email(payload["email"], "Welcome!") # order-independent
|
||||
shared_dedup.mark(payload["idempotencyKey"])
|
||||
sub.ack(message)
|
||||
```
|
||||
|
||||
The default is Fix 1 (independent subscriptions): per-consumer
|
||||
ordering holds, per-consumer dedup is correct, and the code is
|
||||
simpler. Fix 2 is the narrow opt-in for genuinely stateless,
|
||||
order-independent processing — and even then, the dedup must be
|
||||
shared. See `domains/messaging/pubsub.md` for the full
|
||||
shared-vs-independent discussion.
|
||||
|
||||
## Cross-Domain Links
|
||||
|
||||
- `domains/messaging/pubsub.md` — the shared-vs-independent
|
||||
subscriptions section; the `messaging-shared-subscription`
|
||||
anti-pattern lives here (pre-specified in P4 ATELIER-110).
|
||||
- `domains/messaging/first-principles.md` — P2 (Ordering is a
|
||||
Property, Not an Assumption) is the primary breach; P3
|
||||
(Consumers are Idempotent) is the compounding consequence.
|
||||
- `domains/messaging/delivery-semantics.md` — the idempotency-key
|
||||
dedup store; the per-subscription dedup key scoping
|
||||
(`(subscription, idempotencyKey)`) that prevents one
|
||||
subscription's dedup from masking another's redelivery.
|
||||
- `review/anti-patterns.md` — the `messaging-shared-subscription`
|
||||
chaos anti-pattern (messaging P2, P3, C1 — shared subscription
|
||||
breaks ordering and dedup).
|
||||
- `examples/good/messaging-idempotent-consumer.md` — the good-
|
||||
example version of the consumer pattern: an independent consumer
|
||||
with a dedup store and a DLQ routing rule.
|
||||
@@ -1,226 +0,0 @@
|
||||
# Good Example: AI/ML Reproducible Training Run
|
||||
|
||||
> A training run that follows Atelier's AI/ML principles. Each aspect
|
||||
> cites the principle it satisfies. Scope per D-023: this is
|
||||
> engineering discipline (reproducibility, versioning, lineage,
|
||||
> serving), **not** algorithm or model design — no architecture
|
||||
> choice, hyperparameter tuning, or model-family comparison appears
|
||||
> here.
|
||||
|
||||
## The Run
|
||||
|
||||
A training run `2026-08-05T09:12:00Z#run-42` produces model
|
||||
`registry/payments-fraud@sha256:b5e1...aa0`. Every input that shaped
|
||||
the model is pinned, named, and recoverable; the eval was declared
|
||||
before training; the model is an addressed artifact in a registry;
|
||||
the rollback path names the prior model and the prior dataset.
|
||||
|
||||
### The Reproducibility Contract
|
||||
|
||||
```yaml
|
||||
# lineage/run-42.yaml — the lineage root, committed alongside the code
|
||||
run_id: 2026-08-05T09:12:00Z#run-42
|
||||
dataset: s3://ml-data/train@sha256:7f3a...e21
|
||||
splits: dvc.yaml@commit a1b2c4d
|
||||
code: git@a1b2c4d
|
||||
config: configs/train.yaml@commit a1b2c4d
|
||||
environment: ghcr.io/org/train-img@sha256:9c2d...f88
|
||||
eval_spec: configs/eval.yaml@commit a1b2c4d
|
||||
model_digest: registry/payments-fraud@sha256:b5e1...aa0
|
||||
status: passed # eval gate passed -> eligible for promotion
|
||||
```
|
||||
|
||||
- Lose any line and the run is anecdote, not evidence. The record is
|
||||
the lineage root: a prediction cites the `model_digest`, which
|
||||
cites this `run_id`, which cites everything above.
|
||||
|
||||
### Data is Versioned (DVC, content-hashed)
|
||||
|
||||
```ini
|
||||
# dvc.yaml — the split config is versioned in git, the data in the
|
||||
# content-addressed object store. Both are pinned by commit + hash.
|
||||
stages:
|
||||
prepare:
|
||||
cmd: python src/prepare.py --input data/raw --out data/splits
|
||||
deps:
|
||||
- data/raw
|
||||
- src/prepare.py
|
||||
outs:
|
||||
- data/splits/train.parquet
|
||||
- data/splits/val.parquet
|
||||
- data/splits/test.parquet
|
||||
# The dataset hash (sha256:7f3a...e21) is recorded in the lineage
|
||||
# contract above. "s3://ml-data/latest" would be a P2 violation.
|
||||
```
|
||||
|
||||
```bash
|
||||
# The dataset is pinned by content hash, not by a mutable path.
|
||||
$ dvc get s3://ml-data/train --rev sha256:7f3a...e21
|
||||
# The split is a deterministic function of (dataset version, split
|
||||
# config, random seed). Two runs on the same pinned inputs produce
|
||||
# the same splits.
|
||||
```
|
||||
|
||||
### Code and Config are Versioned (git)
|
||||
|
||||
```yaml
|
||||
# configs/train.yaml@commit a1b2c4d — versioned with the code
|
||||
# (No algorithm/hyperparameter content is illustrated here — this is
|
||||
# the engineering discipline of pinning the config, not the model
|
||||
# design inside it. Per D-023, algorithm choice is out of scope.)
|
||||
seed: 42
|
||||
splits:
|
||||
train: data/splits/train.parquet
|
||||
val: data/splits/val.parquet
|
||||
test: data/splits/test.parquet # held out, never touched by training
|
||||
```
|
||||
|
||||
### Environment is Pinned (container digest)
|
||||
|
||||
```dockerfile
|
||||
# The training environment is an image addressed by digest, not :latest.
|
||||
# ghcr.io/org/train-img@sha256:9c2d...f88
|
||||
FROM python:3.11-slim
|
||||
# dependencies pinned in requirements.txt with hashes
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
```
|
||||
|
||||
```text
|
||||
# requirements.txt — pinned + hash-pinned (pip-compile / pip-audit)
|
||||
dvc==3.50.2 \
|
||||
--hash=sha256:1c8a...e7
|
||||
mlflow==2.16.0 \
|
||||
--hash=sha256:9b2f...a1
|
||||
# No unpinned ranges. A rerun pulls the exact same wheels.
|
||||
```
|
||||
|
||||
### Evaluation is Defined Before Training (P4)
|
||||
|
||||
```yaml
|
||||
# configs/eval.yaml@commit a1b2c4d — committed BEFORE training runs.
|
||||
# The metrics, splits, and pass/fail thresholds are a-priori; they
|
||||
# are the contract the model must satisfy to leave the experiment.
|
||||
metrics:
|
||||
- name: precision_at_threshold
|
||||
threshold: ">= 0.92"
|
||||
- name: recall_at_threshold
|
||||
threshold: ">= 0.85"
|
||||
- name: false_positive_rate
|
||||
threshold: "<= 0.03"
|
||||
split: data/splits/test.parquet # held out, never in training
|
||||
gate: all_metrics_pass # AND of all thresholds; no cherry-pick
|
||||
# The eval schema equals the serving input contract (serving.md P8):
|
||||
# feature names, types, ranges match the production boundary exactly.
|
||||
```
|
||||
|
||||
- Metrics chosen after seeing scores would be a P4 violation: the eval
|
||||
would be rationalizing, not measuring. See
|
||||
`domains/ai-ml/model-evaluation.md`.
|
||||
|
||||
### The Model is a Versioned Artifact (MLflow registry)
|
||||
|
||||
```bash
|
||||
# After the eval gate passes, the model is registered as an immutable
|
||||
# artifact addressed by digest, then promoted by stage.
|
||||
$ mlflow models register \
|
||||
--name payments-fraud \
|
||||
--model-uri runs:/run-42/model \
|
||||
--description "run-42, dataset sha256:7f3a...e21, eval passed"
|
||||
# registry/payments-fraud@sha256:b5e1...aa0
|
||||
# Stages: None -> Staging -> Production. Promotion is a registry
|
||||
# operation, not a file copy. Never "latest".
|
||||
```
|
||||
|
||||
### The Pipeline Composes (P9)
|
||||
|
||||
```text
|
||||
# The training flow is a pipeline with explicit stages and contracts,
|
||||
# not a notebook. Each stage has named inputs and named outputs.
|
||||
prepare(dataset@hash) -> split(dvc.yaml) -> train(config, env@digest)
|
||||
-> eval(eval.yaml, test@hash) -> [gate: pass] -> register(model@digest)
|
||||
|
|
||||
+-> [gate: fail] -> abort, no promote
|
||||
# A notebook in this path would be a P9 violation: implicit state,
|
||||
# human-dependent order, unreproducible.
|
||||
```
|
||||
|
||||
## What Makes It Good
|
||||
|
||||
### Reproducibility is First Class (AI/ML P1, C1, C5)
|
||||
- data + code + config + environment are all pinned. A second
|
||||
engineer on a second laptop checks out commit `a1b2c4d`, pulls the
|
||||
dataset by hash, pulls the image by digest, and reproduces the run
|
||||
bit-for-bit. The run is reviewable because it is recreatable.
|
||||
- See `domains/ai-ml/first-principles.md` P1 and
|
||||
`domains/devops/first-principles.md` P1 Reproducibility.
|
||||
|
||||
### Data is Versioned, Not Just Code (AI/ML P2, C5, C7)
|
||||
- The dataset is `s3://ml-data/train@sha256:7f3a...e21`, not
|
||||
`s3://ml-data/latest`. A model trained on "the data" is a model
|
||||
trained on an unknown input — a C1 violation. DVC pins the data the
|
||||
way git pins the code.
|
||||
- See `domains/ai-ml/data-versioning.md` (dataset hashing, the DVC /
|
||||
Delta Lake / LakeFS comparison) and `domains/data/migrations.md`.
|
||||
|
||||
### Lineage is Traceable End-to-End (AI/ML P3, C7, C1)
|
||||
- prediction → model → run-42 → dataset → source. Every edge is
|
||||
named; no orphan model. A serving regression traces back to the
|
||||
exact dataset and code that built the model, which is how drift is
|
||||
diagnosed (data drift vs concept drift vs prediction drift).
|
||||
- See `domains/ai-ml/data-versioning.md` (lineage record) and
|
||||
`domains/observability/logging.md`.
|
||||
|
||||
### Evaluation Defined Before Training (AI/ML P4, C1, C2)
|
||||
- `eval.yaml` was committed before `train` ran. The gate is
|
||||
`all_metrics_pass`; a failing metric aborts promotion. Cherry-
|
||||
picking a metric post-hoc is a correctness violation — the eval
|
||||
would no longer measure the model.
|
||||
- See `domains/ai-ml/model-evaluation.md` (eval-as-a-gate) and
|
||||
`domains/testing/first-principles.md` (tests as specification).
|
||||
|
||||
### Models are Versioned Artifacts (AI/ML P5, C5, C6)
|
||||
- The model is `registry/payments-fraud@sha256:b5e1...aa0`, promoted
|
||||
Staging → Production. A serving endpoint that pulled `latest` would
|
||||
be serving an unknown model with no rollback. The registry is to
|
||||
models what a container registry is to images.
|
||||
- See `domains/ai-ml/serving.md` (the model is an addressed artifact)
|
||||
and `domains/devops/first-principles.md` P7 Immutability.
|
||||
|
||||
### Rollback Includes the Model (AI/ML P10, C5)
|
||||
- If production regresses, the rollback restores the prior model
|
||||
digest `registry/payments-fraud@sha256:a1c4...f09` AND the prior
|
||||
serving code. A rollback that redeploys old code but keeps the new
|
||||
model has not rolled back — the model was the thing that regressed.
|
||||
- See `domains/ai-ml/serving.md` (Rollback Includes the Model) and
|
||||
`domains/devops/first-principles.md` P4 Rollback First.
|
||||
|
||||
## What This Example Does NOT Do (And Why That's Good)
|
||||
|
||||
- Does **not** reference the dataset by a mutable path —
|
||||
`s3://ml-data/latest` would be a P2 violation.
|
||||
- Does **not** choose metrics after seeing scores — that is a P4
|
||||
violation (rationalizing, not measuring).
|
||||
- Does **not** pull `latest` from the model registry — that is a P5
|
||||
violation (unknown model, no rollback).
|
||||
- Does **not** contain algorithm/architecture/hyperparameter content
|
||||
— per D-023, those are research choices, not engineering
|
||||
principles, and have no derivation in the core C-rules.
|
||||
- Does **not** run from a notebook — a notebook in the pipeline path
|
||||
is a P9 violation (implicit state, unreproducible).
|
||||
|
||||
## Cross-Domain Links
|
||||
|
||||
- `domains/ai-ml/data-versioning.md` — the DVC pinning, the lineage
|
||||
record, the tool comparison (DVC / Delta Lake / LakeFS).
|
||||
- `domains/ai-ml/serving.md` — the model is promoted as an addressed
|
||||
artifact; the serving boundary validates inputs against the same
|
||||
schema as the eval.
|
||||
- `domains/ai-ml/model-evaluation.md` — the eval-as-a-gate that this
|
||||
run must pass before promotion.
|
||||
- `domains/devops/first-principles.md` P1 Reproducibility — the
|
||||
non-negotiable this run inherits.
|
||||
- `domains/data/migrations.md` — data versioning parallels schema
|
||||
migration discipline.
|
||||
- `domains/observability/logging.md` — the lineage record is a
|
||||
structured, append-only log of provenance.
|
||||
@@ -1,223 +0,0 @@
|
||||
# Good Example: Edge Offline-First + Sync Reconcile
|
||||
|
||||
> A field-service tablet app that operates through a 90-minute network
|
||||
> partition, queues writes locally, and reconciles on reconnect using
|
||||
> the CRDT-vs-LWW decision matrix from `domains/edge/sync.md`. Each
|
||||
> aspect cites the edge principle it satisfies.
|
||||
|
||||
## The Code
|
||||
|
||||
```typescript
|
||||
// The local store: an offline-first write queue + a CRDT register
|
||||
// per record. The tablet keeps working through the partition; the
|
||||
// queue drains on reconnect; the merge converges (P4 bounded).
|
||||
|
||||
interface LWWRegister<T> {
|
||||
value: T;
|
||||
timestamp: number; // monotonic logical clock (HLC), NOT wall time
|
||||
nodeId: string; // tablet id — deterministic tiebreak (P4)
|
||||
}
|
||||
|
||||
interface QueuedWrite {
|
||||
idempotencyKey: string; // P5: retried syncs are safe
|
||||
collection: string;
|
||||
recordId: string;
|
||||
register: LWWRegister<unknown>;
|
||||
queuedAt: number;
|
||||
}
|
||||
|
||||
class OfflineStore {
|
||||
private queue: QueuedWrite[] = [];
|
||||
private state: Map<string, LWWRegister<unknown>> = new Map();
|
||||
private telemetryBuffer: TelemetryRecord[] = []; // P10 local-first
|
||||
|
||||
// P2: write succeeds offline. The queue is the durable record of
|
||||
// intent; sync is deferred, not blocked.
|
||||
write(collection: string, recordId: string, value: unknown, key: string): void {
|
||||
const register: LWWRegister<unknown> = {
|
||||
value,
|
||||
timestamp: this.hlc.now(), // monotonic; no wall-clock skew
|
||||
nodeId: this.nodeId,
|
||||
};
|
||||
this.state.set(`${collection}:${recordId}`, register);
|
||||
this.queue.push({
|
||||
idempotencyKey: key, // P5: idempotent sync
|
||||
collection, recordId, register,
|
||||
queuedAt: this.hlc.now(),
|
||||
});
|
||||
// P10: buffer the write event locally; forwarded on reconnect.
|
||||
this.telemetryBuffer.push({ event: "local.write", at: Date.now(), key });
|
||||
}
|
||||
|
||||
// P2: read from local state offline. The UI never blocks on the
|
||||
// network.
|
||||
read(collection: string, recordId: string): unknown {
|
||||
return this.state.get(`${collection}:${recordId}`)?.value;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```typescript
|
||||
// The CRDT register merge — a state-based CvRDT (convergent). Per the
|
||||
// CRDT-vs-LWW decision matrix in domains/edge/sync.md, a state-based
|
||||
// CRDT is the choice when the data model fits a register and the
|
||||
// network is unreliable (full-state merge tolerates dropped ops).
|
||||
|
||||
function mergeLWWRegister<T>(
|
||||
local: LWWRegister<T>,
|
||||
remote: LWWRegister<T>,
|
||||
): LWWRegister<T> {
|
||||
// Associative, commutative, idempotent (P4 bound, P5 idempotent).
|
||||
// (timestamp, nodeId) is a total order — no oscillation.
|
||||
if (local.timestamp > remote.timestamp) return local;
|
||||
if (local.timestamp < remote.timestamp) return remote;
|
||||
return local.nodeId > remote.nodeId ? local : remote; // deterministic tie
|
||||
}
|
||||
```
|
||||
|
||||
```typescript
|
||||
// The reconnect reconcile: drain the offline queue, pull remote
|
||||
// state, three-way merge (CRDT registers), push converged state.
|
||||
// Idempotent keys make a retried reconcile safe (P5).
|
||||
|
||||
async function reconcile(store: OfflineStore, server: SyncServer): Promise<ReconcileReport> {
|
||||
// P5: the reconcile is idempotent. The idempotency key on each
|
||||
// queued write means a retry (network flapped mid-reconcile) does
|
||||
// not double-apply.
|
||||
const queued = store.drainQueue();
|
||||
let pushed = 0, merged = 0, conflicts = 0;
|
||||
|
||||
// 1. Push local writes. The server dedups by idempotencyKey (P5).
|
||||
for (const w of queued) {
|
||||
await server.applyWrite(w.idempotencyKey, w.collection, w.recordId, w.register);
|
||||
pushed++;
|
||||
}
|
||||
|
||||
// 2. Pull remote state for every record we touched + every record
|
||||
// the server changed since our last sync cursor. Merge via CRDT.
|
||||
const remoteRecords = await server.fetchChanged(store.syncCursor());
|
||||
for (const [key, remoteReg] of remoteRecords) {
|
||||
const localReg = store.localRegister(key);
|
||||
if (localReg) {
|
||||
// P4: CRDT merge converges. The merge is a pure function of
|
||||
// two inputs; the total order (timestamp, nodeId) guarantees
|
||||
// no oscillation.
|
||||
const converged = mergeLWWRegister(localReg, remoteReg);
|
||||
if (converged !== localReg) { conflicts++; }
|
||||
store.setLocal(key, converged);
|
||||
merged++;
|
||||
} else {
|
||||
store.setLocal(key, remoteReg); // remote-only record
|
||||
merged++;
|
||||
}
|
||||
}
|
||||
|
||||
// 3. P10: flush the buffered telemetry. The partition did not blind
|
||||
// the operator — the events survived on-node.
|
||||
store.flushTelemetry();
|
||||
|
||||
return { pushed, merged, conflicts, converged: true };
|
||||
}
|
||||
```
|
||||
|
||||
```typescript
|
||||
// P10: local-first telemetry. Events are buffered on-node and
|
||||
// forwarded on reconnect. A fire-and-forget pipeline loses data when
|
||||
// the link drops; a local-first buffer survives.
|
||||
|
||||
interface TelemetryRecord { event: string; at: number; key: string; }
|
||||
|
||||
class TelemetryBuffer {
|
||||
private buffer: TelemetryRecord[] = [];
|
||||
|
||||
push(rec: TelemetryRecord): void { this.buffer.push(rec); }
|
||||
|
||||
// Called from reconcile() on reconnect. The buffer is the P10
|
||||
// guarantee: the operator sees the partition-window activity,
|
||||
// not a gap.
|
||||
async flush(sink: TelemetrySink): Promise<void> {
|
||||
for (const rec of this.buffer) { await sink.emit(rec); }
|
||||
this.buffer = [];
|
||||
}
|
||||
|
||||
depth(): number { return this.buffer.length; }
|
||||
}
|
||||
```
|
||||
|
||||
## The Scenario
|
||||
|
||||
A field-service tablet is dispatched to a basement site with no
|
||||
cellular coverage. The technician updates the work-order status
|
||||
(`started`, `parts-ordered`, `completed`) five times over 90
|
||||
minutes. Each write lands in the local store immediately — the UI
|
||||
never blocks on the network. The writes are queued with an
|
||||
idempotency key (the work-order id + a monotonic sequence).
|
||||
|
||||
When the tablet reconnects, the reconcile drains the queue: the
|
||||
server dedups by idempotency key (a retry mid-reconcile does not
|
||||
double-apply). The server also returns a remote update — the
|
||||
dispatcher re-assigned the work order to a different technician at
|
||||
minute 45, then reverted at minute 60. The CRDT merge converges:
|
||||
the register with the later logical timestamp wins; the tiebreak by
|
||||
node id is deterministic. The merge terminates in one pass (P4
|
||||
bounded); it does not oscillate between the dispatcher's revert and
|
||||
the technician's status updates. The buffered telemetry flushes,
|
||||
and the operator sees the full partition-window activity — no gap.
|
||||
|
||||
## Principles Demonstrated
|
||||
|
||||
### Offline is a First-Class State (Edge P2, C1, C5)
|
||||
- The tablet writes and reads through the partition. The UI never
|
||||
blocks on the network; the offline write queue is the durable
|
||||
record of intent. Partition is the norm, not the exception;
|
||||
reconciliation happens on reconnect. An app that crashes on
|
||||
disconnect has no offline state; this app engineers it.
|
||||
- See `domains/edge/offline-first.md` (offline write-queue) and
|
||||
`domains/edge/first-principles.md` P2.
|
||||
|
||||
### Sync Conflicts are Bounded, Not Infinite (Edge P4, C1, C5)
|
||||
- The CRDT register merge is associative, commutative, and
|
||||
idempotent. The total order `(timestamp, nodeId)` guarantees
|
||||
convergence in one pass — no oscillation, no infinite loop. This
|
||||
is the P4 bound: the merge terminates. The CRDT-vs-LWW decision
|
||||
matrix in `domains/edge/sync.md` selected a state-based CRDT
|
||||
because the data model fits a register and the network is
|
||||
unreliable (full-state merge tolerates dropped ops).
|
||||
- See `domains/edge/sync.md` (CRDT-vs-LWW decision matrix, merge
|
||||
semantics) and `domains/edge/first-principles.md` P4.
|
||||
|
||||
### Edge Operations are Idempotent (Edge P5, C1)
|
||||
- Every queued write carries an idempotency key; the server dedups
|
||||
by key. A reconcile retried mid-flap does not double-apply. The
|
||||
merge function is idempotent (`merge(a, a) === a`) — a retried
|
||||
merge of the same two replicas yields the same result. Sync, the
|
||||
retried-by-nature operation, is safe.
|
||||
- See `domains/edge/sync.md` (Merge Semantics, P5) and
|
||||
`domains/edge/first-principles.md` P5.
|
||||
|
||||
### Edge Observability Survives Partition (Edge P10, C7, C5)
|
||||
- Telemetry is buffered on-node (`telemetryBuffer`) and flushed on
|
||||
reconnect. The partition did not blind the operator — the
|
||||
partition-window activity is forwarded, not lost. A
|
||||
fire-and-forget pipeline would have a 90-minute gap; the
|
||||
local-first buffer survives.
|
||||
- See `domains/edge/offline-first.md` (local-first logging) and
|
||||
`domains/edge/first-principles.md` P10.
|
||||
|
||||
## Cross-Domain Links
|
||||
|
||||
- `domains/edge/offline-first.md` — the offline write-queue pattern
|
||||
this app instantiates; the partition-as-norm discipline.
|
||||
- `domains/edge/sync.md` — the CRDT-vs-LWW decision matrix exercised
|
||||
here (state-based CvRDT chosen for register + unreliable network);
|
||||
the merge-semantics properties (associative, commutative,
|
||||
idempotent).
|
||||
- `domains/edge/first-principles.md` — P2, P4, P5, P10 are the
|
||||
principles demonstrated.
|
||||
- `domains/concurrency/patterns` — the in-process immutability
|
||||
analog (`concurrency/P1 Immutability by Default`) that makes the
|
||||
merge a pure function of two inputs.
|
||||
- `domains/observability/metrics` — the generic SLI/SLO discipline
|
||||
the local-first telemetry buffer builds on.
|
||||
- `review/anti-patterns.md` — the `edge-sync-loop` chaos
|
||||
anti-pattern is the inverse of this example's bounded CRDT merge.
|
||||
@@ -1,197 +0,0 @@
|
||||
# Good Example: GitOps Pull Request
|
||||
|
||||
> A pull request that changes ArgoCD Application manifests following
|
||||
> Atelier's GitOps + Operators principles. Each aspect cites the
|
||||
> principle it satisfies.
|
||||
|
||||
## The PR
|
||||
|
||||
A PR titled `promote payments-api 1.2.3 to prod` opened against the
|
||||
GitOps repo `platform/gitops`. It changes the `targetRevision` of the
|
||||
payments-api Application from `1.2.2` to `1.2.3`, adds a sync-wave
|
||||
annotation to a new migration Job, and tightens the AppProject's
|
||||
destination allow-list. CI runs plan/diff; nothing pushes to the
|
||||
cluster.
|
||||
|
||||
### The Commit
|
||||
|
||||
```yaml
|
||||
# manifests/prod/payments-api.yaml — the only file changed
|
||||
apiVersion: argoproj.io/v1alpha1
|
||||
kind: Application
|
||||
metadata:
|
||||
name: payments-api
|
||||
namespace: argocd
|
||||
finalizers:
|
||||
- resources-finalizer.argocd.argoproj.io
|
||||
spec:
|
||||
source:
|
||||
repoURL: https://git.example.com/platform/payments
|
||||
targetRevision: 1.2.3 # was 1.2.2 — pinned, not latest
|
||||
path: manifests/prod
|
||||
destination:
|
||||
server: https://kubernetes.default.svc
|
||||
namespace: payments
|
||||
syncPolicy:
|
||||
automated:
|
||||
prune: true
|
||||
selfHeal: true
|
||||
syncOptions:
|
||||
- CreateNamespace=false
|
||||
- PrunePropagationPolicy=foreground
|
||||
```
|
||||
|
||||
```yaml
|
||||
# manifests/prod/payments-db-migration.yaml — new file, wave-ordered
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: payments-db-migrate-1.2.3
|
||||
namespace: payments
|
||||
annotations:
|
||||
argocd.argoproj.io/sync-wave: "-1" # PreSync: run before the app
|
||||
spec:
|
||||
backoffLimit: 0
|
||||
ttlSecondsAfterFinished: 86400
|
||||
template:
|
||||
spec:
|
||||
restartPolicy: OnFailure
|
||||
serviceAccountName: payments-migrator
|
||||
containers:
|
||||
- name: migrate
|
||||
image: registry.example.com/payments/migrate:1.2.3
|
||||
args: ["up", "--to", "1.2.3"]
|
||||
```
|
||||
|
||||
```yaml
|
||||
# projects/payments.yaml — AppProject tightened in the same PR
|
||||
apiVersion: argoproj.io/v1alpha1
|
||||
kind: AppProject
|
||||
metadata:
|
||||
name: payments
|
||||
namespace: argocd
|
||||
spec:
|
||||
sourceRepos:
|
||||
- https://git.example.com/platform/payments
|
||||
destinations:
|
||||
- server: https://kubernetes.default.svc
|
||||
namespace: payments # prod-app can only deploy to payments ns
|
||||
clusterResourceWhitelist:
|
||||
- group: ""
|
||||
kind: Namespace # allowed to create its own namespace
|
||||
roles:
|
||||
- name: payments-team
|
||||
policies:
|
||||
- p, proj:payments:payments-team, applications, sync, payments/*, allow
|
||||
```
|
||||
|
||||
### The CI Pipeline (runs on the PR, before merge)
|
||||
|
||||
```text
|
||||
# .github/workflows/gitops-plan.yml (illustrative steps)
|
||||
- name: validate manifests
|
||||
run: argocd app manifests manifests/prod/ | kubeconform -strict
|
||||
|
||||
- name: diff against live cluster (read-only, no apply)
|
||||
run: argocd app diff payments-api --server $ARGOCD_SERVER --auth-token $READ_ONLY_TOKEN
|
||||
# CI holds a READ-ONLY ArgoCD token. It never holds kubectl rights.
|
||||
# A non-empty diff is the PR's proposed change, rendered for review.
|
||||
|
||||
- name: opa gate (admission policy pre-check)
|
||||
run: opa eval -i manifests/prod/ -d policies/ "data.k8s.admission.deny"
|
||||
# Policy violations fail the PR before merge, not after deploy.
|
||||
```
|
||||
|
||||
## What Makes It Good
|
||||
|
||||
### Git is the Source of Truth (GitOps P1, C1 Correctness)
|
||||
- The promotion is a commit. The cluster's desired state is a
|
||||
derivative of this repo; the repo is the authority. If the change is
|
||||
wrong, `git revert` is the rollback — the recovery path is the
|
||||
history.
|
||||
- See `domains/gitops-operators/first-principles.md` P1 and
|
||||
`domains/gitops-operators/argocd.md` (Application CRD).
|
||||
|
||||
### Pull, Don't Push (GitOps P3, C4 Locality)
|
||||
- CI holds a **read-only** ArgoCD token for `app diff`. It holds no
|
||||
`kubectl` rights against the production cluster. The cluster's
|
||||
ArgoCD controller pulls the merged commit; nothing pushes to the
|
||||
cluster. A compromised CI token can read, not deploy.
|
||||
- See `domains/gitops-operators/argocd.md` (RBAC and SSO) and
|
||||
`domains/gitops-operators/flux.md` for the same pull boundary from
|
||||
the Flux side.
|
||||
|
||||
### State is Immutable and Versioned (GitOps P5, C5 Reversibility)
|
||||
- `targetRevision: 1.2.3` — the Application pins a specific chart
|
||||
revision, not `latest`. The commit that changed it is a permanent
|
||||
record; `git revert` restores `1.2.2` and ArgoCD's `selfHeal`
|
||||
converges the cluster back. No force-push; history is the audit
|
||||
trail.
|
||||
- See `domains/gitops-operators/first-principles.md` P5 and
|
||||
`domains/infrastructure-as-code/state.md` (State is Truth).
|
||||
|
||||
### Sync Waves Order Correctness (GitOps P4, C1)
|
||||
- The migration Job carries `argocd.argoproj.io/sync-wave: "-1"` so
|
||||
it runs in `PreSync` before the payments-api Deployment that
|
||||
depends on the new schema. Wave ordering is a correctness
|
||||
mechanism, not performance — the app starting before its migration
|
||||
is a correctness bug.
|
||||
- See `domains/gitops-operators/argocd.md` (Sync Waves and Hooks).
|
||||
|
||||
### Reconcile, Don't Mutate by Hand (GitOps P8)
|
||||
- `selfHeal: true` + `prune: true` means a hand-edited drift on a
|
||||
managed resource is overwritten on the next loop. The fix for drift
|
||||
is a new commit, not `kubectl edit`. The PR author does not SSH into
|
||||
the cluster to "fix" anything.
|
||||
- See `domains/gitops-operators/argocd.md` (Diff and Drift) and
|
||||
`domains/gitops-operators/first-principles.md` P8.
|
||||
|
||||
### Least Privilege Reconciliation (GitOps P10, C8 Economy)
|
||||
- The AppProject `payments` restricts the Application to the
|
||||
`payments` namespace and the `payments` repo. The controller's
|
||||
ServiceAccount (not shown) is bound to a namespace-scoped Role, not
|
||||
`cluster-admin`. The PR *tightens* the allow-list — least privilege
|
||||
is a direction, not a one-time setting.
|
||||
- See `domains/gitops-operators/argocd.md` (RBAC and SSO) and
|
||||
`domains/kubernetes/rbac.md`.
|
||||
|
||||
### Policy is a Gate (Compliance P5, cross-link)
|
||||
- The `opa eval` step runs the admission policy against the proposed
|
||||
manifests before merge. A violation fails the PR; the non-compliant
|
||||
state is never realized. Detection is not enforcement; this is
|
||||
enforcement.
|
||||
- See `domains/compliance/policy-as-code.md` and
|
||||
`domains/devops/ci-cd.md`.
|
||||
|
||||
### Failure is Observable (GitOps P9)
|
||||
- A sync failure or health degradation on `payments-api` emits
|
||||
ArgoCD status (`Degraded` / `OutOfSync`) and a notification. Silent
|
||||
drift is the bug; this PR does not disable notifications.
|
||||
- See `domains/gitops-operators/argocd.md` (Health and Status) and
|
||||
`domains/observability/metrics.md`.
|
||||
|
||||
## What This PR Does NOT Do (And Why That's Good)
|
||||
|
||||
- Does **not** run `kubectl apply` from CI — that is the push pattern,
|
||||
a P3 violation (see `examples/bad/` for the anti-pattern).
|
||||
- Does **not** use `argocd app set` as the steady state — the change
|
||||
is in git, not in an imperative command's history.
|
||||
- Does **not** store raw Secrets in the GitOps repo — secrets arrive
|
||||
via Sealed Secrets / SOPS / External Secrets, encrypted in git.
|
||||
- Does **not** float `targetRevision: latest` — the Application pins
|
||||
a version; "latest" is an unknown model of the system.
|
||||
|
||||
## Cross-Domain Links
|
||||
|
||||
- `domains/gitops-operators/argocd.md` — the Application CRD, sync
|
||||
waves, RBAC/AppProjects, and the pull model.
|
||||
- `domains/gitops-operators/flux.md` — the same PR pattern from the
|
||||
Flux side (Kustomization CRD, per-cluster autonomy).
|
||||
- `domains/kubernetes/workloads.md` — the Deployment/Job the
|
||||
Application reconciles.
|
||||
- `domains/kubernetes/rbac.md` — the ServiceAccount + Role the
|
||||
controller and the migration Job run as.
|
||||
- `domains/compliance/policy-as-code.md` — the OPA gate is a
|
||||
compliance-as-a-gate enforcement point.
|
||||
- `domains/devops/P4 Rollback First` — `git revert` is the rollback;
|
||||
`selfHeal` is the convergence.
|
||||
@@ -1,226 +0,0 @@
|
||||
# Good Example: Idempotent Consumer with Dedup + DLQ
|
||||
|
||||
> An orders-consumer that achieves exactly-once-via-idempotency:
|
||||
> at-least-once delivery plus a TTL-bounded dedup store and a DLQ
|
||||
> routing rule. Each aspect cites the messaging principle it
|
||||
> satisfies. Exercises the idempotency + DLQ guidance in
|
||||
> `domains/messaging/delivery-semantics.md`.
|
||||
|
||||
## The Code
|
||||
|
||||
```python
|
||||
# The idempotency-key dedup store (P3). TTL-bounded (P6): a dedup
|
||||
# store with no TTL is a memory leak. The TTL exceeds the broker's
|
||||
# max-redelivery window; beyond it, the key is expired (the broker
|
||||
# has given up).
|
||||
|
||||
import time, json
|
||||
|
||||
DEDUP_TTL_SECONDS = 24 * 3600 # > broker max-redelivery window
|
||||
|
||||
class DedupStore:
|
||||
"""P3 (idempotent), P6 (TTL-bounded). seen() before process;
|
||||
mark() after process; the order gives at-least-once + dedup."""
|
||||
|
||||
def __init__(self, backend):
|
||||
# backend is Redis or a shared DB. MUST be shared across
|
||||
# consumer instances (see messaging/pubsub.md on shared vs
|
||||
# independent subscriptions).
|
||||
self.backend = backend
|
||||
|
||||
def seen(self, key: str) -> bool:
|
||||
ts = self.backend.get(key)
|
||||
if ts is None:
|
||||
return False
|
||||
if time.time() - ts > DEDUP_TTL_SECONDS:
|
||||
self.backend.delete(key) # P6: expired; not a redelivery
|
||||
return False
|
||||
return True
|
||||
|
||||
def mark(self, key: str):
|
||||
self.backend.set(key, time.time(), ttl=DEDUP_TTL_SECONDS)
|
||||
```
|
||||
|
||||
```python
|
||||
# The idempotent consumer. Order: dedup BEFORE process, mark AFTER
|
||||
# process, ack AFTER mark. A crash before mark re-processes (the
|
||||
# dedup store lacks the key); a crash before ack redelivers and the
|
||||
# dedup store makes the redelivery a no-op (P3). For a
|
||||
# non-idempotent process (a payment that must not double-charge),
|
||||
# process+mark are one DB transaction — exactly-once via
|
||||
# idempotency (P4).
|
||||
|
||||
def consume_orders(broker, dedup: DedupStore, process_order):
|
||||
for message in broker.receive():
|
||||
payload = json.loads(message["body"])
|
||||
|
||||
# P3: dedup BEFORE process. A redelivered message is a
|
||||
# no-op, not a double-apply.
|
||||
if dedup.seen(payload["idempotencyKey"]):
|
||||
broker.ack(message) # already processed; skip
|
||||
continue
|
||||
|
||||
try:
|
||||
# P4: the declared semantic is at-least-once + idempotent
|
||||
# dedup = exactly-once-via-idempotency. For a payment,
|
||||
# process_order + dedup.mark run in one DB transaction
|
||||
# so the mark commits iff the process commits.
|
||||
process_order(payload)
|
||||
dedup.mark(payload["idempotencyKey"])
|
||||
broker.ack(message)
|
||||
|
||||
except TransientError as exc:
|
||||
# P6: bounded retry with backoff. Nack for redelivery;
|
||||
# the broker redelivers after exponential backoff.
|
||||
broker.nack(message, delay=backoff(payload.get("attempt", 0)))
|
||||
|
||||
except (ValueError, SchemaError) as exc:
|
||||
# P5: poison message — unparseable. Route immediately,
|
||||
# do NOT retry (no retry will fix a bad schema).
|
||||
route_to_dlq(broker, message, exc, kind="poison")
|
||||
broker.ack(message)
|
||||
|
||||
except PermanentError as exc:
|
||||
# P5: permanent failure (e.g., not-found dependency).
|
||||
# Retry will not fix it — DLQ now.
|
||||
route_to_dlq(broker, message, exc, kind="dlq")
|
||||
broker.ack(message)
|
||||
```
|
||||
|
||||
```python
|
||||
# The DLQ routing rule (P5 dead-letter handling, P10 DLQ depth
|
||||
# alert). Distinguishes poison (unparseable; never retried) from
|
||||
# DLQ (exhausted retry budget on a transient). Both carry audit
|
||||
# metadata; both emit a depth metric.
|
||||
|
||||
DLQ = "orders-dlq"
|
||||
POISON = "orders-poison"
|
||||
MAX_RETRY_TTL_SECONDS = 30 * 60 # 30 min retry window
|
||||
|
||||
def route_to_dlq(broker, message, reason, kind: str):
|
||||
target = POISON if kind == "poison" else DLQ
|
||||
broker.send(target, body=json.dumps({
|
||||
"original": message["body"],
|
||||
"reason": str(reason),
|
||||
"kind": kind, # poison vs dlq
|
||||
"deadLetteredAt": now_iso(),
|
||||
"redeliveryCount": message.get("attempt", 0),
|
||||
}))
|
||||
# P10: emit a metric so DLQ depth alerts fire. A DLQ that grows
|
||||
# with no alert is a silent correctness defect (P5/P10).
|
||||
metrics.increment(f"{kind}.depth", tags={"queue": "orders"})
|
||||
|
||||
|
||||
def consume_with_retry_budget(broker, dedup, process_order):
|
||||
# Combines TTL-with-backoff for transient failures (P6) with
|
||||
# poison-queue + DLQ + alert (P5/P10).
|
||||
for message in broker.receive():
|
||||
payload = json.loads(message["body"])
|
||||
if dedup.seen(payload["idempotencyKey"]):
|
||||
broker.ack(message); continue
|
||||
|
||||
first_attempt_ts = payload.get("firstAttemptTs", time.time())
|
||||
attempt = payload.get("attempt", 0)
|
||||
|
||||
try:
|
||||
process_order(payload)
|
||||
dedup.mark(payload["idempotencyKey"])
|
||||
broker.ack(message)
|
||||
|
||||
except TransientError as exc:
|
||||
# P6: if the retry window is exhausted, route to DLQ;
|
||||
# otherwise redeliver with exponential backoff.
|
||||
if time.time() - first_attempt_ts > MAX_RETRY_TTL_SECONDS:
|
||||
route_to_dlq(broker, message, exc, kind="dlq") # P5
|
||||
broker.ack(message)
|
||||
else:
|
||||
broker.nack(message, delay=backoff(attempt))
|
||||
```
|
||||
|
||||
## The Scenario
|
||||
|
||||
An orders queue delivers `order.created` events to the consumer at
|
||||
**at-least-once** (the declared semantic, P4). The broker redelivers
|
||||
on consumer crash or ack-timeout. Three things happen:
|
||||
|
||||
1. **Normal delivery** — the consumer dedups by `idempotencyKey`,
|
||||
processes, marks, acks.
|
||||
2. **Redelivery after a crash before ack** — the consumer crashed
|
||||
after `mark` but before `ack`. The broker redelivers; `seen()`
|
||||
returns true; the consumer acks without re-processing (P3).
|
||||
3. **Poison message** — a malformed JSON body. The consumer routes
|
||||
it to the poison queue immediately (no retry will fix a parse
|
||||
error), acks the origin, and emits a `poison.depth` metric. The
|
||||
operator is paged on poison-queue growth (P10).
|
||||
|
||||
A transient downstream failure (the payments API is briefly 503)
|
||||
retries with exponential backoff for 30 minutes (P6); if it exceeds
|
||||
the budget, the message routes to the DLQ with `reason`,
|
||||
`redeliveryCount`, and `deadLetteredAt` — auditable, drainable,
|
||||
observable (P5). The DLQ depth metric alerts the operator; the DLQ
|
||||
entry's audit metadata lets the operator replay after the bug is
|
||||
fixed (P5 reversibility).
|
||||
|
||||
## Principles Demonstrated
|
||||
|
||||
### Consumers are Idempotent (Messaging P3, C1)
|
||||
- The consumer dedups by idempotency key before processing. A
|
||||
redelivered message is a no-op, not a double-apply. The
|
||||
`process → mark → ack` order gives at-least-once + idempotent
|
||||
dedup; for a non-idempotent process, `process + mark` are one DB
|
||||
transaction (exactly-once via idempotency, P4).
|
||||
- See `domains/messaging/delivery-semantics.md` (idempotency-key
|
||||
dedup store) and `domains/messaging/first-principles.md` P3.
|
||||
|
||||
### Delivery Semantics are Explicit (Messaging P4, C1, C2)
|
||||
- The channel is declared **at-least-once + idempotent consumer** —
|
||||
the engineering practice that collapses to exactly-once under
|
||||
correct dedup (P3). The semantic is not emergent; it is the
|
||||
declared choice per channel. The tradeoff (dedup-store cost,
|
||||
transactional-process complexity) is conscious and documented.
|
||||
- See `domains/messaging/queues.md` (the three-semantics comparison
|
||||
table) and `domains/messaging/delivery-semantics.md` (exactly-
|
||||
once via idempotency).
|
||||
|
||||
### Dead-Letter Handling is Defined (Messaging P5, C1, C5)
|
||||
- Poison messages (unparseable) route immediately to the poison
|
||||
queue — no retry will fix them. Transient failures retry with
|
||||
backoff until the TTL, then route to the DLQ. Both carry audit
|
||||
metadata (`reason`, `redeliveryCount`, `deadLetteredAt`); both are
|
||||
drainable and observable. The DLQ is the reversibility mechanism
|
||||
— a dead-lettered message can be reprocessed after the bug is
|
||||
fixed.
|
||||
- See `domains/messaging/delivery-semantics.md` (dead-letter
|
||||
strategy comparison table, DLQ routing rule) and
|
||||
`domains/messaging/first-principles.md` P5.
|
||||
|
||||
### Messaging is Observable (Messaging P10, C7, C1)
|
||||
- DLQ depth and poison-queue depth are emitted as metrics and wired
|
||||
to alerts. A DLQ that grows silently is a correctness defect;
|
||||
the alert makes it visible. Silent backlog is a bug, not a
|
||||
feature — the operator is paged on growth, not on a customer
|
||||
report.
|
||||
- See `domains/messaging/delivery-semantics.md` (DLQ depth as an
|
||||
alert) and `domains/messaging/first-principles.md` P10.
|
||||
|
||||
## Cross-Domain Links
|
||||
|
||||
- `domains/messaging/delivery-semantics.md` — the idempotency-key
|
||||
dedup-store pattern and the dead-letter strategy comparison table
|
||||
exercised here (TTL-with-backoff + poison-queue + DLQ + alert).
|
||||
- `domains/messaging/queues.md` — the three-semantics comparison
|
||||
table; the ack/nack/redelivery model this consumer uses.
|
||||
- `domains/messaging/first-principles.md` — P3, P4, P5, P10 are the
|
||||
principles demonstrated.
|
||||
- `domains/concurrency/patterns` — the in-process retry/backoff
|
||||
analog (Pattern 6, Timeout on Every Block); messaging owns the
|
||||
broker-backed instance where redelivery comes across a network.
|
||||
- `domains/errors/patterns` — errors as data: a DLQ entry is the
|
||||
async-messaging instance of an error log (observable, auditable,
|
||||
drainable).
|
||||
- `domains/observability/metrics` — the generic SLI/SLO discipline
|
||||
the DLQ-depth alert builds on.
|
||||
- `review/anti-patterns.md` — the `messaging-unbounded-retry` and
|
||||
`messaging-shared-subscription` chaos anti-patterns are the
|
||||
inverse of this example's bounded retry + independent-consumer
|
||||
discipline.
|
||||
@@ -1,171 +0,0 @@
|
||||
# Go Concurrency — Derived Application
|
||||
|
||||
> Applies Atelier's domain principles to Go's concurrency specifically. Go's distinctive strength (goroutines, channels, context) earns a dedicated concurrency doc rather than a `go-async.md`.
|
||||
> Derives from `domains/` docs; introduces no new P-rules (D-063).
|
||||
> See `languages/go.md` for the language first-principles stub.
|
||||
|
||||
## Goroutines and Structured Concurrency (Concurrency P1 Immutability by Default, C6 Composability)
|
||||
|
||||
- **`go f()` spawns a goroutine; ensure it does not outlive its parent:** an unstructured `go f()` leaks when the parent returns. Use `sync.WaitGroup`, `errgroup.Group`, or a `context`-scoped pattern to bound lifetime.
|
||||
- **`errgroup.WithContext` for structured concurrency:** a `Group` cancels its context on first error; siblings see the cancellation and exit. Mirrors `TaskGroup` semantics cross-language.
|
||||
- **Goroutines share only immutable inputs:** `go process(snap)` where `snap` is a copy. A goroutine sharing a mutable slice with the parent is a race (Concurrency P1 Immutability, P6 No Silent Races).
|
||||
- **No `go` in a library function without a documented lifetime:** a library that spawns unbounded goroutines leaks them into the caller. Either accept a `context.Context` or return a `Stop()` method.
|
||||
|
||||
```go
|
||||
import "golang.org/x/sync/errgroup"
|
||||
|
||||
func fetchAll(ctx context.Context, ids []string) ([]*User, error) {
|
||||
g, ctx := errgroup.WithContext(ctx)
|
||||
results := make([]*User, len(ids))
|
||||
for i, id := range ids {
|
||||
i, id := i, id // capture loop vars
|
||||
g.Go(func() error {
|
||||
u, err := fetchUser(ctx, id)
|
||||
if err != nil { return err }
|
||||
results[i] = u
|
||||
return nil
|
||||
})
|
||||
}
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return results, nil
|
||||
}
|
||||
```
|
||||
|
||||
## Channels: Bounded Queues and Backpressure (Concurrency P9 Bounded Queues, C6 Composability)
|
||||
|
||||
- **Bounded channels apply backpressure:** `make(chan T, N)` blocks the sender when full (Concurrency P9 — bounded queues). Unbounded `make(chan T)` lets the producer run ahead and OOM.
|
||||
- **`select` with `default` for non-blocking send/receive:** a `default` case makes the channel a queue with try semantics; without it, the operation blocks.
|
||||
- **Close channel from the sender, never the receiver:** closing a channel signals "no more sends." A receiver closing it is a race; the sender may still be writing.
|
||||
- **One channel, one responsibility:** do not multiplex control and data on the same channel. Use a `select` over multiple channels instead.
|
||||
- **Applies `messaging/queues`:** a bounded Go channel is an in-process broker — bounded buffer, backpressure, at-most-once handoff. The same semantics apply; the broker is local.
|
||||
|
||||
```go
|
||||
func pipeline(ctx context.Context, in <-chan Job, out chan<- Result) {
|
||||
for {
|
||||
select {
|
||||
case j, ok := <-in:
|
||||
if !ok { return }
|
||||
r := process(j)
|
||||
select {
|
||||
case out <- r:
|
||||
case <-ctx.Done():
|
||||
return
|
||||
}
|
||||
case <-ctx.Done():
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// bounded: backpressure when out is full
|
||||
out := make(chan Result, 16)
|
||||
```
|
||||
|
||||
## context.Context for Cancellation (Concurrency P7 Cancellation Support, Concurrency P8 Timeout Discipline)
|
||||
|
||||
- **`context.Context` is the first parameter of every I/O function:** `func fetchUser(ctx context.Context, id string) (*User, error)`. A function that does I/O without a `ctx` cannot be cancelled (Concurrency P7).
|
||||
- **`context.WithTimeout` for a deadline:** `ctx, cancel := context.WithTimeout(ctx, 5*time.Second); defer cancel()`. Every external call races against a deadline (Concurrency P8).
|
||||
- **`cancel()` always called, even on success:** `defer cancel()` immediately after creating the context. A leaked context leaks its timer.
|
||||
- **Never store a `context.Context` in a struct:** pass it as a parameter. A struct holding a `ctx` captures a request-scoped value into a long-lived object.
|
||||
- **Applies `concurrency/P7`:** cancellation propagates via `ctx.Done()`. A `select` on `<-ctx.Done()` is the cancel-aware wait.
|
||||
|
||||
```go
|
||||
func fetchWithTimeout(ctx context.Context, url string) (*Response, error) {
|
||||
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
|
||||
defer cancel()
|
||||
|
||||
req, _ := http.NewRequestWithContext(ctx, "GET", url, nil)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
if errors.Is(err, context.DeadlineExceeded) {
|
||||
return nil, ErrTimeout
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
return resp, nil
|
||||
}
|
||||
```
|
||||
|
||||
## select and Multiplexed Channels (Concurrency P7 Cancellation Support, C6 Composability)
|
||||
|
||||
- **`select` multiplexes channel operations:** it picks a ready case at random (fair). A `select` with `<-ctx.Done()` plus a data case is the cancel-aware wait.
|
||||
- **`default` makes `select` non-blocking:** use for "send if ready, else drop" (a bounded queue with drop-oldest policy).
|
||||
- **`select {}` blocks forever:** a `select{}` with no cases is a permanent block. Use only in a goroutine that should run until the process exits.
|
||||
- **Applies `concurrency/P7`:** the `select` over `ctx.Done()` and a result channel is the canonical cancel pattern.
|
||||
|
||||
```go
|
||||
func processUntilCancel(ctx context.Context, jobs <-chan Job) {
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case j, ok := <-jobs:
|
||||
if !ok { return }
|
||||
// ...
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## sync Primitives and Lock Scope (Concurrency P3 Boundaries are Locks, Concurrency P5 Lock Minimization)
|
||||
|
||||
- **`sync.Mutex` scoped minimally:** not held across I/O (a `Send` on a channel, an HTTP call). Hold the lock, mutate, release — then do I/O (Concurrency P3 Lock Scope).
|
||||
- **`sync.RWMutex` for read-heavy, `Mutex` for write-heavy:** RWMutex adds overhead; only prefer it when reads dominate by 10x+.
|
||||
- **`sync.Map` for specific cases (append-only, disjoint keys):** not a general `map[K]V` replacement. For most maps, `Mutex` + `map` is clearer and often faster.
|
||||
- **`sync.Once` for one-time init:** `var once sync.Once; once.Do(func(){ init() })`. Idempotent and race-free.
|
||||
- **Applies `concurrency/P5` (lock minimization):** prefer channels over locks; when a lock is needed, hold it for the smallest possible scope.
|
||||
|
||||
```go
|
||||
type Cache struct {
|
||||
mu sync.Mutex
|
||||
items map[string]*User
|
||||
}
|
||||
|
||||
func (c *Cache) Get(id string) (*User, bool) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
u, ok := c.items[id]
|
||||
return u, ok
|
||||
}
|
||||
|
||||
func (c *Cache) Set(id string, u *User) {
|
||||
c.mu.Lock()
|
||||
c.items[id] = u
|
||||
c.mu.Unlock() // explicit unlock before any I/O
|
||||
}
|
||||
```
|
||||
|
||||
## Race Detection (Concurrency P6 No Silent Races)
|
||||
|
||||
- **`go test -race` enforces `P6`:** the race detector instruments memory accesses and fails on data races. See `go-tooling.md` for the CI gate.
|
||||
- **Tests must exercise the concurrent path:** a serial test of a `Mutex`-protected map finds no race. Write tests with N goroutines hitting the map under `-race`.
|
||||
- **Applies `concurrency/P6`:** a race detected at test time is a bug fixed; a race undetected is a production heisenbug.
|
||||
|
||||
```go
|
||||
func TestCacheConcurrent(t *testing.T) {
|
||||
c := &Cache{items: map[string]*User{}}
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 100; i++ {
|
||||
i := i
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
c.Set(strconv.Itoa(i), &User{})
|
||||
c.Get(strconv.Itoa(i))
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
}
|
||||
```
|
||||
|
||||
## Cross-References
|
||||
|
||||
- `domains/concurrency/patterns.md` — the cancellation/timeout/semaphore patterns applied here.
|
||||
- `domains/concurrency/first-principles.md` — Concurrency P1, P3, P5, P6, P7, P8, P9 traced throughout.
|
||||
- `domains/messaging/queues.md` — bounded Go channels as in-process brokers; backpressure parallels (IDEATE-40).
|
||||
- `domains/errors/patterns.md` — `errgroup` and error propagation in concurrent code.
|
||||
- `languages/go-types.md` — typed channels carry the named types defined there.
|
||||
- `languages/go-tooling.md` — the `-race` CI gate that enforces Concurrency P6.
|
||||
- `languages/go-testing.md` — concurrent tests that exercise the race detector.
|
||||
@@ -1,141 +0,0 @@
|
||||
# Go Testing — Derived Application
|
||||
|
||||
> Applies Atelier's domain principles to Go testing specifically.
|
||||
> Derives from `domains/` docs; introduces no new P-rules (D-063).
|
||||
> See `languages/go.md` for the language first-principles stub.
|
||||
|
||||
## Table-Driven Tests (Testing P1 Tests as Specification, C2 Clarity)
|
||||
|
||||
- **Table-driven is the Go idiom:** `cases := []struct{ name string; in X; want Y }{...}`; loop with `t.Run(c.name, ...)`. Each case is a subtest with its own name and failure output.
|
||||
- **Test names read as a spec:** `{"rejects empty email", ...}`, `{"returns persisted id", ...}`. A reader understands the unit from the subtest names (Testing P1).
|
||||
- **No `if got != want { t.Fatal() }` shared across cases:** each case asserts independently; a failure in case 3 does not skip cases 4 and 5.
|
||||
- **`t.Run` enables `-run` filtering:** `go test -run TestCreateUser/rejects_empty_email` runs one case. Essential for debugging a single failure.
|
||||
|
||||
```go
|
||||
func TestCreateUser(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
email string
|
||||
wantErr bool
|
||||
}{
|
||||
{"rejects empty email", "", true},
|
||||
{"rejects missing @", "no-at-sign", true},
|
||||
{"accepts valid email", "a@b.co", false},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
_, err := CreateUser(c.email)
|
||||
if (err != nil) != c.wantErr {
|
||||
t.Fatalf("err=%v, wantErr=%v", err, c.wantErr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## t.Parallel for Independence (Testing P2 Independence, Concurrency P10 Test for Race Conditions)
|
||||
|
||||
- **`t.Parallel()` for independent subtests:** each subtest opts in; the runner executes them concurrently. A test that fails under `Parallel` has hidden state (Testing P2 Independence).
|
||||
- **Capture loop variables:** `c := c` inside the loop, or rely on Go 1.22+ per-iteration scoping. A parallel subtest sharing `c` races on the last value.
|
||||
- **Applies `concurrency/P10` (test for races):** parallel tests are the first line of race detection; combine with `-race` for the full safety net.
|
||||
|
||||
```go
|
||||
for _, c := range cases {
|
||||
c := c // capture for parallel
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
_, err := CreateUser(c.email)
|
||||
if (err != nil) != c.wantErr {
|
||||
t.Fatalf("err=%v, wantErr=%v", err, c.wantErr)
|
||||
}
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
## t.Cleanup for Teardown (Testing P3 Determinism, Testing P2 Independence)
|
||||
|
||||
- **`t.Cleanup(func() { ... })` for teardown:** runs in LIFO order after the test (and its subtests) complete. Replaces `defer` in a helper that does not know when the test ends.
|
||||
- **Per-test state, not shared:** a `setup(t)` helper creates resources and registers cleanup; each test gets its own. A package-level `var` shared across tests is order coupling.
|
||||
- **`t.TempDir()` for filesystem tests:** creates a unique temp dir and cleans up automatically. No manual `os.RemoveAll` and no cross-test contamination.
|
||||
- **Applies `Testing P3` (determinism):** cleanup is tied to the test lifecycle, not a global teardown that may run before or after depending on order.
|
||||
|
||||
```go
|
||||
func setupStore(t *testing.T) *Store {
|
||||
t.Parallel()
|
||||
dir := t.TempDir() // auto-cleaned
|
||||
s, err := OpenStore(filepath.Join(dir, "db"))
|
||||
if err != nil { t.Fatal(err) }
|
||||
t.Cleanup(func() { s.Close() })
|
||||
return s
|
||||
}
|
||||
```
|
||||
|
||||
## Race Detector (Testing P9 Edge Case Coverage, Concurrency P6 No Silent Races)
|
||||
|
||||
- **`go test -race` in CI, always:** see `go-tooling.md`. The detector is the enforcement of `concurrency/P6`.
|
||||
- **Tests must exercise the concurrent path:** a serial test of a `Mutex`-protected map finds no race. Write tests with N goroutines.
|
||||
- **`-count=1` to disable result caching:** by default, Go caches passing tests. `-count=1` forces re-run; combine with `-race` and parallelism to surface heisenbugs.
|
||||
- **Applies `Testing P9` (edge case coverage):** the race detector is the edge-case tool for concurrency — it finds the inputs the test author forgot to write.
|
||||
|
||||
```bash
|
||||
# CI gate
|
||||
go test -race -count=1 ./...
|
||||
```
|
||||
|
||||
## Time and Determinism (Testing P3 Determinism, Testing P9 Edge Case Coverage)
|
||||
|
||||
- **No `time.Now()` in code under test:** inject a `Clock` interface. In tests, a fake clock advances deterministically.
|
||||
- **`time.Sleep` in tests is a smell:** a sleep waits for a real timer, flaky under load. Use a channel or `Eventually`-style polling with a timeout.
|
||||
- **`t.Deadline()` aware helpers:** a helper that may take long checks `t.Deadline()` and bails early. Prevents a slow test from timing out the suite.
|
||||
|
||||
```go
|
||||
type Clock interface { Now() time.Time }
|
||||
|
||||
type fakeClock struct{ t time.Time }
|
||||
func (f *fakeClock) Now() time.Time { return f.t }
|
||||
|
||||
func TestUserHasCreatedAt(t *testing.T) {
|
||||
clk := &fakeClock{time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)}
|
||||
u, _ := CreateUserWithClock("a@b.co", clk)
|
||||
if u.CreatedAt.Year() != 2024 {
|
||||
t.Fatalf("year=%d, want 2024", u.CreatedAt.Year())
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Mocks and Interfaces (Testing P7 Realism, API P1 Contract Fidelity)
|
||||
|
||||
- **Mock at the interface, not the struct:** `type Store interface { Get(id string) (*User, error) }` in production; `type mockStore struct{ ... }` in test. The interface is the contract (applies `api/P1`).
|
||||
- **`httptest` for HTTP servers:** `httptest.NewServer` gives a real server on a loopback port; no manual socket plumbing.
|
||||
- **`testify/mock` or hand-written mocks:** hand-written for one-off, `testify` for complex sequencing. Avoid mocking frameworks that generate code at runtime (reflection-heavy) — they hide failures behind stack traces.
|
||||
- **Applies `Testing P7` (realism):** mock the boundary (HTTP, DB), not the unit. Mocking the unit under test tests the mock.
|
||||
|
||||
```go
|
||||
type mockStore struct {
|
||||
users map[string]*User
|
||||
got []string
|
||||
}
|
||||
func (m *mockStore) Get(id string) (*User, error) {
|
||||
m.got = append(m.got, id)
|
||||
return m.users[id], nil
|
||||
}
|
||||
|
||||
func TestGetUserLogs(t *testing.T) {
|
||||
s := &mockStore{users: map[string]*User{"abc": {}}}
|
||||
svc := NewService(s)
|
||||
svc.GetUser("abc")
|
||||
if len(s.got) != 1 || s.got[0] != "abc" {
|
||||
t.Fatalf("got=%v", s.got)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Cross-References
|
||||
|
||||
- `domains/testing/pyramid.md` — where unit/integration/race tests sit; the race job is its own layer.
|
||||
- `domains/testing/fixtures.md` — `t.TempDir` and `t.Cleanup` as the fixture discipline.
|
||||
- `domains/testing/first-principles.md` — Testing P1 Specification, P2 Independence, P3 Determinism, P9 Edge Coverage.
|
||||
- `domains/concurrency/first-principles.md` — Concurrency P6 (race detector), P10 (test for races).
|
||||
- `languages/go-types.md` — the named types tests assert.
|
||||
- `languages/go-concurrency.md` — concurrent tests exercise the patterns from that doc.
|
||||
- `languages/go-tooling.md` — the `go test` flags (`-race`, `-count`, `-run`) detailed here.
|
||||
@@ -1,97 +0,0 @@
|
||||
# Go Tooling — Derived Application
|
||||
|
||||
> Applies Atelier's domain principles to Go tooling specifically.
|
||||
> Derives from `domains/` docs; introduces no new P-rules (D-063).
|
||||
> See `languages/go.md` for the language first-principles stub.
|
||||
|
||||
## go vet and golangci-lint (DevOps P2 Automation, C2 Clarity)
|
||||
|
||||
- **`go vet` is the stdlib baseline:** it catches `printf` format mismatches, lock-copy-by-value, and unreachable code. Run on every build.
|
||||
- **`golangci-lint` aggregates vet + dozens of linters:** enable `errcheck` (no `_ = err`), `govet`, `staticcheck`, `ineffassign`, `unused`, `gofmt`, `goimports`. Each enabled linter has a one-line `# reason:` in `.golangci.yml`.
|
||||
- **`errcheck` enforces `errors/P2` (fail loudly):** a discarded error is a silent failure. `errcheck` fails the build on `_ = doX()`.
|
||||
- **`goimports` over `gofmt`:** `goimports` adds missing imports and removes unused ones, in addition to formatting. The format is not debated in review (Clarity C2).
|
||||
|
||||
```yaml
|
||||
# .golangci.yml
|
||||
linters:
|
||||
enable:
|
||||
- errcheck # reason: Errors P2 — no swallowed errors
|
||||
- govet
|
||||
- staticcheck
|
||||
- ineffassign
|
||||
- unused
|
||||
- gofmt
|
||||
- goimports
|
||||
linters-settings:
|
||||
errcheck:
|
||||
check-blank: true # fail on _ = fn()
|
||||
```
|
||||
|
||||
## go test -race (Concurrency P6 No Silent Races)
|
||||
|
||||
- **`go test -race` in CI, always:** the race detector instruments memory accesses and fails on data races. It is the primary enforcement of `concurrency/P6` (no silent races).
|
||||
- **`-race` adds overhead; run it in a separate CI job:** the race build is ~2x slower; keep the fast unit-test job and add a race job.
|
||||
- **`-race` requires tests that actually exercise the concurrent path:** a test that calls `Get`/`Set` serially finds no race. Write tests that spawn goroutines hitting the same map.
|
||||
- **Applies `concurrency/P6`:** a race detected is a bug fixed; a race undetected is a heisenbug in production. The detector is the safety net.
|
||||
|
||||
```bash
|
||||
# CI race job
|
||||
go test -race -count=1 ./...
|
||||
```
|
||||
|
||||
## Module Discipline (DevOps P1 Reproducibility)
|
||||
|
||||
- **`go mod tidy` on every change that touches imports:** removes unused deps and adds missing ones. A `go.mod` with stale entries breaks reproducibility.
|
||||
- **`go.sum` committed and verified:** `go mod verify` checks the checksums of the module cache against `go.sum`. A drifted `go.sum` is a supply-chain signal.
|
||||
- **Pinned major versions in `go.mod`:** `require github.com/x/y v1.2.3` pins the minor; a `v1.2.4` patch may auto-update. For applications, consider a `go.mod` proxy that pins to exact commits.
|
||||
- **`go mod vendor` for hermetic CI:** vendoring `vendor/` into the repo means CI builds without network. The trade-off is repo size; the win is reproducibility (DevOps P1).
|
||||
|
||||
```bash
|
||||
# CI build gate
|
||||
go mod tidy
|
||||
go mod verify
|
||||
go build ./...
|
||||
go test -race ./...
|
||||
```
|
||||
|
||||
## Reproducible Builds (DevOps P1 Reproducibility, C3 Simplicity)
|
||||
|
||||
- **One Go toolchain version, pinned:** `goenv` or `asdf` pins the Go version per repo; a `.go-version` file declares it. A CI job that uses "latest" Go drifts.
|
||||
- **`CGO_ENABLED=0` for static binaries:** a static binary runs in a scratch container with no libc dependency. Set in CI for all release builds.
|
||||
- **`-trimpath` and `-ldflags='-s -w'` for reproducible output:** strips the build path from the binary and removes debug info. Two builds of the same commit produce byte-identical binaries.
|
||||
|
||||
```bash
|
||||
# Reproducible release build
|
||||
CGO_ENABLED=0 go build -trimpath -ldflags='-s -w' -o app ./cmd/app
|
||||
```
|
||||
|
||||
## Documentation in the Pipeline (Documentation P1 Documentation is Code, DevOps P9 Documentation in the Pipeline)
|
||||
|
||||
- **`go doc` from comments:** package comments and exported-symbol comments are the API docs; `go doc` and `pkg.go.dev` render them. Missing comments on exported symbols fail `revive`/`golint` (Documentation P1).
|
||||
- **`// Example` functions are run by `go test`:** an `ExampleUser` function with `// Output:` is a tested artifact; a stale output fails the build.
|
||||
- **`README.md` and `docs/` are built by `mkdocs` or similar:** the pipeline validates links and renders; a broken link fails CI (Documentation P1).
|
||||
|
||||
```go
|
||||
// GetUser fetches a user by id.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// u, err := GetUser(id)
|
||||
// if err != nil { ... }
|
||||
func GetUser(id UserId) (*User, error) { /* ... */ }
|
||||
|
||||
func ExampleGetUser() {
|
||||
u, err := GetUser("abc")
|
||||
fmt.Println(u, err)
|
||||
// Output: <nil> not found
|
||||
}
|
||||
```
|
||||
|
||||
## Cross-References
|
||||
|
||||
- `domains/devops/ci-cd.md` — the pipeline gates that host vet/lint/test.
|
||||
- `domains/devops/first-principles.md` — DevOps P1 Reproducibility, P2 Automation.
|
||||
- `domains/concurrency/first-principles.md` — Concurrency P6 No Silent Races (`-race`).
|
||||
- `domains/documentation/first-principles.md` — Documentation P1 Documentation is Code.
|
||||
- `languages/go-types.md` — the type rules staticcheck enforces reference this doc.
|
||||
- `languages/go-testing.md` — the `go test` flags (`-race`, `-count`) detailed here.
|
||||
@@ -1,140 +0,0 @@
|
||||
# Go Type System — Derived Application
|
||||
|
||||
> Applies Atelier's domain principles to Go's type system specifically.
|
||||
> Derives from `domains/` docs; introduces no new P-rules (D-063).
|
||||
> See `languages/go.md` for the language first-principles stub.
|
||||
|
||||
## Named Types for Domain Concepts (C1 Correctness, Data P7 Type Fidelity)
|
||||
|
||||
- **Named types for domain IDs and values:** `type UserId string`, `type OrderId string`. Two named types are distinct even with identical underlying types; the compiler rejects the swap.
|
||||
- **Constructors validate at the boundary:** `func NewUserId(s string) (UserId, error)` returns an error on bad input. A bare `UserId(s)` cast bypasses validation — only the constructor is exported.
|
||||
- **Applies `data/P7` (type fidelity):** a named type carries the domain meaning through the call graph; a `string` parameter does not.
|
||||
- **`any` is the wide type; narrow before use:** Go 1.18+ `any` is an alias for `interface{}`. Use it only at true boundaries (e.g., `json.Unmarshal`); narrow with a type assertion immediately.
|
||||
|
||||
```go
|
||||
type UserId string
|
||||
type OrderId string
|
||||
|
||||
func NewUserId(s string) (UserId, error) {
|
||||
if !regexp.MustCompile(`^[a-z0-9]+$`).MatchString(s) {
|
||||
return "", fmt.Errorf("invalid user id: %q", s)
|
||||
}
|
||||
return UserId(s), nil
|
||||
}
|
||||
|
||||
func GetUser(id UserId) (*User, error) { /* ... */ }
|
||||
|
||||
// GetUser("abc") // compile error: string is not UserId
|
||||
// GetUser(OrderId("abc")) // compile error: distinct named types
|
||||
```
|
||||
|
||||
## Generics (C6 Composability, Data P7 Type Fidelity)
|
||||
|
||||
- **Generics (1.18+) preserve element types across containers:** `type Repository[T any] struct { ... }` keeps `T` through `Get`/`Save`, rather than widening to `any`.
|
||||
- **Constrain with `comparable` for map keys, custom interfaces for behavior:** `func dedupe[T comparable](s []T) []T` uses `comparable`; a `Sortable[T]` constraint expresses the `Less` requirement.
|
||||
- **Avoid generics where an interface suffices:** `io.Reader` is not improved by generics. Generics are for type-preserving containers; interfaces are for behavior.
|
||||
- **No generic methods on generic types (not supported):** `func (r Repository[T]) Map[U any](f func(T) U) Repository[U]` is a compile error. Use a free function.
|
||||
|
||||
```go
|
||||
type Entity interface { ID() string }
|
||||
|
||||
type Repository[T Entity] struct {
|
||||
db map[string]T
|
||||
}
|
||||
|
||||
func (r *Repository[T]) Get(id string) (T, bool) {
|
||||
var zero T
|
||||
t, ok := r.db[id]
|
||||
if !ok { return zero, false }
|
||||
return t, true
|
||||
}
|
||||
|
||||
func (r *Repository[T]) Save(t T) { r.db[t.ID()] = t }
|
||||
```
|
||||
|
||||
## Interfaces (C6 Composability, API P1 Contract Fidelity)
|
||||
|
||||
- **Interfaces defined by the consumer, not the producer:** a package defines its dependencies as interfaces (`type Store interface { Get(id string) (*User, error) }`), and accepts implementations. The producer does not pre-declare "the interface I implement."
|
||||
- **Small interfaces (Go proverb):** `io.Reader` is one method. An interface with 5+ methods is a god-object; split it.
|
||||
- **Accept interfaces, return concrete types:** return a `*UserRepo`, accept a `Store`. The caller gets the implementation; the callee depends on the abstraction.
|
||||
- **Applies `api/P1` (contract fidelity):** the interface is the contract; the concrete type is the implementation. Tests mock the interface, not the struct.
|
||||
|
||||
```go
|
||||
// consumer defines the interface
|
||||
type UserStore interface {
|
||||
Get(id string) (*User, error)
|
||||
}
|
||||
|
||||
type Service struct { store UserStore }
|
||||
|
||||
func NewService(s UserStore) *Service { return &Service{store: s} }
|
||||
|
||||
// producer returns concrete; satisfies UserStore implicitly
|
||||
type UserRepo struct { db map[string]*User }
|
||||
func (r *UserRepo) Get(id string) (*User, error) { return r.db[id], nil }
|
||||
```
|
||||
|
||||
## Type Assertion Discipline (C1 Correctness, Errors P1 Errors are Data)
|
||||
|
||||
- **Type assertions return `(T, bool)` — use the bool:** `v, ok := x.(UserId)` distinguishes "wrong type" from "zero value." A bare `x.(UserId)` panics on mismatch.
|
||||
- **`switch x := x.(type)` for multi-variant narrowing:** each case narrows `x` to the case type. The default case is exhaustive (no `never`-style check; Go relies on review).
|
||||
- **Applies `errors/P1` (errors are data):** a failed type assertion is a value (`ok == false`), not an exception. Handle it as a branch, not a panic.
|
||||
- **Never assert across module boundaries silently:** an assertion on a type from another package couples to its internals. Prefer an interface method.
|
||||
|
||||
```go
|
||||
func describe(x any) string {
|
||||
switch v := x.(type) {
|
||||
case UserId:
|
||||
return "user " + string(v)
|
||||
case OrderId:
|
||||
return "order " + string(v)
|
||||
default:
|
||||
return fmt.Sprintf("unknown: %T", v)
|
||||
}
|
||||
}
|
||||
|
||||
// safe form, never panic
|
||||
id, ok := raw.(UserId)
|
||||
if !ok {
|
||||
return fmt.Errorf("expected UserId, got %T", raw)
|
||||
}
|
||||
```
|
||||
|
||||
## Error Types and errors.Is/As (Errors P1 Errors are Data, Errors P3 Fail Specifically)
|
||||
|
||||
- **Sentinel errors for known cases:** `var ErrNotFound = errors.New("not found")`; check with `errors.Is(err, ErrNotFound)`. The sentinel is a value, not an exception class.
|
||||
- **Custom error types for context:** `type ValidationError struct { Field, Msg string }`; check with `var ve *ValidationError; errors.As(err, &ve)`. The type carries structured data (Errors P4 Preserve Context).
|
||||
- **Wrap with `%w`:** `fmt.Errorf("get user %s: %w", id, err)` preserves the chain. `errors.Is`/`As` unwrap it. Bare `%v` breaks the chain.
|
||||
- **Applies `errors/P3` (fail specifically):** `ErrNotFound` is specific; `ErrFailed` is not. The error type names the failure mode.
|
||||
|
||||
```go
|
||||
var ErrNotFound = errors.New("not found")
|
||||
|
||||
type ValidationError struct {
|
||||
Field string
|
||||
Msg string
|
||||
}
|
||||
func (e *ValidationError) Error() string { return e.Field + ": " + e.Msg }
|
||||
|
||||
func GetUser(id UserId) (*User, error) {
|
||||
u, ok := db[string(id)]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("user %s: %w", id, ErrNotFound)
|
||||
}
|
||||
return u, nil
|
||||
}
|
||||
|
||||
// caller
|
||||
if errors.Is(err, ErrNotFound) { /* 404 */ }
|
||||
var ve *ValidationError
|
||||
if errors.As(err, &ve) { /* 422 with ve.Field */ }
|
||||
```
|
||||
|
||||
## Cross-References
|
||||
|
||||
- `domains/data/schema-design.md` — named types parallel schema design at the Go boundary.
|
||||
- `domains/data/first-principles.md` — Data P7 Type Fidelity is the primary trace.
|
||||
- `domains/api/rest.md` — contract fidelity for HTTP handlers using interfaces.
|
||||
- `domains/errors/patterns.md` — `errors.Is`/`As` and the wrap-with-`%w` pattern.
|
||||
- `languages/go-concurrency.md` — typed channels carry these named types.
|
||||
- `languages/go-testing.md` — table-driven tests assert type-swap safety.
|
||||
@@ -2,13 +2,6 @@
|
||||
|
||||
> How Atelier's domain principles apply in Go specifically. Derives from `domains/` docs.
|
||||
|
||||
## Derived Docs
|
||||
|
||||
- [go-types.md](go-types.md) — named types, generics, interfaces, type assertion discipline.
|
||||
- [go-tooling.md](go-tooling.md) — go vet, golangci-lint, go test -race, module discipline.
|
||||
- [go-concurrency.md](go-concurrency.md) — goroutines, channels, context, select, sync primitives.
|
||||
- [go-testing.md](go-testing.md) — table-driven tests, t.Parallel, t.Cleanup, race detector.
|
||||
|
||||
## Type System (C1 Correctness, Data P7 Type Fidelity)
|
||||
|
||||
- **Named types for domain concepts:** `type UserId string`, not bare `string`.
|
||||
|
||||
@@ -1,117 +0,0 @@
|
||||
# Python Async — Derived Application
|
||||
|
||||
> Applies Atelier's domain principles to Python async specifically.
|
||||
> Derives from `domains/` docs; introduces no new P-rules (D-063).
|
||||
> See `languages/python.md` for the language first-principles stub.
|
||||
|
||||
## asyncio and anyio (Concurrency P7 Cancellation Support, C2 Clarity)
|
||||
|
||||
- **`asyncio` for I/O-bound work; threads only for blocking libraries:** `async def` + `await` for network/disk; `run_in_executor` to wrap a blocking call. Mixing threads for I/O is the wrong default.
|
||||
- **`anyio` for runtime portability:** `anyio` abstracts asyncio/trio; a library written against `anyio` runs on either. Use it for libraries; for applications, asyncio directly is fine.
|
||||
- **One event loop, one thread:** `asyncio.run(main())` creates and runs the loop. Do not call `asyncio.run` inside an existing loop (raises `RuntimeError`); do not share a loop across threads.
|
||||
- **Applies `concurrency/P7`:** every `async def` accepts cancellation as a first-class signal; `CancelledError` propagates unless explicitly suppressed (and suppressing it is almost always a bug).
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
import anyio
|
||||
|
||||
async def fetch_user(id: str) -> User:
|
||||
return await api.get(f'/users/{id}')
|
||||
|
||||
# asyncio application
|
||||
async def main():
|
||||
user = await fetch_user('abc')
|
||||
|
||||
asyncio.run(main())
|
||||
|
||||
# anyio library — portable across asyncio/trio
|
||||
async def fetch_all(ids: list[str]) -> list[User]:
|
||||
return await anyio.gather(*(fetch_user(i) for i in ids))
|
||||
```
|
||||
|
||||
## Structured Concurrency (Concurrency P1 Immutability by Default, C6 Composability)
|
||||
|
||||
- **`asyncio.TaskGroup` (3.11+) for structured concurrency:** tasks created in a `TaskGroup` are awaited or cancelled together on exit. No orphan tasks outlive the block.
|
||||
- **No `asyncio.gather(..., return_exceptions=False)` for fallible tasks:** `gather` returns partial results on first exception; `TaskGroup` cancels siblings and propagates the error atomically. Use `TaskGroup` for new code.
|
||||
- **Applies `concurrency/P1` (immutability):** tasks share only immutable inputs; results are collected, not mutated in place. A task that writes to a shared list is a race waiting to happen.
|
||||
- **`anyio.create_task_group()` mirrors `TaskGroup` cross-runtime:** same structured-concurrency guarantee, portable.
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
|
||||
async def fetch_all(ids: list[str]) -> list[User]:
|
||||
results: list[User] = []
|
||||
async with asyncio.TaskGroup() as tg:
|
||||
tasks = [tg.create_task(fetch_user(i)) for i in ids]
|
||||
# all tasks done (or cancelled) by here
|
||||
return [t.result() for t in tasks]
|
||||
```
|
||||
|
||||
## Cancellation and Timeout (Concurrency P7 Cancellation Support, Concurrency P8 Timeout Discipline)
|
||||
|
||||
- **`asyncio.wait_for(coro, timeout)` for a deadline:** every external `await` races against a timeout. A bare `await` is an unbounded wait (Concurrency P8).
|
||||
- **`asyncio.timeout()` (3.11+) as a context manager:** `async with asyncio.timeout(5): await op` — cleaner than `wait_for` for multi-await blocks.
|
||||
- **`CancelledError` propagates; do not catch broadly:** `except Exception` swallows `CancelledError` in 3.7 (it was `BaseException`); in 3.8+ it's `BaseException` and `except Exception` skips it. Catch specifically, never bare `except:`.
|
||||
- **Applies `concurrency/P7`:** cancellation is cooperative — a long synchronous block inside `async def` ignores cancellation. Yield with `await asyncio.sleep(0)` periodically in CPU-bound loops.
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
|
||||
async def fetch_with_timeout(id: str, timeout: float = 5.0) -> User:
|
||||
async with asyncio.timeout(timeout):
|
||||
return await fetch_user(id)
|
||||
|
||||
async def shutdown(token: asyncio.Event) -> None:
|
||||
# cooperative cancel — long-running loop checks the token
|
||||
while not token.is_set():
|
||||
await do_chunk()
|
||||
await asyncio.sleep(0) # yield so cancel can land
|
||||
```
|
||||
|
||||
## Bounded Concurrency and Queues (Concurrency P9 Bounded Queues)
|
||||
|
||||
- **`asyncio.Semaphore(N)` to bound in-flight tasks:** a `Semaphore(8)` wrapping `gather` caps concurrency. Unbounded `gather` on a 10k-item list exhausts file descriptors (Concurrency P9 — bounded queues).
|
||||
- **`asyncio.Queue(maxsize=N)` for producer/consumer:** a bounded queue applies backpressure to the producer. An unbounded queue lets the producer run ahead and OOM.
|
||||
- **Applies `messaging/queues`:** an `asyncio.Queue` is an in-process broker — the same bounded-queue / backpressure semantics apply; the broker is just local.
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
|
||||
async def map_bounded(items: list[str], limit: int = 8) -> list[User]:
|
||||
sem = asyncio.Semaphore(limit)
|
||||
async def guarded(i: str) -> User:
|
||||
async with sem:
|
||||
return await fetch_user(i)
|
||||
return await asyncio.gather(*(guarded(i) for i in items))
|
||||
```
|
||||
|
||||
## Error Handling in Async (Errors P5 Recoverable When Possible, Errors P1 Errors are Data)
|
||||
|
||||
- **Retry with backoff for transient failures:** network blips are recoverable (Errors P5). Exponential backoff with jitter, capped retries, and an `anyio`-cancellation-aware `sleep`.
|
||||
- **No retry for non-idempotent operations:** a `POST` creating a resource is not safely retryable without an idempotency key (applies `api/P6` Idempotency).
|
||||
- **`except asyncio.CancelledError: raise`** is the only valid handling — re-raise so the cancellation propagates. Catching and continuing breaks structured concurrency.
|
||||
- **Applies `messaging/delivery-semantics`:** a cancelable async operation is at-most-once; retry-on-cancel is at-least-once. The caller must declare which.
|
||||
|
||||
```python
|
||||
import anyio
|
||||
import random
|
||||
|
||||
async def fetch_retry(id: str, attempts: int = 3) -> User:
|
||||
for i in range(attempts):
|
||||
try:
|
||||
return await fetch_user(id)
|
||||
except (TimeoutError, ConnectionError):
|
||||
if i == attempts - 1:
|
||||
raise
|
||||
await anyio.sleep((2 ** i) * 0.1 + random.random() * 0.1)
|
||||
raise RuntimeError('unreachable')
|
||||
```
|
||||
|
||||
## Cross-References
|
||||
|
||||
- `domains/concurrency/patterns.md` — the cancellation/timeout/semaphore patterns applied here.
|
||||
- `domains/concurrency/first-principles.md` — Concurrency P1 Immutability, P7 Cancellation Support, P8 Timeout Discipline, P9 Bounded Queues.
|
||||
- `domains/messaging/queues.md` — `asyncio.Queue` as an in-process broker; backpressure parallels (IDEATE-40).
|
||||
- `domains/errors/patterns.md` — typed async errors and retry-with-backoff.
|
||||
- `languages/py-types.md` — `Result` and exception hierarchy used in async error handling.
|
||||
- `languages/py-tooling.md` — `pytest-asyncio` config that runs these tests.
|
||||
@@ -1,111 +0,0 @@
|
||||
# Python Testing — Derived Application
|
||||
|
||||
> Applies Atelier's domain principles to Python testing specifically.
|
||||
> Derives from `domains/` docs; introduces no new P-rules (D-063).
|
||||
> See `languages/python.md` for the language first-principles stub.
|
||||
|
||||
## pytest and Spec-Driven Tests (Testing P1 Tests as Specification, C2 Clarity)
|
||||
|
||||
- **`pytest` is the default; `unittest` only for stdlib-only libraries:** `pytest` fixtures, parametrize, and assertion rewriting beat `unittest`'s boilerplate (Clarity C2).
|
||||
- **Tests co-located with source:** `user.py` → `test_user.py`. A test far from its subject rots (Documentation P5 Discoverability).
|
||||
- **Test names read as a spec:** `def test_create_user_rejects_invalid_email():` — a reader understands the unit from the name. Avoid `def test_user1():`.
|
||||
- **`assert` over `self.assertEqual`:** pytest rewrites `assert` to show the failing values; `assertEqual` is unittest's escape hatch and loses readability.
|
||||
- **Applies `Testing P1`:** the test is a specification; the failure message is the spec violation.
|
||||
|
||||
```python
|
||||
# test_user.py
|
||||
import pytest
|
||||
from user import create_user, ValidationError
|
||||
|
||||
def test_create_user_rejects_invalid_email():
|
||||
with pytest.raises(ValidationError):
|
||||
create_user(email='not-an-email')
|
||||
|
||||
def test_create_user_returns_persisted_id():
|
||||
u = create_user(email='a@b.co')
|
||||
assert u.id # truthy persisted id
|
||||
```
|
||||
|
||||
## Factories and Fixture Discipline (Testing P2 Independence, Testing P7 Realism)
|
||||
|
||||
- **`factory_boy` or `pytest-factoryboy` over shared fixtures for mutable state:** `UserFactory.build()` returns a fresh object per call; a session-scoped fixture mutated across tests couples them (Testing P2 Independence).
|
||||
- **Fixtures for setup/teardown, factories for data:** a `db` fixture sets up the DB once per test; a `make_user` factory produces fresh data per assertion. Conflating them produces order-dependent tests.
|
||||
- **`scope='function'` is the default and the safe default:** `scope='session'` for read-only resources (a schema migration), never for mutable state.
|
||||
- **Mock at the boundary, not the unit:** `mocker.patch('requests.get')` for HTTP; do not patch `user.User.save` (that mocks the unit under test — Testing P7 realism).
|
||||
|
||||
```python
|
||||
import factory
|
||||
from user import User
|
||||
|
||||
class UserFactory(factory.Factory):
|
||||
class Meta:
|
||||
model = User
|
||||
email = factory.Sequence(lambda n: f'u{n}@b.co')
|
||||
name = 'Test User'
|
||||
|
||||
def test_user_factory_is_fresh():
|
||||
u1 = UserFactory.build()
|
||||
u2 = UserFactory.build()
|
||||
assert u1.email != u2.email # independent
|
||||
```
|
||||
|
||||
## Parametrize and Edge Cases (Testing P9 Edge Case Coverage, Testing P3 Determinism)
|
||||
|
||||
- **`@pytest.mark.parametrize` for input tables:** one parametrized test runs N cases; each is an independent test with its own name and failure output (Testing P9).
|
||||
- **Edge cases as rows, not special tests:** empty list, `None`, max int, unicode — each a row. An ad-hoc `test_handles_edge` with multiple asserts hides which case failed (Testing P6 Failure Specificity).
|
||||
- **`pytest --randomly` catches order coupling:** a test passing alone but failing in a suite has hidden shared state. The random plugin makes it visible (Testing P2 Independence).
|
||||
- **Property tests via `hypothesis`:** for invariants (e.g., "parse(serialize(x)) == x"), `hypothesis` generates hundreds of inputs and shrinks failures to a minimal counterexample.
|
||||
|
||||
```python
|
||||
import pytest
|
||||
|
||||
@pytest.mark.parametrize('email, reason', [
|
||||
('', 'empty'),
|
||||
('a' * 1000 + '@b.co', 'too long'),
|
||||
('no-at-sign', 'missing @'),
|
||||
('a@b', 'missing TLD'),
|
||||
])
|
||||
def test_create_user_rejects(email, reason):
|
||||
with pytest.raises(ValidationError):
|
||||
create_user(email=email)
|
||||
```
|
||||
|
||||
## Determinism and Time (Testing P3 Determinism, Testing P9 Edge Case Coverage)
|
||||
|
||||
- **No `datetime.now()`, `time.time()`, `uuid.uuid4()`, `random.random()` in code under test:** inject a `Clock`, `UUIDGen`, `Random` port. In tests, provide deterministic fakes.
|
||||
- **`freezegun` for time:** `@freeze_time('2024-01-01')` makes `datetime.now()` deterministic. Do not call `datetime.now()` directly in code — wrap it in a `Clock` port so production and tests both inject.
|
||||
- **`pytest --randomly-seed=last` to reproduce a failing order:** when `--randomly` finds an order bug, the seed is logged; re-run with it to debug deterministically.
|
||||
|
||||
```python
|
||||
from freezegun import freeze_time
|
||||
|
||||
@freeze_time('2024-01-01')
|
||||
def test_user_has_created_at():
|
||||
u = create_user(email='a@b.co')
|
||||
assert u.created_at.year == 2024
|
||||
```
|
||||
|
||||
## Async Tests (Concurrency P10 Test for Race Conditions, Testing P1 Tests as Specification)
|
||||
|
||||
- **`pytest-asyncio` (or `anyio`'s pytest plugin) for `async def` tests:** `@pytest.mark.asyncio` runs the coroutine on a loop. Without it, an `async def` test is silently skipped (returns a coroutine, never awaited).
|
||||
- **`anyio`'s plugin runs the same test on asyncio and trio:** one parametrized run across both runtimes catches runtime-specific bugs.
|
||||
- **Race-sensitive tests use `--randomly` and bounded concurrency:** a `Semaphore(1)` test under random order surfaces hidden state.
|
||||
- **Applies `concurrency/P10`:** async tests are the race detector's first line — if a test passes alone but fails under `gather` of N, there's a race.
|
||||
|
||||
```python
|
||||
import pytest
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_async_fetch_returns_user():
|
||||
u = await fetch_user('abc')
|
||||
assert u.email
|
||||
```
|
||||
|
||||
## Cross-References
|
||||
|
||||
- `domains/testing/pyramid.md` — where unit/integration/property tests sit; hypothesis is the property layer.
|
||||
- `domains/testing/fixtures.md` — factory-vs-fixture discipline applied via `factory_boy`.
|
||||
- `domains/testing/first-principles.md` — Testing P1 Specification, P2 Independence, P3 Determinism, P9 Edge Coverage.
|
||||
- `languages/py-types.md` — the `Result` and Pydantic models that tests assert.
|
||||
- `languages/py-async.md` — async tests use the cancellation/timeout patterns from that doc.
|
||||
- `languages/py-tooling.md` — the `pyproject.toml [tool.pytest]` config that runs these tests.
|
||||
@@ -1,99 +0,0 @@
|
||||
# Python Tooling — Derived Application
|
||||
|
||||
> Applies Atelier's domain principles to Python tooling specifically.
|
||||
> Derives from `domains/` docs; introduces no new P-rules (D-063).
|
||||
> See `languages/python.md` for the language first-principles stub.
|
||||
|
||||
## ruff for Lint and Format (DevOps P2 Automation, C2 Clarity)
|
||||
|
||||
- **`ruff` replaces flake8 + black + isort + pyupgrade:** one tool, one config, one order of magnitude faster. Format is not debated in review (Clarity C2).
|
||||
- **Rule selection is principled, not "everything":** `select = ["E", "F", "I", "UP", "B", "SIM"]` — each rule group has a one-line `# reason:` in `pyproject.toml`. Rules without a rationale are noise (Documentation P1 — docs are code).
|
||||
- **`ruff format` is the formatter, `ruff check` is the linter:** run both in CI; the formatter is deterministic, the linter surfaces smells.
|
||||
- **Applies `devops/P2`:** the format/lint gate runs on every push; a developer never waits for a reviewer to comment on style.
|
||||
|
||||
```toml
|
||||
# pyproject.toml
|
||||
[tool.ruff]
|
||||
target-version = "py311"
|
||||
line-length = 100
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = ["E", "F", "I", "UP", "B", "SIM", "RUF"]
|
||||
# reason: E/F = pyflakes+pycodestyle; I = isort; UP = pyupgrade; B = bugbear; SIM = simplification
|
||||
|
||||
[tool.ruff.format]
|
||||
quote-style = "double"
|
||||
```
|
||||
|
||||
## mypy and Type-Check Gate (DevOps P2 Automation, Data P7 Type Fidelity)
|
||||
|
||||
- **`mypy --strict` in CI, not in the editor:** strict flags (`disallow_untyped_defs`, `no_implicit_optional`, `warn_return_any`) are the floor. The editor runs a relaxed mypy for speed; CI runs strict as the gate.
|
||||
- **`pyright` for stricter/async-aware checking:** pyright understands `async` better and reports faster; mypy is the standard. Pick one as the gate, run the other as informational.
|
||||
- **Per-module overrides only with a tracked reason:** `[[tool.mypy.overrides]] module = "legacy.*" ignore_errors = true` — each override block links to a ticket. Untracked overrides accumulate into a permanently untyped core.
|
||||
- **`py.typed` marker for libraries:** ships the type info to consumers. Without it, downstream mypy treats the library as `Any`.
|
||||
|
||||
```bash
|
||||
# CI gate
|
||||
mypy --strict src/
|
||||
pyright src/ || true # informational
|
||||
```
|
||||
|
||||
## Dependency Management: poetry and uv (DevOps P1 Reproducibility)
|
||||
|
||||
- **`poetry` or `uv` for lockfile discipline:** both produce a deterministic lock (`poetry.lock` / `uv.lock`). `pip install` alone does not — it resolves at install time, producing different trees across machines.
|
||||
- **`uv` for speed (Rust-based, 10–100x faster):** newer tool, same lockfile semantics. Either is acceptable; do not mix within a repo.
|
||||
- **Lockfile committed for applications:** for libraries, commit the lock for CI reproducibility even though consumers resolve their own tree.
|
||||
- **`--frozen` install in CI:** `poetry install --no-dev --frozen` fails if the lock is out of sync. Prevents a "works on my machine" drift.
|
||||
|
||||
```bash
|
||||
# CI install — deterministic
|
||||
uv sync --frozen --no-dev
|
||||
# or
|
||||
poetry install --no-dev --frozen
|
||||
```
|
||||
|
||||
## Virtualenv Discipline (DevOps P1 Reproducibility, C3 Simplicity)
|
||||
|
||||
- **One virtualenv per project, never the system Python:** `uv venv` or `python -m venv .venv`. System Python drift breaks reproducibility.
|
||||
- **`uv` creates and pins the Python version:** `uv venv --python 3.12` ensures the same interpreter across machines. A pinned Python is part of the reproducibility contract, not just the lockfile.
|
||||
- **No `pip install` into the system Python in CI:** use `uv`/`poetry`'s venv. A CI step that mutates system Python makes the next job non-hermetic.
|
||||
|
||||
```bash
|
||||
uv venv --python 3.12
|
||||
source .venv/bin/activate
|
||||
uv pip install -r requirements.txt
|
||||
```
|
||||
|
||||
## Documentation in the Pipeline (Documentation P1 Documentation is Code, DevOps P9 Documentation in the Pipeline)
|
||||
|
||||
- **`mkdocs` + `mkdocstrings` from docstrings:** API docs are generated from `google`- or `numpy`-style docstrings; the build fails on missing docstrings for public symbols (Documentation P1).
|
||||
- **`doctest` blocks in docstrings are run by pytest:** a `>>>` example is a tested artifact; a stale example fails the build (Documentation P1, Testing P1).
|
||||
- **`pyproject.toml` is the single source of tool config:** ruff, mypy, pytest, poetry all read from it. Do not scatter `.flake8`, `setup.cfg`, `mypy.ini`. One config file is one place to look (Clarity C2).
|
||||
|
||||
```python
|
||||
def get_user(id: UUID) -> User:
|
||||
"""Fetch a user by id.
|
||||
|
||||
Args:
|
||||
id: the user's UUID.
|
||||
|
||||
Returns:
|
||||
The User.
|
||||
|
||||
Raises:
|
||||
NotFoundError: if the user does not exist.
|
||||
|
||||
Example:
|
||||
>>> get_user(UUID('intentional-example-uuid'))
|
||||
User(...)
|
||||
"""
|
||||
...
|
||||
```
|
||||
|
||||
## Cross-References
|
||||
|
||||
- `domains/devops/ci-cd.md` — the pipeline gates that host ruff/mypy/poetry.
|
||||
- `domains/devops/first-principles.md` — DevOps P1 Reproducibility, P2 Automation.
|
||||
- `domains/documentation/first-principles.md` — Documentation P1 Documentation is Code.
|
||||
- `languages/py-types.md` — the type rules mypy enforces reference this doc.
|
||||
- `languages/py-testing.md` — the pytest config (`pyproject.toml [tool.pytest]`) detailed here.
|
||||
@@ -1,111 +0,0 @@
|
||||
# Python Type System — Derived Application
|
||||
|
||||
> Applies Atelier's domain principles to Python's type system specifically.
|
||||
> Derives from `domains/` docs; introduces no new P-rules (D-063).
|
||||
> See `languages/python.md` for the language first-principles stub.
|
||||
|
||||
## Type Hints and Gradual Typing (C1 Correctness, Data P7 Type Fidelity)
|
||||
|
||||
- **Type hints on every function signature:** `def get_user(id: UUID) -> User | None:`. Hints are annotations, not enforcement, but `mypy`/`pyright` make them a build gate.
|
||||
- **Gradual typing is opt-in, not opt-out:** start with `--strict` on a package, fix the errors, then expand. A repo-wide `# type: ignore` is a gradual-typing failure.
|
||||
- **`from __future__ import annotations` for forward refs:** all annotations are strings until resolved, so `class User: ...` referencing `User` works without quotes in 3.10+.
|
||||
- **`Any` disables the checker; `object` is the wide type:** `Any` allows any operation; `object` requires narrowing. Use `object` for opaque inputs (e.g., `json.loads` return).
|
||||
- **Applies `data/P7` (type fidelity):** a hint is the contract; the checker verifies it. A missing hint is a missing contract.
|
||||
|
||||
- **Pydantic for runtime validation at the boundary:** hints on `BaseModel` fields are validated at construction, catching bad input from network/config before it deepens into the system.
|
||||
- **`extra='forbid'` by default:** Pydantic allows extra fields silently; forbid them to surface schema drift (e.g., a client sending a typo'd field name).
|
||||
- **Custom types via `Annotated` with validators:** `Email = Annotated[str, validate_email]` keeps the type readable and the validator attached to the type, not the model.
|
||||
|
||||
```python
|
||||
from pydantic import BaseModel, ConfigDict
|
||||
from uuid import UUID
|
||||
|
||||
class UserCreate(BaseModel):
|
||||
model_config = ConfigDict(extra='forbid')
|
||||
email: str
|
||||
name: str
|
||||
|
||||
class User(UserCreate):
|
||||
id: UUID
|
||||
```
|
||||
|
||||
## Pydantic and Schema Fidelity (C1 Correctness, API P1 Contract Fidelity, Data P7 Type Fidelity)
|
||||
|
||||
- **Pydantic models are the API contract:** a FastAPI handler taking `UserCreate` rejects malformed JSON with a 422 before the body runs. This is `api/P1` (contract fidelity) at the type boundary.
|
||||
- **`ConfigDict(extra='forbid')` rejects unknown fields:** silently accepting extras is a contract leak — the server appears to handle fields it ignores.
|
||||
- **Validators raise `ValueError`, not `Exception`:** Pydantic converts `ValueError` to a validation error response; a generic `Exception` becomes a 500 and hides the input bug.
|
||||
- **Applies `api/P1`:** the model is the source of truth; the OpenAPI schema is generated from it, not hand-written. Drift between schema and code is impossible.
|
||||
|
||||
```python
|
||||
from typing import Annotated
|
||||
from pydantic import BaseModel, Field, StringConstraints
|
||||
|
||||
EmailStr = Annotated[str, StringConstraints(pattern=r'^[^@\s]+@[^@\s]+$')]
|
||||
|
||||
class Login(BaseModel):
|
||||
email: EmailStr
|
||||
password: Annotated[str, Field(min_length=8)]
|
||||
```
|
||||
|
||||
## Errors as Data (Errors P1 Errors are Data, C1 Correctness)
|
||||
|
||||
- **`Union[T, Error]` over `Optional[T]` for expected failures:** `Optional[User]` cannot distinguish "not found" from "permission denied". A discriminated `Result` carries the cause.
|
||||
- **Custom exception hierarchy rooted at `AppError`:** `class NotFoundError(AppError)` etc. — callers can `except AppError` for the broad case, or a specific subclass for handling.
|
||||
- **`raise` for exceptional paths, `return Result` for expected:** "user not found" is expected (a `Result`); "DB connection lost" is exceptional (a `raise`). Conflating them makes error handling a guess.
|
||||
- **Applies `errors/P1`:** errors are values, not control-flow magic. A `Result` type encodes this at the type level even where exceptions are the runtime mechanism.
|
||||
|
||||
```python
|
||||
from dataclasses import dataclass
|
||||
from typing import Generic, TypeVar, Union
|
||||
|
||||
T = TypeVar('T')
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Ok(Generic[T]):
|
||||
value: T
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Err:
|
||||
error: Exception
|
||||
|
||||
Result = Union[Ok[T], Err]
|
||||
|
||||
def find_user(id: UUID) -> Result[User]:
|
||||
row = db.get(id)
|
||||
if row is None:
|
||||
return Err(NotFoundError(f'user {id}'))
|
||||
return Ok(User.from_row(row))
|
||||
```
|
||||
|
||||
## Generics and Protocols (C6 Composability, C1 Correctness)
|
||||
|
||||
- **`Protocol` for structural typing (PEP 544):** a `Repository` protocol defines `get`/`save` without requiring an inheritance hierarchy; any class matching the shape satisfies it.
|
||||
- **`TypeVar` with bounds for generic functions:** `T = TypeVar('T', bound=Entity)` lets `serialize(t: T) -> dict` access `t.id`.
|
||||
- **`Generic[T]` for container types:** a typed `Repository[T]` preserves the element type across `get`/`save`, rather than widening to `Any`.
|
||||
- **`@overload` for callable overloads:** `def parse(s: str) -> int: ...` vs `def parse(s: bytes) -> int: ...` — the runtime body is one function; the overloads are the type contract.
|
||||
|
||||
```python
|
||||
from typing import Protocol, TypeVar
|
||||
|
||||
T = TypeVar('T')
|
||||
|
||||
class Repository(Protocol[T]):
|
||||
def get(self, id: str) -> T | None: ...
|
||||
def save(self, t: T) -> None: ...
|
||||
|
||||
class UserRepo:
|
||||
def get(self, id: str) -> User | None: ...
|
||||
def save(self, u: User) -> None: ...
|
||||
|
||||
def use_repo(r: Repository[User]) -> None:
|
||||
u = r.get('abc') # type: User | None
|
||||
```
|
||||
|
||||
## Cross-References
|
||||
|
||||
- `domains/data/schema-design.md` — Pydantic models parallel schema design at the TS/JSON boundary.
|
||||
- `domains/data/first-principles.md` — Data P7 Type Fidelity is the primary trace for this doc.
|
||||
- `domains/api/rest.md` — contract fidelity for FastAPI handlers consuming Pydantic models.
|
||||
- `domains/errors/patterns.md` — the `Result` discriminated union as error-as-data encoding.
|
||||
- `languages/py-async.md` — typed async results built on the `Result` union here.
|
||||
- `languages/py-tooling.md` — the `mypy`/`pyright` config that enforces these hints.
|
||||
@@ -2,13 +2,6 @@
|
||||
|
||||
> How Atelier's domain principles apply in Python specifically. Derives from `domains/` docs.
|
||||
|
||||
## Derived Docs
|
||||
|
||||
- [py-types.md](py-types.md) — type hints + Pydantic, mypy/pyright, gradual typing.
|
||||
- [py-tooling.md](py-tooling.md) — ruff, mypy, poetry, uv, virtualenv discipline.
|
||||
- [py-async.md](py-async.md) — asyncio, anyio, cancellation, structured concurrency.
|
||||
- [py-testing.md](py-testing.md) — pytest, factory_boy, fixture discipline, parametrize.
|
||||
|
||||
## Type System (C1 Correctness, Data P7 Type Fidelity)
|
||||
|
||||
- **Type hints on every function:** `def get_user(id: UUID) -> User | None:`.
|
||||
|
||||
@@ -1,129 +0,0 @@
|
||||
# Rust Async — Derived Application
|
||||
|
||||
> Applies Atelier's domain principles to Rust async specifically.
|
||||
> Derives from `domains/` docs; introduces no new P-rules (D-063).
|
||||
> See `languages/rust.md` for the language first-principles stub.
|
||||
|
||||
## tokio and the Async Runtime (Concurrency P5 Lock Minimization, C6 Composability)
|
||||
|
||||
- **`tokio` is the default async runtime:** `#[tokio::main]` for the entry; `tokio::spawn` for a task. The runtime owns the reactor, the I/O driver, and the timer.
|
||||
- **`tokio::spawn` returns a `JoinHandle` like `std::thread::spawn`:** a dropped `JoinHandle` detaches (the task keeps running); `await` the handle to join. Prefer await to detach.
|
||||
- **`tokio::task::JoinSet` for structured concurrency:** a set of tasks awaited together; on drop, all remaining tasks are cancelled. Mirrors `errgroup`/`TaskGroup` semantics.
|
||||
- **`runtime` features are explicit:** `tokio = { version = "1", features = ["full"] }` for a binary; `["rt", "rt-multi-thread", "macros"]` for a library. Pulling `full` into a library bloats downstream.
|
||||
|
||||
```rust
|
||||
#[tokio::main]
|
||||
async fn main() {
|
||||
let mut set = tokio::task::JoinSet::new();
|
||||
for id in ["a", "b", "c"] {
|
||||
set.spawn(fetch_user(id.to_string()));
|
||||
}
|
||||
while let Some(res) = set.join_next().await {
|
||||
match res {
|
||||
Ok(Ok(u)) => println!("{}", u.name),
|
||||
Ok(Err(e)) => eprintln!("err: {e}"),
|
||||
Err(join_err) => eprintln!("panic: {join_err}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Async Traits (Concurrency P7 Cancellation Support, C6 Composability)
|
||||
|
||||
- **`async fn` in traits stabilized in Rust 1.75:** `trait Repo { async fn get(&self, id: &str) -> Result<User, Error>; }`. No `async-trait` crate needed for new code on recent toolchains.
|
||||
- **`Box<dyn Trait>` with async methods needs `dyn`-compatibility:** the returned future is `Pin<Box<dyn Future>>`; the compiler boxes it. For hot paths, use generics (`impl Trait`) over `dyn`.
|
||||
- **`async-trait` crate for older toolchains:** macro that desugars to a `Pin<Box<dyn Future>>`. Migrate to native `async fn in trait` when the toolchain allows.
|
||||
- **`Send` bounds on async traits for cross-thread spawn:** `trait Repo: Send { async fn get(&self, id: &str) -> Result<User, Error>; }` — the returned future must be `Send` to spawn on a multi-thread runtime.
|
||||
|
||||
```rust
|
||||
trait UserRepo: Send + Sync {
|
||||
async fn get(&self, id: &str) -> Result<User, Error>;
|
||||
}
|
||||
|
||||
struct PgRepo { pool: PgPool }
|
||||
impl UserRepo for PgRepo {
|
||||
async fn get(&self, id: &str) -> Result<User, Error> {
|
||||
sqlx::query_as::<_, User>("SELECT * FROM users WHERE id = $1")
|
||||
.bind(id).fetch_one(&self.pool).await.map_err(Error::from)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Cancellation (Concurrency P7 Cancellation Support, Concurrency P8 Timeout Discipline)
|
||||
|
||||
- **Cancellation is cooperative via dropping the future:** `tokio::select!` drops the unselected branch, cancelling it. A dropped future stops at its next `.await` point.
|
||||
- **`tokio::time::timeout` for a deadline:** `timeout(Duration::from_secs(5), op).await` returns `Ok(Ok(v))` on success, `Ok(Err(e))` on inner error, `Err(Elapsed)` on timeout. Every external `await` races against a deadline (Concurrency P8).
|
||||
- **`tokio::select!` for cancel-aware waits:** `select! { res = op => res, _ = cancel => return Err(Cancelled), }`. The unselected branch is dropped, cancelling it.
|
||||
- **Cancellation is not atomic:** a future dropped mid-`await` may have partial state. `Drop` runs on cancellation; clean up there (e.g., rollback a transaction).
|
||||
- **Applies `concurrency/P7`:** cancellation is a first-class signal; the runtime propagates it via drop. No `CancelledError` to catch — the future is gone.
|
||||
|
||||
```rust
|
||||
use tokio::time::timeout;
|
||||
use std::time::Duration;
|
||||
|
||||
async fn fetch_with_timeout(url: &str) -> Result<Response, Error> {
|
||||
match timeout(Duration::from_secs(5), fetch(url)).await {
|
||||
Ok(Ok(r)) => Ok(r),
|
||||
Ok(Err(e)) => Err(e.into()),
|
||||
Err(_elapsed) => Err(Error::Timeout),
|
||||
}
|
||||
}
|
||||
|
||||
async fn cancellable(op: impl Future<Output=()>, mut cancel: tokio::sync::oneshot::Receiver<()>) {
|
||||
tokio::select! {
|
||||
_ = op => {},
|
||||
_ = &mut cancel => println!("cancelled"),
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Pin and Self-Referential Futures (Concurrency P5 Lock Minimization, C1 Correctness)
|
||||
|
||||
- **`async fn` returns a `Future` that is often self-referential:** the generated state machine may hold a borrow into its own stack. Such a future must be `Pin`ned to move safely.
|
||||
- **`Pin<Box<T>>` to box and pin:** `Box::pin(async { ... })` returns a `Pin<Box<dyn Future>>`. The cost is a heap alloc; the win is `Send`/`dyn`-compatibility.
|
||||
- **`Pin<&mut T>` for in-place polling:** `Pin::new(&mut fut)` pins a stack future; the borrow checker prevents moving it. Use for stack-allocated futures in `select!`.
|
||||
- **Do not `unsafe` unpin:** `Pin::get_unchecked_mut` opts out of the pin guarantees. Application code never needs it; library code uses it for `poll` implementations.
|
||||
|
||||
```rust
|
||||
use std::pin::Pin;
|
||||
|
||||
async fn boxed() -> Pin<Box<dyn std::future::Future<Output = ()> + Send>> {
|
||||
Box::pin(async {
|
||||
// self-referential state machine is safe to move once pinned
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
## Bounded Channels and Backpressure (Concurrency P9 Bounded Queues)
|
||||
|
||||
- **`tokio::sync::mpsc::channel(N)` is bounded:** `send().await` blocks when full (backpressure, Concurrency P9). Unbounded `unbounded_channel()` lets the producer run ahead and OOM.
|
||||
- **`tokio::sync::mpsc::Sender::try_send` for non-blocking send:** returns `Err(TrySendError::Full(v))` when full; the caller decides to drop, log, or back off. A bounded queue + `try_send` is the backpressure-aware pattern.
|
||||
- **`tokio::sync::broadcast` for fan-out:** multiple receivers each get a copy; a slow receiver misses (lag). Use for telemetry, not for commands.
|
||||
- **Applies `messaging/queues`:** a bounded tokio channel is an in-process broker — bounded buffer, backpressure, at-most-once handoff. The same semantics apply; the broker is local.
|
||||
|
||||
```rust
|
||||
use tokio::sync::mpsc;
|
||||
|
||||
async fn producer(tx: mpsc::Sender<Job>) {
|
||||
for j in jobs() {
|
||||
if tx.send(j).await.is_err() { return; } // receiver dropped
|
||||
}
|
||||
}
|
||||
|
||||
async fn consumer(rx: mpsc::Receiver<Job>) {
|
||||
while let Some(j) = rx.recv().await {
|
||||
process(j).await;
|
||||
}
|
||||
}
|
||||
|
||||
let (tx, rx) = mpsc::channel::<Job>(16); // bounded: backpressure
|
||||
```
|
||||
|
||||
## Cross-References
|
||||
|
||||
- `domains/concurrency/patterns.md` — the cancellation/timeout/semaphore patterns applied here.
|
||||
- `domains/concurrency/first-principles.md` — Concurrency P5 Lock Minimization, P7 Cancellation Support, P8 Timeout Discipline, P9 Bounded Queues.
|
||||
- `domains/messaging/delivery-semantics.md` — at-most-once vs at-least-once framing for async retry/cancel (IDEATE-40).
|
||||
- `languages/rs-ownership.md` — `Send`/`Sync` bounds on futures build on the ownership model here.
|
||||
- `languages/rs-tooling.md` — `tokio` feature flags and the `cargo` build profiles detailed there.
|
||||
- `languages/rs-testing.md` — `#[tokio::test]` and async test patterns.
|
||||
@@ -1,136 +0,0 @@
|
||||
# Rust Ownership — Derived Application
|
||||
|
||||
> Applies Atelier's domain principles to Rust's ownership model specifically. Rust's distinctive strength (Send/Sync, lifetimes, borrowing) earns a dedicated ownership doc rather than an `rs-types.md`.
|
||||
> Derives from `domains/` docs; introduces no new P-rules (D-063).
|
||||
> See `languages/rust.md` for the language first-principles stub.
|
||||
|
||||
## Ownership and Move Semantics (Concurrency P1 Immutability by Default, C1 Correctness)
|
||||
|
||||
- **Ownership is unique:** at any time, exactly one owner holds a value. Assignment passes ownership (`let y = x;` — `x` is moved, not copied). The compiler rejects use-after-move.
|
||||
- **`Copy` types (integers, `bool`, `&T`) duplicate on assignment; everything else moves.** A `struct` is `Copy` only if all fields are; opt in via `#[derive(Copy, Clone)]` only for small, cheap-to-copy types.
|
||||
- **Pass by `&T` for read-only, `&mut T` for mutation:** a borrow does not transfer ownership; the caller retains the value after the callee returns.
|
||||
- **Applies `concurrency/P1` (immutability by default):** `&T` is shared and immutable; `&mut T` is exclusive and mutable. The compiler enforces "one or many, never both" — aliasing XOR mutation, statically.
|
||||
|
||||
```rust
|
||||
let s = String::from("hello");
|
||||
let t = s; // s moved into t
|
||||
// println!("{}", s); // error: use of moved value
|
||||
|
||||
let n = 5;
|
||||
let m = n; // i32 is Copy: n still usable
|
||||
println!("{} {}", n, m);
|
||||
```
|
||||
|
||||
## Borrowing and Lifetimes (C1 Correctness, Data P7 Type Fidelity, Concurrency P3 Boundaries are Locks)
|
||||
|
||||
- **`&'a T` ties a borrow to a lifetime `'a`:** the borrow cannot outlive the owner. Lifetimes are static — the compiler rejects dangling references.
|
||||
- **Lifetime elision when unambiguous:** `fn first<'a>(s: &'a str) -> &'a str` is elided to `fn first(s: &str) -> &str` (one input → output lifetime). When ambiguous, name the lifetime.
|
||||
- **`'static` is the longest lifetime (the whole program):** not "until I drop it." Use `'static` only for values that genuinely live forever (string literals, `const`s); leaking to `'static` to satisfy the checker is a bug.
|
||||
- **`Ref<'a, T>` and `RefMut<'a, T>` from `RefCell` are runtime-checked borrows:** the borrow rules still apply, checked at runtime instead of compile time. A second `RefMut` panics.
|
||||
- **Applies `concurrency/P3` (boundaries are locks):** `&mut T` is the compile-time lock — exclusive access is the boundary; no runtime mutex needed for single-threaded aliasing discipline.
|
||||
|
||||
```rust
|
||||
fn longest<'a>(a: &'a str, b: &'a str) -> &'a str {
|
||||
if a.len() > b.len() { a } else { b } // borrow tied to both inputs
|
||||
}
|
||||
|
||||
fn dangling() -> &str { // compile error: missing lifetime
|
||||
let s = String::from("local");
|
||||
&s // error: s drops at end of fn
|
||||
}
|
||||
```
|
||||
|
||||
## Send and Sync (Concurrency P1 Immutability by Default, Concurrency P3 Boundaries are Locks, C1 Correctness)
|
||||
|
||||
- **`Send`:** a type `T: Send` may be moved across thread boundaries. Most types are `Send`; `Rc<T>` is not (shared non-atomically refcounted).
|
||||
- **`Sync`:** a type `T: Sync` may be shared (`&T`) across threads. `RefCell<T>` is `!Sync` (interior mutability without atomics); `Mutex<T>` is `Sync` (it synchronizes).
|
||||
- **The compiler enforces `Send`/`Sync` at the thread-spawn boundary:** `std::thread::spawn(move || { ... })` requires the closure's captures to be `Send`.
|
||||
- **Applies `concurrency/P1` and `concurrency/P3`:** `Send` is the move-across-boundary contract; `Sync` is the share-across-boundary contract. Data races are a compile error, not a runtime detector. This is Rust's distinctive strength over Go's race detector.
|
||||
|
||||
```rust
|
||||
use std::rc::Rc;
|
||||
use std::sync::Arc;
|
||||
|
||||
let rc = Rc::new(5);
|
||||
// std::thread::spawn(move || { println!("{}", rc) }); // error: Rc is !Send
|
||||
|
||||
let arc = Arc::new(5);
|
||||
std::thread::spawn(move || { println!("{}", arc) }); // ok: Arc<T> is Send+Sync
|
||||
```
|
||||
|
||||
## Shared Mutation: Arc, Mutex, RwLock (Concurrency P3 Boundaries are Locks, Concurrency P5 Lock Minimization)
|
||||
|
||||
- **`Arc<T>` for shared ownership across threads:** atomic refcounted. Clone increases the count; the last drop frees `T`.
|
||||
- **`Mutex<T>` for exclusive mutation across threads:** `lock()` blocks until exclusive; the guard `MutexGuard<T>` derefs to `&mut T` and releases on drop.
|
||||
- **`RwLock<T>` for read-heavy, `Mutex<T>` for write-heavy:** RwLock allows multiple readers or one writer. For most cases, `Mutex` is simpler and faster; prefer it unless reads dominate by 10x+.
|
||||
- **Hold the lock for the smallest scope:** `let g = m.lock().unwrap();` then drop `g` before I/O. RAII releases on scope exit; explicit `drop(g)` clarifies intent.
|
||||
- **Applies `concurrency/P5` (lock minimization):** prefer message passing (`mpsc` channels) over locks. When a lock is needed, scope it minimally.
|
||||
|
||||
```rust
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::thread;
|
||||
|
||||
let counter = Arc::new(Mutex::new(0));
|
||||
let mut handles = vec![];
|
||||
for _ in 0..10 {
|
||||
let c = Arc::clone(&counter);
|
||||
handles.push(thread::spawn(move || {
|
||||
let mut g = c.lock().unwrap();
|
||||
*g += 1;
|
||||
// g drops here, lock released
|
||||
}));
|
||||
}
|
||||
for h in handles { h.join().unwrap(); }
|
||||
println!("{}", *counter.lock().unwrap());
|
||||
```
|
||||
|
||||
## Interior Mutability (Concurrency P1 Immutability by Default, C1 Correctness)
|
||||
|
||||
- **`Cell<T>` for `Copy` types, `RefCell<T>` for non-`Copy`:** interior mutability moves the borrow check from compile time to runtime. `RefCell::borrow_mut()` panics on a second mutable borrow.
|
||||
- **`Mutex<T>`/`RwLock<T>` for thread-safe interior mutability:** the runtime check is the lock, not a panic. Use these across threads; `RefCell` only single-threaded.
|
||||
- **`UnsafeCell<T>` is the primitive; never use directly:** `Cell`, `RefCell`, `Mutex` are safe wrappers. Direct `UnsafeCell` is `unsafe` and opts out of the aliasing guarantee.
|
||||
- **Applies `concurrency/P1`:** interior mutability is the exception, not the default. Reach for it when an API must present `&self` while mutating internally (e.g., a cache); document why.
|
||||
|
||||
```rust
|
||||
use std::cell::RefCell;
|
||||
|
||||
struct Cache {
|
||||
inner: RefCell<HashMap<String, User>>,
|
||||
}
|
||||
impl Cache {
|
||||
fn get(&self, id: &str) -> Option<User> {
|
||||
// &self (immutable) but mutates internally
|
||||
self.inner.borrow_mut().entry(id.to_string()).or_insert_with(|| fetch()).clone()
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Drop and RAII (C1 Correctness, Concurrency P3 Boundaries are Locks)
|
||||
|
||||
- **`Drop` runs when the owner goes out of scope:** no `defer`, no `finally`. A `MutexGuard` releases, a `File` closes, a `JoinHandle`... does not join (a dropped `JoinHandle` detaches).
|
||||
- **`Drop` is deterministic:** it runs at scope exit, not GC time. This is why `Arc`'s refcount is precise and `Mutex` release is timely.
|
||||
- **`ManuallyDrop<T>` to opt out:** for FFI types whose destructor you must call manually. Rare in application code; common in `unsafe` bindings.
|
||||
- **`Drop` order: fields in declaration order, then the struct itself.** A field that another field's `Drop` depends on must be declared last.
|
||||
|
||||
```rust
|
||||
struct Resource { name: String }
|
||||
impl Drop for Resource {
|
||||
fn drop(&mut self) {
|
||||
println!("dropping {}", self.name); // runs at scope end
|
||||
}
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let _r = Resource { name: "x".into() };
|
||||
// _r drops here, prints "dropping x"
|
||||
}
|
||||
```
|
||||
|
||||
## Cross-References
|
||||
|
||||
- `domains/concurrency/first-principles.md` — Concurrency P1 Immutability, P3 Boundaries are Locks, P5 Lock Minimization.
|
||||
- `domains/data/first-principles.md` — Data P7 Type Fidelity (lifetimes are the type-level fidelity for references).
|
||||
- `domains/concurrency/patterns.md` — message-passing vs lock patterns applied via `Arc`/`Mutex`/`mpsc`.
|
||||
- `domains/errors/patterns.md` — `?` propagation relies on ownership transfer of the error.
|
||||
- `languages/rs-async.md` — async borrows (`Pin`/`&mut`) build on the lifetime model here.
|
||||
- `languages/rs-testing.md` — `Send`/`Sync` tests and ownership-based property tests.
|
||||
@@ -1,159 +0,0 @@
|
||||
# Rust Testing — Derived Application
|
||||
|
||||
> Applies Atelier's domain principles to Rust testing specifically.
|
||||
> Derives from `domains/` docs; introduces no new P-rules (D-063).
|
||||
> See `languages/rust.md` for the language first-principles stub.
|
||||
|
||||
## #[test] and Co-located Tests (Testing P1 Tests as Specification, C2 Clarity)
|
||||
|
||||
- **`#[test]` on functions in a `#[cfg(test)] mod tests` block:** tests co-located with source, compiled only in `cargo test`. A test file far from its subject rots (Documentation P5 Discoverability).
|
||||
- **Test names read as a spec:** `fn create_user_rejects_invalid_email()` — a reader understands the unit from the name. Avoid `fn test_user_1()`.
|
||||
- **`assert!` / `assert_eq!` / `assert_ne!` over raw `panic!`:** the macros produce readable failure output (`assertion failed: left == right, left: 5, right: 3`). Raw `panic!` gives a message only (Testing P6 Failure Specificity).
|
||||
- **Applies `Testing P1`:** the test is a specification; the failure message is the spec violation.
|
||||
|
||||
```rust
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn create_user_rejects_invalid_email() {
|
||||
let r = create_user("not-an-email");
|
||||
assert!(matches!(r, Err(Error::Validation(_))));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn create_user_returns_persisted_id() {
|
||||
let u = create_user("a@b.co").unwrap();
|
||||
assert!(!u.id.is_empty());
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## proptest and Property Tests (Testing P9 Edge Case Coverage, Testing P1 Tests as Specification)
|
||||
|
||||
- **`proptest` (or `quickcheck`) for invariant tests:** declare a property (`parse(serialize(x)) == x`), the framework generates hundreds of inputs and shrinks failures to a minimal counterexample (Testing P9).
|
||||
- **Strategy over hand-written generators:** `proptest::collection::vec(any::<u32>(), 0..100)` generates arbitrary `Vec<u32>`; do not hand-roll a generator for each property.
|
||||
- **`proptest!` macro or `proptest! { ... }` block:** each `case (name) => { ... }` is a property. The block is the spec (Testing P1).
|
||||
- **Property tests complement, not replace, example tests:** examples document the happy path; properties cover the edge space. Both are required.
|
||||
|
||||
```rust
|
||||
use proptest::prelude::*;
|
||||
|
||||
proptest! {
|
||||
#[test]
|
||||
fn roundtrips_id(s in "[a-z0-9]{1,32}") {
|
||||
let id = UserId::new(&s).unwrap();
|
||||
assert_eq!(id.as_str(), s);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_invalid_id(s in "[^a-z0-9]+") {
|
||||
assert!(UserId::new(&s).is_err());
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Mock Discipline (Testing P2 Independence, Testing P7 Realism)
|
||||
|
||||
- **Mock at the trait, not the struct:** `trait Store { fn get(&self, id: &str) -> Result<User, Error>; }` in production; `#[automock] trait Store` (via `mockall`) in test. The trait is the contract.
|
||||
- **`mockall` for generated mocks:** `#[automock] trait Repo {}` generates `MockRepo` with `expect_*` methods. Each expectation is per-test; no shared mock state (Testing P2 Independence).
|
||||
- **Mock the boundary, not the unit:** mock `Repo`, not `UserService` (the unit). Mocking the unit under test tests the mock (Testing P7 realism).
|
||||
- **No `#[cfg(test)]` on production code paths to inject mocks:** instead, accept the trait as a generic or `dyn` parameter. Test-only branches in production code are dead code in prod.
|
||||
|
||||
```rust
|
||||
use mockall::*;
|
||||
|
||||
#[automock]
|
||||
trait UserRepo {
|
||||
fn get(&self, id: &str) -> Result<User, Error>;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn get_user_returns_not_found() {
|
||||
let mut repo = MockUserRepo::new();
|
||||
repo.expect_get()
|
||||
.with(eq("abc"))
|
||||
.returning(|_| Err(Error::NotFound));
|
||||
let svc = UserService::new(Box::new(repo));
|
||||
assert!(matches!(svc.get_user("abc"), Err(Error::NotFound)));
|
||||
}
|
||||
```
|
||||
|
||||
## Async Tests (Concurrency P10 Test for Race Conditions, Testing P1 Tests as Specification)
|
||||
|
||||
- **`#[tokio::test]` for `async fn` tests:** runs the coroutine on a tokio runtime. Without it, an `async fn` test returns a future, never awaited (silently passes).
|
||||
- **`#[tokio::test(flavor = "multi_thread")]` for concurrency-sensitive tests:** multi-thread runtime surfaces races that single-thread misses (Concurrency P10).
|
||||
- **`tokio::time::pause()` and `advance()` for time:** freeze and advance the runtime clock deterministically. No `tokio::time::sleep(real)` in tests.
|
||||
- **Race-sensitive tests use `loom` for model-checking:** `loom` simulates all thread interleavings; it catches races `-race`-style detectors miss. Use for lock-free data structures.
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn async_fetch_returns_user() {
|
||||
let u = fetch_user("abc").await.unwrap();
|
||||
assert!(!u.name.is_empty());
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
async fn concurrent_cache_is_safe() {
|
||||
let c = Arc::new(Cache::new());
|
||||
let mut h = vec![];
|
||||
for i in 0..10 {
|
||||
let c = c.clone();
|
||||
h.push(tokio::spawn(async move { c.get(&i.to_string()).await; }));
|
||||
}
|
||||
for x in h { x.await.unwrap(); }
|
||||
}
|
||||
```
|
||||
|
||||
## Determinism and Time (Testing P3 Determinism, Testing P9 Edge Case Coverage)
|
||||
|
||||
- **No `SystemTime::now()` or `Instant::now()` in code under test:** inject a `Clock` trait. In tests, a fake clock advances deterministically.
|
||||
- **`tokio::time::pause()` for async time:** freezes the runtime clock; `tokio::time::advance(dur)` moves it. A `sleep(5s)` in test resolves instantly.
|
||||
- **`--test-threads=1` to reproduce order coupling:** by default, `cargo test` runs tests in parallel; a test that passes alone but fails in a suite has hidden shared state. `-1` reproduces.
|
||||
|
||||
```rust
|
||||
trait Clock { fn now(&self) -> std::time::Instant; }
|
||||
|
||||
struct FakeClock(std::time::Instant);
|
||||
impl Clock for FakeClock {
|
||||
fn now(&self) -> std::time::Instant { self.0 }
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn user_has_created_at() {
|
||||
let clk = FakeClock(std::time::Instant::now());
|
||||
let u = create_user_with_clock("a@b.co", &clk).unwrap();
|
||||
assert_eq!(u.created_at, clk.now());
|
||||
}
|
||||
```
|
||||
|
||||
## Doc Tests (Documentation P1 Documentation is Code, Testing P1 Tests as Specification)
|
||||
|
||||
- **`cargo test --doc` runs `///` fenced blocks:** a `///` example with `#`-hidden setup is a tested artifact; a stale output fails the build (Documentation P1).
|
||||
- **`no_run` for examples that should compile but not run:** ```` ```rust,no_run ```` — type-checks the example without executing. Use for examples that need a DB.
|
||||
- **`ignore` for examples that should not compile-check:** ```` ```rust,ignore ```` — skips entirely. Rare; prefer `no_run`.
|
||||
- **Applies `Testing P1`:** the doc example is the spec; the doc test is the spec's regression test.
|
||||
|
||||
```rust
|
||||
/// Fetch a user by id.
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```
|
||||
/// # use mycrate::{get_user, Error};
|
||||
/// let u = get_user("abc").unwrap();
|
||||
/// assert!(!u.name.is_empty());
|
||||
/// ```
|
||||
pub fn get_user(id: &str) -> Result<User, Error> { /* ... */ }
|
||||
```
|
||||
|
||||
## Cross-References
|
||||
|
||||
- `domains/testing/pyramid.md` — where unit/property/doc tests sit; proptest is the property layer.
|
||||
- `domains/testing/fixtures.md` — `t.Cleanup`-equivalent (`Drop` in tests) as fixture discipline.
|
||||
- `domains/testing/first-principles.md` — Testing P1 Specification, P2 Independence, P3 Determinism, P9 Edge Coverage.
|
||||
- `domains/concurrency/first-principles.md` — Concurrency P10 (test for races), `loom` model-checking.
|
||||
- `languages/rs-ownership.md` — `Send`/`Sync` tests and ownership-based property tests.
|
||||
- `languages/rs-async.md` — `#[tokio::test]` patterns from that doc.
|
||||
- `languages/rs-tooling.md` — `cargo test` flags (`--doc`, `--test-threads`) detailed here.
|
||||
@@ -1,95 +0,0 @@
|
||||
# Rust Tooling — Derived Application
|
||||
|
||||
> Applies Atelier's domain principles to Rust tooling specifically.
|
||||
> Derives from `domains/` docs; introduces no new P-rules (D-063).
|
||||
> See `languages/rust.md` for the language first-principles stub.
|
||||
|
||||
## cargo and Build Discipline (DevOps P2 Automation, DevOps P1 Reproducibility)
|
||||
|
||||
- **`cargo build` for dev, `cargo build --release` for release:** release enables optimizations (LTO, codegen-units=1). The default profile is for fast iteration, not perf.
|
||||
- **`Cargo.lock` committed for applications and CI:** for libraries, commit the lock for CI reproducibility even though consumers resolve their own tree. A drifted lock breaks reproducibility (DevOps P1).
|
||||
- **`cargo update` periodically, with a CI check:** `cargo update` bumps patch versions in the lock; a CI job that fails on lock drift catches a forgotten `cargo update`.
|
||||
- **`cargo vendor` for hermetic CI:** vendors `vendor/` into the repo; CI builds without network. The trade-off is repo size; the win is reproducibility.
|
||||
|
||||
```toml
|
||||
# Cargo.toml — profile discipline
|
||||
[profile.release]
|
||||
lto = true
|
||||
codegen-units = 1
|
||||
panic = "abort" # smaller binary, no unwinding
|
||||
```
|
||||
|
||||
## clippy (DevOps P2 Automation, C2 Clarity)
|
||||
|
||||
- **`cargo clippy` is the lint layer over `rustc`:** it catches `clone()` where a borrow would do, `unwrap()` in library code, and needless `Box`. Run on every build.
|
||||
- **`cargo clippy -- -D warnings` in CI:** warnings are errors. A clippy warning is a smell; accumulating them erodes the signal (Clarity C2).
|
||||
- **Per-lint allow only with a tracked reason:** `#[allow(clippy::needless_collect)] // reason: GH-123 — collect needed for len` — each allow links to a ticket. Untracked allows accumulate into a permanently lint-bypassed core.
|
||||
- **`cargo clippy --fix` for safe auto-fixes:** applies the linter's suggested change. Review the diff; do not run blindly on a large commit.
|
||||
|
||||
```bash
|
||||
# CI gate
|
||||
cargo clippy --all-targets --all-features -- -D warnings
|
||||
```
|
||||
|
||||
## cargo fmt (DevOps P2 Automation, C2 Clarity)
|
||||
|
||||
- **`cargo fmt` is the formatter; format is not debated in review:** run in CI as a check (`cargo fmt --check`), not a fix. A failing check blocks the PR.
|
||||
- **`rustfmt.toml` for repo-wide settings:** if the defaults are wrong for the repo, override once and stop. Do not relitigate per-PR.
|
||||
- **Applies `devops/P2`:** the format gate is automated; a reviewer never comments on style.
|
||||
|
||||
```bash
|
||||
# CI gate — fail if unformatted
|
||||
cargo fmt --check
|
||||
```
|
||||
|
||||
## Edition Discipline (DevOps P1 Reproducibility, C5 Reversibility)
|
||||
|
||||
- **`edition` in `Cargo.toml` pins the language edition:** 2015, 2018, 2021, 2024. An edition is a coherent set of language changes; bumping it is a deliberate migration.
|
||||
- **Edition is not the compiler version:** `rustc 1.75` supports edition 2021; edition 2024 needs a newer `rustc`. Pin the toolchain with `rust-toolchain.toml`.
|
||||
- **Bump editions deliberately, not opportunistically:** `cargo fix --edition` applies the migration lint; review the diff. A bump mid-feature conflates two changes.
|
||||
- **Applies `devops/P1` and `C5` (reversibility):** pinning the edition and toolchain makes the build reproducible; bumping is a controlled, reversible change.
|
||||
|
||||
```toml
|
||||
# Cargo.toml
|
||||
[package]
|
||||
edition = "2021"
|
||||
rust-version = "1.75"
|
||||
```
|
||||
|
||||
```toml
|
||||
# rust-toolchain.toml
|
||||
[toolchain]
|
||||
channel = "1.75"
|
||||
components = ["clippy", "rustfmt"]
|
||||
```
|
||||
|
||||
## Documentation in the Pipeline (Documentation P1 Documentation is Code, DevOps P9 Documentation in the Pipeline)
|
||||
|
||||
- **`cargo doc` from doc comments:** `///` on items generates API docs; `cargo doc --open` previews. The build fails on broken intra-doc links (`#![warn(rustdoc::broken_intra_doc_links)]`).
|
||||
- **Doc tests are run by `cargo test`:** a `///` fenced block with `#`-hidden setup is a tested artifact; a stale example fails `cargo test --doc` (Documentation P1).
|
||||
- **`#![warn(missing_docs)]` for libraries:** public items without doc comments fail the build. Documentation is a build gate, not an afterthought.
|
||||
- **`cargo readme` or `cargo docs-rs` for landing pages:** the crate's `README.md` is rendered on docs.rs; keep it in sync with `lib.rs`'s top-level doc.
|
||||
|
||||
```rust
|
||||
#![warn(missing_docs, rustdoc::broken_intra_doc_links)]
|
||||
|
||||
/// Fetch a user by id.
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```
|
||||
/// # use mycrate::get_user;
|
||||
/// let u = get_user("abc").unwrap();
|
||||
/// println!("{}", u.name);
|
||||
/// ```
|
||||
pub fn get_user(id: &str) -> Result<User, Error> { /* ... */ }
|
||||
```
|
||||
|
||||
## Cross-References
|
||||
|
||||
- `domains/devops/ci-cd.md` — the pipeline gates that host clippy/fmt/test.
|
||||
- `domains/devops/first-principles.md` — DevOps P1 Reproducibility, P2 Automation.
|
||||
- `domains/documentation/first-principles.md` — Documentation P1 Documentation is Code.
|
||||
- `languages/rs-ownership.md` — `Send`/`Sync` clippy lints reference this doc.
|
||||
- `languages/rs-async.md` — async-runtime tooling (`tokio` features) detailed here.
|
||||
- `languages/rs-testing.md` — `cargo test` flags (`--doc`, `--no-run`) detailed here.
|
||||
@@ -2,13 +2,6 @@
|
||||
|
||||
> How Atelier's domain principles apply in Rust specifically. Derives from `domains/` docs.
|
||||
|
||||
## Derived Docs
|
||||
|
||||
- [rs-ownership.md](rs-ownership.md) — Send/Sync, lifetimes, borrowing, ownership transfer.
|
||||
- [rs-tooling.md](rs-tooling.md) — cargo, clippy, fmt, edition discipline.
|
||||
- [rs-async.md](rs-async.md) — tokio, async traits, cancellation, pin.
|
||||
- [rs-testing.md](rs-testing.md) — #[test], proptest, property testing, mock discipline.
|
||||
|
||||
## Type System (C1 Correctness, Data P7 Type Fidelity)
|
||||
|
||||
- **Newtypes for domain concepts:** `struct UserId(String);` — zero-cost, type-safe.
|
||||
|
||||
@@ -1,115 +0,0 @@
|
||||
# TypeScript Async — Derived Application
|
||||
|
||||
> Applies Atelier's domain principles to TypeScript async specifically.
|
||||
> Derives from `domains/` docs; introduces no new P-rules (D-063).
|
||||
> See `languages/typescript.md` for the language first-principles stub.
|
||||
|
||||
## Promises and AbortSignal (Concurrency P7 Cancellation Support, C1 Correctness)
|
||||
|
||||
- **Every async function accepts an optional `AbortSignal`:** cancellation is a first-class parameter, not a side channel. The signal propagates to `fetch`, `setTimeout`, and downstream awaits.
|
||||
- **`AbortController` is the producer side; `AbortSignal` is the consumer side:** a function takes a `signal` (read-only), the caller owns the `controller` and decides when to abort.
|
||||
- **Abort propagates as a rejected `Promise`:** `fetch` rejects with `AbortError`; downstream code sees the rejection, not a silent no-op. This preserves `errors/P5` (recoverable when possible) — the caller can distinguish cancellation from a real failure.
|
||||
- **Applies `concurrency/P7`:** no async operation runs without a path to cancel it. A long-running `await` with no signal is a hung request.
|
||||
- **Never swallow `AbortError`:** re-throw or handle distinctly; cancellation is the caller's intent, not an error to log.
|
||||
|
||||
```typescript
|
||||
async function fetchUser(id: UserId, signal?: AbortSignal): Promise<User> {
|
||||
const ctrl = new AbortController();
|
||||
signal?.addEventListener('abort', () => ctrl.abort());
|
||||
const res = await fetch(`/users/${id}`, { signal: ctrl.signal });
|
||||
if (!res.ok) throw new HttpError(res.status);
|
||||
return res.json() as Promise<User>;
|
||||
}
|
||||
|
||||
// caller controls cancellation
|
||||
const ctrl = new AbortController();
|
||||
const timer = setTimeout(() => ctrl.abort(), 5000);
|
||||
try {
|
||||
const u = await fetchUser(id, ctrl.signal);
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
```
|
||||
|
||||
## async/await Discipline (Concurrency P8 Timeout Discipline, C2 Clarity)
|
||||
|
||||
- **`await` is the only async primitive in application code:** no `.then` chains, no callback pyramids. `async`/`await` reads top-to-bottom (Clarity C2).
|
||||
- **Never `await` in a hot loop without batching:** sequential `await` in a `for` loop is O(n) latency. Use `Promise.all` for parallelism; `for await...of` only for genuine streams.
|
||||
- **`Promise.race` for a timeout:** every external `await` has a deadline. `Promise.race([op, timeout])` rejects when the deadline passes.
|
||||
- **`return` vs `return await`:** inside `try`/`finally`, `return await` runs the `finally`; bare `return` of a Promise defers the `finally` to the microtask. Prefer `return await` when cleanup must run.
|
||||
- **Applies `concurrency/P8`:** a bare `await` with no timeout is an unbounded wait. External calls (network, disk) always race against a deadline.
|
||||
|
||||
```typescript
|
||||
async function fetchWithTimeout(url: string, ms = 5000, signal?: AbortSignal): Promise<Response> {
|
||||
const ctrl = new AbortController();
|
||||
signal?.addEventListener('abort', () => ctrl.abort());
|
||||
const timer = new Promise<never>((_, reject) =>
|
||||
setTimeout(() => reject(new TimeoutError(ms)), ms)
|
||||
);
|
||||
try {
|
||||
return await Promise.race([fetch(url, { signal: ctrl.signal }), timer]);
|
||||
} finally {
|
||||
clearTimeout(timer); // cleanup runs on success and on race-loss
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Error Handling in Async (Errors P5 Recoverable When Possible, Errors P1 Errors are Data)
|
||||
|
||||
- **Catch `unknown`, narrow with a type guard:** `catch (e: unknown)` — TS does not infer the error type. `instanceof` or a discriminator narrows it.
|
||||
- **Retry with backoff for transient failures:** network blips are recoverable (Errors P5). Exponential backoff with jitter, capped retry count, and an `AbortSignal`-aware `setTimeout`.
|
||||
- **No retry for non-idempotent operations:** a `POST` that creates a resource is not safely retryable without an idempotency key (applies `api/P6` Idempotency).
|
||||
- **Typed errors over `Error` subclasses:** a discriminated union `AppError = Network | Timeout | Cancelled` carries context (Errors P4 Preserve Context) without `instanceof` chains.
|
||||
|
||||
```typescript
|
||||
async function fetchRetry(url: string, attempts = 3, signal?: AbortSignal): Promise<Response> {
|
||||
for (let i = 0; i < attempts; i++) {
|
||||
try {
|
||||
return await fetchWithTimeout(url, 5000, signal);
|
||||
} catch (e: unknown) {
|
||||
if (e instanceof AbortError) throw e; // do not retry cancellation
|
||||
if (e instanceof TimeoutError && i < attempts - 1) {
|
||||
await sleep(jitter(i), signal); // backoff before retry
|
||||
continue;
|
||||
}
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
throw new Error('unreachable');
|
||||
}
|
||||
```
|
||||
|
||||
## Cancellation Propagation (Concurrency P7 Cancellation Support, Concurrency P9 Bounded Queues)
|
||||
|
||||
- **One signal, many consumers:** pass the same `AbortSignal` to every async call in a request. Aborting once cancels the whole tree.
|
||||
- **Bounded concurrency with a semaphore:** a `Semaphore(N)` wrapping `Promise.all` caps in-flight requests (Concurrency P9 — bounded queues). Unbounded `Promise.all` on a 10k-item array exhausts file descriptors.
|
||||
- **Cancellation is cooperative, not preemptive:** a long synchronous block inside an `async` function ignores the signal. Yield with `await Promise.resolve()` periodically in CPU-bound loops, or move to a worker.
|
||||
- **Applies `messaging/delivery-semantics`:** a cancelable async operation is an at-most-once delivery — the caller may stop listening, the result may or may not arrive. Retry-on-cancel is at-least-once; the caller must declare which.
|
||||
|
||||
```typescript
|
||||
async function mapBounded<T, U>(items: readonly T[], fn: (t: T, s: AbortSignal) => Promise<U>, limit = 8, signal?: AbortSignal): Promise<U[]> {
|
||||
const ctrl = new AbortController();
|
||||
signal?.addEventListener('abort', () => ctrl.abort());
|
||||
const results: U[] = new Array(items.length);
|
||||
let next = 0;
|
||||
const workers = Array.from({ length: limit }, async () => {
|
||||
while (true) {
|
||||
const i = next++;
|
||||
if (i >= items.length) break;
|
||||
if (ctrl.signal.aborted) throw new AbortError();
|
||||
results[i] = await fn(items[i], ctrl.signal);
|
||||
}
|
||||
});
|
||||
await Promise.all(workers);
|
||||
return results;
|
||||
}
|
||||
```
|
||||
|
||||
## Cross-References
|
||||
|
||||
- `domains/concurrency/patterns.md` — the cancellation/timeout/semaphore patterns applied here.
|
||||
- `domains/concurrency/first-principles.md` — Concurrency P7 Cancellation Support, P8 Timeout Discipline, P9 Bounded Queues.
|
||||
- `domains/messaging/delivery-semantics.md` — at-most-once vs at-least-once framing for async retry/cancel (IDEATE-40).
|
||||
- `domains/errors/patterns.md` — typed async errors and retry-with-backoff.
|
||||
- `languages/ts-types.md` — `Result<T, E>` and discriminated `AppError` used in async error handling.
|
||||
- `languages/ts-tooling.md` — `no-floating-promises` lint rule that enforces these awaits.
|
||||
@@ -1,117 +0,0 @@
|
||||
# TypeScript Testing — Derived Application
|
||||
|
||||
> Applies Atelier's domain principles to TypeScript testing specifically.
|
||||
> Derives from `domains/` docs; introduces no new P-rules (D-063).
|
||||
> See `languages/typescript.md` for the language first-principles stub.
|
||||
|
||||
## Vitest and Jest (Testing P1 Tests as Specification, C2 Clarity)
|
||||
|
||||
- **Vitest for new TS projects; Jest for legacy:** Vitest shares `vite`'s transform pipeline (no separate `ts-jest` config); Jest's ecosystem is broader. Either is acceptable — pick one per repo, do not mix.
|
||||
- **Tests co-located with source:** `user.ts` → `user.test.ts`. A test file far from its subject rots (Documentation P5 Discoverability).
|
||||
- **`describe`/`it` mirror the public API:** the test block names read as a specification ("User", "rejects an invalid email", "returns the persisted id"). A reader should understand the unit from test names alone (Testing P1).
|
||||
- **`expect` over `assert`:** Vitest/Jest matchers produce readable failure output (`expect(x).toBe(y)` → "expected 5, received 3"). Raw `assert` gives a stack trace and nothing else (Testing P6 Failure Specificity).
|
||||
|
||||
```typescript
|
||||
// user.test.ts
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { createUser } from './user';
|
||||
|
||||
describe('createUser', () => {
|
||||
it('rejects an invalid email', async () => {
|
||||
await expect(createUser({ email: 'not-an-email' })).rejects.toThrow(ValidationError);
|
||||
});
|
||||
it('returns the persisted id', async () => {
|
||||
const u = await createUser({ email: 'a@b.co' });
|
||||
expect(u.id).toMatch(/^[a-z0-9]+$/);
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
## Mock Discipline (Testing P2 Independence, Testing P7 Realism)
|
||||
|
||||
- **Mock at the boundary, not the unit:** replace `fetch` or the DB client, not the function under test. Mocking the unit under test tests the mock, not the code (Testing P7 — realism).
|
||||
- **No partial mocks of the system under test:** if a method must be stubbed, the unit is too large. Extract a collaborator and mock that.
|
||||
- **Each test sets up and tears down its own state:** no shared mutable fixtures. A `beforeEach`/`afterEach` resets; a top-level `let` shared across tests is order-coupling (Testing P2 Independence).
|
||||
- **`vi.useFakeTimers()` for time-dependent code:** never call `Date.now()` directly in code under test; inject a `Clock` port. In tests, fake timers make `setTimeout` synchronous.
|
||||
|
||||
```typescript
|
||||
import { vi, beforeEach, afterEach } from 'vitest';
|
||||
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers();
|
||||
global.fetch = vi.fn(); // boundary mock
|
||||
});
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
```
|
||||
|
||||
## Type-Level Tests (Testing P1 Tests as Specification, Data P7 Type Fidelity)
|
||||
|
||||
- **Type-level tests assert the type system, not runtime behavior:** `expectTypeOf<T>().toMatchTypeOf<U>` and `tsd`/`expect-type` fail the build when a type assertion is wrong.
|
||||
- **Negative type tests are required:** `// @ts-expect-error` proves the compiler rejects what it should. A `@ts-expect-error` that no longer errors is itself an error (the comment must be consumed).
|
||||
- **Branded types and utility types get type tests:** a `UserId` should not be assignable to `string`; a `Readonly<T>` should not allow assignment. These invariants are part of the spec (Testing P1).
|
||||
- **Applies `data/P7` (type fidelity):** a type-level test is a regression test for the type checker — if a refactor silently widens a type, the test fails.
|
||||
|
||||
```typescript
|
||||
import { expectTypeOf } from 'expect-type';
|
||||
import type { User, UserPatch, UserId } from './user';
|
||||
|
||||
test('UserPatch omits id and makes fields optional', () => {
|
||||
expectTypeOf<UserPatch>().toMatchTypeOf<{ name?: string; email?: string }>();
|
||||
expectTypeOf<UserPatch>().not.toHaveProperty('id');
|
||||
});
|
||||
|
||||
test('UserId is not assignable to bare string', () => {
|
||||
// @ts-expect-error — brand prevents widening
|
||||
const s: string = {} as UserId;
|
||||
expect(s).toBeDefined();
|
||||
});
|
||||
```
|
||||
|
||||
## Parametrize and Factories (Testing P3 Determinism, Testing P9 Edge Case Coverage)
|
||||
|
||||
- **`it.each` / `test.each` for parametrized cases:** one table drives many runs; each row is an independent test with its own name and failure output.
|
||||
- **Factories over fixtures:** `makeUser(overrides)` returns a fresh object per call. A shared `const user = {...}` across tests couples them and breaks determinism when one test mutates it (Testing P3).
|
||||
- **Edge cases as rows, not special tests:** empty array, single element, max int, null, undefined — each a row in a `test.each` table. An ad-hoc `it('handles edge')` with multiple asserts hides which case failed (Testing P9 — edge case coverage, P6 failure specificity).
|
||||
- **Property-style tests via `fast-check`:** for invariants (e.g., "parse(serialize(x)) === x"), `fast-check` generates hundreds of inputs and shrinks failures to a minimal counterexample.
|
||||
|
||||
```typescript
|
||||
import { test, expect } from 'vitest';
|
||||
import { makeUser } from './user.factory';
|
||||
|
||||
test.each([
|
||||
{ input: '', reason: 'empty' },
|
||||
{ input: 'a'.repeat(1000), reason: 'too long' },
|
||||
{ input: 'not-an-email', reason: 'no @' },
|
||||
])('rejects email: $reason', async ({ input }) => {
|
||||
await expect(makeUser({ email: input })).rejects.toThrow(ValidationError);
|
||||
});
|
||||
```
|
||||
|
||||
## Determinism and Time (Testing P3 Determinism, Testing P9 Edge Case Coverage)
|
||||
|
||||
- **No `Date.now()`, `Math.random()`, or `crypto.randomUUID()` in code under test:** inject a `Clock`, `Random`, and `IdGen` port. In tests, provide deterministic fakes.
|
||||
- **`--random` test order (Vitest `sequence.shuffle: true` default) catches order coupling:** a test that passes alone but fails in a suite has hidden state. The shuffle makes that state visible (Testing P2).
|
||||
- **Race-detector parallelism for async tests:** run async tests concurrently by default; a test that assumes serial execution breaks under parallelism. Vitest's `concurrent` flag surfaces the bug.
|
||||
|
||||
```typescript
|
||||
import { vi, test, expect } from 'vitest';
|
||||
|
||||
test.concurrent('parallel fetch does not interleave state', async () => {
|
||||
const store = new Store();
|
||||
await Promise.all([store.put('a', 1), store.put('b', 2)]);
|
||||
expect(store.get('a')).toBe(1);
|
||||
expect(store.get('b')).toBe(2);
|
||||
});
|
||||
```
|
||||
|
||||
## Cross-References
|
||||
|
||||
- `domains/testing/pyramid.md` — where unit/type/integration tests sit; the type-level tests here are the base layer.
|
||||
- `domains/testing/fixtures.md` — factory-vs-fixture discipline applied via `makeUser`.
|
||||
- `domains/testing/first-principles.md` — Testing P1 Specification, P2 Independence, P3 Determinism, P9 Edge Coverage.
|
||||
- `languages/ts-types.md` — the branded types and utility types that type-level tests assert.
|
||||
- `languages/ts-async.md` — async tests use the cancellation/timeout patterns from that doc.
|
||||
- `languages/ts-tooling.md` — `ts-jest`/`vitest` config and the `expect-type`/`tsd` toolchain.
|
||||
@@ -1,114 +0,0 @@
|
||||
# TypeScript Tooling — Derived Application
|
||||
|
||||
> Applies Atelier's domain principles to TypeScript tooling specifically.
|
||||
> Derives from `domains/` docs; introduces no new P-rules (D-063).
|
||||
> See `languages/typescript.md` for the language first-principles stub.
|
||||
|
||||
## tsc and tsconfig Discipline (DevOps P2 Automation, DevOps P1 Reproducibility)
|
||||
|
||||
- **`strict: true` is the floor, not the ceiling:** it enables `strictNullChecks`, `noImplicitAny`, `strictFunctionTypes`, and more. Disable sub-flags only with a justification comment.
|
||||
- **`tsc --noEmit` in CI:** type-checking is a build gate; emission is the bundler's job. Separate the two so a type error fails CI even when the bundler would have succeeded.
|
||||
- **`tsconfig` is per-project, not inherited verbatim:** a shared base (`extends`) encodes org defaults; each project overrides the deltas it needs. Avoids the "one monoreto-config-fits-all" trap.
|
||||
- **`noUncheckedIndexedAccess` for safety:** `arr[i]` becomes `T | undefined`, forcing narrowing. Costs little, prevents a class of out-of-bounds deref bugs.
|
||||
- **Applies `devops/P1` (reproducibility):** pinned `typescript` version in `package.json` and `lockfile` ensure every CI run type-checks against the same compiler.
|
||||
|
||||
```jsonc
|
||||
// tsconfig.json — base
|
||||
{
|
||||
"compilerOptions": {
|
||||
"strict": true,
|
||||
"noUncheckedIndexedAccess": true,
|
||||
"exactOptionalPropertyTypes": true,
|
||||
"noEmit": true,
|
||||
"moduleResolution": "bundler",
|
||||
"isolatedModules": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## ESLint and @typescript-eslint (DevOps P2 Automation, Documentation P9 Living Documents)
|
||||
|
||||
- **ESLint with `@typescript-eslint` strict ruleset:** `recommended-type-checked` enables rules that require the type checker (`no-floating-promises`, `no-misused-promises`).
|
||||
- **Rules encode decisions, not taste:** every custom rule in the config has a one-line `// reason:` comment linking to the principle it enforces. This makes the config a living document (Documentation P9).
|
||||
- **Format is Prettier's job; ESLint lints:** `eslint-config-prettier` disables conflicting format rules. Do not relitigate formatting in code review.
|
||||
- **`no-floating-promises` enforces `concurrency/P8` (timeout discipline):** an un-awaited `Promise` is a fire-and-forget that swallows errors and timeouts. The rule forces `.catch()` or `await`.
|
||||
|
||||
```jsonc
|
||||
// .eslintrc.json
|
||||
{
|
||||
"extends": [
|
||||
"eslint:recommended",
|
||||
"plugin:@typescript-eslint/recommended-type-checked",
|
||||
"prettier"
|
||||
],
|
||||
"parserOptions": { "project": "./tsconfig.json" },
|
||||
"rules": {
|
||||
// reason: enforce Concurrency P8 — no un-awaited promises
|
||||
"@typescript-eslint/no-floating-promises": "error",
|
||||
// reason: enforce Data P7 — no `any` escaping the type checker
|
||||
"@typescript-eslint/no-explicit-any": "error"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Project References and ts-jest (DevOps P2 Automation, C6 Composability)
|
||||
|
||||
- **Project references for monorepos:** `composite: true` + `references` let `tsc --build` incrementally type-check only changed projects, and enforce the dependency graph at the type level.
|
||||
- **`paths` aliases mirror the import structure:** `@app/*` → `src/*`. Configure once in `tsconfig`, mirror in the bundler and the test runner so all three agree.
|
||||
- **`ts-jest` (or `vitest`) with `isolatedModules: true`:** each test file is type-checked in isolation, matching how the bundler transpiles. Catches the "passes in `tsc` but fails in the bundler" gap.
|
||||
- **Applies `devops/P2`:** the build pipeline (tsc → lint → test → bundle) is automated; a developer never runs a manual sequence.
|
||||
|
||||
```jsonc
|
||||
// tsconfig.references.json
|
||||
{
|
||||
"files": [],
|
||||
"references": [
|
||||
{ "path": "./packages/core" },
|
||||
{ "path": "./packages/api" },
|
||||
{ "path": "./packages/web" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Lockfile and Reproducible Install (DevOps P1 Reproducibility)
|
||||
|
||||
- **`npm ci` in CI, not `npm install`:** `ci` reads the lockfile exactly and fails on drift. `install` mutates the lockfile.
|
||||
- **Lockfile committed for applications:** for libraries, commit `package-lock.json` for CI reproducibility even though consumers resolve their own tree.
|
||||
- **No floating ranges in `package.json`:** `^` and `~` are CI's job to resolve; pin the resolved version in the lockfile. An unpinned `*` is a supply-chain attack surface.
|
||||
|
||||
```bash
|
||||
# CI install step — deterministic
|
||||
npm ci
|
||||
# Type-check gate
|
||||
npx tsc --noEmit
|
||||
# Lint gate
|
||||
npx eslint .
|
||||
```
|
||||
|
||||
## Documentation in the Pipeline (Documentation P1 Documentation is Code, DevOps P9 Documentation in the Pipeline)
|
||||
|
||||
- **Type-checked JSDoc:** `typedoc` (or `TypeDoc`) generates API docs from `tsdoc` comments. The compiler enforces that `@param` names match real parameters.
|
||||
- **`@example` blocks are compiled:** a `tsdoc` `@example` fenced block is type-checked as part of the doc build. Stale examples fail the pipeline (Documentation P1 — docs are code).
|
||||
- **README badges reflect CI status:** the build/lint/test/type-check gates are the source of truth; badges surface them. Do not hand-edit status tables.
|
||||
|
||||
```typescript
|
||||
/**
|
||||
* Fetch a user by ID.
|
||||
*
|
||||
* @param id - a branded UserId (see ts-types.md).
|
||||
* @throws {NotFoundError} if the user does not exist.
|
||||
* @example
|
||||
* ```ts
|
||||
* const u = await getUser(userId('abc'));
|
||||
* ```
|
||||
*/
|
||||
async function getUser(id: UserId): Promise<User> { /* ... */ }
|
||||
```
|
||||
|
||||
## Cross-References
|
||||
|
||||
- `domains/devops/ci-cd.md` — the pipeline gates that host tsc/ESLint/ts-jest.
|
||||
- `domains/devops/first-principles.md` — DevOps P1 Reproducibility, P2 Automation.
|
||||
- `domains/documentation/first-principles.md` — Documentation P1 Documentation is Code.
|
||||
- `languages/ts-types.md` — the type rules ESLint enforces reference this doc.
|
||||
- `languages/ts-testing.md` — the test-runner config (`ts-jest`/`vitest`) detailed here.
|
||||
@@ -1,111 +0,0 @@
|
||||
# TypeScript Type System — Derived Application
|
||||
|
||||
> Applies Atelier's domain principles to TypeScript's type system specifically.
|
||||
> Derives from `domains/` docs; introduces no new P-rules (D-063).
|
||||
> See `languages/typescript.md` for the language first-principles stub.
|
||||
|
||||
## Nominal vs Structural Typing (C1 Correctness, Data P7 Type Fidelity, API P1 Contract Fidelity)
|
||||
|
||||
- **TypeScript is structurally typed:** two types with the same shape are assignable. This is convenient but erases domain boundaries — a `UserId` and `PostId` both `string` are interchangeable.
|
||||
- **Branded (nominal) types for domain IDs:** intersect with a phantom brand to simulate nominal typing. The brand is never constructed at runtime; it exists only to the type checker.
|
||||
- **Applies `data/P7` (type fidelity)** at the value boundary: a branded `UserId` cannot be passed where a `PostId` is expected, preventing an entire class of swap bugs.
|
||||
- **Applies `api/P1` (contract fidelity):** branded types make API contracts explicit — handlers cannot accept "any string" for an ID.
|
||||
- **Brand is opaque to consumers:** do not export the brand symbol; construction goes through a validated factory.
|
||||
|
||||
```typescript
|
||||
type UserId = string & { readonly __brand: 'UserId' };
|
||||
type PostId = string & { readonly __brand: 'PostId' };
|
||||
|
||||
function userId(s: string): UserId {
|
||||
if (!/^[a-zA-Z0-9]+$/.test(s)) throw new Error('invalid id');
|
||||
return s as UserId;
|
||||
}
|
||||
|
||||
function getUser(id: UserId): User { /* ... */ }
|
||||
getUser('abc'); // type error
|
||||
getUser(userId('abc')); // ok
|
||||
getUser(postId('xyz')); // type error — distinct brands
|
||||
```
|
||||
|
||||
## Generics (C6 Composability, Data P7 Type Fidelity)
|
||||
|
||||
- **Generics preserve type information across boundaries:** a `Repository<T>` keeps the element type through `find`/`save` rather than widening to `any`.
|
||||
- **Constrain with `extends`:** `<T extends Entity>` documents the contract and gives the body access to `T.id`.
|
||||
- **Avoid unnecessary generics:** if a function accepts "any value and returns it unchanged," `T` is noise. Prefer `unknown` for truly opaque inputs.
|
||||
- **Variance is structural:** TS does not enforce sound variance; mark mutation points with `readonly` to keep `T[]` assignable to `readonly T[]`.
|
||||
|
||||
```typescript
|
||||
interface Entity { id: string }
|
||||
class Repository<T extends Entity> {
|
||||
constructor(private db: Map<string, T>) {}
|
||||
find(id: string): T | undefined { return this.db.get(id); }
|
||||
save(t: T): void { this.db.set(t.id, t); }
|
||||
}
|
||||
```
|
||||
|
||||
## Narrowing and Type Guards (C1 Correctness, Errors P1 Errors are Data)
|
||||
|
||||
- **Narrowing is how TS handles `unknown` and union types safely:** `typeof`, `in`, `instanceof`, and discriminators collapse a wide type to a precise one before use.
|
||||
- **User-defined type guards (`x is T`) encode domain predicates:** `isUser(x): x is User` lets the checker track the narrow across call sites.
|
||||
- **Applies `errors/P1` (errors are data):** a `Result<T, E>` discriminated union is narrowed with `if (r.ok)` — no `try`/`catch` needed for expected failures.
|
||||
- **Never use `as` to widen past a check:** `as` lies to the compiler. If narrowing does not reach the type you need, the predicate is wrong, not the cast.
|
||||
|
||||
```typescript
|
||||
type Result<T, E> = { ok: true; value: T } | { ok: false; error: E };
|
||||
|
||||
function unwrap<T, E>(r: Result<T, E>): T {
|
||||
if (r.ok) return r.value; // narrowed to { ok: true; value: T }
|
||||
throw new Error(JSON.stringify(r.error));
|
||||
}
|
||||
|
||||
function isUser(x: unknown): x is User {
|
||||
return typeof x === 'object' && x !== null && 'id' in x && 'name' in x;
|
||||
}
|
||||
```
|
||||
|
||||
## Utility Types (C5 Reversibility, C6 Composability)
|
||||
|
||||
- **`Partial<T>`, `Pick<T,K>`, `Omit<T,K>`, `Readonly<T>` are derived views:** they derive from a source-of-truth `T` rather than redeclaring fields, so the source change propagates (reversibility).
|
||||
- **`Readonly<T>` enforces immutability at the type level** — applies `concurrency/P1` (immutability by default) without runtime cost.
|
||||
- **`Record<K, V>` over `{ [k: string]: V }`:** the index signature form allows any string key including prototype pollution vectors; `Record` is exact.
|
||||
- **Compose, don't accumulate:** `type Patch<T> = Partial<Omit<T, 'id'>>` reads as a transformation; restate it if `T` changes shape, rather than maintaining a parallel `Patch` type.
|
||||
|
||||
```typescript
|
||||
interface User { id: string; name: string; email: string; }
|
||||
type UserPatch = Partial<Omit<User, 'id'>>;
|
||||
type ReadonlyUser = Readonly<User>;
|
||||
type UsersById = Record<string, User>;
|
||||
```
|
||||
|
||||
## Discriminated Unions (C1 Correctness, Data P7 Type Fidelity, Errors P1 Errors are Data)
|
||||
|
||||
- **Discriminated unions over enums:** `type Status = { type: 'pending' } | { type: 'paid'; amount: number }` is exhaustive and carries payload per variant; an `enum` carries neither.
|
||||
- **The discriminant is a literal `type` (or `kind`) field:** the checker narrows on it in `switch` and `if` without a custom guard.
|
||||
- **Exhaustiveness via `never`:** assign the narrowed value to `never` in the default branch; if a variant is added, the default fails to compile.
|
||||
- **Applies `errors/P1`:** model domain errors as a discriminated union `AppError = NotFound | Validation | Conflict`, not as exception classes — the type system carries the error set.
|
||||
|
||||
```typescript
|
||||
type Status =
|
||||
| { type: 'pending' }
|
||||
| { type: 'paid'; amount: number }
|
||||
| { type: 'refunded'; reason: string };
|
||||
|
||||
function describe(s: Status): string {
|
||||
switch (s.type) {
|
||||
case 'pending': return 'awaiting payment';
|
||||
case 'paid': return `paid ${s.amount}`;
|
||||
case 'refunded': return `refunded: ${s.reason}`;
|
||||
default:
|
||||
const _exhaustive: never = s; // compile error if a variant is added
|
||||
throw new Error('unhandled');
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Cross-References
|
||||
|
||||
- `domains/data/schema-design.md` — schema-level fidelity parallels branded types at the TS boundary.
|
||||
- `domains/data/first-principles.md` — Data P7 Type Fidelity, the primary trace for this doc.
|
||||
- `domains/api/rest.md` — contract fidelity for API handlers consuming branded IDs.
|
||||
- `domains/errors/patterns.md` — discriminated unions as the error-as-data encoding.
|
||||
- `languages/ts-async.md` — typed async results built on the `Result` union here.
|
||||
@@ -2,13 +2,6 @@
|
||||
|
||||
> How Atelier's domain principles apply in TypeScript specifically. Derives from `domains/` docs; this file is the language-specific lens.
|
||||
|
||||
## Derived Docs
|
||||
|
||||
- [ts-types.md](ts-types.md) — TS type system: nominal-via-branding, generics, narrowing, utility types, discriminated unions.
|
||||
- [ts-tooling.md](ts-tooling.md) — tsc, ESLint, ts-jest, project references, tsconfig discipline.
|
||||
- [ts-async.md](ts-async.md) — Promises + AbortSignal, async/await, error handling, cancellation.
|
||||
- [ts-testing.md](ts-testing.md) — Vitest/Jest, mock discipline, type-level tests.
|
||||
|
||||
## Type System (C1 Correctness, Data P7 Type Fidelity)
|
||||
|
||||
- **Strict mode on:** `strict: true` in `tsconfig.json`. No `any` without justification.
|
||||
|
||||
+11
-48
@@ -6,14 +6,14 @@
|
||||
|
||||
| Core Principle | Domains that derive from it | Count |
|
||||
|----------------|---------------------------|-------|
|
||||
| C1 Correctness | All 19 (v0.1: 11; v0.2: infrastructure-as-code, kubernetes; v0.3: gitops-operators, ai-ml, i18n, compliance; v0.4: edge, messaging) | Universal |
|
||||
| C2 Clarity | v0.1: uiux, api, data, testing, observability, errors, documentation, devops; v0.2: infrastructure-as-code, kubernetes; v0.3: gitops-operators, ai-ml, i18n, compliance; v0.4: messaging | 16 |
|
||||
| C3 Simplicity | v0.1: security, data, testing, performance, documentation, concurrency, devops; v0.2: infrastructure-as-code; v0.3: gitops-operators, i18n, compliance; v0.4: edge | 12 |
|
||||
| C4 Locality | v0.1: testing, concurrency; v0.2: infrastructure-as-code, kubernetes; v0.3: gitops-operators, i18n; v0.4: edge, messaging | 8 |
|
||||
| C5 Reversibility | v0.1: api, data, uiux, concurrency, devops; v0.2: infrastructure-as-code, kubernetes; v0.3: gitops-operators, ai-ml, i18n, compliance; v0.4: edge, messaging | 13 |
|
||||
| C6 Composability | v0.1: api, security, observability, errors, documentation, concurrency; v0.2: infrastructure-as-code, kubernetes; v0.3: gitops-operators, ai-ml, i18n, compliance; v0.4: edge, messaging | 14 |
|
||||
| C7 Observability | v0.1: api, data, testing, performance, observability, errors, devops; v0.2: infrastructure-as-code, kubernetes; v0.3: gitops-operators, ai-ml, i18n, compliance; v0.4: edge, messaging | 15 |
|
||||
| C8 Economy | v0.1: security, testing, performance, observability, concurrency; v0.2: kubernetes; v0.3: gitops-operators, i18n, compliance; v0.4: edge, messaging | 11 |
|
||||
| C1 Correctness | All 17 (v0.1: 11; v0.2: infrastructure-as-code, kubernetes; v0.3: gitops-operators, ai-ml, i18n, compliance) | Universal |
|
||||
| C2 Clarity | v0.1: uiux, api, data, testing, observability, errors, documentation, devops; v0.2: infrastructure-as-code, kubernetes; v0.3: gitops-operators, ai-ml, i18n, compliance | 14 |
|
||||
| C3 Simplicity | v0.1: security, data, testing, performance, documentation, concurrency, devops; v0.2: infrastructure-as-code; v0.3: gitops-operators, i18n, compliance | 11 |
|
||||
| C4 Locality | v0.1: testing, concurrency; v0.2: infrastructure-as-code, kubernetes; v0.3: gitops-operators, i18n | 6 |
|
||||
| C5 Reversibility | v0.1: api, data, uiux, concurrency, devops; v0.2: infrastructure-as-code, kubernetes; v0.3: gitops-operators, ai-ml, i18n, compliance | 11 |
|
||||
| C6 Composability | v0.1: api, security, observability, errors, documentation, concurrency; v0.2: infrastructure-as-code, kubernetes; v0.3: gitops-operators, ai-ml, i18n, compliance | 12 |
|
||||
| C7 Observability | v0.1: api, data, testing, performance, observability, errors, devops; v0.2: infrastructure-as-code, kubernetes; v0.3: gitops-operators, ai-ml, i18n, compliance | 13 |
|
||||
| C8 Economy | v0.1: security, testing, performance, observability, concurrency; v0.2: kubernetes; v0.3: gitops-operators, i18n, compliance | 9 |
|
||||
|
||||
## Interpretation
|
||||
|
||||
@@ -34,17 +34,15 @@
|
||||
| Performance | C1, C3, C6, C7, C8 | Economy + measurement |
|
||||
| Observability | C1, C2, C6, C7, C8 | Self-referential (P7 derives from C7) |
|
||||
| Errors | C1, C2, C5, C6, C7 | Errors as data + reversibility |
|
||||
| Documentation | C1, C2, C3, C5, C6, C8 | Docs-as-code + living docs; economy of process (P1) |
|
||||
| Documentation | C1, C2, C3, C5, C6 | Docs-as-code + living docs |
|
||||
| Concurrency | C1, C3, C4, C5, C6, C7, C8 | Broadest derivation; locality (C4) unique |
|
||||
| DevOps | C1, C2, C3, C5, C7, C8 | Reproducibility + rollback; economy of effort (P2) |
|
||||
| DevOps | C1, C2, C3, C5, C7 | Reproducibility + rollback |
|
||||
| Infrastructure as Code | C1, C2, C3, C4, C5, C6, C7 | Declarative + state + composition; broadest derivation alongside Concurrency |
|
||||
| Kubernetes | C1, C2, C4, C5, C6, C7, C8 | Declarative + reversibility + economy; broad derivation (7 C-rules) |
|
||||
| GitOps + Operators | C1, C2, C3, C4, C5, C6, C7, C8 | Source-of-truth + reconciliation + pull-locality + least privilege; broadest derivation (8 C-rules, tied with i18n) |
|
||||
| AI / ML | C1, C2, C5, C6, C7 | Reproducibility + lineage + serving observability |
|
||||
| i18n | C1, C2, C3, C4, C5, C6, C7, C8 | Locale + formatting + direction + reversibility; broadest derivation (8 C-rules) |
|
||||
| Compliance | C1, C2, C3, C5, C6, C7, C8 | Audit + policy-as-code + retention + posture |
|
||||
| Edge | C1, C3, C4, C5, C6, C7, C8 | Proximity + offline + constraint; broad derivation (7 C-rules) |
|
||||
| Messaging | C1, C2, C4, C5, C6, C7, C8 | Cross-process delivery + ordering + idempotency; broad derivation (7 C-rules) |
|
||||
|
||||
## v0.2 Domain Coverage (per IDEATE-03 schema)
|
||||
|
||||
@@ -62,39 +60,6 @@
|
||||
| i18n | 10 | 4 (locale-resources, formatting, rtl-bidi, testing-i18n) | ✓ | complete |
|
||||
| Compliance | 10 | 4 (audit-logs, data-retention, policy-as-code, evidence) | ✓ | complete |
|
||||
|
||||
## v0.4 Domain Coverage (per IDEATE-03 schema)
|
||||
|
||||
| Domain | P-count | Derived-doc-count | Manifest-listed | Status |
|
||||
|--------|---------|-------------------|-----------------|--------|
|
||||
| Edge | 10 | 4 (cdn, offline-first, iot, sync) | ✓ | complete |
|
||||
| Messaging | 10 | 4 (queues, pubsub, streams, delivery-semantics) | ✓ | complete |
|
||||
|
||||
## Languages Coverage (v0.4, per D-066)
|
||||
|
||||
> Language-derived docs apply existing domain P-rules. They introduce
|
||||
> no new matrix rows (D-063, D-066). The sub-table is the verifiable
|
||||
> aggregate; inline section-headers are the per-doc detail. Every
|
||||
> derived doc traces to ≥1 domain P-rule (D-066, no orphans).
|
||||
|
||||
| Language | Derived Doc | Traced Domain P-rules | Traced Core C-rules |
|
||||
|----------|-------------|------------------------|---------------------|
|
||||
| TypeScript | ts-types.md | Data P7, API P1, Errors P1 | C1, C5, C6 |
|
||||
| TypeScript | ts-tooling.md | DevOps P2, DevOps P1, Documentation P1, DevOps P9 | C1, C2, C6, C8 |
|
||||
| TypeScript | ts-async.md | Concurrency P7, Concurrency P8, Concurrency P9, Errors P5, Errors P1 | C1, C2, C5 |
|
||||
| TypeScript | ts-testing.md | Testing P1, Testing P2, Testing P3, Testing P7, Testing P9, Data P7 | C1, C2 |
|
||||
| Python | py-types.md | Data P7, API P1, Errors P1 | C1, C2, C6 |
|
||||
| Python | py-tooling.md | DevOps P2, DevOps P1, Documentation P1, DevOps P9 | C1, C2, C3, C8 |
|
||||
| Python | py-async.md | Concurrency P1, Concurrency P7, Concurrency P8, Concurrency P9, Errors P5, Errors P1 | C1, C2, C5, C6 |
|
||||
| Python | py-testing.md | Testing P1, Testing P2, Testing P3, Testing P7, Testing P9, Concurrency P10 | C1, C2 |
|
||||
| Go | go-types.md | Data P7, API P1, Errors P1, Errors P3 | C1, C2, C6 |
|
||||
| Go | go-tooling.md | DevOps P2, DevOps P1, Concurrency P6, Documentation P1, DevOps P9 | C1, C2, C3, C7, C8 |
|
||||
| Go | go-concurrency.md | Concurrency P1, Concurrency P3, Concurrency P5, Concurrency P6, Concurrency P7, Concurrency P8, Concurrency P9 | C1, C3, C4, C5, C6, C8 |
|
||||
| Go | go-testing.md | Testing P1, Testing P2, Testing P3, Testing P7, Testing P9, Concurrency P6, Concurrency P10, API P1 | C1, C2 |
|
||||
| Rust | rs-ownership.md | Concurrency P1, Concurrency P3, Concurrency P5, Data P7 | C1, C3, C4 |
|
||||
| Rust | rs-tooling.md | DevOps P2, DevOps P1, Documentation P1, DevOps P9 | C1, C2, C3, C5, C8 |
|
||||
| Rust | rs-async.md | Concurrency P5, Concurrency P7, Concurrency P8, Concurrency P9 | C1, C3, C5, C6, C8 |
|
||||
| Rust | rs-testing.md | Testing P1, Testing P2, Testing P3, Testing P7, Testing P9, Concurrency P10, Documentation P1 | C1, C2 |
|
||||
|
||||
## Gaps and Notes
|
||||
|
||||
- No domain derives from only one C-rule. The minimum is 4 (UI/UX: C1, C2, C3, C5, C7 — actually 5). Every domain is multi-rooted.
|
||||
@@ -103,6 +68,4 @@
|
||||
- **UI/UX** and **API** are the most user-facing; they emphasize C2 (Clarity) heavily.
|
||||
- **Security** is the only domain with explicit non-tradeable declarations; this promotes 8 of its rules to C1-equivalent per `core/conflict-resolution.md` §6.
|
||||
- **v0.2 expansion:** C4 (Locality) grew from 2 to 4 domains (added infrastructure-as-code state locality, kubernetes namespace blast-radius). C6 (Composability) grew from 6 to 8. The two new domains are broad-derivation domains (7 C-rules each), consistent with Concurrency's breadth.
|
||||
- **v0.3 expansion:** C3 (Simplicity) grew from 7 to 11 (added gitops-operators declarative simplicity, i18n flexible layout, compliance structural redaction). C4 (Locality) grew from 4 to 6 (added gitops-operators pull-credential locality, i18n resource/text-direction locality). C5 (Reversibility) grew from 7 to 11 (added all four v0.3 domains — gitops history, ai-ml reproducibility, i18n translation versioning, compliance append-only/retention). C6 (Composability) grew from 8 to 12. C7 (Observability) grew from 9 to 13 (added all four v0.3 domains — reconciliation, drift detection, format correctness, posture). C2 (Clarity) grew from 10 to 14. The v0.3 expansion broadens every non-universal C-rule's coverage, confirming the four new domains are cross-cutting and well-rooted.
|
||||
- **v0.4 expansion:** C2 (Clarity) grew from 14 to 16 (added messaging P1/P2/P4 — contract clarity). C4 (Locality) grew from 6 to 8 (added edge P1/P8 geographic locality, messaging P7 partition locality). C5 (Reversibility) grew from 11 to 13 (added edge P2/P4/P7/P10 reconciliation + degradation, messaging P5/P8/P9 replay + evolution). C6 (Composability) grew from 12 to 14 (added edge P8 fleet composition, messaging P7/P9 partition + version composition). C7 (Observability) grew from 13 to 15 (added edge P10 partition-survivable telemetry, messaging P8/P10 durable-log + lag). C8 (Economy) grew from 9 to 11 (added edge P3/P9 constrained resources + trust, messaging P6 bounded backpressure). C3 (Simplicity) grew from 11 to 12 (added edge P6 explicit invalidation simplicity). Both v0.4 domains are broad-derivation (7 C-rules each), consistent with the v0.2/v0.3 precedent.
|
||||
- **Languages Coverage (v0.4, D-066):** The 16 language-derived docs (4 languages × 4 docs) apply existing domain P-rules; they introduce no new matrix rows (D-063, D-066). The sub-table above is the verifiable aggregate — every derived doc traces to ≥1 domain P-rule (no orphans). Languages are an application lens, not a principle source.
|
||||
- **v0.3 expansion:** C3 (Simplicity) grew from 7 to 11 (added gitops-operators declarative simplicity, i18n flexible layout, compliance structural redaction). C4 (Locality) grew from 4 to 6 (added gitops-operators pull-credential locality, i18n resource/text-direction locality). C5 (Reversibility) grew from 7 to 11 (added all four v0.3 domains — gitops history, ai-ml reproducibility, i18n translation versioning, compliance append-only/retention). C6 (Composability) grew from 8 to 12. C7 (Observability) grew from 9 to 13 (added all four v0.3 domains — reconciliation, drift detection, format correctness, posture). C2 (Clarity) grew from 10 to 14. The v0.3 expansion broadens every non-universal C-rule's coverage, confirming the four new domains are cross-cutting and well-rooted.
|
||||
@@ -265,39 +265,8 @@ C5=Reversibility · C6=Composability · C7=Observability · C8=Economy
|
||||
| P9 Secrets and Sensitive Data are Redacted in Audit | C1, C3 | Correctness via security; simplicity of structural redaction |
|
||||
| P10 Compliance Posture is Observable | C7, C1 | Observability of compliance; correctness of posture |
|
||||
|
||||
## Edge
|
||||
## Coverage Summary (post-v0.3)
|
||||
|
||||
| Edge Principle | Core | Why |
|
||||
|-------------------------------------------------------|-------|--------------------------------------------------|
|
||||
| P1 Proximity is the Design Driver | C4, C1 | Locality of compute near user/data; correctness via latency |
|
||||
| P2 Offline is a First-Class State | C1, C5 | Correctness under partition; reversibility of reconciliation |
|
||||
| P3 Resources are Constrained and Declared | C8, C1 | Economy of constrained nodes; correctness of declared bounds |
|
||||
| P4 Sync Conflicts are Bounded, Not Infinite | C1, C5 | Correctness of convergence; reversibility of divergent state |
|
||||
| P5 Edge Operations are Idempotent | C1 | Correctness under retry |
|
||||
| P6 Cache Invalidation is Explicit | C1, C3 | Correctness of cached state; simplicity of defined invalidation |
|
||||
| P7 Partial Degradation is Engineered | C1, C5 | Correctness of degraded modes; reversibility of recovery |
|
||||
| P8 Geographic Distribution is a First-Class Constraint | C4, C6 | Locality of placement; composability of the fleet |
|
||||
| P9 Identity is Constrained at the Edge | C1, C8 | Correctness via security; economy of trust |
|
||||
| P10 Edge Observability Survives Partition | C7, C5 | Observability of the fleet; reversibility of buffered telemetry |
|
||||
|
||||
## Messaging
|
||||
|
||||
| Messaging Principle | Core | Why |
|
||||
|--------------------------------------------------|-------|--------------------------------------------------|
|
||||
| P1 Messages are Contracts | C1, C2 | Correctness of the exchange; clarity of the schema boundary |
|
||||
| P2 Ordering is a Property, Not an Assumption | C1, C2 | Correctness of order; clarity of the guarantee |
|
||||
| P3 Consumers are Idempotent | C1 | Correctness under redelivery |
|
||||
| P4 Delivery Semantics are Explicit | C1, C2 | Correctness of the chosen semantic; clarity of the tradeoff |
|
||||
| P5 Dead-Letter Handling is Defined | C1, C5 | Correctness of poison-message routing; reversibility of reprocessing |
|
||||
| P6 Backpressure is Bounded | C1, C8 | Correctness of bounded backlog; economy of broker memory |
|
||||
| P7 Partitioning is Intentional | C4, C6 | Locality of order; composability of parallelism |
|
||||
| P8 Replay and Retention are Configured | C5, C7 | Reversibility of replay; observability of the durable log |
|
||||
| P9 Schemas Evolve Compatibly | C5, C6 | Reversibility of schema changes; composability of versions |
|
||||
| P10 Messaging is Observable | C7, C1 | Observability of lag/DLQ; correctness of backlog detection |
|
||||
|
||||
## Coverage Summary (post-v0.4)
|
||||
|
||||
- 19 domains (11 v0.1 + 2 v0.2: infrastructure-as-code, kubernetes; 4 v0.3: gitops-operators, ai-ml, i18n, compliance; 2 v0.4: edge, messaging)
|
||||
- 190 domain principles total (110 v0.1 + 20 v0.2 + 40 v0.3 + 20 v0.4)
|
||||
- Every domain P-rule traces to ≥1 core C-rule (C1–C8). No orphans.
|
||||
- Language-derived docs (16) apply existing domain P-rules; they introduce no new matrix rows (D-063, D-066). See `matrix/domain-coverage.md` `## Languages Coverage` sub-table.
|
||||
- 17 domains (11 v0.1 + 2 v0.2: infrastructure-as-code, kubernetes; 4 v0.3: gitops-operators, ai-ml, i18n, compliance)
|
||||
- 170 domain principles total (110 v0.1 + 20 v0.2 + 40 v0.3)
|
||||
- Every domain P-rule traces to ≥1 core C-rule (C1–C8). No orphans.
|
||||
@@ -197,37 +197,6 @@ If the task touches a domain, run that domain's checklist:
|
||||
- [ ] Audit logs do not leak secrets; redaction is structural, not opportunistic (P9)
|
||||
- [ ] System reports its own compliance state (drift from policy, open violations, retention status) (P10)
|
||||
|
||||
### If Edge (see `domains/edge/`)
|
||||
- [ ] Compute is placed near the user/data source; latency is treated as a correctness constraint, not a perf preference (P1)
|
||||
- [ ] The system continues to operate offline; partition is the norm, not the exception; reconciliation happens on reconnect (P2)
|
||||
- [ ] Edge-node resource constraints (CPU/memory/power/bandwidth) are declared per node class, not assumed infinite (P3)
|
||||
- [ ] Sync conflicts converge; no oscillation or infinite sync loops (P4)
|
||||
- [ ] Sync, cache-fill, and device commands are idempotent — retries are safe (P5)
|
||||
- [ ] Edge caches have an explicit TTL or invalidation strategy; no TTL-less caches under partition (P6)
|
||||
- [ ] Partial degradation is a designed state with a defined contract, not a crash (P7)
|
||||
- [ ] Routing, fan-out, and data placement are location-aware decisions, not accidents of deployment (P8)
|
||||
- [ ] Edge-device credentials are scoped and per-device; no edge node is a cluster-admin-equivalent (P9)
|
||||
- [ ] Telemetry is local-first: buffered on-node and forwarded on reconnect; partition does not blind the operator (P10)
|
||||
|
||||
### If Messaging (see `domains/messaging/`)
|
||||
- [ ] Messages have an explicit, versioned schema; producer and consumer agree on shape before exchange (P1)
|
||||
- [ ] Ordering guarantees (per-partition, global, none) are explicit and documented; "FIFO" is backed by the broker contract (P2)
|
||||
- [ ] Consumers are idempotent — redelivery is deduped via idempotency keys or deterministic processing (P3)
|
||||
- [ ] Delivery semantics (at-least-once/at-most-once/exactly-once) are a declared choice per channel (P4)
|
||||
- [ ] Poison messages route to a dead-letter queue; the DLQ is observable and drainable (P5)
|
||||
- [ ] Backpressure is bounded — consumer lag visible, max-unacked bounded, retry budget capped (P6)
|
||||
- [ ] Partition key choice is a documented design decision (ordering vs parallelism vs hotspots) (P7)
|
||||
- [ ] Retention windows and replay-from-offset are explicit; the broker is a durable log, not a pipe (P8)
|
||||
- [ ] Schema changes are backward/forward-compatible; breaking changes are versioned migrations, not silent shape edits (P9)
|
||||
- [ ] Consumer lag, DLQ depth, throughput, and consumer-group health are observed; silent backlog is a bug (P10)
|
||||
|
||||
### If Language-Derived Docs (see `languages/`)
|
||||
- [ ] Scope check: no new P-rules introduced — every section traces to an existing domain P-rule (D-063, D-066)
|
||||
- [ ] Every section header names ≥1 traced domain P-rule AND the core C-rule(s) inline (e.g., `## Nominal vs Structural Typing (C1 Correctness, Data P7 Type Fidelity, API P1 Contract Fidelity)`)
|
||||
- [ ] Fenced code examples are in-language and illustrative only — no standalone `.ts`/`.py`/`.go`/`.rs` runtime artifacts (D-020)
|
||||
- [ ] The first-principles stub retains its section structure — no P1–P10 numbering added to languages (D-063)
|
||||
- [ ] Cross-links to traced domain docs are present (≥1 outbound per derived doc, ATELIER-114)
|
||||
|
||||
## Final Gate
|
||||
|
||||
- [ ] Have I read the relevant domain's first-principles?
|
||||
|
||||
@@ -188,72 +188,6 @@ The following standalone file types are forbidden under `examples/` and elsewher
|
||||
| Signed manifests as standalone files (`.sig`, `.att`, `.intoto.jsonl`) | fenced code in `examples/*.md` | Runtime attestation artifact; violates docs-only contract |
|
||||
| Standalone `.yaml` / `.tf` / `.sh` | fenced code in `examples/*.md` | (Carried forward from v0.2) Runtime deployable artifact |
|
||||
|
||||
## v0.4 Chaos Anti-Patterns (from IDEATE-36, IDEATE-37, D-068, ATELIER-110)
|
||||
|
||||
These are named, cross-cutting violations specific to the v0.4 domains. Single-breach per D-068. Reject on sight.
|
||||
|
||||
| Anti-Pattern | Breaches | Why |
|
||||
|--------------|----------|-----|
|
||||
| Edge sync conflict loop (two nodes oscillate divergent state forever; no convergence) | edge P4 Sync Conflicts are Bounded, Not Infinite; C1, C5 | Infinite oscillation is a correctness failure, not eventual consistency; the sync is a livelock |
|
||||
| Edge TTL-less cache (edge cache with no TTL and no explicit invalidation; stale forever under partition) | edge P6 Cache Invalidation is Explicit; C1, C3 | Stale-forever under partition is a silent correctness defect |
|
||||
| Edge blocking call on constrained device (a synchronous blocking I/O call on a low-power IoT device with no timeout) | edge P3 Resources are Constrained, P5 (retry safety); C8, C1 | Blocks the constrained node; no timeout = hang; retry is unsafe without idempotency |
|
||||
| Messaging shared subscription (two consumers share one subscription; each message processed by an arbitrary consumer, breaking per-consumer ordering and idempotency) | messaging P2 Ordering, P3 Idempotency; C1 | Shared subscription breaks ordering and dedup; the `messaging-shared-subscription` bad example lives here |
|
||||
| Messaging blocking consumer (a consumer blocks on a slow downstream call with no timeout; the broker re-delivers to the same stuck consumer) | messaging P6 Backpressure is Bounded; C1, C8 | Blocks the partition; lag grows; no backpressure escape |
|
||||
| Messaging unbounded retry / no DLQ (a poison message is retried forever with no dead-letter queue; the consumer never makes progress) | messaging P5 Dead-Letter Handling is Defined; C1, C5 | Infinite retry is a livelock; poison messages must DLQ |
|
||||
|
||||
### v0.4 Deployable Artifact Types (IDEATE-36, D-020)
|
||||
|
||||
The following standalone file types are forbidden under `languages/` and elsewhere in the framework (carried forward from v0.2/v0.3 and extended for language-derived docs). All examples and derived-doc code are `.md` files with fenced code only.
|
||||
|
||||
| Forbidden standalone artifact | Belongs in | Why |
|
||||
|-------------------------------|-----------|-----|
|
||||
| Standalone `.ts` file | fenced code in `languages/*.md` (or `examples/*.md`) | Runtime TypeScript artifact; violates docs-only contract (v0.4 extension) |
|
||||
| Standalone `.py` file | fenced code in `languages/*.md` (or `examples/*.md`) | Runtime Python artifact; violates docs-only contract (v0.4 extension) |
|
||||
| Standalone `.go` file | fenced code in `languages/*.md` (or `examples/*.md`) | Runtime Go artifact; violates docs-only contract (v0.4 extension) |
|
||||
| Standalone `.rs` file | fenced code in `languages/*.md` (or `examples/*.md`) | Runtime Rust artifact; violates docs-only contract (v0.4 extension) |
|
||||
| Standalone `.yaml` / `.tf` / `.sh` | fenced code in `examples/*.md` | (Carried forward from v0.2/v0.3) Runtime deployable artifact |
|
||||
|
||||
### v0.4 Language-Derived-Doc Anti-Patterns
|
||||
|
||||
| Anti-Pattern | Breaches | Why |
|
||||
|--------------|----------|-----|
|
||||
| Language-derived doc introducing a new P-rule (e.g., "TS P1", "Python P2") | D-063, ARCHITECTURE.md hierarchy, matrix row-count invariant | Languages apply, not derive; breaks the hierarchy (`core > domains > languages`) and the 190-row invariant |
|
||||
| Language-derived doc with no domain P-rule trace (orphan section — header names no domain P-rule) | D-066, matrix completeness | No trace = no traceability; the doc is not connected to the framework |
|
||||
| Language-derived doc with runtime artifact (standalone `.ts`/`.py`/`.go`/`.rs` file) | D-020, PROJECT.md no-runtime-code | Violates the docs-only contract; must be fenced code in `.md` |
|
||||
| Language first-principles stub gaining P1–P10 numbering | D-063 | Languages are not domains; no P-rules to number; breaks the apply-don't-derive scope |
|
||||
|
||||
## v0.4 Domain-Specific Anti-Patterns
|
||||
|
||||
### Edge
|
||||
|
||||
| Anti-Pattern | Breaches | Why |
|
||||
|--------------|----------|-----|
|
||||
| Central-region-only deployment for a latency-bound workload | P1 Proximity is the Design Driver | Violates the proximity constraint; latency is a correctness defect |
|
||||
| App that crashes on disconnect (no offline state) | P2 Offline is a First-Class State | Partition is the norm; crash-on-disconnect is unengineered |
|
||||
| Undeclared edge-node resource budget (assumes infinite CPU/memory) | P3 Resources are Constrained and Declared | Unbounded = OOM on constrained nodes |
|
||||
| Sync loop that oscillates forever (CRDT without merge-semantics, LWW without monotonic clock) | P4 Sync Conflicts are Bounded | Livelock; convergence is never reached |
|
||||
| Non-idempotent edge write (cache-fill or device command retried with side effects) | P5 Edge Operations are Idempotent | Retry doubles the side effect |
|
||||
| TTL-less edge cache under partition | P6 Cache Invalidation is Explicit | Stale-forever; silent staleness |
|
||||
| Crash-on-node-failure (no partial-degradation contract) | P7 Partial Degradation is Engineered | Whole-system crash on one node's failure |
|
||||
| Random geographic placement (no location-aware routing) | P8 Geographic Distribution is a First-Class Constraint | Latency and data-residency uncontrolled |
|
||||
| Shared edge-device credential (one key for the whole fleet) | P9 Identity is Constrained at the Edge | One compromise = fleet compromise |
|
||||
| Fire-and-forget telemetry (no on-node buffer; data lost on partition) | P10 Edge Observability Survives Partition | Partition blinds the operator |
|
||||
|
||||
### Messaging
|
||||
|
||||
| Anti-Pattern | Breaches | Why |
|
||||
|--------------|----------|-----|
|
||||
| Schemaless message (no versioned contract; consumer parses by guess) | P1 Messages are Contracts | No boundary; consumer breaks silently on shape change |
|
||||
| "It's FIFO" with no documented partition contract | P2 Ordering is a Property, Not an Assumption | Undocumented ordering = assumed ordering = wrong ordering |
|
||||
| Non-idempotent consumer under at-least-once delivery | P3 Consumers are Idempotent | Redelivery doubles the effect |
|
||||
| Unstated delivery semantic (at-least-once vs exactly-once guessed) | P4 Delivery Semantics are Explicit | The tradeoff is made blindly |
|
||||
| No dead-letter queue (poison message retried forever or dropped) | P5 Dead-Letter Handling is Defined | Livelock or silent loss |
|
||||
| Unbounded retry budget (no cap; slow consumer stalls the partition) | P6 Backpressure is Bounded | Lag grows without bound |
|
||||
| Default partition key (no rationale; hotspot or wrong-order) | P7 Partitioning is Intentional | Hotspot or ordering violation |
|
||||
| Ephemeral broker (no retention; no replay) | P8 Replay and Retention are Configured | Not a durable log; replay impossible |
|
||||
| Silent schema change (producer breaks consumers with no version bump) | P9 Schemas Evolve Compatibly | Breaking change without migration |
|
||||
| Silent backlog (no lag metric; consumer falls behind invisibly) | P10 Messaging is Observable | Cannot fix what you cannot see |
|
||||
|
||||
## v0.3 Domain-Specific Anti-Patterns
|
||||
|
||||
### GitOps + Operators
|
||||
|
||||
@@ -129,39 +129,6 @@ Run the relevant domain section from `agent-checklist.md` (UI/UX, API, Security,
|
||||
- [ ] Do audit logs avoid leaking secrets (redaction is structural, not opportunistic)?
|
||||
- [ ] Does the system report its own compliance state (drift from policy, open violations, retention status)?
|
||||
|
||||
### If Edge
|
||||
- [ ] Scope check: is this the proximity/location/constraint angle, NOT generic perf or in-process concurrency? (D-061 — reject generic-performance or in-process-concurrency content as out-of-scope)
|
||||
- [ ] Is compute placed near the user/data source, with latency treated as a correctness constraint (not a perf preference)? (P1)
|
||||
- [ ] Does the system continue to operate offline; is partition the norm, not the exception? (P2)
|
||||
- [ ] Are edge-node resource constraints (CPU/memory/power/bandwidth) declared per node class, not assumed infinite? (P3)
|
||||
- [ ] Do sync conflicts converge; is there no oscillation or infinite sync loop? (P4)
|
||||
- [ ] Are sync, cache-fill, and device commands idempotent (retries safe)? (P5)
|
||||
- [ ] Do edge caches have an explicit TTL or invalidation strategy (no TTL-less caches under partition)? (P6)
|
||||
- [ ] Is partial degradation a designed state with a defined contract (not a crash)? (P7)
|
||||
- [ ] Are routing, fan-out, and data placement location-aware decisions? (P8)
|
||||
- [ ] Are edge-device credentials scoped and per-device (no edge cluster-admin)? (P9)
|
||||
- [ ] Is telemetry local-first (buffered on-node, forwarded on reconnect; partition does not blind the operator)? (P10)
|
||||
|
||||
### If Messaging
|
||||
- [ ] Scope check: is this cross-process/network-failure-model, NOT in-process concurrency? (D-062 — reject in-process-concurrency content as out-of-scope)
|
||||
- [ ] Do messages have an explicit, versioned schema (producer and consumer agree on shape)? (P1)
|
||||
- [ ] Are ordering guarantees (per-partition, global, none) explicit and documented? (P2)
|
||||
- [ ] Are consumers idempotent (redelivery deduped via idempotency keys or deterministic processing)? (P3)
|
||||
- [ ] Are delivery semantics (at-least-once/at-most-once/exactly-once) a declared choice per channel? (P4)
|
||||
- [ ] Do poison messages route to a dead-letter queue (DLQ observable and drainable)? (P5)
|
||||
- [ ] Is backpressure bounded (consumer lag visible, max-unacked bounded, retry budget capped)? (P6)
|
||||
- [ ] Is partition key choice a documented design decision (ordering vs parallelism vs hotspots)? (P7)
|
||||
- [ ] Are retention windows and replay-from-offset explicit (durable log, not pipe)? (P8)
|
||||
- [ ] Are schema changes backward/forward-compatible (breaking changes are versioned migrations)? (P9)
|
||||
- [ ] Are consumer lag, DLQ depth, throughput, and consumer-group health observed (silent backlog is a bug)? (P10)
|
||||
|
||||
### If Language-Derived Docs
|
||||
- [ ] Scope check: apply, don't derive — no new P-rules, no new matrix rows (D-063, D-066 — reject any "TS P1"/"Python P2" etc. new-P-rule content)
|
||||
- [ ] Does every section header name ≥1 traced domain P-rule AND the core C-rule(s) inline?
|
||||
- [ ] Are fenced code examples in-language and illustrative only (no standalone `.ts`/`.py`/`.go`/`.rs` runtime artifacts)? (D-020)
|
||||
- [ ] Does the first-principles stub retain its section structure (no P1–P10 numbering added to languages)? (D-063)
|
||||
- [ ] Are cross-links to traced domain docs present (≥1 outbound per derived doc)? (ATELIER-114)
|
||||
|
||||
## Review Etiquette
|
||||
|
||||
- **Comment, don't command.** "This could be X" not "Change this to X."
|
||||
|
||||
Reference in New Issue
Block a user