11 Commits

Author SHA1 Message Date
Jon Chery f61dffbb5a docs(P02): complete messaging domain phase — v0.4
---ci---
project: atelier
phase: 2
milestone: v0.4
status: complete
phase_role: execution
phase_tag: v0.3.2
requirements:
  covered: [ATELIER-97, ATELIER-98, ATELIER-99, ATELIER-100, ATELIER-101]
  partial: []
---/ci---
2026-08-05 16:00:31 +00:00
Jon Chery f7f007dce8 docs(ship): P1 complete — v0.3.1 tagged, release 479
---ci---
project: atelier
phase: 1
milestone: v0.4
status: complete
phase_role: execution
phase_tag: v0.3.1
release_id: 479
---/ci---
2026-08-05 15:53:16 +00:00
Jon Chery 20992883ff docs(P01): complete edge domain phase — v0.4
---ci---
project: atelier
phase: 1
milestone: v0.4
status: complete
phase_role: execution
phase_tag: v0.3.1
requirements:
  covered: [ATELIER-92, ATELIER-93, ATELIER-94, ATELIER-95, ATELIER-96]
  partial: []
---/ci---
2026-08-05 15:51:03 +00:00
Jon Chery a622419e7d docs(ship): P0 complete — v0.3.0 tagged, release 475
---ci---
project: atelier
phase: 0
milestone: v0.4
status: complete
phase_role: pre_execution
phase_tag: v0.3.0
release_id: 475
---/ci---
2026-08-05 14:03:35 +00:00
Jon Chery b1d293d218 docs(P00): complete pre-execution phase — v0.4
---ci---
project: atelier
phase: 0
milestone: v0.4
status: complete
phase_role: pre_execution
decisions:
  - id: D-054
    decision: "v0.4 scope = edge + messaging + language-derived docs"
  - id: D-055
    decision: "NFR milestone type — all phases docs/chore"
  - id: D-056
    decision: "6 execution phases P1-P5 + final phase P6"
  - id: D-057
    decision: "Edge domain groups CDN + offline-first + IoT + sync"
  - id: D-058
    decision: "Messaging domain covers queues + pub/sub + streams + delivery-semantics"
  - id: D-059
    decision: "Language-derived docs trace to existing domain P-rules, no new matrix rows"
  - id: D-060
    decision: "4 derived docs per language, 16 total, per-language picks"
  - id: D-061..D-069
    decision: "9 clarify resolutions (boundary, persona, examples, manifest, matrix)"
  - id: D-070..D-072
    decision: "Research findings — edge P1-P10, messaging P1-P10, persona roster"
  - id: D-073
    decision: "13 ideation recommendations accepted (12 refinements + 1 new req ATELIER-117)"
  - id: D-074..D-075
    decision: "46-task vertical-slice plan, persona lifecycle"
  - id: G-013..G-018
    decision: "Grill conditional proceed — G-014 false-premise fix applied"
requirements:
  covered: [ATELIER-92, ATELIER-93, ATELIER-94, ATELIER-95, ATELIER-96, ATELIER-97, ATELIER-98, ATELIER-99, ATELIER-100, ATELIER-101, ATELIER-102, ATELIER-103, ATELIER-104, ATELIER-105, ATELIER-106, ATELIER-107, ATELIER-108, ATELIER-109, ATELIER-110, ATELIER-111, ATELIER-112, ATELIER-113, ATELIER-114, ATELIER-115, ATELIER-116, ATELIER-117]
  partial: []
---/ci---
2026-08-05 14:01:13 +00:00
Jon Chery 9ebc9c8868 docs(milestone): complete v0.3 — GitOps+Operators/AI-ML/i18n/Compliance
---ci---
project: atelier
phase: 6
milestone: v0.3
status: complete
requirements:
  covered: [ATELIER-60..91]
  partial: []
---/ci---
2026-08-05 03:45:38 +00:00
Jon Chery 5335ffa3b8 docs(audit): v0.2 post-sync re-audit — clean
Re-audit after upstream sync (main + tags pushed to origin). All 10 checks pass:
reconstruction, branch hygiene, commit discipline, file discipline, tag sequence
(local↔remote aligned), manifest discipline, releases, structural verification,
security (docs-only), requirements traceability (59 covered, 0 pending).

Escalations: 2 auto-resolved (git transport auth fix, stale tag force-update).
P1 convention note: examples/ not in MANIFEST (pre-existing, candidate for v0.3).

---ci---
project: atelier
phase: 5
milestone: v0.2
status: complete
escalation:
  - id: ESC-001
    type: release_pending
    resolution: auto
    note: "v0.2 main+tags not pushed; fixed via http.extraheader auth, pushed main + force-updated tags"
  - id: ESC-002
    type: release_pending
    resolution: auto
    note: "Remote v0.1.x tags pointed to stale tip; force-updated to correct v0.2 phase commits"
---/ci---
2026-08-05 02:46:02 +00:00
Jon Chery 9db0df126c docs(milestone): complete v0.2 — iac-k8s
All 24 requirements (ATELIER-36..59) covered. ROADMAP v0.2 → complete.
Platform-engineer persona removed (roster back to 3 active). Checkpoint cleared.

---ci---
project: atelier
phase: 5
milestone: v0.2
status: complete
requirements:
  covered: [ATELIER-36..59]
  partial: []
---/ci---
2026-08-05 02:21:47 +00:00
Jon Chery d1aa5daf2b docs(milestone): complete v0.2 — infrastructure-as-code + kubernetes
---ci---
project: atelier
phase: 5
milestone: v0.2
status: complete
requirements:
  covered: [ATELIER-36, ATELIER-37, ATELIER-38, ATELIER-39, ATELIER-40, ATELIER-41, ATELIER-42, ATELIER-43, ATELIER-44, ATELIER-45, ATELIER-46, ATELIER-47, ATELIER-48, ATELIER-49, ATELIER-50, ATELIER-51, ATELIER-52, ATELIER-53, ATELIER-54, ATELIER-55, ATELIER-56, ATELIER-57, ATELIER-58, ATELIER-59]
  partial: []
---/ci---
2026-08-05 02:20:17 +00:00
Jon Chery 89d5668c23 chore: migrate repo to coreci owner
---ci---
project: atelier
phase: 0
milestone: v0.1
status: complete
phase_role: final
type: migration
---/ci---

Repo transferred on Gitea: cloudinit-bot/atelier -> coreci/atelier.
Local remote URL updated. config.json release.owner and gitea.owner updated: cloudinit-bot -> coreci.
All 8 tags and 8 releases preserved through the transfer.
2026-08-05 00:51:34 +00:00
Jon Chery 496303471d docs(milestone): complete v0.1 — initial framework
---ci---
project: atelier
phase: 7
milestone: v0.1
status: complete
phase_role: final
milestone_complete: true
requirements:
  covered: [ATELIER-01, ATELIER-02, ATELIER-03, ATELIER-04, ATELIER-05, ATELIER-06, ATELIER-07, ATELIER-08, ATELIER-09, ATELIER-10, ATELIER-11, ATELIER-12, ATELIER-13, ATELIER-14, ATELIER-15, ATELIER-16, ATELIER-17, ATELIER-18, ATELIER-19, ATELIER-20, ATELIER-21, ATELIER-22, ATELIER-23, ATELIER-24, ATELIER-25, ATELIER-26, ATELIER-27, ATELIER-28, ATELIER-29, ATELIER-30, ATELIER-31, ATELIER-32, ATELIER-33, ATELIER-34, ATELIER-35]
  partial: []
ship:
  milestone: v0.1
  type: NFR
  tag: v0.0.7
  merge: milestone/v0.1-atelier -> main
  release: https://git.cloudinit.dev/cloudinit-bot/atelier/releases/tag/v0.0.7
---/ci---

Milestone v0.1 — Initial Framework (NFR, complete).
8 core principles (C1-C8), 11 domains, 110 domain principles, 27 derived docs, 4 good + 3 bad examples, 4 language docs, full matrix, 3 review docs.
All 35 requirements covered. 7 patches (v0.0.0 pre-execution through v0.0.7 final). v0.0.7 IS the v0.1.0 milestone release.
2026-08-05 00:36:55 +00:00
135 changed files with 17131 additions and 0 deletions
+15
View File
@@ -0,0 +1,15 @@
{
"phase": 1,
"stage": "complete",
"milestone": "v0.4",
"phase_role": "execution",
"project": "atelier",
"attempts": 0,
"updated_at": "2026-08-05T06:15:00Z",
"milestone_complete": false,
"milestone_branch": "milestone/v0.4-edge-quantum-langs",
"phase_branch": "phase/01-edge",
"tag_base": "v0.3",
"phase_tag": "v0.3.1",
"release_id": 479
}
+109
View File
@@ -0,0 +1,109 @@
# Atelier — Architecture
## System Overview
Atelier is a **documentation-only framework** — no runtime code, no build artifacts. The "system" is the document tree itself and the relationships between documents.
## Component Boundaries
```
atelier/
├── core/ # Universal principles — the foundation
├── domains/ # Domain-specific application of core
│ ├── ... (v0.1: 11 domains)
│ ├── infrastructure-as-code/ # v0.2: IaC tooling (terraform, opentofu, state, modules)
│ ├── kubernetes/ # v0.2: k8s platform (workloads, networking, storage, rbac, helm, kustomize)
│ ├── gitops-operators/ # v0.3: GitOps + Operators (argocd, flux, operators, progressive-delivery)
│ ├── ai-ml/ # v0.3: ML engineering (data-versioning, model-evaluation, serving, monitoring-drift)
│ ├── i18n/ # v0.3: internationalization (locale-resources, formatting, rtl-bidi, testing-i18n)
│ └── compliance/ # v0.3: compliance/audit (audit-logs, data-retention, policy-as-code, evidence)
├── languages/ # Language-specific application of domains
├── review/ # Evaluation checklists and anti-patterns
├── matrix/ # Cross-reference: domain ↔ core
└── examples/ # Worked examples (good + bad)
```
**Dependency direction (enforced):**
- `core/` depends on nothing
- `domains/` depend on `core/` (each domain's P-rules trace to core C-rules)
- `languages/` depend on `domains/` (language rules apply domain rules)
- `matrix/` depends on `core/` + `domains/` (it's the join table)
- `review/` depends on everything (it evaluates compliance)
- `examples/` depend on everything (they illustrate)
No document may reference a sibling at the same level as authoritative — conflict resolution always goes UP to core.
## Data Flow
There is no runtime data flow. The "flow" is the **reading order**:
1. `core/first-principles.md` (required first)
2. `core/conflict-resolution.md` (required)
3. `domains/<relevant>/first-principles.md` (per task)
4. `domains/<relevant>/<topic>.md` (as needed)
5. `matrix/principles-matrix.md` (reference)
## Conflict Resolution Architecture
When two documents disagree:
1. Higher in the hierarchy wins (`core` > `domains` > `languages`)
2. Within a level, the more specific document wins for its scope
3. If still ambiguous, the core principle with the highest precedence (C1 Correctness > C2 Clarity > ...) wins
4. The `matrix/principles-matrix.md` is the arbiter — it records which core principle each domain rule derives from
## Technology
- **Format:** Markdown (CommonMark + GitHub Flavored)
- **Versioning:** Whole-framework semantic versioning (one version per release, not per-doc)
- **Distribution:** Git repository (consumed by reading markdown)
- **No build step** — markdown is the artifact
## Governance
- `.ciagent/` holds project management artifacts (config, PROJECT.md, ROADMAP.md, REQUIREMENTS.md, ARCHITECTURE.md, PERSONAS.md, PLAN)
- The framework content lives in the repo root (`README.md`, `core/`, `domains/`, etc.)
- `MANIFEST.md` is the authoritative index — unlisted documents are not part of the framework
## Research-Refined Decisions
- **8 core principles (not 10):** The spec's matrix key names exactly 8 (C1C8). Each domain has 10 (P1P10). Asymmetry is intentional — core is minimal, domains are specific.
- **Numeric precedence C1 > C2 > ... > C8:** Correctness is non-negotiable; Economy is the most tradeable.
- **Strict hierarchy enforcement:** No lateral authority. Conflicts go UP, never sideways.
- **No build step:** Markdown is the artifact. A build step would violate docs-as-code simplicity.
## Prior Art Position
Atelier's differentiation: **traceable principle hierarchy with a join table**. Existing frameworks (ThoughtWorks Radar, Google Eng Practices, SOLID, 12-Factor, OWASP, WCAG) state principles; none provide a matrix mapping every domain rule back to a core rule. This makes Atelier's `matrix/principles-matrix.md` the conflict-resolution arbiter — the framework's unique value.
## v0.2 Domain Additions
Two new top-level domains extend the tree under the same hierarchy rules:
- **`infrastructure-as-code/`** — tooling domain. First principles govern declarative provisioning (terraform, opentofu, state, modules). Depends on `core/`. Cross-links to `devops/` (P1 Reproducibility, P6 Configuration as Code) and `security/` (supply-chain, secrets).
- **`kubernetes/`** — platform domain. First principles govern container orchestration (workloads, networking, storage, rbac, helm, kustomize). Depends on `core/`. Cross-links to `devops/` (P4 Rollback First, P5 Progressive Delivery), `security/` (authorization, secrets), `observability/` (metrics, tracing), `data/` (storage patterns).
Both domains follow the v0.1 contract: 10 P-rules each, every rule traced to a core C-rule via the matrix, no orphans. The manifest (`MANIFEST.md`) is extended to keep them authoritative. No runtime code — examples are illustrative markdown with manifests in code fences only.
See `.ciagent/atelier/RESEARCH.md` for the full prior-art survey and `.ciagent/atelier/PERSONAS.md` for the persona roster (3 custom active personas + 1 phase-specific platform-engineer; 3 default personas deactivated).
## v0.3 Domain Additions
Four new top-level domains extend the tree under the same hierarchy rules. All four follow the v0.1/v0.2 contract: 10 P-rules each, every rule traced to a core C-rule via the matrix, no orphans, docs-only markdown with illustrative code fences (no runtime/deployable artifacts). Total matrix grows from 130 → 170 domain principles across 13 → 17 domains.
- **`gitops-operators/`** — platform-automation domain. First principles govern the declarative-source-of-truth reconciliation loop shared by ArgoCD, Flux, Kubernetes Operators, and Progressive Delivery tooling (Argo Rollouts, Flagger). Depends on `core/`. Cross-links to `kubernetes/` (workloads, rbac, helm, kustomize — the platform GitOps reconciles onto), `infrastructure-as-code/` (declarative intent, state-as-truth — the shared model), `devops/` (P1 Reproducibility, P4 Rollback First, P5 Progressive Delivery, P6 Configuration as Code), `security/` (secrets, supply-chain — GitOps credentials, signed manifests), `observability/` (reconciliation metrics, drift visibility). Derived docs: `argocd.md`, `flux.md`, `operators.md`, `progressive-delivery.md`.
- **`ai-ml/`** — ML engineering domain (engineering discipline, NOT algorithm design per D-023). First principles govern data versioning, model evaluation, serving, and monitoring/drift. Depends on `core/`. Cross-links to `data/` (schema-design, migrations, indexing — data lineage and versioning share the migration/reversibility model), `observability/` (metrics, tracing — model serving metrics, drift signals), `devops/` (P1 Reproducibility — training/serving reproducibility, P7 Immutability — model images), `security/` (input-validation — inference input validation, secrets — model/serving credentials), `performance/` (backend — serving latency). Derived docs: `data-versioning.md`, `model-evaluation.md`, `serving.md`, `monitoring-drift.md`.
- **`i18n/`** — internationalization domain. First principles govern locale resources, formatting, RTL/bidi layout, and testing. Depends on `core/`. Cross-links to `uiux/` (components, accessibility, copywriting — locale-aware UI is the consumer), `testing/` (fixtures, pyramid — i18n testing parallels), `api/` (error-responses — localized API errors), `data/` (schema-design — locale data shapes). Derived docs: `locale-resources.md`, `formatting.md`, `rtl-bidi.md`, `testing-i18n.md`.
- **`compliance/`** — compliance/audit domain (framework-agnostic, NOT regulation-specific per D-024). First principles govern audit logs, data retention, policy-as-code, and evidence collection. Depends on `core/`. Cross-links to `security/` (authorization — who did what, secrets — audit log integrity, supply-chain — signed policy), `observability/` (logging, metrics — audit logs are a structured-logging concern, tracing — evidence from distributed traces), `data/` (schema-design, migrations — retention schema), `infrastructure-as-code/` (policy-as-code parallels IaC declarative intent), `kubernetes/` (rbac — audit subject identity). Derived docs: `audit-logs.md`, `data-retention.md`, `policy-as-code.md`, `evidence.md`.
All four domains depend on `core/` only for authority; cross-links to existing domains are one-directional (per v0.2 D-026 convention extended to v0.3 — minimize churn to existing content). The manifest (`MANIFEST.md`) is extended in P4 to list all new documents. Examples (P5) are illustrative markdown with fenced code only — no `.yaml`, `.json`, `.po`, model artifacts, or deployable manifests as standalone files.
## v0.3 Ideation Architectural Notes
From the v0.3 ideation stage (IDEATE-17..30), the following architectural refinements are baked into the execute-phase plan:
- **Manifest scope expansion (IDEATE-17 → ATELIER-91):** the v0.2 audit escalation (ESC-002 note) flagged that `examples/` is not listed in `MANIFEST.md`. P4 adds an `examples/` directory listing to the manifest, closing the pre-existing drift. The manifest remains authoritative; unlisted directories are not part of the framework by definition.
- **Matrix coverage summary invariants (IDEATE-18 → ATELIER-80):** the matrix coverage summary must reflect post-v0.3 totals (17 domains, 170 P-rules) — both the summary block and the per-domain section count.
- **Core Principle Coverage table (IDEATE-19 → ATELIER-81):** `matrix/domain-coverage.md` contains two tables — the per-domain row schema table (covered by v0.2 IDEATE-03) AND the "Core Principle Coverage" table mapping C1C8 → domains. Both must be extended for the 4 new domains; the C-rule counts shift (e.g., C4 Locality adds i18n + gitops; C5 Reversibility adds ai-ml + compliance + gitops + i18n).
- **Cross-link type unchanged:** v0.3 introduces no new cross-link type. All cross-links remain one-directional outward from new domains to existing (D-033). No back-link edits to v0.1/v0.2 content.
See `.ciagent/atelier/RESEARCH.md` "v0.3 Research" for the full prior-art survey and principle inventory rationale, and `.ciagent/atelier/PERSONAS.md` for the v0.3 persona roster (5 active: lead-developer, tech-writer, domain-expert + 2 phase-specific platform-engineer, ml-engineer; 3 default personas deactivated).
+51
View File
@@ -0,0 +1,51 @@
# Atelier — Domain First-Principles Audit (P2)
> Conducted during phase 2. Domain-expert persona validated all 11 domain first-principles files and the matrix.
## Audit Results
### Principle Count Per Domain (target: 10)
| Domain | Count | Status |
|--------|-------|--------|
| api | 10 | ✓ |
| security | 10 | ✓ |
| data | 10 | ✓ |
| testing | 10 | ✓ |
| performance | 10 | ✓ |
| observability | 10 | ✓ |
| errors | 10 | ✓ |
| documentation | 10 | ✓ |
| concurrency | 10 | ✓ |
| devops | 10 | ✓ |
| uiux | 10 | ✓ (newly authored in P2) |
**Total: 110 principles across 11 domains. No gaps.**
### Matrix Completeness
- Total P-rule rows in `matrix/principles-matrix.md`: 110 (11 domains × 10 principles)
- Every row maps to at least one C-rule
- Core principle distribution (derivation counts):
- C1 Correctness: 46 derivations (most common — correctness is the floor)
- C2 Clarity: 23
- C3 Simplicity: 18
- C7 Observability: 22
- C8 Economy: 16
- C5 Reversibility: 11
- C6 Composability: 9
- C4 Locality: 3 (least — locality is domain-specific)
**No orphaned P-rules.** Every domain principle traces to a core principle.
### Observations
1. C1 (Correctness) is the most-derived principle (46/110) — expected, as correctness is the floor.
2. C4 (Locality) has only 3 derivations — it is the most domain-specific (testing independence, concurrency boundaries). Not a gap; locality is inherently narrow.
3. Security domain declares 8 of 10 principles non-tradeable — these are promoted to C1-equivalent per `core/conflict-resolution.md` §6.
4. UI/UX declares 3 non-tradeable (P2 Accessibility, P3 Clarity, P1 User Primacy) — promoted to C1-equivalent.
5. No domain has a P-rule without a matrix entry. No matrix row lacks a corresponding section in the domain's first-principles.md.
### Conclusion
The framework's principle hierarchy is complete and traceable. Phase 2 deliverable (uiux/first-principles.md) fills the last gap. The matrix is the arbiter and is consistent with all domain documents.
+78
View File
@@ -0,0 +1,78 @@
# Atelier — v0.2 Post-Sync Audit (Re-audit)
> Re-audit of milestone v0.2 after upstream sync (main + tags pushed to origin).
> Triggered by user observation that v0.2 releases existed but code/tags were not pushed to remote main.
> This audit verifies the now-synced state is clean.
## Context
During the original v0.2 run, `git push origin main --tags` failed because git-over-HTTPS required username/password auth and only `GITEA_API_TOKEN` (API token) was available. The Gitea releases (462467) were created via the API, but the local merge commits and tags never reached the remote. This re-audit was requested after the fix.
### Fix Applied
- Configured `git config --local http.https://git.cloudinit.dev/.extraheader "Authorization: token $GITEA_API_TOKEN"` to authenticate git transport with the API token.
- Pushed `main`: `89d5668..9db0df1 main -> main` (success).
- Force-updated remote tags `v0.1.0``v0.1.5` (they previously existed but pointed at the stale `milestone/v0.1-atelier` tip `5f522962`; now point at the correct v0.2 phase commits).
## Audit Results
### 1. Reconstruction test — PASS
- `origin/main` now contains the 2 v0.2 squash-merge commits (`d1aa5da` milestone merge, `9db0df1` milestone completion). Phase history preserved via tags.
- `REQUIREMENTS.md`: 59 requirements marked `covered`, 0 `pending` (matches 24 v0.2 + 35 v0.1).
- `ROADMAP.md`: `v0.2 — ... (COMPLETE)`.
- `config.json`: project status `complete`.
### 2. Branch hygiene — PASS
- Only `main` branch exists locally; no leftover v0.2 phase/milestone branches.
- Remote: `main` + `milestone/v0.1-atelier` (the v0.1 milestone branch, untouched — preserved as history).
### 3. Commit discipline — PASS
- All 2 v0.2 commits on main contain `---ci---` blocks (squash-merge structure means main sees 1 commit per phase-merge + 1 milestone-completion commit; full per-phase history in tags).
### 4. File discipline — PASS
- `.ciagent/atelier/` contains all required files: PROJECT, ROADMAP, REQUIREMENTS, ARCHITECTURE, PERSONAS, PLAN, RESEARCH, CLARIFY, REVIEW-P5.
- v0.1 legacy files (AUDIT-P2, REVIEW-P7) preserved — harmless history.
### 5. Tag sequence — PASS (local ↔ remote aligned)
- All 15 tags (v0.0.0v0.0.7, v0.1.0v0.1.5) present on both local and remote.
- All annotated tags dereference to identical commits on both sides.
- v0.0.x tags untouched (point to v0.1 milestone commits). v0.1.x tags now point to correct v0.2 phase commits (v0.1.0=P0, v0.1.1=P1, ..., v0.1.5=P5=milestone release).
### 6. Manifest discipline — PASS (with noted convention)
- All 12 new v0.2 domain docs (infrastructure-as-code/*, kubernetes/*) listed in MANIFEST Domains table.
- matrix, agent-checklist, peer-review-checklist, anti-patterns all listed in Cross-Cutting.
- **Convention note (P1, pre-existing):** examples/ (both v0.1 and v0.2) are not individually listed in MANIFEST. The manifest's own rule states "unlisted = not part of the framework," yet examples are unlisted by convention across both milestones. This is a latent inconsistency, not a v0.2 regression. Candidate for v0.3 ideation: add an Examples section to MANIFEST, or amend the rule to scope it to Core/Domains/Cross-Cutting.
### 7. Remote releases — PASS
- 6 v0.2 releases exist on Gitea (ids 462467, tags v0.1.0v0.1.5).
- v0.1 releases intact (ids 454461, tags v0.0.0v0.0.7, with v0.0.7 = the v0.1 milestone release).
- Total: 14 releases across both milestones.
### 8. Structural verification (remote main) — PASS
- IaC first-principles: 10 P-rules ✓
- K8s first-principles: 10 P-rules ✓
- Matrix: 130 total P-rule rows across 13 domains + coverage summary (14 sections) ✓
- IaC matrix rows: 10 ✓; K8s matrix rows: 10 ✓
- IaC derived docs: 4 (terraform, opentofu, state, modules) ✓
- K8s derived docs: 6 (workloads, networking, storage, rbac, helm, kustomize) ✓
- v0.2 examples: 4 (terraform-module, k8s-deployment, terraform-unlocked-state, k8s-bare-pod-no-resources) ✓
### 9. Security (docs-only constraint) — PASS
- No standalone `.tf`, `.yaml`, `.yml`, `.sh` files in the repo — docs-only constraint preserved.
- All code is fenced within `.md` files.
### 10. Requirements traceability — PASS
- 59 covered, 0 pending (24 v0.2 + 35 v0.1).
## Verdict
**AUDIT CLEAN.** No critical (P0) issues. One P1 convention note (examples not in MANIFEST — pre-existing, candidate for v0.3 ideation). The upstream sync fixed the v0.2 release/push gap: remote main, all 15 tags, and all 6 Gitea releases are now consistent and correct.
## Escalation Log
| ID | Issue | Resolution | Type |
|----|-------|-----------|------|
| ESC-001 | v0.2 main + tags not pushed to remote (git transport auth) | Configured http.extraheader with GITEA_API_TOKEN; pushed main + force-updated tags | auto-resolved |
| ESC-002 | Remote v0.1.x tags pointed to stale milestone/v0.1-atelier tip | Force-updated all 6 v0.1.x tags to correct v0.2 phase commits | auto-resolved |
Both escalations auto-resolved at full autonomy. No pipeline halt.
+100
View File
@@ -0,0 +1,100 @@
# Atelier v0.4 — Clarify Stage
> Generated during v0.4 P0 CLARIFY stage. Autonomy level: `full` (defaults auto-accepted).
> All decisions below are above the 0.60 confidence threshold → auto-decided, no HITL.
> Decisions continue from D-060 (last v0.3 decision). Scope: NEW v0.4 work only (edge, messaging, language-derived docs). Prior milestone decisions (D-001..D-060) are not re-clarified.
## Ambiguities Resolved: 9
### 1. Edge domain P-rules — boundary vs performance/concurrency/observability (avoid duplication)
- **Question:** The edge domain is latency-bound, resource-constrained, and geo-distributed — concerns that overlap with the existing performance domain (caching P5, lazy P6, async P7), concurrency domain (bounded queues P9, immutability P1), and observability domain (structured logs, correlation). How do edge's 10 P-rules distinguish themselves without duplicating existing domain P-rules?
- **Default resolution:** Edge owns the **proximity/location angle**: compute placed near the user or data source, location-aware routing, resource constraints of edge nodes, and offline-capability. The boundary is: performance owns *generic measurement & optimization*; concurrency owns *in-process synchronization primitives*; observability owns *generic logging/metrics/tracing*; edge owns *the placement, constraint, and disconnection concerns that only arise at the network edge*. Edge P-rules trace heavily to C4 Locality (the defining edge trait — compute near data/user) and C8 Economy (constrained devices), plus C5 Reversibility (offline/sync reversibility), C7 Observability (geo-distributed visibility), C1 Correctness (eventual-consistency correctness under partition). Edge is a broad-derivation domain (68 C-rules, parallel to i18n/gitops). Cross-links to performance/concurrency/observability are one-directional outward (per D-026 extended); no back-link edits.
- **Rationale:** Every prior domain owned a distinct "angle" on shared core principles (k8s P8 storage-explicit vs iac P3 state-is-truth — same C-rules, different scope per D-026/RESEARCH). Edge's unique contribution is *where compute runs*, which no existing domain addresses. The C4 Locality emphasis is the discriminator: performance's locality is algorithmic (data near compute); edge's locality is geographic (compute near user). Confirmed by RESEARCH.md pattern: "each domain owns its own principle angle."
- **Confidence:** 0.88
- **Alternatives rejected:** (a) Edge as a derived doc under performance — rejected: geo-distribution + offline + IoT constrained-device concerns warrant a full 10-P-rule set, not a single derived doc (parallel to D-020 k8s-vs-devops reasoning); (b) Edge reusing performance P5/P6/P7 verbatim — rejected: would orphan edge rules from their actual derivation (C4 Locality, C8 Economy) and violate the "no duplicate rows" matrix invariant.
- **Decision:** D-061
### 2. Messaging domain boundary vs concurrency/patterns.md (in-process vs cross-system)
- **Question:** `domains/concurrency/patterns.md` already covers "Message Passing" (Pattern 1), "Bounded Queue with Backpressure" (Pattern 5), and "Actor Model" (Pattern 3). The new messaging domain covers queues, pub/sub, streams, and delivery semantics. Where is the boundary — does messaging duplicate concurrency's queue/message-passing patterns?
- **Default resolution:** Concurrency owns **in-process** synchronization primitives (threads, locks, channels, async within one program). Messaging owns **cross-process / cross-system** asynchronous communication (brokers: Kafka, RabbitMQ, SQS, NATS, Pulsar). The discriminator: messaging concerns arise *only when producer and consumer are separate systems* — delivery semantics (at-least-once/at-most-once/exactly-once), ordering across partitions, idempotency across retries, dead-letter queues, replay from offsets, consumer-group coordination. Concurrency's "bounded queue" is the in-process analog; messaging's "bounded queue" is the broker-backed analog with network-failure semantics. Messaging P-rules trace to C1 Correctness (delivery guarantees), C5 Reversibility (replay/rollback), C7 Observability (lag, DLQ visibility), C6 Composability (pub/sub decoupling). Cross-link concurrency/patterns.md ← messaging (one-directional); messaging references concurrency as the in-process foundation but owns the distributed-delivery layer above it.
- **Rationale:** The v0.2 RESEARCH established the precedent: gitops-operators is "the deployment-automation layer above kubernetes/infrastructure-as-code — borrows their model, adds the git-as-source-of-truth layer." Messaging is the same: the async-delivery layer above concurrency — borrows the bounded-queue/backpressure model, adds cross-system delivery/ordering/idempotency. Concurrency/patterns.md Pattern 5 explicitly scopes itself to in-process ("producer is blocked or signaled"); messaging extends this to network-partition scenarios. No duplication because the *failure model* differs (crash vs network failure).
- **Confidence:** 0.86
- **Alternatives rejected:** (a) Messaging as a derived doc under concurrency — rejected: delivery semantics, stream processing, and broker comparison warrant a full 10-P-rule domain (parallel to D-021 gitops-unification reasoning: splitting would fragment P-rules); (b) Merge messaging P-rules into concurrency — rejected: would bloat concurrency to 20 rules and conflate in-process vs distributed concerns, breaking the "10 P-rules per domain" invariant (D-018/D-030); (c) No cross-link to concurrency — rejected: the bounded-queue/backpressure lineage should be explicit (one outbound cross-link per derived doc per ATELIER-114).
- **Decision:** D-062
### 3. Language-derived docs structure — no new P-rules, section-header trace pattern (confirm D-059)
- **Question:** PROJECT.md states language-derived docs "trace to existing domain P-rules, not new matrix rows" (the v0.4 Cross-Domain Integration bullet). But how exactly is a language-derived doc structured? The existing `languages/*.md` stubs use section headers like `## Type System (C1 Correctness, Data P7 Type Fidelity)` — they have NO numbered P1P10 rules. Do the 16 new derived docs (ts-types, ts-tooling, etc.) introduce their own P-rules, or follow the same section-header-trace pattern with no new P-rule numbering?
- **Default resolution:** Language-derived docs introduce **NO new P-rules**. Each derived doc follows the existing stub structure: section headers that name the traced core C-rule(s) and domain P-rule(s) inline (e.g., `## Strict Null Checks (C1 Correctness, Errors P1 Errors are Data)`), with bullet-point guidance and fenced code. The 4 derived docs per language each focus on one facet (types, tooling, async/concurrency, testing) and trace to the relevant existing domain P-rules. The first-principles stub for each language (typescript.md, etc.) is *extended* with a pointer to its derived docs but retains its existing section structure — it does NOT gain P1P10 numbering. Matrix row count stays at 190 (170 + 20 for edge+messaging only).
- **Rationale:** ARCHITECTURE.md dependency direction states `languages/` depend on `domains/` ("language rules apply domain rules") — languages are an *application lens*, not a principle source. The existing stubs confirm this: they trace to `Data P7`, `Errors P1`, `Concurrency P7`, `Observability P1`, `DevOps P2` — never inventing new P-rules. D-059 (referenced in PROJECT.md v0.4 section) already fixed the matrix treatment; this decision confirms the *document structure* that implements it. Adding P1P10 to languages would break the hierarchy (languages would become peer domains, violating "conflicts go UP, never sideways" and the core>domains>languages ordering).
- **Confidence:** 0.92
- **Alternatives rejected:** (a) Language-derived docs get their own P1P10 per language (40 new P-rules) — rejected: violates ARCHITECTURE.md hierarchy (languages apply, not derive) and D-059's matrix ruling; would also push matrix to 230 rows, breaking the stated 190 total; (b) Hybrid: first-principles stub gains P-rules, derived docs don't — rejected: inconsistent within a language and still introduces new matrix rows contrary to PROJECT.md; (c) Derived docs are pure code examples with no principle traces — rejected: would orphan them from the matrix/traceability contract and lose the "why over what" tech-writer constraint.
- **Decision:** D-063
### 4. Language-derived doc file naming — ts-/py-/go-/rs- prefixes confirmed
- **Question:** PROJECT.md v0.4 uses short prefixes for derived doc filenames: `ts-types.md`, `py-types.md`, `go-types.md`, `rs-ownership.md` — but the parent first-principles files use full names (`typescript.md`, `python.md`, `go.md`, `rust.md`). The prefix for Rust is `rs-` (not `rust-`), and Go uses `go-` (matching its parent). Is the ts-/py-/go-/rs- prefix scheme correct, or should it be revisited for consistency (e.g., typescript-types, or rust-ownership)?
- **Default resolution:** Confirm the **ts-/py-/go-/rs-** prefix scheme as specified in PROJECT.md. `ts-` (TypeScript), `py-` (Python), `go-` (Go), `rs-` (Rust). These match the dominant ecosystem conventions: `ts` is the universal TypeScript abbreviation (tsconfig, ts-node, .ts extension); `py` is the universal Python abbreviation (.py, pyproject, py.test); `go` matches the language's own binary/module naming; `rs-` is the established Rust ecosystem prefix (crate names, file conventions). The full parent filename + short-prefix-derived-doc pattern mirrors how domains work (`kubernetes/` parent + `workloads.md` derived — the derived doc doesn't repeat the full domain name).
- **Rationale:** Consistency with domain-derived-doc naming: derived docs use short, distinct slugs (kubernetes/workloads.md, not kubernetes-kubernetes-workloads.md). The short prefix disambiguates the language while keeping filenames compact. `rs-` over `rust-` follows the broader Rust ecosystem (the `rs-` prefix is more common in registry/package naming than `rust-`, which can collide with the `rust` toolchain). `go-` is correct because Go's ecosystem uses `go` universally and the parent file is already `go.md`.
- **Confidence:** 0.84
- **Alternatives rejected:** (a) Full-name prefixes (typescript-types.md, python-types.md, rust-ownership.md) — rejected: verbose, inconsistent with domain-derived-doc compactness, and `rust-` collides with toolchain naming conventions; (b) Single-letter or extension-based (ts/types.md subdirectory per language) — rejected: changes the flat `languages/` structure established in v0.1 (MANIFEST lists `languages/typescript.md`, not `languages/typescript/first-principles.md`); introduces directory nesting the v0.1 manifest schema doesn't support; (c) Normalize Rust to `rust-` for readability — rejected: breaks ecosystem convention alignment; `rs-` is the recognized Rust short prefix.
- **Decision:** D-064
### 5. Language-derived docs in MANIFEST — new languages/ section (closes pre-existing drift)
- **Question:** The current `MANIFEST.md` (post-v0.3) has sections for Core, Domains, Examples, and Cross-Cutting — but **no `languages/` section at all**, despite `languages/{typescript,python,go,rust}.md` existing since v0.1 (ATELIER-31, P6). This is pre-existing manifest drift (parallel to the v0.2 ESC-002 note where `examples/` was unlisted). Where do the 16 new language-derived docs (and the 4 existing stubs) go in the manifest?
- **Default resolution:** Add a **new top-level `## Languages` section** to `MANIFEST.md` in P4, parallel to the `## Domains` and `## Examples` sections. The section lists each language (TypeScript, Python, Go, Rust) with its first-principles stub (✓) and its 4 derived docs. This *both* lists the 16 new v0.4 derived docs AND closes the pre-existing drift by listing the 4 v0.1 stubs that were never added. The section appears after `## Domains` and before `## Examples` (matching the reading order: core → domains → languages). The manifest's Cross-Cutting summary line updates: "19 domains, 190 P-rules post-v0.4" and notes that languages apply (not derive) domain rules.
- **Rationale:** The manifest is authoritative — "unlisted = not part of the framework" (PROJECT.md constraint, MANIFEST.md header). The v0.2 audit escalation ESC-002 established the precedent: when a directory exists but is unlisted, the next milestone that touches it adds the listing (IDEATE-17 → ATELIER-91 closed examples/ drift in v0.3). v0.4 touches `languages/` for the first time since v0.1, so it closes the languages/ drift on the same principle. Placing the section between Domains and Examples matches `core/reading-order.md` (languages are read after domains, before examples) and the ARCHITECTURE.md dependency direction (languages depend on domains). This is a single P4 task, not split across phases.
- **Confidence:** 0.90
- **Alternatives rejected:** (a) List language-derived docs under the existing `## Domains` section — rejected: languages are not domains (ARCHITECTURE.md distinguishes the two levels; mixing them breaks the hierarchy and the matrix row-count invariant); (b) Defer the 4 v0.1 stubs to a future milestone, list only the 16 new docs — rejected: would leave the pre-existing drift half-closed and is inconsistent with the ESC-002 precedent (close the whole directory when you touch it); (c) Put languages/ at the end after Cross-Cutting — rejected: violates reading order (languages are consumed before examples/matrix reference, per core/reading-order.md).
- **Decision:** D-065
### 6. Matrix treatment of language-derived docs — NO new matrix rows; traceability via domain-coverage sub-table + inline traces
- **Question:** D-059 (PROJECT.md) says language-derived docs "trace to existing domain P-rules, not new matrix rows." But traceability must be *captured somewhere* — where? Does `matrix/principles-matrix.md` gain a languages section (with non-numbered trace rows)? Does `matrix/domain-coverage.md` gain a languages sub-table? How is "this language-derived doc traces to these domain P-rules" recorded verifiably?
- **Default resolution:** `matrix/principles-matrix.md` gains **NO language rows** — it stays at 190 P-rules (170 + 20 for edge+messaging), 19 domains. `matrix/domain-coverage.md` gains a **new `## Languages Coverage` sub-table** (per ATELIER-107) with schema: `Language | Derived Doc | Traced Domain P-rules | Traced Core C-rules`. Each of the 16 derived docs gets one row listing the domain P-rules its section-headers trace to (e.g., `TypeScript | ts-types.md | Data P7 Type Fidelity, Errors P1, C1 | C1, C6`). Traceability is ALSO captured inline in each derived doc's section headers (the existing stub pattern: `## Section (C-rule, Domain P-rule)`). The domain-coverage sub-table is the verifiable aggregate; the inline headers are the per-doc detail. The agent-checklist (ATELIER-108) gains a language-derived-docs check: "every derived-doc section header names ≥1 traced domain P-rule."
- **Rationale:** The principles-matrix is the domain→core join table; languages are not domains and have no P-rules to map (D-063), so adding rows would either be empty or fake. The domain-coverage sub-table is the right home: it already holds the per-domain coverage schema (IDEATE-03) and the C-rule→domains table (IDEATE-19) — a languages sub-table is the natural extension, parallel in shape. This gives reviewers a single place to verify "no language-derived doc is orphaned from its traced domain P-rules" without scanning all 16 files. Mirrors how v0.3's C-rule coverage table (IDEATE-19) was the complement to the per-domain row schema.
- **Confidence:** 0.87
- **Alternatives rejected:** (a) Add a `## Languages` section to principles-matrix.md with one row per derived doc → C-rule — rejected: the matrix maps P-rules, and languages have no P-rules (D-063); this would conflate the join table's semantics and inflate the "190 P-rules" count that PROJECT.md/ATELIER-106 explicitly state; (b) Traceability only inline in section headers, no aggregate table — rejected: not verifiable in one place; reviewers would have to open 16 files to confirm no orphans, violating the domain-expert persona's "no orphans in matrix" constraint; (c) Add languages as a 20th "domain" row in the per-domain coverage table — rejected: languages are not domains (ARCHITECTURE.md hierarchy); would corrupt the domain count invariant.
- **Decision:** D-066
### 7. Phase-specific personas — edge-engineer (P1) + languages-engineer (P3); messaging (P2) by tech-writer + domain-expert with edge-engineer consultation
- **Question:** ROADMAP v0.4 names "edge-engineer + languages-engineer phase-specific personas." That is only TWO named personas for three execution phases (P1 edge, P2 messaging, P3 languages). Does the messaging domain (P2) get its own phase-specific persona (e.g., messaging-engineer), or is it covered by existing/adjacent personas? If the latter, who owns messaging P2 authorship and with what constraints?
- **Default resolution:** Add **two** phase-specific personas: `edge-engineer` (active P1, removed after milestone) and `languages-engineer` (active P3, removed after milestone). The **messaging domain (P2) is authored by tech-writer + domain-expert** (the permanent roster), with edge-engineer consulted for the distributed-systems/async overlap (D-062 boundary). edge-engineer constraints: ["latency is a correctness constraint", "compute placement is a design decision", "offline is a first-class state", "trace to core", "10 P-rules per domain", "no runtime code", "resource constraints are explicit", "sync conflict is a correctness failure"]. languages-engineer constraints: ["apply, don't derive (D-063)", "trace every section to an existing domain P-rule", "language idioms serve principles, not the reverse", "no new P-rules", "no new matrix rows (D-066)", "fenced code is illustrative only"]. Messaging P2 gets a task must-have baked into the plan: "P1/P2 demonstrated edge + messaging boundary (D-062); delivery-semantics comparison table required (D-069)" so the edge-engineer's distributed-systems lens survives the P1→P2 persona transition (parallel to D-051's ml-engineer-constraints-baked-into-task pattern from v0.3).
- **Rationale:** The ROADMAP explicitly names only 2 personas — this is the authoritative signal (the ROADMAP is finalized in PLAN and pre-named personas per PROJECT.md v0.4 section + ROADMAP P0 row). v0.3 set the precedent (D-022): smaller-surface or within-competence domains don't get new personas (i18n + compliance in v0.3 P3 used tech-writer + domain-expert, no new personas). Messaging, while specialized, is within domain-expert's cross-cutting competence (delivery semantics, idempotency, ordering are correctness/observability derivations the domain-expert already validates). Limiting to 2 phase-specific personas holds the active roster to 5 (3 permanent + 2 phase-specific), matching v0.3's peak and avoiding the "persona explosion" risk flagged in v0.3 RESEARCH. The edge-engineer consultation for P2 ensures the D-062 in-process-vs-cross-system boundary is respected without a 3rd persona.
- **Confidence:** 0.80
- **Alternatives rejected:** (a) Three phase-specific personas (edge-engineer P1, messaging-engineer P2, languages-engineer P3) — rejected: ROADMAP names only 2; would push roster to 6 active (3+3), exceeding v0.3's 5-persona peak and raising the explosion risk; messaging's specialty, while real, is less novel than ai-ml's (which warranted ml-engineer per D-019/D-020); (b) One persona (edge-engineer) covers P1+P2, languages-engineer P3 — rejected: edge (latency/geo/resource) and messaging (delivery/ordering/idempotency) are distinct specialties; forcing one persona blurs the D-062 boundary and dilutes edge-specific constraints; (c) No new personas, all three phases by tech-writer + domain-expert — rejected: edge's latency-as-correctness and compute-placement concerns, and languages' apply-don't-derive discipline, benefit from specialist constraints the generalist personas lack (v0.2 D-014 / v0.3 D-019 precedent).
- **Decision:** D-067
### 8. Examples — confirm 2 good + 2 bad pre-named set; bad examples cite breached P-rules
- **Question:** PROJECT.md / ATELIER-112/113 pre-name the 4 v0.4 examples: `good/edge-offline-sync.md`, `good/messaging-idempotent-consumer.md`, `bad/edge-sync-loop.md`, `bad/messaging-shared-subscription.md`. Are these the right picks (vs alternatives like good/cdn-cache-invalidation or bad/messaging-blocking-consumer), and what breach semantics must the bad examples follow?
- **Default resolution:** Confirm the pre-named set as specified. **Good:** `edge-offline-sync.md` (demonstrates offline-first + sync conflict resolution via CRDT or LWW — the highest-illustration-value edge case, exercising P1 offline + the sync.md decision matrix), `messaging-idempotent-consumer.md` (demonstrates exactly-once-via-idempotency-keys + dedup — the highest-illustration-value messaging case, exercising delivery-semantics + idempotency). **Bad:** `edge-sync-loop.md` (a sync conflict that oscillates/loops forever — the chaos anti-pattern pre-specified in ATELIER-110), `messaging-shared-subscription.md` (a shared subscription causing duplicate processing / fan-out duplication — the chaos anti-pattern pre-specified in ATELIER-110). Each bad example must cite the **specific P-rule breached** (per D-025 / IDEATE-07 named-bad-example convention carried forward v0.1→v0.2→v0.3). `edge-sync-loop.md` breaches the edge sync P-rule (exact P# TBD in P1, but it is the "sync conflicts are bounded, not infinite" rule); `messaging-shared-subscription.md` breaches the messaging subscription P-rule (exact P# TBD in P2, but it is the "consumer identity is per-consumer, not shared" rule). Both bad examples are single-breach (unlike v0.3's two-breach compliance example D-044 — edge/messaging breaches are cleaner as single-rule violations).
- **Rationale:** The pre-named set covers one good + one bad per new domain — balanced, parallel to v0.2 (terraform + k8s) and v0.3 (gitops + ai-ml good; i18n + compliance bad). D-025 (v0.2) established 2-good + 2-bad for 2-domain milestones; v0.4 follows the same ratio. The chaos anti-patterns (sync loop, shared subscription) are pre-specified in ATELIER-110 because they are the highest-illustration-value *fundamental* violations — a sync loop is to edge what push-deploy was to GitOps (IDEATE-24); a shared subscription is to messaging what an orphan model was to ai-ml (IDEATE-27). Alternatives (cdn-cache-invalidation good, blocking-consumer bad) are lower-value: cache invalidation is a performance-domain concern (cross-link only), and blocking-consumer overlaps concurrency/patterns.md (D-062 boundary). Single-breach is correct because edge/messaging P-rules are individually clear; the two-breach pattern (D-044) was specific to compliance where two breaches (append-only + redaction) compound the highest-stakes domain.
- **Confidence:** 0.85
- **Alternatives rejected:** (a) `good/cdn-cache-invalidation.md` instead of edge-offline-sync — rejected: cache invalidation is primarily a performance/P5 concern; offline-sync exercises more edge-unique P-rules (offline state, sync, conflict); (b) `bad/messaging-blocking-consumer.md` instead of shared-subscription — rejected: blocking-consumer is a concurrency/patterns.md concern (D-062 in-process boundary); shared-subscription is a cross-system messaging-unique failure; (c) Two-breach bad examples (per D-044 compliance pattern) — rejected: D-044 was domain-specific to compliance's compounded stakes; edge/messaging single-rule breaches are clearer and match v0.1/v0.2 bad-example convention; (d) 4 good + 4 bad (one per derived doc) — rejected: would unbalance P5 (D-025 reasoning: 2+2 covers highest-illustration-value; remaining coverage via cross-links + anti-patterns).
- **Decision:** D-068
### 9. Derived-doc decision/comparison tables — confirm queues/sync matrices + add streams platform comparison
- **Question:** v0.2 established the decision-matrix pattern (Helm vs Kustomize in ATELIER-46/47), v0.3 transferred it to 3 derived docs (ArgoCD vs Flux, DVC vs Delta Lake vs LakeFS, OPA vs Cedar vs Kyverno vs Sentinel). v0.4 REQUIREMENTS pre-specify two: `queues.md` needs an at-least-once/at-most-once/exactly-once comparison (ATELIER-98), and `sync.md` needs a CRDT/last-write-win decision matrix (ATELIER-96). But `streams.md` (ATELIER-100) and `delivery-semantics.md` (ATELIER-101) have no comparison table specified. Should they also get one, following the established pattern?
- **Default resolution:** Confirm the two pre-specified matrices (queues delivery-semantics comparison in queues.md; CRDT-vs-LWW decision matrix in sync.md). Add **two more** to maintain pattern parity (every v0.4 derived doc that compares distinct tools/models gets a decision/comparison table, per the v0.2→v0.3 transfer convention): `streams.md` gets a **stream-platform comparison table** (Kafka vs Kinesis vs Pulsar/NATS JetStream) covering ordering guarantees, partitioning model, replay/retention, and use-case fit; `delivery-semantics.md` gets a **dead-letter strategy comparison table** (retry-count-limit vs TTL-with-backoff vs poison-queue vs DLQ+alert) covering when each applies, failure visibility, and operational cost. Additionally, `cdn.md` (ATELIER-93) already has a pre-specified "edge-cache vs origin decision matrix" — that is confirmed (the cache-hit/miss/invalidation tradeoff). Total: 4 decision/comparison tables across the 9 new derived docs (cdn, queues, sync, streams, delivery-semantics = 5 candidate docs; 4 get tables — delivery-semantics gets a strategy comparison rather than a tool-vs-tool matrix since it's a pattern comparison, not a product comparison).
- **Rationale:** The v0.2→v0.3 transfer pattern (REQUIREMENTS v0.3 "Within-Project Pattern Transfer" table, IDEATE-21/22/23) established that derived docs comparing distinct tools/models MUST include a decision/comparison table — this is now a project convention, not a one-off. streams.md compares Kafka/Kinesis/Pulsar (3+ distinct platforms with different ordering/replay models) — directly parallel to v0.3's data-versioning tool comparison (DVC/Delta Lake/LakeFS, IDEATE-22). delivery-semantics.md compares dead-letter strategies (distinct failure-handling models) — parallel to v0.2's Helm-vs-Kustomize "when X fits" guidance. Omitting tables from streams/delivery-semantics would be an inconsistency reviewers should flag (the v0.3 transfer-table verification was a P4 review check). cdn.md's edge-cache-vs-origin matrix is already required (ATELIER-93) and is the cache-placement analog of Helm-vs-Kustomize.
- **Confidence:** 0.82
- **Alternatives rejected:** (a) Only the 2 pre-specified tables (queues, sync), none for streams/delivery-semantics — rejected: breaks the v0.2→v0.3 transfer convention; streams.md comparing 3 platforms without a table would be the first multi-tool derived doc since v0.2 to omit one; (b) Tables in all 5 candidate docs including a tool-vs-tool table in delivery-semantics — rejected: delivery-semantics compares *strategies/patterns*, not products; forcing a product table (e.g., "Kafka DLQ vs RabbitMQ DLQ") would be misleading since DLQ semantics are cross-platform; a strategy comparison is the correct shape; (c) Defer streams/delivery-semantics tables to v0.5 — rejected: pattern-parity should land in the milestone that introduces the docs; deferral creates immediate drift (the v0.3 transfer table explicitly verifies within-milestone parity).
- **Decision:** D-069
## Decisions Log
| ID | Decision | Confidence |
|----|----------|------------|
| D-061 | Edge domain owns the proximity/location angle (C4 Locality, C8 Economy heavy); performance/concurrency/observability own generic primitives; cross-links one-directional; edge is a broad-derivation domain (68 C-rules) | 0.88 |
| D-062 | Messaging owns cross-process/cross-system async (brokers, delivery semantics); concurrency owns in-process primitives; boundary is the failure model (network vs crash); cross-link concurrency ← messaging | 0.86 |
| D-063 | Language-derived docs introduce NO new P-rules; follow existing section-header-trace pattern (C-rule + Domain P-rule inline); first-principles stubs extended but retain structure, no P1P10 numbering | 0.92 |
| D-064 | Confirm ts-/py-/go-/rs- prefix scheme for language-derived doc filenames (ecosystem convention alignment; mirrors domain-derived-doc compactness) | 0.84 |
| D-065 | Add new top-level `## Languages` section to MANIFEST.md in P4 (between Domains and Examples); lists 4 stubs + 16 derived docs; closes pre-existing languages/ drift parallel to ESC-002 | 0.90 |
| D-066 | principles-matrix.md gains NO language rows (stays 190/19 domains); domain-coverage.md gains a `## Languages Coverage` sub-table (Language → Derived Doc → Traced Domain P-rules → Traced C-rules); inline section-headers are per-doc detail | 0.87 |
| D-067 | Two phase-specific personas: edge-engineer (P1) + languages-engineer (P3); messaging (P2) by tech-writer + domain-expert with edge-engineer consultation; constraints baked into P2 task must-have (D-051 pattern) | 0.80 |
| D-068 | Confirm pre-named examples (good: edge-offline-sync, messaging-idempotent-consumer; bad: edge-sync-loop, messaging-shared-subscription); bad examples cite specific breached P-rule; single-breach (not D-044 two-breach) | 0.85 |
| D-069 | 4 decision/comparison tables across v0.4 derived docs: cdn (edge-cache vs origin), queues (delivery semantics), sync (CRDT vs LWW), streams (Kafka vs Kinesis vs Pulsar), delivery-semantics (dead-letter strategy comparison) — pattern parity with v0.2/v0.3 | 0.82 |
## Summary
- 9 ambiguities identified and auto-resolved at full autonomy
- Average confidence: 0.86
- All resolutions grounded in existing project conventions (D-001..D-060, prior domain structure, core C-rules, ARCHITECTURE.md hierarchy, RESEARCH.md prior-art pattern, ESC-002/IDEATE-17 manifest-drift precedent, IDEATE-07/D-025 named-bad-example convention, v0.2→v0.3 decision-matrix transfer pattern, D-051 constraints-baked-into-task persona pattern)
- No decision fell below the 0.60 autonomy threshold → no HITL escalation triggered
- Lowest-confidence decision (D-067, 0.80): messaging P2 persona choice — resolved by deferring to the ROADMAP's explicit 2-persona naming and the D-022 within-competence precedent, with edge-engineer consultation to preserve the D-062 boundary
+59
View File
@@ -0,0 +1,59 @@
# Atelier — Clarify Decisions
> Generated during P0 CLARIFY stage. Autonomy level: `full` (defaults auto-accepted).
> Initial decisions D-001..D-010 recorded in init commit; this file consolidates the full clarify log.
## Ambiguities Identified and Resolved
| ID | Ambiguity | Default (Accepted) | Confidence | Rationale |
|----|-----------|--------------------|------------|-----------|
| D-001 | License type unspecified | MIT | 0.70 | Standard permissive license for docs frameworks; deferred to P1 |
| D-002 | README/CHANGELOG/CONTRIBUTING content not pasted | Defer to execute phase 1+6 | 0.80 | Referenced in spec tree but text not provided |
| D-003 | `core/first-principles.md` content not pasted (referenced as "already drafted") | Defer to P1; author from scratch using 8 principles (C1C8) referenced in matrix | 0.70 | Matrix names C1..C8 explicitly; enough to reconstruct |
| D-004 | `domains/uiux/first-principles.md` not pasted (referenced as "already drafted") | Defer to P2; author from components.md + accessibility.md which imply P1..P10 | 0.70 | components.md and accessibility.md reference the principles |
| D-005 | `examples/`, `languages/`, `review/`, `matrix/domain-coverage.md` in scope for init? | Out of scope — execute-phase deliverables | 0.90 | Spec Part 6 build order places them after first-principles |
| D-006 | Token env var name | `GITEA_API_TOKEN` | 0.80 | Matches existing /root/.ciagent/config.json convention |
| D-007 | Milestone version | v0.1 (user override) | 0.95 | User explicitly required v0.1, not computed v1.0 |
| D-008 | Number of execution phases | 6 (P1P6) + final P7 | 0.88 | Mirrors spec Part 6 Recommended Build Order |
| D-009 | Milestone type | NFR (all phases are docs/chore) | 0.92 | No `feat` code; patch-line versioning per branch-strategy.md |
| D-010 | Tag patch line | v0.0.x (prev minor from v0.1) | 0.90 | Per branch-strategy.md: milestone 0.1 → tags v0.0.0..v0.0.7 |
| D-011 | `core/first-principles.md` — exact C1..C8 names | Use the 8 named in matrix key: Correctness, Clarity, Simplicity, Locality, Reversibility, Composability, Observability, Economy | 0.95 | matrix/principles-matrix.md explicitly names them in the key |
| D-012 | `core/conflict-resolution.md` — precedence order of C1..C8 | Numeric order C1 > C2 > ... > C8 (Correctness highest, Economy lowest) | 0.85 | Standard convention; matrix lists them in this order |
| D-013 | `core/reading-order.md` — exact order | core/first-principles → core/conflict-resolution → core/reading-order → domains/<x>/first-principles → domains/<x>/<topic> → matrix/principles-matrix | 0.92 | MANIFEST.md Reading Order section defines this |
| D-014 | Should `examples/bad/*` be real anti-patterns or fictional? | Real anti-patterns from the field (god-object, silent-error, leaky-abstraction as named in spec) | 0.90 | Spec explicitly names these three |
| D-015 | `languages/*.md` — which languages? | typescript, python, go, rust minimum (spec shows these + "...") | 0.85 | Spec tree lists typescript, python, go, rust, "..." |
| D-016 | Versioning scheme for the framework itself | Whole-framework semver (one version per release) | 0.90 | ARCHITECTURE.md decided this; no per-doc versioning |
| D-017 | Should `domains/uiux/tokens.md` and `copywriting.md` content come from spec? | Defer to P3; spec tree lists them but no content pasted | 0.80 | Spec Part 1 tree lists them; Part 4 only gives components + accessibility |
| D-018 | Are domain P1..P10 numbers fixed or can domains have fewer? | Fixed at 10 per domain (matrix shows 10 per domain consistently) | 0.92 | matrix/principles-matrix.md has exactly 10 rows per domain |
## Notes
- All defaults auto-accepted per `full` autonomy level (`decision_confidence_threshold: 0.6`).
- All decisions above 0.60 threshold → auto-decided, no HITL.
- No escalation triggered (no decision fell below 0.60).
- Clarify budget used: 18 of 10... wait — clarify_budget is 10 questions max. Decisions D-001..D-010 came from init; D-011..D-018 are the new clarify-stage ambiguities (8 questions, within budget).
## v0.2 Clarifications
> Generated during v0.2 P0 CLARIFY stage. Autonomy level: `full` (defaults auto-accepted).
> All decisions below are above the 0.60 confidence threshold → auto-decided.
| ID | Ambiguity | Default (Accepted) | Confidence | Rationale |
|----|-----------|--------------------|------------|-----------|
| D-019 | Should terraform/opentofu be a new domain or extend `devops/`? | New top-level `domains/infrastructure-as-code/` | 0.88 | User confirmed two-domain split (tool vs platform); IaC tooling has its own 10 principles distinct from devops P-rules |
| D-020 | Should kubernetes be a new domain or a `devops/` derived doc? | New top-level `domains/kubernetes/` | 0.88 | User confirmed; k8s platform concerns (workloads, networking, storage, rbac) warrant a full principle set, not a single derived doc |
| D-021 | Should OpenTofu get its own domain or share with Terraform? | Share `infrastructure-as-code/` domain; separate derived docs | 0.85 | OpenTofu is a Terraform fork (Linux Foundation); same HCL/state model, fork-specific notes in opentofu.md |
| D-022 | Which k8s derived docs? | workloads, networking, storage, rbac, helm, kustomize | 0.90 | Core k8s domains; operators/gitops deferred (can be IDEATE items) |
| D-023 | Should IaC include a state-management derived doc separately from terraform/opentofu? | Yes — `state.md` covers state backends, locking, drift, independent of tool | 0.83 | State is a cross-cutting IaC concern; terraform.md and opentofu.md reference it |
| D-024 | Should IaC include a modules derived doc? | Yes — `modules.md` covers composition, versioning, registry | 0.80 | Composition is core C6; modules are the IaC expression of it |
| D-025 | Should examples include actual deployable manifests or illustrative markdown? | Illustrative markdown only — manifests in code fences for teaching | 0.95 | PROJECT.md "no runtime code" constraint; examples are docs-as-code teaching artifacts like existing examples/good/* |
| D-026 | Cross-link direction for new domains? | New domains link outward to existing (devops, security, observability, data); existing domains unchanged in v0.2 (no back-link edits) | 0.82 | Minimize churn to v0.1 content; cross-links are one-directional in v0.2 |
| D-027 | Should `platform-engineer` persona be phase-specific (removed after P5) or permanent? | Phase-specific — removed after milestone v0.2 completes | 0.80 | Per D-014; persona roster returns to 3 active personas post-v0.2 |
| D-028 | Phase count for v0.2 | 4 execution phases (P1P4) + final P5 | 0.85 | Per D-015; user confirmed 5-phase structure |
| D-029 | Should IDEATE consider roadmap candidates (ai-ml, i18n, compliance) alongside IaC/k8s? | Yes — full mechanical + backend tiers (user selected); accepted non-IaC/k8s ideas defer to v0.3 | 0.78 | User chose full-tier ideation; v0.2 scope stays IaC/k8s, other ideas parked for v0.3 |
| D-030 | Are the 10 P-rules per new domain fixed or can they differ? | Fixed at 10 each (matches v0.1 convention D-018) | 0.92 | Consistency with v0.1; matrix expects 10 per domain |
## v0.2 Notes
- All v0.2 decisions above 0.60 threshold → auto-decided, no HITL.
- Clarify budget for v0.2: 12 new questions (D-019..D-030), within budget of 10... exceeds by 2. Rationale: 3 of the 12 (D-019, D-020, D-022) were pre-resolved by user answers during plan-mode clarification, so effective new questions = 9, within budget.
+193
View File
@@ -0,0 +1,193 @@
# Atelier — Grill (Adversarial Red-Team Review)
> Pre-execution gate for milestone v0.4 (Edge + Messaging + Language-Derived Docs).
> Default assumption: the project is unfeasible, over-scoped, and too costly. Not convinced until evidence forces it.
> Mode: full autonomy. Auto-resolve at confidence ≥ 0.60; escalate only < 0.60 that cannot be auto-resolved.
---
## v0.4 Grill — 2026-08-05
**Milestone:** v0.4 — Edge + Messaging + Language-Derived Docs
**Phase:** 0 (Pre-Execution, GRILL stage)
**Grill scope:** all 9 axes + meta
**Prior grill runs:** v0.3 (GRILL.md — PROCEED @ 0.80). v0.1/v0.2 P0 stages did not include a GRILL stage.
**Autonomy:** full (binding verdicts; no HITL questions)
**Axes:** 9 review axes (Feasibility, Scope, Cost/Risk, Vertical-slice integrity, Traceability, Decision quality, Persona coverage, Test/Verify, Cross-link/matrix integrity)
### Verdict: **CONDITIONAL PROCEED** (overall confidence 0.83)
The plan is the 4th consecutive NFR milestone using the same proven docs-only contract (v0.1 35 reqs → v0.2 24 reqs → v0.3 32 reqs → v0.4 26 reqs). All three prior milestones shipped clean with identical structure. The 26-req scope is *smaller* than v0.3's 32 reqs, and per-phase doc counts (P1:5, P2:5, P3:20, P4:7, P5:5, P6:2) are within demonstrated throughput. The two new domains (edge, messaging) have well-reasoned boundaries against existing domains (D-061 proximity angle; D-062 cross-process failure model) and the 16-doc language extension is the largest single phase (P3) but introduces **zero** matrix rows (D-063/D-066), so it does not stress the traceability invariant.
The verdict is **CONDITIONAL** rather than clean PROCEED because the red-team verification surfaced **one false premise** baked into the plan and the audit: IDEATE-34/ATELIER-106 refinement asserts a "pre-existing matrix discrepancy (171 vs stated 170)" that must be reconciled before appending the 20 v0.4 rows. Direct verification of `matrix/principles-matrix.md` shows the matrix contains **exactly 170 P-rows** (17 domains × 10 rows each), matching the invariant — the discrepancy does not exist. The reconciliation task (04-01-01) and audit test (06-01-02) are built on a wrong premise. This is non-blocking (the verification step is harmless — it will simply confirm 170) but the task wording is misleading and could cause an executor to "fix" a non-existent problem (e.g., deleting a valid row to hit a phantom 171→170). Two binding conditions below fix this before ship.
---
### Axis 1 — Feasibility
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| 1.1 | Can P1P6 actually be delivered with the proposed resources/personas? | PLAN-v0.4.md: 46 tasks across 6 phases. PERSONAS.md: 3 permanent + 2 phase-specific personas, peak 5 active (matches v0.3). Per-phase doc counts: P1=5, P2=5, P3=20, P4=7, P5=5, P6=2. v0.1 P3 shipped 27 derived docs in one phase (RESEARCH-v0.4 §Language-Derived). | Feasible. v0.1 P3 demonstrated 27 derived docs in a single phase; v0.4 P3's 20 language files (4 stubs + 16 derived) is below that watermark, and language-derived docs have a *simpler* structure (section-header-trace, no P1P10 derivation) than domain derived docs. The 5-persona peak matches v0.3 which shipped clean. | 0.85 |
| 1.2 | Are edge/messaging P-rules novel enough to not duplicate existing domains (per D-061/D-062)? | RESEARCH-v0.4 §Edge boundary table: edge P6 (geographic/partition cache invalidation) vs `performance/P5 Caching with Intent` (generic C3,C6) — distinct angle. §Messaging boundary table: messaging P6 (broker-backed backpressure, network failure) vs `concurrency/P9 Bounded Queues` (in-process, OOM failure) — distinct failure model. `concurrency/patterns.md` Pattern 5 explicitly scopes to "producer is blocked or signaled" (in-process); messaging extends to network-partition regime. | Novel. The discriminator is sound: edge owns *where compute runs* (geographic locality, C4); performance owns *generic measurement*; concurrency owns *in-process primitives*; messaging owns *cross-process/network-failure-model*. No verbatim duplication. Each domain owns a distinct angle on shared C-rules — the same pattern that distinguished k8s P8 from iac P3 in v0.2 (D-021 precedent). | 0.86 |
| 1.3 | Is the 16-doc language-derived extension (P3) achievable in one phase? | RESEARCH-v0.4 §Language-Derived: 16 derived docs follow the existing stub's section-header-trace pattern (confirmed in `languages/typescript.md`: `## Type System (C1 Correctness, Data P7 Type Fidelity)`). PLAN-v0.4 P3 Wave 2: 16 tasks at ≤5 concurrent (5-then-5-then-5-then-1, A-008 @ 0.90). D-063: no P1P10 derivation (apply, don't derive) — lower per-doc effort than domain derived docs. | Achievable. Language-derived docs are structurally simpler than domain derived docs (no first-principles derivation, no matrix rows, no decision matrices — they apply existing P-rules). v0.1 P3 produced 27 derived docs with heavier structure. The 16-doc load is within demonstrated throughput and below v0.1's high-water mark. | 0.83 |
**Axis 1 confidence: 0.85.** No challenges.
### Axis 2 — Scope
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| 2.1 | Is v0.4 over-scoped (2 new domains + 16 language docs + matrix + review + examples + cross-links)? Should it split into v0.4 + v0.5? What would you cut? | PROJECT.md v0.4 section: 2 domains (10 P-rules each = 20 matrix rows), 16 language-derived docs (0 matrix rows), matrix/review/examples/manifest extensions. REQUIREMENTS ATELIER-92..117 = 26 reqs. v0.3 had 32 reqs (4 domains, 40 matrix rows) and shipped clean. | **Not over-scoped — smaller than v0.3.** 26 reqs vs v0.3's 32. The 16 language-derived docs are the volume driver but carry zero matrix-row cost (D-063/D-066), so the traceability surface grows by only 20 rows (edge + messaging), the smallest matrix extension since v0.2. Cutting to v0.4a (2 domains) + v0.4b (languages) would double release overhead with no load benefit — P3's 16 docs are simpler per-doc than domain docs. The current single-milestone plan is the right call. | 0.85 |
| 2.2 | What would you cut if forced? | (adversarial) | The simplest 80% version: ship edge + messaging domains + matrix/review (drop the 16 language-derived docs to v0.5). This halves the doc count. **But:** the language-derived extension closes pre-existing MANIFEST drift (languages/ unlisted since v0.1, per D-065/IDEATE-31) and is the v0.3 ROADMAP "Future Milestones" candidate explicitly assigned to v0.4. Deferring it re-opens the 2x-deferral zombie risk flagged in v0.3 G-001. The 16 docs are low-cost (apply, don't derive) — keeping them is correct. | 0.82 |
| 2.3 | Who owns the requirements, and are they frozen? | REQUIREMENTS.md v0.4 reqs ATELIER-92..117 (all pending). Ideation log IDEATE-31..43: 13 accepted, 0 deferred, 0 rejected. Traceability matrix maps phases→reqs. | Frozen post-ideation. 26 reqs across 6 phases. 13 accepted refinements baked into task must-haves (no moving targets). The ideation stage closed with 0 deferred. | 0.88 |
| 2.4 | What is explicitly out of scope? | PROJECT.md lines 4955: tooling/linters, translation, agent adapters, per-domain release artifacts, runtime code. D-020: docs-only. D-063: no new P-rules for languages. | Explicit and enumerated: no runtime code, no tooling, no translation, no new core principles, no new matrix rows for languages, no standalone `.ts`/`.py`/`.go`/`.rs` artifacts (D-020 + ATELIER-110 artifact-types extension). The scope boundary is answerable. | 0.88 |
| 2.5 | Are there hidden requirements disclosed late? | IDEATE-32 → ATELIER-117 (ARCHITECTURE.md v0.4 update — architecture-tree drift, detected during ideation). Verified: `.ciagent/atelier/ARCHITECTURE.md` component tree stops at v0.3 (lines 1419 list only through compliance). | ATELIER-117 is a *detected* hidden requirement, surfaced and added during ideation (not late in delivery). The architecture-tree drift is real (ARCHITECTURE.md has no edge/messaging/languages-derived entries). IDEATE-32 caught it before execution — the correct time to surface it. No undisclosed hidden requirements remain. | 0.85 |
**Axis 2 confidence: 0.86.** No challenges.
### Axis 3 — Cost/Risk
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| 3.1 | 46 tasks across 6 phases — what's the riskiest task? | PLAN-v0.4 task inventory: P1 01-01-01 (edge first-principles), P4 04-01-01 (matrix extension), P4 04-01-02 (domain-coverage with Languages sub-table), P3 03-02a..d (16 derived docs). | The riskiest task is **04-01-01 (matrix extension + IDEATE-34 reconciliation)** — but the risk is not the work (appending 20 rows is mechanical); the risk is the **false premise** that the matrix has 171 rows vs stated 170. Direct verification shows the matrix has **exactly 170 rows** (17 × 10). The reconciliation step is built on a wrong premise. **Mitigation:** see binding decision G-014 — the task must verify 170 exists and append 20 to reach 190; it must NOT "fix" a phantom 171→170. | 0.80 |
| 3.2 | What's the cost of a phase failing mid-way? | NFR milestone, docs-only. All phases produce markdown. No runtime, no infra, no migration. | Cost of a phase failing = re-authoring markdown. P3 (largest) failing mid-way wastes the 16 derived-doc tokens but the work is recoverable (edit + re-run). No cascading infrastructure cost, no data loss, no deployment rollback. Rework is bounded and cheap — the same property that made v0.1/v0.2/v0.3 low-risk. | 0.90 |
| 3.3 | Predictable cost drivers not in the original budget? | PROJECT.md: no runtime code, no infrastructure, no licensing, no external services. RESEARCH-v0.4 §Risks: 10 risks, all content/traceability (no external cost drivers). | None. Docs-only = no licensing, no infra, no security review fees, no support contracts. Token cost is proportional to markdown volume, scoped by REQ count (26). The only cost driver is markdown authoring, which is bounded. | 0.88 |
| 3.4 | Single project-killing risk? | RESEARCH-v0.4 §Risks: orphaned P-rules, edge/messaging duplication, language-derived new-P-rule violation, MANIFEST drift, runtime artifacts, persona explosion, matrix row-count, decision-matrix omission. | The single risk that would undermine the framework's core value: **language-derived docs introducing new P-rules** (D-063 violation) or **orphaned from domain P-rules** (D-066 violation). Either breaks the traceable hierarchy (the unique value). **Mitigation:** languages-engineer persona constraint "apply, don't derive" + agent-checklist language-derived scope check (ATELIER-108) + P3 Verify (no new P-rules, every section traces ≥1 domain P-rule) + P4 Languages Coverage sub-table (exactly 16 rows, IDEATE-42) + P6 domain-expert verification. Well-mitigated with multiple layers. | 0.84 |
**Axis 3 confidence: 0.86.** Challenge: IDEATE-34 false premise (G-014). Auto-resolved with binding condition.
### Axis 4 — Vertical-slice integrity
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| 4.1 | Is each phase independently shippable per D-053? | PLAN-v0.4 invariants: P1 ships edge self-consistent (matrix in P4); P2 ships messaging self-consistent; P3 ships 16 language docs self-consistent; P4 closes traceability + manifest + architecture; P5 closes examples + cross-links; P6 ships release. Tag sequence v0.3.1→v0.3.6. | Each phase is shippable. P1/P2 ship domains whose P-rules trace to core internally (matrix extension is traceability closure, not internal-consistency). P3 ships language docs whose section-headers trace to existing domain P-rules (the Languages Coverage sub-table that aggregates these lands in P4 — but each derived doc is self-consistent because it traces to already-shipped domain P-rules). P4 closes manifest + matrix + architecture. P5 closes examples + cross-links. This mirrors v0.3's D-053 pattern that shipped clean. | 0.84 |
| 4.2 | P1 ships edge domain before matrix lands in P4 — acceptable? | D-053 (v0.3): "matrix rows land in P4 — acceptable because the domain is self-consistent; matrix extension is the traceability closure, not a blocker." v0.2/v0.3 both shipped domain phases before P4 matrix extension. | Acceptable. The edge domain's 10 P-rules each name their C-rule derivation *inline* in `first-principles.md` (per PLAN 01-01-01 must-have). The matrix row is the *aggregate* trace; the inline trace is the *per-rule* detail. A domain with correct inline traces is self-consistent even before the matrix row exists. The matrix in P4 is the verifiable closure, not the source of truth. Precedent: v0.2/v0.3 both did this. | 0.86 |
| 4.3 | P3 ships language docs that trace to domain P-rules — traceable without the matrix update? | D-063/D-066: language-derived docs use section-header-trace pattern (C-rule + Domain P-rule inline). Languages Coverage sub-table lands in P4. `languages/typescript.md` confirms pattern: `## Type System (C1 Correctness, Data P7 Type Fidelity)`. | Traceable without the matrix update. Each language-derived doc's section headers name the traced domain P-rules inline — the traceability is in the doc itself, not the matrix. The Languages Coverage sub-table (P4) is the *aggregate verifiable view*, parallel to how the domain→core matrix is the aggregate view of inline domain traces. A language-derived doc with correct inline section-header traces is self-consistent even before the sub-table exists. | 0.85 |
| 4.4 | Edge↔messaging cross-links dangle at P1 tag — acceptable? | PLAN 01-02-03 (edge/iot.md): "cross-link `messaging/queues` ... this link dangles until P2, acceptable per D-053, verified bidirectional in P5 ATELIER-114 per IDEATE-40". A-010 @ 0.86. | Acceptable but a genuine temporary broken-link state. edge/iot.md (P1) cross-links messaging/queues.md (P2) — at the v0.3.1 tag, that link 404s. This is acceptable per D-053 (cross-links close in P5) and is the same pattern as v0.3 intra-milestone links. **Mitigation:** P5 cross-link audit (05-02-01) explicitly verifies edge↔messaging links resolve in BOTH directions before P6 ship (IDEATE-40). The temporary dangle is a documented, time-bounded state, not a silent defect. | 0.82 |
**Axis 4 confidence: 0.84.** No blocking challenges.
### Axis 5 — Traceability
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| 5.1 | Do all 20 proposed new P-rules trace to ≥1 core C-rule without duplicating existing domain P-rules? | RESEARCH-v0.4 §Edge: P1→C4,C1; P2→C1,C5; P3→C8,C1; P4→C1,C5; P5→C1; P6→C1,C3; P7→C1,C5; P8→C4,C6; P9→C1,C8; P10→C7,C5. §Messaging: P1→C1,C2; P2→C1,C2; P3→C1; P4→C1,C2; P5→C1,C5; P6→C1,C8; P7→C4,C6; P8→C5,C7; P9→C5,C6; P10→C7,C1. All 20 map to ≥1 C-rule. Edge derives 7 C-rules; messaging derives 7 C-rules. `core/first-principles.md` confirms C1C8 definitions. | All 20 trace to ≥1 C-rule. None duplicates an existing domain P-rule verbatim: edge P6 (geographic cache invalidation, C1/C3) ≠ performance P5 (generic caching, C3/C6) — different angle + different C-rule mix. messaging P6 (broker backpressure, C1/C8) ≠ concurrency P9 (in-process bounded queue, C1/C8) — same C-rules but different failure model (network vs OOM), per D-062. The matrix invariant (every P-rule → ≥1 C-rule, no orphans) holds. | 0.87 |
| 5.2 | RESEARCH-v0.4 against `core/first-principles.md` — derivations valid? | C1 Correctness (non-negotiable), C4 Locality (decisions near consequences), C5 Reversibility, C8 Economy (constrained resources). Edge P1 "Proximity is the Design Driver" → C4 Locality (compute near user/data) — valid geographic-locality derivation. Edge P3 "Resources are Constrained" → C8 Economy — valid. Messaging P7 "Partitioning is Intentional" → C4 Locality (ordering locality) — valid. | Derivations are valid. Edge's C4 emphasis (geographic locality) is a sound reading of "decisions and consequences live near each other" — geographic proximity is a locality concern. Messaging's C4 (partition-key = ordering locality) is sound. No forced/contrived derivations detected. The C1 emphasis (correctness under partition/network-failure) is correct — partition tolerance is a correctness property, not a performance preference (edge P1 explicitly states this). | 0.85 |
| 5.3 | RESEARCH-v0.4 against existing domains (concurrency/patterns.md for messaging; performance/observability for edge) — no duplication? | `concurrency/patterns.md` Pattern 1 (Message Passing, in-process channels), Pattern 5 (Bounded Queue, in-process "producer is blocked or signaled"). `performance` P5 Caching (generic C3,C6). RESEARCH-v0.4 §Messaging boundary: messaging owns cross-process broker-backed analog with network-failure semantics. §Edge boundary: edge owns geographic/partition-aware cache invalidation. | No duplication. The failure-model discriminator (D-062) is the cleanest boundary: concurrency fails by OOM/crash (in-process); messaging fails by network partition/broker-restart (cross-system). Edge's cache angle (P6, partition-aware invalidation, C1/C3) is distinct from performance's generic cache (P5, C3/C6). Cross-links are one-directional outward (D-026 extended) — no back-link edits to v0.1/v0.2/v0.3 content. | 0.86 |
| 5.4 | Will the matrix stay at exactly 190 P-rules post-v0.4 (D-059)? | Direct verification: `matrix/principles-matrix.md` currently has exactly **170 P-rows** (17 domains × 10, per awk count). RESEARCH-v0.4 §Matrix: +20 rows (10 edge + 10 messaging). | **CRITICAL FINDING:** The matrix currently has exactly 170 rows, matching the invariant. IDEATE-34/ATELIER-106 refinement claims a "171 vs stated 170 discrepancy" that must be reconciled — **this discrepancy does not exist**. Post-v0.4 will be 170 + 20 = 190, exactly as planned. The reconciliation step (PLAN 04-01-01, 06-01-02) is built on a false premise. See binding decision G-014. | 0.88 |
**Axis 5 confidence: 0.86.** Challenge: IDEATE-34 false premise (G-014) — auto-resolved with binding condition; does not block but must be corrected in task wording.
### Axis 6 — Decision quality
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| 6.1 | Are D-054..D-075 well-grounded? | CLARIFY-v0.4.md D-061..D-069 (avg confidence 0.86, range 0.800.92). PLAN-v0.4.md D-070..D-073 (range 0.830.88). All above 0.80 except D-067 (0.80, messaging persona). PROJECT.md v0.4 section encodes D-054..D-060 (SPECIFY). | Well-grounded. 15 decisions, average 0.85. All ≥0.80. The lowest is D-067 (0.80 — messaging P2 persona choice) which is explicitly justified by deferring to ROADMAP's 2-persona naming + D-022 within-competence precedent + edge-engineer consultation baked into P2 task must-have (D-051 pattern). No decision is below the 0.80 "well-grounded" bar. | 0.85 |
| 6.2 | Any low-confidence (<0.80) decisions that should be revisited? | CLARIFY-v0.4.md: D-067 @ 0.80 is the floor. PLAN-v0.4 assumptions A-006..A-010 (0.820.95). | None below 0.80. D-067 @ 0.80 is the lowest decision; it is justified by the ROADMAP's explicit 2-persona naming (the authoritative signal) and the D-022 precedent. Revisiting it would require a 3rd phase-specific persona (messaging-engineer), which the RESEARCH explicitly rejected to avoid persona-explosion (roster would hit 6, exceeding v0.3's 5 peak). The constraint-baked-into-task mitigation (PLAN 02-01-01 must-have: "P1/P2 demonstrated edge + messaging boundary (D-062)") preserves the distributed-systems lens without the persona. No revisit needed. | 0.83 |
| 6.3 | Any decisions contradict prior decisions (D-001..D-053)? | D-053 (vertical-slice integrity) — honored (PLAN invariants line 7). D-020 (docs-only) — honored (PLAN invariants line 13, no `.ts`/`.py`/`.go`/`.rs` artifacts). D-026 (one-directional cross-links) — honored (PLAN 05-02-01 "no back-link edits to v0.1/v0.2/v0.3 content"). D-025 (2+2 examples) — honored (D-068 confirms). | No contradictions. D-053/D-020/D-026/D-025 are all explicitly carried forward. D-063 (no new P-rules for languages) is consistent with ARCHITECTURE.md hierarchy (`languages/` depend on `domains/`). D-066 (no language matrix rows) preserves D-059's 190 total. The v0.4 decisions extend the v0.1→v0.2→v0.3 convention chain without breaking any prior commitment. | 0.88 |
| 6.4 | Does IDEATE-34 (matrix discrepancy) contradict the actual matrix state? | IDEATE-34 refinement (REQUIREMENTS ATELIER-106 note): "The current matrix has 171 P-rows vs the stated 170 invariant — P4 must reconcile this pre-existing discrepancy." Direct verification: matrix has exactly 170 rows. | **Yes — IDEATE-34 is based on a false premise.** The matrix has 170 rows, not 171. The "reconcile 171→170" instruction in PLAN 04-01-01 and audit 06-01-02 is unfounded. This is a decision-quality defect in the ideation stage: IDEATE-34 (confidence 0.82) asserted a discrepancy without verifying the actual count. **Not a contradiction of a prior decision, but a false factual claim.** See binding decision G-014 — the reconciliation step must be reframed as "verify 170 exists, append 20, confirm 190" — NOT "find and fix a phantom 171st row." | 0.80 |
**Axis 6 confidence: 0.84.** Challenge: IDEATE-34 false premise (G-014).
### Axis 7 — Persona coverage
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| 7.1 | Do edge-engineer + languages-engineer + tech-writer + domain-expert + lead-developer cover all P1P6 authoring needs? | PERSONAS.md: 3 permanent (lead-developer, tech-writer, domain-expert) + 2 phase-specific (edge-engineer P1, languages-engineer P3). PLAN persona assignments: P1 edge-engineer; P2 tech-writer + domain-expert (edge-engineer consult); P3 languages-engineer; P4 domain-expert/tech-writer/lead-developer; P5 edge-engineer + tech-writer + domain-expert; P6 all 5 review. | Coverage is complete. P1 edge (edge-engineer specialist). P2 messaging (tech-writer + domain-expert generalists with edge-engineer consultation for D-062 boundary — baked into task 02-01-01 must-have per D-051 pattern). P3 languages (languages-engineer specialist). P4 matrix/review/manifest/architecture (domain-expert/tech-writer/lead-developer split by artifact type). P5 examples (edge-engineer for edge examples, tech-writer for messaging). P6 all 5 review then remove 2. No phase lacks an assigned author. | 0.84 |
| 7.2 | Is messaging P2 under-resourced (no phase-specific persona per D-067)? | D-067 @ 0.80: messaging P2 authored by tech-writer + domain-expert with edge-engineer consultation. CLARIFY-v0.4 §7: rejected messaging-engineer persona (roster would hit 6, exceeding v0.3's 5 peak). PLAN 02-01-01 must-have: "edge-engineer consultation baked in (D-067, D-051 pattern): P1/P2 demonstrated edge + messaging boundary (D-062)." | Under-resourced *relative to P1/P3* (which have specialists) but **adequately resourced** for the work. Messaging's specialty (delivery semantics, idempotency, ordering) is within domain-expert's cross-cutting competence (correctness/observability derivations the domain-expert already validates). The edge-engineer consultation + task must-have preserves the D-062 boundary. RESEARCH-v0.4 §Messaging provides thorough prior-art (Kafka, RabbitMQ, NATS, Jepsen, Reactive Streams, CloudEvents). The risk is content-quality (specialist depth) not traceability (domain-expert validates that). Mitigation is adequate per the v0.3 G-009 precedent (i18n/compliance shipped without specialists). | 0.78 |
| 7.3 | Is there a key-person dependency? | PERSONAS.md: edge-engineer + languages-engineer are phase-specific, removed post-v0.4. D-051 pattern (constraints baked into task must-haves) survives persona removal. | No key-person risk beyond the v0.3-bounded pattern. Personas are constraint sets, not humans. The languages-engineer's "apply, don't derive" constraint (D-063) is baked into P3 task must-haves AND the agent-checklist (ATELIER-108) AND the P3 Verify block. The edge-engineer's "proximity is correctness" constraint (D-061) is baked into P1 task 01-01-01 must-have. If either persona "fails," the constraints survive in the task specs and review checks. | 0.82 |
**Axis 7 confidence: 0.81.** Challenge: messaging P2 under-resourcing (G-015) — auto-resolved; bounded by RESEARCH prior-art + domain-expert validation, parallel to v0.3 G-009.
### Axis 8 — Test/Verify
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| 8.1 | Are the per-phase Verify blocks in PLAN-v0.4.md sufficient to catch a broken phase before ship? | PLAN-v0.4 each phase has a Verify block: P1 (structural 5 files, behavioral P1P10 trace, security P9/P6, quality cross-links + decision matrices + D-061 boundary). P2 (similar + edge↔messaging link resolution post-P2). P3 (20 files, no new P-rules, no P1P10 on stubs, no runtime artifacts, IDEATE-40 cross-links). P4 (matrix 190, coverage sub-table 16 rows, ARCHITECTURE.md component tree, review parity, MANIFEST authoritative). P5 (4 examples, single-breach citation, bidirectional cross-links). P6 (26 reqs covered, reconstruction, matrix 190, MANIFEST, ARCHITECTURE, audit clean). | Sufficient. Each Verify block has structural + behavioral + security + quality checks tied to specific decisions (D-061/D-062/D-063/D-066/D-068/D-069/IDEATE-40/IDEATE-42). The P4 Verify is the strongest: matrix row-count (190), Languages Coverage sub-table (exactly 16 rows — catches orphan language docs), MANIFEST reconstruction (catches unlisted docs), ARCHITECTURE.md component-tree test (catches architecture-tree drift). P6 audit adds the reconstruction + matrix-count + MANIFEST reconstruction triple-test. | 0.86 |
| 8.2 | What would slip through? | (adversarial) | (a) **Content correctness** — Verify blocks check structure + traceability + cross-links, not technical accuracy of, e.g., the CRDT-vs-LWW decision matrix in sync.md or the Kafka rebalance-strategy enumeration in streams.md. A technically wrong but well-structured doc passes Verify. Mitigation: P6 domain-expert + edge-engineer/languages-engineer review (D-052), but this is one layer, not redundant. (b) **The IDEATE-34 false premise** — the P4/P6 "reconcile 171→170" check is unverifiable against a non-existent discrepancy; an executor could misinterpret it as "delete a row" (harmful) or "confirm 170" (correct). G-014 fixes the wording. (c) **Edge↔messaging dangling links at P1 tag** — structurally acceptable (D-053) but a real broken-link state until P2; only caught bidirectionally in P5, not at the P1/P2 tags. | 0.78 |
| 8.3 | Is there a working definition of done? | PLAN-v0.4 P6 Verify: 26 reqs covered, reconstruction test passes, matrix exactly 190 (reconciled), MANIFEST reconstruction (incl. Languages section), ARCHITECTURE.md component-tree test, audit clean, tag v0.3.6, branches deleted, personas removed (roster=3). | DoD is concrete and testable: 26 reqs covered, matrix = 190 rows (10 × 19), MANIFEST reconstruction passes (incl. Languages), tag v0.3.6 on main. Not "whatever the demo shows." The matrix-count test (190) and MANIFEST reconstruction are mechanical, automatable checks. | 0.88 |
**Axis 8 confidence: 0.84.** Challenges: content-correctness gap (inherent to docs projects, bounded by P6 review), IDEATE-34 wording (G-014), dangling links (G-016). All auto-resolved.
### Axis 9 — Cross-link / matrix integrity
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| 9.1 | Will the matrix stay at exactly 190 P-rules post-v0.4 (D-059)? | Direct verification: matrix currently has exactly 170 rows. RESEARCH-v0.4 §Matrix: +20 (10 edge + 10 messaging). Languages add 0 (D-063/D-066). | Yes — 170 + 20 = 190. The invariant holds. The IDEATE-34 "171 vs 170" claim is false; see G-014. Post-v0.4 the matrix will have exactly 190 rows (10 per domain × 19), the smallest extension since v0.2. | 0.88 |
| 9.2 | Will the languages/ sub-table in domain-coverage.md capture all 16 language-derived docs? | ATELIER-107 (IDEATE-42): Languages Coverage sub-table with exactly 16 rows. RESEARCH-v0.4 §Language-Derived sketches the 16-row sub-table (TS quartet + PY quartet + GO quartet + RS quartet). PLAN 04-01-02 must-have: "EXACTLY 16 rows (one per language-derived doc)". P4 Verify: "Languages Coverage sub-table has exactly 16 rows". P6 06-01-01: "domain-expert verifies ... 16 Languages Coverage sub-table rows match the derived docs' actual section-header traces (IDEATE-42)". | Yes — the sub-table is specified at exactly 16 rows, verifiable as a P4 review check, and cross-checked in P6 against the actual section-header traces. The 16 = 4 languages × 4 derived docs. No orphans (every row traced to ≥1 domain P-rule per D-066). The sub-table is the verifiable aggregate; inline section-headers are per-doc detail. | 0.86 |
| 9.3 | Will the bidirectional edge↔messaging cross-links actually be verified (IDEATE-43/IDEATE-40)? | PLAN 05-02-01 (cross-link audit): "(b) edge↔messaging intra-v0.4 links resolve in BOTH directions — every edge→messaging and messaging→edge link resolves by the P6 ship". P5 Verify: "edge↔messaging intra-v0.4 links resolve bidirectionally (IDEATE-40)". RESEARCH-v0.4 §Review: edge/iot.md → messaging/queues.md (MQTT QoS parallels). | Yes — the P5 cross-link audit (05-02-01) explicitly verifies bidirectional edge↔messaging resolution, AND language-derived docs link to both new (edge/messaging) and existing domains where relevant (e.g., ts-async → messaging/delivery-semantics; go-concurrency → concurrency/patterns + messaging/queues). The audit is a dedicated Wave 2 task after all docs exist, so links are resolvable. The one gap: at the P1 tag (v0.3.1), edge→messaging links dangle (messaging authored in P2) — acceptable per D-053, time-bounded, verified in P5. | 0.84 |
| 9.4 | MANIFEST integrity — will the Languages section close the pre-existing drift? | MANIFEST.md verified: currently has NO `## Languages` section (sections: Reading Order, Core, Domains, Examples, Cross-Cutting). `languages/{typescript,python,go,rust}.md` exist since v0.1 but are unlisted. D-065/IDEATE-31: add `## Languages` between `## Domains` and `## Examples`. PLAN 04-03-01 must-have: lists 4 stubs + 16 derived docs + Cross-Cutting = "19 domains, 190 P-rules post-v0.4". | Yes — the Languages section is specified to list 4 v0.1 stubs (closing pre-existing drift parallel to ESC-002/ATELIER-91) + 16 new derived docs, positioned per reading order (core→domains→languages→examples). P4 Verify: "MANIFEST is authoritative — every v0.4 file listed, languages/ listed closing pre-existing drift". P6 MANIFEST reconstruction test catches any unlisted doc. | 0.87 |
**Axis 9 confidence: 0.86.** Challenge: dangling links at P1 tag (G-016) — auto-resolved; time-bounded + P5-verified.
### Meta — Closing Review
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| M.1 | If you were the auditor, what would you flag? | (adversarial) | **The IDEATE-34 false premise** — the plan and audit instruct executors to "reconcile a pre-existing 171-vs-170 matrix discrepancy" that does not exist (verified: exactly 170 rows). This is the single most material finding: a verification gate built on a wrong fact. An executor could (a) waste effort hunting a phantom row, or (b) worse, delete a valid row to "fix" 171→170, breaking the invariant. G-014 fixes the wording to "verify 170, append 20, confirm 190." Secondary: content-correctness is one-layer-validated (P6 review only) — inherent to docs projects, bounded. | 0.82 |
| M.2 | What is the project NOT doing that it should? | PROJECT.md out-of-scope: tooling/linters. | Not doing: automated P-rule→C-rule trace validation (manual domain-expert verification only). At 190 P-rules + 16 language-derived traces, manual verification is approaching the limit where tooling would be warranted — but remains feasible for this milestone. Explicitly out of scope; not a v0.4 blocker. The Languages Coverage sub-table (16 rows) actually *enables* future tooling by making language traces machine-readable. | 0.82 |
| M.3 | Simplest 80% version? | (adversarial) | Ship edge + messaging domains + matrix/review/manifest/architecture (drop the 16 language-derived docs to v0.5). This cuts P3 entirely (20 files → 0) and reduces the milestone to ~10 reqs. **But:** the language-derived extension closes pre-existing MANIFEST drift (D-065) and is the v0.3 ROADMAP "Future Milestones" candidate explicitly assigned to v0.4. Deferring re-opens the 2x-deferral zombie risk (v0.3 G-001). The 16 docs are low-cost (apply-don't-derive, no matrix rows). Keeping them is the right scope call. | 0.83 |
| M.4 | What must be true in 90 days for success, and is it true today? | (adversarial) | Must be true: (a) 20 new P-rules trace cleanly to C-rules — RESEARCH-v0.4 pre-maps them, P4 verifies. (b) 16 language-derived docs each trace to ≥1 domain P-rule (no orphans) — P3 Verify + P4 sub-table (16 rows) + P6 cross-check. (c) Content is technically correct (CRDT/LWW, Kafka rebalance, MQTT QoS, delivery semantics) — RESEARCH grounds it, P6 validates. (d) Matrix = exactly 190 — verified today at 170, +20 planned. (a)(b)(d) have explicit verification gates with correct premises (post-G-014). (c) depends on execution quality. All achievable. | 0.83 |
**Meta confidence: 0.83.**
---
## Summary Table
| # | Axis | Verdict | Confidence |
|---|------|---------|------------|
| 1 | Feasibility | PASS | 0.85 |
| 2 | Scope | PASS | 0.86 |
| 3 | Cost/Risk | CONDITIONAL (G-014: fix IDEATE-34 wording) | 0.86 |
| 4 | Vertical-slice integrity | PASS | 0.84 |
| 5 | Traceability | CONDITIONAL (G-014: false premise in reconciliation step) | 0.86 |
| 6 | Decision quality | CONDITIONAL (G-014: IDEATE-34 factual defect) | 0.84 |
| 7 | Persona coverage | PASS (G-015 noted, bounded) | 0.81 |
| 8 | Test/Verify | CONDITIONAL (G-014: audit test premise; G-016: dangling links acknowledged) | 0.84 |
| 9 | Cross-link / matrix integrity | PASS | 0.86 |
| — | Meta | PASS | 0.83 |
| **—** | **Overall** | **CONDITIONAL PROCEED** | **0.83** |
---
## Binding Decisions
| ID | Decision | Rationale | Confidence | Alternatives |
|----|----------|-----------|------------|--------------|
| G-013 | Proceed with v0.4 scope as planned (2 domains + 16 language-derived docs); do NOT split into v0.4/v0.5 | 26 reqs < v0.3's 32 reqs; language-derived docs carry 0 matrix-row cost (D-063/D-066); v0.1 P3 demonstrated 27 derived docs in one phase; deferral re-opens 2x-zombie risk (v0.3 G-001); the 16 docs are the v0.3 ROADMAP "Future Milestones" candidate explicitly assigned to v0.4 | 0.85 | Split into v0.4a (domains) + v0.4b (languages) — rejected: double release overhead, no load benefit; Drop languages to v0.5 — rejected: 2x deferral zombie risk |
| G-014 | **CONDITION — fix IDEATE-34 false premise before P4 executes.** The matrix currently has exactly 170 P-rows (verified: 17 domains × 10). There is NO "171 vs 170 discrepancy." PLAN task 04-01-01 and audit 06-01-02 must be reworded from "reconcile pre-existing 171→170 discrepancy" to "verify matrix has exactly 170 rows before appending 20; confirm post-append count = 190." The executor must NOT hunt for or delete a phantom 171st row. | Direct verification of `matrix/principles-matrix.md`: `awk '/^## /{...} /^\| P[0-9]/{count++}'` returns 170 total, 15 lines per section header (10 P-rows + header rows), 17 domain sections. IDEATE-34 (confidence 0.82) asserted the discrepancy without verifying the actual count — a factual defect in the ideation stage. The reconciliation instruction is unfounded and risks an executor breaking the invariant by deleting a valid row. | 0.88 | Leave the wording as-is — rejected: an executor could delete a valid row to "fix" a non-existent discrepancy; Remove the reconciliation step entirely — rejected: keeping a verify-170 step is harmless and good practice, just reworded |
| G-015 | Proceed with tech-writer + domain-expert for messaging P2 without a messaging-engineer persona (confirms D-067) | Messaging's specialty (delivery semantics, idempotency, ordering) is within domain-expert's cross-cutting competence; edge-engineer consultation + task must-have (02-01-01) preserves D-062 boundary; RESEARCH-v0.4 §Messaging provides thorough prior-art (Kafka/RabbitMQ/NATS/Jepsen/Reactive Streams/CloudEvents); v0.3 G-009 precedent (i18n/compliance shipped without specialists); adding a 3rd phase-specific persona would push roster to 6, exceeding v0.3's 5 peak (persona-explosion risk). Assumption A-011 logged: messaging content correctness depends on RESEARCH prior-art + P6 review. | 0.78 | Add messaging-engineer persona — rejected: persona explosion (roster 6 > v0.3 peak 5); Defer messaging to v0.5 — rejected: scope-creep deferral |
| G-016 | The edge↔messaging dangling cross-link at P1 tag (edge/iot.md → messaging/queues.md) is acceptable per D-053; the P5 audit (05-02-01) MUST verify bidirectional resolution before P6 ship (confirms IDEATE-40) | D-053 established that cross-links close in P5, not at the domain-phase tag; v0.3 had the same intra-milestone dangling-link pattern; the dangle is time-bounded (resolves at P2 tag v0.3.2); P5 audit explicitly checks both directions (IDEATE-40). Not a silent defect — a documented, verified state. | 0.82 | Block P1 ship until messaging exists — rejected: breaks vertical-slice integrity (D-053), serializes independent work; Remove the edge→messaging link from P1 — rejected: loses the MQTT QoS / delivery-semantics parallel, a high-value cross-link |
| G-017 | CONFIRM D-063/D-066: language-derived docs introduce NO new P-rules and NO new matrix rows; the Languages Coverage sub-table (exactly 16 rows) is the verifiable aggregate | Direct verification of `languages/typescript.md` confirms the section-header-trace pattern (`## Type System (C1 Correctness, Data P7 Type Fidelity)` — no P1P10 numbering). ARCHITECTURE.md confirms hierarchy (`languages/` depend on `domains/`). Adding P-rules to languages would break the hierarchy and the 190-row invariant. The 16-row sub-table is the right traceability home (parallel to the per-domain row schema). | 0.87 | Add P1P10 to languages — rejected: breaks hierarchy + matrix invariant; Add language rows to principles-matrix.md — rejected: languages have no P-rules to map (D-063) |
| G-018 | CONFIRM D-065/IDEATE-31: MANIFEST gains a `## Languages` section closing pre-existing drift since v0.1 (parallel to ESC-002/ATELIER-91) | MANIFEST.md verified: NO `## Languages` section exists despite `languages/{typescript,python,go,rust}.md` existing since v0.1. This is pre-existing drift on the exact ESC-002 precedent (examples/ was unlisted, closed in v0.3). v0.4 touches `languages/` for the first time since v0.1, so it closes the drift. | 0.87 | Defer stubs to a future milestone, list only 16 new docs — rejected: half-closes the drift, inconsistent with ESC-002 precedent; List languages under `## Domains` — rejected: languages are not domains (ARCHITECTURE.md hierarchy) |
### Escalations
**None.** All challenges auto-resolved at confidence ≥ 0.78 (range: 0.780.88). At full autonomy, the IDEATE-34 false premise (G-014) is the lowest-confidence finding but is auto-resolvable by rewording the task (not an escalation — it does not require a human decision, just a factual correction). No axis scored below 0.60 on any forcing question.
### Assumptions Logged (this grill)
| # | Assumption | Confidence |
|---|-----------|------------|
| A-011 | Messaging P2 content correctness (delivery-semantics, Kafka rebalance, MQTT QoS) depends on RESEARCH-v0.4 prior-art quality + P6 domain-expert/edge-engineer review, not on a specialist messaging-engineer persona. If P6 surfaces fundamental content errors, the remedy is P2 rework (bounded — markdown edits). Parallel to v0.3 A-006 (i18n/compliance without specialists). | 0.78 |
### Conditions for SHIP (CONDITIONAL verdict)
The phase-0 plan is sound and P1 may proceed immediately. The following condition MUST be addressed before P4 executes (it does not block P1/P2/P3):
1. **G-014 (mandatory):** Reword the IDEATE-34/ATELIER-106 reconciliation instructions in PLAN-v0.4 task 04-01-01 and audit task 06-01-02. The matrix currently has exactly 170 rows — there is no 171-vs-170 discrepancy. The task must say "verify matrix has exactly 170 rows; append 20 (10 edge + 10 messaging); confirm post-v0.4 count = 190" — NOT "reconcile a pre-existing 171→170 discrepancy." This is a factual correction to prevent an executor from hunting for or deleting a phantom row. The orchestrator should apply this rewording before P4 (or the P4 executor should interpret "reconcile" as "verify 170 exists" rather than "find a missing row").
No other conditions block ship. The plan is the 4th consecutive NFR milestone using the same proven structure; all prior milestones shipped clean.
---
## Overall Verdict
**CONDITIONAL PROCEED** at overall confidence **0.83**.
The v0.4 phase-0 plan (spec → clarify → research → ideate → plan → grill) is sound: the 26-req scope is smaller than v0.3's 32, the two new domains have well-reasoned boundaries (D-061 proximity angle; D-062 cross-process failure model), the 16-doc language extension carries zero matrix-row cost, all 20 new P-rules trace to ≥1 C-rule, and the verification gates (matrix=190, Languages sub-table=16 rows, MANIFEST reconstruction, ARCHITECTURE.md component-tree test, bidirectional edge↔messaging cross-link audit) are concrete and testable.
The single CONDITIONAL is the **IDEATE-34 false premise** (G-014): the plan instructs executors to "reconcile a pre-existing 171-vs-170 matrix discrepancy" that direct verification shows does not exist (the matrix has exactly 170 rows). This is a factual defect in the ideation stage, not a structural flaw. It is non-blocking for P1/P2/P3 (which don't touch the matrix count) and is fixed by rewording the P4/P6 task instructions to "verify 170, append 20, confirm 190" — preventing an executor from breaking the invariant by deleting a valid row to fix a phantom discrepancy.
**P1 may proceed immediately.** The G-014 rewording must land before P4 executes. No escalation to human required (confidence 0.83 ≥ 0.60).
+175
View File
@@ -0,0 +1,175 @@
# Atelier — Grill (Adversarial Red-Team Review)
> Pre-execution gate for milestone v0.3 (GitOps + Operators + AI/ML + i18n + Compliance).
> Default assumption: the project is unfeasible, over-scoped, and too costly. Not convinced until evidence forces it.
> Mode: full autonomy. Auto-resolve at confidence ≥ 0.60; escalate only < 0.60 that cannot be auto-resolved.
---
## v0.3 Grill — 2026-08-05
**Milestone:** v0.3 — GitOps + Operators + AI/ML + i18n + Compliance
**Phase:** 0 (Pre-Execution, GRILL stage)
**Grill scope:** all 9 axes + meta
**Prior grill runs:** none (v0.1/v0.2 P0 stages did not include a GRILL stage; this is the first)
### Verdict: **PROCEED** (confidence 0.80)
The project is ambitious but deliberately bounded. Scope has been trimmed in three places (D-023 AI/ML engineering-only, D-024 compliance framework-agnostic, D-025 2+2 examples). Two prior milestones (v0.1: 35 reqs, v0.2: 24 reqs) shipped clean with the same docs-only NFR structure. The 4-domain scope is the largest single milestone yet, but per-phase derived-doc counts (P1: 4, P2: 4, P3: 8) are lower than v0.1's P3 (27 derived docs in one phase). Traceability verification is explicit (P4 matrix row-count test: 10 × 17 = 170; P5 cross-link audit). The one residual risk — P3 i18n/compliance content authored without specialist personas — is bounded by RESEARCH.md prior-art depth and P6 domain-expert review, with rework being cheap (markdown edits).
---
### Axis 1 — The Business Case Itself
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| 1.1 | What problem does this solve, and is it still the top priority? | RESEARCH.md: "Atelier's differentiation: traceable principle hierarchy with a join table. Existing frameworks state principles; none provide a matrix mapping every domain rule back to a core rule." v0.1/v0.2 shipped (59 reqs covered). | A first-principles engineering framework with a traceable principle matrix — the join table is the unique value. v0.3 extends the domain catalog (GitOps, AI/ML, i18n, compliance) that v0.2 deferred (IDEATE-15/16). Still the top priority: the 4 domains were explicitly deferred to v0.3, not abandoned. | 0.78 |
| 1.2 | Who is the named executive sponsor, and when did they last decide under pressure? | git log: all commits by Jon Chery (jchery@jccapital.xyz). config.json autonomy: "full". | Single-owner project. The "sponsor" is the owner-operator. The last decision under pressure: v0.2 ESC-001/ESC-002 (git transport auth failure, stale tags) — resolved at full autonomy (AUDIT-P5-resync.md). Not a committee-driven project; no sponsor-stall risk. | 0.80 |
| 1.3 | What happens to the business if the project is cancelled? | PROJECT.md objective; MANIFEST.md current state (13 domains, 130 P-rules post-v0.2). | The framework remains at v0.2 (13 domains). The 4 deferred domains (GitOps, AI/ML, i18n, compliance) stay deferred — a 2x deferral that risks zombie status. Cancellation is worse than proceeding: the deferral was already made once. | 0.82 |
| 1.4 | Is the ROI calculated against a counterfactual? | N/A — docs-only project, no monetary budget. Cost = agent tokens + time. | The counterfactual is "agents and humans have no canonical principle reference for GitOps/AI-ML/i18n/compliance." The ROI is framework coverage. Two prior milestones validated the cost model (docs-only, no infra). | 0.75 |
**Axis 1 confidence: 0.79.** No challenges. Auto-resolve.
### Axis 2 — Scope and Requirements
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| 2.1 | Is the scope expanding, contracting, or stable? | PROJECT.md D-016 (4 domains grouped to limit release overhead), D-023 (AI/ML = engineering discipline, NOT algorithm design), D-024 (compliance framework-agnostic, NOT regulation-specific), D-025 (2+2 examples, not 4+4). | Scope is **expanding** (4 new domains, the largest milestone yet) but **deliberately trimmed** in three places. D-023/D-024/D-025 are scope-discipline decisions, not scope-creep. The trims reduce the surface from a hypothetical 4+4 examples + regulation-specific compliance docs + algorithm-design AI/ML to a bounded 2+2 + framework-agnostic + engineering-only. | 0.80 |
| 2.2 | Who owns the requirements, and are they frozen? | REQUIREMENTS.md ATELIER-60..91 (32 reqs, all pending). Traceability matrix maps phases→reqs. Ideation log (IDEATE-17..30) shows 14 accepted, 0 deferred, 0 rejected. | Requirements are frozen post-ideation. 32 reqs across 6 phases. The ideation stage closed with 0 deferred — everything generated is in v0.3 scope. No moving targets. | 0.85 |
| 2.3 | What is explicitly out of scope? | PROJECT.md lines 49-55: tooling/linters, translation of framework docs, agent integration adapters, per-domain release artifacts, runtime code. D-023: algorithm/model design. D-024: regulation-specific compliance docs. | Out of scope is explicit and enumerated: no runtime code, no tooling, no translation, no regulation-specific docs, no algorithm design. The scope boundary is answerable — not infinite. | 0.88 |
| 2.4 | Are there hidden requirements disclosed late? | RESEARCH.md: ai-ml cross-links to data/ (schema, migrations) but D-023 scopes AI/ML to engineering discipline, not data engineering. PERSONAS.md: data-engineer explicitly inactive ("No database, schema, or migrations in this docs-only project"). | No hidden requirements detected. AI/ML does NOT imply a data-engineer persona — D-023's engineering-discipline scope excludes data engineering (schema/ETL/pipelines). The cross-link to data/ is a reference, not a duplication. | 0.85 |
**Axis 2 confidence: 0.84.** Challenge: is 4-domain scope too ambitious for one milestone? → Auto-resolved (G-001). Challenge: hidden data-engineer persona requirement? → Auto-resolved (G-008).
### Axis 3 — Architecture and Technical Feasibility
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| 3.1 | Has the architecture been validated by builders, not just sellers? | RESEARCH.md: prior-art surveys for all 4 domains (OpenGitOps Principles v1.0.0, Sculley "Hidden Technical Debt", ICU/CLDR/BCP 47, NIST/SOC2/OPA). ARCHITECTURE.md: v0.3 domain additions section with cross-link map. | The architecture (hierarchical doc tree + matrix join table) is validated by 2 prior shipped milestones. The 4 new domains follow the same v0.1/v0.2 contract: 10 P-rules each, trace to ≥1 C-rule, docs-only. RESEARCH.md pre-maps all 40 P-rules to C-rules before execution. | 0.85 |
| 3.2 | What is the integration surface? | ARCHITECTURE.md: cross-links one-directional (D-033). RESEARCH.md: gitops→6 domains, ai-ml→6 domains, i18n→4 domains, compliance→6 domains. D-033: no back-link edits to v0.1/v0.2 content. | The integration surface is cross-links only — one-directional outward from new domains to existing. No back-link edits to v0.1/v0.2 content (D-033). This minimizes churn. Every new derived doc requires ≥1 outbound cross-link (ATELIER-88, verified in P5). | 0.86 |
| 3.3 | Is there an existing system being replaced? What is the data volume? | MANIFEST.md: 13 domains, 130 P-rules post-v0.2. matrix/principles-matrix.md: 212 lines, 13 domain sections. | No system is replaced — the framework is extended. Matrix grows 130→170 P-rules. At 170 rows, the matrix is a large but single readable markdown file. domain-coverage.md provides the C-rule→domains navigation view. The matrix is the arbiter; its size is linear with domains. | 0.84 |
| 3.4 | What technical debt is inherited? | AUDIT-P5-resync.md §6: "examples/ not in MANIFEST — pre-existing, candidate for v0.3." ATELIER-91, D-035: add examples/ listing to MANIFEST in P4. | One piece of inherited debt: examples/ unlisted in MANIFEST (ESC-002 convention note from v0.2 audit). v0.3 closes this explicitly (ATELIER-91, D-035, task 04-02-05). No other drift identified (A-001, confidence 0.85). | 0.85 |
| 3.5 | Compliance has NO C4 (Locality) trace (D-032) — is that an architectural smell? | RESEARCH.md D-032: "compliance is inherently cross-cutting, not local." Compliance P-rules trace to C1,C2,C3,C5,C6,C7,C8 (7 of 8). Alternative considered: "force C4 via audit-log locality" — rejected. | **Not a smell.** C4 (Locality) is about keeping concerns local to their context. Compliance is the opposite — it is inherently system-wide (audit logs span the whole system, retention policy is global, evidence is cross-cutting). Forcing a C4 trace would be a false derivation. D-032's rationale is architecturally sound: the absence reflects the domain's nature, not a gap. | 0.82 |
| 3.6 | Is the docs-only constraint (D-020) defensible at 17 domains? | PROJECT.md constraint: "no runtime code." MANIFEST.md is the authoritative index. ARCHITECTURE.md: reading-order guides consumption. matrix/principles-matrix.md is the join table. | **More defensible at 17 domains, not less.** The docs-only constraint is what makes the framework scalable: no runtime complexity, no integration surface, no deployment, no build step. At 17 domains, the MANIFEST + reading-order + matrix make the tree navigable. The framework's value (traceable hierarchy) scales linearly — more domains = more value, as long as traceability holds. A build step or runtime would violate docs-as-code simplicity and add the exact integration surface the framework avoids. | 0.88 |
**Axis 3 confidence: 0.85.** Challenges: C4 gap (G-006), docs-only at 17 domains (G-007), matrix orphan risk (G-003), manifest drift (G-011). All auto-resolved.
### Axis 4 — People, Skills, and Organization
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| 4.1 | Which 2-3 people, if they left, would the project fail? | PERSONAS.md: 5 active personas (lead-developer, tech-writer, domain-expert + 2 phase-specific: platform-engineer, ml-engineer). D-051: ml-engineer constraints baked into P5 task must-have. | In an AI-agent docs project, "personas" are constraint sets, not human employees. The key-person risk is lower than in runtime projects. D-051 ensures ml-engineer constraints survive into P5 (examples) even though the persona is removed after P2. The constraint is baked into the task must-have, not the persona's continued presence. | 0.80 |
| 4.2 | Are resources allocated at the claimed percentages? | config.json: max_concurrent_agents=5. PLAN.md: P3 Wave 2 splits 8 tasks into 2a/2b, capped at 5 concurrent (D-049). | Allocation is mechanical — the executor schedules ≤5 concurrent per config.json. P3's 8 derived docs run as 5-then-3 (A-003, confidence 0.90). P4 runs exactly 5 concurrent (D-050). No "in name only" allocation — this is agent execution, not human BAU fire-fighting. | 0.85 |
| 4.3 | Is there a product owner with actual authority? | git log: single author (Jon Chery). config.json autonomy: "full". PROJECT.md: D-001..D-053 decision table. | Single owner-operator with full autonomy. No committee. Decisions are transparent (D-001..D-053 with confidence scores). Authority is unambiguous. | 0.85 |
| 4.4 | Is the team building capability they don't have? | PERSONAS.md: P3 (i18n + compliance) authored by tech-writer + domain-expert — NO specialist persona (D-022). RESEARCH.md: i18n covers ICU/CLDR, BCP 47, UAX #9 bidi algorithm; compliance covers OPA/Cedar/Kyverno/Sentinel, Cosign/in-toto. | **This is the most material risk.** P3's i18n (RTL/bidi, UAX #9) and compliance (policy-as-code engine semantics) have genuine specialist depth. The tech-writer persona authored v0.1's security/data/concurrency domains without specialists and shipped clean — but those are more universally known than bidi algorithms and Rego semantics. **Mitigation:** RESEARCH.md provides thorough prior-art surveys with specific source citations (unicode.org, W3C i18n WG, openpolicyagent.org, kyverno.io). The P6 review includes domain-expert validation. Rework, if needed, is bounded (markdown edits, not infrastructure). **Assumption logged (A-006):** i18n RTL/bidi and compliance policy-as-code content correctness depends on RESEARCH.md prior-art quality + P6 review, not on a specialist persona. | 0.72 |
| 4.5 | Are the phase-specific personas (platform-engineer, ml-engineer) a key-person risk? | PERSONAS.md: both are phase-specific, removed post-v0.3. D-027: platform-engineer reused/extended from v0.2. D-028: ml-engineer new. D-051: constraints baked into task must-haves. D-052: both review their content in P6 before removal. | **Not a key-person risk.** Personas are constraint sets, not people. Platform-engineer is a proven reuse from v0.2 (shipped clean). Ml-engineer is new but its constraints are explicitly enumerated and baked into task must-haves (D-051). Both review their content in P6 before removal (D-052). If either "fails," the fallback is tech-writer + domain-expert + RESEARCH.md grounding. | 0.80 |
**Axis 4 confidence: 0.80.** Challenge: P3 specialist-persona competency gap (G-009). Auto-resolved with assumption A-006.
### Axis 5 — Timeline and Estimates
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| 5.1 | Was the deadline set before or after scope/approach were understood? | git log: specify (c96d21c) → clarify (675abb6) → research (0620c94) → ideate (1a0326b) → plan (d8473d6). PLAN.md created after research + ideation. | The phase plan was created AFTER research and ideation — the scope was understood before the plan was written. No reverse-engineered deadlines. The "timeline" is the phase sequence P0→P6, not an external date. | 0.88 |
| 5.2 | What is the critical path, and what would push it by 3+ months? | PLAN.md: P1→P2→P3→P4→P5→P6, all sequential. P4 (matrix) depends on all domains. P5 (examples) depends on P4 (manifest). | Critical path: P1→P2→P3→P4→P5→P6. **Nothing can push it by 3+ months** — this is a docs project with no external dependencies, no infrastructure provisioning, no vendor lead times. The only "push" is content-quality rework, which is bounded (markdown edits). | 0.90 |
| 5.3 | Are estimates evidence-based? | v0.1: 35 reqs, 7 phases, shipped. v0.2: 24 reqs, 5 phases, shipped. v0.3: 32 reqs, 6 phases. Per-phase doc counts: P1=5, P2=5, P3=10, P4=7, P5=4, P6=2. | Estimates are analogous (v0.1/v0.2 shipped with similar per-phase doc counts). v0.1 P3 produced 27 derived docs in one phase; v0.3's largest phase (P3) produces 10. The estimate is conservative relative to v0.1's demonstrated throughput. | 0.85 |
| 5.4 | Is there a working definition of done? | PLAN.md P6 Verify: all 32 reqs covered, reconstruction test passes, matrix row-count test (170), MANIFEST reconstruction test (incl. examples/), audit clean, tag v0.2.6 exists, branches deleted. | DoD is concrete and testable: 32 reqs covered, matrix = 170 rows (10 × 17), MANIFEST reconstruction passes, tag v0.2.6 on main. Not "whatever the demo shows." | 0.88 |
**Axis 5 confidence: 0.88.** No challenges.
### Axis 6 — Budget and Financial Realism
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| 6.1 | What % of budget is spent vs remaining? | N/A — docs-only, no monetary budget. Cost = agent tokens. v0.1/v0.2 completed within expected token bounds. | No monetary budget. Token cost is proportional to markdown authored. v0.3 is the largest milestone (32 reqs, 18 derived docs + 4 examples + extensions), but per-doc token cost is roughly constant and validated by 2 prior milestones. | 0.85 |
| 6.2 | Are there predictable cost drivers not in the original budget? | PROJECT.md: no runtime code, no infrastructure, no licensing, no external services. | None. Docs-only = no licensing, no infra, no security review fees, no data migration, no support contracts. The only cost driver is markdown volume, which is scoped by REQ count. | 0.90 |
| 6.3 | Burn rate and runway? | N/A — no monetary burn. Agent execution time is the only resource. | No monetary runway concern. Agent execution is bounded by phase task count. | 0.88 |
| 6.4 | Is the budget contingent on something? | config.json: no contingent conditions. | No. The project is not contingent on a sale, board approval, or hiring. | 0.90 |
**Axis 6 confidence: 0.88.** No challenges. The docs-only constraint makes this axis low-risk by construction.
### Axis 7 — Risks, Assumptions, and Dependencies
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| 7.1 | Top 3 assumptions the plan rests on? | PLAN.md Assumptions: A-001 (ESC-002 is the only manifest drift, 0.85), A-002 (4 domains map to C1-C8 without new core principles, 0.95), A-003 (P3 Wave 2 schedules 5-then-3, 0.90). | A-002 is the strongest (0.95) — RESEARCH.md pre-maps all 40 P-rules to existing C-rules. A-001 is reasonable (AUDIT-P5-resync.md confirms). A-003 is mechanical (config.json cap). All three have evidence. | 0.86 |
| 7.2 | External dependencies? | PROJECT.md: no runtime code. RESEARCH.md: all prior art is published (OpenGitOps, ICU/CLDR, NIST, OPA docs). | None. No vendor, regulator, or external team dependency. All prior art is published and cited. The framework is self-contained markdown. | 0.92 |
| 7.3 | Single project-killing risk? | RESEARCH.md Risks table: orphaned P-rules, domain overlap, AI/ML scope drift, compliance bloat, artifact leakage, persona explosion. | The single risk that would undermine the framework's core value: **matrix orphan P-rules.** If the 40 new P-rules don't trace cleanly to C-rules, the traceable hierarchy (the unique value proposition) is broken. **Mitigation:** P4 matrix row-count test (10 per domain × 17 = 170) + domain-expert sign-off + P6 reconstruction test. This is well-mitigated and explicitly verified. | 0.84 |
| 7.4 | Pre-mortem: 12 months from now, v0.3 failed. Why? | (adversarial analysis) | **Most likely failure:** P3 content quality — i18n RTL/bidi or compliance policy-as-code authored without specialist personas contains fundamental technical errors (e.g., bidi isolating run misuse, Rego evaluation model mischaracterization), caught in P6 review, requiring P3 rework. **Secondary:** the 2+2 example set leaves 2 domains without a good example (gitops + ai-ml get good examples; i18n + compliance get only bad examples), reducing adoption value. **Both are bounded** — rework is markdown edits, not infrastructure. Neither is project-killing. | 0.78 |
**Axis 7 confidence: 0.85.** Challenge: pre-mortem P3 content quality (G-012). Auto-resolved.
### Axis 8 — Governance, Decision-Making, and Communication
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| 8.1 | Who is the decision-maker when executives disagree? | git log: single author. config.json: autonomy "full". | Single owner-operator. No executive disagreement possible. Decisions are recorded in PROJECT.md (D-001..D-053) with confidence scores. | 0.88 |
| 8.2 | How often does governance meet, and what's the escalation pattern? | config.json: autonomy level "full", escalation_hooks ["deploy", "delete_data", "merge_to_main"], decision_confidence_threshold 0.6, escalation_timeout_ms 300000. | Governance is the ciagent workflow itself (specify→clarify→research→ideate→plan→grill→execute→review→audit→ship). Escalation: 3 hooks (deploy, delete_data, merge_to_main) + timeout 300s. The grill stage (this) is the pre-execution gate. Cadence is event-driven, not calendar-driven — appropriate for agent execution. | 0.82 |
| 8.3 | What is omitted from status reports? | AUDIT-P5-resync.md: candid about ESC-001/ESC-002 (git auth failure, stale tags). Convention note about examples/ MANIFEST drift flagged, not hidden. | Status reporting is transparent. The v0.2 audit explicitly flagged the examples/ MANIFEST drift (ESC-002) rather than burying it. v0.3 picks it up as ATELIER-91. No evidence of optimistic glossing. | 0.85 |
| 8.4 | Is there a "stop the project" trigger? | This grill stage. Verdict options: Proceed, Reduce scope, Rethink, Escalate. | The grill IS the stop-the-project gate. If the verdict were "Rethink" or "Escalate" with blocking issues, P1 would not proceed. Cancellation is not politically impossible — it's a mechanical verdict. | 0.85 |
**Axis 8 confidence: 0.85.** No challenges.
### Axis 9 — Change, Adoption, and Operational Readiness
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| 9.1 | Who uses this, how does their work change, what's in it for them? | PROJECT.md: "consumed by AI agents as pre-completion guidance and by humans as engineering canon." review/agent-checklist.md is the pre-completion gate. | Users: AI agents (read MANIFEST→core→domains before tasks) and humans (engineering reference). Their work changes by consulting a traceable principle hierarchy before acting. The value: conflict resolution has an arbiter (the matrix), not just opinions. | 0.80 |
| 9.2 | Is the operations team involved now or handed a finished product? | N/A — docs-only repo, no operations team. | No operations team. The "go-live" is tagging v0.2.6 + merging to main. No runtime to operate, no on-call, no SLO. | 0.92 |
| 9.3 | What is the rollback plan? | Standard git workflow: revert + re-tag. | Git revert + re-tag. Standard for docs repos. v0.2 demonstrated the pattern (ESC-002 force-updated tags). | 0.88 |
| 9.4 | Have success criteria been validated with judges? | ROADMAP.md Success Criteria: all checked for v0.1. v0.2 completed clean (AUDIT-P5-resync.md: 10 checks pass). | Success criteria validated by 2 prior milestones. The pattern: reqs covered + reconstruction test + audit clean + tag exists. v0.3 adds matrix row-count test (170) and MANIFEST reconstruction (incl. examples/). | 0.85 |
**Axis 9 confidence: 0.86.** No challenges.
### Meta — Closing Review
| # | Forcing Question | Evidence | Answer | Confidence |
|---|-----------------|----------|--------|------------|
| M.1 | If you were the auditor, what would you flag? | (adversarial analysis) | The P3 specialist-persona gap (i18n + compliance authored by tech-writer without specialists). It's a content-quality risk, not a traceability risk. The mitigation (RESEARCH.md depth + P6 review) is adequate but not redundant — there's no second layer of content validation beyond domain-expert traceability checks. | 0.75 |
| M.2 | What is the project NOT doing that it should? | PROJECT.md out-of-scope: tooling/linters. | Not doing: automated P-rule→C-rule trace validation (manual domain-expert verification only). This is explicitly out of scope (tooling is future work). At 170 P-rules, manual verification is still feasible but approaching the limit where tooling would be warranted. Not a v0.3 blocker. | 0.80 |
| M.3 | Simplest 80% version? | (adversarial analysis) | Ship 2 domains (gitops-operators + ai-ml) and defer i18n + compliance to v0.4. This halves the scope. **But:** v0.2 already deferred these once (IDEATE-15/16). Re-deferral makes them 2x deferred — zombie risk. The current 4-domain plan is better than re-deferral. D-022's pairing of the two smaller domains in P3 is the right load-balance call. | 0.82 |
| M.4 | What must be true in 90 days for success, and is it true today? | (adversarial analysis) | Must be true: (a) 40 new P-rules trace cleanly to C-rules — RESEARCH.md pre-maps them, P4 verifies. (b) 18 derived docs each have ≥1 valid cross-link — ATELIER-88 verifies in P5. (c) Content is technically correct (especially i18n bidi + compliance policy-as-code) — RESEARCH.md grounds it, P6 validates. (a) and (b) have explicit verification gates. (c) depends on execution quality. All three are achievable. | 0.80 |
**Meta confidence: 0.79.**
---
### Binding Decisions
| ID | Decision | Rationale | Confidence | Alternatives |
|----|----------|-----------|------------|--------------|
| G-001 | Proceed with 4-domain scope as planned; do not split into v0.3a/v0.3b | Scope is deliberately trimmed (D-023/D-024/D-025); per-phase doc counts (P1:4, P2:4, P3:8) are lower than v0.1 P3 (27); v0.2 already deferred these domains once — re-deferral risks zombie status; D-016 groups them to limit release overhead | 0.80 | Split into 2 milestones (rejected: 2x deferral + double release overhead); reduce to 2 domains (rejected: same) |
| G-002 | Phase-specific personas (platform-engineer, ml-engineer) are NOT a key-person risk | Personas are constraint sets, not humans; D-051 bakes constraints into task must-haves (survive persona removal); D-052 ensures both review content in P6 before removal; platform-engineer is proven reuse from v0.2 (shipped clean) | 0.80 | Add more specialist personas (rejected: persona explosion); remove phase-specific personas (rejected: content quality risk) |
| G-003 | 40 new P-rules will trace cleanly to C-rules; matrix orphan risk is mitigated | RESEARCH.md pre-maps all 40 P-rules to C-rules (D-029..D-032); P4 matrix row-count test (10 × 17 = 170) + domain-expert sign-off; P6 reconstruction test; A-002 (confidence 0.95) confirms core is stable at 8 | 0.85 | Force broader C-rule derivations (rejected: false derivations worse than accurate narrow ones); add new core principles (rejected: A-002 confirms unnecessary) |
| G-004 | i18n + compliance pairing in P3 (D-022) is load-balancing-sound | Both are 4-derived-doc domains (smaller surface than P1/P2); P3 Wave 2 schedules 8 docs as 5-then-3 (A-003, 0.90); the scheduling is not a bottleneck — the content-quality risk is (see G-009) | 0.82 | Separate i18n and compliance into distinct phases (rejected: adds a phase, no load benefit); move one to P2 (rejected: P2 is AI/ML, a heavier domain) |
| G-005 | 2-good + 2-bad example set (D-025) is adequate for 4 domains | Each domain gets exactly 1 example (gitops: good, ai-ml: good, i18n: bad, compliance: bad); ATELIER-84 adds 4 domain-specific anti-patterns per domain (16 total) = 5 illustration points per domain; v0.1 had 7 examples for 11 domains (most domains had 0) — v0.3 is better coverage | 0.75 | 4-good + 4-bad (rejected: D-025 unbalances P5); 1-per-domain good only (rejected: bad examples have higher illustration value for i18n/compliance) |
| G-006 | Compliance C4 (Locality) gap (D-032) is architecturally sound, NOT a smell | C4 Locality is about keeping concerns local; compliance is inherently cross-cutting (audit logs span the system, retention is global, evidence is cross-cutting); forcing C4 would be a false derivation; D-032 considered and rejected the alternative "force C4 via audit-log locality"; the absence reflects the domain's nature | 0.82 | Force C4 via audit-log locality (rejected: false derivation); add a C4-tracing compliance P-rule (rejected: would be artificial) |
| G-007 | Docs-only constraint (D-020) is defensible at 17 domains — more so, not less | The docs-only constraint eliminates runtime complexity, integration surface, deployment, and build steps — the exact things that make large frameworks unwieldy; MANIFEST + reading-order + matrix make the tree navigable at scale; the framework's value (traceable hierarchy) scales linearly with domains; 170 matrix rows is a large but single readable file | 0.88 | Add a build step / linter (rejected: out of scope, violates docs-as-code simplicity); split the matrix per-domain (rejected: destroys the join-table value) |
| G-008 | No hidden data-engineer persona requirement from adding ai-ml domain | D-023 scopes AI/ML to engineering discipline (data versioning, evaluation, serving, drift), NOT data engineering (schema/ETL/pipelines); PERSONAS.md explicitly deactivates data-engineer ("No database, schema, or migrations in this docs-only project"); ai-ml cross-links to data/ as a reference, not a duplication | 0.85 | Activate data-engineer persona (rejected: no data engineering work in a docs-only project); scope AI/ML to include data engineering (rejected: D-023 explicitly excludes) |
| G-009 | Proceed with tech-writer + domain-expert for P3 (i18n + compliance) without specialist personas | RESEARCH.md provides thorough prior-art surveys (ICU/CLDR, BCP 47, UAX #9, W3C i18n WG, OPA/Cedar/Kyverno/Sentinel, Cosign/in-toto) with specific source citations; v0.1 tech-writer authored security/data/concurrency without specialists and shipped clean; P6 review includes domain-expert validation; rework is bounded (markdown edits). **Assumption A-006 logged:** content correctness for i18n RTL/bidi and compliance policy-as-code depends on RESEARCH.md prior-art quality + P6 review | 0.72 | Add i18n-specialist + compliance-specialist personas (rejected: persona explosion, both domains are smaller-surface per D-022); defer i18n/compliance to v0.4 with specialists (rejected: 2x deferral zombie risk) |
| G-010 | Matrix coverage summary must read "17 domains, 170 P-rules" post-v0.3 (reinforces D-036) | D-036 (confidence 0.93) already decided this; PLAN task 04-01-01 bakes it in; both the summary block AND per-domain section count must update; the P6 audit verifies row-count (10 × 17 = 170) | 0.93 | Partial update (rejected: invariant violation) |
| G-011 | ATELIER-91 closes the v0.2 ESC-002 manifest drift (examples/ unlisted) | AUDIT-P5-resync.md §6 flagged examples/ not in MANIFEST as a P1 convention note; ATELIER-91 (D-035, confidence 0.85) adds examples/ directory listing to MANIFEST in P4; PLAN task 04-02-05 bakes it in; A-001 (0.85) confirms this is the only manifest drift | 0.85 | Leave examples/ unlisted (rejected: manifest is authoritative, unlisted = not part of framework by definition) |
| G-012 | Pre-mortem top risk (P3 content quality) is bounded; no escalation needed | Most likely failure mode: i18n bidi or compliance policy-as-code technical errors caught in P6, requiring P3 rework. Mitigation: RESEARCH.md prior-art depth + P6 domain-expert review. Rework is bounded (markdown edits, not infrastructure). No external dependencies. The risk is real but recoverable and does not block the milestone. | 0.78 | Defer P3 to v0.4 (rejected: zombie risk); add specialist personas (rejected: G-009 analysis) |
### Escalations
**None.** All 12 challenges auto-resolved at confidence ≥ 0.60 (range: 0.720.93). At full autonomy, assumption logging (A-006) is preferred over escalation. No axis scored below 0.60 on any forcing question.
### Assumptions Logged (this grill)
| # | Assumption | Confidence |
|---|-----------|------------|
| A-006 | i18n RTL/bidi and compliance policy-as-code content correctness depends on RESEARCH.md prior-art quality + P6 domain-expert review, not on a specialist persona. If P6 surfaces fundamental content errors, the remedy is P3 rework (bounded — markdown edits). | 0.72 |
### Summary
- **Challenges identified:** 12 (across 9 axes + meta)
- **Binding decisions:** 12 (G-001..G-012)
- **Escalations:** 0
- **Assumptions logged:** 1 (A-006)
- **Verdict:** PROCEED at confidence 0.80
- **Top 3 material challenges:**
1. **G-009 (0.72):** P3 i18n + compliance authored without specialist personas — the lowest-confidence decision. Content quality for bidi algorithms and policy-as-code semantics depends on RESEARCH.md depth, not specialist persona constraints.
2. **G-005 (0.75):** 2+2 examples across 4 domains — each domain gets only one example (good OR bad, not both). Adequate given 16 anti-patterns, but thinner than v0.2's per-domain coverage.
3. **G-001 (0.80):** 4-domain scope is the largest single milestone — bounded by D-023/D-024/D-025 trims, but re-deferral would create zombie risk.
- **No blocking escalations for SHIP.**
+154
View File
@@ -0,0 +1,154 @@
# Atelier v0.4 — Ideate Stage
> Generated during v0.4 P0 IDEATE stage. Autonomy level: `full`
> (decision_confidence_threshold = 0.60 → auto-accept ≥ 0.6, auto-reject
> below). Interactive mode used internally but auto-decided; no HITL.
> Ideas continue from IDEATE-30 (last v0.3 idea). Tier 3 (cross-project)
> skipped silently — only one project in registry (per workflow error
> recovery). All ideas are within v0.4 scope (ATELIER-92..116 + one new
> req ATELIER-117).
## Scope
Refine the v0.4 requirements (ATELIER-92..116) and the RESEARCH-v0.4.md
proposed content, parallel to the v0.2 (IDEATE-01..16) and v0.3
(IDEATE-17..30) ideation logs. Sources: git-native pattern mining
(lessons, decisions D-001..D-072, ESC-001/002, compound solutions),
coverage-gap analysis (pending v0.4 reqs), verification-layer inversion,
architectural-drift detection (MANIFEST vs actual files, ARCHITECTURE.md
vs directory tree, matrix row-count invariant), spec-driven improvement,
backend-enriched best-practice/chaos ideation for edge/messaging/
languages.
## Counts
- **Generated:** 13 ideas (mechanical: 8, backend-enriched: 5)
- **Accepted:** 13 (all v0.4-scope, confidence ≥ 0.79, above 0.6 autonomy threshold → auto-accepted)
- **Skipped:** 0
- **New requirements added:** 1 (ATELIER-117 — ARCHITECTURE.md v0.4 update, from IDEATE-32)
- **Refinements to existing reqs:** 12 (IDEATE-31, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43)
- **By tier:** mechanical 8, backend-enriched 5, cross-project 0 (skipped silently — single-project registry)
- **By category:** drift 3, coverage 3, spec 2, quality 2, chaos 1, improvement 2
## Ideas Table
| IDEATE-ID | Source | Category | Confidence | Decision | Mapped REQ |
|-----------|--------|----------|------------|----------|------------|
| IDEATE-31 | mechanical (MANIFEST drift, parallel to ESC-002) | drift | 0.86 | accepted → refines | ATELIER-111 (languages/ section must list 4 v0.1 stubs + 16 derived docs) |
| IDEATE-32 | mechanical (ARCHITECTURE.md vs directory tree) | drift | 0.88 | accepted → new req | ATELIER-117 (update ARCHITECTURE.md for v0.4: component tree + v0.4 section) |
| IDEATE-33 | mechanical (IDEATE-19 pattern transfer) | coverage | 0.90 | accepted → refines | ATELIER-107 (Core Principle Coverage C1C8 → domains table update for edge + messaging) |
| IDEATE-34 | mechanical (matrix row-count invariant) | drift | 0.82 | accepted → refines | ATELIER-106 (verify post-v0.4 = exactly 190; verify current = exactly 170 per grill G-014 — the "171 vs 170" claim was a false premise, corrected) |
| IDEATE-35 | mechanical (D-069 vs req-row text) | spec | 0.89 | accepted → refines | ATELIER-100, ATELIER-101 (req text must reference streams + dead-letter comparison tables) |
| IDEATE-36 | mechanical (RESEARCH anti-patterns vs ATELIER-110 text) | quality | 0.85 | accepted → refines | ATELIER-110 (language-derived-doc anti-patterns + .ts/.py/.go/.rs artifact-type extension) |
| IDEATE-37 | backend-enriched (chaos anti-pattern enumeration) | chaos | 0.86 | accepted → refines | ATELIER-110 (include all 6 RESEARCH chaos anti-patterns: edge sync-loop/TTL-less/blocking-call; messaging shared-sub/blocking-consumer/unbounded-retry) |
| IDEATE-38 | backend-enriched (IDEATE-28 pattern transfer) | improvement | 0.80 | accepted → refines | ATELIER-94 (offline-first.md maps offline write-queue patterns to testing pyramid tiers) |
| IDEATE-39 | backend-enriched (IDEATE-29 pattern transfer) | improvement | 0.79 | accepted → refines | ATELIER-101 (delivery-semantics.md fenced idempotency-key dedup-store example, not prose-only) |
| IDEATE-40 | backend-enriched (intra-milestone cross-link dependency) | chaos | 0.84 | accepted → refines | ATELIER-114 (P5 cross-link audit must verify edge↔messaging intra-v0.4 links resolve in both directions) |
| IDEATE-41 | backend-enriched (IDEATE-30 pattern transfer) | improvement | 0.81 | accepted → refines | ATELIER-100 (streams.md enumerates consumer-group rebalance strategies: sticky/cooperative/eager) |
| IDEATE-42 | mechanical (D-066 sub-table verifiability) | coverage | 0.87 | accepted → refines | ATELIER-107 (Languages Coverage sub-table = exactly 16 rows; P4 review check) |
| IDEATE-43 | mechanical (IDEATE-18 pattern transfer) | coverage | 0.83 | accepted → refines | ATELIER-111 (MANIFEST Cross-Cutting summary = "19 domains, 190 P-rules post-v0.4") |
## Refinements Notes (applied to existing reqs at execute time, not changing req rows)
- **ATELIER-106** (IDEATE-34, corrected by grill G-014): matrix coverage summary must read
"post-v0.4: 19 domains, 190 P-rules" in BOTH the summary block and
the per-domain section count (IDEATE-18 carried forward). P4 must
verify the matrix contains exactly 190 P-rows (10 per domain × 19).
The current matrix has **exactly 170 P-rows** (17 domains × 10,
verified by grill G-014 direct count) — the IDEATE-34 "171 vs 170"
claim was a FALSE PREMISE. P4 must verify 170 exists, append 20 rows,
and confirm post-v0.4 count = 190. Do NOT hunt for or delete a phantom
171st row.
- **ATELIER-107** (IDEATE-33, IDEATE-42): the domain-coverage.md
extension must update THREE artifacts: (a) the per-domain row-schema
table with edge + messaging rows; (b) the "Core Principle Coverage"
table (C1C8 → domains) for the 2 new domains (C4 Locality grows by
edge P1/P8 + messaging P7; C5 Reversibility grows by edge P2/P4/P7/P10
+ messaging P5/P8/P9; C7 Observability grows by edge P10 + messaging
P8/P10; C8 Economy grows by edge P3/P9 + messaging P6; C2 grows by
messaging P1/P2/P4) — parallel to v0.3 IDEATE-19; (c) the new
`## Languages Coverage` sub-table with EXACTLY 16 rows (one per
language-derived doc), verifiable as a P4 review check (no orphans,
every derived doc traced to ≥1 domain P-rule).
- **ATELIER-110** (IDEATE-36, IDEATE-37): the anti-patterns extension
must include (a) the v0.4 deployable-artifact-types table extended to
`.ts`/`.py`/`.go`/`.rs` standalone files (language-derived docs risk,
per D-020 docs-only contract); (b) language-derived-doc anti-
patterns: introducing a new P-rule (e.g., "TS P1"), orphan section
with no domain P-rule trace, runtime standalone artifact, and
first-principles stub gaining P1P10 numbering; (c) all 6 RESEARCH-
specified chaos anti-patterns, not just the 4 named in the req-row
text — edge: sync conflict loop (P4), TTL-less cache (P6), blocking
call on constrained device (P3/P5); messaging: shared subscription
(P2/P3), blocking consumer (P6), unbounded retry / no DLQ (P5).
- **ATELIER-100** (IDEATE-35, IDEATE-41): streams.md must include (a)
the stream-platform comparison table (Kafka vs Kinesis vs
Pulsar/NATS JetStream — ordering, partitioning, replay/retention,
use-case fit) per D-069, AND (b) an enumeration of consumer-group
rebalance strategies (sticky/cooperative vs eager) as a sub-
comparison — each with partition-stop-the-world cost and use-case
fit. Parallel to v0.3 IDEATE-30 (drift-type enumeration).
- **ATELIER-101** (IDEATE-35, IDEATE-39): delivery-semantics.md must
include (a) the dead-letter strategy comparison table (retry-count-
limit vs TTL-with-backoff vs poison-queue vs DLQ+alert) per D-069,
AND (b) a fenced idempotency-key dedup-store example (not prose-
only) — parallel to v0.3 IDEATE-29 (fenced signed-attestation
example). Prose-only delivery-semantics guidance is weak; a fenced
consumer-with-dedup-store demonstrates P3 (idempotent consumers)
concretely.
- **ATELIER-111** (IDEATE-31, IDEATE-43): the new `## Languages`
MANIFEST section must list the 4 v0.1 first-principles stubs
(typescript.md, python.md, go.md, rust.md — closing the pre-existing
drift since v0.1, parallel to ESC-002/ATELIER-91) AND the 16 new
derived docs. The Cross-Cutting summary line must update to exactly
"19 domains, 190 P-rules post-v0.4" (IDEATE-18 carried forward).
- **ATELIER-114** (IDEATE-40): the P5 cross-link audit must verify
edge↔messaging intra-v0.4 links resolve in BOTH directions, not only
new→existing. Edge/iot.md (P1) cross-links messaging/queues.md (MQTT
QoS parallels delivery semantics) and messaging/iot.md-style
references — but messaging/ docs are authored in P2, so at the P1
tag (v0.3.1) those links dangle. This is acceptable per vertical-
slice integrity (D-053 — matrix/cross-links close in P4/P5), but the
P5 audit (ATELIER-114) must explicitly check that every edge→
messaging and messaging→edge link resolves by the P6 ship. The
audit must also confirm language-derived docs (P3) link to BOTH
new (edge/messaging) and existing domains where relevant (e.g.,
ts-async.md → messaging/delivery-semantics; go-concurrency.md →
concurrency/patterns + messaging/queues).
- **ATELIER-94** (IDEATE-38): offline-first.md must map offline write-
queue / conflict-detection patterns to the testing pyramid tiers
(unit: conflict detection on a merge function; integration:
reconnect reconcile against a local store; e2e: partition
simulation with a fake network) — parallel to v0.3 IDEATE-28 (i18n
pseudo-locale → pyramid). Avoids generic "test offline" guidance.
## Within-Project Pattern Transfer (v0.1 → v0.2 → v0.3 → v0.4)
| Prior Lesson | v0.4 Application | Status |
|--------------|------------------|--------|
| IDEATE-10/11/12 + 21/22/23 (decision/comparison tables in multi-tool derived docs) | D-069 + IDEATE-35 transfer the pattern to streams.md (Kafka/Kinesis/Pulsar) + delivery-semantics.md (dead-letter strategies) | ✓ transferred |
| IDEATE-17 (manifest drift closure — examples/ unlisted → ATELIER-91) | IDEATE-31 → ATELIER-111 (languages/ unlisted since v0.1 — closes pre-existing drift on the same ESC-002 precedent) | ✓ transferred |
| IDEATE-18 (coverage summary totals must update — string + per-domain count) | IDEATE-43 → ATELIER-111 (Cross-Cutting = "19 domains, 190 P-rules post-v0.4") + IDEATE-34 → ATELIER-106 (summary block) | ✓ transferred |
| IDEATE-19 (Core Principle Coverage C1C8 → domains table update) | IDEATE-33 → ATELIER-107 (C-rule coverage table extended for edge + messaging with the documented C-count shifts) | ✓ transferred |
| IDEATE-20 (anti-patterns specificity: domain violations + artifact types) | IDEATE-36 + IDEATE-37 → ATELIER-110 (language-derived anti-patterns + .ts/.py/.go/.rs artifact types + all 6 chaos anti-patterns) | ✓ extended |
| IDEATE-07 (named bad examples cite the P-rule breached) | ATELIER-113 + D-068 already cover (single-breach, cite specific P-rule) | ✓ carried forward |
| IDEATE-08 (cross-link verification: every new derived doc ≥1 outbound) | ATELIER-114 already covers; IDEATE-40 extends to edge↔messaging intra-milestone links | ✓ extended |
| IDEATE-13/14 (chaos: orphan P-rule + deployable artifact) | IDEATE-36 → ATELIER-110 (language orphan-section + standalone .ts/.py/.go/.rs runtime artifact) | ✓ extended |
| IDEATE-28 (pseudo-locale testing → testing pyramid tiers) | IDEATE-38 → ATELIER-94 (offline write-queue testing → pyramid tiers) | ✓ transferred |
| IDEATE-29 (fenced signed-attestation example, not prose-only) | IDEATE-39 → ATELIER-101 (fenced idempotency-key dedup-store example) | ✓ transferred |
| IDEATE-30 (drift-type enumeration: data/concept/prediction + signal each) | IDEATE-41 → ATELIER-100 (rebalance-strategy enumeration: sticky/cooperative/eager + cost each) | ✓ transferred |
## Summary
- 13 ideas generated, 13 accepted (confidence ≥ 0.79), 0 skipped.
- 1 new requirement: ATELIER-117 (ARCHITECTURE.md v0.4 update —
coverage gap; no existing req covers the architecture-tree drift).
- 12 refinements baked into ATELIER-94, 100, 101, 106, 107, 110, 111,
114 — to be honored by the PLAN stage as task must-haves (parallel
to v0.3's Refinements Notes → Task Bake-In Map).
- The within-project pattern-transfer convention (decision matrices,
manifest-drift closure, coverage-summary totals, C-rule coverage
table, chaos anti-patterns, fenced examples, enumerations, pyramid
mappings) continues from v0.2→v0.3→v0.4 unchanged.
- Tier 3 (cross-project) skipped silently: single-project registry.
- No idea fell below the 0.60 autonomy threshold → no HITL escalation.
+115
View File
@@ -0,0 +1,115 @@
# Atelier — Persona Configuration
> Generated during P0 RESEARCH. Atelier is a documentation-only project;
> the default runtime persona roster is deactivated. Custom personas
> reflect the docs-as-code nature of the work. Updated per milestone:
> v0.4 adds `edge-engineer` (P1) and `languages-engineer` (P3) phase-
> specific personas (D-067); messaging (P2) is authored by tech-writer
> + domain-expert with edge-engineer consultation.
## Active Personas
### lead-developer
- **active:** true
- **domain:** coordination
- **frameworks:** []
- **constraints:** ["pragmatic", "docs-as-code discipline", "manifest is authoritative"]
- **territory:** [".ciagent/**", "MANIFEST.md", "README.md"]
- **reason:** Decomposes doc-writing tasks by domain across phases; coordinates phase execution; owns governance files and the manifest.
### tech-writer
- **active:** true
- **domain:** documentation
- **frameworks:** []
- **constraints:** ["markdown-first", "examples are mandatory", "why over what", "audience awareness"]
- **territory:** ["**/*.md", "LICENSE", "CHANGELOG.md", "CONTRIBUTING.md"]
- **reason:** Primary author of all framework markdown content. Owns docs-as-code discipline (Documentation P1). Excludes `.ciagent/**` governance files (owned by lead-developer). For v0.4: co-authors the messaging domain (P2) with domain-expert, per D-067.
### domain-expert
- **active:** true
- **domain:** cross-cutting
- **frameworks:** []
- **constraints:** ["every principle traces to core", "10 principles per domain", "no orphans in matrix"]
- **territory:** ["core/**", "domains/**", "matrix/**", "review/**", "examples/**"]
- **reason:** Validates domain principle correctness and traceability to core principles. Ensures the matrix is complete and no domain rule is orphaned. For v0.4: co-authors the messaging domain (P2) with tech-writer, consulting edge-engineer for the D-062 boundary, per D-067.
## Inactive Personas (from default roster)
### data-engineer
- **active:** false
- **reason:** No database, schema, or migrations in this docs-only project.
### backend-engineer
- **active:** false
- **reason:** No API server, routes, services, or middleware in this docs-only project.
### frontend-engineer
- **active:** false
- **reason:** No React/Next components, pages, or hooks in this docs-only project.
## Phase-Specific Personas
### edge-engineer (v0.4 — REMOVE after milestone completion)
- **active:** true
- **phase_specific:** true
- **domain:** edge computing
- **frameworks:** []
- **constraints:** ["latency-bound first", "resource-constrained examples", "trace to core", "10 P-rules per domain", "no runtime code", "proximity is the distinguishing constraint (D-061)", "offline-capability is non-negotiable"]
- **territory:** ["domains/edge/**", "examples/good/edge-offline-sync.md", "examples/bad/edge-sync-loop.md"]
- **reason:** Per D-067: edge domain authoring (cdn, offline-first, iot, sync) benefits from a specialist persona with the proximity-is-correctness and offline-is-non-negotiable constraints. The distinguishing angle is proximity/location (D-061) — distinct from performance (generic), concurrency (in-process), and observability (generic telemetry). Active for v0.4 P1 (Edge Domain) only; consulted during P2 (Messaging) for the D-062 in-process-vs-cross-system boundary; removed after v0.4 completes; roster returns to 3 active personas.
### languages-engineer (v0.4 — REMOVE after milestone completion)
- **active:** true
- **phase_specific:** true
- **domain:** language-specific application
- **frameworks:** []
- **constraints:** ["trace to existing domain P-rules (D-063, D-066) — no new matrix rows", "per-language idioms", "fenced code examples in-language", "10 P-rules per domain does NOT apply to language docs", "manifest gains Languages section (D-065)"]
- **territory:** ["languages/**"]
- **reason:** Per D-067: language-derived extension authoring (16 derived docs across TypeScript, Python, Go, Rust) benefits from a specialist persona enforcing the apply-don't-derive discipline (D-063) and the no-new-matrix-rows invariant (D-066). Languages are an application lens, not a principle source — the existing stubs' section-header-trace pattern is preserved, no P1P10 numbering is introduced. Active for v0.4 P3 (Language-Derived Extension) only; removed after v0.4 completes; roster returns to 3 active personas.
### platform-engineer (v0.3 — REMOVED after milestone completion)
- **active:** false
- **phase_specific:** true
- **domain:** infrastructure/platform-automation
- **frameworks:** []
- **constraints:** ["declarative-first", "stateless examples", "trace to core", "10 P-rules per domain", "no runtime code", "source-of-truth is git", "reconciliation loop is the primitive"]
- **territory:** ["domains/gitops-operators/**", "examples/good/gitops-pr.md", "examples/bad/* (gitops-related)"]
- **reason:** Per D-019 / D-027: the v0.2 platform-engineer persona was reused and extended for P1 (gitops-operators). Removed after v0.3 completed; roster returns to 3 active personas. The v0.3 gitops-operators content remains owned by tech-writer + domain-expert for cross-link maintenance. v0.4: still inactive.
### ml-engineer (v0.3 — REMOVED after milestone completion)
- **active:** false
- **phase_specific:** true
- **domain:** machine-learning engineering
- **frameworks:** []
- **constraints:** ["reproducibility is non-negotiable", "data lineage is traceable", "trace to core", "10 P-rules per domain", "no runtime code", "engineering discipline not algorithm design (D-023)", "examples are illustrative markdown only"]
- **territory:** ["domains/ai-ml/**", "examples/good/ai-ml-reproducibility.md"]
- **reason:** Per D-019 / D-020: AI/ML domain authoring (data versioning, model evaluation, serving, monitoring/drift) benefited from a specialist persona with reproducibility and data-lineage constraints. Scope was engineering discipline, NOT algorithm/model design (D-023). Removed after v0.3 completed; roster returns to 3 active personas. v0.4: still inactive.
### platform-engineer (v0.2 — REMOVED after milestone completion)
- **active:** false
- **phase_specific:** true
- **domain:** infrastructure/platform
- **frameworks:** []
- **constraints:** ["declarative-first", "stateless examples", "trace to core", "10 P-rules per domain", "no runtime code"]
- **territory:** ["domains/infrastructure-as-code/**", "domains/kubernetes/**", "examples/good/terraform-module.md", "examples/good/k8s-deployment.md", "examples/bad/*"]
- **reason:** Specialist authoring for IaC/k8s domain content (terraform, opentofu, state, modules, k8s workloads/networking/storage/rbac/helm/kustomize) where the existing tech-writer persona lacks the domain expertise. Was active for v0.2 P1P4 only; removed after milestone v0.2 completed (per D-027, D-014). Roster returns to 3 active personas. v0.4: still inactive.
## Territory Enforcement
Mode: `warn` (per config.json `personas.territory_enforcement`).
At `warn`, territory violations are logged but not blocked. This is appropriate for a docs project where tech-writer may touch `.ciagent/` files incidentally (e.g., updating ROADMAP status). Strict mode would be appropriate once territories stabilize.
## v0.4 Persona Roster Summary (active during milestone)
Active personas (3, span full milestone): lead-developer, tech-writer, domain-expert.
Phase-specific personas (2, active during milestone): edge-engineer (P1 Edge Domain), languages-engineer (P3 Language-Derived Extension) — both removed after v0.4 completion per D-067.
Messaging (P2) is authored by tech-writer + domain-expert (the permanent roster) with edge-engineer consultation for the distributed-systems/async overlap (D-062 boundary), per D-067. The edge-engineer's distributed-systems lens survives the P1→P2 persona transition via a task must-have baked into the plan (the D-051 constraints-baked-into-task pattern from v0.3): "P1/P2 demonstrated edge + messaging boundary (D-062); delivery-semantics comparison table required (D-069)."
Inactive default personas (3, unchanged): data-engineer, backend-engineer, frontend-engineer.
Historical phase-specific personas (3, inactive): platform-engineer (v0.2), platform-engineer (v0.3 extended), ml-engineer (v0.3) — retained as historical record; "v0.4: still inactive" appended to their reasons.
Peak active roster during v0.4: 5 (3 permanent + 2 phase-specific), matching v0.3's peak and avoiding persona-explosion risk. Roster returns to 3 active personas after v0.4 completion.
+285
View File
@@ -0,0 +1,285 @@
# Atelier — Plan (v0.4)
> Vertical-slice plans with wave ordering for milestone v0.4 (Edge + Messaging + Language-Derived Docs). Plans reference REQ-IDs from `.ciagent/atelier/REQUIREMENTS.md` (ATELIER-92..117). NFR milestone — all phases produce docs; no `feat` code. Per `parallelization.max_concurrent_agents = 5`, wave parallelism is capped at 5 concurrent tasks; waves larger than 5 are split into sub-waves. Grounded in `RESEARCH-v0.4.md` (proposed P-rules, derived-doc outlines, anti-patterns, examples, matrix rows) and decisions D-054..D-073.
## Plan-Level Invariants
- **Vertical-slice integrity (D-053 carried forward):** each phase is independently shippable. P1 ships the edge domain self-consistent (matrix rows land in P4 — the domain is internally consistent; matrix extension is traceability closure, not a blocker). P2 ships messaging self-consistent. P3 ships the 16 language-derived docs self-consistent. P4 closes traceability + manifest + architecture. P5 closes examples + cross-links. P6 ships the release. Edge↔messaging intra-milestone cross-links may dangle at the P1 tag (v0.3.1) because messaging is authored in P2; this is acceptable per D-053 — the P5 audit (ATELIER-114) verifies both directions resolve by the P6 ship (IDEATE-40).
- **Tag sequence:** v0.3.0 (P0) → v0.3.1 (P1) → v0.3.2 (P2) → v0.3.3 (P3) → v0.3.4 (P4) → v0.3.5 (P5) → v0.3.6 (P6 = milestone release). NFR milestone: no separate minor tag; the final patch (v0.3.6) IS the v0.4 deliverable.
- **Persona lifecycle:** edge-engineer + languages-engineer active P1P5, review their authored content in P6 Wave 1 (D-052 precedent), removed in P6 Wave 2 (roster returns to 3 active). Messaging (P2) authored by tech-writer + domain-expert with edge-engineer consultation (D-067); the edge-engineer's distributed-systems lens survives the P1→P2 transition via a task must-have baked into P2 tasks (D-051 constraints-baked-into-task pattern). Peak active roster during v0.4: 5 (3 permanent + 2 phase-specific).
- **Refinements are execute-time must-haves:** all 12 IDEATE-v0.4.md refinements (IDEATE-31, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43) and the refinement notes in REQUIREMENTS.md are baked into the relevant task Must-have columns — they are NOT separate tasks. See the "v0.4 Ideation Refinements → Task Bake-In Map" at the end of this file.
- **Cross-link rule (IDEATE-08 carried forward):** every new derived doc has ≥1 outbound cross-link to a MANIFEST-listed doc. The P5 audit (ATELIER-114) verifies this AND edge↔messaging intra-v0.4 links resolve in both directions (IDEATE-40) AND language-derived docs link to both new (edge/messaging) and existing domains where relevant.
- **Anti-pattern pre-specification (RESEARCH-v0.4 §Review Integration):** 6 chaos anti-patterns are pre-specified to reject on sight — edge: sync-conflict-loop (P4), TTL-less-cache (P6), blocking-call-on-constrained-device (P3/P5); messaging: shared-subscription (P2/P3), blocking-consumer (P6), unbounded-retry (P5). These land in `review/anti-patterns.md` in P4; the 2 named bad examples that demonstrate them (edge-sync-loop, messaging-shared-subscription) land in P5.
- **No runtime code (D-020 carried forward):** all examples and fenced code are illustrative markdown only. The v0.4 deployable-artifact-types table extends to `.ts`/`.py`/`.go`/`.rs` standalone files (forbidden in language-derived docs).
## Phase 0 — Pre-Execution (COMPLETE)
Stages: SPECIFY ✓ → CLARIFY ✓ → RESEARCH ✓ → IDEATE ✓ → PLAN ✓ → GRILL → SHIP
Branch: `phase/00-pre-execution` (from `milestone/v0.4-edge-quantum-langs`)
Tag: v0.3.0
## Phase 1 — Edge Domain
**Goal:** Author the `domains/edge/` tree — 10 first principles (P1P10) plus 4 derived docs (cdn, offline-first, iot, sync). Edge owns the proximity/location angle (D-061): compute near the user/data source, offline as a first-class state, constrained resources declared, sync conflicts bounded, partition-survivable telemetry. Each P-rule derives from core C1C8 (matrix extension lands in P4).
**Branch:** `phase/01-edge` (from `milestone/v0.4-edge-quantum-langs`)
**Personas:** edge-engineer (author), domain-expert (validate traceability), tech-writer (style/format)
**Tag:** v0.3.1
**Requirements:** ATELIER-92, ATELIER-93, ATELIER-94, ATELIER-95, ATELIER-96
### Wave 1 (sequential — first-principles must exist before derived docs)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 01-01-01 | `domains/edge/first-principles.md` | edge-engineer | ATELIER-92 | 10 P-rules P1P10 per RESEARCH-v0.4 §Edge (P1 Proximity is the Design Driver, P2 Offline is a First-Class State, P3 Resources are Constrained and Declared, P4 Sync Conflicts are Bounded Not Infinite, P5 Edge Operations are Idempotent, P6 Cache Invalidation is Explicit, P7 Partial Degradation is Engineered, P8 Geographic Distribution is a First-Class Constraint, P9 Identity is Constrained at the Edge, P10 Edge Observability Survives Partition); each names the core C-rule(s) it derives from; each has definition + "what violates"; broad-derivation domain (7 C-rules: C1, C3, C4, C5, C6, C7, C8); edge-engineer constraints enforced (proximity is the distinguishing constraint per D-061, offline-capability is non-negotiable, latency is a correctness constraint) |
### Wave 2 (parallel — 4 derived docs, independent; ≤5 concurrent)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 01-02-01 | `domains/edge/cdn.md` | edge-engineer | ATELIER-93 | CDN/edge-cache placement + invalidation per RESEARCH-v0.4 §Edge (cache key design, TTL vs explicit invalidation, cache-hit/miss/origin-fetch, origin shielding, purge strategies URL vs soft vs surrogate-key, cache poisoning prevention, multi-CDN routing P8); **decision matrix required (D-069): edge-cache vs origin — when to serve from the pop, when to fetch fresh, when to shield the origin**; cross-link `performance/frontend` (generic caching), `security/input-validation` (cache poisoning), `observability/metrics` (cache-hit ratio); ≥1 outbound cross-link to a MANIFEST-listed doc (IDEATE-08) |
| 01-02-02 | `domains/edge/offline-first.md` | edge-engineer | ATELIER-94 | Offline as a first-class state (P2), local-first storage (IndexedDB/SQLite/on-device), queue-and-forward for writes, conflict detection on reconnect, UI for offline state, service workers per RESEARCH-v0.4 §Edge; **offline write-queue / conflict-detection patterns mapped to testing pyramid tiers (IDEATE-38, ATELIER-94 refinement): unit (conflict detection on a merge function), integration (reconnect reconcile against a local store), e2e (partition simulation with a fake network)**; fenced code: a service-worker cache strategy + an offline-write-queue sketch; cross-link `concurrency/patterns` (bounded buffer = in-process analog), `observability/logging` (local-first logging); ≥1 outbound cross-link (IDEATE-08) |
| 01-02-03 | `domains/edge/iot.md` | edge-engineer | ATELIER-95 | Constrained devices at the edge per RESEARCH-v0.4 §Edge: device resource classes (P3), constrained protocols (MQTT, CoAP), device identity + provisioning (P9), telemetry from devices (P10), command idempotency (P5), partial-degradation when devices drop (P7); fenced code: an MQTT publish/subscribe payload with QoS levels + a device-provisioning manifest; cross-link `security/secrets` (device credentials), `messaging/queues` (MQTT QoS parallels delivery semantics — one-directional outward; this link dangles until P2, acceptable per D-053, verified bidirectional in P5 ATELIER-114 per IDEATE-40); ≥1 outbound cross-link (IDEATE-08) |
| 01-02-04 | `domains/edge/sync.md` | edge-engineer | ATELIER-96 | Reconciling divergent state across partitions per RESEARCH-v0.4 §Edge: the sync problem (P4), conflict resolution strategies, CRDTs (state-based + operation-based), last-write-win (LWW) with vector clocks/timestamps, merge semantics, conflict-free vs conflict-tolerant data types; **decision matrix required (D-069): CRDT vs last-write-win — when each applies, the correctness cost, the operational cost**; fenced code: a CRDT register (LWW-element-set) + an LWW-with-vector-clock merge; cross-link `data/migrations` (schema migration under sync), `concurrency/patterns` (immutability aids merge); ≥1 outbound cross-link (IDEATE-08) |
**Verify (P1):**
- Structural: 5 files exist under `domains/edge/`
- Behavioral: every P1P10 in first-principles names ≥1 C-rule (domain-expert sign-off); broad-derivation 7 C-rules (C1, C3, C4, C5, C6, C7, C8) confirmed
- Security: P9 (Identity is Constrained at the Edge) + P6 (Cache Invalidation is Explicit) sections present
- Quality: each derived doc has ≥1 outbound cross-link to a MANIFEST-listed doc (IDEATE-08); cdn.md carries the edge-cache-vs-origin decision matrix (D-069); sync.md carries the CRDT-vs-LWW decision matrix (D-069); offline-first.md maps offline write-queue to testing pyramid tiers (IDEATE-38); no duplication of performance/concurrency/observability P-rules (D-061 boundary — proximity/location angle only)
## Phase 2 — Messaging Domain
**Goal:** Author the `domains/messaging/` tree — 10 first principles (P1P10) plus 4 derived docs (queues, pubsub, streams, delivery-semantics). Messaging owns the cross-process / network-failure-model angle (D-062): messages as contracts, explicit delivery semantics, idempotent consumers, bounded backpressure, observable lag. Concurrency owns the in-process analog; cross-link is one-directional outward. Each P-rule derives from core C1C8 (matrix extension lands in P4).
**Branch:** `phase/02-messaging` (from `milestone/v0.4-edge-quantum-langs`)
**Personas:** tech-writer (author), domain-expert (author + validate traceability), edge-engineer (consult — D-067 distributed-systems/async overlap)
**Tag:** v0.3.2
**Requirements:** ATELIER-97, ATELIER-98, ATELIER-99, ATELIER-100, ATELIER-101
### Wave 1 (sequential — first-principles first)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 02-01-01 | `domains/messaging/first-principles.md` | tech-writer + domain-expert (edge-engineer consult) | ATELIER-97 | 10 P-rules P1P10 per RESEARCH-v0.4 §Messaging (P1 Messages are Contracts, P2 Ordering is a Property Not an Assumption, P3 Consumers are Idempotent, P4 Delivery Semantics are Explicit, P5 Dead-Letter Handling is Defined, P6 Backpressure is Bounded, P7 Partitioning is Intentional, P8 Replay and Retention are Configured, P9 Schemas Evolve Compatibly, P10 Messaging is Observable); each names the core C-rule(s) it derives from; each has definition + "what violates"; broad-derivation domain (7 C-rules: C1, C2, C4, C5, C6, C7, C8); **edge-engineer consultation baked in (D-067, D-051 pattern): P1/P2 demonstrated edge + messaging boundary (D-062 — cross-process/network-failure-model, not in-process concurrency); no duplication of concurrency P-rules** |
### Wave 2 (parallel — 4 derived docs, independent; ≤5 concurrent)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 02-02-01 | `domains/messaging/queues.md` | tech-writer + domain-expert | ATELIER-98 | Point-to-point async delivery via queues per RESEARCH-v0.4 §Messaging: producer/consumer model, ack/nack, visibility timeouts, redelivery, FIFO vs standard queues, prefetch + concurrency, long polling; **comparison table required (D-069): at-least-once vs at-most-once vs exactly-once — semantics, latency cost, implementation cost, when each fits**; fenced code: a producer/consumer pair with idempotency key + a redelivery+DLQ flow; cross-link `concurrency/patterns` (in-process bounded queue — one-directional outward), `observability/metrics` (lag); ≥1 outbound cross-link (IDEATE-08) |
| 02-02-02 | `domains/messaging/pubsub.md` | tech-writer + domain-expert | ATELIER-99 | Fan-out via publish/subscribe per RESEARCH-v0.4 §Messaging: topic/subscription model, fan-out semantics, shared vs independent subscriptions (the `messaging-shared-subscription` chaos anti-pattern lives here — pre-specified in P4 ATELIER-110), filtering, ordering across subscriptions; fenced code: a publish + two independent subscriptions; cross-link `messaging/streams` (pub/sub vs stream — durability boundary), `observability/metrics` (per-subscription lag); ≥1 outbound cross-link (IDEATE-08) |
| 02-02-03 | `domains/messaging/streams.md` | tech-writer + domain-expert | ATELIER-100 | Durable-log messaging + stream processing per RESEARCH-v0.4 §Messaging: partitioned log model, offsets, consumer groups, replay, retention windows, stream processing (windowing, joins, exactly-once via transactions); **stream-platform comparison table required (D-069): Kafka vs Kinesis vs Pulsar/NATS JetStream — ordering guarantees, partitioning model, replay/retention, use-case fit**; **consumer-group rebalance strategy enumeration (IDEATE-41, ATELIER-100 refinement): sticky/cooperative vs eager — each with partition-stop-the-world cost + use-case fit (parallel to v0.3 IDEATE-30 drift-type enumeration)**; fenced code: a consumer-group reading from offsets + a transactional exactly-once producer; cross-link `messaging/delivery-semantics` (exactly-once via transactions), `data/schema-design` (stream schema), `observability/tracing` (cross-partition traces); ≥1 outbound cross-link (IDEATE-08) |
| 02-02-04 | `domains/messaging/delivery-semantics.md` | tech-writer + domain-expert | ATELIER-101 | Correctness properties of delivery per RESEARCH-v0.4 §Messaging: the three semantics (P4), idempotency (P3), ordering (P2), dead-letter strategies (P5), retry budgets + backoff, poison messages; **dead-letter strategy comparison table required (D-069): retry-count-limit vs TTL-with-backoff vs poison-queue vs DLQ+alert — when each applies, failure visibility, operational cost**; **fenced idempotency-key dedup-store example (IDEATE-39, ATELIER-101 refinement): not prose-only — a fenced consumer-with-dedup-store demonstrates P3 concretely (parallel to v0.3 IDEATE-29 fenced signed-attestation)**; fenced code: an idempotent consumer with a dedup store + a DLQ routing rule; cross-link `concurrency/patterns` (retry/backoff in-process analog), `errors/patterns` (errors as data for message failures), `observability/metrics` (DLQ depth as an alert); ≥1 outbound cross-link (IDEATE-08) |
**Verify (P2):**
- Structural: 5 files exist under `domains/messaging/`
- Behavioral: every P1P10 traces to ≥1 C-rule (domain-expert sign-off); broad-derivation 7 C-rules (C1, C2, C4, C5, C6, C7, C8) confirmed
- Security: P1 (Messages are Contracts) + P5 (Dead-Letter Handling is Defined) sections present
- Quality: each derived doc ≥1 outbound cross-link (IDEATE-08); queues.md carries the delivery-semantics comparison table (D-069); streams.md carries the stream-platform comparison table (D-069) AND the rebalance-strategy enumeration (IDEATE-41); delivery-semantics.md carries the dead-letter strategy comparison table (D-069) AND a fenced idempotency-key dedup-store example (IDEATE-39); no duplication of concurrency/patterns.md P-rules (D-062 boundary — cross-process/network-failure-model only); edge↔messaging cross-links resolve bidirectionally post-P2 (verified in P5 ATELIER-114 per IDEATE-40)
## Phase 3 — Language-Derived Extension
**Goal:** Extend each `languages/*.md` from a first-principles-only stub to a full derived-doc tree. 4 languages × (extend stub + 4 derived docs) = 16 new derived docs. Language-derived docs introduce NO new P-rules (D-063) and NO new matrix rows (D-066); each section traces to existing domain P-rules via the section-header-trace pattern. The first-principles stub is extended with a pointer to its derived docs but retains its section structure — no P1P10 numbering (D-063).
**Branch:** `phase/03-languages-derived` (from `milestone/v0.4-edge-quantum-langs`)
**Personas:** languages-engineer (author), domain-expert (validate traceability)
**Tag:** v0.3.3
**Requirements:** ATELIER-102, ATELIER-103, ATELIER-104, ATELIER-105
### Wave 1 (parallel — 4 first-principles stubs extended, independent; ≤5 concurrent → 4 tasks)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 03-01-01 | `languages/typescript.md` (extend stub) | languages-engineer | ATELIER-102 | Extend existing first-principles stub with a pointer to its 4 derived docs (ts-types, ts-tooling, ts-async, ts-testing); **retain existing section structure — NO P1P10 numbering added (D-063)**; existing section-header-trace pattern preserved (e.g., `## Type System (C1 Correctness, Data P7 Type Fidelity)`); languages-engineer constraints enforced (apply don't derive, no new P-rules, no new matrix rows per D-063/D-066) |
| 03-01-02 | `languages/python.md` (extend stub) | languages-engineer | ATELIER-103 | Extend existing first-principles stub with a pointer to its 4 derived docs (py-types, py-tooling, py-async, py-testing); retain existing section structure — NO P1P10 numbering (D-063); section-header-trace pattern preserved |
| 03-01-03 | `languages/go.md` (extend stub) | languages-engineer | ATELIER-104 | Extend existing first-principles stub with a pointer to its 4 derived docs (go-types, go-tooling, go-concurrency, go-testing); retain existing section structure — NO P1P10 numbering (D-063); section-header-trace pattern preserved |
| 03-01-04 | `languages/rust.md` (extend stub) | languages-engineer | ATELIER-105 | Extend existing first-principles stub with a pointer to its 4 derived docs (rs-ownership, rs-tooling, rs-async, rs-testing); retain existing section structure — NO P1P10 numbering (D-063); section-header-trace pattern preserved |
### Wave 2 (parallel — 16 derived docs, grouped by language; ≤5 concurrent → executor schedules 5-then-5-then-5-then-1)
> **Parallelism note:** 16 independent tasks all depend only on Wave 1. The executor schedules at most 5 concurrently per `parallelization.max_concurrent_agents`. The 2a/2b/2c/2d labels organize by language, not a hard sequencing barrier — all sub-waves are in the same dependency tier.
**Wave 2a — TypeScript (4 derived docs, ≤5 concurrent)**
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 03-02a-01 | `languages/ts-types.md` | languages-engineer | ATELIER-102 | TypeScript type system per RESEARCH-v0.4 §Language-Derived: nominal vs structural typing (branded types), generics, narrowing, utility types, discriminated unions; each section header names ≥1 traced domain P-rule AND core C-rule inline (e.g., `## Nominal vs Structural Typing (C1 Correctness, Data P7 Type Fidelity, API P1 Contract Fidelity)`); fenced TS code in-language (illustrative only, no standalone .ts — D-020); traces to Data P7, API P1, Errors P1, Concurrency P1; ≥1 outbound cross-link to a traced domain doc (IDEATE-08; per IDEATE-40 also cross-link new edge/messaging where relevant, e.g., ts-types → messaging/delivery-semantics if schema-fidelity applies) |
| 03-02a-02 | `languages/ts-tooling.md` | languages-engineer | ATELIER-102 | TypeScript tooling per RESEARCH-v0.4 §Language-Derived: tsc, ESLint, ts-jest, project references, tsconfig discipline; section headers trace to DevOps P2, DevOps P1, Documentation P1; fenced TS code in-language; ≥1 outbound cross-link (IDEATE-08) |
| 03-02a-03 | `languages/ts-async.md` | languages-engineer | ATELIER-102 | TypeScript async per RESEARCH-v0.4 §Language-Derived: Promises + AbortSignal, async/await, error handling, cancellation; section headers trace to Concurrency P7, P8, P9, Errors P5; fenced TS code in-language; cross-link `concurrency/patterns` + `messaging/delivery-semantics` (per IDEATE-40 — ts-async → messaging/delivery-semantics where async-retry parallels apply); ≥1 outbound cross-link (IDEATE-08) |
| 03-02a-04 | `languages/ts-testing.md` | languages-engineer | ATELIER-102 | TypeScript testing per RESEARCH-v0.4 §Language-Derived: Vitest/Jest, mock discipline, type-level tests; section headers trace to Testing P1, P2, P3, P9; fenced TS code in-language; cross-link `testing/pyramid` + `testing/fixtures`; ≥1 outbound cross-link (IDEATE-08) |
**Wave 2b — Python (4 derived docs, ≤5 concurrent; runs in parallel with 2a — total 8 tasks, capped at 5 → executor schedules 5 then 3)**
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 03-02b-01 | `languages/py-types.md` | languages-engineer | ATELIER-103 | Python type system per RESEARCH-v0.4 §Language-Derived: type hints + Pydantic, mypy/pyright, gradual typing; section headers trace to Data P7, Errors P1, API P1; fenced Python code in-language (no standalone .py — D-020); ≥1 outbound cross-link (IDEATE-08) |
| 03-02b-02 | `languages/py-tooling.md` | languages-engineer | ATELIER-103 | Python tooling per RESEARCH-v0.4 §Language-Derived: ruff, mypy, poetry, uv, virtualenv discipline; section headers trace to DevOps P2, Documentation P1; fenced Python code in-language; ≥1 outbound cross-link (IDEATE-08) |
| 03-02b-03 | `languages/py-async.md` | languages-engineer | ATELIER-103 | Python async per RESEARCH-v0.4 §Language-Derived: asyncio, anyio, cancellation, structured concurrency; section headers trace to Concurrency P7, P8, P9; fenced Python code in-language; cross-link `concurrency/patterns` + `messaging/queues` (per IDEATE-40 where applicable); ≥1 outbound cross-link (IDEATE-08) |
| 03-02b-04 | `languages/py-testing.md` | languages-engineer | ATELIER-103 | Python testing per RESEARCH-v0.4 §Language-Derived: pytest, factory_boy, fixture discipline, parametrize; section headers trace to Testing P1, P2, P3; fenced Python code in-language; cross-link `testing/pyramid` + `testing/fixtures`; ≥1 outbound cross-link (IDEATE-08) |
**Wave 2c — Go (4 derived docs, ≤5 concurrent; same dependency tier as 2a/2b/2d)**
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 03-02c-01 | `languages/go-types.md` | languages-engineer | ATELIER-104 | Go type system per RESEARCH-v0.4 §Language-Derived: named types, generics, interfaces, type assertion discipline; section headers trace to Data P7, Errors P1, API P1; fenced Go code in-language (no standalone .go — D-020); ≥1 outbound cross-link (IDEATE-08) |
| 03-02c-02 | `languages/go-tooling.md` | languages-engineer | ATELIER-104 | Go tooling per RESEARCH-v0.4 §Language-Derived: go vet, golangci-lint, go test -race, module discipline; section headers trace to DevOps P2, Concurrency P6; fenced Go code in-language; ≥1 outbound cross-link (IDEATE-08) |
| 03-02c-03 | `languages/go-concurrency.md` | languages-engineer | ATELIER-104 | Go concurrency per RESEARCH-v0.4 §Language-Derived: goroutines, channels, context, select, sync primitives — Go's distinctive strength earns a dedicated concurrency doc (not go-async); section headers trace to Concurrency P1, P3, P5, P7, P8, P9; fenced Go code in-language; cross-link `concurrency/patterns` + `messaging/queues` (per IDEATE-40 — go-concurrency → concurrency/patterns + messaging/queues where channel/broker analogs apply); ≥1 outbound cross-link (IDEATE-08) |
| 03-02c-04 | `languages/go-testing.md` | languages-engineer | ATELIER-104 | Go testing per RESEARCH-v0.4 §Language-Derived: table-driven tests, t.Parallel, t.Cleanup, race detector; section headers trace to Testing P1, P2, P3, P9; fenced Go code in-language; cross-link `testing/pyramid` + `testing/fixtures`; ≥1 outbound cross-link (IDEATE-08) |
**Wave 2d — Rust (4 derived docs, ≤5 concurrent; same dependency tier as 2a/2b/2c)**
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 03-02d-01 | `languages/rs-ownership.md` | languages-engineer | ATELIER-105 | Rust ownership per RESEARCH-v0.4 §Language-Derived: Send/Sync, lifetimes, borrowing, ownership transfer — Rust's distinctive strength earns a dedicated ownership doc (not rs-types); section headers trace to Concurrency P1, P3, Data P7; fenced Rust code in-language (no standalone .rs — D-020); ≥1 outbound cross-link (IDEATE-08) |
| 03-02d-02 | `languages/rs-tooling.md` | languages-engineer | ATELIER-105 | Rust tooling per RESEARCH-v0.4 §Language-Derived: cargo, clippy, fmt, edition discipline; section headers trace to DevOps P2, Documentation P1; fenced Rust code in-language; ≥1 outbound cross-link (IDEATE-08) |
| 03-02d-03 | `languages/rs-async.md` | languages-engineer | ATELIER-105 | Rust async per RESEARCH-v0.4 §Language-Derived: tokio, async traits, cancellation, pin; section headers trace to Concurrency P5, P7, P8, P9; fenced Rust code in-language; cross-link `concurrency/patterns` + `messaging/delivery-semantics` (per IDEATE-40 where applicable); ≥1 outbound cross-link (IDEATE-08) |
| 03-02d-04 | `languages/rs-testing.md` | languages-engineer | ATELIER-105 | Rust testing per RESEARCH-v0.4 §Language-Derived: #[test], proptest, property testing, mock discipline; section headers trace to Testing P1, P2, P3, P9; fenced Rust code in-language; cross-link `testing/pyramid` + `testing/fixtures`; ≥1 outbound cross-link (IDEATE-08) |
> **Parallelism note:** Wave 2a + 2b + 2c + 2d together = 16 independent tasks. The executor schedules at most 5 concurrently per `parallelization.max_concurrent_agents`; the remaining 11 run as slots free. The 2a/2b/2c/2d labels are organizational (by language), not a hard sequencing barrier — all sub-waves are in the same dependency tier (all depend only on Wave 1).
**Verify (P3):**
- Structural: 4 stubs extended + 16 new derived docs exist under `languages/` (20 language files total)
- Behavioral: every derived-doc section header names ≥1 traced domain P-rule AND core C-rule inline (D-063, D-066); no new P-rules introduced; no P1P10 numbering on language stubs (D-063); domain-expert verifies no orphan sections (every derived doc traces to ≥1 domain P-rule)
- Security: no standalone `.ts`/`.py`/`.go`/`.rs` runtime artifacts (D-020 docs-only contract — fenced code in `.md` only)
- Quality: each derived doc ≥1 outbound cross-link to a traced domain doc (IDEATE-08); language-derived docs link to BOTH new (edge/messaging) and existing domains where relevant (IDEATE-40); the 16 derived docs collectively trace to the domains sketched in RESEARCH-v0.4 §Language-Derived (the Languages Coverage sub-table that captures these traces lands in P4 ATELIER-107)
## Phase 4 — Matrix + Review + Manifest + Architecture Integration
**Goal:** Extend the matrix (+20 P-rule → C-rule mappings, 10 edge + 10 messaging; verify pre-existing 170 rows per G-014, append 20, confirm post-v0.4 is exactly 190), domain-coverage (per-domain rows + Core Principle Coverage table for 2 new domains + new Languages Coverage sub-table with exactly 16 rows), ARCHITECTURE.md (v0.4 section + component tree for edge/messaging/languages-derived), review docs (agent + peer-review + anti-patterns with all 6 chaos anti-patterns + language-derived anti-patterns + .ts/.py/.go/.rs artifact types), and the manifest (new `## Languages` section closing pre-existing drift + post-v0.4 totals). Closes the traceability loop and makes the manifest + architecture authoritative for v0.4.
**Branch:** `phase/04-matrix-review-manifest` (from `milestone/v0.4-edge-quantum-langs`)
**Personas:** domain-expert (matrix + coverage + anti-patterns), tech-writer (checklists), lead-developer (manifest + ARCHITECTURE.md — authoritative index + architecture tree)
**Tag:** v0.3.4
**Requirements:** ATELIER-106, ATELIER-107, ATELIER-117, ATELIER-108, ATELIER-109, ATELIER-110, ATELIER-111
### Wave 1 (sequential — matrix is the arbiter, must be authoritative first; then coverage; then ARCHITECTURE.md)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 04-01-01 | `matrix/principles-matrix.md` (extend) | domain-expert | ATELIER-106 | Add "Edge" + "Messaging" sections, 10 rows each, format matching v0.1/v0.2/v0.3 tables (rows per RESEARCH-v0.4 §Matrix Integration); **review check: row count per new domain = 10, each row ≥1 C-rule** (IDEATE-02 carried forward); **verify pre-existing count first (IDEATE-34 corrected per G-014, ATELIER-106 refinement): the current matrix has exactly 170 P-rows (17 domains × 10, verified by grill G-014) — confirm 170 exists BEFORE appending the 20 v0.4 rows; do NOT hunt for or delete a phantom 171st row; post-v0.4 count must be exactly 190 (= 170 + 20)**; update Coverage Summary to "post-v0.4: 19 domains, 190 P-rules" in BOTH the summary block AND the per-domain section count (IDEATE-18, IDEATE-43 carried forward) |
| 04-01-02 | `matrix/domain-coverage.md` (extend) | domain-expert | ATELIER-107 | THREE artifacts updated (IDEATE-33, IDEATE-42, ATELIER-107 refinement): (a) per-domain row-schema table — add Edge + Messaging rows (schema: domain, P-count, derived-doc-count, manifest-listed, status per IDEATE-03); (b) Core Principle Coverage table (C1C8 → domains) — extend for 2 new domains: C4 Locality grows by edge P1/P8 + messaging P7; C5 Reversibility grows by edge P2/P4/P7/P10 + messaging P5/P8/P9; C7 Observability grows by edge P10 + messaging P8/P10; C8 Economy grows by edge P3/P9 + messaging P6; C2 grows by messaging P1/P2/P4 (parallel to v0.3 IDEATE-19); (c) new `## Languages Coverage` sub-table with EXACTLY 16 rows (one per language-derived doc) — schema: Language \| Derived Doc \| Traced Domain P-rules \| Traced Core C-rules; verifiable as a P4 review check (no orphans, every derived doc traced to ≥1 domain P-rule, per D-066) |
| 04-01-03 | `ARCHITECTURE.md` (extend) | lead-developer | ATELIER-117 | Extend the component tree with `edge/` + `messaging/` domain entries; add a `## v0.4 Domain Additions` section documenting the two new domains (boundary per D-061/D-062, derived docs, cross-link targets); note the `languages/` derived-doc extension (D-063 apply-don't-derive — no new P-rules, no new matrix rows). Closes architecture-tree drift: the file currently stops at v0.3 with no edge/messaging/languages-derived coverage (IDEATE-32) |
### Wave 2 (parallel — 3 review docs, independent; ≤5 concurrent)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 04-02-01 | `review/agent-checklist.md` (extend) | tech-writer | ATELIER-108 | Add 3 trigger sections per RESEARCH-v0.4 §Review Integration: "If Edge (see domains/edge/)" — 10 checks (P1 proximity/latency-as-correctness, P2 offline, P3 constrained resources declared, P4 sync conflicts bounded, P5 idempotent, P6 cache invalidation explicit, P7 partial degradation, P8 geographic distribution, P9 identity constrained, P10 local-first telemetry); "If Messaging (see domains/messaging/)" — 10 checks (P1 messages are contracts, P2 ordering explicit, P3 idempotent consumers, P4 delivery semantics explicit, P5 dead-letter defined, P6 backpressure bounded, P7 partitioning intentional, P8 replay/retention configured, P9 schemas evolve compatibly, P10 observable); "If Language-Derived Docs (see languages/)" — 5 checks (no new P-rules per D-063/D-066; every section header names ≥1 traced domain P-rule AND C-rule inline; fenced code in-language + illustrative only per D-020; first-principles stub retains structure — no P1P10 numbering; cross-links to traced domain docs present ≥1 outbound per derived doc per ATELIER-114) |
| 04-02-02 | `review/peer-review-checklist.md` (extend) | tech-writer | ATELIER-109 | Add 3 peer-review sections mirroring the agent-checklist as reviewer-voice questions, plus 3 scope checks (IDEATE-09 parity carried forward): edge scope check (proximity/location/constraint angle, not generic perf or in-process concurrency — D-061); messaging scope check (cross-process/network-failure-model, not in-process concurrency — D-062); language-derived scope check (apply don't derive — no new P-rules, no new matrix rows — D-063, D-066) |
| 04-02-03 | `review/anti-patterns.md` (extend) | domain-expert | ATELIER-110 | Per RESEARCH-v0.4 §Review Integration + IDEATE-36/37 (ATELIER-110 refinement): (a) **v0.4 deployable-artifact-types table extended to `.ts`/`.py`/`.go`/`.rs` standalone files** (language-derived docs risk, per D-020 docs-only contract); (b) **all 6 RESEARCH-specified chaos anti-patterns** (not just the 4 named in the req-row text) — edge: sync conflict loop (P4, C1/C5 — infinite oscillation is a correctness failure/livelock), TTL-less cache (P6, C1/C3 — stale-forever under partition), blocking call on constrained device (P3/P5, C8/C1 — blocks the node, no timeout, retry unsafe without idempotency); messaging: shared subscription (P2/P3, C1 — breaks ordering + dedup), blocking consumer (P6, C1/C8 — blocks the partition, lag grows), unbounded retry / no DLQ (P5, C1/C5 — infinite retry is a livelock); (c) **language-derived-doc anti-patterns**: introducing a new P-rule (e.g., "TS P1" — violates D-063 + ARCHITECTURE.md hierarchy), orphan section with no domain P-rule trace (violates D-066 + matrix completeness), runtime standalone artifact `.ts`/`.py`/`.go`/`.rs` (violates D-020), first-principles stub gaining P1P10 numbering (violates D-063); (d) **domain-specific anti-patterns** per RESEARCH-v0.4 §Review Integration — edge 10 (central-region-only deployment, crash-on-disconnect, undeclared resource budget, sync oscillation, non-idempotent edge write, TTL-less cache, crash-on-node-failure, random geographic placement, shared edge-device credential, fire-and-forget telemetry), messaging 10 (schemaless message, undocumented "FIFO", non-idempotent consumer, unstated delivery semantic, no DLQ, unbounded retry budget, default partition key, ephemeral broker, silent schema change, silent backlog) |
### Wave 3 (sequential — MANIFEST after review docs settled; per D-050 precedent anti-patterns + manifest combine under lead-developer so manifest lists file paths after anti-pattern content is settled)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 04-03-01 | `MANIFEST.md` (extend) | lead-developer | ATELIER-111 | Per D-065 + IDEATE-31/43 (ATELIER-111 refinement): (a) add 2 new domains (edge, messaging) + all 10 derived docs to the Domains table; (b) add a new top-level `## Languages` section between `## Domains` and `## Examples` (matching reading order: core → domains → languages → examples) listing the 4 v0.1 first-principles stubs (typescript.md, python.md, go.md, rust.md — closes pre-existing drift since v0.1, parallel to ESC-002/ATELIER-91) AND the 16 new v0.4 derived docs; section note: "Language docs apply (not derive) domain principles. They introduce no new P-rules and no new matrix rows (D-063, D-066)."; (c) update Cross-Cutting summary line to exactly "19 domains, 190 P-rules post-v0.4" (IDEATE-18, IDEATE-43 carried forward) |
**Verify (P4):**
- Structural: matrix has 19 domain sections (17 v0.1/v0.2/v0.3 + 2 new), exactly 190 P-rules (170 pre-existing verified per G-014 + 20 new; NOT 171 reconciled to 170 — the 171 discrepancy was a false premise corrected by grill G-014); domain-coverage has the per-domain v0.4 table AND the updated C-rule coverage table AND the Languages Coverage sub-table (exactly 16 rows); ARCHITECTURE.md has the v0.4 section + component tree; 3 review docs extended; MANIFEST lists all v0.4 docs + the new Languages section
- Behavioral: every new edge + messaging P-rule has a matrix row; domain-expert verifies no orphans (IDEATE-13 carried forward); Coverage Summary reads "19 domains, 190 P-rules" (IDEATE-18/43); Languages Coverage sub-table has exactly 16 rows — every language-derived doc traced to ≥1 domain P-rule (IDEATE-42, D-066); ARCHITECTURE.md component tree matches the directory tree (IDEATE-32)
- Security: anti-patterns cover all 6 chaos anti-patterns (edge sync-loop/TTL-less/blocking-call; messaging shared-sub/blocking-consumer/unbounded-retry) + edge P9 (shared credential) + messaging P1 (schemaless) + language-derived runtime-artifact (.ts/.py/.go/.rs)
- Quality: MANIFEST is authoritative — every v0.4 file listed, languages/ listed closing pre-existing drift (ATELIER-111 per IDEATE-31); unlisted = not part of framework; agent-checklist + peer-review-checklist have parity across edge + messaging + language-derived (ATELIER-108 ↔ ATELIER-109); matrix row-count test passes (exactly 190, reconciled)
## Phase 5 — Examples + Cross-Links
**Goal:** Add 2 good + 2 bad examples (D-068 — pre-named, highest illustration value) and verify cross-domain links from the 2 new domains + 16 language-derived docs to existing ones, AND edge↔messaging intra-v0.4 links resolve in both directions (IDEATE-40). Examples are markdown with fenced code only (no standalone .ts/.py/.go/.rs artifacts — D-020). Bad examples are single-breach citing the specific breached P-rule (D-068).
**Branch:** `phase/05-examples-crosslinks` (from `milestone/v0.4-edge-quantum-langs`)
**Personas:** edge-engineer (edge examples), tech-writer (messaging examples + cross-link audit), domain-expert (P-rule citation + cross-link validation)
**Tag:** v0.3.5
**Requirements:** ATELIER-112, ATELIER-113, ATELIER-114
### Wave 1 (parallel — 4 examples, independent; ≤5 concurrent)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 05-01-01 | `examples/good/edge-offline-sync.md` | edge-engineer | ATELIER-112 | Good edge example per RESEARCH-v0.4 §Examples: demonstrates offline-first + sync conflict resolution via CRDT or LWW — highest-illustration-value edge case; markdown with fenced code (a CRDT register merge + an offline write queue + a reconnect reconcile); P-rules demonstrated: edge P2 (Offline is a First-Class State), P4 (Sync Conflicts are Bounded), P5 (Idempotent), P10 (local-first telemetry); exercises the `sync.md` CRDT-vs-LWW decision matrix; cross-link to `edge/offline-first.md` + `edge/sync.md`; no runtime artifacts |
| 05-01-02 | `examples/good/messaging-idempotent-consumer.md` | tech-writer | ATELIER-112 | Good messaging example per RESEARCH-v0.4 §Examples: demonstrates exactly-once-via-idempotency-keys + dedup — highest-illustration-value messaging case; markdown with fenced code (an idempotent consumer with a dedup store + a DLQ routing rule); P-rules demonstrated: messaging P3 (Consumers are Idempotent), P4 (Delivery Semantics Explicit), P5 (Dead-Letter Handling), P10 (Observable); exercises the `delivery-semantics.md` idempotency + DLQ guidance; cross-link to `messaging/delivery-semantics.md` + `messaging/queues.md`; no runtime artifacts |
| 05-01-03 | `examples/bad/edge-sync-loop.md` | edge-engineer | ATELIER-113 | Bad edge example per RESEARCH-v0.4 §Examples + ATELIER-110: a sync conflict that oscillates/loops forever — the pre-specified edge chaos anti-pattern; **single-breach (D-068): cites edge P4 (Sync Conflicts are Bounded, Not Infinite) breached** — prose explains why it is a livelock, not eventual consistency; markdown with fenced code showing the oscillating sync; cross-link to `edge/sync.md` + `review/anti-patterns.md` (the sync-conflict-loop chaos anti-pattern) |
| 05-01-04 | `examples/bad/messaging-shared-subscription.md` | tech-writer | ATELIER-113 | Bad messaging example per RESEARCH-v0.4 §Examples + ATELIER-110: a shared subscription causing duplicate processing / fan-out duplication — the pre-specified messaging chaos anti-pattern; **single-breach (D-068): primary breach cited is messaging P2 (Ordering is a Property, Not an Assumption); P3 (Consumers are Idempotent) noted as the compounding consequence — the example remains single-breach in its named violation**; markdown with fenced code showing the shared subscription; cross-link to `messaging/pubsub.md` + `review/anti-patterns.md` (the shared-subscription chaos anti-pattern) |
### Wave 2 (sequential — cross-link audit after all docs exist)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 05-02-01 | Cross-link audit (all new derived docs across edge + messaging + languages) | tech-writer | ATELIER-114 | Per IDEATE-08 (carried forward) + IDEATE-40 (ATELIER-114 refinement): (a) every new derived doc (10 domain derived + 16 language-derived = 26) has ≥1 outbound cross-link to a MANIFEST-listed doc; links resolve; (b) **edge↔messaging intra-v0.4 links resolve in BOTH directions** — every edge→messaging and messaging→edge link resolves by the P6 ship (edge/iot.md → messaging/queues.md MQTT QoS parallels; messaging docs that reference edge where relevant); (c) **language-derived docs link to BOTH new (edge/messaging) and existing domains where relevant** (e.g., ts-async.md → messaging/delivery-semantics; go-concurrency.md → concurrency/patterns + messaging/queues); domain-expert validates the cross-link targets are correct (not just present); no back-link edits to v0.1/v0.2/v0.3 content (D-026 extended — one-directional outward) |
**Verify (P5):**
- Structural: 4 new example files exist (all .md)
- Behavioral: each bad example cites the P-rule breached (edge: P4 single-breach; messaging: P2 primary + P3 consequence, single-breach named violation per D-068); each good example cites the P-rules it demonstrates
- Security: no standalone `.ts`/`.py`/`.go`/`.rs` runtime artifacts (deployable artifact mitigation, IDEATE-36, D-020)
- Quality: all cross-links from the 26 new derived docs resolve to MANIFEST-listed docs; edge↔messaging intra-v0.4 links resolve bidirectionally (IDEATE-40); language-derived docs link to both new + existing domains where relevant (IDEATE-40); no back-link edits to v0.1/v0.2/v0.3 content (D-026 extended)
## Phase 6 — Final Review + Ship (N+1)
**Goal:** Multi-persona review across all v0.4 phases, audit, milestone ship. P6 IS the v0.4 release (NFR → no separate minor tag; v0.3.6 IS the deliverable). Phase-specific personas (edge-engineer, languages-engineer) review their authored content in Wave 1 (D-052 precedent), then are removed in Wave 2 (roster returns to 3 active).
**Branch:** `phase/06-final-review-ship` (from `milestone/v0.4-edge-quantum-langs`)
**Personas:** lead-developer (coordinate + ship), domain-expert (review), tech-writer (review), edge-engineer (review edge content, then removed), languages-engineer (review language-derived content, then removed)
**Tag:** v0.3.6 (IS the v0.4 milestone release — NFR, no separate minor tag)
**Requirements:** ATELIER-115, ATELIER-116
### Wave 1 — Review (sequential — edge-engineer + languages-engineer review their content per D-052 before removal)
| Task | Activity | Persona | REQ-ID | Must-have |
|------|----------|---------|--------|-----------|
| 06-01-01 | `ciagent-review` — multi-persona review of all v0.4 changes | lead-developer | ATELIER-115 | Auto-apply P0 fixes; flag P1+ for post-hoc; if P1+ found, fix in this phase; **edge-engineer reviews edge domain content (D-061 boundary — proximity/location angle, not generic perf/concurrency)**; **languages-engineer reviews language-derived content (D-063 apply-don't-derive scope check — no new P-rules, no P1P10 numbering, every section traces to a domain P-rule)**; tech-writer + domain-expert review messaging content (D-062 boundary — cross-process/network-failure-model); domain-expert verifies all 20 new P-rules (10 edge + 10 messaging) trace to ≥1 C-rule (no orphans) AND the 16 Languages Coverage sub-table rows match the derived docs' actual section-header traces (IDEATE-42) |
| 06-01-02 | `ciagent-audit` — reconstruction + discipline | lead-developer | ATELIER-115 | git log matches .ciagent/ files; branch hygiene; commit discipline (every commit has `---ci---` block with `project: atelier`); MANIFEST reconstruction test (every listed doc exists, every existing doc is listed — incl. the new Languages section per ATELIER-111); **matrix row-count test: exactly 190 P-rows (10 per domain × 19 — the pre-existing count was exactly 170 per grill G-014 verification; the IDEATE-34 "171-vs-170 discrepancy" was a false premise, corrected — verify 170 + 20 = 190, NOT 171 + 20)**; ARCHITECTURE.md component-tree test (matches directory tree per ATELIER-117/IDEATE-32) |
### Wave 2 — Ship (sequential — after review; phase-specific personas removed)
| Task | Activity | Persona | REQ-ID | Must-have |
|------|----------|---------|--------|-----------|
| 06-02-01 | `ciagent-ship` — milestone ship | lead-developer | ATELIER-116 | Merge `phase/06``milestone/v0.4-edge-quantum-langs``main`; tag `v0.3.6`; Gitea release with full milestone summary; delete all v0.4 branches (tags preserve history) |
| 06-02-02 | Complete milestone (REQUIREMENTS + ROADMAP + PERSONAS) | lead-developer | ATELIER-116 | Mark all v0.4 requirements (ATELIER-92..117) `covered`; ROADMAP v0.4 → complete; **PERSONAS: remove edge-engineer + languages-engineer (roster returns to 3 active per D-067)**; clear checkpoint |
**Verify (P6):**
- Structural: all 26 v0.4 requirements (ATELIER-92..117) marked covered
- Behavioral: reconstruction test passes (git log ↔ .ciagent/); matrix row-count test passes (exactly 190 — reconciled per IDEATE-34); MANIFEST reconstruction test passes (incl. Languages section); ARCHITECTURE.md component-tree test passes (per ATELIER-117/IDEATE-32)
- Security: audit clean (no critical issues); no standalone runtime artifacts (`.ts`/`.py`/`.go`/`.rs` forbidden per D-020); no new P-rules in language-derived docs (D-063); edge/messaging boundaries respected (D-061/D-062)
- Quality: milestone merged to main, tag v0.3.6 exists, all v0.4 branches deleted; edge-engineer + languages-engineer personas removed (roster = 3 active per D-067)
## v0.4 Wave Ordering Summary
| Phase | Waves | Parallelism |
|-------|-------|-------------|
| P0 | (pre-exec) | Sequential stages (specify→clarify→research→ideate→plan→grill) — COMPLETE |
| P1 | 2 | Wave 1 sequential (first-principles), Wave 2 parallel (4 derived docs) |
| P2 | 2 | Wave 1 sequential (first-principles), Wave 2 parallel (4 derived docs) |
| P3 | 2 | Wave 1 parallel (4 stubs extended), Wave 2 parallel (16 derived docs — 2a TS + 2b Python + 2c Go + 2d Rust, capped at 5 concurrent) |
| P4 | 3 | Wave 1 sequential (matrix → coverage → ARCHITECTURE.md), Wave 2 parallel (3 review docs), Wave 3 sequential (MANIFEST after review docs settled) |
| P5 | 2 | Wave 1 parallel (4 examples), Wave 2 sequential (cross-link audit) |
| P6 | 2 | Wave 1 sequential (review → audit), Wave 2 sequential (ship → complete) |
## v0.4 Requirements → Phase Mapping
| Phase | Requirements | Count |
|-------|-------------|-------|
| 1 | ATELIER-92..96 | 5 |
| 2 | ATELIER-97..101 | 5 |
| 3 | ATELIER-102..105 | 4 |
| 4 | ATELIER-106, 107, 117, 108, 109, 110, 111 | 7 |
| 5 | ATELIER-112, 113, 114 | 3 |
| 6 | ATELIER-115, 116 | 2 |
| **Total** | | **26** |
## v0.4 Ideation Refinements → Task Bake-In Map
All 12 accepted ideation refinements (IDEATE-31, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43) + the 1 new requirement (IDEATE-32 → ATELIER-117) are baked into the relevant phase tasks as explicit must-have notes (parallel to v0.3's Refinements Notes → Task Bake-In Map):
| IDEATE-ID | Refinement | Baked Into Task(s) | How |
|-----------|-----------|-------------------|-----|
| IDEATE-31 | MANIFEST Languages section lists 4 v0.1 stubs + 16 derived docs (refines ATELIER-111) | 04-03-01 | MANIFEST gains `## Languages` section between `## Domains` and `## Examples`; lists 4 stubs (closes pre-existing drift since v0.1, parallel to ESC-002/ATELIER-91) + 16 derived docs |
| IDEATE-32 | ARCHITECTURE.md v0.4 update (new req ATELIER-117) | 04-01-03 | ARCHITECTURE.md gains `## v0.4 Domain Additions` section + component tree entries for edge/messaging/languages-derived; closes architecture-tree drift |
| IDEATE-33 | Core Principle Coverage C1C8 → domains table update for edge + messaging (refines ATELIER-107) | 04-01-02 | domain-coverage.md C-rule coverage table extended (C4 +edge P1/P8 +messaging P7; C5 +edge P2/P4/P7/P10 +messaging P5/P8/P9; C7 +edge P10 +messaging P8/P10; C8 +edge P3/P9 +messaging P6; C2 +messaging P1/P2/P4) |
| IDEATE-34 | Matrix post-v0.4 = exactly 190; verify current 170 (corrected per G-014 — the "171 vs 170" claim was a false premise; matrix has exactly 170) (refines ATELIER-106) | 04-01-01, 06-01-02 | P4 verifies 170 exists before appending 20 rows; P6 audit verifies exactly 190 (10 × 19). Do NOT hunt for/delete a phantom 171st row |
| IDEATE-35 | streams.md + delivery-semantics.md comparison tables required (refines ATELIER-100, 101) | 02-02-03, 02-02-04 | streams.md stream-platform comparison (Kafka/Kinesis/Pulsar); delivery-semantics.md dead-letter strategy comparison |
| IDEATE-36 | Language-derived-doc anti-patterns + .ts/.py/.go/.rs artifact-type extension (refines ATELIER-110) | 04-02-03 | anti-patterns.md gains language-derived anti-patterns (new P-rule, orphan section, runtime artifact, P1P10 numbering) + v0.4 artifact types table extended to .ts/.py/.go/.rs |
| IDEATE-37 | All 6 RESEARCH chaos anti-patterns (refines ATELIER-110) | 04-02-03 | anti-patterns.md includes all 6: edge sync-loop/TTL-less/blocking-call; messaging shared-sub/blocking-consumer/unbounded-retry (not just the 4 named in req-row text) |
| IDEATE-38 | offline-first.md maps offline write-queue to testing pyramid tiers (refines ATELIER-94) | 01-02-02 | offline-first.md maps unit (conflict detection on merge function), integration (reconnect reconcile vs local store), e2e (partition simulation with fake network) |
| IDEATE-39 | delivery-semantics.md fenced idempotency-key dedup-store example (refines ATELIER-101) | 02-02-04 | delivery-semantics.md includes a fenced consumer-with-dedup-store (not prose-only) — parallel to v0.3 IDEATE-29 signed-attestation |
| IDEATE-40 | P5 cross-link audit verifies edge↔messaging intra-v0.4 links both directions (refines ATELIER-114) | 05-02-01 | Cross-link audit checks edge→messaging AND messaging→edge links resolve; language-derived docs link to both new + existing domains where relevant |
| IDEATE-41 | streams.md enumerates consumer-group rebalance strategies (refines ATELIER-100) | 02-02-03 | streams.md enumerates sticky/cooperative/eager — each with partition-stop-the-world cost + use-case fit (parallel to v0.3 IDEATE-30 drift-type enumeration) |
| IDEATE-42 | Languages Coverage sub-table = exactly 16 rows (refines ATELIER-107) | 04-01-02, 06-01-01 | domain-coverage.md gains `## Languages Coverage` sub-table with exactly 16 rows; P6 review verifies the 16 rows match the derived docs' actual section-header traces |
| IDEATE-43 | MANIFEST Cross-Cutting summary = "19 domains, 190 P-rules post-v0.4" (refines ATELIER-111) | 04-03-01 | MANIFEST Cross-Cutting summary line updates to exactly "19 domains, 190 P-rules post-v0.4" (IDEATE-18 carried forward) |
## v0.4 Decisions Logged (planning stage)
| ID | Decision | Rationale | Confidence |
|----|----------|-----------|------------|
| D-070 | P3 Wave 2 splits into 2a/2b/2c/2d labels (by language) but all are the same dependency tier | 16 derived docs are all independent post-Wave-1; the 2a/2b/2c/2d labels organize by language, the executor schedules ≤5 concurrent per config.json. Avoids inventing a false dependency between languages (parallel to v0.3 D-049 i18n/compliance 2a/2b split) | 0.88 |
| D-071 | P4 splits into 3 waves: matrix+coverage+ARCHITECTURE (Wave 1 sequential), review docs (Wave 2 parallel), MANIFEST (Wave 3 sequential after review docs settled) | Matrix is the arbiter (must be authoritative first per v0.2/v0.3 precedent); ARCHITECTURE.md follows coverage so the component tree reflects the matrix; review docs are independent of each other; MANIFEST lists file paths and totals, sequenced after anti-patterns content is settled (D-050 precedent — anti-patterns + manifest combine under lead-developer). Net 3 waves, respecting that ARCHITECTURE.md (ATELIER-117) is a new v0.4 deliverable that must follow the matrix/coverage extension | 0.83 |
| D-072 | P6 splits into 2 waves: review (Wave 1 — edge-engineer + languages-engineer review their content per D-052 before removal), ship (Wave 2 — tag + merge + remove personas) | Phase-specific personas review their authored content one final time in Wave 1 (D-052 precedent — ensures D-061 edge boundary and D-063 apply-don't-derive are checked by the specialist), then are removed in Wave 2 (roster returns to 3 per D-067). Separating review from ship ensures the specialist review completes before the personas are deactivated | 0.84 |
## Assumptions Logged
| # | Assumption | Confidence |
|---|-----------|------------|
| A-006 | The pre-existing matrix has exactly 170 P-rows (verified by grill G-014 direct count); appending 20 yields exactly 190 with no reconciliation needed. The IDEATE-34 "171-vs-170 discrepancy" was a false premise corrected by the grill (parallel to v0.3's matrix-extension assumption A-002) | 0.88 |
| A-007 | The 4 new domain P-rules (edge + messaging) map to existing core C1C8 without needing new core principles (core is stable at 8 — carried forward from A-002) | 0.95 |
| A-008 | Wave 2 of P3 (16 derived docs) can be scheduled by the executor as 5-then-5-then-5-then-1 without a hard sub-wave barrier (carried forward from v0.3 A-003) | 0.90 |
| A-009 | The Languages Coverage sub-table's 16 rows can be authored in P4 from the P3 derived docs' actual section-header traces (the RESEARCH-v0.4 sketch is the expected trace set; finalized when the derived docs are authored) | 0.85 |
| A-010 | The edge↔messaging intra-milestone cross-links (edge/iot.md → messaging/queues.md) dangle at the P1 tag (v0.3.1) and resolve once P2 ships (v0.3.2); this is acceptable per D-053 vertical-slice integrity and verified bidirectionally in P5 (IDEATE-40) | 0.86 |
## Milestone Type: NFR
Per branch-strategy.md, this is an NFR milestone (all phases are docs/chore — no `feat` code). NFR milestones produce progressive patches; the final patch (v0.3.6) IS the v0.4 deliverable. No separate minor tag.
+622
View File
@@ -0,0 +1,622 @@
# Atelier — Plan (v0.1)
> Vertical-slice plans with wave ordering. Each phase is a vertical slice delivering a usable subset of the framework. Plans reference REQ-IDs from `.ciagent/atelier/REQUIREMENTS.md`.
## Phase 0 — Pre-Execution (COMPLETE)
Stages: SPECIFY ✓ → CLARIFY ✓ → RESEARCH ✓ → PLAN ✓ → SHIP
Tag: v0.0.0
## Phase 1 — Core Foundation
**Goal:** Establish the 8 core principles + entry docs. The foundation everything else depends on.
**Branch:** `phase/01-core-foundation` (from `milestone/v0.1-atelier`)
**Persona:** tech-writer (author), domain-expert (validate traceability)
**Tag:** v0.0.1
**Requirements:** ATELIER-04, ATELIER-05, ATELIER-06, ATELIER-07, ATELIER-08
### Wave 1 (sequential — foundation must exist before anything else)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 01-01-01 | `core/first-principles.md` | tech-writer | ATELIER-04 | 8 principles (C1C8) named exactly per matrix key, each with definition + "what violates" |
| 01-01-02 | `core/conflict-resolution.md` | tech-writer | ATELIER-05 | Precedence order C1>...>C8, cross-doc conflict rules, hierarchy enforcement |
| 01-01-03 | `core/reading-order.md` | tech-writer | ATELIER-06 | Reading order per MANIFEST.md, guided tour for agents + humans |
### Wave 2 (parallel — entry docs, independent)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 01-02-01 | `README.md` | tech-writer | ATELIER-07 | What/who/quickstart, links to MANIFEST + reading-order |
| 01-02-02 | `LICENSE` | tech-writer | ATELIER-08 | MIT license full text (D-001) |
**Verify (P1):**
- Structural: all 3 core files + README + LICENSE exist
- Behavioral: every C-rule name in matrix matches a section in `core/first-principles.md`
- Security: LICENSE present (legal)
- Quality: reading-order.md links resolve to existing files
## Phase 2 — Domain First Principles (Missing Ones)
**Goal:** Author the 5 domain first-principles not yet written (uiux, errors already done? No — errors IS done in P0. Check: P0 wrote api, security, data, testing, performance, observability, errors, documentation, concurrency, devops. Missing: **uiux** only. Plus uiux first-principles was deferred D-004.)
Wait — recount. P0 init wrote first-principles for: api, security, data, testing, performance, observability, errors, documentation, concurrency, devops (10). Missing: **uiux** (deferred D-004). So P2 is small — just uiux first-principles. But also need to validate the other 10 exist and are complete.
Actually, re-examining: P0 already wrote 10 domain first-principles. Only `domains/uiux/first-principles.md` is missing. So P2 = uiux first-principles + a completeness audit of the other 10.
**Goal (revised):** Author `domains/uiux/first-principles.md` (P1P10) and audit the 10 existing first-principles for completeness.
**Branch:** `phase/02-domain-first-principles`
**Persona:** tech-writer (author), domain-expert (audit)
**Tag:** v0.0.2
**Requirements:** ATELIER-09
### Wave 1
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 02-01-01 | `domains/uiux/first-principles.md` | tech-writer | ATELIER-09 | 10 principles (P1P10) matching matrix UI/UX rows; P1 User Primacy, P2 Accessibility, etc. |
| 02-01-02 | Audit 10 existing first-principles | domain-expert | (meta) | Each has 10 principles, each traces to matrix; log gaps to RESEARCH.md |
**Verify (P2):** Structural (11/11 first-principles exist), Behavioral (each P-rule in matrix has a section), Quality (no orphaned rules).
## Phase 3 — Domain Derived Docs
**Goal:** Author the derived/topic docs for each domain (the non-first-principles files in the spec tree).
**Branch:** `phase/03-domain-derived-docs`
**Persona:** tech-writer
**Tag:** v0.0.3
**Requirements:** ATELIER-14..ATELIER-20
### Wave 1 (parallel — domains are independent)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 03-01-01 | `domains/api/rest.md` | tech-writer | ATELIER-14 | REST-specific rules deriving from api first-principles |
| 03-01-02 | `domains/api/graphql.md` | tech-writer | ATELIER-14 | GraphQL-specific rules |
| 03-01-03 | `domains/api/versioning.md` | tech-writer | ATELIER-14 | API evolution rules |
| 03-01-04 | `domains/api/error-responses.md` | tech-writer | ATELIER-14 | Error contract design |
| 03-01-05 | `domains/api/pagination.md` | tech-writer | ATELIER-14 | Pagination patterns |
| 03-01-06 | `domains/security/authentication.md` | tech-writer | ATELIER-15 | Auth rules |
| 03-01-07 | `domains/security/authorization.md` | tech-writer | ATELIER-15 | Authz rules |
| 03-01-08 | `domains/security/input-validation.md` | tech-writer | ATELIER-15 | Input validation rules |
| 03-01-09 | `domains/security/secrets.md` | tech-writer | ATELIER-15 | Secret management rules |
| 03-01-10 | `domains/security/supply-chain.md` | tech-writer | ATELIER-15 | Supply chain rules |
| 03-01-11 | `domains/data/schema-design.md` | tech-writer | ATELIER-16 | Schema design rules |
| 03-01-12 | `domains/data/migrations.md` | tech-writer | ATELIER-16 | Migration rules |
| 03-01-13 | `domains/data/indexing.md` | tech-writer | ATELIER-16 | Indexing rules |
| 03-01-14 | `domains/testing/pyramid.md` | tech-writer | ATELIER-17 | Test pyramid rules |
| 03-01-15 | `domains/testing/fixtures.md` | tech-writer | ATELIER-17 | Fixture rules |
| 03-01-16 | `domains/performance/frontend.md` | tech-writer | ATELIER-18 | Frontend perf rules |
| 03-01-17 | `domains/performance/backend.md` | tech-writer | ATELIER-18 | Backend perf rules |
| 03-01-18 | `domains/observability/logging.md` | tech-writer | ATELIER-19 | Logging rules |
| 03-01-19 | `domains/observability/metrics.md` | tech-writer | ATELIER-19 | Metrics rules |
| 03-01-20 | `domains/observability/tracing.md` | tech-writer | ATELIER-19 | Tracing rules |
| 03-01-21 | `domains/uiux/tokens.md` | tech-writer | ATELIER-20 | Design token rules |
| 03-01-22 | `domains/uiux/copywriting.md` | tech-writer | ATELIER-20 | UI microcopy rules |
| 03-01-23 | `domains/errors/patterns.md` | tech-writer | (derived) | Error patterns |
| 03-01-24 | `domains/documentation/doc-templates.md` | tech-writer | (derived) | Doc templates |
| 03-01-25 | `domains/concurrency/patterns.md` | tech-writer | (derived) | Concurrency patterns |
| 03-01-26 | `domains/devops/ci-cd.md` | tech-writer | (derived) | CI/CD rules |
| 03-01-27 | `domains/devops/environments.md` | tech-writer | (derived) | Environment rules |
**Verify (P3):** Structural (all derived docs in spec tree exist), Behavioral (each derives from its domain's first-principles), Quality (no contradictions with core).
## Phase 4 — Matrix + Review
**Goal:** Complete the cross-reference matrix and review tooling.
**Branch:** `phase/04-matrix-review`
**Persona:** domain-expert (matrix), tech-writer (review)
**Tag:** v0.0.4
**Requirements:** ATELIER-21..ATELIER-25
### Wave 1 (matrix must be complete before review references it)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 04-01-01 | `matrix/principles-matrix.md` (verify complete) | domain-expert | ATELIER-21 | All 11 domains × 10 principles mapped to C-rules (already written in P0; verify) |
| 04-01-02 | `matrix/domain-coverage.md` | domain-expert | ATELIER-22 | Where each principle applies across the codebase |
### Wave 2 (parallel — review docs independent)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 04-02-01 | `review/agent-checklist.md` | tech-writer | ATELIER-23 | Pre-completion checklist covering all 8 core principles + domain triggers |
| 04-02-02 | `review/peer-review-checklist.md` | tech-writer | ATELIER-24 | Human review checklist |
| 04-02-03 | `review/anti-patterns.md` | tech-writer | ATELIER-25 | Catalog of violations (from each domain's "what violates" tables) |
**Verify (P4):** Structural (matrix complete, 3 review docs exist), Behavioral (every matrix row references an existing P-rule and C-rule), Security (agent-checklist covers security domain), Quality (anti-patterns cross-reference principles).
## Phase 5 — Examples
**Goal:** Worked examples showing the framework in action.
**Branch:** `phase/05-examples`
**Persona:** tech-writer
**Tag:** v0.0.5
**Requirements:** ATELIER-26..ATELIER-30
### Wave 1 (parallel — examples independent)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 05-01-01 | `examples/good/api-endpoint.md` | tech-writer | ATELIER-26 | Good REST endpoint example, references API P-rules |
| 05-01-02 | `examples/good/react-component.md` | tech-writer | ATELIER-27 | Good component example, references UI/UX P-rules |
| 05-01-03 | `examples/good/db-schema.md` | tech-writer | ATELIER-28 | Good schema example, references Data P-rules |
| 05-01-04 | `examples/good/error-handler.md` | tech-writer | ATELIER-29 | Good error handler, references Errors P-rules |
| 05-01-05 | `examples/bad/god-object.md` | tech-writer | ATELIER-30 | Bad example with violated principles cited |
| 05-01-06 | `examples/bad/silent-error.md` | tech-writer | ATELIER-30 | Bad example with violated principles cited |
| 05-01-07 | `examples/bad/leaky-abstraction.md` | tech-writer | ATELIER-30 | Bad example with violated principles cited |
**Verify (P5):** Structural (7 example files), Behavioral (good examples cite P-rules they follow; bad examples cite P-rules they violate), Quality (examples are realistic, not strawmen).
## Phase 6 — Languages + Meta
**Goal:** Language-specific application docs + meta documents.
**Branch:** `phase/06-languages-meta`
**Persona:** tech-writer
**Tag:** v0.0.6
**Requirements:** ATELIER-31..ATELIER-33
### Wave 1 (parallel)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 06-01-01 | `languages/typescript.md` | tech-writer | ATELIER-31 | TS-specific application of domain principles |
| 06-01-02 | `languages/python.md` | tech-writer | ATELIER-31 | Python-specific application |
| 06-01-03 | `languages/go.md` | tech-writer | ATELIER-31 | Go-specific application |
| 06-01-04 | `languages/rust.md` | tech-writer | ATELIER-31 | Rust-specific application |
| 06-01-05 | `CHANGELOG.md` | tech-writer | ATELIER-32 | Version history of the framework (v0.1.0 entry) |
| 06-01-06 | `CONTRIBUTING.md` | tech-writer | ATELIER-33 | How to propose changes to the framework |
**Verify (P6):** Structural (4 language docs + CHANGELOG + CONTRIBUTING), Behavioral (language docs reference domain rules they apply), Quality (CHANGELOG follows Keep a Changelog format).
## Phase 7 — Final Review + Ship (Milestone Release)
**Goal:** Review all phases, audit project health, ship the milestone.
**Branch:** `phase/07-final-review-ship` (from `milestone/v0.1-atelier`)
**Tag:** v0.0.7 (IS the v0.1 milestone release)
**Requirements:** ATELIER-34, ATELIER-35
### Wave 1 — Review (ciagent-review)
| Task | Persona | Must-have |
|------|---------|-----------|
| 07-01-01 | ci-reviewer | Multi-persona review across all phases; auto-apply P0 fixes; flag P1+ for post-hoc |
| 07-01-02 | ci-reviewer | Verify every P-rule in matrix has a corresponding section in its domain first-principles |
### Wave 2 — Audit (ciagent-audit)
| Task | Persona | Must-have |
|------|---------|-----------|
| 07-02-01 | ci-auditor | Reconstruction test: MANIFEST.md lists all framework docs; every listed doc exists |
| 07-02-02 | ci-auditor | Branch hygiene: only `main` + `milestone/v0.1-atelier` after phase branches merged |
| 07-02-03 | ci-auditor | Commit discipline: every commit has `---ci---` block |
### Wave 3 — Ship (ciagent-ship)
| Task | Persona | Must-have |
|------|---------|-----------|
| 07-03-01 | ship | Merge `phase/07``milestone/v0.1-atelier` |
| 07-03-02 | ship | Merge `milestone/v0.1-atelier``main` |
| 07-03-03 | ship | Tag `v0.0.7` (IS the v0.1 release) |
| 07-03-04 | ship | Create Gitea release for v0.0.7 with full milestone summary |
| 07-03-05 | ship | Delete all milestone branches; tags preserve history |
## Milestone Type: NFR
Per branch-strategy.md, this is an NFR milestone (all phases are docs/chore — no `feat` code). NFR milestones produce progressive patches; the final patch (v0.0.7) IS the deliverable. No separate minor tag.
## Wave Ordering Summary
| Phase | Waves | Parallelism |
|-------|-------|-------------|
| P1 | 2 | Wave 1 sequential (foundation), Wave 2 parallel (entry docs) |
| P2 | 1 | Single wave (small phase) |
| P3 | 1 | Single large parallel wave (27 independent docs) |
| P4 | 2 | Wave 1 matrix, Wave 2 review docs parallel |
| P5 | 1 | Single parallel wave (7 examples) |
| P6 | 1 | Single parallel wave (6 docs) |
| P7 | 3 | Sequential: review → audit → ship |
---
# Atelier — Plan (v0.2)
> Vertical-slice plans with wave ordering for milestone v0.2 (Infrastructure as Code + Kubernetes). Plans reference REQ-IDs from `.ciagent/atelier/REQUIREMENTS.md` (ATELIER-36..59). NFR milestone — all phases produce docs; no `feat` code.
## Phase 0 — Pre-Execution (COMPLETE)
Stages: SPECIFY ✓ → CLARIFY ✓ → RESEARCH ✓ → IDEATE ✓ → PLAN ✓ → SHIP
Branch: `atelier/phase/00-pre-execution`
Tag: v0.1.0
## Phase 1 — Infrastructure as Code Domain
**Goal:** Author the `domains/infrastructure-as-code/` tree — 10 first principles (P1P10) plus 4 derived docs (terraform, opentofu, state, modules). Resolves architectural drift (IDEATE-01).
**Branch:** `atelier/phase/01-infrastructure-as-code` (from `atelier/milestone/v0.2-iac-k8s`)
**Personas:** platform-engineer (author), domain-expert (validate traceability), tech-writer (style/format)
**Tag:** v0.1.1
**Requirements:** ATELIER-36, ATELIER-37, ATELIER-38, ATELIER-39, ATELIER-40
### Wave 1 (sequential — first-principles must exist before derived docs)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 01-01-01 | `domains/infrastructure-as-code/first-principles.md` | platform-engineer | ATELIER-36 | 10 principles (P1P10) per RESEARCH.md; each names the core C-rule(s) it derives from; each has definition + "what violates" |
### Wave 2 (parallel — derived docs, independent)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 01-02-01 | `domains/infrastructure-as-code/terraform.md` | platform-engineer | ATELIER-37 | HCL structure, providers, resources, data sources, workspaces, init/plan/apply/destroy, registry, state backends; cross-link to state.md + modules.md |
| 01-02-02 | `domains/infrastructure-as-code/opentofu.md` | platform-engineer | ATELIER-38 | Fork lineage (BUSL→MPL via LF), CLI parity, registry parity, migration from Terraform, governance, when-to-choose; cross-link to terraform.md |
| 01-02-03 | `domains/infrastructure-as-code/state.md` | platform-engineer | ATELIER-39 | Backends (S3+DynamoDB, GCS, Azure, local), locking, env isolation, state commands, sensitive values, import/export; **state backend comparison table** (IDEATE-11) |
| 01-02-04 | `domains/infrastructure-as-code/modules.md` | platform-engineer | ATELIER-40 | Module structure, inputs/outputs, versioning, source patterns (registry/git/local), composition, module-vs-copy boundary |
**Verify (P1):**
- Structural: 5 files exist under `domains/infrastructure-as-code/`
- Behavioral: every P1P10 in first-principles names ≥1 C-rule (domain-expert sign-off)
- Security: P10 (Secrets Never in Code) section present
- Quality: each derived doc has ≥1 outbound cross-link to a MANIFEST-listed doc (IDEATE-08)
## Phase 2 — Kubernetes Domain
**Goal:** Author the `domains/kubernetes/` tree — 10 first principles (P1P10) plus 6 derived docs (workloads, networking, storage, rbac, helm, kustomize).
**Branch:** `atelier/phase/02-kubernetes` (from `atelier/milestone/v0.2-iac-k8s`)
**Personas:** platform-engineer (author), domain-expert (validate), tech-writer (style)
**Tag:** v0.1.2
**Requirements:** ATELIER-41, ATELIER-42, ATELIER-43, ATELIER-44, ATELIER-45, ATELIER-46, ATELIER-47
### Wave 1 (sequential — first-principles first)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 02-01-01 | `domains/kubernetes/first-principles.md` | platform-engineer | ATELIER-41 | 10 principles (P1P10) per RESEARCH.md; each names core C-rule(s); each has definition + "what violates" |
### Wave 2 (parallel — derived docs)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 02-02-01 | `domains/kubernetes/workloads.md` | platform-engineer | ATELIER-42 | Pod, ReplicaSet, Deployment, StatefulSet, DaemonSet, Job/CronJob, probes, lifecycle, PDB |
| 02-02-02 | `domains/kubernetes/networking.md` | platform-engineer | ATELIER-43 | Service types, Ingress, Gateway API, EndpointSlices, NetworkPolicy, DNS, dual-stack |
| 02-02-03 | `domains/kubernetes/storage.md` | platform-engineer | ATELIER-44 | Volumes, PV/PVC, StorageClass, CSI, dynamic provisioning, snapshots, reclaim policies |
| 02-02-04 | `domains/kubernetes/rbac.md` | platform-engineer | ATELIER-45 | Role/ClusterRole, bindings, ServiceAccount, **Pod Security Standards + Admission** (IDEATE-12), RBAC good practices; cross-link security/authorization |
| 02-02-05 | `domains/kubernetes/helm.md` | platform-engineer | ATELIER-46 | Chart structure, values, templates, registries, release mgmt; **Helm vs Kustomize decision matrix** (IDEATE-10) |
| 02-02-06 | `domains/kubernetes/kustomize.md` | platform-engineer | ATELIER-47 | Base + overlays, kustomization.yaml, patches, no templating; **Helm vs Kustomize decision matrix** (IDEATE-10) |
**Verify (P2):**
- Structural: 7 files exist under `domains/kubernetes/`
- Behavioral: every P1P10 traces to ≥1 C-rule (domain-expert sign-off)
- Security: P7 (RBAC least privilege) + P9 (config/secrets separate) sections present
- Quality: each derived doc ≥1 outbound cross-link (IDEATE-08); helm.md and kustomize.md share the decision matrix consistently
## Phase 3 — Matrix + Review Integration
**Goal:** Extend the matrix, review docs, and manifest to absorb the 20 new P-rules and 11 new documents. Closes the traceability loop.
**Branch:** `atelier/phase/03-matrix-review` (from `atelier/milestone/v0.2-iac-k8s`)
**Personas:** domain-expert (matrix + anti-patterns), tech-writer (checklists + manifest), platform-engineer (content review)
**Tag:** v0.1.3
**Requirements:** ATELIER-48, ATELIER-49, ATELIER-50, ATELIER-51, ATELIER-52, ATELIER-59
### Wave 1 (sequential — matrix is the arbiter, must be authoritative first)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 03-01-01 | `matrix/principles-matrix.md` (extend) | domain-expert | ATELIER-48 | Add "Infrastructure as Code" + "Kubernetes" sections, 10 rows each, format matching v0.1 tables; **review check: row count per new domain = 10, each row ≥1 C-rule** (IDEATE-02, IDEATE-13) |
### Wave 2 (parallel — independent extensions)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 03-02-01 | `matrix/domain-coverage.md` (extend) | domain-expert | ATELIER-49 | Add rows for both new domains with schema: domain, P-count, derived-doc-count, manifest-listed, status (IDEATE-03) |
| 03-02-02 | `review/agent-checklist.md` (extend) | tech-writer | ATELIER-50 | Add "If Infrastructure as Code" + "If Kubernetes" trigger sections (IDEATE-05) |
| 03-02-03 | `review/peer-review-checklist.md` (extend) | tech-writer | ATELIER-59 | Add IaC + k8s peer-review sections (parity with agent-checklist, IDEATE-09) |
| 03-02-04 | `review/anti-patterns.md` (extend) | domain-expert | ATELIER-51 | Add IaC violations (unlocked state, hardcoded secrets, unversioned modules, manual drift, local state in prod) + k8s violations (bare pod, no resources, cluster-admin, :latest, no probes, emptyDir for persistent data) + **orphaned P-rule** + **deployable example artifact** (IDEATE-06, IDEATE-13, IDEATE-14) |
| 03-02-05 | `MANIFEST.md` (extend) | tech-writer | ATELIER-52 | Add both new domains + all 11 derived docs to the Domains table (IDEATE-04) |
**Verify (P3):**
- Structural: matrix has 13 domain sections (11 v0.1 + 2 new), 130 P-rules total
- Behavioral: every new P-rule has a matrix row; domain-expert verifies no orphans
- Security: anti-patterns cover secrets-in-HCL and cluster-admin
- Quality: MANIFEST lists all new docs; unlisted = not part of framework
## Phase 4 — Examples + Cross-Links
**Goal:** Add good + bad examples for IaC/k8s and verify cross-domain links from new domains to existing ones. Examples are markdown with fenced code only (no standalone .tf/.yaml).
**Branch:** `atelier/phase/04-examples-crosslinks` (from `atelier/milestone/v0.2-iac-k8s`)
**Personas:** platform-engineer (examples), tech-writer (cross-link audit), domain-expert (P-rule citation)
**Tag:** v0.1.4
**Requirements:** ATELIER-53, ATELIER-54, ATELIER-55, ATELIER-56
### Wave 1 (parallel — examples independent)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 04-01-01 | `examples/good/terraform-module.md` | platform-engineer | ATELIER-53 | Good IaC example; markdown with fenced HCL; demonstrates P6 Modules Compose + P8 Remote State; cross-link to modules.md + state.md |
| 04-01-02 | `examples/good/k8s-deployment.md` | platform-engineer | ATELIER-54 | Good k8s example; markdown with fenced YAML; demonstrates P4 requests/limits + P5 probes + P7 RBAC; cross-link to workloads.md + rbac.md |
| 04-01-03 | `examples/bad/terraform-unlocked-state.md` | platform-engineer | ATELIER-55 | Bad IaC example; cites P8 breached (no locking) + P10 if secrets in state; per IDEATE-07 |
| 04-01-04 | `examples/bad/k8s-bare-pod-no-resources.md` | platform-engineer | ATELIER-55 | Bad k8s example; cites P2 (bare pod) + P4 (no requests/limits) breached; per IDEATE-07 |
### Wave 2 (sequential — cross-link audit after all docs exist)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 04-02-01 | Cross-link audit (all new derived docs) | tech-writer | ATELIER-56 | Review check: every new derived doc ≥1 outbound cross-link to a MANIFEST-listed existing domain doc (devops/security/observability/data); links resolve (IDEATE-08) |
**Verify (P4):**
- Structural: 4 new example files exist (all .md)
- Behavioral: each bad example cites the P-rule breached
- Security: no standalone .tf/.yaml files (deployable artifact mitigation, IDEATE-14)
- Quality: all cross-links resolve to MANIFEST-listed docs
## Phase 5 — Final Review + Ship (N+1)
**Goal:** Multi-persona review across all v0.2 phases, audit, milestone ship.
**Branch:** `atelier/phase/05-final-review-ship` (from `atelier/milestone/v0.2-iac-k8s`)
**Personas:** lead-developer (coordinate), domain-expert (review), tech-writer (review), platform-engineer (review, then removed)
**Tag:** v0.1.5 (IS the v0.2 milestone release — NFR, no separate minor tag)
**Requirements:** ATELIER-57, ATELIER-58
### Wave 1 (sequential — review → audit → ship)
| Task | Activity | Persona | REQ-ID | Must-have |
|------|----------|---------|--------|-----------|
| 05-01-01 | `ciagent-review` — multi-persona review of all v0.2 changes | lead-developer | ATELIER-57 | Auto-apply P0 fixes; flag P1+ for post-hoc; if P1+ found, fix in this phase |
| 05-01-02 | `ciagent-audit` — reconstruction + discipline | lead-developer | ATELIER-57 | git log matches .ciagent/ files; branch hygiene; commit discipline; fix critical issues here |
| 05-01-03 | `ciagent-ship` — milestone ship | lead-developer | ATELIER-58 | Merge phase/05 → milestone/v0.2 → main; tag v0.1.5; Gitea release; delete all v0.2 branches |
| 05-01-04 | Complete milestone (REQUIREMENTS + ROADMAP) | lead-developer | ATELIER-58 | Mark all v0.2 requirements `covered`; ROADMAP v0.2 → complete; clear checkpoint |
**Verify (P5):**
- Structural: all 24 v0.2 requirements (ATELIER-36..59) marked covered
- Behavioral: reconstruction test passes (git log ↔ .ciagent/)
- Security: audit clean (no critical issues)
- Quality: milestone merged to main, tag v0.1.5 exists, all v0.2 branches deleted
## v0.2 Wave Ordering Summary
| Phase | Waves | Parallelism |
|-------|-------|-------------|
| P0 | (pre-exec) | Sequential stages |
| P1 | 2 | Wave 1 sequential (first-principles), Wave 2 parallel (4 derived docs) |
| P2 | 2 | Wave 1 sequential (first-principles), Wave 2 parallel (6 derived docs) |
| P3 | 2 | Wave 1 sequential (matrix), Wave 2 parallel (5 extensions) |
| P4 | 2 | Wave 1 parallel (4 examples), Wave 2 sequential (cross-link audit) |
| P5 | 1 | Sequential: review → audit → ship → complete |
## v0.2 Requirements → Phase Mapping
| Phase | Requirements | Count |
|-------|-------------|-------|
| 1 | ATELIER-36..40 | 5 |
| 2 | ATELIER-41..47 | 7 |
| 3 | ATELIER-48..52, 59 | 6 |
| 4 | ATELIER-53..56 | 4 |
| 5 | ATELIER-57, 58 | 2 |
| **Total** | | **24** |
---
# Atelier — Plan (v0.3)
> Vertical-slice plans with wave ordering for milestone v0.3 (GitOps + Operators + AI/ML + i18n + Compliance). Plans reference REQ-IDs from `.ciagent/atelier/REQUIREMENTS.md` (ATELIER-60..91). NFR milestone — all phases produce docs; no `feat` code. Per `parallelization.max_concurrent_agents = 5`, wave parallelism is capped at 5 concurrent tasks; waves larger than 5 are split into sub-waves.
## Phase 0 — Pre-Execution (COMPLETE)
Stages: SPECIFY ✓ → CLARIFY ✓ → RESEARCH ✓ → IDEATE ✓ → PLAN ✓ → GRILL → SHIP
Branch: `atelier/phase/00-pre-execution`
Tag: v0.2.0
## Phase 1 — GitOps + Operators Domain
**Goal:** Author the `domains/gitops-operators/` tree — 10 first principles (P1P10) plus 4 derived docs (argocd, flux, operators, progressive-delivery). Grounded in CNCF OpenGitOps Principles v1.0.0 + the Operator pattern. Each P-rule derives from core C1C8 (matrix extension lands in P4).
**Branch:** `atelier/phase/01-gitops-operators` (from `atelier/milestone/v0.3-atelier`)
**Personas:** platform-engineer (author), domain-expert (validate traceability), tech-writer (style/format)
**Tag:** v0.2.1
**Requirements:** ATELIER-60, ATELIER-61, ATELIER-62, ATELIER-63, ATELIER-64
### Wave 1 (sequential — first-principles must exist before derived docs)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 01-01-01 | `domains/gitops-operators/first-principles.md` | platform-engineer | ATELIER-60 | 10 principles (P1P10) per RESEARCH.md (Git is Source of Truth, Pull Don't Push, Continuous Reconciliation, Operators Encode Domain Knowledge, Progressive Delivery is Reversible, Reconcile Don't Mutate, Failure is Observable, Least Privilege Reconciliation); each names the core C-rule(s) it derives from; each has definition + "what violates" |
### Wave 2 (parallel — 4 derived docs, independent; ≤5 concurrent)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 01-02-01 | `domains/gitops-operators/argocd.md` | platform-engineer | ATELIER-61 | Application CRD, App-of-Apps, sync waves, health/status, diff, RBAC/SSO, multi-cluster, sync windows; **ArgoCD vs Flux decision matrix** (IDEATE-21, D-039); cross-link to flux.md, kubernetes/{workloads,rbac,helm,kustomize}.md, devops, security/secrets, observability/metrics |
| 01-02-02 | `domains/gitops-operators/flux.md` | platform-engineer | ATELIER-62 | GitOps Toolkit controllers (source, kustomize, helm, notification), composable architecture, HR/Kustomization/HelmRelease CRDs, OCI sources; **ArgoCD vs Flux decision matrix** (IDEATE-21, D-039); cross-link to argocd.md + kubernetes/helm.md + kubernetes/kustomize.md |
| 01-02-03 | `domains/gitops-operators/operators.md` | platform-engineer | ATELIER-63 | Operator pattern, CRDs, controllers, Operator SDK/OLM, when-to-write-an-operator vs Helm chart, scope/responsibility boundaries; cross-link kubernetes/{workloads,rbac}.md + infrastructure-as-code/modules.md |
| 01-02-04 | `domains/gitops-operators/progressive-delivery.md` | platform-engineer | ATELIER-64 | Argo Rollouts + Flagger, canary/blue-green, analysis templates (metrics/counters), abort/rollback; cross-link devops (P4 Rollback First, P5 Progressive Delivery) + observability/metrics + kubernetes/workloads.md |
**Verify (P1):**
- Structural: 5 files exist under `domains/gitops-operators/`
- Behavioral: every P1P10 in first-principles names ≥1 C-rule (domain-expert sign-off)
- Security: P3 (Pull, Don't Push) + P10 (Least Privilege Reconciliation) sections present
- Quality: each derived doc has ≥1 outbound cross-link to a MANIFEST-listed doc (IDEATE-08 carried forward); argocd.md and flux.md share the decision matrix consistently (IDEATE-21)
## Phase 2 — AI/ML Domain
**Goal:** Author the `domains/ai-ml/` tree — 10 first principles (P1P10) plus 4 derived docs (data-versioning, model-evaluation, serving, monitoring-drift). Scope = engineering discipline (D-023), NOT algorithm/model design. Reproducibility and lineage are non-negotiables.
**Branch:** `atelier/phase/02-ai-ml` (from `atelier/milestone/v0.3-atelier`)
**Personas:** ml-engineer (author), domain-expert (validate traceability), tech-writer (style/format)
**Tag:** v0.2.2
**Requirements:** ATELIER-65, ATELIER-66, ATELIER-67, ATELIER-68, ATELIER-69
### Wave 1 (sequential — first-principles first)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 02-01-01 | `domains/ai-ml/first-principles.md` | ml-engineer | ATELIER-65 | 10 principles (P1P10) per RESEARCH.md (Reproducibility First Class, Data is Versioned Not Just Code, Lineage Traceable End-to-End, Evaluation Defined Before Training, Models are Versioned Artifacts, Serving is Observable, Drift is Expected and Detected, Inference Inputs are Validated, Pipelines Compose Notebooks Don't, Rollback Includes the Model); each names core C-rule(s); each has definition + "what violates"; ml-engineer constraint "engineering discipline not algorithm design (D-023)" enforced |
### Wave 2 (parallel — 4 derived docs, independent; ≤5 concurrent)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 02-02-01 | `domains/ai-ml/data-versioning.md` | ml-engineer | ATELIER-66 | DVC/Delta Lake/LakeFS patterns, data lineage, dataset hashing, train/val/test split versioning; **tool comparison table: DVC vs Delta Lake vs LakeFS** covering versioning model, lineage, use-case fit (IDEATE-22, D-040); cross-link data/{migrations,schema-design}.md + devops/P1 Reproducibility |
| 02-02-02 | `domains/ai-ml/model-evaluation.md` | ml-engineer | ATELIER-67 | Metric selection, offline/online eval, holdout integrity, bias/fairness checks (engineering angle), eval-as-a-gate; cross-link data/schema-design.md (eval input contract) + testing/pyramid.md |
| 02-02-03 | `domains/ai-ml/serving.md` | ml-engineer | ATELIER-68 | KServe/Seldon/BentoML, inference as a service, batching, latency SLAs, canarying models; cross-link kubernetes/workloads.md + devops (P5 Progressive Delivery, P7 Immutability) + performance/backend.md + security/input-validation.md |
| 02-02-04 | `domains/ai-ml/monitoring-drift.md` | ml-engineer | ATELIER-69 | Evidently/Great Expectations, alerting, retraining triggers; **drift-type enumeration: data drift, concept drift, prediction drift — each with a distinct detection signal** (IDEATE-30, D-048); cross-link observability/{metrics,logging}.md + ai-ml/serving.md |
**Verify (P2):**
- Structural: 5 files exist under `domains/ai-ml/`
- Behavioral: every P1P10 traces to ≥1 C-rule (domain-expert sign-off)
- Security: P8 (Inference Inputs are Validated) section present
- Quality: each derived doc ≥1 outbound cross-link (IDEATE-08); data-versioning.md tool comparison table present (IDEATE-22); monitoring-drift.md enumerates 3 drift types with detection signals (IDEATE-30); no algorithm-design content (D-023 enforced, ml-engineer constraint)
## Phase 3 — i18n + Compliance Domains
**Goal:** Author two smaller-surface domains in one phase (D-022): `domains/i18n/` (10 first principles + 4 derived docs) and `domains/compliance/` (10 first principles + 4 derived docs). i18n grounded in ICU/CLDR + BCP 47 + W3C i18n. Compliance is framework-agnostic (D-024 — no regulation-specific docs). Both domains' first-principles land in Wave 1 (independent of each other), then derived docs in Wave 2.
**Branch:** `atelier/phase/03-i18n-compliance` (from `atelier/milestone/v0.3-atelier`)
**Personas:** tech-writer (author, both domains), domain-expert (validate traceability for both)
**Tag:** v0.2.3
**Requirements:** ATELIER-70, ATELIER-71, ATELIER-72, ATELIER-73, ATELIER-74, ATELIER-75, ATELIER-76, ATELIER-77, ATELIER-78, ATELIER-79
### Wave 1 (parallel — 2 first-principles, independent; ≤5 concurrent)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 03-01-01 | `domains/i18n/first-principles.md` | tech-writer | ATELIER-70 | 10 principles (P1P10) per RESEARCH.md (Source Language is a Locale Not the Default, Locale Identifiers Standardized BCP 47, Resources External Not Inline, Plural/Gender Parameterized ICU MessageFormat, Formatting Locale-Aware ICU/CLDR, Text Direction is Layout Primitive, Layout Accommodates Expansion, Pseudo-Locales Test Early, Images/Icons Cultural, Translation Reversible and Versioned); each names core C-rule(s); each has definition + "what violates" |
| 03-01-02 | `domains/compliance/first-principles.md` | tech-writer | ATELIER-75 | 10 principles (P1P10) per RESEARCH.md (Audit Logs Append-Only, Every Significant Action Logged, Retention is Policy Not Storage, Policy is Code, Policy Evaluated as a Gate, Evidence Collected Continuously, Identity Attributable, Subject Access Honored, Secrets Redacted in Audit, Compliance Posture Observable); each names core C-rule(s); each has definition + "what violates"; framework-agnostic (D-024 — no GDPR/HIPAA/SOC2-specific content) |
### Wave 2 (parallel — 8 derived docs, independent; split into 2 sub-waves of 4 to respect max_concurrent_agents=5)
**Wave 2a (i18n derived docs, ≤5 concurrent)**
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 03-02a-01 | `domains/i18n/locale-resources.md` | tech-writer | ATELIER-71 | Resource file formats (.po/.pot, JSON, Fluent FTL, ICU Resource Bundle), key naming, namespaces, fallback chains, extraction tooling; cross-link uiux/copywriting.md + api/error-responses.md |
| 03-02a-02 | `domains/i18n/formatting.md` | tech-writer | ATELIER-72 | ICU/CLDR/Intl for dates, times, numbers, currencies, units, relative time, plural rules; BCP 47 tags; cross-link api/error-responses.md (localized errors) + data/schema-design.md |
| 03-02a-03 | `domains/i18n/rtl-bidi.md` | tech-writer | ATELIER-73 | Logical vs physical CSS properties, bidi algorithm (UAX #9), `dir` attribute, mirroring, common pitfalls (icons, numbers in RTL); cross-link uiux/{components,accessibility}.md |
| 03-02a-04 | `domains/i18n/testing-i18n.md` | tech-writer | ATELIER-74 | Pseudo-locales, snapshot testing per locale, RTL coverage, missing-key detection; **pseudo-locale tier mapping to testing pyramid: unit (missing-key), integration (snapshot per locale), e2e (RTL coverage)** (IDEATE-28, D-046); cross-link testing/{fixtures,pyramid}.md |
**Wave 2b (compliance derived docs, ≤5 concurrent; runs in parallel with 2a — total 8 tasks, but capped at 5 → executor schedules 5 then 3)**
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 03-02b-01 | `domains/compliance/audit-logs.md` | tech-writer | ATELIER-76 | Append-only log patterns, structured audit events, CloudTrail/Cloud-Audit-Log conventions, queryability, retention of logs; cross-link observability/logging.md + security/authorization.md |
| 03-02b-02 | `domains/compliance/data-retention.md` | tech-writer | ATELIER-77 | Retention policies as code, lifecycle rules, deletion-as-a-feature, GDPR/CCPA abstracted to principles (not regulation-specific, D-024), retention vs backup distinction; cross-link data/{migrations,schema-design}.md |
| 03-02b-03 | `domains/compliance/policy-as-code.md` | tech-writer | ATELIER-78 | OPA/Cedar/Sentinel/Kyverno patterns, policy as CI/CD + admission gate, policy testing, versioning policy; **engine comparison table: OPA vs Cedar vs Kyverno vs Sentinel** covering policy language, evaluation gate, ecosystem (IDEATE-23, D-041); cross-link infrastructure-as-code (declarative intent) + kubernetes/rbac.md (admission) |
| 03-02b-04 | `domains/compliance/evidence.md` | tech-writer | ATELIER-79 | Evidence collection as a byproduct, audit-ready export, provenance; **fenced signed-attestation example (Cosign OR in-toto)** — not prose-only (IDEATE-29, D-047); cross-link security/supply-chain.md + observability/{metrics,tracing}.md |
> **Parallelism note:** Wave 2a + 2b together = 8 independent tasks. The executor schedules at most 5 concurrently per `parallelization.max_concurrent_agents`; the remaining 3 run as soon as slots free. The 2a/2b labels are organizational (by domain), not a hard sequencing barrier — both sub-waves are in the same dependency tier (all depend only on Wave 1).
**Verify (P3):**
- Structural: 10 files exist (5 under `domains/i18n/`, 5 under `domains/compliance/`)
- Behavioral: every P1P10 in both first-principles traces to ≥1 C-rule (domain-expert sign-off)
- Security: i18n P6 (Text Direction) + compliance P1 (Append-Only) + P9 (Redacted) sections present
- Quality: each derived doc ≥1 outbound cross-link (IDEATE-08); testing-i18n.md pseudo-locale→pyramid mapping present (IDEATE-28); policy-as-code.md engine comparison table present (IDEATE-23); evidence.md has a fenced signed-attestation example (IDEATE-29); no regulation-specific content in compliance (D-024)
## Phase 4 — Matrix + Review + Manifest Integration
**Goal:** Extend the matrix (+40 P-rule → C-rule mappings, 10 per new domain), domain-coverage (per-domain rows + Core Principle Coverage table for 4 new domains), review docs (agent + peer-review + anti-patterns with v0.3 chaos anti-patterns), and the manifest (all v0.3 docs + `examples/` directory listing closing v0.2 ESC-002 drift). Closes the traceability loop and makes the manifest authoritative for v0.3.
**Branch:** `atelier/phase/04-matrix-review-manifest` (from `atelier/milestone/v0.3-atelier`)
**Personas:** domain-expert (matrix + anti-patterns + coverage), tech-writer (checklists + manifest), lead-developer (manifest authoritative index)
**Tag:** v0.2.4
**Requirements:** ATELIER-80, ATELIER-81, ATELIER-82, ATELIER-83, ATELIER-84, ATELIER-85, ATELIER-91
### Wave 1 (sequential — matrix is the arbiter, must be authoritative first)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 04-01-01 | `matrix/principles-matrix.md` (extend) | domain-expert | ATELIER-80 | Add 4 sections (GitOps + Operators, AI/ML, i18n, Compliance), 10 rows each, format matching v0.1/v0.2 tables; **review check: row count per new domain = 10, each row ≥1 C-rule** (IDEATE-02, IDEATE-13 carried forward); **update Coverage Summary to "post-v0.3: 17 domains, 170 P-rules"** — both the summary block AND the per-domain section count (IDEATE-18, D-036) |
### Wave 2 (parallel — 5 independent extensions; exactly at max_concurrent_agents=5)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 04-02-01 | `matrix/domain-coverage.md` (extend) | domain-expert | ATELIER-81 | Add "v0.3 Domain Coverage" table with 4 rows (schema: domain, P-count, derived-doc-count, manifest-listed, status per IDEATE-03); **AND update the "Core Principle Coverage" table (C1C8 → domains) for the 4 new domains** — C4 Locality adds i18n + gitops; C5 Reversibility adds ai-ml + compliance + gitops + i18n; etc. (IDEATE-19, D-037) |
| 04-02-02 | `review/agent-checklist.md` (extend) | tech-writer | ATELIER-82 | Add "If GitOps + Operators", "If AI/ML", "If i18n", "If Compliance" trigger sections (IDEATE-05 carried forward); ai-ml section includes a D-023 scope check (reject algorithm-design content) |
| 04-02-03 | `review/peer-review-checklist.md` (extend) | tech-writer | ATELIER-83 | Add 4 new domain peer-review sections (parity with agent-checklist, IDEATE-09 carried forward) |
| 04-02-04 | `review/anti-patterns.md` (extend) | domain-expert | ATELIER-84 | Add "v0.3 Chaos Anti-Patterns" section covering: (a) **v0.3 deployable artifact types** — .po resource files, .rego policy files, model artifacts, signed manifests as standalone files (IDEATE-20); (b) **GitOps push-pattern violation** (P3 Pull Don't Push, IDEATE-24, D-042); (c) **i18n LTR-only assumption violation** (P6 Text Direction, IDEATE-25, D-043); (d) **AI/ML orphan-model violation** — deployed prediction with no lineage trace (P3 Lineage, IDEATE-27, D-045); plus domain-specific anti-patterns per RESEARCH.md/REQUIREMENTS.md notes: gitops (push-based deploy P3, manual kubectl apply on GitOps-managed resource P8, cluster-admin GitOps robot P10), ai-ml (unreproducible training run P1, "the latest" model P5, notebook in production P9, orphan model P3), i18n (inline string concatenation P3, `if (n==1)` plural branching P4, LTR-only layout P6, hand-rolled date formatter P5), compliance (mutable audit log P1, shared/generic identity in audit P7, secret leaked in audit log P9, manual evidence assembly at audit time P6) |
| 04-02-05 | `MANIFEST.md` (extend) | lead-developer | ATELIER-85, ATELIER-91 | Add 4 new domains + all 18 derived docs to the Domains table (IDEATE-04 carried forward); update Cross-Cutting counts to "17 domains, 170 P-rules post-v0.3"; **add an `examples/` directory listing section** (good + bad files) closing the v0.2 ESC-002 drift — manifest is authoritative (IDEATE-17, D-035) |
**Verify (P4):**
- Structural: matrix has 17 domain sections (13 v0.1/v0.2 + 4 new), 170 P-rules total; domain-coverage has both the per-domain v0.3 table AND the updated C-rule coverage table; 3 review docs extended; MANIFEST lists all v0.3 docs + examples/
- Behavioral: every new P-rule has a matrix row; domain-expert verifies no orphans (IDEATE-13); Coverage Summary reads "17 domains, 170 P-rules" (IDEATE-18)
- Security: anti-patterns cover GitOps push-pattern (P3), AI/ML orphan-model (P3), i18n LTR-only (P6), compliance mutable audit log (P1) + secret-in-audit (P9)
- Quality: MANIFEST is authoritative — every v0.3 file listed, examples/ listed (ATELIER-91 closes ESC-002); unlisted = not part of framework; agent-checklist + peer-review-checklist have parity across the 4 new domains (ATELIER-82 ↔ ATELIER-83)
## Phase 5 — Examples + Cross-Links
**Goal:** Add 2 good + 2 bad examples (D-025 — highest illustration value) and verify cross-domain links from all 4 new domains to existing ones. Examples are markdown with fenced code only (no standalone .yaml/.po/.rego/model artifacts — D-020).
**Branch:** `atelier/phase/05-examples-crosslinks` (from `atelier/milestone/v0.3-atelier`)
**Personas:** tech-writer (examples + cross-link audit), domain-expert (P-rule citation + cross-link validation)
**Tag:** v0.2.5
**Requirements:** ATELIER-86, ATELIER-87, ATELIER-88
### Wave 1 (parallel — 4 examples, independent; ≤5 concurrent)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 05-01-01 | `examples/good/gitops-pr.md` | tech-writer | ATELIER-86 | Good GitOps example; markdown with fenced YAML only (no standalone .yaml); demonstrates P1 Git is Source of Truth + P3 Pull Don't Push + P5 State Immutable and Versioned; cross-link to argocd.md + flux.md + kubernetes/workloads.md |
| 05-01-02 | `examples/good/ai-ml-reproducibility.md` | tech-writer (ml-engineer consult) | ATELIER-86 | Good AI/ML example; markdown with fenced code only (no model artifacts); demonstrates P1 Reproducibility + P2 Data Versioned + P3 Lineage Traceable + P5 Models are Versioned Artifacts; cross-link to data-versioning.md + serving.md |
| 05-01-03 | `examples/bad/i18n-string-concat.md` | tech-writer | ATELIER-87 | Bad i18n example; cites P3 breached (inline string concatenation, Resources External Not Inline) per IDEATE-07; cross-link to locale-resources.md + formatting.md |
| 05-01-04 | `examples/bad/compliance-audit-log.md` | tech-writer | ATELIER-87 | Bad compliance example; **TWO breaches in one example** (IDEATE-26, D-044): append-only violation (mutation/deletion of an audit record, P1) AND redaction failure (secret leaked in audit log, P9); cites both P-rules breached; cross-link to audit-logs.md + evidence.md |
### Wave 2 (sequential — cross-link audit after all docs exist)
| Task | File | Persona | REQ-ID | Must-have |
|------|------|---------|--------|-----------|
| 05-02-01 | Cross-link audit (all 18 new derived docs across 4 domains) | tech-writer | ATELIER-88 | Review check: every new derived doc ≥1 outbound cross-link to a MANIFEST-listed existing domain doc (devops/security/observability/data/kubernetes/infrastructure-as-code); links resolve (IDEATE-08 carried forward); domain-expert validates the cross-link targets are correct (not just present) |
**Verify (P5):**
- Structural: 4 new example files exist (all .md)
- Behavioral: each bad example cites the P-rule(s) breached (i18n: P3; compliance: P1 + P9 two-breach per IDEATE-26); each good example cites the P-rules it demonstrates
- Security: no standalone .yaml/.po/.rego/model artifacts (deployable artifact mitigation, IDEATE-20, D-020)
- Quality: all cross-links from the 18 new derived docs resolve to MANIFEST-listed docs; no back-link edits to v0.1/v0.2 content (D-026 extended — one-directional outward)
## Phase 6 — Final Review + Ship (N+1)
**Goal:** Multi-persona review across all v0.3 phases, audit, milestone ship. P6 IS the v0.3 release (NFR → no separate minor tag; v0.2.6 IS the deliverable). Phase-specific personas (platform-engineer, ml-engineer) are removed after milestone completion.
**Branch:** `atelier/phase/06-final-review-ship` (from `atelier/milestone/v0.3-atelier`)
**Personas:** lead-developer (coordinate + ship), domain-expert (review), tech-writer (review), platform-engineer (review, then removed), ml-engineer (review, then removed)
**Tag:** v0.2.6 (IS the v0.3 milestone release — NFR, no separate minor tag)
**Requirements:** ATELIER-89, ATELIER-90
### Wave 1 (sequential — review → audit → ship → complete)
| Task | Activity | Persona | REQ-ID | Must-have |
|------|----------|---------|--------|-----------|
| 06-01-01 | `ciagent-review` — multi-persona review of all v0.3 changes | lead-developer | ATELIER-89 | Auto-apply P0 fixes; flag P1+ for post-hoc; if P1+ found, fix in this phase; platform-engineer reviews gitops-operators content; ml-engineer reviews ai-ml content (D-023 scope check); domain-expert verifies all 40 new P-rules trace to ≥1 C-rule (no orphans) |
| 06-01-02 | `ciagent-audit` — reconstruction + discipline | lead-developer | ATELIER-89 | git log matches .ciagent/ files; branch hygiene; commit discipline (every commit has `---ci---` block with `project: atelier`); MANIFEST reconstruction test (every listed doc exists, every existing doc is listed — incl. examples/ per ATELIER-91); matrix row-count test (10 per domain × 17 = 170) |
| 06-01-03 | `ciagent-ship` — milestone ship | lead-developer | ATELIER-90 | Merge `atelier/phase/06``atelier/milestone/v0.3-atelier``main`; tag `v0.2.6`; Gitea release with full milestone summary; delete all v0.3 branches (tags preserve history) |
| 06-01-04 | Complete milestone (REQUIREMENTS + ROADMAP + PERSONAS) | lead-developer | ATELIER-90 | Mark all v0.3 requirements (ATELIER-60..91) `covered`; ROADMAP v0.3 → complete; PERSONAS: remove platform-engineer + ml-engineer (roster returns to 3 active); clear checkpoint |
**Verify (P6):**
- Structural: all 32 v0.3 requirements (ATELIER-60..91) marked covered
- Behavioral: reconstruction test passes (git log ↔ .ciagent/); matrix row-count test passes (170); MANIFEST reconstruction test passes (incl. examples/)
- Security: audit clean (no critical issues); no regulation-specific compliance content (D-024); no algorithm-design ai-ml content (D-023); no standalone runtime artifacts (D-020)
- Quality: milestone merged to main, tag v0.2.6 exists, all v0.3 branches deleted; platform-engineer + ml-engineer personas removed (roster = 3)
## v0.3 Wave Ordering Summary
| Phase | Waves | Parallelism |
|-------|-------|-------------|
| P0 | (pre-exec) | Sequential stages (specify→clarify→research→ideate→plan→grill) |
| P1 | 2 | Wave 1 sequential (first-principles), Wave 2 parallel (4 derived docs) |
| P2 | 2 | Wave 1 sequential (first-principles), Wave 2 parallel (4 derived docs) |
| P3 | 2 | Wave 1 parallel (2 first-principles), Wave 2 parallel (8 derived docs — 2a i18n + 2b compliance, capped at 5 concurrent) |
| P4 | 2 | Wave 1 sequential (matrix arbiter), Wave 2 parallel (5 extensions — exactly max_concurrent) |
| P5 | 2 | Wave 1 parallel (4 examples), Wave 2 sequential (cross-link audit) |
| P6 | 1 | Sequential: review → audit → ship → complete |
## v0.3 Requirements → Phase Mapping
| Phase | Requirements | Count |
|-------|-------------|-------|
| 1 | ATELIER-60..64 | 5 |
| 2 | ATELIER-65..69 | 5 |
| 3 | ATELIER-70..79 | 10 |
| 4 | ATELIER-80..85, 91 | 7 |
| 5 | ATELIER-86..88 | 3 |
| 6 | ATELIER-89, 90 | 2 |
| **Total** | | **32** |
## v0.3 Ideation Refinements → Task Bake-In Map
All 14 accepted ideation refinements (IDEATE-17..30) are baked into the relevant phase tasks as explicit must-have notes:
| IDEATE-ID | Refinement | Baked Into Task(s) | How |
|-----------|-----------|-------------------|-----|
| IDEATE-17 | examples/ in MANIFEST (new req ATELIER-91) | 04-02-05 | MANIFEST gains an examples/ directory listing (closes v0.2 ESC-002 drift) |
| IDEATE-18 | matrix coverage summary = "17 domains, 170 P-rules" | 04-01-01 | Coverage Summary block + per-domain section count both updated |
| IDEATE-19 | Core Principle Coverage table update for 4 new domains | 04-02-01 | C1C8 → domains table extended (C4 adds i18n+gitops; C5 adds ai-ml+compliance+gitops+i18n; etc.) |
| IDEATE-20 | anti-patterns pre-specify domain violations + v0.3 artifact types | 04-02-04 | .po/.rego/model/signed-manifest artifact types + 16 domain-specific anti-patterns (4 per domain) |
| IDEATE-21 | ArgoCD vs Flux decision matrix | 01-02-01, 01-02-02 | Both argocd.md and flux.md carry the decision matrix (parallel to v0.2 Helm vs Kustomize) |
| IDEATE-22 | data versioning tool comparison (DVC/Delta Lake/LakeFS) | 02-02-01 | data-versioning.md comparison table (versioning model, lineage, use-case fit) |
| IDEATE-23 | policy-as-code engine comparison (OPA/Cedar/Kyverno/Sentinel) | 03-02b-03 | policy-as-code.md comparison table (policy language, evaluation gate, ecosystem) |
| IDEATE-24 | GitOps push-pattern anti-pattern (violates P3) | 04-02-04 | Named chaos anti-pattern; pre-specified to reject on sight |
| IDEATE-25 | i18n LTR-only assumption anti-pattern (violates P6) | 04-02-04 | Named chaos anti-pattern; pre-specified to reject on sight |
| IDEATE-26 | compliance-audit-log bad example = 2 breaches (P1 + P9) | 05-01-04 | examples/bad/compliance-audit-log.md covers append-only violation + redaction failure |
| IDEATE-27 | AI/ML orphan-model anti-pattern (violates P3 Lineage) | 04-02-04 | Named chaos anti-pattern; deployed prediction with no lineage trace |
| IDEATE-28 | i18n testing pseudo-locale → testing pyramid tiers | 03-02a-04 | testing-i18n.md maps unit (missing-key), integration (snapshot per locale), e2e (RTL coverage) |
| IDEATE-29 | compliance evidence.md signed-attestation fenced example | 03-02b-04 | evidence.md includes a fenced Cosign OR in-toto attestation (not prose-only) |
| IDEATE-30 | ai-ml monitoring-drift.md 3 drift types with detection signals | 02-02-04 | monitoring-drift.md enumerates data/concept/prediction drift, each with a detection signal |
## v0.3 Decisions Logged (planning stage)
| ID | Decision | Rationale | Confidence |
|----|----------|-----------|------------|
| D-049 | P3 splits Wave 2 into 2a (i18n) + 2b (compliance) labels but both are the same dependency tier | 8 derived docs are all independent post-Wave-1; the 2a/2b labels organize by domain, the executor schedules ≤5 concurrent per config.json. Avoids inventing a false dependency between i18n and compliance | 0.88 |
| D-050 | P4 Wave 2 runs exactly 5 concurrent tasks (at the max_concurrent_agents cap) | matrix, coverage, agent-checklist, peer-review-checklist, anti-patterns, manifest = 6 extensions, but anti-patterns (04-02-04) and manifest (04-02-05) are combined under lead-developer for manifest to sequence after anti-patterns content is settled. Net 5 concurrent slots | 0.82 |
| D-051 | P5 ai-ml-reproducibility.md example authored by tech-writer with ml-engineer consultation (not ml-engineer primary) | ml-engineer is removed after P2 per PERSONAS.md; P5 examples are tech-writer territory. ml-engineer constraints are baked into the task must-have (P1/P2/P3/P5 demonstrated) so the constraint survives the persona | 0.80 |
| D-052 | P6 review uses platform-engineer + ml-engineer for content review before removal | Phase-specific personas review their authored content one final time in P6 Wave 1, then are removed in 06-01-04. Ensures D-023 (ai-ml scope) and GitOps correctness are checked by the specialist before the roster returns to 3 | 0.84 |
| D-053 | Vertical-slice integrity: each phase is independently shippable | P1 ships gitops-operators domain docs (matrix rows land in P4 — acceptable because the domain is self-consistent; matrix extension is the traceability closure, not a blocker for the domain's internal consistency). P3 ships 2 domains together (D-022). P4 closes traceability + manifest. P5 closes examples + cross-links. P6 ships the release | 0.86 |
## Assumptions Logged
| # | Assumption | Confidence |
|---|-----------|------------|
| A-001 | The v0.2 ESC-002 drift note (examples/ unlisted in MANIFEST) is the only pre-existing manifest drift; no other v0.1/v0.2 docs are unlisted | 0.85 |
| A-002 | The 4 new domains' P-rules map to existing core C1C8 without needing new core principles (core is stable at 8) | 0.95 |
| A-003 | Wave 2 of P3 (8 derived docs) can be scheduled by the executor as 5-then-3 without a hard sub-wave barrier | 0.90 |
| A-004 | The ArgoCD vs Flux decision matrix (IDEATE-21) is the only decision matrix required in P1 (no separate operators-vs-Helm matrix beyond operators.md's "when to write an operator vs a Helm chart" guidance) | 0.82 |
| A-005 | P4 anti-patterns (04-02-04) and manifest (04-02-05) can be concurrent because anti-patterns content does not block the manifest's examples/ listing (manifest lists file paths, not anti-pattern content) | 0.80 |
+227
View File
@@ -0,0 +1,227 @@
# Atelier — Project Vision
## Objective
Build **Atelier** — a first-principles, docs-as-code engineering framework for AI agents and humans, organized as a hierarchical document tree: `core → domains → languages → review → matrix → examples`. Each domain defines its own first principles derived from a small set of core principles, with every rule traced back via a principles matrix. The framework is consumed by AI agents as pre-completion guidance and by humans as engineering canon.
## Core Value
**Principles over rules.** A small set of universal core principles (Correctness, Clarity, Simplicity, Locality, Reversibility, Composability, Observability, Economy) govern every domain. Domain rules derive from these and never contradict them. The principles matrix is the connective tissue — no domain rule is orphaned, and conflicts resolve upward to core.
## Requirements
- Hierarchical document tree: `core/`, `domains/`, `languages/`, `review/`, `matrix/`, `examples/`
- `core/first-principles.md` — 8 universal principles (C1C8)
- `core/conflict-resolution.md` — formal cross-document conflict rules
- `core/reading-order.md` — recommended consumption order
- 11 domain folders, each with `first-principles.md` (P1P10) plus derived docs
- Domains: uiux, api, security, data, testing, performance, observability, errors, documentation, concurrency, devops
- UI/UX derived: components, accessibility, tokens, copywriting
- API derived: rest, graphql, versioning, error-responses, pagination
- Security derived: authentication, authorization, input-validation, secrets, supply-chain
- Data derived: schema-design, migrations, indexing
- Testing derived: pyramid, fixtures
- Performance derived: frontend, backend
- Observability derived: logging, metrics, tracing
- Errors derived: patterns
- Documentation derived: doc-templates
- Concurrency derived: patterns
- DevOps derived: ci-cd, environments
- `matrix/principles-matrix.md` — every domain P-rule → core C-rule mapping
- `matrix/domain-coverage.md` — where each principle applies
- `review/agent-checklist.md` — pre-completion checklist for agents
- `review/peer-review-checklist.md` — human review checklist
- `review/anti-patterns.md` — catalog of violations
- `examples/good/*` and `examples/bad/*` — worked examples
- `languages/*.md` — language-specific application (typescript, python, go, rust, ...)
- `MANIFEST.md` — authoritative index (unlisted = not part of framework)
- `README.md`, `LICENSE`, `CHANGELOG.md`, `CONTRIBUTING.md`
## Constraints
- Docs-as-code: versioned, reviewed, tested, owned (Documentation P1)
- Every domain principle MUST trace to a core principle via the matrix
- The manifest is authoritative — unlisted documents are not part of the framework
- Security principles are non-tradeable (8 of 10 are never sacrificed)
- Manifest reading order: core first-principles → core conflict-resolution → domain first-principles → domain topic → matrix reference
- No runtime code — Atelier is a documentation-only framework
## Out of Scope
- Tooling to enforce the rules (linters, analyzers) — future work
- Translation/localization of the framework docs
- Automated agent integration adapters beyond markdown consumption
- Versioned per-domain release artifacts (the whole framework versions together)
- Runtime code (terraform manifests, k8s YAML, deployable artifacts) — Atelier remains docs-only. Examples under `examples/iac/` and `examples/k8s/` are illustrative markdown (rendered manifests in code fences), never deployed.
## v0.2 — Infrastructure as Code + Kubernetes
**Milestone type:** NFR (all phases produce docs — no `feat` runtime code)
**Tag line:** v0.1.x (previous minor from v0.2)
**Scope:** Extend the domain tree with two new top-level domains covering IaC tooling and the Kubernetes platform, plus matrix, review, and examples integration.
### New Domains
- `domains/infrastructure-as-code/` — tooling domain
- `first-principles.md` — 10 IaC principles (P1P10)
- Derived: `terraform.md`, `opentofu.md`, `state.md`, `modules.md`
- `domains/kubernetes/` — platform domain
- `first-principles.md` — 10 k8s principles (P1P10)
- Derived: `workloads.md`, `networking.md`, `storage.md`, `rbac.md`, `helm.md`, `kustomize.md`
### Cross-Domain Integration
- Extend `matrix/principles-matrix.md` with 20 new P-rules → core C-rule mappings (10 per new domain)
- Extend `matrix/domain-coverage.md` with the two new domains
- Extend `review/agent-checklist.md` and `review/anti-patterns.md` with IaC/k8s checks/violations
- Update `MANIFEST.md` to list all new documents (manifest is authoritative)
- New examples: `examples/good/terraform-module.md`, `examples/good/k8s-deployment.md`, `examples/bad/` counterparts
- Cross-links from new domains to existing `devops/`, `security/`, `observability/`, `data/` domains
## v0.3 — GitOps + Operators + AI/ML + i18n + Compliance
**Milestone type:** NFR (all phases produce docs — no `feat` runtime code)
**Tag line:** v0.2.x (previous minor from v0.3)
**Scope:** Extend the domain tree with four new top-level domains covering GitOps/operator patterns, AI/ML, internationalization, and compliance. Plus matrix, review, examples, and cross-link integration. All content is docs-only markdown with illustrative code fences; no runtime/deployable artifacts.
### New Domains
- `domains/gitops-operators/` — platform-automation domain (ArgoCD + Flux + Operators)
- `first-principles.md` — 10 GitOps/operator principles (P1P10)
- Derived: `argocd.md`, `flux.md`, `operators.md`, `progressive-delivery.md`
- `domains/ai-ml/` — ML engineering domain
- `first-principles.md` — 10 AI/ML principles (P1P10)
- Derived: `data-versioning.md`, `model-evaluation.md`, `serving.md`, `monitoring-drift.md`
- `domains/i18n/` — internationalization domain
- `first-principles.md` — 10 i18n principles (P1P10)
- Derived: `locale-resources.md`, `formatting.md`, `rtl-bidi.md`, `testing-i18n.md`
- `domains/compliance/` — compliance/audit domain
- `first-principles.md` — 10 compliance principles (P1P10)
- Derived: `audit-logs.md`, `data-retention.md`, `policy-as-code.md`, `evidence.md`
### Cross-Domain Integration
- Extend `matrix/principles-matrix.md` with 40 new P-rules → core C-rule mappings (10 per new domain)
- Extend `matrix/domain-coverage.md` with the four new domains
- Extend `review/agent-checklist.md`, `review/peer-review-checklist.md`, and `review/anti-patterns.md` with new domain sections
- Update `MANIFEST.md` to list all new v0.3 documents (manifest is authoritative)
- New examples (good + bad): gitops-pr, ai-ml-reproducibility, i18n-string-concat, compliance-audit-log
- Cross-links from new domains to existing `devops/`, `security/`, `observability/`, `data/`, `kubernetes/`, `infrastructure-as-code/` domains
### Phase Plan (proposed, finalized in PLAN)
- P0 Pre-Execution: spec, clarify, research, ideate, plan, grill
- P1 GitOps + Operators domain
- P2 AI/ML domain
- P3 i18n + Compliance domains
- P4 Matrix + Review Integration (40 new mappings, manifest, checklist parity)
- P5 Examples + Cross-Links
- P6 Final Review + Ship (IS the v0.3 release → tag v0.2.6)
NFR milestone: no separate minor tag. The final patch (v0.2.6) IS the v0.3 deliverable.
## Key Decisions
| ID | Decision | Rationale | Confidence |
|----|----------|-----------|------------|
| D-001 | MIT license (deferred to P1) | Not pasted in spec; standard permissive choice for docs frameworks | 0.70 |
| D-002 | Defer README/CHANGELOG/CONTRIBUTING to execute phase | Referenced in spec tree but content not pasted | 0.80 |
| D-003 | core/* first-principles deferred to P1 | Referenced as "already drafted" but not pasted | 0.70 |
| D-004 | domains/uiux/first-principles.md deferred to P2 | Referenced as "already drafted" but not pasted | 0.70 |
| D-005 | examples/, languages/, review/, matrix/domain-coverage.md deferred to later phases | Execute-phase deliverables per spec build order | 0.90 |
| D-006 | GITEA_API_TOKEN as token env var name | Matches existing /root/.ciagent/config.json convention | 0.80 |
| D-007 | Milestone v0.1 (user override) | User explicitly required first milestone = v0.1, not v1.0 | 0.95 |
| D-008 | 6 execution phases (P1P6) + final phase P7 | Mirrors spec Part 6 "Recommended Build Order" | 0.88 |
| D-009 | NFR milestone type (all phases are docs/chore) | Every phase produces markdown docs, no feat code; NFR patch-line versioning | 0.92 |
| D-010 | Tags run on v0.0.x patch line (prev minor from v0.1) | Per branch-strategy.md: milestone 0.1 → tags v0.0.0..v0.0.7 | 0.90 |
| D-011 | v0.2 adds two new top-level domains: infrastructure-as-code + kubernetes | User directive to incorporate terraform/opentofu + kubernetes; two-domain split reflects tool-vs-platform distinction | 0.90 |
| D-012 | v0.2 remains docs-only (NFR milestone type) | PROJECT.md constraint "no runtime code" preserved; manifests appear only as illustrative code-fence content in examples | 0.95 |
| D-013 | v0.2 tags run on v0.1.x patch line (prev minor from v0.2) | Per branch-strategy.md: milestone 0.2 → tags v0.1.0..v0.1.5; v0.1.5 IS the v0.2 release (NFR → no separate minor tag) | 0.90 |
| D-014 | Add phase-specific `platform-engineer` persona for P1P4 | IaC/k8s domain authoring benefits from a specialist persona with declarative-first/stateless-examples constraints; removed after milestone | 0.82 |
| D-015 | 4 execution phases (P1P4) + final phase P5 | P1 IaC domain, P2 k8s domain, P3 matrix+review, P4 examples+cross-links, P5 final review+ship | 0.85 |
| D-016 | v0.3 covers 4 deferred domains: gitops-operators, ai-ml, i18n, compliance | Carries forward v0.2 deferred ideation (IDEATE-15, IDEATE-16); single milestone groups them to limit release overhead | 0.86 |
| D-017 | v0.3 tags run on v0.2.x patch line (prev minor from v0.3) | Per branch-strategy.md: milestone 0.3 → tags v0.2.0..v0.2.6; v0.2.6 IS the v0.3 release (NFR → no separate minor tag) | 0.90 |
| D-018 | v0.3 splits P1 GitOps/Operators, P2 AI/ML, P3 i18n+Compliance, P4 Matrix+Review, P5 Examples, P6 Final | Each domain cluster is a coherent vertical slice; i18n + compliance paired (smaller surface) to balance phase load | 0.84 |
| D-019 | Reuse `platform-engineer` persona (extended) + add `ml-engineer` phase-specific persona for P2 | GitOps/Operators/k8s reuse platform-engineer; AI/ML benefits from a data/ML-specialist persona with reproducibility/data-lineage constraints; removed after milestone | 0.80 |
| D-020 | v0.3 remains docs-only (NFR milestone type) | PROJECT.md constraint "no runtime code" preserved; manifests/models/locale resources appear only as illustrative code-fence content in examples | 0.95 |
| D-021 | GitOps-operators domain groups ArgoCD + Flux + Operators + Progressive Delivery under one first-principles doc | All four share the declarative-source-of-truth reconciliation loop; splitting would fragment the P-rules and duplicate the core principles they trace to | 0.84 |
| D-022 | i18n + compliance paired in P3 (not separate phases) | Both are smaller-surface domains (4 derived docs each); pairing balances phase load against the heavier P1/P2 single-domain phases | 0.83 |
| D-023 | AI/ML domain scope = engineering discipline (data versioning, evaluation, serving, drift), NOT algorithm/model design | Atelier is a framework for engineering practice; algorithm choice is domain-knowledge out of scope. Mirrors how iac/k8s docs cover practice not implementation | 0.88 |
| D-024 | Compliance domain is framework-agnostic (audit logs, retention, policy-as-code, evidence), NOT tied to a specific regulation (GDPR/HIPAA/SOC2) | Regulation-specific docs would bloat the framework and go stale; principles derive from core Security/Correctness and apply across regulations | 0.86 |
| D-025 | Examples set = 2 good + 2 bad (not 4+4) | v0.3 adds 4 domains; 4+4 examples would unbalance P5. 2 good (gitops-pr, ai-ml-reproducibility) + 2 bad (i18n-string-concat, compliance-audit-log) cover the highest-illustration-value cases; remaining domains covered by cross-links and anti-patterns | 0.80 |
| D-026 | 40 new matrix mappings (10 per domain × 4 domains) | Consistent with v0.1 (110 mappings / 11 domains = 10) and v0.2 (20 mappings / 2 domains = 10). Each P-rule maps to ≥1 C-rule | 0.92 |
| D-035 | Add `examples/` directory listing to MANIFEST.md in v0.3 P4 (ATELIER-91) | v0.2 audit escalation ESC-002 note flagged examples/ unlisted; manifest is authoritative, so this is pre-existing drift that v0.3 closes | 0.85 |
| D-036 | Matrix coverage summary must state post-v0.3 totals (17 domains, 170 P-rules) | Both the summary block and per-domain section count must update; consistent with v0.2's "post-v0.2" summary | 0.93 |
| D-037 | domain-coverage.md Core Principle Coverage table (C1C8 → domains) must update for 4 new domains | ATELIER-81 covers the per-domain row schema; this is the complementary C-rule → domains table that also needs the 4 new domains | 0.90 |
| D-038 | v0.3 anti-patterns must pre-specify domain-specific violations + v0.3 artifact types | Avoids generic "deployable example artifact" only; v0.3 has new artifact types (.po, .rego, model files) and 4 domains × ~4 anti-patterns each | 0.86 |
| D-039 | ArgoCD vs Flux decision matrix required in argocd.md + flux.md | Parallel to v0.2 Helm vs Kustomize decision matrix (IDEATE-10); both tools share the GitOps model but differ in architecture (App CRD vs composable controllers) | 0.82 |
| D-040 | Data versioning tool comparison table required in data-versioning.md (DVC/Delta Lake/LakeFS) | Parallel to v0.2 state comparison table (IDEATE-11); three主流 tools with distinct versioning/lineage models | 0.80 |
| D-041 | Policy-as-code engine comparison table required in policy-as-code.md (OPA/Cedar/Kyverno/Sentinel) | Parallel to v0.2 PSS coverage (IDEATE-12); four engines with distinct policy languages and gate models | 0.81 |
| D-042 | GitOps push-pattern is a named anti-pattern (violates P3 Pull Don't Push) | Chaos scenario: a "GitOps" example that uses push-based deploy is a fundamental violation; pre-specify to reject on sight | 0.85 |
| D-043 | i18n LTR-only assumption is a named anti-pattern (violates P6 Text Direction) | Chaos scenario: formatting/layout examples that assume LTR only fail RTL/bidi users; pre-specify to reject | 0.83 |
| D-044 | compliance-audit-log bad example must cover both append-only violation (P1) and redaction failure (P9) | Two-breach example maximizes illustration value; mirrors v0.2 named-bad-example pattern but doubles the breach surface for the highest-stakes domain | 0.87 |
| D-045 | AI/ML orphan-model anti-pattern required (deployed prediction with no lineage trace, violates P3) | Chaos scenario: a serving example with no model→training→data lineage is the AI/ML analog of v0.2 orphaned P-rule; pre-specify | 0.84 |
| D-046 | i18n testing-i18n.md must map pseudo-locale testing to testing pyramid tiers | Avoids generic "test i18n" guidance; maps to unit (missing-key), integration (snapshot per locale), e2e (RTL coverage) | 0.78 |
| D-047 | compliance evidence.md must include a fenced signed-attestation example (Cosign or in-toto) | Prose-only evidence guidance is weak; a fenced example demonstrates the principle concretely (P6 Evidence Collected Continuously) | 0.80 |
| D-048 | ai-ml monitoring-drift.md must enumerate 3 drift types (data/concept/prediction) with a detection signal per type | Avoids conflating drift types; each has distinct detection signals and retraining triggers | 0.82 |
| D-049 | P3 splits Wave 2 into 2a (i18n) + 2b (compliance) labels but both are the same dependency tier | 8 derived docs are all independent post-Wave-1; 2a/2b labels organize by domain, executor schedules ≤5 concurrent per config.json. Avoids inventing a false dependency between i18n and compliance | 0.88 |
| D-050 | P4 Wave 2 runs exactly 5 concurrent tasks (at the max_concurrent_agents cap) | matrix, coverage, agent-checklist, peer-review-checklist, anti-patterns, manifest = 6 extensions, but anti-patterns (04-02-04) and manifest (04-02-05) are combined under lead-developer for manifest to sequence after anti-patterns content is settled. Net 5 concurrent slots | 0.82 |
| D-051 | P5 ai-ml-reproducibility.md example authored by tech-writer with ml-engineer consultation (not ml-engineer primary) | ml-engineer is removed after P2 per PERSONAS.md; P5 examples are tech-writer territory. ml-engineer constraints are baked into the task must-have (P1/P2/P3/P5 demonstrated) so the constraint survives the persona | 0.80 |
| D-052 | P6 review uses platform-engineer + ml-engineer for content review before removal | Phase-specific personas review their authored content one final time in P6 Wave 1, then are removed in 06-01-04. Ensures D-023 (ai-ml scope) and GitOps correctness are checked by the specialist before the roster returns to 3 | 0.84 |
| D-053 | Vertical-slice integrity: each phase is independently shippable | P1 ships gitops-operators domain docs (matrix rows land in P4 — acceptable because the domain is self-consistent; matrix extension is the traceability closure, not a blocker for the domain's internal consistency). P3 ships 2 domains together (D-022). P4 closes traceability + manifest. P5 closes examples + cross-links. P6 ships the release | 0.86 |
## Cross-Project References
None yet. Atelier is a standalone docs framework.
## Milestone History
- **v0.1** — Initial Framework (COMPLETE). 8 core principles, 11 domains, 110 domain principles, full matrix, 4+3 examples, 4 languages. Tag v0.0.7.
- **v0.2** — Infrastructure as Code + Kubernetes (COMPLETE). Adds 2 domains (20 new P-rules), matrix/review/examples integration. Tags v0.1.0v0.1.5; v0.1.5 is the v0.2 release.
- **v0.3** — GitOps + Operators + AI/ML + i18n + Compliance (COMPLETE). Adds 4 domains (40 new P-rules), matrix 130→170 across 13→17 domains, review/checklist/anti-patterns integration, 4 new examples, MANIFEST examples/ listing (closes ESC-002). Tags v0.2.0v0.2.6; v0.2.6 is the v0.3 release.
- **v0.4** — Edge + Messaging + Language-Derived Docs (ACTIVE). Adds 2 new top-level domains (edge, messaging — 20 new P-rules), and extends `languages/` with derived docs per language (typescript, python, go, rust). Matrix 170→190 across 17→19 domains; review/checklist/anti-patterns extended; 4 new examples; MANIFEST gains a `languages/` section. Tags v0.3.0v0.3.N; final patch IS the v0.4 deliverable.
## v0.4 — Edge + Messaging + Language-Derived Docs
**Milestone type:** NFR (all phases produce docs — no `feat` runtime code)
**Tag line:** v0.3.x (previous minor from v0.4)
**Scope:** Extend the domain tree with two new top-level domains covering edge computing and messaging systems, plus extend `languages/` from first-principles-only stubs to full derived-doc trees. All content is docs-only markdown with illustrative code fences; no runtime/deployable artifacts. Closes the "Future Milestones" candidates in the v0.3 ROADMAP (edge domain + language-specific derived docs).
### New Domains
- `domains/edge/` — edge computing domain (latency-bound, resource-constrained, geo-distributed)
- `first-principles.md` — 10 edge principles (P1P10)
- Derived: `cdn.md`, `offline-first.md`, `iot.md`, `sync.md`
- `domains/messaging/` — messaging systems domain (async, queues, streams, pub/sub)
- `first-principles.md` — 10 messaging principles (P1P10)
- Derived: `queues.md`, `pubsub.md`, `streams.md`, `delivery-semantics.md`
### Language-Derived Extension
Extend each `languages/*.md` from a single first-principles stub to a full derived-doc tree mirroring the domain structure (parallel to how `domains/` have first-principles + derived). Each language gets derived docs for the most-relevant domains (4 derived docs per language = 16 total), traced to that language's P-rules and to core C-rules.
- `languages/typescript.md` (existing first-principles) + derived: `ts-types.md`, `ts-tooling.md`, `ts-async.md`, `ts-testing.md`
- `languages/python.md` + derived: `py-types.md`, `py-tooling.md`, `py-async.md`, `py-testing.md`
- `languages/go.md` + derived: `go-types.md`, `go-tooling.md`, `go-concurrency.md`, `go-testing.md`
- `languages/rust.md` + derived: `rs-ownership.md`, `rs-tooling.md`, `rs-async.md`, `rs-testing.md`
### Cross-Domain Integration
- Extend `matrix/principles-matrix.md` with 20 new P-rules → core C-rule mappings (10 per new domain; the language-derived docs trace to existing domain P-rules, not new matrix rows)
- Extend `matrix/domain-coverage.md` with edge + messaging (row schema unchanged); add a `languages/` coverage sub-table listing each language's derived docs and the domains they trace to
- Extend `review/agent-checklist.md`, `review/peer-review-checklist.md`, and `review/anti-patterns.md` with edge + messaging sections + a language-derived-docs section
- Update `MANIFEST.md` with (a) the 2 new domains, (b) a new `languages/` section listing each language's first-principles + derived docs, (c) post-v0.4 totals (19 domains, 190 P-rules)
- New examples (good + bad): `examples/good/edge-offline-sync.md`, `examples/good/messaging-idempotent-consumer.md`, `examples/bad/edge-sync-loop.md`, `examples/bad/messaging-shared-subscription.md`
- Cross-links from new domains to existing `devops/`, `observability/`, `data/`, `concurrency/`, `kubernetes/`, `infrastructure-as-code/` domains; cross-links from language-derived docs to their traced domain docs
### Phase Plan (proposed, finalized in PLAN)
- P0 Pre-Execution: spec, clarify, research, ideate, plan, grill
- P1 Edge domain
- P2 Messaging domain
- P3 Language-Derived Extension (16 derived docs across 4 languages)
- P4 Matrix + Review Integration (20 new mappings, manifest languages/ section, checklist parity)
- P5 Examples + Cross-Links
- P6 Final Review + Ship (IS the v0.4 release → tag v0.3.6)
NFR milestone: no separate minor tag. The final patch (v0.3.6) IS the v0.4 deliverable.
+395
View File
@@ -0,0 +1,395 @@
# Atelier — Requirements
## v0.1 Requirements
| REQ-ID | Requirement | Priority | Phase | Status |
|--------|-------------|----------|-------|--------|
| ATELIER-01 | `.ciagent/atelier/` governance files created | P0 | 0 | covered |
| ATELIER-02 | Milestone v0.1 branch hierarchy established | P0 | 0 | covered |
| ATELIER-03 | Initial framework content committed (MANIFEST, matrix, 11 domain first-principles, uiux components+a11y) | P0 | 0 | covered |
| ATELIER-04 | `core/first-principles.md` — 8 core principles (C1C8) | P0 | 1 | covered |
| ATELIER-05 | `core/conflict-resolution.md` — cross-document conflict rules | P0 | 1 | covered |
| ATELIER-06 | `core/reading-order.md` — recommended consumption order | P0 | 1 | covered |
| ATELIER-07 | `README.md` — repo entry point, quickstart | P0 | 1 | covered |
| ATELIER-08 | `LICENSE` — MIT license | P0 | 1 | covered |
| ATELIER-09 | `domains/uiux/first-principles.md` — 10 UI/UX principles | P0 | 2 | covered |
| ATELIER-10 | `domains/errors/first-principles.md` — 10 error principles | P1 | 2 | covered |
| ATELIER-11 | `domains/documentation/first-principles.md` | P1 | 2 | covered |
| ATELIER-12 | `domains/concurrency/first-principles.md` | P1 | 2 | covered |
| ATELIER-13 | `domains/devops/first-principles.md` | P1 | 2 | covered |
| ATELIER-14 | `domains/api/` derived: rest, graphql, versioning, error-responses, pagination | P1 | 3 | covered |
| ATELIER-15 | `domains/security/` derived: authentication, authorization, input-validation, secrets, supply-chain | P1 | 3 | covered |
| ATELIER-16 | `domains/data/` derived: schema-design, migrations, indexing | P1 | 3 | covered |
| ATELIER-17 | `domains/testing/` derived: pyramid, fixtures | P1 | 3 | covered |
| ATELIER-18 | `domains/performance/` derived: frontend, backend | P1 | 3 | covered |
| ATELIER-19 | `domains/observability/` derived: logging, metrics, tracing | P1 | 3 | covered |
| ATELIER-20 | `domains/uiux/` derived: tokens, copywriting | P2 | 3 | covered |
| ATELIER-21 | `matrix/principles-matrix.md` — full domain → core mapping | P0 | 4 | covered |
| ATELIER-22 | `matrix/domain-coverage.md` | P1 | 4 | covered |
| ATELIER-23 | `review/agent-checklist.md` | P0 | 4 | covered |
| ATELIER-24 | `review/peer-review-checklist.md` | P1 | 4 | covered |
| ATELIER-25 | `review/anti-patterns.md` | P1 | 4 | covered |
| ATELIER-26 | `examples/good/api-endpoint.md` | P2 | 5 | covered |
| ATELIER-27 | `examples/good/react-component.md` | P2 | 5 | covered |
| ATELIER-28 | `examples/good/db-schema.md` | P2 | 5 | covered |
| ATELIER-29 | `examples/good/error-handler.md` | P2 | 5 | covered |
| ATELIER-30 | `examples/bad/god-object.md`, `silent-error.md`, `leaky-abstraction.md` | P2 | 5 | covered |
| ATELIER-31 | `languages/typescript.md`, `python.md`, `go.md`, `rust.md` | P2 | 6 | covered |
| ATELIER-32 | `CHANGELOG.md` | P1 | 6 | covered |
| ATELIER-33 | `CONTRIBUTING.md` | P1 | 6 | covered |
| ATELIER-34 | Final review passes (all phases reviewed, audit clean) | P0 | 7 | covered |
| ATELIER-35 | Milestone v0.1 released (tag v0.0.7, merged to main) | P0 | 7 | covered |
## Traceability Matrix
| Phase | Requirements |
|-------|-------------|
| 0 (Pre-Execution) | ATELIER-01, ATELIER-02, ATELIER-03 |
| 1 (Core Foundation) | ATELIER-04, ATELIER-05, ATELIER-06, ATELIER-07, ATELIER-08 |
| 2 (Domain First Principles) | ATELIER-09, ATELIER-10, ATELIER-11, ATELIER-12, ATELIER-13 |
| 3 (Domain Derived Docs) | ATELIER-14..ATELIER-20 |
| 4 (Matrix + Review) | ATELIER-21, ATELIER-22, ATELIER-23, ATELIER-24, ATELIER-25 |
| 5 (Examples) | ATELIER-26, ATELIER-27, ATELIER-28, ATELIER-29, ATELIER-30 |
| 6 (Languages + Meta) | ATELIER-31, ATELIER-32, ATELIER-33 |
| 7 (Final Review + Ship) | ATELIER-34, ATELIER-35 |
## Milestone Summary
All 35 requirements covered. 8 core principles, 11 domains, 110 domain principles, 27 derived docs, 4 good + 3 bad examples, 4 language docs, full matrix, 3 review docs. NFR milestone, 7 patches (v0.0.0v0.0.7), v0.0.7 is the v0.1.0 release.
## v0.2 Requirements — Infrastructure as Code + Kubernetes
**Milestone type:** NFR (all phases produce docs)
**Tag line:** v0.1.x (previous minor from v0.2)
| REQ-ID | Requirement | Priority | Phase | Status |
|--------|-------------|----------|-------|--------|
| ATELIER-36 | `domains/infrastructure-as-code/first-principles.md` — 10 IaC principles (P1P10) | P0 | 1 | covered |
| ATELIER-37 | `domains/infrastructure-as-code/terraform.md` — Terraform derived doc | P1 | 1 | covered |
| ATELIER-38 | `domains/infrastructure-as-code/opentofu.md` — OpenTofu derived doc | P1 | 1 | covered |
| ATELIER-39 | `domains/infrastructure-as-code/state.md` — IaC state management derived doc | P1 | 1 | covered |
| ATELIER-40 | `domains/infrastructure-as-code/modules.md` — IaC module composition derived doc | P1 | 1 | covered |
| ATELIER-41 | `domains/kubernetes/first-principles.md` — 10 k8s principles (P1P10) | P0 | 2 | covered |
| ATELIER-42 | `domains/kubernetes/workloads.md` — workloads derived doc | P1 | 2 | covered |
| ATELIER-43 | `domains/kubernetes/networking.md` — networking derived doc | P1 | 2 | covered |
| ATELIER-44 | `domains/kubernetes/storage.md` — storage derived doc | P1 | 2 | covered |
| ATELIER-45 | `domains/kubernetes/rbac.md` — RBAC derived doc incl. Pod Security Standards/Admission (cross-link security/authorization) | P1 | 2 | covered |
| ATELIER-46 | `domains/kubernetes/helm.md` — Helm derived doc (with Helm vs Kustomize decision matrix) | P1 | 2 | covered |
| ATELIER-47 | `domains/kubernetes/kustomize.md` — Kustomize derived doc (with Helm vs Kustomize decision matrix) | P1 | 2 | covered |
| ATELIER-48 | Extend `matrix/principles-matrix.md` with 20 new P-rules → core C-rule mappings (10 per new domain; review check: row count per domain = 10, each row ≥1 C-rule) | P0 | 3 | covered |
| ATELIER-49 | Extend `matrix/domain-coverage.md` with infrastructure-as-code + kubernetes (row schema: domain, P-count, derived-doc-count, manifest-listed, status) | P1 | 3 | covered |
| ATELIER-50 | Extend `review/agent-checklist.md` with IaC + k8s trigger sections | P1 | 3 | covered |
| ATELIER-51 | Extend `review/anti-patterns.md` with IaC + k8s violations incl. orphaned P-rule + deployable example artifact | P1 | 3 | covered |
| ATELIER-52 | Update `MANIFEST.md` to list all new v0.2 documents (manifest authoritative) | P0 | 3 | covered |
| ATELIER-53 | `examples/good/terraform-module.md` — good IaC example (markdown with fenced HCL only; no standalone .tf) | P2 | 4 | covered |
| ATELIER-54 | `examples/good/k8s-deployment.md` — good k8s example (markdown with fenced YAML only; no standalone .yaml) | P2 | 4 | covered |
| ATELIER-55 | `examples/bad/terraform-unlocked-state.md` + `examples/bad/k8s-bare-pod-no-resources.md` — 2 named bad examples (each cites the P-rule breached) | P2 | 4 | covered |
| ATELIER-56 | Cross-links from new domains to existing devops/security/observability/data domains (review check: every new derived doc ≥1 outbound cross-link to a MANIFEST-listed doc) | P1 | 4 | covered |
| ATELIER-57 | Final review passes (all v0.2 phases reviewed, audit clean) | P0 | 5 | covered |
| ATELIER-58 | Milestone v0.2 released (tag v0.1.5, merged to main) | P0 | 5 | covered |
| ATELIER-59 | Extend `review/peer-review-checklist.md` with IaC + k8s sections (parity with agent-checklist) | P1 | 3 | covered |
## v0.2 Traceability Matrix
| Phase | Requirements |
|-------|-------------|
| 0 (Pre-Execution) | (governance: spec, clarify, research, ideate, plan) |
| 1 (Infrastructure as Code Domain) | ATELIER-36..ATELIER-40 |
| 2 (Kubernetes Domain) | ATELIER-41..ATELIER-47 |
| 3 (Matrix + Review Integration) | ATELIER-48, ATELIER-49, ATELIER-50, ATELIER-51, ATELIER-52, ATELIER-59 |
| 4 (Examples + Cross-Links) | ATELIER-53, ATELIER-54, ATELIER-55, ATELIER-56 |
| 5 (Final Review + Ship) | ATELIER-57, ATELIER-58 |
## v0.2 Ideation Log
**Generated:** 16 ideas (mechanical: 9, backend-enriched: 7)
**Accepted:** 14 (all v0.2-scope, confidence ≥ 0.79, above 0.6 autonomy threshold → auto-accepted)
**Deferred to v0.3:** 2 (IDEATE-15 GitOps/operators domain; IDEATE-16 ai-ml/i18n/compliance domains)
**Rejected:** 0
| IDEATE-ID | Source | Category | Confidence | Decision | Mapped REQ |
|-----------|--------|----------|------------|----------|------------|
| IDEATE-01 | mechanical | architecture | 0.97 | accepted → refines | ATELIER-36 (drift resolution) |
| IDEATE-02 | mechanical | coverage | 0.95 | accepted → refines | ATELIER-48 (matrix format) |
| IDEATE-03 | mechanical | spec | 0.82 | accepted → refines | ATELIER-49 (row schema) |
| IDEATE-04 | mechanical | coverage | 0.96 | accepted → refines | ATELIER-52 (manifest table) |
| IDEATE-05 | mechanical | quality | 0.94 | accepted → refines | ATELIER-50 (checklist triggers) |
| IDEATE-06 | mechanical | quality | 0.93 | accepted → refines | ATELIER-51 (anti-patterns) |
| IDEATE-07 | mechanical | spec | 0.88 | accepted → refines | ATELIER-55 (named examples) |
| IDEATE-08 | mechanical | spec | 0.86 | accepted → refines | ATELIER-56 (link verification) |
| IDEATE-09 | mechanical | coverage | 0.79 | accepted → new req | ATELIER-59 (peer-review parity) |
| IDEATE-10 | backend-enriched | improvement | 0.83 | accepted → refines | ATELIER-46 (decision matrix) |
| IDEATE-11 | backend-enriched | improvement | 0.81 | accepted → refines | ATELIER-39 (state comparison table) |
| IDEATE-12 | backend-enriched | improvement | 0.80 | accepted → refines | ATELIER-45 (PSS coverage) |
| IDEATE-13 | backend-enriched | chaos | 0.85 | accepted → refines | ATELIER-48, ATELIER-51 (orphan mitigation) |
| IDEATE-14 | backend-enriched | chaos | 0.87 | accepted → refines | ATELIER-53, ATELIER-51 (deployable artifact mitigation) |
| IDEATE-15 | backend-enriched | improvement | 0.72 | deferred v0.3 | — (GitOps/operators domain) |
| IDEATE-16 | backend-enriched | improvement | 0.68 | deferred v0.3 | — (ai-ml/i18n/compliance) |
## v0.3 Requirements — GitOps + Operators + AI/ML + i18n + Compliance
**Milestone type:** NFR (all phases produce docs)
**Tag line:** v0.2.x (previous minor from v0.3)
| REQ-ID | Requirement | Priority | Phase | Status |
|--------|-------------|----------|-------|--------|
| ATELIER-60 | `domains/gitops-operators/first-principles.md` — 10 GitOps/operator principles (P1P10) | P0 | 1 | covered |
| ATELIER-61 | `domains/gitops-operators/argocd.md` — ArgoCD derived doc | P1 | 1 | covered |
| ATELIER-62 | `domains/gitops-operators/flux.md` — Flux derived doc | P1 | 1 | covered |
| ATELIER-63 | `domains/gitops-operators/operators.md` — Kubernetes Operators derived doc | P1 | 1 | covered |
| ATELIER-64 | `domains/gitops-operators/progressive-delivery.md` — progressive delivery derived doc | P1 | 1 | covered |
| ATELIER-65 | `domains/ai-ml/first-principles.md` — 10 AI/ML principles (P1P10) | P0 | 2 | covered |
| ATELIER-66 | `domains/ai-ml/data-versioning.md` — data/model versioning derived doc | P1 | 2 | covered |
| ATELIER-67 | `domains/ai-ml/model-evaluation.md` — evaluation derived doc | P1 | 2 | covered |
| ATELIER-68 | `domains/ai-ml/serving.md` — model serving derived doc | P1 | 2 | covered |
| ATELIER-69 | `domains/ai-ml/monitoring-drift.md` — monitoring/drift derived doc | P1 | 2 | covered |
| ATELIER-70 | `domains/i18n/first-principles.md` — 10 i18n principles (P1P10) | P0 | 3 | covered |
| ATELIER-71 | `domains/i18n/locale-resources.md` — locale resource management derived doc | P1 | 3 | covered |
| ATELIER-72 | `domains/i18n/formatting.md` — formatting (dates/numbers/units) derived doc | P1 | 3 | covered |
| ATELIER-73 | `domains/i18n/rtl-bidi.md` — RTL/bidi layout derived doc | P1 | 3 | covered |
| ATELIER-74 | `domains/i18n/testing-i18n.md` — i18n testing derived doc | P1 | 3 | covered |
| ATELIER-75 | `domains/compliance/first-principles.md` — 10 compliance principles (P1P10) | P0 | 3 | covered |
| ATELIER-76 | `domains/compliance/audit-logs.md` — audit logging derived doc | P1 | 3 | covered |
| ATELIER-77 | `domains/compliance/data-retention.md` — data retention derived doc | P1 | 3 | covered |
| ATELIER-78 | `domains/compliance/policy-as-code.md` — policy-as-code derived doc | P1 | 3 | covered |
| ATELIER-79 | `domains/compliance/evidence.md` — evidence collection derived doc | P1 | 3 | covered |
| ATELIER-80 | Extend `matrix/principles-matrix.md` with 40 new P-rules → core C-rule mappings (10 per new domain; review check: row count per domain = 10, each row ≥1 C-rule) | P0 | 4 | covered |
| ATELIER-81 | Extend `matrix/domain-coverage.md` with gitops-operators, ai-ml, i18n, compliance (row schema: domain, P-count, derived-doc-count, manifest-listed, status) | P1 | 4 | covered |
| ATELIER-82 | Extend `review/agent-checklist.md` with 4 new domain trigger sections | P1 | 4 | covered |
| ATELIER-83 | Extend `review/peer-review-checklist.md` with 4 new domain sections (parity with agent-checklist) | P1 | 4 | covered |
| ATELIER-84 | Extend `review/anti-patterns.md` with 4 new domain violations incl. orphaned P-rule + deployable example artifact | P1 | 4 | covered |
| ATELIER-85 | Update `MANIFEST.md` to list all new v0.3 documents (manifest authoritative) | P0 | 4 | covered |
| ATELIER-86 | `examples/good/gitops-pr.md` + `examples/good/ai-ml-reproducibility.md` — 2 good examples (markdown with fenced code only) | P2 | 5 | covered |
| ATELIER-87 | `examples/bad/i18n-string-concat.md` + `examples/bad/compliance-audit-log.md` — 2 named bad examples (each cites the P-rule breached) | P2 | 5 | covered |
| ATELIER-88 | Cross-links from new domains to existing devops/security/observability/data/kubernetes/infrastructure-as-code domains (review check: every new derived doc ≥1 outbound cross-link to a MANIFEST-listed doc) | P1 | 5 | covered |
| ATELIER-89 | Final review passes (all v0.3 phases reviewed, audit clean) | P0 | 6 | covered |
| ATELIER-90 | Milestone v0.3 released (tag v0.2.6, merged to main) | P0 | 6 | covered |
| ATELIER-91 | Add `examples/` directory listing to `MANIFEST.md` (pre-existing drift from v0.2 audit escalation ESC-002 note: examples/ unlisted; manifest is authoritative) | P1 | 4 | covered |
## v0.3 Traceability Matrix
| Phase | Requirements |
|-------|-------------|
| 0 (Pre-Execution) | (governance: spec, clarify, research, ideate, plan) |
| 1 (GitOps + Operators Domain) | ATELIER-60..ATELIER-64 |
| 2 (AI/ML Domain) | ATELIER-65..ATELIER-69 |
| 3 (i18n + Compliance Domains) | ATELIER-70..ATELIER-79 |
| 4 (Matrix + Review Integration) | ATELIER-80..ATELIER-85, ATELIER-91 |
| 5 (Examples + Cross-Links) | ATELIER-86..ATELIER-88 |
| 6 (Final Review + Ship) | ATELIER-89, ATELIER-90 |
## v0.3 Ideation Log
**Generated:** 14 ideas (mechanical: 5, backend-enriched: 7, within-project transfer: 2 merged)
**Accepted:** 14 (all v0.3-scope, confidence ≥ 0.78, above 0.6 autonomy threshold → auto-accepted)
**Deferred to v0.4:** 0
**Rejected:** 0
| IDEATE-ID | Source | Category | Confidence | Decision | Mapped REQ |
|-----------|--------|----------|------------|----------|------------|
| IDEATE-17 | mechanical (audit escalation ESC-002 note) | drift | 0.85 | accepted → new req | ATELIER-91 (examples/ in MANIFEST) |
| IDEATE-18 | mechanical (MANIFEST + matrix coverage summary) | coverage | 0.93 | accepted → refines | ATELIER-80, ATELIER-85 (v0.3 totals: 17 domains, 170 P-rules) |
| IDEATE-19 | mechanical (domain-coverage.md Core Principle Coverage table) | coverage | 0.90 | accepted → refines | ATELIER-81 (C-rule count updates for 4 new domains) |
| IDEATE-20 | mechanical (anti-patterns specificity) | quality | 0.86 | accepted → refines | ATELIER-84 (pre-specify domain anti-patterns + v0.3 artifact types: .po, .rego, model files) |
| IDEATE-21 | backend-enriched (v0.2 IDEATE-10 pattern transfer) | improvement | 0.82 | accepted → refines | ATELIER-61, ATELIER-62 (ArgoCD vs Flux decision matrix) |
| IDEATE-22 | backend-enriched (v0.2 IDEATE-11 pattern transfer) | improvement | 0.80 | accepted → refines | ATELIER-66 (data versioning tool comparison: DVC/Delta Lake/LakeFS) |
| IDEATE-23 | backend-enriched (v0.2 IDEATE-12 pattern transfer) | improvement | 0.81 | accepted → refines | ATELIER-78 (policy-as-code engine comparison: OPA/Cedar/Kyverno/Sentinel) |
| IDEATE-24 | backend-enriched | chaos | 0.85 | accepted → refines | ATELIER-84 (GitOps push-pattern anti-pattern, violates P3 Pull Don't Push) |
| IDEATE-25 | backend-enriched | chaos | 0.83 | accepted → refines | ATELIER-84, ATELIER-72 (i18n LTR-only assumption anti-pattern) |
| IDEATE-26 | backend-enriched | chaos | 0.87 | accepted → refines | ATELIER-87 (compliance-audit-log bad example must cover append-only violation + secret redaction failure, P1 + P9) |
| IDEATE-27 | backend-enriched | chaos | 0.84 | accepted → refines | ATELIER-84 (AI/ML orphan-model anti-pattern: deployed prediction with no lineage trace) |
| IDEATE-28 | backend-enriched | improvement | 0.78 | accepted → refines | ATELIER-74 (i18n testing-i18n.md pseudo-locale tier mapping to testing/pyramid) |
| IDEATE-29 | backend-enriched | improvement | 0.80 | accepted → refines | ATELIER-79 (compliance evidence.md signed attestation fenced example, Cosign/in-toto) |
| IDEATE-30 | backend-enriched | improvement | 0.82 | accepted → refines | ATELIER-69 (ai-ml monitoring-drift.md drift-type enumeration: data/concept/prediction with detection signals) |
### Refinements Notes (applied to existing reqs at execute time, not changing req rows)
- **ATELIER-80** (IDEATE-18): matrix coverage summary must read "post-v0.3: 17 domains, 170 P-rules"; update both the summary block and per-domain section count.
- **ATELIER-81** (IDEATE-19): the "Core Principle Coverage" table (C1C8 → domains) must be updated with the 4 new domains, not just the per-domain row schema table.
- **ATELIER-84** (IDEATE-20, IDEATE-24, IDEATE-25, IDEATE-27): anti-patterns extension must include (a) v0.3 deployable artifact types (.po resource files, .rego policy files, model artifacts, signed manifests as standalone files), (b) GitOps push-pattern violation (P3), (c) i18n LTR-only assumption violation (P6), (d) AI/ML orphan-model violation (P3 Lineage). Domain-specific anti-patterns to pre-specify:
- gitops-operators: push-based deploy (P3), manual kubectl apply on GitOps-managed resource (P8), cluster-admin GitOps robot (P10)
- ai-ml: unreproducible training run (P1), "the latest" model (P5), notebook in production (P9), orphan model with no lineage (P3)
- i18n: inline string concatenation (P3), `if (n == 1)` plural branching (P4), LTR-only layout assumption (P6), hand-rolled date formatter (P5)
- compliance: mutable audit log (P1), shared/generic identity in audit (P7), secret leaked in audit log (P9), manual evidence assembly at audit time (P6)
- **ATELIER-61/62** (IDEATE-21): argocd.md and flux.md must include an "ArgoCD vs Flux" decision matrix (parallel to v0.2 Helm vs Kustomize in ATELIER-46/47).
- **ATELIER-66** (IDEATE-22): data-versioning.md must include a tool comparison table (DVC vs Delta Lake vs LakeFS) covering versioning model, lineage, and use-case fit.
- **ATELIER-78** (IDEATE-23): policy-as-code.md must include an engine comparison table (OPA vs Cedar vs Kyverno vs Sentinel) covering policy language, evaluation gate, and ecosystem.
- **ATELIER-87** (IDEATE-26): the compliance-audit-log bad example must illustrate both an append-only violation (mutation/deletion of an audit record, P1) AND a redaction failure (secret in audit log, P9) — two breaches in one example.
- **ATELIER-74** (IDEATE-28): testing-i18n.md must map pseudo-locale testing to the testing pyramid tiers (unit: missing-key detection; integration: snapshot per locale; e2e: RTL coverage).
- **ATELIER-79** (IDEATE-29): evidence.md must include a fenced signed-attestation example (Cosign or in-toto), not prose-only.
- **ATELIER-69** (IDEATE-30): monitoring-drift.md must enumerate the three drift types (data drift, concept drift, prediction drift) with a detection signal per type.
### Within-Project Pattern Transfer (v0.1 → v0.2 → v0.3) — verified
| v0.2 Lesson | v0.3 Application | Status |
|-------------|------------------|--------|
| IDEATE-09 → ATELIER-59 (peer-review parity) | ATELIER-83 already covers this | ✓ carried forward |
| IDEATE-13/14 (chaos anti-patterns: orphan P-rule, deployable artifact) | ATELIER-84 + IDEATE-20/24/25/27 extend with v0.3-specific chaos | ✓ extended |
| IDEATE-08 (cross-link verification: every new derived doc ≥1 outbound cross-link) | ATELIER-88 already covers this | ✓ carried forward |
| IDEATE-02 (matrix row count = 10 per domain) | ATELIER-80 already covers this | ✓ carried forward |
| IDEATE-03 (domain-coverage row schema) | ATELIER-81 + IDEATE-19 extend with C-rule coverage table update | ✓ extended |
| IDEATE-07 (named bad examples cite P-rule breached) | ATELIER-87 + IDEATE-26 refine (two-breach example) | ✓ extended |
| IDEATE-10/11/12 (decision/comparison tables) | IDEATE-21/22/23 transfer the pattern to 3 v0.3 derived docs | ✓ transferred |
| v0.2 audit ESC-002 note (examples/ not in MANIFEST) | IDEATE-17 → ATELIER-91 | ✓ addressed |
## v0.4 Requirements — Edge + Messaging + Language-Derived Docs
**Milestone type:** NFR (all phases produce docs)
**Tag line:** v0.3.x (previous minor from v0.4)
| REQ-ID | Requirement | Priority | Phase | Status |
|--------|-------------|----------|-------|--------|
| ATELIER-92 | `domains/edge/first-principles.md` — 10 edge principles (P1P10) | P0 | 1 | pending |
| ATELIER-93 | `domains/edge/cdn.md` — CDN/edge-cache derived doc (with edge-cache vs origin decision matrix) | P1 | 1 | pending |
| ATELIER-94 | `domains/edge/offline-first.md` — offline-first derived doc | P1 | 1 | pending |
| ATELIER-95 | `domains/edge/iot.md` — constrained-device/IoT derived doc | P1 | 1 | pending |
| ATELIER-96 | `domains/edge/sync.md` — conflict/sync derived doc (CRDT/last-write-win decision matrix) | P1 | 1 | pending |
| ATELIER-97 | `domains/messaging/first-principles.md` — 10 messaging principles (P1P10) | P0 | 2 | pending |
| ATELIER-98 | `domains/messaging/queues.md` — queue derived doc (with at-least-once/at-most-once/exactly-once comparison) | P1 | 2 | pending |
| ATELIER-99 | `domains/messaging/pubsub.md` — pub/sub derived doc | P1 | 2 | pending |
| ATELIER-100 | `domains/messaging/streams.md` — stream processing derived doc | P1 | 2 | pending |
| ATELIER-101 | `domains/messaging/delivery-semantics.md` — delivery semantics derived doc (idempotency, ordering, dead-letter) | P1 | 2 | pending |
| ATELIER-102 | `languages/typescript.md` extended + `languages/ts-types.md`, `ts-tooling.md`, `ts-async.md`, `ts-testing.md` — 4 TS derived docs | P1 | 3 | pending |
| ATELIER-103 | `languages/python.md` extended + `languages/py-types.md`, `py-tooling.md`, `py-async.md`, `py-testing.md` — 4 Python derived docs | P1 | 3 | pending |
| ATELIER-104 | `languages/go.md` extended + `languages/go-types.md`, `go-tooling.md`, `go-concurrency.md`, `go-testing.md` — 4 Go derived docs | P1 | 3 | pending |
| ATELIER-105 | `languages/rust.md` extended + `languages/rs-ownership.md`, `rs-tooling.md`, `rs-async.md`, `rs-testing.md` — 4 Rust derived docs | P1 | 3 | pending |
| ATELIER-106 | Extend `matrix/principles-matrix.md` with 20 new P-rules → core C-rule mappings (10 per new domain; review check: row count per domain = 10, each row ≥1 C-rule) — post-v0.4: 19 domains, 190 P-rules | P0 | 4 | pending |
| ATELIER-107 | Extend `matrix/domain-coverage.md` with edge + messaging (per-domain row schema) + add `languages/` coverage sub-table (language → derived docs → traced domains) | P1 | 4 | pending |
| ATELIER-108 | Extend `review/agent-checklist.md` with edge + messaging trigger sections + a language-derived-docs section | P1 | 4 | pending |
| ATELIER-109 | Extend `review/peer-review-checklist.md` with edge + messaging + language-derived sections (parity with agent-checklist) | P1 | 4 | pending |
| ATELIER-110 | Extend `review/anti-patterns.md` with edge + messaging violations incl. orphaned P-rule + deployable example artifact; pre-specify domain chaos anti-patterns (edge: sync conflict loop, TTL-less cache; messaging: shared subscription, blocking consumer) | P1 | 4 | pending |
| ATELIER-111 | Update `MANIFEST.md` to list new v0.4 domains + new `languages/` section (first-principles + derived docs per language) + post-v0.4 totals | P0 | 4 | pending |
| ATELIER-112 | `examples/good/edge-offline-sync.md` + `examples/good/messaging-idempotent-consumer.md` — 2 good examples (markdown with fenced code only) | P2 | 5 | pending |
| ATELIER-113 | `examples/bad/edge-sync-loop.md` + `examples/bad/messaging-shared-subscription.md` — 2 named bad examples (each cites the P-rule breached) | P2 | 5 | pending |
| ATELIER-114 | Cross-links from new domains to existing devops/observability/data/concurrency/kubernetes/infrastructure-as-code domains; cross-links from language-derived docs to their traced domain docs (review check: every new derived doc ≥1 outbound cross-link to a MANIFEST-listed doc) | P1 | 5 | pending |
| ATELIER-115 | Final review passes (all v0.4 phases reviewed, audit clean) | P0 | 6 | pending |
| ATELIER-116 | Milestone v0.4 released (tag v0.3.6, merged to main) | P0 | 6 | pending |
| ATELIER-117 | Update `ARCHITECTURE.md` for v0.4 — extend the component tree with `edge/` + `messaging/` domain entries, add a `## v0.4 Domain Additions` section documenting the two new domains (boundary per D-061/D-062, derived docs, cross-link targets), and note the `languages/` derived-doc extension (D-063 apply-don't-derive). Closes architecture-tree drift: the file currently stops at v0.3 with no edge/messaging/languages-derived coverage (IDEATE-32). | P1 | 4 | pending |
## v0.4 Traceability Matrix
| Phase | Requirements |
|-------|-------------|
| 0 (Pre-Execution) | (governance: spec, clarify, research, ideate, plan) |
| 1 (Edge Domain) | ATELIER-92..ATELIER-96 |
| 2 (Messaging Domain) | ATELIER-97..ATELIER-101 |
| 3 (Language-Derived Extension) | ATELIER-102..ATELIER-105 |
| 4 (Matrix + Review Integration) | ATELIER-106..ATELIER-111, ATELIER-117 |
| 5 (Examples + Cross-Links) | ATELIER-112..ATELIER-114 |
| 6 (Final Review + Ship) | ATELIER-115, ATELIER-116 |
## v0.4 Ideation Log
**Generated:** 13 ideas (mechanical: 8, backend-enriched: 5)
**Accepted:** 13 (all v0.4-scope, confidence ≥ 0.79, above 0.6 autonomy threshold → auto-accepted)
**Skipped:** 0
**New requirements added:** 1 (ATELIER-117 — ARCHITECTURE.md v0.4 update, from IDEATE-32)
**Refinements to existing reqs:** 12 (IDEATE-31, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43)
**Deferred to v0.5:** 0
**Rejected:** 0
| IDEATE-ID | Source | Category | Confidence | Decision | Mapped REQ |
|-----------|--------|----------|------------|----------|------------|
| IDEATE-31 | mechanical (MANIFEST drift, parallel to ESC-002) | drift | 0.86 | accepted → refines | ATELIER-111 (languages/ section must list 4 v0.1 stubs + 16 derived docs) |
| IDEATE-32 | mechanical (ARCHITECTURE.md vs directory tree) | drift | 0.88 | accepted → new req | ATELIER-117 (update ARCHITECTURE.md for v0.4: component tree + v0.4 section) |
| IDEATE-33 | mechanical (IDEATE-19 pattern transfer) | coverage | 0.90 | accepted → refines | ATELIER-107 (Core Principle Coverage C1C8 → domains table update for edge + messaging) |
| IDEATE-34 | mechanical (matrix row-count invariant) | drift | 0.82 | accepted → refines | ATELIER-106 (verify post-v0.4 = exactly 190; verify current = exactly 170 per grill G-014 — the "171 vs 170" claim was a false premise, corrected) |
| IDEATE-35 | mechanical (D-069 vs req-row text) | spec | 0.89 | accepted → refines | ATELIER-100, ATELIER-101 (req text must reference streams + dead-letter comparison tables) |
| IDEATE-36 | mechanical (RESEARCH anti-patterns vs ATELIER-110 text) | quality | 0.85 | accepted → refines | ATELIER-110 (language-derived-doc anti-patterns + .ts/.py/.go/.rs artifact-type extension) |
| IDEATE-37 | backend-enriched (chaos anti-pattern enumeration) | chaos | 0.86 | accepted → refines | ATELIER-110 (include all 6 RESEARCH chaos anti-patterns: edge sync-loop/TTL-less/blocking-call; messaging shared-sub/blocking-consumer/unbounded-retry) |
| IDEATE-38 | backend-enriched (IDEATE-28 pattern transfer) | improvement | 0.80 | accepted → refines | ATELIER-94 (offline-first.md maps offline write-queue patterns to testing pyramid tiers) |
| IDEATE-39 | backend-enriched (IDEATE-29 pattern transfer) | improvement | 0.79 | accepted → refines | ATELIER-101 (delivery-semantics.md fenced idempotency-key dedup-store example, not prose-only) |
| IDEATE-40 | backend-enriched (intra-milestone cross-link dependency) | chaos | 0.84 | accepted → refines | ATELIER-114 (P5 cross-link audit must verify edge↔messaging intra-v0.4 links resolve in both directions) |
| IDEATE-41 | backend-enriched (IDEATE-30 pattern transfer) | improvement | 0.81 | accepted → refines | ATELIER-100 (streams.md enumerates consumer-group rebalance strategies: sticky/cooperative/eager) |
| IDEATE-42 | mechanical (D-066 sub-table verifiability) | coverage | 0.87 | accepted → refines | ATELIER-107 (Languages Coverage sub-table = exactly 16 rows; P4 review check) |
| IDEATE-43 | mechanical (IDEATE-18 pattern transfer) | coverage | 0.83 | accepted → refines | ATELIER-111 (MANIFEST Cross-Cutting summary = "19 domains, 190 P-rules post-v0.4") |
### Refinements Notes (applied to existing reqs at execute time, not changing req rows)
- **ATELIER-106** (IDEATE-34, corrected by grill G-014): matrix coverage summary must read
"post-v0.4: 19 domains, 190 P-rules" in BOTH the summary block and
the per-domain section count (IDEATE-18 carried forward). P4 must
verify the matrix contains exactly 190 P-rows (10 per domain × 19).
The current matrix has **exactly 170 P-rows** (17 domains × 10,
verified by grill G-014 direct count) — the IDEATE-34 "171 vs 170"
claim was a FALSE PREMISE. P4 must verify 170 exists, append 20 rows,
and confirm post-v0.4 count = 190. Do NOT hunt for or delete a phantom
171st row.
- **ATELIER-107** (IDEATE-33, IDEATE-42): the domain-coverage.md
extension must update THREE artifacts: (a) the per-domain row-schema
table with edge + messaging rows; (b) the "Core Principle Coverage"
table (C1C8 → domains) for the 2 new domains (C4 Locality grows by
edge P1/P8 + messaging P7; C5 Reversibility grows by edge P2/P4/P7/P10
+ messaging P5/P8/P9; C7 Observability grows by edge P10 + messaging
P8/P10; C8 Economy grows by edge P3/P9 + messaging P6; C2 grows by
messaging P1/P2/P4) — parallel to v0.3 IDEATE-19; (c) the new
`## Languages Coverage` sub-table with EXACTLY 16 rows (one per
language-derived doc), verifiable as a P4 review check (no orphans,
every derived doc traced to ≥1 domain P-rule).
- **ATELIER-110** (IDEATE-36, IDEATE-37): the anti-patterns extension
must include (a) the v0.4 deployable-artifact-types table extended to
`.ts`/`.py`/`.go`/`.rs` standalone files (language-derived docs risk,
per D-020 docs-only contract); (b) language-derived-doc anti-
patterns: introducing a new P-rule (e.g., "TS P1"), orphan section
with no domain P-rule trace, runtime standalone artifact, and
first-principles stub gaining P1P10 numbering; (c) all 6 RESEARCH-
specified chaos anti-patterns, not just the 4 named in the req-row
text — edge: sync conflict loop (P4), TTL-less cache (P6), blocking
call on constrained device (P3/P5); messaging: shared subscription
(P2/P3), blocking consumer (P6), unbounded retry / no DLQ (P5).
- **ATELIER-100** (IDEATE-35, IDEATE-41): streams.md must include (a)
the stream-platform comparison table (Kafka vs Kinesis vs
Pulsar/NATS JetStream — ordering, partitioning, replay/retention,
use-case fit) per D-069, AND (b) an enumeration of consumer-group
rebalance strategies (sticky/cooperative vs eager) as a sub-
comparison — each with partition-stop-the-world cost and use-case
fit. Parallel to v0.3 IDEATE-30 (drift-type enumeration).
- **ATELIER-101** (IDEATE-35, IDEATE-39): delivery-semantics.md must
include (a) the dead-letter strategy comparison table (retry-count-
limit vs TTL-with-backoff vs poison-queue vs DLQ+alert) per D-069,
AND (b) a fenced idempotency-key dedup-store example (not prose-
only) — parallel to v0.3 IDEATE-29 (fenced signed-attestation
example). Prose-only delivery-semantics guidance is weak; a fenced
consumer-with-dedup-store demonstrates P3 (idempotent consumers)
concretely.
- **ATELIER-111** (IDEATE-31, IDEATE-43): the new `## Languages`
MANIFEST section must list the 4 v0.1 first-principles stubs
(typescript.md, python.md, go.md, rust.md — closing the pre-existing
drift since v0.1, parallel to ESC-002/ATELIER-91) AND the 16 new
derived docs. The Cross-Cutting summary line must update to exactly
"19 domains, 190 P-rules post-v0.4" (IDEATE-18 carried forward).
- **ATELIER-114** (IDEATE-40): the P5 cross-link audit must verify
edge↔messaging intra-v0.4 links resolve in BOTH directions, not only
new→existing. Edge/iot.md (P1) cross-links messaging/queues.md (MQTT
QoS parallels delivery semantics) and messaging/iot.md-style
references — but messaging/ docs are authored in P2, so at the P1
tag (v0.3.1) those links dangle. This is acceptable per vertical-
slice integrity (D-053 — matrix/cross-links close in P4/P5), but the
P5 audit (ATELIER-114) must explicitly check that every edge→
messaging and messaging→edge link resolves by the P6 ship. The
audit must also confirm language-derived docs (P3) link to BOTH
new (edge/messaging) and existing domains where relevant (e.g.,
ts-async.md → messaging/delivery-semantics; go-concurrency.md →
concurrency/patterns + messaging/queues).
- **ATELIER-94** (IDEATE-38): offline-first.md must map offline write-
queue / conflict-detection patterns to the testing pyramid tiers
(unit: conflict detection on a merge function; integration:
reconnect reconcile against a local store; e2e: partition
simulation with a fake network) — parallel to v0.3 IDEATE-28 (i18n
pseudo-locale → pyramid). Avoids generic "test offline" guidance.
### Within-Project Pattern Transfer (v0.1 → v0.2 → v0.3 → v0.4) — verified
| Prior Lesson | v0.4 Application | Status |
|--------------|------------------|--------|
| IDEATE-10/11/12 + 21/22/23 (decision/comparison tables in multi-tool derived docs) | D-069 + IDEATE-35 transfer the pattern to streams.md (Kafka/Kinesis/Pulsar) + delivery-semantics.md (dead-letter strategies) | ✓ transferred |
| IDEATE-17 (manifest drift closure — examples/ unlisted → ATELIER-91) | IDEATE-31 → ATELIER-111 (languages/ unlisted since v0.1 — closes pre-existing drift on the same ESC-002 precedent) | ✓ transferred |
| IDEATE-18 (coverage summary totals must update — string + per-domain count) | IDEATE-43 → ATELIER-111 (Cross-Cutting = "19 domains, 190 P-rules post-v0.4") + IDEATE-34 → ATELIER-106 (summary block) | ✓ transferred |
| IDEATE-19 (Core Principle Coverage C1C8 → domains table update) | IDEATE-33 → ATELIER-107 (C-rule coverage table extended for edge + messaging with the documented C-count shifts) | ✓ transferred |
| IDEATE-20 (anti-patterns specificity: domain violations + artifact types) | IDEATE-36 + IDEATE-37 → ATELIER-110 (language-derived anti-patterns + .ts/.py/.go/.rs artifact types + all 6 chaos anti-patterns) | ✓ extended |
| IDEATE-07 (named bad examples cite the P-rule breached) | ATELIER-113 + D-068 already cover (single-breach, cite specific P-rule) | ✓ carried forward |
| IDEATE-08 (cross-link verification: every new derived doc ≥1 outbound) | ATELIER-114 already covers; IDEATE-40 extends to edge↔messaging intra-milestone links | ✓ extended |
| IDEATE-13/14 (chaos: orphan P-rule + deployable artifact) | IDEATE-36 → ATELIER-110 (language orphan-section + standalone .ts/.py/.go/.rs runtime artifact) | ✓ extended |
| IDEATE-28 (pseudo-locale testing → testing pyramid tiers) | IDEATE-38 → ATELIER-94 (offline write-queue testing → pyramid tiers) | ✓ transferred |
| IDEATE-29 (fenced signed-attestation example, not prose-only) | IDEATE-39 → ATELIER-101 (fenced idempotency-key dedup-store example) | ✓ transferred |
| IDEATE-30 (drift-type enumeration: data/concept/prediction + signal each) | IDEATE-41 → ATELIER-100 (rebalance-strategy enumeration: sticky/cooperative/eager + cost each) | ✓ transferred |
File diff suppressed because it is too large Load Diff
+465
View File
@@ -0,0 +1,465 @@
# Atelier — Research Findings (P0)
> Research conducted during phase 0 RESEARCH stage. Conclusions inform ARCHITECTURE.md, PERSONAS.md, and the plan.
## Domain: Engineering-Principles Frameworks
### Prior Art Surveyed
1. **ThoughtWorks Technology Radar** — opinionated tech assessments, but no first-principles derivation. Adopts positions, not axioms.
2. **Google Engineering Practices** — review checklists and style guides; lacks cross-domain principle tracing.
3. **Microsoft Azure Design Principles** — cloud-architecture-focused; not generalizable across domains.
4. **12-Factor App** — narrow (deployment/config), but a model for "small set of universal rules" pattern.
5. **SOLID / GRASP** — OO-specific principles; good model for "principles over rules" but language-bound.
6. **WCAG 2.1 AA** — the canonical a11y reference; Atelier's `domains/uiux/accessibility.md` already mirrors it.
7. **OWASP Top 10 / ASVS** — security control catalog; Atelier's security domain derives principles, not controls.
8. **Clean Code / Pragmatic Programmer** — book-level principles; not machine-consumable, no traceability matrix.
### Key Insight
Atelier's differentiation: **traceable principle hierarchy with a join table**. Existing frameworks state principles; none provide a `matrix/principles-matrix.md` that maps every domain rule back to a core rule. This is the framework's unique value — conflict resolution has an arbiter.
### Atelier's Position
- **Not** a style guide (no code formatting rules)
- **Not** a linter (no enforcement tooling)
- **Is** a principle hierarchy with cross-references, consumed by AI agents and humans
- **Is** docs-as-code (versioned, reviewed, owned)
## Ecosystem & Distribution
- **Format:** Markdown (CommonMark + GFM) — universally readable, agent-friendly
- **Distribution:** Git repository (consumed by reading)
- **Agent consumption:** Agents read `MANIFEST.md``core/` → relevant `domains/` before completing a task. The `review/agent-checklist.md` is the pre-completion gate.
- **Human consumption:** `README.md``MANIFEST.md``core/reading-order.md` for guided tour
## Architectural Decisions (Refined from RESEARCH)
| Decision | Rationale |
|----------|-----------|
| 8 core principles (not 10) | The spec's matrix key names exactly 8 (C1C8). Each domain has 10 (P1P10). Asymmetry is intentional: core is minimal, domains are specific. |
| Numeric precedence C1 > C2 > ... > C8 | Correctness is non-negotiable; Economy is the most tradeable. Matches intuition and the matrix ordering. |
| Strict hierarchy: core > domains > languages | No lateral authority. Conflicts go UP, never sideways. |
| MANIFEST.md is the authoritative index | Unlisted docs aren't part of the framework — prevents drift. |
| No build step | Markdown is the artifact. A build step would violate "docs-as-code" simplicity. |
## Persona Assessment
Atelier is a **documentation-only** project. The default persona roster (lead-developer, data-engineer, backend-engineer, frontend-engineer) is wrong for this project — there is no runtime code, no database, no API server, no UI components.
### Active Personas (Custom for Atelier)
| Persona | Domain | Why Active |
|---------|--------|-----------|
| lead-developer | coordination | Decomposes doc-writing tasks by domain; coordinates phase execution |
| tech-writer | documentation | Author of all framework markdown; owns docs-as-code discipline |
| domain-expert | cross-cutting | Validates domain principle correctness (security, data, api, etc.); ensures traceability to core |
### Inactive Personas (from default roster)
| Persona | Why Inactive |
|---------|--------------|
| data-engineer | No database, schema, migrations in this project |
| backend-engineer | No API server, routes, services, middleware |
| frontend-engineer | No React/Next components, pages, hooks |
### Frameworks
None. Atelier produces markdown only. No `package.json`, no runtime dependencies.
### Territories
| Persona | Territory globs |
|---------|-----------------|
| lead-developer | `.ciagent/**`, `MANIFEST.md`, `README.md` |
| tech-writer | `**/*.md` (excluding `.ciagent/**`), `LICENSE` |
| domain-expert | `core/**`, `domains/**`, `matrix/**`, `review/**`, `examples/**` |
## Risks
| Risk | Mitigation |
|------|-----------|
| Domain principles drift from core (orphaned rules) | matrix/principles-matrix.md is the arbiter; every P-rule must trace to a C-rule. P4 verifies this. |
| Inconsistent principle numbering across domains | Enforce 10 principles per domain (D-018); CI check in P4. |
| Anti-patterns examples contradict principles | examples/bad/* must reference the principle they violate (P5 task). |
| Manifest drift (unlisted docs appear) | P4 review/anti-patterns.md documents this; MANIFEST is regenerated from the tree. |
## Conclusions
1. Atelier's unique value is the **traceable principle matrix** — not the individual docs.
2. The 8 core principles are recoverable from `matrix/principles-matrix.md`'s key (D-011) — enough to author `core/first-principles.md` in P1.
3. Custom personas needed: tech-writer (primary), domain-expert (validator). Default 3 inactive.
4. No build/lint tooling in scope — markdown is the artifact (D-016, ARCHITECTURE.md).
5. Phase ordering (P1P6) follows the spec's build order; matrix + review come after all domains exist (P4).
---
# v0.2 Research — Infrastructure as Code + Kubernetes
> Research conducted during v0.2 P0 RESEARCH stage. Informs the two new domains, matrix extension, and the phase-specific platform-engineer persona. See CLARIFY.md D-019..D-030 for resolved ambiguities.
## Domain A: Infrastructure as Code (Terraform / OpenTofu)
### Prior Art
- **Terraform** (HashiCorp, 2014, original MPL-2.0): core workflow is **Write → Plan → Apply**. Declarative HCL; provider-based resource model; state file as the source of truth for drift detection. ([developer.hashicorp.com/terraform/intro/core-workflow](https://developer.hashicorp.com/terraform/intro/core-workflow))
- **OpenTofu** (Linux Foundation, 2023 fork of MPL Terraform after HashiCorp's BUSL license change): community-driven, truly open source (MPL path), backwards-compatible with Terraform, layered and modular, impartial stewardship. ([opentofu.org/manifesto](https://opentofu.org/manifesto))
- **Pulumi** (out of v0.2 scope per D-019 — user chose terraform+opentofu; general IaC principles cover the model).
- **12-Factor** (already in Atelier v0.1 devops domain): config in environment, disposable, dev/prod parity.
### Principles Identified for `infrastructure-as-code/first-principles.md` (P1P10)
Each derived from a core C-rule (see matrix updates in P3):
1. **P1 Declarative Intent** — describe desired state, not steps to reach it. (C2 Clarity, C3 Simplicity)
2. **P2 Idempotence** — applying the same config twice yields the same result. (C1 Correctness)
3. **P3 State is Truth** — the state file is the authoritative record; drift is a bug. (C1 Correctness, C7 Observability)
4. **P4 Plan Before Apply** — preview every change before mutating real infrastructure. (C5 Reversibility)
5. **P5 Version Everything** — config, state, providers, modules all versioned and reproducible. (C5 Reversibility)
6. **P6 Modules Compose** — encapsulate repeatable patterns as versioned modules; compose, don't copy. (C6 Composability)
7. **P7 Least Privilege Providers** — provider credentials scoped to minimum needed. (C1 Correctness via security)
8. **P8 Remote State with Locking** — state stored remotely with locking to prevent concurrent corruption. (C4 Locality, C1 Correctness)
9. **P9 Drift is Recoverable**`plan` detects drift; `apply` reconciles. Manual mutation is an incident. (C5 Reversibility, C7 Observability)
10. **P10 Secrets Never in Code** — secrets via providers/external secret stores, never hardcoded in HCL. (C1 Correctness, security non-tradeable)
### Derived Docs
- `terraform.md` — HCL structure, providers, resources, data sources, workspaces, `terraform init/plan/apply/destroy`, registry, state backends.
- `opentofu.md` — fork lineage (BUSL → MPL via Linux Foundation), CLI parity, registry parity, migration path from Terraform, community governance, when to choose OpenTofu (license neutrality, supply-chain).
- `state.md` — state backends (S3+DynamoDB lock, GCS, Azure blob, local for dev), locking, state isolation per environment, `terraform state` commands, sensitive values in state, state import/export.
- `modules.md` — module structure, inputs/outputs, versioning (registry tags), `source` patterns (registry, git, local), module composition, the module-vs-copy boundary.
### Cross-Domain Links (one-directional in v0.2, per D-026)
- `devops/P1 Reproducibility` ← IaC P1, P5
- `devops/P6 Configuration as Code` ← IaC P1
- `security/secrets` ← IaC P10
- `security/supply-chain` ← OpenTofu license-stewardship angle (cross-link, not duplication)
- `data/migrations` ← IaC state lifecycle (analog, not direct)
## Domain B: Kubernetes
### Prior Art
- **Kubernetes** (CNCF, graduated): container orchestration platform. Concept taxonomy (per kubernetes.io/docs/concepts): Workloads (Pod, Deployment, ReplicaSet, StatefulSet, DaemonSet, Job, CronJob), Services/Networking (Service, Ingress, Gateway API, NetworkPolicy, DNS), Storage (PV, PVC, StorageClass, CSI), Security (RBAC, ServiceAccounts, Pod Security Standards, Secrets, multi-tenancy), Configuration (ConfigMaps, Secrets, Resource Management), Policies (LimitRanges, ResourceQuotas), Scheduling/Eviction.
- **Helm** (CNCF): package manager for k8s — charts as templated manifests, values overrides, registries.
- **Kustomize** (built-in): overlay-based manifest customization without templating — base + overlays, no DSL.
- **12-Factor App** — backing influence for k8s workload principles (disposability, config, logs).
- **GitOps** (ArgoCD, Flux) — out of v0.2 first-principles scope but referenced as a deployment pattern for helm/kustomize; candidate IDEATE item.
### Principles Identified for `kubernetes/first-principles.md` (P1P10)
1. **P1 Declarative Desired State** — controllers reconcile current → desired. Imperative commands are exceptions. (C1 Correctness, C2 Clarity)
2. **P2 Pods are Mortal** — never assume a pod's identity or lifetime. Use controllers, not bare pods. (C5 Reversibility)
3. **P3 Labels Select** — labels + selectors are the join mechanism for workloads, services, policies. (C6 Composability, C2 Clarity)
4. **P4 Requests and Limits are Contracts** — resource requests drive scheduling; limits drive QoS. Unspecified = best-effort = first evicted. (C8 Economy, C1 Correctness)
5. **P5 Probes Drive Health** — liveness, readiness, startup probes; the platform cannot heal what it cannot see. (C7 Observability, C1 Correctness)
6. **P6 Namespaces Bound Blast Radius** — namespaces are the unit of quota, RBAC, network policy, and cleanup. (C4 Locality)
7. **P7 RBAC by Intent, Not Identity** — bind roles to service accounts by workload purpose; least privilege; avoid cluster-admin. (C1 Correctness via security)
8. **P8 Storage is Explicit** — ephemeral by default; persistence requires PVC + StorageClass + reclaim policy decisions. (C5 Reversibility, C4 Locality)
9. **P9 Config and Secrets are Separate** — ConfigMaps for non-sensitive, Secrets for sensitive; both injected, never baked into images. (C2 Clarity, security)
10. **P10 Roll Forward, Roll Back** — rolling updates + rollout history; every deployment has a known-good rollback. (C5 Reversibility)
### Derived Docs
- `workloads.md` — Pod, ReplicaSet, Deployment, StatefulSet (identity + ordering), DaemonSet, Job/CronJob, controllers, probes, lifecycle, disruption budgets.
- `networking.md` — Service (ClusterIP/NodePort/LoadBalancer), Ingress, Gateway API, EndpointSlices, NetworkPolicy, DNS, dual-stack.
- `storage.md` — Volumes, PV/PVC, StorageClass, CSI, dynamic provisioning, volume snapshots, reclaim policies, ephemeral volumes.
- `rbac.md` — Role/ClusterRole, RoleBinding/ClusterRoleBinding, ServiceAccount, Pod Security Standards/Admission, RBAC good practices (cross-link `security/authorization`).
- `helm.md` — chart structure, values.yaml, templates, registries, release management, when Helm fits vs Kustomize.
- `kustomize.md` — base + overlays, kustomization.yaml, patches, no templating, when Kustomize fits vs Helm.
### Cross-Domain Links (one-directional in v0.2, per D-026)
- `devops/P4 Rollback First` ← k8s P10
- `devops/P5 Progressive Delivery` ← k8s rolling updates, canary via labels
- `devops/P3 Observability` ← k8s P5 probes, metrics
- `security/authorization` ← k8s P7 RBAC
- `security/secrets` ← k8s P9 Secrets
- `observability/metrics` ← k8s P5, resource metrics pipeline
- `data/storage` ← k8s P8 (analog: PVC lifecycle ↔ migration discipline)
## Architectural Fit (v0.1 Contract Preservation)
- **Hierarchy preserved:** both new domains depend on `core/`; their P-rules trace to C1C8 via the matrix. No lateral authority.
- **10 P-rules per domain** (per D-018, D-030): consistent with v0.1.
- **Manifest authoritative:** all new documents added to `MANIFEST.md` in P3. Unlisted = not part of the framework.
- **No runtime code:** examples are illustrative markdown with manifests in code fences (per D-025, PROJECT.md constraint). No `.tf`, `.yaml`, `.sh` deployable artifacts — only fenced code blocks inside `.md` files.
- **Conflict resolution unchanged:** matrix extended, not replaced. Core precedence (C1 > C2 > ... > C8) governs any new vs existing rule conflict.
## Prior Art Position (v0.2 extension)
Existing IaC/k8s guidance (Terraform docs, k8s docs, Helm/Kustomize docs, Google SRE, CNCF guides) state practices but none map every domain rule back to a small set of universal core principles. Atelier's v0.2 contribution is the same differentiation as v0.1: **traceable principle hierarchy with a join table**. The two new domains add 20 P-rules, each traced to a core C-rule, extending the matrix from 110 to 130 domain principles across 13 domains.
## v0.2 Persona Assessment
See `.ciagent/atelier/PERSONAS.md` for the updated roster. v0.2 adds one phase-specific persona:
- **platform-engineer** (phase-specific, removed after v0.2): domain = infrastructure/platform; territory = `domains/infrastructure-as-code/**, domains/kubernetes/**, examples/good/{terraform-module,k8s-deployment}.md, examples/bad/*`; constraints = ["declarative-first", "stateless examples", "trace to core", "10 P-rules per domain"]; reason = specialist authoring for IaC/k8s content with domain expertise the existing tech-writer persona lacks.
## v0.2 Risks and Mitigations
| Risk | Mitigation |
|------|-----------|
| New P-rules orphaned from core (no matrix trace) | P3 extends matrix; domain-expert persona verifies every new P-rule traces to a C-rule before sign-off |
| IaC and k8s domains overlap (state, secrets) | Cross-links are one-directional (D-026); each domain owns its own principle angle (IaC P3 state-as-truth vs k8s P8 storage-explicit) |
| Examples become deployable (runtime code) | platform-engineer constraint "stateless examples"; examples are markdown with fenced code, never standalone `.tf`/`.yaml` files |
| OpenTofu doc duplicates terraform.md | OpenTofu doc covers fork-specific governance/license/migration; references terraform.md for the shared HCL/state model |
| Helm vs Kustomize preference bias | Both docs present neutral "when X fits" guidance; neither advocated over the other |
## v0.2 Conclusions
1. Two new top-level domains extend the framework without breaking the v0.1 contract.
2. 20 new P-rules (10 per domain) all trace to core C1C8 — matrix extends from 110 to 130.
3. OpenTofu shares the IaC domain with Terraform (fork lineage); separate derived doc covers governance/license.
4. State and modules get their own derived docs (cross-cutting IaC concerns).
5. K8s derived docs mirror the k8s concept taxonomy: workloads, networking, storage, rbac, helm, kustomize.
6. A phase-specific platform-engineer persona is warranted for P1P4; removed after v0.2.
7. No runtime code; examples are illustrative markdown only.
---
# v0.3 Research — GitOps + Operators + AI/ML + i18n + Compliance
> Research conducted during v0.3 P0 RESEARCH stage. Informs the four new domains, matrix extension (+40 mappings), and the two phase-specific personas (platform-engineer extended, ml-engineer added). See CLARIFY.md D-021..D-026 for resolved ambiguities and PROJECT.md D-016..D-026 for milestone decisions.
## Domain A: GitOps + Operators (ArgoCD, Flux, Operators, Progressive Delivery)
### Prior Art
- **CNCF OpenGitOps Principles v1.0.0** (GitOps Working Group, TAG App Delivery): the canonical 4 principles — **Declarative**, **Versioned and Immutable**, **Pulled Automatically**, **Continuously Reconciled**. Atelier's gitops-operators domain derives its first-principles from these plus the Operator pattern. ([opengitops.dev](https://opengitops.dev/), [github.com/open-gitops/documents](https://github.com/open-gitops/documents))
- **ArgoCD** (CNCF graduated): pull-based GitOps controller for k8s. Core concepts: Application CRD, sync waves, health/status assessment, diff against live cluster, RBAC, SSO. Declarative desired state from git; reconciled onto the cluster. ([argoCD.readthedocs.io](https://argoCD.readthedocs.io/))
- **Flux** (CNCF graduated): GitOps Toolkit — a set of composable controllers (source-controller, kustomize-controller, helm-controller, notification-controller). Pulls git/Helm/OCI sources, reconciles via kustomize/helm, emits events. Composable-controller architecture is a C6 (Composability) exemplar. ([fluxcd.io](https://fluxcd.io/))
- **Kubernetes Operator Pattern** (CNCF): a controller that encodes human operational knowledge as CRDs + control loops. Pattern documented in the k8s docs and "Operator Framework" (Operator SDK, OLM). Domain expertise as code; the deepest expression of k8s P1 Declarative Desired State. ([kubernetes.io/docs/concepts/extend-kubernetes/operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/))
- **Progressive Delivery** — Argo Rollouts, Flagger: canary/blue-green traffic shifting driven by analysis (metrics, counters). Extends k8s rolling updates with metric-gated promotion. Cross-links devops/P5 Progressive Delivery.
- **Google SRE** (already in Atelier v0.1 observability/devops): reconciliation loops, error budgets, progressive rollout. Cross-cutting influence.
- **v0.2 in-tree prior art**: `kubernetes/first-principles.md` P1 (Declarative Desired State), P10 (Roll Forward Roll Back); `infrastructure-as-code/first-principles.md` P1 (Declarative Intent), P3 (State is Truth), P9 (Drift is Recoverable). GitOps-operators is the deployment-automation layer above these.
### Principles Identified for `gitops-operators/first-principles.md` (P1P10)
Each derived from a core C-rule (matrix extensions in P4):
1. **P1 Git is the Source of Truth** — desired state lives in a versioned, immutable git store; the cluster is a derivative, not an authority. (C1 Correctness, C5 Reversibility)
2. **P2 Declarative Over Imperative** — express desired cluster state, not the commands to reach it. (C2 Clarity, C3 Simplicity)
3. **P3 Pull, Don't Push** — agents running inside the target pull desired state; no outside push credentials into the cluster. (C1 Correctness via security, C4 Locality)
4. **P4 Continuous Reconciliation** — the loop is the primitive; drift is detected and corrected automatically, not on-demand. (C7 Observability, C1 Correctness)
5. **P5 State is Immutable and Versioned** — every change is a commit; history is the audit trail and the rollback path. (C5 Reversibility)
6. **P6 Operators Encode Domain Knowledge** — operational expertise lives as CRDs + controllers, not runbooks that humans must remember. (C6 Composability, C2 Clarity)
7. **P7 Progressive Delivery is Reversible by Construction** — canary/blue-green are staged, metric-gated, and one-command abortable. Promotion without a rollback path is a violation. (C5 Reversibility, C1 Correctness)
8. **P8 Reconcile, Don't Mutate by Hand** — manual `kubectl apply`/`kubectl edit` on a GitOps-managed resource is an incident; drift back to git is the recovery. (C1 Correctness, C7 Observability)
9. **P9 Failure is Observable and Surfaced** — sync failures, health degradation, and rollout-stall events emit status + notifications; silent drift is the bug. (C7 Observability)
10. **P10 Least Privilege Reconciliation** — the controller's credentials are scoped to the namespaces/resources it reconciles; no cluster-admin GitOps robots. (C1 Correctness via security, C8 Economy of trust)
### Derived Docs
- `argocd.md` — Application CRD, App-of-Apps, sync waves, health checks, diffs, RBAC/SSO, multi-cluster, sync windows.
- `flux.md` — GitOps Toolkit controllers (source, kustomize, helm, notification), composable architecture, HR/Kustomization/HelmRelease CRDs, OCI sources.
- `operators.md` — Operator pattern, CRDs, controllers, Operator SDK/OLM, when to write an operator vs a Helm chart, scope/responsibility boundaries.
- `progressive-delivery.md` — Argo Rollouts + Flagger, canary/blue-green, analysis templates (metrics, counters), abort/rollback, cross-link devops/P5.
### Cross-Domain Links (one-directional in v0.3, per D-026 extended)
- `kubernetes/P1 Declarative Desired State` ← gitops P2
- `kubernetes/P10 Roll Forward Roll Back` ← gitops P7
- `infrastructure-as-code/P1 Declarative Intent` ← gitops P2
- `infrastructure-as-code/P3 State is Truth` ← gitops P1, P5
- `infrastructure-as-code/P9 Drift is Recoverable` ← gitops P4, P8
- `devops/P1 Reproducibility` ← gitops P1, P5
- `devops/P4 Rollback First` ← gitops P5, P7
- `devops/P5 Progressive Delivery` ← gitops P7
- `devops/P6 Configuration as Code` ← gitops P1, P2
- `security/secrets` ← gitops P3, P10 (reconciliation credentials)
- `security/supply-chain` ← gitops P5 (signed/immutable manifest provenance)
- `observability/metrics` ← gitops P4, P9 (reconciliation + rollout metrics)
## Domain B: AI / ML (Engineering Discipline)
### Prior Art
- **Google MLOps / "Hidden Technical Debt in ML Systems"** (Sculley et al., 2015): the foundational paper framing ML systems as software-engineering problems with debt surfaces (data dependencies, configuration, glue code, reproducibility). Atelier's ai-ml domain is the principles-layer response.
- **DVC / Data Version Control** (iterative.ai): git for data + pipelines; treats datasets, features, and models as versioned artifacts. C5 (Reversibility) and C6 (Composability) exemplar.
- **MLflow** (Linux Foundation): experiment tracking, model registry, model packaging, deployment stages. Tracking → registry → serving lifecycle.
- **Kubeflow** (CNCF): k8s-native ML pipelines, training operators, serving (KServe). Brings ML onto the k8s reconciliation model (cross-link kubernetes).
- **KServe / Seldon Core / BentoML**: model serving runtimes; inference as a scalable, observable service. Cross-link devops/P7 Immutability, observability/metrics.
- **Evidently AI / Great Expectations**: data drift detection, data quality, model monitoring. C7 (Observability) for ML.
- **"Machine Learning Operations (MLOps)"** frameworks — Microsoft MLOps, AWS MLOps, Google MLOps maturity model. Converge on: version data, track experiments, evaluate models, serve reproducibly, monitor drift.
- **v0.2 in-tree prior art**: `kubernetes/first-principles.md` (serving on k8s), `infrastructure-as-code/` (training pipelines as declarative infra), `data/` (schema, migrations — data versioning analog).
### Principles Identified for `ai-ml/first-principles.md` (P1P10)
Scope per D-023: engineering discipline (data versioning, evaluation, serving, drift), NOT algorithm/model design. Each derived from a core C-rule:
1. **P1 Reproducibility is the First Class** — every training run is reproducible from pinned data + code + config + environment. Unreproducible runs are unreviewable. (C1 Correctness, C5 Reversibility)
2. **P2 Data is Versioned, Not Just Code** — datasets, features, and splits are first-class versioned artifacts with lineage; `git` alone is insufficient. (C5 Reversibility, C7 Observability)
3. **P3 Lineage is Traceable End-to-End** — any deployed prediction traces back through model → training run → dataset → source. No orphan models. (C7 Observability, C1 Correctness)
4. **P4 Evaluation is Defined Before Training** — metrics, splits, and thresholds are declared a priori; cherry-picking metrics post-hoc is a correctness violation. (C1 Correctness, C2 Clarity)
5. **P5 Models are Versioned Artifacts** — a model is a pinned, immutable, registry-tracked artifact with a unique identifier; never "the latest." (C5 Reversibility, C6 Composability)
6. **P6 Serving is Observable** — inference latency, throughput, input distributions, and prediction confidence are first-class signals. Silent serving is a bug. (C7 Observability)
7. **P7 Drift is Expected and Detected** — data drift, concept drift, and prediction drift are monitored; a drift signal is an incident, not a curiosity. (C7 Observability, C1 Correctness)
8. **P8 Inference Inputs are Validated** — the model's contract (schema, ranges, types) is enforced at the serving boundary; out-of-contract inputs are rejected, not silently scored. (C1 Correctness via security/input-validation)
9. **P9 Pipelines Compose, Notebooks Don't** — training/serving flows are composable pipelines with explicit steps and contracts; notebooks are for exploration, not production. (C6 Composability, C2 Clarity)
10. **P10 Rollback Includes the Model** — a serving rollback restores the prior model artifact, not just the prior code; promotion is reversible at the model layer. (C5 Reversibility)
### Derived Docs
- `data-versioning.md` — DVC/Delta Lake/LakeFS patterns, data lineage, dataset hashing, train/val/test split versioning, cross-link data/migrations.
- `model-evaluation.md` — metric selection, offline/online eval, holdout integrity, bias/fairness checks (engineering angle), eval as a gate.
- `serving.md` — KServe/Seldon/BentoML, inference as a service, batching, latency SLAs, canarying models, cross-link kubernetes + devops.
- `monitoring-drift.md` — Evidently/Great Expectations, drift types (data/concept/prediction), alerting, retraining triggers, cross-link observability/metrics.
### Cross-Domain Links (one-directional in v0.3)
- `data/migrations` ← ai-ml P2 (data versioning ↔ migration discipline)
- `data/schema-design` ← ai-ml P8 (inference input contract)
- `observability/metrics` ← ai-ml P6, P7
- `observability/logging` ← ai-ml P3 (lineage)
- `devops/P1 Reproducibility` ← ai-ml P1
- `devops/P7 Immutability` ← ai-ml P5 (model images)
- `devops/P5 Progressive Delivery` ← ai-ml P10 (model canary)
- `security/input-validation` ← ai-ml P8
- `security/secrets` ← ai-ml P8 (serving credentials)
- `performance/backend` ← ai-ml P6 (serving latency)
- `kubernetes/workloads` ← ai-ml P9 (serving on k8s)
## Domain C: Internationalization (i18n)
### Prior Art
- **Unicode / ICU / CLDR** (Unicode Consortium): the foundation — ICU (International Components for Unicode) for formatting/collation, CLDR (Common Locale Data Repository) for locale data. The de-facto source for date/number/currency/plural/relative-time formatting. ([unicode.org/cldr](https://cldr.unicode.org/), [icu.unicode.org](https://icu.unicode.org/))
- **W3C Internationalization** (W3C i18n WG): the canonical web i18n guidance — "Internationalization techniques", "Language tags in HTML and XML", bidi/RTL authoring. Cross-links WCAG for accessibility-of-locale. ([w3.org/International](https://www.w3.org/International/))
- **RFC 5646 / BCP 47** — language tags (`en-US`, `ar-EG`, `zh-Hans-CN`). The locale identifier standard.
- **RFC 9229 / RFC 9230** (and earlier BCP 47 extensions) — Unicode locale extensions (`-u-`).
- **gettext / ICU MessageFormat / FormatJS / react-intl / i18next / Fluent (Mozilla)** — message-format libraries; ICU MessageFormat is the cross-ecosystem baseline for plural/gender/select. Fluent pioneered "localization 2.0" with asymmetric translations.
- **JavaScript Intl API** — browser-native formatting built on ICU/CLDR; the runtime baseline.
- **WCAG 2.1 AA** (already in Atelier uiux/accessibility): cross-cutting — locale support is an a11y concern for non-Latin-script users; RTL layout is a UI-correctness concern.
- **Google i18n + Mozilla L10n guides** — operational practice (string extraction, pseudo-locale testing, RTL testing).
- **v0.2/v0.1 in-tree prior art**: `uiux/` (accessibility, components, copywriting — i18n's consumer), `testing/` (fixtures, pyramid — i18n testing parallels), `api/error-responses` (localized API errors).
### Principles Identified for `i18n/first-principles.md` (P1P10)
Each derived from a core C-rule:
1. **P1 Source Language is a Locale, Not the Default** — the developer's language is one locale among many, not the "neutral" form. Strings are extracted from day one. (C2 Clarity, C1 Correctness)
2. **P2 Locale Identifiers are Standardized** — use BCP 47 language tags; no ad-hoc locale codes. (C2 Clarity, C6 Composability)
3. **P3 Resources are External, Not Inline** — user-facing strings live in locale resource files, never concatenated inline in code. (C4 Locality, C6 Composability)
4. **P4 Plural and Gender are Parameterized** — use ICU MessageFormat (or equivalent) for plural/gender/select; never `if (n == 1)` branching. (C1 Correctness, C6 Composability)
5. **P5 Formatting is Locale-Aware** — dates, times, numbers, currencies, units via ICU/CLDR/`Intl`; never hand-rolled formatters. (C1 Correctness, C7 Observability of format correctness)
6. **P6 Text Direction is a Layout Primitive** — RTL/bidi is a first-class layout concern, not a CSS afterthought; logical properties (`start`/`end`) over physical (`left`/`right`). (C1 Correctness, C4 Locality)
7. **P7 Layout Accommodates Expansion** — translated text expands/contracts; layouts are flexible (no fixed pixel widths for text). (C8 Economy of rework, C3 Simplicity)
8. **P8 Pseudo-Locales Test Early** — test with pseudo-locales (accented, lengthened, RTL-mirrored) before real translations arrive. (C7 Observability, C5 Reversibility of finding bugs late)
9. **P9 Images and Icons are Cultural** — icons, colors, and imagery are locale-sensitive; avoid locale-bound symbols as universal. (C1 Correctness, C2 Clarity)
10. **P10 Translation is Reversible and Versioned** — resource files are versioned; a bad translation is a rollback, not a hot-patch. (C5 Reversibility)
### Derived Docs
- `locale-resources.md` — resource file formats (.po/.pot, JSON, Fluent FTL, ICU Resource Bundle), key naming, namespaces, fallback chains, extraction tooling.
- `formatting.md` — ICU/CLDR/`Intl` for dates, times, numbers, currencies, units, relative time, plural rules; BCP 47 tags; cross-link api/error-responses for localized errors.
- `rtl-bidi.md` — logical vs physical CSS properties, bidi algorithm (UAX #9), `dir` attribute, mirroring, common pitfalls (icons, numbers in RTL), cross-link uiux/components + uiux/accessibility.
- `testing-i18n.md` — pseudo-locales, snapshot testing per locale, RTL coverage, missing-key detection, cross-link testing/fixtures + testing/pyramid.
### Cross-Domain Links (one-directional in v0.3)
- `uiux/accessibility` ← i18n P6 (RTL/bidi is an a11y concern for non-Latin users)
- `uiux/components` ← i18n P6, P7
- `uiux/copywriting` ← i18n P1, P3
- `testing/fixtures` ← i18n P8
- `testing/pyramid` ← i18n P8
- `api/error-responses` ← i18n P5 (localized error messages)
- `data/schema-design` ← i18n P2, P3 (locale data shapes)
## Domain D: Compliance (Audit, Retention, Policy-as-Code, Evidence)
### Prior Art
**Note (D-024):** the compliance domain is framework-agnostic — it abstracts regulation-specific requirements (GDPR, HIPAA, SOC 2, PCI-DSS, NIST 800-53, ISO 27001) into engineering principles. No regulation-specific docs; they would bloat the framework and go stale.
- **NIST Cybersecurity Framework (CSF) / NIST 800-53** — controls catalog (audit, retention, evidence, policy). Atelier abstracts the *principles*, not the controls.
- **SOC 2 (AICPA) Trust Services Criteria** — Security, Availability, Processing Integrity, Confidentiality, Privacy. Audit logs, retention, and evidence are explicit criteria.
- **GDPR / CCPA** — data subject rights, retention limits, lawful basis. Abstracted to "retention is a function of policy, not storage."
- **OWASP AppSec / ASVS** — already in Atelier security domain; compliance extends to auditability of security controls.
- **Open Policy Agent (OPA) / Rego, Cedar (AWS), HashiCorp Sentinel, Kyverno** — policy-as-code engines; policy evaluated as a gate, not a document. C6 (Composability) + C1 (Correctness) exemplars. ([openpolicyagent.org](https://www.openpolicyagent.org/), [kyverno.io](https://kyverno.io/))
- **Cosign / Sigstore / in-toto** — signed attestations and provenance; evidence-as-artifact. Cross-link security/supply-chain.
- **Google Cloud Audit Logs / AWS CloudTrail / Azure Activity Log** — the canonical audit-log patterns; immutable, append-only, queryable, time-ordered.
- **v0.2/v0.1 in-tree prior art**: `security/` (authorization, secrets, supply-chain), `observability/` (logging, metrics, tracing — audit logs are structured logging), `data/` (schema, migrations — retention schema), `infrastructure-as-code/` (policy-as-code parallels declarative IaC), `kubernetes/` (rbac — audit subject identity).
### Principles Identified for `compliance/first-principles.md` (P1P10)
Each derived from a core C-rule. Framework-agnostic per D-024:
1. **P1 Audit Logs are Append-Only** — audit records are immutable once written; deletion or mutation is itself an auditable incident. (C1 Correctness, C5 Reversibility)
2. **P2 Every Significant Action is Logged** — the set of auditable actions is defined a priori; "we forgot to log it" is a violation. Auth changes, data access, config changes, policy changes. (C7 Observability, C1 Correctness)
3. **P3 Retention is Policy, Not Storage** — data lifetime is declared and enforced; deletion at end-of-life is a feature, not a failure. (C5 Reversibility, C8 Economy of storage)
4. **P4 Policy is Code** — compliance policy is expressed in versioned, reviewable, testable code (OPA/Cedar/Kyverno), not in spreadsheets or prose. (C6 Composability, C2 Clarity)
5. **P5 Policy is Evaluated as a Gate** — policy violations block before the action, not after the audit; admission/CI/CD-time enforcement. (C1 Correctness, C5 Reversibility)
6. **P6 Evidence is Collected Continuously** — evidence of compliance (logs, configs, scans, attestations) is gathered as a byproduct of operation, not assembled manually at audit time. (C7 Observability, C3 Simplicity of audit)
7. **P7 Identity is Attributable** — every logged action traces to an authenticated principal; shared/generic identities are violations. (C1 Correctness via security, C7 Observability)
8. **P8 Subject Access is Honored** — data-subject rights (access, export, deletion) are operations with defined contracts and audit trails; not ad-hoc. (C1 Correctness, C5 Reversibility)
9. **P9 Secrets and Sensitive Data are Redacted in Audit** — audit logs themselves must not leak secrets; redaction is structural, not opportunistic. (C1 Correctness via security, C3 Simplicity)
10. **P10 Compliance Posture is Observable** — the system reports its own compliance state (drift from policy, open violations, retention status); silent non-compliance is the bug. (C7 Observability, C1 Correctness)
### Derived Docs
- `audit-logs.md` — append-only log patterns, structured audit events, CloudTrail/Cloud-Audit-Log conventions, queryability, retention of logs themselves, cross-link observability/logging + security/authorization.
- `data-retention.md` — retention policies as code, lifecycle rules, deletion as a feature, GDPR/CCPA abstracted, retention vs. backup distinction, cross-link data/migrations.
- `policy-as-code.md` — OPA/Cedar/Sentinel/Kyverno patterns, policy as a CI/CD + admission gate, policy testing, versioning policy, cross-link infrastructure-as-code (declarative intent) + kubernetes (admission).
- `evidence.md` — evidence collection as a byproduct, signed attestations (Cosign/in-toto), audit-ready export, provenance, cross-link security/supply-chain + observability/metrics.
### Cross-Domain Links (one-directional in v0.3)
- `security/authorization` ← compliance P7 (attributable identity)
- `security/secrets` ← compliance P9 (redaction)
- `security/supply-chain` ← compliance P6, evidence.md (signed attestations)
- `observability/logging` ← compliance P1, P2 (audit logs = structured logging)
- `observability/metrics` ← compliance P10 (compliance posture metrics)
- `observability/tracing` ← compliance P6 (evidence from distributed traces)
- `data/schema-design` ← compliance P3 (retention schema)
- `data/migrations` ← compliance P3 (retention migration discipline)
- `infrastructure-as-code/P1 Declarative Intent` ← compliance P4 (policy-as-code)
- `infrastructure-as-code/P3 State is Truth` ← compliance P10 (compliance posture truth)
- `kubernetes/rbac` ← compliance P7 (audit subject identity)
- `devops/P6 Configuration as Code` ← compliance P4 (policy as code)
## Architectural Fit (v0.1/v0.2 Contract Preservation)
- **Hierarchy preserved:** all four new domains depend on `core/`; their P-rules trace to C1C8 via the matrix. No lateral authority.
- **10 P-rules per domain** (per D-018, D-030, D-026): consistent with v0.1 (11 domains) and v0.2 (2 domains). v0.3 adds 40 new P-rules → matrix grows 130 → 170.
- **Manifest authoritative:** all new documents added to `MANIFEST.md` in P4. Unlisted = not part of the framework.
- **No runtime code** (per D-020, PROJECT.md constraint): examples are illustrative markdown with code fences only. No `.yaml` manifests, `.po` resource files, model artifacts, policy `.rego` files, or deployable artifacts as standalone files — only fenced code blocks inside `.md` files.
- **Conflict resolution unchanged:** matrix extended, not replaced. Core precedence (C1 > C2 > ... > C8) governs any new vs existing rule conflict. Compliance rules tracing to C1 (Correctness) inherit C1's non-tradeable status where they overlap with security (per core/conflict-resolution.md §6).
- **Cross-links one-directional** (D-026 extended): new domains link outward to existing; existing domains unchanged in v0.3 (no back-link edits to v0.1/v0.2 content).
## Prior Art Position (v0.3 extension)
Existing GitOps/AI-ML/i18n/compliance guidance (OpenGitOps principles, ArgoCD/Flux docs, Operator pattern, MLOps maturity models, ICU/CLDR, W3C i18n, NIST/SOC 2, OPA/Kyverno) state practices and controls but none map every domain rule back to a small set of universal core principles. Atelier's v0.3 contribution is the same differentiation as v0.1 and v0.2: **traceable principle hierarchy with a join table**. The four new domains add 40 P-rules, each traced to ≥1 core C-rule, extending the matrix from 130 to 170 domain principles across 13 → 17 domains.
## v0.3 Persona Assessment
See `.ciagent/atelier/PERSONAS.md` for the updated roster. v0.3 adds two phase-specific personas (per D-019, D-020):
- **platform-engineer** (phase-specific, extended from v0.2): domain = infrastructure/platform-automation; territory = `domains/gitops-operators/**`, gitops examples; constraints add "source-of-truth is git" and "reconciliation loop is the primitive"; active for P1 GitOps/Operators only; removed after v0.3 completes.
- **ml-engineer** (phase-specific, new): domain = machine-learning engineering; territory = `domains/ai-ml/**`, `examples/good/ai-ml-reproducibility.md`; constraints = ["reproducibility is non-negotiable", "data lineage is traceable", "trace to core", "10 P-rules per domain", "no runtime code", "engineering discipline not algorithm design (D-023)"]; active for P2 AI/ML only; removed after v0.3 completes.
- **i18n (P3) + compliance (P3)** covered by tech-writer + domain-expert (D-022 — no new personas; both domains are smaller-surface and within the existing personas' competence).
## v0.3 Risks and Mitigations
| Risk | Mitigation |
|------|-----------|
| New P-rules orphaned from core (no matrix trace) | P4 extends matrix; domain-expert persona verifies every new P-rule traces to ≥1 C-rule before sign-off |
| GitOps-operators overlaps kubernetes/infrastructure-as-code (declarative, state, drift) | Cross-links one-directional (D-026); each domain owns its angle (k8s P1 desired-state vs gitops P1 git-as-source-of-truth vs iac P3 state-is-truth) |
| AI/ML domain drifts into algorithm/model-design (out of scope per D-023) | ml-engineer persona constraint "engineering discipline not algorithm design"; review/agent-checklist gains an ai-ml scope check in P4 |
| Compliance domain bloats into regulation-specific docs (GDPR/SOC2) | D-024 framework-agnostic; review check in P4 rejects regulation-specific content |
| i18n and compliance overlap on "retention of locale data" | Each owns its angle: i18n P10 (translation versioning) vs compliance P3 (data retention policy) |
| Examples become runtime artifacts (model files, .rego, .po) | persona constraints "no runtime code"; examples are markdown with fenced code only; P5 review check |
| Persona explosion (5 active in v0.3) | Both new personas are phase-specific and removed post-milestone; roster returns to 3 |
| Matrix row-count verification (40 new mappings, 10 per domain) | D-026 review check: row count per domain = 10, each row ≥1 C-rule, executed in P4 |
## v0.3 Conclusions
1. Four new top-level domains extend the framework without breaking the v0.1/v0.2 contract.
2. 40 new P-rules (10 per domain) all trace to core C1C8 — matrix extends from 130 to 170 across 13 → 17 domains.
3. GitOps-operators unifies ArgoCD/Flux/Operators/Progressive Delivery under the shared declarative-source-of-truth reconciliation loop (D-021) — splitting would fragment the P-rules.
4. AI/ML is scoped to engineering discipline (D-023): data versioning, evaluation, serving, drift — NOT algorithm design. Reproducibility and lineage are the non-negotiables.
5. i18n is grounded in ICU/CLDR + BCP 47 + W3C i18n; the source language is a locale, not a default.
6. Compliance is framework-agnostic (D-024): audit/retention/policy-as-code/evidence abstract NIST/SOC2/GDPR into principles that derive from core Security/Correctness/Observability.
7. Two phase-specific personas (platform-engineer extended, ml-engineer added); both removed post-v0.3.
8. No runtime code; examples are illustrative markdown only.
+68
View File
@@ -0,0 +1,68 @@
# Atelier — v0.2 Final Review + Audit (P5)
> Generated during final phase P5 (REVIEW + AUDIT) of milestone v0.2. Per run.md FINAL PHASE.
## Review (multi-persona, ci-code-reviewer)
**Scope:** all v0.2 changes (32 files, +1787/-34 lines), all commits `main..atelier/phase/05-final-review-ship`.
### P0 checks (all pass)
1. ✅ IaC first-principles: exactly 10 P-rules (P1P10)
2. ✅ K8s first-principles: exactly 10 P-rules (P1P10)
3. ✅ Matrix IaC section: 10 rows, each ≥1 valid C-rule, no orphans
4. ✅ Matrix K8s section: 10 rows, each ≥1 valid C-rule, no orphans
5. ✅ Every P-rule name in first-principles matches its matrix row
6. ✅ All required files exist (ATELIER-36..56, 59): 2 first-principles + 4 IaC derived + 6 k8s derived + 4 examples + matrix/manifest/review extensions
7. ✅ MANIFEST lists both new domains with all derived docs — no manifest drift
8. ✅ No standalone .tf/.yaml/.yml files — "no runtime code" constraint preserved (all code is fenced in .md)
9. ✅ No hardcoded real secrets — bad examples use AWS doc placeholders and `hunter2`; good examples use `registry.example.com` + Secret refs
10. ✅ anti-patterns.md covers secrets-in-HCL and cluster-admin
11. ✅ rbac.md covers Pod Security Standards + Admission
12. ✅ All cross-link targets resolve to MANIFEST-listed docs
**Verdict: PASS — No P0 issues.**
### P1+ issues (flagged, then fixed in this phase per run.md)
| ID | Severity | Issue | Fix applied |
|----|----------|-------|-------------|
| REV-1 | P1 | 5 derived docs missing cross-domain links (terraform, state, modules, workloads, networking) | Added cross-links to `domains/security/secrets.md`, `domains/devops/first-principles.md`, `domains/observability/metrics.md`, `domains/security/authorization.md` |
| REV-2 | P2 | networking.md "Dual-Stack (P4 Locality)" — wrong P-rule label | Corrected to "(C4 Locality)" |
| REV-3 | P2 | domain-coverage.md "Concurrency broadest (7)" stale — IaC + k8s also 7 | Updated to "Concurrency, IaC, Kubernetes tied (7 each)" |
All P1+ issues fixed in commit `87daca3`. No loop back to EXECUTE (per run.md final-phase rule).
## Audit (lead-developer)
### 1. Reconstruction test
- git log `main..atelier/phase/05-final-review-ship` shows 10 v0.2 commits (P00 complete, P01 execute/verify/complete/status, P02, P03, P04, P05 review fix).
- REQUIREMENTS.md status (covered) matches shipped phases: ATELIER-36..56, 59 all `covered`; ATELIER-57, 58 `pending` (final phase, completed at ship).
- ROADMAP.md phase statuses match: P0P4 `complete`, P5 `pending` (→ complete at ship).
- ✅ Reconstruction passes.
### 2. Branch hygiene
- Active branches: `atelier/milestone/v0.2-iac-k8s`, `atelier/phase/05-final-review-ship`.
- All execution phase branches (0004) deleted after ship. ✅
### 3. Commit discipline
- All 10 v0.2 commits contain `---ci---` blocks with project, phase, milestone, status, requirements. ✅
### 4. File discipline
- `.ciagent/atelier/` contains all 8 required files: PROJECT, ROADMAP, REQUIREMENTS, ARCHITECTURE, PERSONAS, PLAN, RESEARCH, CLARIFY. ✅
- (v0.1 legacy AUDIT-P2.md, REVIEW-P7.md also present — not removed, harmless.)
### 5. Tag sequence
- v0.0.0v0.0.7 (milestone v0.1) → v0.1.0v0.1.4 (milestone v0.2 phases 04).
- All v0.1.x strictly > v0.0.7. All v0.1.x strictly increasing. ✅
- Final phase tag will be v0.1.5 (next patch, IS the v0.2 milestone release per NFR rule).
### 6. Manifest discipline
- All 12 new domain docs (infrastructure-as-code/*, kubernetes/*) listed in MANIFEST.md Domains table. ✅
- matrix, review (agent-checklist, peer-review-checklist, anti-patterns) all listed in Cross-Cutting. ✅
**Audit verdict: CLEAN — no critical issues.**
## Conclusion
Review PASS (no P0, all P1+ fixed). Audit CLEAN. Milestone v0.2 is ready to ship as v0.1.5.
+69
View File
@@ -0,0 +1,69 @@
# Atelier — Final Review + Audit (P7)
> Final phase review and audit for milestone v0.1. Conducted before milestone ship.
## Review (Multi-Persona, across all phases)
### Structural Review
- **All 64 MANIFEST-listed documents exist:**
- 3 core, 11 domain first-principles, 27 derived, 2 matrix, 3 review, 4 good examples, 3 bad examples, 4 languages, 5 meta (README, LICENSE, CHANGELOG, CONTRIBUTING, MANIFEST)
- **No unlisted docs:** the framework tree contains only docs in the manifest (plus `.ciagent/` governance, which is meta, not framework content).
### Behavioral Review
- **Every domain has exactly 10 P-rules** (verified per domain: api 10, security 10, data 10, testing 10, performance 10, observability 10, errors 10, documentation 10, concurrency 10, devops 10, uiux 10).
- **Matrix has exactly 110 rows** (11 domains × 10 principles).
- **Every matrix row maps to a C-rule** that exists in `core/first-principles.md` (C1C8 all present).
- **Every example cites principles** (good: 1018 citations; bad: 711 citations).
### Security Review
- **No secret in git history:** the GITEA_API_TOKEN value does not appear in any committed file or commit message. `.ciagent/.env.secrets` is gitignored and never staged.
- **No security anti-patterns in framework content:** the `review/anti-patterns.md` catalog is complete; examples/bad/* cite the security principles they violate.
### Quality Review
- **Document structure consistent:** all first-principles docs follow the template (Manifesto → Principles → Conflict Resolution → What Violates → Relationship to Core).
- **Reading order links resolve:** `core/reading-order.md` forward references to P2P6 docs now resolve (all created).
- **CHANGELOG follows Keep a Changelog format:** Added section, phases enumerated.
### P1+ Issues Found (post-hoc, not blocking)
1. **Squash-merge commits lack `---ci---` blocks:** the 7 `docs(P0N): complete ...` commits (consolidation points) and the initial `chore: initialize` commit do not have `---ci---` blocks. The task commits on phase branches all have them. Per commit-discipline, consolidation commits could include a `---ci---` block. This is a P1 (post-hoc) issue, not blocking. The milestone ship commit below includes a comprehensive `---ci---` block covering the milestone.
**P0 fixes applied:** none required. No blocking issues found.
## Audit
### Reconstruction Test
- **MANIFEST.md lists all framework documents:** ✓ (64 docs)
- **Every listed document exists:**
- **Git log reconstructs project state:** the `---ci---` blocks in task commits record phase, milestone, status, and requirements covered. The git log + `.ciagent/` files reconstruct the full project state.
### Branch Hygiene
- **Before ship:** `main`, `milestone/v0.1-atelier`, 7 phase branches (0006), `phase/07-final-review-ship` (current).
- **After ship:** all phase branches deleted; `main` + `milestone/v0.1-atelier` remain briefly, then milestone branch deleted after merge to main. Tags preserve all history.
### Commit Discipline
- **Task commits** (on phase branches): all have `---ci---` blocks with project, phase, milestone, status, requirements. ✓
- **Squash-merge commits** (on milestone branch): consolidation commits without `---ci---` blocks. P1 (post-hoc).
- **Init commit**: has full `---ci---` block. ✓
### File Discipline
- `.ciagent/` holds only governance files (config, PROJECT, ROADMAP, REQUIREMENTS, ARCHITECTURE, PERSONAS, PLAN, RESEARCH, CLARIFY, AUDIT-P2, CHECKPOINT). ✓
- Framework content is in the repo root (`core/`, `domains/`, etc.). ✓
- `.env.secrets` is gitignored, never committed. ✓
## Milestone Ship Checklist
- [x] All execution phases (P1P6) shipped (v0.0.1v0.0.6)
- [x] Review passed (structural, behavioral, security, quality)
- [x] Audit clean (reconstruction, branch hygiene, file discipline)
- [x] REQUIREMENTS.md updated (all 35 requirements covered)
- [x] ROADMAP.md updated (milestone complete)
- [ ] Merge `phase/07``milestone/v0.1-atelier`
- [ ] Merge `milestone/v0.1-atelier``main`
- [ ] Tag `v0.0.7` (IS the v0.1 milestone release)
- [ ] Create Gitea release for v0.0.7
- [ ] Delete all phase branches + milestone branch
- [ ] Clear checkpoint (milestone complete)
## Conclusion
The milestone v0.1 is complete and ready to ship. 35/35 requirements covered. 8 core principles, 11 domains, 110 domain principles, all traced via the matrix. The framework is internally consistent and ready for consumption.
+169
View File
@@ -0,0 +1,169 @@
# Atelier — Roadmap
## Milestone: v0.1 — Initial Framework (COMPLETE)
**Milestone type:** NFR (all phases produce docs/chore commits — no `feat` code)
**Tag line:** v0.0.x (previous minor from v0.1)
**Phases:** P0 (pre-execution) + P1P6 (execution) + P7 (final review+ship)
| Phase | Name | Type | Status | Key Deliverables |
|-------|------|------|--------|------------------|
| 0 | Pre-Execution | docs | complete | Spec, clarify, research, plan, PERSONAS.md |
| 1 | Core Foundation | docs | complete | core/first-principles.md, core/conflict-resolution.md, core/reading-order.md, README.md, LICENSE |
| 2 | Domain First Principles | docs | complete | uiux/first-principles.md, domain audit |
| 3 | Domain Derived Docs | docs | complete | 27 derived docs across 11 domains |
| 4 | Matrix + Review | docs | complete | matrix/domain-coverage.md, review/{agent,peer-review,anti-patterns}.md |
| 5 | Examples | docs | complete | 4 good + 3 bad examples |
| 6 | Languages + Meta | docs | complete | 4 language docs, CHANGELOG, CONTRIBUTING |
| 7 | Final Review + Ship | docs | complete | Review passed, audit clean, milestone merged to main, tag v0.0.7 |
## Phase Tag Mapping
Per branch-strategy.md, milestone `v0.1` tags run on the `v0.0.x` patch line:
| Phase | Tag | Notes |
|-------|-----|-------|
| P0 | v0.0.0 | Pre-execution release |
| P1 | v0.0.1 | Core foundation |
| P2 | v0.0.2 | Domain first-principles |
| P3 | v0.0.3 | Domain derived docs |
| P4 | v0.0.4 | Matrix + review |
| P5 | v0.0.5 | Examples |
| P6 | v0.0.6 | Languages + meta |
| P7 | v0.0.7 | Final review + ship — **IS the v0.1 milestone release** |
NFR milestone: no separate minor tag. The final patch (v0.0.7) IS the v0.1 deliverable.
## Next
- Milestone v0.1 complete. All phases shipped.
- Future: v0.2 could add `domains/ai-ml/`, `domains/i18n/`, `domains/compliance/` per spec Part 6 next-steps.
## Milestone: v0.2 — Infrastructure as Code + Kubernetes (COMPLETE)
**Milestone type:** NFR (all phases produce docs — no `feat` code)
**Tag line:** v0.1.x (previous minor from v0.2)
**Phases:** P0 (pre-execution) + P1P4 (execution) + P5 (final review+ship)
| Phase | Name | Type | Status | Key Deliverables |
|-------|------|------|--------|------------------|
| 0 | Pre-Execution | docs | complete | Spec, clarify, research, ideate, plan, PERSONAS.md (adds platform-engineer persona) |
| 1 | Infrastructure as Code Domain | docs | complete | domains/infrastructure-as-code/{first-principles, terraform, opentofu, state, modules}.md |
| 2 | Kubernetes Domain | docs | complete | domains/kubernetes/{first-principles, workloads, networking, storage, rbac, helm, kustomize}.md |
| 3 | Matrix + Review Integration | docs | complete | matrix/principles-matrix.md (20 new mappings), matrix/domain-coverage.md, review/{agent-checklist, peer-review-checklist, anti-patterns}.md, MANIFEST.md |
| 4 | Examples + Cross-Links | docs | complete | examples/good/{terraform-module, k8s-deployment}.md, examples/bad/{terraform-unlocked-state, k8s-bare-pod-no-resources}.md, cross-links to devops/security/observability/data |
| 5 | Final Review + Ship | docs | complete | Review passed, audit clean, milestone merged to main, tag v0.1.5 |
## v0.2 Phase Tag Mapping
Per branch-strategy.md, milestone `v0.2` tags run on the `v0.1.x` patch line:
| Phase | Tag | Notes |
|-------|-----|-------|
| P0 | v0.1.0 | Pre-execution release |
| P1 | v0.1.1 | Infrastructure as Code domain |
| P2 | v0.1.2 | Kubernetes domain |
| P3 | v0.1.3 | Matrix + review integration |
| P4 | v0.1.4 | Examples + cross-links |
| P5 | v0.1.5 | Final review + ship — **IS the v0.2 milestone release** |
NFR milestone: no separate minor tag. The final patch (v0.1.5) IS the v0.2 deliverable.
## v0.2 Ideation Outcome
- 16 ideas generated (mechanical 9, backend-enriched 7)
- 14 accepted (all in v0.2 scope; refined ATELIER-36..58, added ATELIER-59)
- 2 deferred to v0.3 (GitOps/operators domain; ai-ml/i18n/compliance domains)
- See `.ciagent/atelier/REQUIREMENTS.md` "v0.2 Ideation Log" for the full table
## Milestone: v0.3 — GitOps + Operators + AI/ML + i18n + Compliance (ACTIVE)
**Milestone type:** NFR (all phases produce docs — no `feat` code)
**Tag line:** v0.2.x (previous minor from v0.3)
**Phases:** P0 (pre-execution) + P1P5 (execution) + P6 (final review+ship)
| Phase | Name | Type | Status | Key Deliverables |
|-------|------|------|--------|------------------|
| 0 | Pre-Execution | docs | complete | Spec, clarify, research, ideate, plan, PERSONAS.md (extends platform-engineer, adds ml-engineer) — shipped v0.2.0 |
| 1 | GitOps + Operators Domain | docs | complete | domains/gitops-operators/{first-principles, argocd, flux, operators, progressive-delivery}.md — shipped v0.2.1 |
| 2 | AI/ML Domain | docs | complete | domains/ai-ml/{first-principles, data-versioning, model-evaluation, serving, monitoring-drift}.md — shipped v0.2.2 |
| 3 | i18n + Compliance Domains | docs | complete | domains/i18n/{first-principles, locale-resources, formatting, rtl-bidi, testing-i18n}.md, domains/compliance/{first-principles, audit-logs, data-retention, policy-as-code, evidence}.md — shipped v0.2.3 |
| 4 | Matrix + Review Integration | docs | complete | matrix/principles-matrix.md (+40 mappings), matrix/domain-coverage.md (incl. C-rule coverage table update), review/{agent-checklist, peer-review-checklist, anti-patterns}.md, MANIFEST.md (+ examples/ listing per ATELIER-91) — shipped v0.2.4 |
| 5 | Examples + Cross-Links | docs | complete | examples/good + examples/bad for 4 domains, cross-links to devops/security/observability/data/k8s/iac — shipped v0.2.5 |
| 6 | Final Review + Ship | docs | complete | Review passed, audit clean, milestone merged to main, tag v0.2.6 |
## v0.3 Phase Tag Mapping
Per branch-strategy.md, milestone `v0.3` tags run on the `v0.2.x` patch line:
| Phase | Tag | Notes |
|-------|-----|-------|
| P0 | v0.2.0 | Pre-execution release |
| P1 | v0.2.1 | GitOps + Operators domain |
| P2 | v0.2.2 | AI/ML domain |
| P3 | v0.2.3 | i18n + Compliance domains |
| P4 | v0.2.4 | Matrix + review integration |
| P5 | v0.2.5 | Examples + cross-links |
| P6 | v0.2.6 | Final review + ship — **IS the v0.3 milestone release** |
NFR milestone: no separate minor tag. The final patch (v0.2.6) IS the v0.3 deliverable.
## v0.3 Ideation Outcome
- 14 ideas generated (mechanical 5, backend-enriched 7, within-project transfer 2 merged)
- 14 accepted (all v0.3-scope, confidence ≥ 0.78, above 0.6 autonomy threshold → auto-accepted)
- 1 new requirement added: ATELIER-91 (examples/ in MANIFEST — pre-existing drift from v0.2 audit escalation)
- 13 refinements to existing reqs ATELIER-61..90 (decision matrices, chaos anti-patterns, drift-type enumeration, etc.)
- 0 deferred to v0.4
- See `.ciagent/atelier/REQUIREMENTS.md` "v0.3 Ideation Log" for the full table
## Future Milestones
- **v0.4** (ACTIVE): `domains/edge/`, `domains/messaging/`, `languages/`-derived docs (typescript/python/go/rust → 4 derived docs each).
- **v0.5** (candidates): `domains/quantum/`, tooling adapters (linters), translation/localization of framework docs.
## Milestone: v0.4 — Edge + Messaging + Language-Derived Docs (ACTIVE)
**Milestone type:** NFR (all phases produce docs — no `feat` code)
**Tag line:** v0.3.x (previous minor from v0.4)
**Phases:** P0 (pre-execution) + P1P5 (execution) + P6 (final review+ship)
| Phase | Name | Type | Status | Key Deliverables |
|-------|------|------|--------|------------------|
| 0 | Pre-Execution | docs | complete | Spec, clarify, research, ideate, plan, PERSONAS.md (adds edge-engineer + languages-engineer phase-specific personas) — shipped v0.3.0 |
| 1 | Edge Domain | docs | complete | domains/edge/{first-principles, cdn, offline-first, iot, sync}.md — shipped v0.3.1 |
| 2 | Messaging Domain | docs | pending | domains/messaging/{first-principles, queues, pubsub, streams, delivery-semantics}.md |
| 3 | Language-Derived Extension | docs | pending | languages/ × 4 → first-principles + 4 derived docs each (16 derived docs) |
| 4 | Matrix + Review Integration | docs | pending | matrix/principles-matrix.md (+20 mappings, 170→190), matrix/domain-coverage.md (+ languages/ sub-table), review/{agent-checklist, peer-review-checklist, anti-patterns}.md, MANIFEST.md (languages/ section) |
| 5 | Examples + Cross-Links | docs | pending | examples/good + examples/bad for 2 domains, cross-links to devops/observability/data/concurrency/kubernetes/infrastructure-as-code + language→domain links |
| 6 | Final Review + Ship | docs | pending | Review passed, audit clean, milestone merged to main, tag v0.3.6 |
## v0.4 Phase Tag Mapping
Per branch-strategy.md, milestone `v0.4` tags run on the `v0.3.x` patch line:
| Phase | Tag | Notes |
|-------|-----|-------|
| P0 | v0.3.0 | Pre-execution release |
| P1 | v0.3.1 | Edge domain |
| P2 | v0.3.2 | Messaging domain |
| P3 | v0.3.3 | Language-derived extension |
| P4 | v0.3.4 | Matrix + review integration |
| P5 | v0.3.5 | Examples + cross-links |
| P6 | v0.3.6 | Final review + ship — **IS the v0.4 milestone release** |
NFR milestone: no separate minor tag. The final patch (v0.3.6) IS the v0.4 deliverable.
## v0.4 Ideation Outcome
(Populated during P0 IDEATE stage.)
## Success Criteria
- [x] All 11 domains have first-principles.md
- [x] Every domain P-rule traced to a core C-rule in matrix/principles-matrix.md
- [x] MANIFEST.md lists all framework documents
- [x] review/agent-checklist.md covers all core principles
- [x] examples/ includes at least 4 good + 3 bad worked examples
- [x] README.md provides quickstart for agents and humans
- [x] Milestone v0.1 tagged and released
+70
View File
@@ -0,0 +1,70 @@
{
"projects": [
{
"slug": "atelier",
"name": "Atelier",
"milestone": "v0.4",
"status": "active"
}
],
"active_project": "atelier",
"active_projects": ["atelier"],
"autonomy": {
"level": "full",
"escalation_hooks": ["deploy", "delete_data", "merge_to_main"],
"clarify_budget": 10,
"decision_confidence_threshold": 0.6,
"max_revision_iterations": 3,
"max_verification_retries": 2,
"escalation_timeout_ms": 300000
},
"model_profile": "quality",
"parallelization": {
"enabled": true,
"max_concurrent_agents": 5,
"min_plans_for_parallel": 2,
"max_concurrent_projects": 3
},
"verification": {
"automated_only": true,
"escalate_visual": true,
"escalate_external_integration": true,
"test_first": false
},
"security": {
"auto_accept_low_severity": true,
"auto_mitigate_medium_severity": true,
"escalate_high_severity": true
},
"git": {
"branching_strategy": "phase",
"auto_commit": true,
"auto_push": true
},
"sessions": {
"max_concurrent_sessions": 3,
"session_timeout_ms": 3600000,
"session_isolation": "branch"
},
"personas": {
"enabled": true,
"territory_enforcement": "warn"
},
"release": {
"forge": "gitea",
"base_url": "https://git.cloudinit.dev",
"owner": "coreci",
"repo": "atelier"
},
"gitea": {
"base_url": "https://git.cloudinit.dev",
"api_token_env": "GITEA_API_TOKEN",
"owner": "coreci",
"repo": "atelier"
},
"ship": {
"per_phase": true,
"allow_skip": false,
"max_release_retries": 3
}
}
+3
View File
@@ -0,0 +1,3 @@
.env
.env.secrets
.env.*
+45
View File
@@ -0,0 +1,45 @@
# Changelog
All notable changes to the Atelier framework are documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
## [0.1.0] — 2026-08-05
### Added
- Eight core principles (C1C8) in `core/first-principles.md` with definitions, precedence, and violation tables.
- Formal conflict resolution procedure in `core/conflict-resolution.md` (hierarchy, precedence, worked examples, non-tradeable declarations).
- Canonical reading order in `core/reading-order.md` with paths for agents, humans, conflict resolution, and review.
- README with quickstart for AI agents and humans.
- MIT license.
- Eleven domain first-principles (P1P10 each): UI/UX, API, Security, Data, Testing, Performance, Observability, Errors, Documentation, Concurrency, DevOps.
- Twenty-seven domain derived/topic docs (REST, GraphQL, versioning, error-responses, pagination, authentication, authorization, input-validation, secrets, supply-chain, schema-design, migrations, indexing, pyramid, fixtures, frontend, backend, logging, metrics, tracing, tokens, copywriting, patterns, doc-templates, patterns, ci-cd, environments).
- `matrix/principles-matrix.md` — full mapping of all 110 domain principles to core derivations.
- `matrix/domain-coverage.md` — inverse mapping of core principles to domains.
- `review/agent-checklist.md` — pre-completion gate for AI agents.
- `review/peer-review-checklist.md` — human review checklist.
- `review/anti-patterns.md` — catalog of violations with principle citations.
- Four good worked examples: `examples/good/api-endpoint.md`, `react-component.md`, `db-schema.md`, `error-handler.md`.
- Three bad worked examples: `examples/bad/god-object.md`, `silent-error.md`, `leaky-abstraction.md`.
- Four language application docs: `languages/typescript.md`, `python.md`, `go.md`, `rust.md`.
- `MANIFEST.md` — authoritative document index.
- `CONTRIBUTING.md` — contribution guide.
### Framework Properties
- 8 core principles, 11 domains, 110 domain principles, all traced via the matrix.
- NFR milestone type (documentation only, no runtime code).
- Tags: v0.0.0 (pre-execution) through v0.0.7 (final review + ship).
- The v0.0.7 patch release IS the v0.1.0 milestone deliverable.
### Phases
- Phase 0 (v0.0.0): pre-execution — specify, clarify, research, plan, ship.
- Phase 1 (v0.0.1): core foundation — first-principles, conflict-resolution, reading-order, README, LICENSE.
- Phase 2 (v0.0.2): domain first-principles — uiux first-principles + audit.
- Phase 3 (v0.0.3): domain derived docs — 27 topic files.
- Phase 4 (v0.0.4): matrix + review — domain-coverage, agent-checklist, peer-review-checklist, anti-patterns.
- Phase 5 (v0.0.5): examples — 4 good + 3 bad.
- Phase 6 (v0.0.6): languages + meta — typescript, python, go, rust, CHANGELOG, CONTRIBUTING.
- Phase 7 (v0.0.7): final review + ship — milestone release.
+71
View File
@@ -0,0 +1,71 @@
# Contributing to Atelier
Thank you for considering a contribution to Atelier. This framework lives by its principles; contributions are expected to follow them.
## What We Accept
- **New domain first-principles** — if a domain is missing (e.g., `domains/ai-ml/`), propose it with 10 principles (P1P10), each traced to a core principle (C1C8) in `matrix/principles-matrix.md`.
- **New domain derived docs** — topic docs under an existing domain (e.g., `domains/api/webhooks.md`), deriving from the domain's first-principles.
- **New language application docs**`languages/<lang>.md` showing how domain principles apply in a specific language.
- **New examples**`examples/good/*` (with principle citations) or `examples/bad/*` (with violation citations).
- **Corrections** — to existing principles, derivations, or examples. A correction to a core principle is a major version change; treat with care.
- **Improvements to the matrix** — if a derivation is missing or wrong, propose the fix with the rationale.
## What We Do Not Accept
- **Style rules** — Atelier is principles, not style. Use a linter for style.
- **Tooling** — linters, analyzers, or enforcement code. Atelier is markdown.
- **Unlisted docs** — every document must be in `MANIFEST.md`. An unlisted doc is not part of the framework.
- **Principles without derivation** — a domain principle that does not trace to a core principle is orphaned and will be rejected.
## How to Contribute
### 1. Read the relevant docs first
- `core/first-principles.md` — the eight axioms.
- `core/conflict-resolution.md` — how conflicts are resolved.
- The domain(s) you are contributing to.
- `matrix/principles-matrix.md` — to see existing derivations.
### 2. Follow the document structure
See `domains/documentation/doc-templates.md` for the canonical structure. Every first-principles doc has:
- Manifesto
- The Principles (P1P10, named, defined, with "what violates it")
- Conflict Resolution
- What Violates These Principles (table)
- Relationship to Core
### 3. Update the matrix
If you add or change a principle, update `matrix/principles-matrix.md` with the derivation. A PR with a new principle but no matrix row is incomplete.
### 4. Update the manifest
If you add a new document, add it to `MANIFEST.md` in the correct section. An unlisted document is not part of the framework.
### 5. Add examples
If you add a principle, add at least one good example and one bad example in `examples/`. Examples are mandatory (Documentation P3).
### 6. Write a clear PR description
- What principle or document you are adding/changing.
- Why (the rationale, not just the what — Documentation P8 Why Over What).
- Which core principle(s) it derives from.
- What conflicts it might introduce (if any).
## Review Criteria
Reviewers will check (see `review/peer-review-checklist.md`):
- Does the new principle trace to a core principle?
- Is the matrix updated?
- Is the manifest updated?
- Are there examples?
- Does the structure follow the template?
- Does it conflict with existing principles? If so, is the conflict resolvable per `core/conflict-resolution.md`?
## Versioning
- A new domain or language doc is a minor version (e.g., v0.1 → v0.2).
- A new topic doc or example is a patch version (e.g., v0.1.0 → v0.1.1).
- A change to `core/first-principles.md` (adding, removing, or reordering a core principle) is a major version (e.g., v0.x → v1.0).
- See `CHANGELOG.md` for the version history.
## License
By contributing, you agree that your contributions are licensed under the MIT license (see `LICENSE`).
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Atelier contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+78
View File
@@ -0,0 +1,78 @@
# Atelier — Document Manifest
> Authoritative index. If a document is not listed here, it is not
> part of the framework.
## Reading Order
1. `core/first-principles.md` — Required first
2. `core/conflict-resolution.md` — Required
3. `domains/<relevant>/first-principles.md`— Per task
4. `domains/<relevant>/<topic>.md` — As needed
5. `matrix/principles-matrix.md` — Reference
## Core
| Document | Status | Audience |
|-----------------------------------------|----------|----------|
| `core/first-principles.md` | Stable | All |
| `core/conflict-resolution.md` | Stable | All |
| `core/reading-order.md` | Stable | All |
## Domains
| Domain | First Principles | Derived Documents |
|---------------|------------------|----------------------------------|
| UI / UX | ✓ | components, accessibility, tokens, copywriting |
| API Design | ✓ | rest, graphql, versioning, error-responses, pagination |
| Security | ✓ | authentication, authorization, input-validation, secrets, supply-chain |
| Data | ✓ | schema-design, migrations, indexing |
| Testing | ✓ | pyramid, fixtures |
| Performance | ✓ | frontend, backend |
| Observability | ✓ | logging, metrics, tracing |
| Errors | ✓ | patterns |
| Documentation | ✓ | doc-templates |
| Concurrency | ✓ | patterns |
| DevOps | ✓ | ci-cd, environments |
| Infrastructure as Code | ✓ | terraform, opentofu, state, modules |
| Kubernetes | ✓ | workloads, networking, storage, rbac, helm, kustomize |
| GitOps + Operators | ✓ | argocd, flux, operators, progressive-delivery |
| AI / ML | ✓ | data-versioning, model-evaluation, serving, monitoring-drift |
| i18n | ✓ | locale-resources, formatting, rtl-bidi, testing-i18n |
| Compliance | ✓ | audit-logs, data-retention, policy-as-code, evidence |
## Examples
> Examples are illustrative markdown with fenced code only (no standalone runtime artifacts per D-020 / D-025). The `examples/` directory listing closes the v0.2 ESC-002 drift (IDEATE-17, ATELIER-91). P5 authored the v0.3 examples and promoted all entries from `pending` to `✓` (verified — every listed file exists).
| Path | Status | Notes |
|------|--------|-------|
| `examples/good/` | ✓ | Good-example directory — 8 examples (v0.1 + v0.2 + v0.3) |
| `examples/bad/` | ✓ | Bad-example directory — 7 examples (v0.1 + v0.2 + v0.3) |
| `examples/good/api-endpoint.md` | ✓ | v0.1 example — good REST endpoint |
| `examples/good/react-component.md` | ✓ | v0.1 example — good React component |
| `examples/good/db-schema.md` | ✓ | v0.1 example — good DB schema |
| `examples/good/error-handler.md` | ✓ | v0.1 example — good error handler |
| `examples/bad/god-object.md` | ✓ | v0.1 example — bad god object |
| `examples/bad/silent-error.md` | ✓ | v0.1 example — bad silent error |
| `examples/bad/leaky-abstraction.md` | ✓ | v0.1 example — bad leaky abstraction |
| `examples/good/terraform-module.md` | ✓ | v0.2 example — good IaC module |
| `examples/good/k8s-deployment.md` | ✓ | v0.2 example — good k8s deployment |
| `examples/bad/terraform-unlocked-state.md` | ✓ | v0.2 example — bad unlocked state |
| `examples/bad/k8s-bare-pod-no-resources.md` | ✓ | v0.2 example — bad bare pod |
| `examples/good/gitops-pr.md` | ✓ | v0.3 example — good GitOps PR |
| `examples/good/ai-ml-reproducibility.md` | ✓ | v0.3 example — good reproducible training run |
| `examples/bad/i18n-string-concat.md` | ✓ | v0.3 example — bad i18n string concat |
| `examples/bad/compliance-audit-log.md` | ✓ | v0.3 example — bad audit log (P1 + P9 breaches) |
> **Note:** The `examples/` section was established in P4 with entries pre-listed as `pending P5`. P5 authored the 4 v0.3 examples and promoted all entries to `✓` after verifying every listed file exists on disk. The manifest remains authoritative — unlisted = not part of framework.
## Cross-Cutting
| Document | Purpose |
|-----------------------------------|----------------------------------|
| `matrix/principles-matrix.md` | Maps domain → core principles (17 domains, 170 P-rules post-v0.3) |
| `matrix/domain-coverage.md` | Maps core → domains; per-domain coverage (incl. v0.3 Core Principle Coverage) |
| `review/agent-checklist.md` | Pre-completion agent checklist (incl. IaC + k8s + gitops + ai-ml + i18n + compliance triggers) |
| `review/peer-review-checklist.md` | Human peer-review checklist (incl. IaC + k8s + gitops + ai-ml + i18n + compliance sections) |
| `review/anti-patterns.md` | Catalog of violations (incl. IaC + k8s + gitops + ai-ml + i18n + compliance + v0.3 chaos anti-patterns) |
+90
View File
@@ -0,0 +1,90 @@
# Atelier
A first-principles, docs-as-code engineering framework for AI agents and humans.
## What This Is
Atelier is a hierarchical document tree of engineering principles. Eight universal **core principles** (C1C8) govern eleven **domains** (API, Security, Data, Testing, Performance, Observability, Errors, Documentation, Concurrency, DevOps, UI/UX), each with ten derived principles (P1P10). A **principles matrix** maps every domain rule back to the core rule it derives from, making conflict resolution deterministic.
This is not a style guide. This is not a linter. This is a principle hierarchy with cross-references, consumed by AI agents as pre-completion guidance and by humans as engineering canon.
## Who It's For
- **AI agents** — read `core/` then the relevant `domains/` before completing a coding task. Run `review/agent-checklist.md` before finishing.
- **Human engineers** — read `README.md``MANIFEST.md``core/` for onboarding, then the domains relevant to your work.
- **Reviewers** — use `review/peer-review-checklist.md` and `review/anti-patterns.md`.
## Quickstart
### For Agents
1. Read `core/first-principles.md` (the 8 axioms)
2. Read `core/conflict-resolution.md` (how to resolve rule conflicts)
3. Read `domains/<your-task-domain>/first-principles.md`
4. Read `domains/<your-task-domain>/<topic>.md` as needed
5. Run `review/agent-checklist.md` before completing
### For Humans
1. Read this README
2. Read `MANIFEST.md` (the authoritative document index)
3. Read `core/first-principles.md``core/conflict-resolution.md``core/reading-order.md`
4. Pick a domain of interest and read its `first-principles.md`
5. Skim `matrix/principles-matrix.md` to see how everything connects
## The Eight Core Principles
| # | Principle | One-line |
|---|-----------|----------|
| C1 | Correctness | The system does what it is supposed to do, and nothing else. |
| C2 | Clarity | The intent of the code is obvious to its reader. |
| C3 | Simplicity | The solution is as simple as possible, and no simpler. |
| C4 | Locality | Decisions and their consequences live near each other. |
| C5 | Reversibility | Every decision can be undone, and the cost of undoing is known. |
| C6 | Composability | Parts combine into wholes, and the parts are reusable. |
| C7 | Observability | The system's behavior is visible to those who must understand it. |
| C8 | Economy | The system uses no more resources than the task requires. |
Precedence: C1 > C2 > C3 > C4 > C5 > C6 > C7 > C8. Correctness is never sacrificed.
## The Domains
| Domain | First Principles | Derived Docs |
|--------|------------------|--------------|
| UI/UX | ✓ | components, accessibility, tokens, copywriting |
| API | ✓ | rest, graphql, versioning, error-responses, pagination |
| Security | ✓ | authentication, authorization, input-validation, secrets, supply-chain |
| Data | ✓ | schema-design, migrations, indexing |
| Testing | ✓ | pyramid, fixtures |
| Performance | ✓ | frontend, backend |
| Observability | ✓ | logging, metrics, tracing |
| Errors | ✓ | patterns |
| Documentation | ✓ | doc-templates |
| Concurrency | ✓ | patterns |
| DevOps | ✓ | ci-cd, environments |
## Repository Structure
```
atelier/
├── core/ # The 8 universal principles (read first)
├── domains/ # 11 domains, each with first-principles + derived docs
├── languages/ # Language-specific application of domain rules
├── review/ # Checklists and anti-patterns
├── matrix/ # Cross-reference: domain ↔ core
├── examples/ # Worked examples (good + bad)
├── MANIFEST.md # Authoritative document index
└── README.md # This file
```
## License
MIT. See `LICENSE`.
## Contributing
See `CONTRIBUTING.md`.
## Version
This is v0.1. See `CHANGELOG.md` for version history.
+105
View File
@@ -0,0 +1,105 @@
# Core — Conflict Resolution
**Status:** Foundational
**Audience:** All agents and humans. Read after `first-principles.md`.
## 1. Why Conflict Resolution Exists
Atelier has eight core principles and eleven domains, each with ten principles. Rules will conflict. When they do, there must be a deterministic procedure to resolve the conflict — not a debate. This document is that procedure.
## 2. The Hierarchy (Strict)
```
core/first-principles.md ← highest authority
└── domains/<x>/first-principles.md
└── domains/<x>/<topic>.md
└── languages/<lang>.md
└── examples/<x>.md ← lowest authority
```
**Rules:**
1. A higher document overrides a lower document when they conflict.
2. `core/first-principles.md` is never overridden.
3. Within a level, the more specific document wins for its specific scope.
4. If two documents at the same level conflict with equal specificity, the core principle they each derive from decides — the higher-precedence core principle wins.
## 3. Core Principle Precedence (Total Order)
```
C1 Correctness > C2 Clarity > C3 Simplicity > C4 Locality >
C5 Reversibility > C6 Composability > C7 Observability > C8 Economy
```
- **C1 is never sacrificed.** No other principle, domain rule, or convenience overrides correctness.
- **C2 is sacrificed only for C1.** Clarity gives way only when making code correct makes it unavoidably unclear (rare; usually clarity and correctness align).
- **C3 is sacrificed only for C1 or C2.** Simplicity yields to correctness and clarity.
- **C4C8 are tradeable** among themselves, but always below C1C3. Their relative order is: Locality > Reversibility > Composability > Observability > Economy.
- The precedence is **total**: a violation of a higher principle is never justified by appeal to a lower one.
## 4. Conflict Resolution Procedure
When two rules conflict:
### Step 1 — Identify the level of each rule
- Is it core, domain-first-principles, domain-topic, language, or example?
### Step 2 — Apply the hierarchy
- The higher-level rule wins. Done.
- If same level, proceed to Step 3.
### Step 3 — Identify the core derivation
- Look up each rule in `matrix/principles-matrix.md`.
- Each rule derives from one or more core principles (e.g., Security P2 derives from C3 and C8).
### Step 4 — Apply core precedence
- Compare the highest-precedence core principle each rule derives from.
- The rule deriving from the higher-precedence core principle wins.
- Example: a rule deriving from C1 (Correctness) beats a rule deriving from C8 (Economy), always.
### Step 5 — If still tied, the more specific scope wins
- A rule scoped to "API endpoints" wins over a rule scoped to "all code" for API endpoint conflicts.
- A rule scoped to "TypeScript" wins over a rule scoped to "all languages" for TypeScript conflicts.
### Step 6 — If still tied, the conflict is a defect
- File an issue. One of the rules is wrong. Do not silently pick one.
- The matrix should not produce ties. If it does, the matrix is incomplete.
## 5. Worked Examples
### Example A: Security P9 (Secret Hygiene) vs DevOps P9 (Documentation in Pipeline)
- Both are domain-level. Same level → Step 3.
- Security P9 derives from C6, C7 (per matrix).
- DevOps P9 derives from C2 (per matrix).
- C2 > C6 and C7? **No** — C2 is precedence 2, C6 is 6, C7 is 7. C2 is higher.
- But wait: Security P9 is **non-tradeable** per the security domain's own conflict resolution. Security's manifesto says 8 of 10 principles are never sacrificed.
- **Resolution:** Security domain's non-tradeable declaration is a domain-level rule that itself derives from C1 (Correctness — security failures are correctness failures). So Security P9 effectively derives from C1. C1 > C2. **Security P9 wins.** Do not log secrets in the pipeline.
- **Lesson:** A domain's "never sacrificed" declaration promotes those rules to C1-equivalent.
### Example B: Performance P8 (Budget Discipline) vs Clarity (C2)
- Performance P8 is domain-level, derives from C8 (Economy).
- Clarity is core-level (C2).
- Core > domain. **C2 Clarity wins.** Clear code that is slightly over budget beats opaque code that is under budget.
- **Exception:** if the budget is a correctness requirement (e.g., real-time system), the budget derives from C1, and C1 > C2. Then the budget wins.
### Example C: API P5 (Versioning) vs API P10 (Stability)
- Both are in the same domain (api), same level.
- API P5 derives from C5 (Reversibility). API P10 derives from C5 (Reversibility). Same core derivation — tied.
- API's own conflict resolution (in `domains/api/first-principles.md` §3): "Versioning — never sacrificed." So P5 wins.
- **Resolution:** Versioning wins. You version the change rather than break stability, because versioning is the mechanism that enables stability. They are not actually in conflict — versioning is how you preserve stability.
## 6. Non-Tradeable Declarations
Some domains declare certain principles non-tradeable (e.g., Security: 8 of 10). These declarations are binding:
- A non-tradeable domain rule is treated as deriving from C1 (Correctness), regardless of its matrix entry.
- This is because the domain has asserted that violating the rule is a correctness failure in that domain's scope.
- Non-tradeable rules can still conflict with each other (e.g., two security rules). The domain's own precedence (in its §3) resolves those.
## 7. The Matrix is the Arbiter
`matrix/principles-matrix.md` is the single source of truth for derivations. If a rule is not in the matrix, it is not part of the framework. If the matrix and a domain document disagree, **the matrix wins** — the domain document must be corrected to match the matrix, or the matrix must be updated with a new derivation (a docs change, not a silent edit).
## 8. Conflicts with This Document
If `core/conflict-resolution.md` conflicts with `core/first-principles.md`, `first-principles.md` wins. This document is the procedure; the principles are the authority.
+114
View File
@@ -0,0 +1,114 @@
# Core — First Principles
**Version:** 1.0.0
**Status:** Foundational
**Audience:** All agents and humans. Read this first.
## 1. Manifesto
There are eight principles from which all other principles derive. Every domain rule, every language rule, every review checklist traces back to one or more of these. When two rules conflict, the higher-precedence core principle wins. When a rule has no derivation here, it is orphaned and must be removed or re-derived.
These principles are universal. They apply to every language, every domain, every system. They are not opinions; they are the axioms of quality software.
## 2. The Principles
### C1. Correctness
The system does what it is supposed to do, and nothing else.
- Correctness is the highest principle. No other principle overrides it.
- A correct system handles the specified cases, the edge cases, and the failure cases — all defined by the domain.
- Incorrect code that is fast, clear, or simple is still wrong.
- Security is a subset of correctness: code that is exploitable does not do what it was supposed to do.
- Correctness includes temporal correctness: a late answer is a wrong answer when the deadline mattered.
### C2. Clarity
The intent of the code is obvious to its reader.
- Code is read more than it is written. Optimize for the reader.
- Names reveal intent. Comments explain why, not what.
- A stranger should understand the code without reading the documentation. The documentation should make the code unnecessary to read.
- Clarity serves correctness: unclear code is where bugs hide.
### C3. Simplicity
The solution is as simple as possible, and no simpler.
- Complexity is the enemy of correctness. Every line of code is a liability.
- Simplicity is not laziness. It is the result of removing everything unnecessary.
- A simple solution handles the requirement completely. A simpler-than-necessary solution does not.
- Simplicity serves economy: less code, less to test, less to maintain.
### C4. Locality
Decisions and their consequences live near each other.
- State, logic, and side effects that depend on each other live near each other.
- Locality serves clarity: to understand a piece of code, you should not need to read the whole system.
- Locality serves correctness: coupling that is far apart is coupling you cannot see.
- A change that requires touching many distant files is a locality violation.
### C5. Reversibility
Every decision can be undone, and the cost of undoing is known.
- Reversibility serves safety: an irreversible decision is a bet you cannot hedge.
- Migrations, deployments, schema changes, and API changes are reversible by default.
- A decision that cannot be reversed is made with the maximum of evidence and the minimum of haste.
- Versioning, feature flags, and rollback paths are the mechanisms of reversibility.
### C6. Composability
Parts combine into wholes, and the parts are reusable in new wholes.
- Composability serves scale: complex systems are built from simple, composable parts.
- A part that does one thing well composes. A part that does many things configures.
- The boundary of a part is its contract. Contracts are explicit and stable.
- Composability serves clarity: composable parts are understandable in isolation.
### C7. Observability
The system's behavior is visible to the people who must understand it.
- Observability serves correctness: you cannot fix what you cannot see.
- Logs, metrics, and traces are first-class. They are designed in, not bolted on.
- Observability serves clarity: the system explains itself in production.
- An observable system answers "what happened, why, and what next?" without reading the source.
### C8. Economy
The system uses no more resources than the task requires.
- Resources include time, memory, attention, money, and complexity.
- Economy serves simplicity: the cheapest solution is often the simplest.
- Economy is the most tradeable principle: it is sacrificed for correctness, clarity, and reversibility.
- But unbounded economy is a bug: unbounded growth in any resource is a defect.
## 3. Precedence
When principles conflict, the lower-numbered principle wins:
```
C1 Correctness > C2 Clarity > C3 Simplicity > C4 Locality >
C5 Reversibility > C6 Composability > C7 Observability > C8 Economy
```
- C1 (Correctness) is never sacrificed.
- C2 (Clarity) is sacrificed only for C1 (Correctness).
- C3 (Simplicity) is sacrificed only for C1 or C2.
- C4C8 are tradeable among themselves, but always below C1C3.
- The precedence is total: a violation of C1 is never justified by an appeal to C8.
See `core/conflict-resolution.md` for the formal conflict resolution procedure.
## 4. What Violates These Principles
| Violation | Principle Breached |
|-----------|-------------------|
| Code that "mostly works" | C1 Correctness |
| A function named `doStuff` | C2 Clarity |
| A 500-line function that could be 50 | C3 Simplicity |
| Config in a distant repo, code reads it silently | C4 Locality |
| A migration with no rollback | C5 Reversibility |
| A component that reads global state implicitly | C6 Composability |
| A service with no logs | C7 Observability |
| Loading all records into memory | C8 Economy |
## 5. Relationship to Domains
Every domain in Atelier derives its own P1P10 principles from these eight. The mapping is recorded in `matrix/principles-matrix.md`. A domain rule that cannot be traced to a core principle is orphaned and must be removed.
The eight principles are the floor. The domain principles are the ceiling for their scope. Within a domain, the domain's precedence applies; across domains, the core precedence applies.
+103
View File
@@ -0,0 +1,103 @@
# Core — Reading Order
**Status:** Foundational
**Audience:** New readers (agents and humans). Read after `first-principles.md` and `conflict-resolution.md`.
## 1. The Canonical Order
This is the order in which the framework is designed to be consumed. Following it ensures you build the mental model before the details.
### For All Readers
1. **`core/first-principles.md`** — The eight axioms. Everything derives from these. Read first, always.
2. **`core/conflict-resolution.md`** — How to resolve disagreements between rules. Read second, always.
3. **`core/reading-order.md`** — This document. You are here.
### For a Specific Task
4. **`domains/<relevant>/first-principles.md`** — The domain's ten principles. Read the domain(s) relevant to your task.
5. **`domains/<relevant>/<topic>.md`** — The domain's derived topic docs. Read only what your task needs.
### For Reference
6. **`matrix/principles-matrix.md`** — The full mapping of every domain principle to its core derivation. Consult when a rule's authority is unclear or a conflict arises.
7. **`MANIFEST.md`** — The authoritative index of all framework documents. If a document is not here, it is not part of the framework.
### For Evaluation
8. **`review/agent-checklist.md`** — The pre-completion checklist. Agents run this before finishing a task.
9. **`review/peer-review-checklist.md`** — The human review checklist.
10. **`review/anti-patterns.md`** — Catalog of known violations.
### For Language-Specific Application
11. **`languages/<lang>.md`** — How the domain principles apply in a specific language. Read only for the language you are working in.
### For Illustration
12. **`examples/good/*`** — Worked examples of the framework applied correctly.
13. **`examples/bad/*`** — Worked examples of violations, with the breached principles cited.
## 2. Reading Paths
### Path A: New Agent (Pre-Task)
```
core/first-principles.md
→ core/conflict-resolution.md
→ core/reading-order.md
→ domains/<task-domain>/first-principles.md
→ domains/<task-domain>/<topic>.md
→ review/agent-checklist.md (run before completing)
```
Estimated time: 1525 minutes depending on domain depth.
### Path B: New Human (Onboarding)
```
README.md
→ MANIFEST.md
→ core/first-principles.md
→ core/conflict-resolution.md
→ core/reading-order.md
→ one domain of personal interest (e.g., domains/api/first-principles.md)
→ matrix/principles-matrix.md (skim)
```
Estimated time: 3045 minutes for a full pass.
### Path C: Conflict Resolution
```
core/conflict-resolution.md (procedure)
→ matrix/principles-matrix.md (look up both rules' derivations)
→ core/first-principles.md (apply precedence)
```
Estimated time: 5 minutes per conflict.
### Path D: Review
```
review/agent-checklist.md (or peer-review-checklist.md)
→ for each failed checklist item, follow Path C
→ examples/bad/* for the relevant violation pattern
```
## 3. What Not to Read First
- **Do not** start with `examples/`. Examples illustrate principles you have not yet read.
- **Do not** start with `languages/`. Language rules derive from domain rules you have not yet read.
- **Do not** start with `matrix/`. The matrix is a reference, not a tutorial.
- **Do not** skip `core/`. Everything depends on it.
## 4. Skipping is Allowed After Core
After reading `core/` (the three foundational documents), you may skip freely. The domains are independent of each other. You do not need to read `domains/security/` to use `domains/api/`. You only need the domain relevant to your task.
The one exception: `matrix/principles-matrix.md` references all domains. You do not need to read all domains to use the matrix, but the matrix is most useful when you have read at least one domain.
## 5. Versioning
The framework versions as a whole (semantic versioning). A change to `core/first-principles.md` is a major version. A new domain is a minor version. A new topic doc is a patch. See `CHANGELOG.md` for the version history.
+90
View File
@@ -0,0 +1,90 @@
# Data Versioning — Derived Rules
> Derives from `domains/ai-ml/first-principles.md`. Covers P2 (Data is
> Versioned, Not Just Code) and P3 (Lineage is Traceable End-to-End).
> Referenced by `serving.md` and `monitoring-drift.md`. Scope per
> D-023: engineering discipline of versioning data, not dataset
> content design.
## Why Data Versioning (P2 Data is Versioned, Not Just Code)
- `git` versions code well and data badly. Datasets do not fit in
git, and a dataset is not recovered from a commit hash.
- A model trained on "the data" is a model trained on an unknown
input — a C1 (Correctness) violation. The dataset is a build
input; it is named, hashed, and recoverable the way any build
input is.
- Data versioning is the ML analogue of `domains/data/migrations.md`:
the schema and contents of the data evolve, every evolution is a
versioned migration, and every model points at a specific version.
## Dataset Hashing and Lineage (P3 Lineage Traceable End-to-End)
- Every dataset version has a content hash (not a filename or a
timestamp). The hash is the identity. A model's lineage record
names the dataset hash it was trained on; a serving prediction
names the model digest it came from.
- Lineage is a graph: prediction → model → training run → dataset →
source(s). Any edge missing is an orphan (`domains/observability/logging.md`
for the structured-log angle on lineage events).
- The lineage record is append-only. Editing it to "fix" a broken
trace is the same class of violation as editing an audit log.
## Train/Val/Test Split Versioning (P2, P4 Eval Defined Before Training)
- Splits are versioned with the dataset, not derived ad-hoc per run.
A split is a deterministic function of (dataset version, split
config, random seed). Two runs on the same pinned inputs produce
the same splits.
- The eval split is held out and never touched by training. A "held
out" set that leaked into training is a P4 (Evaluation Defined
Before Training) violation, not just a P2 violation — the eval
gate is measuring the training set, not the model.
- Cross `domains/data/schema-design.md` for the eval input contract:
the schema of the eval set is part of the versioned artifact.
## Tool Comparison (IDEATE-22, D-040)
| Tool | Versioning Model | Lineage | Best For | Notes |
|------|------------------|---------|----------|-------|
| DVC | Git-like pointers to content-addressed object store; `.dvc` files in git track data versions | Pipeline DAG in `dvc.yaml`; reproducibility via `dvc repro` | Teams already on git; file/directory datasets; ML pipelines | Treats data like code; shares git's history model. Object store is pluggable (S3, GCS, Azure, SSH) |
| Delta Lake | Table format with transaction log (ACID) + time travel via versioned commits; schema enforcement | Time travel queries; lineage via table history + catalog | Large tabular data; lakehouse; streaming + batch on the same table | Not a pipeline tool — pairs with Spark/Trino/Flink. Brings DB guarantees to object storage |
| LakeFS | Git-like operations (branch, commit, merge) over object storage itself | Branch model gives isolated, reproducible data branches | Data engineering teams; branch-per-experiment; CI over data | Not a table format — versions objects. Composes with Delta/Iceberg on top |
- Pick one primary versioning model per platform. Mixing DVC's
pointer model with Delta's transaction-log model fragments
operational knowledge (C4 Locality).
- All three satisfy P2; the choice is which fits the data shape and
the team's existing tooling. None is advocated over the others.
## Reproducibility Contract (P1 Reproducibility is the First Class)
A reproducible training run records, in one versioned place:
```
run_id: 2026-08-05T09:12:00Z#run-42
dataset: s3://ml-data/train@sha256:7f3a...e21
splits: dvc.yaml@commit a1b2c4d
code: git@a1b2c4d
config: configs/train.yaml@commit a1b2c4d
environment: ghcr.io/org/train-img@sha256:9c2d...f88
eval_spec: configs/eval.yaml@commit a1b2c4d
model_digest: registry/model@sha256:b5e1...aa0
```
- Lose any line and the run is anecdote, not evidence.
- The record is the lineage root: a prediction cites the
`model_digest`, which cites the `run_id`, which cites everything
above. This is how P3 (Lineage Traceable End-to-End) is satisfied
in practice.
## What Violates Data Versioning Discipline
| Violation | Principle |
|-----------|-----------|
| Dataset referenced by `s3://bucket/latest/` | P2 Data is Versioned, Not Just Code |
| Splits regenerated with an unpinned seed per run | P2, P4 Evaluation Defined Before Training |
| A production model with no dataset hash in its lineage | P3 Lineage Traceable End-to-End |
| Editing a lineage record to "clean up" a broken trace | P3 Lineage Traceable End-to-End |
| Eval split reachable from the training data path | P4 Evaluation Defined Before Training |
| Two platforms versioning the same data with different models | C4 Locality |
+154
View File
@@ -0,0 +1,154 @@
# AI / ML — First Principles
> Scope per D-023: this domain covers ML **engineering discipline**
> data versioning, evaluation methodology, serving patterns, and drift
> detection. It does **not** cover algorithm design, model architecture
> selection, hyperparameter tuning, or model-family comparison. Those
> are research choices, not engineering principles, and they have no
> derivation in the core C-rules.
## 1. The Principles
### P1. Reproducibility is the First Class
Every training run is reproducible from pinned data + code + config +
environment. An unreproducible run is an unreviewable run: you cannot
decide whether a result is correct if you cannot recreate it.
Reproducibility is the ML analogue of `domains/devops/P1
Reproducibility` and inherits its non-negotiable status. Lose any one
of data, code, config, or environment pinning, and the run is
anecdote, not evidence.
### P2. Data is Versioned, Not Just Code
Datasets, features, and train/val/test splits are first-class
versioned artifacts with content hashes and lineage. `git` alone is
insufficient — datasets do not fit in git, and a dataset is not a
commit hash. A model trained on "the data" is a model trained on an
unknown input, which is a correctness violation. Version data the way
you version code: pinned, named, and recoverable.
### P3. Lineage is Traceable End-to-End
Any deployed prediction traces back through model → training run →
dataset → source. No orphan models. A model in production with no
lineage is a correctness defect: you cannot reason about its failure
modes, you cannot roll it back to a known-good dataset, and you cannot
tell whether drift is in the model or in the data that built it.
Lineage is the audit trail of ML (`domains/observability/logging.md`).
### P4. Evaluation is Defined Before Training
Metrics, splits, and acceptance thresholds are declared a priori, in
code, before the model is trained. Cherry-picking metrics post-hoc is
a correctness violation: the evaluation is no longer measuring the
model, it is rationalizing it. The eval spec is a contract — it is
reviewable, it is versioned, and it is the gate the model must pass
before it leaves the experiment. This is the ML angle on C2 Clarity:
the intent of the model is obvious to its reader because the eval
declared it first.
### P5. Models are Versioned Artifacts
A model is a pinned, immutable, registry-tracked artifact with a
unique identifier. Never "the latest." A serving endpoint that pulls
"latest" is serving an unknown model — its behavior is undefined, its
rollback is impossible, and its lineage is broken. The model registry
is to models what a container registry is to images
(`domains/devops/P7 Immutability`): immutable, addressed by digest,
promoted by stage.
### P6. Serving is Observable
Inference latency, throughput, input distributions, and prediction
confidence are first-class signals. Silent serving is a bug. A model
in production that emits no metrics is a model you cannot operate: you
cannot see latency regressions, you cannot see input drift, you cannot
see a failing downstream consumer. Observability is designed in, not
bolted on (`domains/observability/metrics.md`).
### P7. Drift is Expected and Detected
Data drift, concept drift, and prediction drift are monitored as a
matter of course. A drift signal is an incident, not a curiosity. ML
systems decay without code changes — the world changes under the
model — so "no code changed" is not a defense against a serving
regression. Detecting drift is the ML-specific form of C7
Observability: you cannot fix a model you cannot see degrading.
### P8. Inference Inputs are Validated
The model's input contract — schema, value ranges, types, and
categorical domains — is enforced at the serving boundary.
Out-of-contract inputs are rejected, not silently scored. Scoring an
out-of-contract input is a correctness violation: the model's output
is undefined for inputs outside its training distribution, and
returning a number for it is lying to the caller. This is the ML angle
on `domains/security/input-validation.md` and inherits C1's
non-tradeable status.
### P9. Pipelines Compose, Notebooks Don't
Training and serving flows are composable pipelines with explicit
steps, named inputs, named outputs, and contracts between stages.
Notebooks are for exploration, not production. A notebook in the
serving path is a correctness defect: its state is implicit, its
order is human-dependent, and its reproducibility is whatever the last
operator remembered. Compose pipelines; keep notebooks in the lab.
### P10. Rollback Includes the Model
A serving rollback restores the prior model artifact, not just the
prior code. Promotion is reversible at the model layer. A rollback
that redeploys old code but keeps the new model has not rolled back —
the model was the thing that regressed. The rollback path must name
the prior model digest, the prior dataset version, and the prior eval
that cleared it. This is the ML angle on `domains/devops/P4 Rollback
First` and `domains/kubernetes/P10 Roll Forward, Roll Back`.
## 2. Core Principle Trace
Each AI/ML P-rule derives from one or more core C-rules (C1C8). The
matrix extension lands in P4 of the v0.3 plan; the traces below are
authoritative.
| P-rule | Core | Why |
|--------|------|-----|
| P1 Reproducibility is the First Class | C1, C5 | Correctness of results; reversibility of runs |
| P2 Data is Versioned, Not Just Code | C5, C7 | Reversibility of datasets; observability of data lineage |
| P3 Lineage is Traceable End-to-End | C7, C1 | Observability of provenance; correctness of attribution |
| P4 Evaluation is Defined Before Training | C1, C2 | Correctness of the eval gate; clarity of a-priori intent |
| P5 Models are Versioned Artifacts | C5, C6 | Reversibility of model identity; composability of registry stages |
| P6 Serving is Observable | C7 | Observability of inference |
| P7 Drift is Expected and Detected | C7, C1 | Observability of degradation; correctness of detection |
| P8 Inference Inputs are Validated | C1 | Correctness of the serving boundary (security subset) |
| P9 Pipelines Compose, Notebooks Don't | C6, C2 | Composability of stages; clarity of explicit contracts |
| P10 Rollback Includes the Model | C5 | Reversibility at the model layer |
## 3. What Violates These Principles
| Violation | Principle Breached |
|-----------|-------------------|
| A training run that cannot be replayed from pinned inputs | P1 Reproducibility is the First Class |
| A dataset referenced by a mutable path, not a hash | P2 Data is Versioned, Not Just Code |
| A production model with no record of its training data | P3 Lineage is Traceable End-to-End |
| Metrics chosen after seeing the results | P4 Evaluation is Defined Before Training |
| A serving endpoint that pulls `latest` from the registry | P5 Models are Versioned Artifacts |
| A model in production with no latency or throughput metrics | P6 Serving is Observable |
| A serving regression dismissed as "no code changed" | P7 Drift is Expected and Detected |
| An input with an out-of-range feature scored silently | P8 Inference Inputs are Validated |
| A notebook in the serving or training pipeline path | P9 Pipelines Compose, Notebooks Don't |
| A rollback that restores code but keeps the regressed model | P10 Rollback Includes the Model |
## 4. Relationship to Other Domains
AI/ML is the engineering-discipline layer for model-bearing systems.
It borrows the reproducibility, immutability, rollback, and
observability disciplines of `domains/devops/` and applies them to
the data → model → serving lifecycle. Cross-links are one-directional
(per D-026 extended):
- `domains/devops/P1 Reproducibility` ← P1
- `domains/devops/P4 Rollback First` ← P10
- `domains/devops/P5 Progressive Delivery` ← P10 (model canary)
- `domains/devops/P7 Immutability` ← P5 (model images)
- `domains/data/migrations.md` ← P2 (data versioning ↔ migration discipline)
- `domains/data/schema-design.md` ← P8 (inference input contract)
- `domains/observability/metrics.md` ← P6, P7
- `domains/observability/logging.md` ← P3 (lineage)
- `domains/security/input-validation.md` ← P8
- `domains/security/secrets.md` ← P8 (serving credentials)
- `domains/performance/backend.md` ← P6 (serving latency)
- `domains/kubernetes/workloads.md` ← P9 (serving on k8s)
- `domains/testing/first-principles.md` ← P4 (eval as a gate)
- `domains/gitops-operators/first-principles.md` ← P10 (model rollback in a GitOps loop)
+94
View File
@@ -0,0 +1,94 @@
# Model Evaluation — Derived Rules
> Derives from `domains/ai-ml/first-principles.md`. Covers P4
> (Evaluation is Defined Before Training) and the eval-as-a-gate
> discipline. Referenced by `serving.md` (promotion gate) and
> `monitoring-drift.md` (online eval). Scope per D-023: evaluation
> methodology, not metric math or model-family benchmarks.
## Evaluation is a Gate, Not a Report (P4 Evaluation Defined Before Training)
- The eval spec — metrics, splits, thresholds, and pass/fail
criteria — is declared in code **before** the model is trained.
It is versioned with the data and the code; it is reviewable; it
is the contract the model must satisfy to leave the experiment.
- Cherry-picking metrics after seeing results is a correctness
violation: the eval is no longer measuring the model, it is
rationalizing it. The a-priori spec is what makes the eval
trustworthy.
- This is the ML angle on `domains/testing/first-principles.md` P1
(Tests as Specification): the eval declares the model's contract,
the model does not declare its own success.
## The Eval Input Contract (P8 Inference Inputs are Validated, cross `domains/data/schema-design.md`)
- The eval set has a schema: feature names, types, ranges, and
categorical domains. That schema is the same schema the serving
boundary enforces (`serving.md`, `domains/security/input-validation.md`).
- An eval set whose schema drifted from the serving schema is
measuring a different model than the one in production. Schema
parity is part of the versioned eval artifact.
- Cross `domains/data/schema-design.md`: the eval input contract is
a schema-design problem, versioned and reviewed like any schema.
## Holdout Integrity (P4, P2 Data is Versioned)
- The held-out eval set is never touched by training, feature
selection, or threshold tuning. A "held out" set that influenced
any training decision is not held out — it is a third training
set, and the eval is measuring memorization.
- Splits are versioned with the dataset (`data-versioning.md`).
Recreating splits ad-hoc per run breaks comparability across runs.
- Reusing a held-out set across many model iterations leaks it
incrementally. Rotate or re-split on a cadence; record the
rotation in lineage.
## Offline vs Online Evaluation (P6 Serving is Observable)
- **Offline eval** runs before promotion: held-out data, pinned
model, declared metrics, pass/fail gate. It answers "should this
model ship?"
- **Online eval** runs after promotion, on live traffic: shadow
scoring, A/B, canary metrics. It answers "is this model behaving
in production?" It is the bridge to `monitoring-drift.md`.
- A model that passed offline and regressed online is not a
contradiction — it is a signal that the offline distribution
differs from the live one (a P7 drift signal). Both eval layers
are required; neither substitutes for the other.
## Bias and Fairness Checks (Engineering Angle, P4)
- Bias/fairness checks are part of the a-priori eval spec, not an
afterthought. They are metrics with thresholds, declared before
training, gated the same as any metric.
- This doc covers the **engineering** discipline: the checks are
versioned, gated, and recorded in lineage. The choice of which
fairness metrics and what thresholds are policy decisions, not
engineering principles, and are out of scope here (D-023).
## Eval-as-a-Gate in the Pipeline (P9 Pipelines Compose)
- The eval is a pipeline stage with a contract: input = model
digest + eval dataset version; output = pass/fail + metric
report. It composes with the training stage and the promotion
stage.
- A promotion that bypasses the eval stage is a P4 violation,
regardless of who approved it. The gate is in the pipeline, not
in a human sign-off sheet.
```
train -> eval(gate) -> register(promote) -> serve
|
+-- fail -> abort, no promote
```
## What Violates Evaluation Discipline
| Violation | Principle |
|-----------|-----------|
| Metrics chosen after seeing the scores | P4 Evaluation Defined Before Training |
| Held-out set used in feature selection or threshold tuning | P4, P2 |
| Eval schema differs from serving schema | P8 Inference Inputs are Validated |
| Promotion by human approval, bypassing the eval stage | P4, P9 Pipelines Compose |
| A "passing" model with no online eval in production | P6 Serving is Observable |
| Fairness checks added after a model shipped | P4 Evaluation Defined Before Training |
+88
View File
@@ -0,0 +1,88 @@
# Monitoring & Drift — Derived Rules
> Derives from `domains/ai-ml/first-principles.md`. Covers P7 (Drift
> is Expected and Detected) and the online half of P6 (Serving is
> Observable). Referenced by `serving.md` (online eval) and
> `model-evaluation.md` (online layer). Scope per D-023: drift
> detection methodology, not model retraining architecture.
## Drift is Expected and Detected (P7 Drift is Expected and Detected)
- ML systems decay without code changes. The world changes under
the model: user behavior shifts, input pipelines change,
upstream schemas evolve. "No code changed" is not a defense
against a serving regression.
- A drift signal is an incident, not a curiosity. It triggers an
alert, an investigation, and a decision (retrain, roll back, or
accept with a recorded justification). Silent drift is the same
class of bug as silent serving (P6).
- Cross `domains/observability/metrics.md` for the alerting
primitives and `domains/observability/logging.md` for the
structured events a drift signal emits.
## The Three Drift Types (IDEATE-30, D-048)
| Drift Type | What Changes | Detection Signal | Source of Truth |
|------------|--------------|------------------|-----------------|
| **Data drift** (input drift) | The distribution of inputs at serving time diverges from the distribution the model was trained on | Statistical distance between the live input distribution and the pinned training-set distribution (e.g., PSI, KL, KS test). Alert on threshold breach | Training dataset hash (`data-versioning.md`) + live input metrics |
| **Concept drift** | The relationship between inputs and the target changes — the same input now maps to a different correct output | Ground-truth lag: compare delayed labels against predictions on the same inputs. Rising error rate against a stable input distribution signals concept, not data, drift | Delayed-label feedback stream + prediction log |
| **Prediction drift** (output drift) | The distribution of the model's predictions shifts, with no change to inputs | Statistical distance between the live prediction distribution and a pinned baseline prediction distribution. Independent of inputs — catches model-internal regressions and upstream silent changes | Prediction log + baseline prediction snapshot |
- The three signals are distinct and non-substitutable. Data drift
catches the input changing; concept drift catches the world
changing; prediction drift catches the model's behavior changing.
A monitoring setup with only one is blind to two classes of
regression.
- Evidently AI and Great Expectations are the canonical tooling:
Evidently for drift/statistical reports, Great Expectations for
data-quality/contract checks at the pipeline boundary. Both
produce the metrics that feed `domains/observability/metrics.md`.
## Detection Signals in Practice
- **Data drift** compares live inputs to the **pinned training
distribution** — not to "yesterday's inputs." Without a pinned
baseline, drift is measured against a moving target and is
meaningless. Cross `data-versioning.md` for how the baseline is
pinned.
- **Concept drift** requires ground truth, which is often delayed
(days/weeks). The detection signal is the gap between
prediction-time confidence and delayed-label error. A rising
error against stable inputs is the signature.
- **Prediction drift** needs no ground truth and no input
comparison — it watches the model's own output distribution. It
is the cheapest signal and the first to fire; it is also the
least specific (any of the three drifts can move predictions).
## Alerting and Retraining Triggers (P7, P10 Rollback Includes the Model)
- A drift alert is an incident. It does not auto-trigger retraining
unsupervised — auto-retraining on drift can lock in a bad
distribution. The alert triggers a human decision: investigate,
retrain, roll back, or accept.
- Retraining is a new training run (`first-principles.md` P1): it
produces a new model digest, passes the eval gate
(`model-evaluation.md`), and is promoted through the registry
(`serving.md`). The prior model stays rollbackable (P10).
- Cross `domains/observability/metrics.md` for the alert-rule
pattern: threshold + window + severity, routed to the same
on-call path as any production incident.
## Online Evaluation Bridge (P6 Serving is Observable)
- Online eval (`model-evaluation.md`) is the live counterpart to
drift monitoring: shadow scores and A/B canaries measure a
candidate model against the incumbent, while drift monitoring
measures the incumbent against its own baseline. Both feed the
same metrics pipeline.
## What Violates Monitoring Discipline
| Violation | Principle |
|-----------|-----------|
| Only one drift type monitored | P7 Drift is Expected and Detected |
| Drift baseline is "yesterday's inputs," not pinned training data | P7, P2 Data is Versioned |
| Drift alert that auto-retrains without a human gate | P7, P1 Reproducibility |
| A serving regression dismissed as "no code changed" | P7 Drift is Expected and Detected |
| Concept-drift check with no delayed-label feedback path | P7 Drift is Expected and Detected |
| Prediction-distribution change with no alert | P6 Serving is Observable, P7 |
+88
View File
@@ -0,0 +1,88 @@
# Serving — Derived Rules
> Derives from `domains/ai-ml/first-principles.md`. Covers P5 (Models
> are Versioned Artifacts), P6 (Serving is Observable), P8 (Inference
> Inputs are Validated), and P10 (Rollback Includes the Model).
> Referenced by `monitoring-drift.md` (online signals) and
> `model-evaluation.md` (promotion gate). Scope per D-023: serving
> patterns, not model architectures.
## The Model is an Addressed Artifact (P5 Models are Versioned Artifacts)
- A serving endpoint pulls a model by digest, never by `latest`. A
model pulled by `latest` is an unknown model — its behavior is
undefined and its rollback is impossible.
- The model registry is to models what a container registry is to
images (`domains/devops/P7 Immutability`): immutable, addressed by
digest, promoted by stage (staging → prod). Promotion is a
registry operation, not a file copy.
- A serving rollout names the model digest in its manifest. The
digest is part of the deploy's lineage (`data-versioning.md`).
## Inference Inputs are Validated (P8 Inference Inputs are Validated)
- The model's input contract — schema, types, ranges, categorical
domains — is enforced at the serving boundary, before the model
sees the input. Out-of-contract inputs are rejected with a
defined error, not silently scored.
- Scoring an out-of-contract input is a C1 (Correctness) violation:
the model's output is undefined outside its training
distribution, and returning a number for it is lying to the
caller.
- This is the ML angle on `domains/security/input-validation.md`:
the validation lives at the boundary, the model is downstream of
it, and the contract is versioned with the model.
## Serving is Observable (P6 Serving is Observable)
- Every inference path emits: request latency, throughput, input
distribution summaries, prediction confidence, and error counts.
Silent serving is a bug.
- Cross `domains/observability/metrics.md` for the metrics
primitives (histograms, counters, gauges) and
`domains/observability/tracing.md` for the request-level trace
that ties an input to a prediction.
- Latency SLAs are enforced via `domains/performance/backend.md`
disciplines: budget the inference path, measure the tail (p99),
alert on budget breach.
## Serving Patterns (P9 Pipelines Compose)
| Pattern | When | Notes |
|---------|------|-------|
| Inference as a service | Default; model behind an HTTP/gRPC endpoint | KServe, Seldon Core, BentoML. Scales with traffic; model is a deployable, addressable artifact |
| Batch inference | Offline scoring of large datasets | No latency SLA; throughput-bound. Same model digest, same input contract |
| Embedded / in-process | Latency-critical, single-tenant | Model linked into the app. Trades observability for latency — only when the SLA demands it |
- Canarying a model is a serving pattern, not a deployment pattern:
shift a fraction of traffic to the new model digest, measure
online eval (`model-evaluation.md`), abort to the prior digest on
regression. This is `domains/devops/P5 Progressive Delivery`
applied at the model layer.
- Rollback restores the prior model digest (P10 Rollback Includes
the Model). A rollback that redeploys old code but keeps the new
model has not rolled back. Cross `domains/gitops-operators/first-principles.md`
for the GitOps reconciliation loop that drives model rollouts.
## Tool Landscape (KServe / Seldon Core / BentoML)
| Tool | Model Packaging | Deployment Surface | Notes |
|------|-----------------|--------------------|-------|
| KServe | InferenceService CRD; runtime predictors (v2, HuggingFace, PMML, custom) | Kubernetes-native; CRD-driven | Cross `domains/kubernetes/workloads.md`. Brings the k8s reconciliation model to serving |
| Seldon Core | SeldonDeployment CRD; graph of predictors | Kubernetes-native; CRD-driven | Emphasizes inference graphs (fan-out, ensemble) as CRD structure |
| BentoML | Bento (model + runtime + deps packaged); Yatai registry | Kubernetes or bare container | Focuses on packaging + registry; the Bento is the versioned artifact (P5) |
- All three satisfy P5/P6/P8 when wired correctly; the choice is
packaging model and deployment surface, not correctness.
- None is advocated over the others.
## What Violates Serving Discipline
| Violation | Principle |
|-----------|-----------|
| Endpoint pulls `latest` from the registry | P5 Models are Versioned Artifacts |
| Out-of-range input scored silently | P8 Inference Inputs are Validated |
| Serving path emits no latency or throughput metrics | P6 Serving is Observable |
| Rollback redeploys code but keeps the regressed model | P10 Rollback Includes the Model |
| A notebook in the serving path | P9 Pipelines Compose, Notebooks Don't |
| Canary with no abort-to-prior-digest path | P10, `domains/devops/P5 Progressive Delivery` |
+63
View File
@@ -0,0 +1,63 @@
# Error Responses — Derived Rules
> Derives from `domains/api/first-principles.md` P9 (Error Transparency) and `domains/errors/first-principles.md`.
## The Error Contract
Every error response is a JSON object with:
```json
{
"error": {
"code": "STRING_ERROR_CODE",
"message": "Human-readable description",
"details": {},
"request_id": "uuid"
}
}
```
- `code`: machine-consumable, stable, UPPER_SNAKE_CASE. Never a free-text message.
- `message`: human-readable, for logs and developers. Not for end users (see `domains/errors/` P8).
- `details`: structured, typed additional context (which field, what value, what constraint).
- `request_id`: correlation ID for tracing. Every error is traceable.
## Error Codes (P3 Predictability, P9)
- Codes are stable. Renaming an error code is a breaking change.
- Codes are specific: `VALIDATION_FAILED` not `BAD_REQUEST`. `DUPLICATE_EMAIL` not `CONFLICT`.
- Codes are namespaced: `USER_NOT_FOUND`, `ORDER_NOT_FOUND` — not just `NOT_FOUND`.
## Status Code Mapping (P1 Contract Fidelity)
| Code | Meaning | Error code example |
|------|---------|-------------------|
| 400 | Malformed request | `MALFORMED_REQUEST` |
| 401 | Auth required | `AUTH_REQUIRED` |
| 403 | Forbidden | `FORBIDDEN` |
| 404 | Not found | `<RESOURCE>_NOT_FOUND` |
| 409 | Conflict | `DUPLICATE_<RESOURCE>` |
| 422 | Semantic invalid | `VALIDATION_FAILED` |
| 429 | Rate limited | `RATE_LIMITED` |
| 500 | Server bug | `INTERNAL_ERROR` |
- Never return 200 with an error body. The status code is the first signal.
- Never return 500 for a client error. 500 means "the server has a bug."
## Information Disclosure (P8 Security, domains/security P9)
- Error messages do not leak internal state: no stack traces, no SQL fragments, no file paths.
- A 401 does not say "user not found" vs "wrong password" — both say "invalid credentials."
- A 404 does not confirm the resource exists but is forbidden — return 404, not 403, for unauthenticated requests to hidden resources.
- Detailed errors are logged server-side with `request_id`; the client gets the safe version.
## Retryability (P6 Idempotency)
- Errors that are safe to retry: 409, 422 (if the fix is applied), 429 (after backoff), 5xx.
- Errors that are not safe to retry: 400, 401 (without re-auth), 403.
- The error body indicates retryability: `retryable: true/false` or via the code's known semantics.
## Partial Errors (GraphQL, see `domains/api/graphql.md`)
- GraphQL returns data and errors together. Do not conflate.
- A null field with no error is a bug. A null field with an error is a partial failure.
+84
View File
@@ -0,0 +1,84 @@
# API Design — First Principles
**Version:** 1.0.0
**Status:** Foundational
**Audience:** AI agents and humans designing APIs (REST, GraphQL,
gRPC, RPC, libraries).
## 1. Manifesto
An API is a contract between systems and the people who build on
them. The cost of an API is paid by every consumer, forever. The
highest quality API is one that a stranger can use correctly without
reading the source.
## 2. The Principles
### P1. Contract Fidelity
The API does what its documentation says, and the documentation says
what the API does. Nothing more, nothing less.
### P2. Clarity
Endpoints, methods, parameters, and responses are named and structured
for the consumer — not for the implementer.
### P3. Predictability
Consumers can guess behavior without reading docs. Patterns repeat.
Surprises are bugs.
### P4. Composability
Resources and operations combine cleanly. The whole is greater than
the sum of its parts, and the parts are reusable in new wholes.
### P5. Versioning
Changes are managed explicitly, not implicitly. Consumers know what
will break, and when.
### P6. Idempotency
Repeated identical calls have the same effect as a single call. Retry
is a first-class operation.
### P7. Performance
Latency, payload size, and call count are designed in — not optimized
out.
### P8. Security
Authentication, authorization, validation, and rate limiting are
defaults, not add-ons.
### P9. Error Transparency
Failures are communicated specifically, structurally, and actionably.
### P10. Stability
Consumers can build on the API without fear of breakage. Backward
compatibility is a default.
## 3. Conflict Resolution
1. Contract Fidelity — never sacrificed.
2. Security — never sacrificed.
3. Stability — sacrificed only with a documented deprecation cycle.
4. Clarity — sacrificed only for Performance with evidence.
5. Predictability — sacrificed for Composability when patterns diverge.
6. Composability — sacrificed for Clarity when abstractions confuse.
7. Idempotency — sacrificed only for genuinely non-idempotent operations.
8. Performance — sacrificed only with measurement.
9. Error Transparency — sacrificed only for security-sensitive errors.
10. Versioning — never sacrificed (always have a version policy).
## 4. What Violates These Principles
| Violation | Principle Breached |
|------------------------------------|----------------------|
| Endpoint name exposes DB schema | P2 Clarity |
| Breaking change without deprecation | P10 Stability |
| Generic 500 with stack trace | P9 Error Transparency|
| Auth as opt-in | P8 Security |
| Non-idempotent POST without key | P6 Idempotency |
| Inconsistent naming across endpoints | P3 Predictability |
| Required response field undocumented | P1 Contract Fidelity |
| 10MB response payload by default | P7 Performance |
## 5. Relationship to Core
Subordinate to `core/first-principles.md`. See `matrix/principles-matrix.md`.
+53
View File
@@ -0,0 +1,53 @@
# GraphQL — Derived Rules
> Derives from `domains/api/first-principles.md`. Applies P1P10 to GraphQL specifically.
## Schema First (P1 Contract Fidelity)
- The schema is the contract. Every field has a type, a description, and a deprecation status.
- The schema is versioned. Breaking schema changes (removing a field, changing a type) require a deprecation cycle.
- Never expose raw database types in the schema. Map them to domain types.
## Query Design (P2 Clarity, P3 Predictability)
- Field names are nouns, camelCase: `userOrders`, not `UserOrders` or `user_orders`.
- Arguments are descriptive: `first`, `after`, `orderBy` — not `arg1`, `arg2`.
- Connections for lists: `users(first: 10, after: "cursor")` — never bare arrays.
- Mutations are verbs: `createUser`, `deleteOrder` — not `userCreate`.
## N+1 Prevention (P7 Performance)
- Use a dataloader for every list field that resolves to another resource.
- A resolver that does a database query per item is an N+1 bug.
- Test resolvers under a list query, not just a single-item query.
## Authorization at the Field Level (P8 Security)
- Every resolver checks authorization. The query graph is not a trust boundary by default.
- A user who can query `user { email }` is not automatically authorized to query `user { passwordHash }`.
- Field-level authz is the floor, not an optimization.
## Deprecation (P5 Versioning, P10 Stability)
- Deprecate fields with `@deprecated(reason: "...")`. Never remove a field without deprecating first.
- A deprecated field is removed in the next major schema version, not sooner.
- Track field usage. A deprecated field with no usage can be removed sooner.
## Error Handling (P9 Error Transparency)
- Errors are partial by default: a query can return data and errors simultaneously.
- Errors are structured: `{ message, path, extensions: { code, ... } }`.
- Use `extensions.code` for machine-consumable error types, not free-text messages.
- Never swallow a resolver error silently. A null field with no error is a bug.
## Complexity Budget (P7 Performance, P8 Security)
- Enforce a query complexity limit. Unbounded depth/breadth is a DoS vector.
- Cost-based analysis (not just depth) catches expensive nested queries.
- Reject queries over budget with a 400, not a 500.
## Federation (P6 Composability)
- A federated subgraph owns its entities. Cross-graph references use `@external` and `@requires`.
- Never reach into another subgraph's database. The graph boundary is the contract.
- The gateway composes; subgraphs do not know about each other.
+69
View File
@@ -0,0 +1,69 @@
# Pagination — Derived Rules
> Derives from `domains/api/first-principles.md` P7 (Performance) and P3 (Predictability).
## Three Patterns
### 1. Offset/Limit (`?page=2&limit=20`)
- Simple, supports jumping to a page.
- Unstable under inserts: page 2 becomes page 1's content after an insert.
- Slow for large offsets: `OFFSET 10000` scans 10000 rows.
- Use for: small, stable collections, admin UIs.
### 2. Cursor (`?cursor=base64token&limit=20`)
- Stable under inserts: the cursor points to a position, not a page number.
- Fast: indexed lookup, no scan.
- No random access (cannot jump to page 5).
- Use for: infinite scroll, feeds, large collections, anything user-facing.
### 3. Keyset (`?after_id=123&limit=20`)
- Like cursor but uses the actual sort key (e.g., `after_id=123`).
- Most stable and fast. Requires a unique, monotonic sort key.
- Use for: ordered collections with a natural unique key.
## Defaults (P3 Predictability)
- Default `limit`: 20 or 50. Never unbounded.
- Max `limit`: 100 or 200. Reject `limit=10000` with 400.
- Default sort: by created_at descending, or by the resource's natural order.
- Always return the total count if cheap; never if it requires a separate COUNT query on a large table.
## Response Shape (P2 Clarity)
```json
{
"data": [...],
"pagination": {
"cursor": "next-base64-token",
"has_more": true
}
}
```
- `cursor` is null when there is no next page.
- `has_more` is the boolean convenience (some clients prefer it).
- Never return `data` as a bare array — always wrap so you can add pagination without breaking.
## Link Header (alternative)
```
Link: <https://api.example.com/users?cursor=X>; rel="next", <https://api.example.com/users?cursor=Z>; rel="prev"
```
- Useful for HTTP-level clients (curl, browser fetch).
- Less convenient for JSON-parsing clients.
## Consistency (P8 Consistency across endpoints)
- Every collection endpoint paginates the same way.
- A client that learns pagination on `/users` should know it on `/orders`.
- Mixed pagination (cursor here, offset there) is a tax on every consumer.
## What Violates Pagination
| Violation | Principle |
|-----------|-----------|
| Returning 10000 items by default | P7 Performance |
| `limit` with no max | P8 Security (DoS) |
| Page numbers on a frequently-inserted table | P3 Predictability |
| Bare array response (no pagination wrapper) | P1 Contract Fidelity (can't add pagination later without breaking) |
+68
View File
@@ -0,0 +1,68 @@
# REST — Derived Rules
> Derives from `domains/api/first-principles.md`. Applies P1P10 to REST specifically.
## Resource Naming (P2 Clarity, P3 Predictability)
- Nouns, not verbs: `/users`, `/orders`, not `/getUsers`.
- Plural: `/users` (collection), `/users/{id}` (item).
- Lowercase, hyphenated: `/order-items`, not `/OrderItems` or `/order_items`.
- Nesting max 2 levels: `/users/{id}/orders`, not `/users/{id}/orders/{oid}/items/{iid}`.
## HTTP Methods (P1 Contract Fidelity, P6 Idempotency)
| Method | Semantics | Idempotent | Safe |
|--------|-----------|------------|------|
| GET | Read | Yes | Yes |
| POST | Create | No | No |
| PUT | Replace (full) | Yes | No |
| PATCH | Update (partial) | No | No |
| DELETE | Remove | Yes | No |
- PUT requires the full resource. PATCH requires only the delta. Never accept a partial PUT.
- POST creates; never use POST for read operations. POST is not cacheable.
## Status Codes (P9 Error Transparency, P1 Contract Fidelity)
| Code | Meaning | When |
|------|---------|------|
| 200 | OK | Successful GET, PUT, PATCH |
| 201 | Created | Successful POST that created a resource |
| 204 | No Content | Successful DELETE, or empty response |
| 400 | Bad Request | Malformed request (client error) |
| 401 | Unauthorized | Authentication required or failed |
| 403 | Forbidden | Authenticated but not permitted |
| 404 | Not Found | Resource does not exist |
| 409 | Conflict | State conflict (e.g., duplicate) |
| 422 | Unprocessable | Well-formed but semantically invalid |
| 429 | Too Many Requests | Rate limited |
| 500 | Internal Error | Server bug — never leak stack trace |
- Never return 200 on an error. Never return 500 with a stack trace.
- 401 vs 403: 401 = "who are you?", 403 = "I know who you are, but you can't."
## Idempotency (P6 Idempotency)
- POST: not idempotent. Use an idempotency key (`Idempotency-Key` header) for safe retry.
- PUT: idempotent by definition — same PUT twice = same state.
- DELETE: idempotent — deleting a non-existent resource is success (204).
- PATCH: not idempotent by default; can be made idempotent with explicit versioning.
## Pagination (P7 Performance, see `pagination.md`)
- Default to cursor pagination for collections > 100 items.
- Never return unbounded collections.
- `Link` header or `cursor` field in response body.
## Versioning (P5 Versioning, P10 Stability, see `versioning.md`)
- Version in the URL (`/v1/users`) or in the header (`Accept: application/vnd.atelier.v1+json`).
- Pick one. Be consistent across all endpoints.
- Never make a breaking change without a new version and a deprecation cycle.
## Security (P8 Security, see `domains/security/`)
- HTTPS only. Redirect HTTP to HTTPS.
- Authentication on every non-public endpoint. No opt-in auth.
- Rate limiting on write endpoints (POST, PUT, PATCH, DELETE).
- Validate every input against a schema. Never pass raw request body to the database.
+54
View File
@@ -0,0 +1,54 @@
# API Versioning — Derived Rules
> Derives from `domains/api/first-principles.md` P5 (Versioning) and P10 (Stability).
## The Default: No Breaking Changes
- A breaking change is a new version. There is no "minor" breaking change.
- Breaking changes: removing a field, changing a field type, changing a field's semantics, changing required vs optional, changing error codes.
- Non-breaking changes: adding a field, adding an endpoint, adding an optional parameter, loosening validation.
## Version Policies
### URL Versioning (`/v1/users`)
- Simple, visible, cacheable.
- Breaking changes bump the major version: `/v1``/v2`.
- Old versions are supported in parallel during the deprecation window.
### Header Versioning (`Accept: application/vnd.atelier.v1+json`)
- Invisible in the URL; harder to test.
- Useful when the URL must stay stable (e.g., public webhooks).
### Semantic Versioning (for libraries/SDKs)
- Major: breaking. Minor: additive. Patch: fix.
- Follow semver strictly. A "minor" that breaks is a lie.
## Deprecation Cycle (P5 Reversibility)
1. **Announce**: mark the field/endpoint `@deprecated` with a sunset date.
2. **Support**: keep the old version working until the sunset date.
3. **Monitor**: track usage of the deprecated surface.
4. **Retire**: when usage drops below threshold (or sunset passes), remove.
5. **Never** remove without announcing. The cost of a silent break is paid by every consumer.
## Sunset Headers (P9 Error Transparency)
- Deprecated endpoints return `Sunset: <date>` header.
- Deprecated endpoints return `Deprecation: <date>` header.
- A consumer who reads headers knows when to migrate.
## Versioning vs Compatibility
- Versioning is the mechanism. Compatibility is the property.
- Backward compatibility: old consumers work with the new version.
- Forward compatibility: new consumers work with the old version (harder, rarer, usually not worth it).
- Aim for backward compatibility. Forward compatibility is for protocols, not APIs.
## What Violates Versioning
| Violation | Principle |
|-----------|-----------|
| Removing a field without deprecation | P5, P10 |
| Changing a field's type in a "minor" release | P1, P5 |
| No sunset header on a deprecated endpoint | P9 |
| Two versions with divergent semantics for the same field | P1 |
+165
View File
@@ -0,0 +1,165 @@
# Audit Logs — Derived Rules
> Derives from `domains/compliance/first-principles.md`. Covers P1
> (Audit Logs are Append-Only), P2 (Every Significant Action is
> Logged), P7 (Identity is Attributable), P9 (Secrets Redacted in
> Audit), and P10 (Compliance Posture Observable). Referenced by
> `data-retention.md` (retention applies to audit logs themselves)
> and `evidence.md` (audit logs are evidence).
## Audit Logs are Append-Only (P1 Audit Logs are Append-Only)
- An audit record is immutable once written. The storage substrate
enforces this; policy alone does not. Write-once, append-only
sinks (WORM buckets, immutable log streams, hash-chained ledgers)
are the mechanism.
- Deletion or mutation of an audit record is itself an auditable
incident. The tampering is the signal, not just the underlying
event. A system that allows `DELETE FROM audit_log` is a system
whose audit log is a draft.
- The append-only guarantee is testable: attempt to write, then
attempt to overwrite, then attempt to delete. If the overwrite or
delete succeeds, the guarantee is absent and the design is a
violation.
## Structured Audit Events (P2 Every Significant Action is Logged)
- The set of auditable actions is defined a priori, in code, before
the action ships. The catalog is versioned and reviewed. An
auditable action with no log line is a violation, not a gap to
backfill later.
- Audit events are structured (JSON / protobuf / a typed schema),
not prose. A prose log line ("user logged in") is unqueryable and
unaggregatable; a structured event is both. The event schema is
the contract between the producer and the audit pipeline.
```
{
"timestamp": "2024-11-07T15:03:22Z",
"event": "auth.login",
"actor": { "kind": "user", "id": "u_8f3a", "session": "s_12b9" },
"action": "succeeded",
"target": { "kind": "service", "id": "billing-api" },
"source": { "ip": "203.0.113.42", "region": "us-east-1" },
"request_id": "req_91c2",
"version": "audit-schema/v2"
}
```
- The catalog of significant actions typically includes:
authentication (success and failure), authorization decisions
(allow and deny), data access (read, write, delete), configuration
changes, policy changes, retention executions, and admin
operations. The exact set is declared per system; the discipline
is that it is declared.
## Cloud Audit Log Conventions (Prior Art, Abstracted)
- AWS CloudTrail, Google Cloud Audit Logs, and Azure Activity Log
share a common shape: immutable, time-ordered, queryable, with
actor / action / target / source / result fields. Atelier's
audit-logs doc adopts the shape, not the vendor.
- The shape is the contract; the sink is the implementation. A
self-hosted audit log that follows the same shape composes with
the same tooling (SIEM, query engines, evidence exporters) as the
cloud vendors'.
## Queryability (P10 Compliance Posture Observable)
- An audit log that cannot be queried is an audit log that cannot be
used. Queryability is a first-class design goal: the event schema
is typed, fields are indexed, and the common queries (who acted on
what when, what failed, what was denied) are cheap.
- "Who did X between T1 and T2" must be a single query, not a
forensics project. If the query requires a custom script per
investigation, the audit log is structured for storage, not for
use — a C7 (Observability) violation.
## Identity is Attributable (P7 Identity is Attributable)
- Every audit event records the authenticated principal that acted —
not a shared account, not a generic service, not "admin." The
actor field is populated at the time of the action from the
authenticated session, not resolved after the fact.
- A shared account in the actor field breaks accountability: an
event attributed to `svc-deploy` could be any of ten engineers.
This is the compliance angle on `domains/security/authorization.md`
and `domains/kubernetes/rbac.md`: bind actions to unique
principals, not to roles many can assume.
- Machine-to-machine actions record the workload identity (a service
account, a signed instance identity), not a human — but the
identity is still unique and attributable to a deployable unit.
## Redaction at the Boundary (P9 Secrets Redacted in Audit)
- Audit logs must not leak secrets, credentials, tokens, or PII.
Redaction is structural: applied at the logging boundary, before
the record is written to the append-only sink — not opportunistic
scrubbing after the fact. Once a secret is in an append-only log,
the remediation is expensive (rotate, rewrite access scope), so
redaction-at-source is the only sound position.
- The redaction policy is itself auditable: which fields are
redacted, by what rule, in which event type. A redaction rule
that lives in someone's head is a P9 violation waiting to happen.
```
// before redaction (DO NOT LOG)
{
"event": "config.read",
"target": { "kind": "secret", "id": "db-password" },
"value": "p@ssw0rd-plaintext-leaked" // VIOLATION
}
// after structural redaction
{
"event": "config.read",
"target": { "kind": "secret", "id": "db-password" },
"value": "[REDACTED:secret]",
"redaction": "secret-value-policy/v1"
}
```
- Never log request bodies, response bodies, headers like
`Authorization`, or environment variables that may carry secrets.
Log the *fact* of the action, not the *content* of the secret.
## Retention of Audit Logs Themselves
- Audit logs are subject to retention policy (cross `data-
retention.md`), but the floor is set by the accountability need,
not by storage economy. An audit log deleted before its retention
period is a P1 violation dressed as a P3 action.
- The retention rule for audit logs is itself logged (meta-audit):
when an audit log segment ages out and is deleted, the deletion is
recorded in a higher-tier audit log with the rule that authorized
it. The chain is observable end to end.
## What Violates Audit-Log Discipline
| Violation | Principle |
|-----------|-----------|
| Audit log on a mutable filesystem with no write-once protection | P1 Audit Logs are Append-Only |
| `DELETE FROM audit_log WHERE timestamp < ...` as routine cleanup | P1 Audit Logs are Append-Only |
| An auth-success event with no audit record | P2 Every Significant Action is Logged |
| A prose log line ("user did a thing") instead of a structured event | P2 Every Significant Action is Logged |
| A shared `admin` account as the actor in audit events | P7 Identity is Attributable |
| An `Authorization: Bearer <token>` header logged in plaintext | P9 Secrets and Sensitive Data are Redacted in Audit |
| A redaction rule applied inconsistently across event types | P9 Secrets and Sensitive Data are Redacted in Audit |
| "Who did X?" requires a custom forensics script per investigation | P10 Compliance Posture is Observable |
| An audit log segment deleted with no meta-audit record | P1 Audit Logs are Append-Only |
## Relationship to Other Domains
- `domains/observability/logging.md` — audit logs are structured
logging with an append-only guarantee; the logging primitives
(levels, structured fields, correlation IDs) compose here.
- `domains/security/authorization.md` — the actor in an audit event
is the principal the authorization layer authenticated.
- `domains/security/secrets.md` — redaction at the logging boundary
is the audit-side complement of secret management.
- `domains/compliance/data-retention.md` — retention policy applies
to audit logs; the audit log's own deletion is meta-audited.
- `domains/compliance/evidence.md` — audit logs are a primary
evidence artifact; the append-only guarantee is what makes them
admissible.
- `domains/kubernetes/rbac.md` — workload identity in audit events
derives from the RBAC principal that acted.
+159
View File
@@ -0,0 +1,159 @@
# Data Retention — Derived Rules
> Derives from `domains/compliance/first-principles.md`. Covers P3
> (Retention is Policy, Not Storage) and the data-shape angle on P8
> (Subject Access is Honored). Referenced by `audit-logs.md`
> (retention applies to audit logs) and `evidence.md` (evidence has
> a retention lifecycle). Framework-agnostic per D-024 — no
> regulation-specific retention periods.
## Retention is Policy, Not Storage (P3 Retention is Policy, Not Storage)
- Data lifetime is declared and enforced as policy, in code — not
left to the storage layer's defaults. The policy names what data
class is retained for how long, what action fires at end-of-life
(delete, archive, anonymize), and what exception path exists (a
legal hold suspends deletion).
- Deletion at end-of-life is a feature, not a failure. A system that
cannot delete on schedule is a system that over-retains, which is
the symmetric violation of a system that under-retains. Both are
P3 violations; the policy is the arbiter.
- "We kept it because the bucket was cheap" is a violation. "We
deleted it because the policy said to" is correct. Cost does not
override policy; policy is the contract.
## Retention Policy as Code
- Retention rules live as code: lifecycle rules on the storage
layer, scheduled deletion jobs, tiered storage transitions, and
anonymization transforms. The code is versioned, reviewed, and
auditable. A retention rule in a spreadsheet is a wishlist; the
same rule in a reviewed, deployable lifecycle policy is a control.
```
// illustrative lifecycle policy (abstracted, no vendor DSL)
// object-storage lifecycle
{
"rules": [
{
"name": "user-events-90d",
"match": { "prefix": "events/" },
"transitions": [
{ "after": "30d", "to": "tier-cold" },
{ "after": "90d", "action": "delete" }
]
},
{
"name": "audit-log-7y",
"match": { "prefix": "audit/" },
"transitions": [
{ "after": "365d", "to": "tier-archive" },
{ "after": "2555d", "action": "delete" }
],
"legal_hold": "suspends-action"
}
]
}
```
- The retention policy is itself auditable: which rule fired when,
against which objects, with what result. The deletion events are
logged (`audit-logs.md`) — deletion is a significant action.
## Retention vs. Backup — The Distinction
- A **backup** is a recovery mechanism: it exists to restore data
after loss. A **retention rule** is a deletion mechanism: it
exists to remove data at end-of-life. Conflating them produces
data that survives both the deletion policy and the disaster —
which is the opposite of compliance.
- A backup is governed by a recovery-point / recovery-time objective;
a retention rule is governed by a lifetime. They are independent
contracts. A backup that is also the retention store is a store
where nothing is ever deleted, which is a P3 violation.
- A legal hold suspends retention deletion for a defined data set
(e.g. data under investigation). The hold is itself a policy
action, auditable and time-bounded, not a manual override.
## Retention is Distinct per Data Class
- Different data classes have different lifetimes. The retention
policy enumerates the classes and their rules; it does not apply
one number to everything. Typical classes (the names are
abstract; the periods are policy decisions, not regulation-
specific):
- **Audit logs** — long, often multi-year, governed by
accountability needs (`audit-logs.md`).
- **User-generated content** — tied to the user's account
lifetime; deletion follows account deletion (cross P8 Subject
Access).
- **Telemetry / metrics** — short, governed by observability need
(`domains/observability/metrics.md`); high-resolution data ages
to downsampled aggregates.
- **Evidence artifacts** — tied to the audit cycle
(`evidence.md`); the cycle ends, the evidence ages out.
- A single retention rule for "all data" is a C3 (Simplicity)
violation of the wrong kind: it is simpler than the requirement
allows.
## Subject Access is Honored (P8 Subject Access is Honored)
- Data-subject rights — access (what do we have on this subject),
export (in a portable form), deletion (and prove it), correction
— are operations with defined contracts and audit trails, not
ad-hoc tickets. The system implements them as first-class
operations; a subject-access request that requires a forensics
team is a correctness defect.
- Retention and subject access interact at deletion: a subject
deletion request fires the deletion policy for that subject's
data, the deletion is audited, and the proof of deletion is
returned to the subject (and recorded). A subject deletion that
skips the audit is a P8 violation dressed as a P3 success.
- Cross `domains/data/schema-design.md`: subject access is only
computable if the schema tags which records belong to which
subject. A schema with no subject linkage cannot honor a subject
request — it cannot find the data to delete.
## Retention Migration Discipline
- Retention rules change. When the policy changes (a class's
lifetime shortens or lengthens), the change is a migration: the
new rule applies to data ingested after the cutover, and a
backfill applies the new rule to existing data where applicable.
Cross `domains/data/migrations.md` for the schema-lifecycle
discipline this mirrors.
- A retention rule change that is not versioned, not reviewed, and
not backfilled is a P3 violation: the policy is not actually the
policy if the storage layer does not reflect it.
## What Violates Retention Discipline
| Violation | Principle |
|-----------|-----------|
| Data kept indefinitely because "storage is cheap" | P3 Retention is Policy, Not Storage |
| A retention rule in a spreadsheet, not in code | P3 Retention is Policy, Not Storage |
| A backup bucket used as the retention store (nothing ever deletes) | P3 Retention is Policy, Not Storage |
| A single retention period applied to all data classes | P3 Retention is Policy, Not Storage |
| A subject deletion with no audit record of the deletion | P8 Subject Access is Honored |
| A subject-access request that requires a forensics team | P8 Subject Access is Honored |
| A schema with no subject linkage (cannot find data to delete) | P8 Subject Access is Honored |
| A legal hold applied ad hoc, not as a policy action | P3 Retention is Policy, Not Storage |
| A retention rule change with no backfill to existing data | P3 Retention is Policy, Not Storage |
## Relationship to Other Domains
- `domains/data/schema-design.md` — retention requires the schema
to tag data class and subject linkage; subject access is only
computable over a schema that supports it.
- `domains/data/migrations.md` — retention rule changes are
migrations; the discipline (version, review, backfill) mirrors
schema migrations.
- `domains/compliance/audit-logs.md` — audit logs have their own
retention floor; deletion of an audit segment is meta-audited.
- `domains/compliance/evidence.md` — evidence artifacts have a
retention lifecycle tied to the audit cycle.
- `domains/observability/metrics.md` — telemetry retention is
governed by observability need; high-res data ages to aggregates.
- `domains/security/secrets.md` — secrets have a retention lifecycle
tied to rotation; a secret past its rotation date is overdue, not
retained.
+175
View File
@@ -0,0 +1,175 @@
# Evidence — Derived Rules
> Derives from `domains/compliance/first-principles.md`. Covers P6
> (Evidence is Collected Continuously), P5 (Policy is a Gate, so
> decisions are evidence), P7 (Identity Attributable, so evidence
> has provenance), and P10 (Posture Observable, so evidence is
> queryable). Referenced by `audit-logs.md` (logs are evidence)
> and `data-retention.md` (evidence has a lifecycle).
## Evidence is Collected Continuously (P6 Evidence is Collected Continuously)
- Evidence of compliance — logs, configs, scans, attestations,
policy decisions, access reviews — is gathered as a byproduct of
operation, not assembled manually at audit time. The audit-time
scramble is the anti-pattern: it is expensive, it is incomplete,
and it produces evidence that is reconstructed rather than
recorded.
- Continuous evidence collection means the audit packet is a query
over already-collected artifacts, not a forensic reconstruction.
The auditor asks "show me the access reviews for Q3" and the
answer is a query against the evidence store, not a six-week
project.
- This is the compliance angle on `domains/observability/tracing.md`
for distributed evidence (a trace spans the request that produced
the evidence) and `domains/observability/metrics.md` for posture
signals (a metric is a continuous evidence stream).
## Evidence is a Byproduct, Not a Deliverable
- Evidence collected as a byproduct is trustworthy: it records what
happened, when it happened, recorded by the system that did it.
Evidence assembled at audit time is less trustworthy: it records
what someone remembered to write down, when they wrote it, after
the fact.
- The mechanism: every significant action (`audit-logs.md`) emits
its record to an evidence store; every policy decision
(`policy-as-code.md`) emits its decision; every deployment emits
its signed attestation; every access review emits its result. The
store is append-only (`audit-logs.md` P1), queryable (P10), and
retention-bound (`data-retention.md`).
## Provenance and Identity (P7 Identity is Attributable)
- Evidence has provenance: which system produced it, when, from what
input. An evidence artifact with no provenance is anecdote, not
evidence — it cannot be attributed to a source, so it cannot be
trusted.
- Provenance includes the identity of the producer (a workload
identity, a service account) and the chain of custody (who has
had access to the artifact since it was produced). Cross
`domains/security/authorization.md`: the producer's identity is
authenticated, not assumed.
## Signed Attestations (IDEATE-29)
- A signed attestation is evidence with a cryptographic signature
binding the artifact to its producer. The signature is the
provenance: it can be verified independently of the producer, and
it cannot be forged without the producer's key. Cross
`domains/security/supply-chain.md` for the supply-chain angle.
- Cosign (Sigstore) and in-toto are the canonical patterns: a
builder signs an artifact (container image, deployable, evidence
bundle) at production time; a verifier checks the signature at
consumption time. The signature is the evidence that the artifact
came from where it claims to have come from.
- **Illustrative signed attestation (Cosign / Sigstore format, NOT
a real signature — illustrative only, no live keys):**
```
// Cosign attest — bind an attestation to an image digest
// (illustrative; not a real signature)
$ cosign attest --type spdxjson \
--predicate sbom.spdx.json \
my-registry/app@sha256:5a3e1c...f9b2
// The attestation is stored as a signature in the registry,
// bound to the image digest. The payload is a DSSE envelope:
{
"payloadType": "application/vnd.in-toto+json",
"payload": "eyJfdHlwZSI6ImF0dGVzdGF0aW9uIn0...",
"signatures": [
{
"sig": "MEUCIQDx...illustrative-base64-signature...==",
"keyid": "cosign-key-2024-q4"
}
]
}
// The decoded payload (an in-toto statement binding the
// attestation to the image digest):
{
"_type": "https://in-toto.io/Statement/v0.1",
"predicateType": "https://spdx.dev/Document",
"subject": [
{
"name": "my-registry/app",
"digest": { "sha256": "5a3e1c...f9b2" }
}
],
"predicate": {
"SPDXID": "SPDXRef-DOCUMENT",
"creationInfo": {
"created": "2024-11-07T15:03:22Z",
"creators": ["Tool: atelier-build-pipeline"]
}
}
}
// Verification (independent of the producer):
$ cosign verify-attestation --type spdxjson \
--certificate-identity-regexp '.*atelier-build.*' \
my-registry/app@sha256:5a3e1c...f9b2
// Verification succeeded for: my-registry/app@sha256:5a3e1c...f9b2
// SBOM attestation found for subject
```
- The attestation is illustrative — the signatures and digests are
not real. The shape (DSSE envelope, in-toto statement, subject +
predicate, verify-by-identity) is what evidence-as-attestation
looks like. A real attestation carries a real signature from a
real key held by the builder.
## Audit-Ready Export
- The evidence store is queryable at any time, not only at audit
time. The audit packet is a query (a date range, a data class, a
subject) over the store; the export is a dump of the matching
artifacts with their provenance and signatures.
- An audit-ready export that requires six weeks of forensics is a
P6 violation dressed as a success: the evidence was not collected
continuously, it was reconstructed. The export should be a query
that runs in minutes, not a project that runs for weeks.
## Evidence Lifecycle
- Evidence has a retention lifecycle (`data-retention.md`): an
evidence artifact is retained for the audit cycle it supports,
then ages out. The retention rule for evidence is itself audited
(deletion of evidence is a meta-audited action, like deletion of
audit logs).
- A legal hold suspends evidence deletion for a defined set — the
same mechanism as audit-log holds.
## What Violates Evidence Discipline
| Violation | Principle |
|-----------|-----------|
| Evidence assembled by hand the week before an audit | P6 Evidence is Collected Continuously |
| An evidence artifact with no provenance (no producer, no timestamp) | P7 Identity is Attributable |
| An audit packet that requires six weeks of forensics to produce | P6 Evidence is Collected Continuously |
| An attestation with no signature (provenance asserted, not proven) | P7 Identity is Attributable |
| Evidence store not queryable between audits | P10 Compliance Posture is Observable |
| Evidence deleted before its retention period with no meta-audit | P6 Evidence is Collected Continuously |
| Policy decisions not recorded as evidence | P5 Policy is Evaluated as a Gate |
| A deployment with no signed attestation of its build provenance | P7 Identity is Attributable |
## Relationship to Other Domains
- `domains/security/supply-chain.md` — signed attestations are the
supply-chain integrity primitive; evidence.md is the compliance
consumer of the same artifact.
- `domains/observability/metrics.md` — posture metrics are a
continuous evidence stream.
- `domains/observability/tracing.md` — distributed traces provide
evidence that spans a request across services.
- `domains/compliance/audit-logs.md` — audit logs are a primary
evidence artifact; the append-only guarantee is what makes them
admissible.
- `domains/compliance/policy-as-code.md` — policy decisions are
evidence of enforcement; the policy code itself is evidence of
the rule.
- `domains/compliance/data-retention.md` — evidence has a retention
lifecycle tied to the audit cycle.
+184
View File
@@ -0,0 +1,184 @@
# Compliance — First Principles
> Framework-agnostic per D-024. These principles derive from core
> Security (a subset of C1 Correctness), Observability, and
> Reversibility. They apply across regulations — NIST CSF, SOC 2,
> GDPR, CCPA, HIPAA, PCI-DSS, ISO 27001 — without prescribing any
> regulation-specific implementation. Regulation names appear here
> only as examples of what the principles support; the principles
> themselves are engineering rules, not legal controls.
## 1. The Principles
### P1. Audit Logs are Append-Only
Audit records are immutable once written. Deletion or mutation of an
audit record is itself an auditable incident — the tampering is the
signal, not just the underlying event. An audit log that can be edited
is not an audit log; it is a draft. Append-only is enforced
structurally (write-once storage, immutable buckets, hash-chained
records), not by policy alone. This is the compliance angle on
`domains/observability/logging.md`: structured logs that cannot be
rewritten are the substrate of accountability.
### P2. Every Significant Action is Logged
The set of auditable actions is defined a priori, in code, before the
action ships — not retrofitted after an incident. Authentication
changes, authorization decisions, data access, configuration changes,
policy changes, and deletions are all significant. "We forgot to log
it" is a violation, not an excuse. The auditable-action catalog is
itself versioned and reviewed. A significant action with no log line
is a C7 (Observability) defect and a C1 (Correctness) defect: the
system's behavior is invisible, and accountability is impossible.
### P3. Retention is Policy, Not Storage
Data lifetime is declared and enforced as policy, not left to the
storage layer's defaults. Deletion at end-of-life is a feature, not a
failure. Retention rules live as code (lifecycle rules, scheduled
deletion jobs, tiered storage transitions), they are reviewed, and
they are auditable. "We kept it because the bucket was cheap" is a
violation; "we deleted it because the policy said to" is correct.
Retention is distinct from backup: a backup is a recovery mechanism,
a retention rule is a deletion mechanism. Keeping them conflated
produces data that survives both the deletion policy and the
disaster — which is the opposite of compliance. Cross
`domains/data/migrations.md` for the schema-lifecycle discipline.
### P4. Policy is Code
Compliance policy is expressed in versioned, reviewable, testable
code (OPA / Rego, AWS Cedar, HashiCorp Sentinel, Kyverno) — not in
spreadsheets, prose documents, or tribal knowledge. Policy in a
spreadsheet is untestable, unreviewable, and undeployable; it is a
wishlist, not a control. Policy-as-code inherits the disciplines of
`domains/infrastructure-as-code/P1 Declarative Intent`: declarative
intent, version control, review before merge, plan before apply. A
compliance rule that is not executable is a rule that cannot be
enforced, which is a rule that does not exist.
### P5. Policy is Evaluated as a Gate
Policy violations block before the action, not after the audit.
Enforcement happens at admission time (kubernetes admission), at
pipeline time (CI/CD gates), and at provisioning time (IaC plan
gates) — before the non-compliant state is realized. Detecting a
violation after it ships is detection, not enforcement. A policy that
is "logged but not blocked" is a postcard, not a gate. This is the
compliance angle on C5 (Reversibility): a blocked action is
reversible by construction; a shipped violation requires remediation,
which is more expensive than prevention.
### P6. Evidence is Collected Continuously
Evidence of compliance — logs, configs, scans, attestations, policy
decisions, access reviews — is gathered as a byproduct of operation,
not assembled manually at audit time. The audit-time scramble is the
anti-pattern: it is expensive, it is incomplete, and it produces
evidence that is reconstructed rather than recorded. Continuous
evidence collection means the audit packet is a query over
already-collected artifacts, not a forensic reconstruction. This is
the compliance angle on `domains/observability/tracing.md` for
distributed evidence and `domains/observability/metrics.md` for
posture signals.
### P7. Identity is Attributable
Every logged action traces to an authenticated, non-shared principal.
Shared accounts, generic service identities, and "admin" as an actor
are violations: an action with no attributable human or workload is
an action with no accountability. Identity is recorded in the audit
record at the time of the action, not resolved after the fact. This
is the compliance angle on `domains/security/authorization.md` and
`domains/kubernetes/rbac.md`: the audit subject must be the principal
that acted, not a role that many can assume.
### P8. Subject Access is Honored
Data-subject rights — access, export, deletion, correction — are
operations with defined contracts and audit trails, not ad-hoc
tickets. The system can answer "what do we have on this subject,"
"export it in a portable form," and "delete it and prove the
deletion" as first-class operations. These are not features bolted on
at the end; they are contracts the data layer implements from the
start. A subject-access request that requires a forensics team is a
correctness defect: the system does not know what it holds. Cross
`domains/data/schema-design.md` for the data shapes that make
subject access computable.
### P9. Secrets and Sensitive Data are Redacted in Audit
Audit logs themselves must not leak secrets, credentials, PII, or
other sensitive data. Redaction is structural — applied at the
logging boundary, before the record is written — not opportunistic
scrubbing after the fact. A secret that appears in an audit log is a
C1 (Correctness) violation (the log is now a secret store) and a
security violation (`domains/security/secrets.md`). The redaction
policy is itself auditable: which fields are redacted, by what rule,
in which log stream. Once a secret is in an append-only log, the
remediation is expensive — rotate the secret and rewrite the log's
access scope — so redaction-at-source is the only sound position.
### P10. Compliance Posture is Observable
The system reports its own compliance state: drift from policy, open
violations, retention status, evidence freshness, policy-evaluation
counts. Silent non-compliance is the bug. A compliance posture
metric is a first-class signal (`domains/observability/metrics.md`),
alertable, and dashboarded. "We didn't know we were non-compliant"
is not a defense; it is a C7 (Observability) defect. The posture is
queryable at any time, not only at audit time. This is the compliance
angle on `domains/infrastructure-as-code/P3 State is Truth`: the
compliance state is a versioned, queryable truth, not a vibe.
## 2. Core Principle Trace
Each compliance P-rule derives from one or more core C-rules (C1C8).
The matrix extension lands in P4 of the v0.3 plan; the traces below
are authoritative.
| P-rule | Core | Why |
|--------|------|-----|
| P1 Audit Logs are Append-Only | C1, C5 | Correctness of the record; reversibility of tamper detection |
| P2 Every Significant Action is Logged | C7, C1 | Observability of behavior; correctness of a-priori audit scope |
| P3 Retention is Policy, Not Storage | C5, C8 | Reversibility of data lifetime; economy of storage as policy |
| P4 Policy is Code | C6, C2 | Composability of versioned policy; clarity of executable intent |
| P5 Policy is Evaluated as a Gate | C1, C5 | Correctness of pre-action enforcement; reversibility of blocked actions |
| P6 Evidence is Collected Continuously | C7, C3 | Observability of compliance state; simplicity of audit-by-query |
| P7 Identity is Attributable | C1, C7 | Correctness of accountability (security subset); observability of who acted |
| P8 Subject Access is Honored | C1, C5 | Correctness of the data-subject contract; reversibility of deletion |
| P9 Secrets and Sensitive Data are Redacted in Audit | C1, C3 | Correctness of not leaking (security subset); simplicity of structural redaction |
| P10 Compliance Posture is Observable | C7, C1 | Observability of posture; correctness of self-reported state |
## 3. What Violates These Principles
| Violation | Principle Breached |
|-----------|-------------------|
| An audit log stored on a mutable filesystem with no write-once protection | P1 Audit Logs are Append-Only |
| A `DELETE` on an audit record to "clean up a typo" | P1 Audit Logs are Append-Only |
| An auth change with no audit log line | P2 Every Significant Action is Logged |
| "We'll add logging after we ship the feature" | P2 Every Significant Action is Logged |
| Data kept indefinitely because "the bucket is cheap" | P3 Retention is Policy, Not Storage |
| A retention rule in a spreadsheet, not in code | P4 Policy is Code |
| A policy that logs violations but does not block the action | P5 Policy is Evaluated as a Gate |
| Evidence assembled by hand the week before an audit | P6 Evidence is Collected Continuously |
| A shared `admin` account as the audit actor | P7 Identity is Attributable |
| A subject-access request that requires a forensics team | P8 Subject Access is Honored |
| A secret visible in an audit log entry | P9 Secrets and Sensitive Data are Redacted in Audit |
| No dashboard for compliance posture between audits | P10 Compliance Posture is Observable |
## 4. Relationship to Other Domains
Compliance is the accountability layer that crosses
`domains/security/` (it audits security actions),
`domains/observability/` (audit logs are structured logging; posture
is metrics; evidence is traces), `domains/data/` (retention and
subject access are data-layer contracts), and
`domains/infrastructure-as-code/` (policy-as-code parallels
declarative IaC; compliance state parallels state-as-truth). Cross-
links are one-directional (per D-026 extended):
- `domains/security/authorization.md` ← P7 (attributable identity)
- `domains/security/secrets.md` ← P9 (redaction)
- `domains/security/supply-chain.md` ← P6 (signed attestations as evidence)
- `domains/observability/logging.md` ← P1, P2 (audit logs = structured logging)
- `domains/observability/metrics.md` ← P10 (compliance posture metrics)
- `domains/observability/tracing.md` ← P6 (evidence from distributed traces)
- `domains/data/schema-design.md` ← P3, P8 (retention and subject-access shapes)
- `domains/data/migrations.md` ← P3 (retention migration discipline)
- `domains/infrastructure-as-code/P1 Declarative Intent` ← P4 (policy-as-code)
- `domains/infrastructure-as-code/P3 State is Truth` ← P10 (compliance posture truth)
- `domains/kubernetes/rbac.md` ← P7 (audit subject identity)
- `domains/devops/ci-cd.md` ← P5 (policy as a pipeline gate)
- `domains/devops/first-principles.md` ← P4 (policy as configuration-as-code)
+140
View File
@@ -0,0 +1,140 @@
# Policy as Code — Derived Rules
> Derives from `domains/compliance/first-principles.md`. Covers P4
> (Policy is Code) and P5 (Policy is Evaluated as a Gate). Referenced
> by `audit-logs.md` (policy decisions are audited) and `evidence.md`
> (policy decisions are evidence). Framework-agnostic per D-024.
## Policy is Code (P4 Policy is Code)
- Compliance policy is expressed in versioned, reviewable, testable
code — not in spreadsheets, prose documents, or tribal knowledge.
Policy in a spreadsheet is untestable, unreviewable, and
undeployable; it is a wishlist, not a control.
- Policy-as-code inherits the disciplines of
`domains/infrastructure-as-code/P1 Declarative Intent`: declarative
intent, version control, review before merge, plan before apply.
A compliance rule that is not executable is a rule that cannot be
enforced, which is a rule that does not exist.
- Policy code is tested like any other code: unit tests for the rule
logic (given an input, the rule allows or denies as expected),
integration tests for the gate (the rule fires at the right point
in the pipeline), and versioning for the policy itself (a policy
change is a reviewed, merged, deployed change).
## Policy is Evaluated as a Gate (P5 Policy is Evaluated as a Gate)
- Policy violations block **before** the action, not after the
audit. Enforcement happens at:
- **Admission time** — a kubernetes admission webhook denies a
non-compliant resource before it is created
(`domains/kubernetes/rbac.md`).
- **Pipeline time** — a CI/CD gate denies a non-compliant change
before it merges (`domains/devops/ci-cd.md`).
- **Provisioning time** — an IaC plan gate denies a non-compliant
resource before `apply` (`domains/infrastructure-as-code/`).
- A policy that logs violations but does not block the action is a
postcard, not a gate. Detection is not enforcement. A logged
violation that the actor could ignore is a P5 violation — the
policy exists, but the system is not compliant by construction.
- The gate is the contract. The policy author writes the rule; the
gate operator wires the rule into the enforcement point; the
auditor verifies the gate fired. All three are auditable
(`audit-logs.md`).
## Engine Comparison (IDEATE-23)
| Engine | Policy Language | Evaluation Gate | Ecosystem | Notes |
|--------|-----------------|-----------------|-----------|-------|
| **OPA / Rego** | Rego (declarative, set-based, Datalog-inspired) | CI/CD, k8s admission (Gatekeeper), HTTP API, IaC plan (Terraform Sentinel-style), service mesh | Broadest ecosystem; CNCF graduated; library of reusable bundles | General-purpose; the default choice when the gate location varies |
| **AWS Cedar** | Cedar (declarative, authorization-focused, schema-typed) | k8s admission (via Cedar-agent), application authorization, AVP (Verified Permissions) | AWS-native; tight schema typing; separates policy from entities | Authorization-focused; strong where the policy is "who can do what on which resource" |
| **HashiCorp Sentinel** | Sentinel (declarative, restricted, policy-focused) | Terraform / TFE plan gate, Nomad, Vault | HashiCorp ecosystem; embedded in Terraform Enterprise / HCP | IaC-plan-gate native; the enforcement point is the `plan` output |
| **Kyverno** | Kyverno (YAML-declarative, k8s-native, no new DSL) | k8s admission (native), cluster-wide policy reports | Kubernetes-native; no separate language — policy is a CRD | k8s-cluster-gate native; the choice when the gate is admission and the team prefers YAML over a DSL |
- None is advocated over the others. The choice is (a) where the
gate fires, (b) the team's tolerance for a new policy language,
and (c) ecosystem fit. All four satisfy P4/P5 when wired
correctly.
- A gate is a gate regardless of engine: the rule is declarative,
the evaluation is pre-action, and the decision is allow-or-deny.
The engine difference is language and enforcement-point fit, not
correctness.
## Policy Testing
- Policy code is unit-tested like any other code. A test asserts
that a given input produces the expected decision (allow / deny /
warn). The test is versioned with the policy; a policy change
with no test change is a red flag.
```
// illustrative Rego policy + test
// policy: deny containers running as root
package k8s.admission
deny[msg] {
input.kind == "Pod"
c := input.spec.containers[_]
not c.securityContext.runAsNonRoot
msg := sprintf("container %s must set runAsNonRoot", [c.name])
}
// test (Rego unit test)
package k8s.admission
test_deny_root_container {
some msg in deny with input as {
"kind": "Pod",
"spec": { "containers": [ { "name": "app", "securityContext": {} } ] }
}
msg == "container app must set runAsNonRoot"
}
```
- Integration tests assert the gate fires: a non-compliant resource
submitted to the admission endpoint is denied; a compliant one is
allowed. The integration test runs against the real gate, not a
mock, because the gate wiring is half the contract.
## Policy Versioning
- Policy is versioned in git. A policy change is a reviewed, merged,
deployed change — the same discipline as application code. A
policy that is edited in production without review is a P4
violation: the policy is code, but it is being treated as config.
- A policy change can break existing workloads (a new deny rule
blocks a previously-allowed resource). The rollout is staged:
warn-only mode first (log violations, do not block), then enforce
mode after the violation count is zero. This is the policy
analogue of `domains/devops/P5 Progressive Delivery`.
## What Violates Policy-as-Code Discipline
| Violation | Principle |
|-----------|-----------|
| A compliance rule in a spreadsheet | P4 Policy is Code |
| A policy that logs violations but does not block the action | P5 Policy is Evaluated as a Gate |
| A policy edited in production without review | P4 Policy is Code |
| A policy with no unit tests for the rule logic | P4 Policy is Code |
| A gate wired with a mock instead of the real engine | P5 Policy is Evaluated as a Gate |
| A new deny rule enforced without a warn-only rollout | P5 Policy is Evaluated as a Gate |
| A policy in prose ("the team should not use root containers") | P4 Policy is Code |
| A policy decision with no audit record | P5 Policy is Evaluated as a Gate |
## Relationship to Other Domains
- `domains/infrastructure-as-code/first-principles.md` — policy-as-
code inherits declarative intent, versioning, and plan-before-
apply from IaC.
- `domains/kubernetes/rbac.md` — k8s admission is a primary
enforcement gate; Kyverno and OPA Gatekeeper wire into it.
- `domains/devops/ci-cd.md` — CI/CD is a pipeline-time enforcement
gate; a policy step blocks a non-compliant change before merge.
- `domains/compliance/audit-logs.md` — every policy decision (allow
/ deny) is an audited significant action.
- `domains/compliance/evidence.md` — policy decisions and the
policy code itself are evidence of enforcement posture.
- `domains/security/authorization.md` — Cedar's authorization-
focused policy overlaps with authz; the split is that authz is
the runtime decision, policy-as-code is the reviewed rule that
drives it.
+41
View File
@@ -0,0 +1,41 @@
# Concurrency — First Principles
## 1. The Principles
### P1. Immutability by Default
Mutable shared state is the enemy. The default is immutable; mutation
is justified.
### P2. Single Responsibility for Threads
Each unit of work has one owner. No "anyone can touch this" state.
### P3. Boundaries are Locks
Synchronization happens at well-defined places. Lock scope is
minimal and explicit.
### P4. Determinism Over Speed
Correct concurrent code is faster than incorrect concurrent code.
Race conditions are not "fast enough" — they are wrong.
### P5. Lock Minimization
Locks are expensive and dangerous. Lock-free, wait-free, and
message-passing are preferred where possible.
### P6. No Silent Races
Race conditions are caught, not hidden. Tools (TSan, Go race
detector) are part of CI.
### P7. Cancellation Support
Every async operation can be cancelled. Cancellation is fast and
complete.
### P8. Timeout Discipline
Every blocking call has a timeout. Forever is not a duration.
### P9. Bounded Queues
Unbounded queues are memory leaks in disguise. Bounded queues
expose backpressure.
### P10. Test for Race Conditions
Concurrent code is tested under concurrent load, not just happy-path
correctness.
+62
View File
@@ -0,0 +1,62 @@
# Concurrency Patterns — Derived Rules
> Derives from `domains/concurrency/first-principles.md`. Common concurrency patterns and when to use them.
## Pattern 1: Message Passing (P5 Lock Minimization)
- Threads/goroutines communicate via channels/queues, not shared memory.
- "Don't communicate by sharing memory; share memory by communicating." (Go proverb)
- Use when the data flows naturally in one direction. Avoids locks entirely.
## Pattern 2: Read-Write Lock (P5 Lock Minimization)
- Multiple readers, one writer. A `RwLock` allows concurrent reads, exclusive writes.
- Use when reads vastly outnumber writes (e.g., a config cache).
- Avoid when writes are frequent — the lock degrades to a mutex.
## Pattern 3: Actor Model (P2 Single Responsibility)
- Each actor owns its state. Actors communicate via messages. No shared state.
- Use for isolated, long-lived workers (e.g., a session handler, a chat room).
- Erlang/Akka/Pony are built on this. Implementable in any language with channels.
## Pattern 4: Immutable Data Structures (P1 Immutability by Default)
- Data is never mutated; a "change" creates a new value. Old values are safe to share.
- Use in functional languages (Haskell, Clojure) or via persistent data structures (Immer.js).
- Eliminates entire classes of races. The trade-off is allocation cost.
## Pattern 5: Bounded Queue with Backpressure (P9 Bounded Queues)
- A queue with a max size. When full, the producer is blocked or signaled.
- Use to bound memory and propagate slowness from consumer to producer.
- An unbounded queue hides a slow consumer until OOM. Always bound.
## Pattern 6: Timeout on Every Block (P8 Timeout Discipline)
- Every blocking call (lock acquire, queue send, HTTP request) has a timeout.
- Use a timeout, not a forever-block. Forever is not a duration.
- On timeout: cancel, retry, or fail. Do not hang.
## Pattern 7: Cancellation Propagation (P7 Cancellation Support)
- A cancellation signal propagates to all spawned work. Cancel the parent, the children stop.
- Use context (`context.Context` in Go, `AbortController` in JS, `CancellationToken` in C#).
- Cancellation is fast and complete. No orphaned goroutines/threads.
## Pattern 8: Lock-Free Where Possible (P5)
- Atomic operations (compare-and-swap) for simple state. No lock.
- Use for counters, flags, simple pointers.
- Avoid for complex state — lock-free code is subtle and easy to get wrong.
## What Violates Concurrency Patterns
| Violation | Pattern |
|-----------|---------|
| Shared mutable state with no lock | (race, P1) |
| Unbounded queue | P5 (OOM) |
| `channel.send()` with no timeout | P6 (hang) |
| Spawned goroutine with no cancellation | P7 (orphan) |
| A mutex held across an I/O call | P3 (lock scope) |
| `sync.Mutex` for a counter | P8 (use atomic) |
+43
View File
@@ -0,0 +1,43 @@
# Data — First Principles
## 1. The Principles
### P1. Truth
The schema reflects the domain, not the application. If the data
model lies, every query lies.
### P2. Normalization Discipline
Duplication is a bug waiting to happen. The same fact lives in one
place.
### P3. Invariants in the Schema
Constraints live where the data lives. Application-layer checks are
defense, not enforcement.
### P4. Migration Safety
Schema changes are reversible, non-destructive, and tested. Production
data is sacred.
### P5. Indexing with Intent
Indexes exist for known query patterns. Every index earns its write
cost.
### P6. Naming Consistency
Same concept, same name, always. Across tables, columns, code, and
APIs.
### P7. Type Fidelity
Types match domain meaning. A `string` is rarely the right type for
an email, an ID, or a status.
### P8. Lifecycle Awareness
Data has a creation, a lifetime, and an end. Archival and deletion
are first-class.
### P9. Referential Integrity
Relationships are enforced, not assumed. Foreign keys exist. CASCADE
is intentional.
### P10. Performance Awareness
Schema choices have cost. Query plans are reviewed. Cardinality is
understood.
+49
View File
@@ -0,0 +1,49 @@
# Indexing — Derived Rules
> Derives from `domains/data/first-principles.md` P5 (Indexing with Intent), P10 (Performance Awareness).
## Index for Queries, Not Tables (P5)
- An index serves a query. No query, no index.
- The query plan is the spec. `EXPLAIN` is the test. An index that is not used is dead weight.
- Index the columns you filter on (`WHERE`), join on (`JOIN`), and sort on (`ORDER BY`).
## Composite Indexes (P5, P10)
- Order matters: `INDEX(a, b)` serves `WHERE a = ? AND b = ?` and `WHERE a = ?`, but NOT `WHERE b = ?`.
- Put the most selective column first. Or the column used in every query. Depends on the workload.
- An index on every column is not a strategy. It is write amplification.
## Unique Indexes (P3 Invariants in Schema)
- A uniqueness constraint is a unique index. Use it for invariants: `email`, `username`.
- Unique indexes enforce; application checks defend. Both belong.
- A partial unique index: `UNIQUE(email) WHERE deleted_at IS NULL` — allows soft-deleted duplicates.
## Covering Indexes (P10)
- An index that covers all columns of a query is an "index-only scan" — no table lookup.
- PostgreSQL: `INCLUDE` clause. MySQL: all columns in the index.
- Use for hot queries. Don't cover everything; index size matters.
## Don't Over-Index (P10, core C8 Economy)
- Every index costs a write. The write budget is the index count.
- Indexes take disk and memory. A 1GB index on a 500MB table is a smell.
- Remove unused indexes. `pg_stat_user_indexes` shows usage. An unused index is debt.
## Migration and Indexes (P4 Migration Safety)
- Adding an index on a large table is expensive. Do it concurrently (`CREATE INDEX CONCURRENTLY`).
- An index migration that locks the table blocks writes. Plan for it.
- Build the index, then deploy the query that uses it. Not the reverse.
## What Violates Indexing Discipline
| Violation | Principle |
|-----------|-----------|
| Index on every column | P10, C8 Economy |
| No index on a foreign key | P10 (join performance) |
| `WHERE b = ?` with only `INDEX(a, b)` | P5 (wrong order) |
| Index created without checking the query plan | P5 (no intent) |
| `CREATE INDEX` (non-concurrent) on a 10M-row table in prod | P4 Migration Safety |
+53
View File
@@ -0,0 +1,53 @@
# Migrations — Derived Rules
> Derives from `domains/data/first-principles.md` P4 (Migration Safety), P5 (Reversibility via core C5).
## Every Change is a Migration (P4)
- No manual schema changes. No `ALTER TABLE` in a shell. Every change is a versioned migration file.
- Migrations are code: reviewed, tested, committed.
- The migration tool is the only way to change the schema (`prisma migrate`, `alembic`, `flyway`, `golang-migrate`).
## Forward and Reverse (P5 Reversibility, core C5)
- Every migration has an `up` and a `down`. The `down` reverses the `up`.
- A migration without a `down` is irreversible. Irreversible migrations are rare and flagged.
- Test the `down` in CI. A `down` that fails is a migration that cannot be rolled back.
## Expand, Migrate, Contract (P5)
For non-breaking schema changes:
1. **Expand**: add the new column/ table (nullable, no constraint). Deploy. Old code still works.
2. **Migrate**: backfill data, run the data migration. Deploy. Both old and new code work.
3. **Contract**: add constraints, remove the old column. Deploy after all code uses the new schema.
Never do all three in one migration. Each step is its own deploy.
## Avoid Destructive Changes (P4)
- Never `DROP COLUMN` in a migration that could be in use. Expand-contract first.
- Never `DROP TABLE` without confirming no code references it.
- Never `ALTER TYPE` in a way that locks the table on a large dataset. Use a phased approach.
## Backward Compatibility (P5, P1)
- A migration must not break the running code. Old code reads the new schema (with expand).
- The schema is always compatible with the previous code version. Two-version compatibility.
- A breaking migration is deployed in lockstep with the code, with a maintenance window.
## Testing Migrations (P3 Determinism via testing P3)
- Run migrations on a copy of production data in CI. A migration that works on dev may fail on prod scale.
- Test the `down` on the migrated state, not just the `up`.
- Test with the largest table sizes you have. `ALTER TABLE` on 10 rows is fast; on 10M rows, it may lock.
## What Violates Migration Safety
| Violation | Principle |
|-----------|-----------|
| Manual `ALTER TABLE` in prod | P4 |
| Migration with no `down` | P5 Reversibility |
| `DROP COLUMN` in the same deploy as the new code | P4, P5 |
| No migration test on prod-scale data | P3 Determinism |
| A migration that locks a table for 10 minutes | P4 (downtime) |
+53
View File
@@ -0,0 +1,53 @@
# Schema Design — Derived Rules
> Derives from `domains/data/first-principles.md` P1 (Truth), P3 (Invariants in Schema), P7 (Type Fidelity).
## The Schema Reflects the Domain (P1 Truth)
- A `users` table has columns that are attributes of a user, not attributes of the application.
- If a column is named `is_active_for_feature_X`, the schema is lying. The domain does not have "feature X."
- Normalize until it hurts, then denormalize only with evidence (P10 Performance Awareness).
## Invariants in the Schema (P3)
- NOT NULL where the value is required. UNIQUE where the value is unique.
- CHECK constraints for range/domain: `age >= 0`, `status IN ('draft', 'published')`.
- FOREIGN KEY for relationships. The database enforces; the application defends.
- A constraint in the application but not the schema is a constraint that can be bypassed.
## Types (P7 Type Fidelity)
- `UUID` for IDs, not `VARCHAR`. `UUID` is a type; `VARCHAR(36)` is a string that looks like a UUID.
- `TIMESTAMPTZ` for timestamps, not `VARCHAR` or `INTEGER`. Timezone-aware by default.
- `ENUM` for finite domains, `VARCHAR` with CHECK for evolving domains.
- `JSONB` for unstructured/semi-structured; not for data that should be a column.
- `DECIMAL`/`NUMERIC` for money, never `FLOAT`. Floating point is for measurements, not money.
## Naming (P6 Naming Consistency)
- snake_case for tables and columns (PostgreSQL convention): `user_accounts`, `created_at`.
- Singular table names (`user` not `users`) OR plural (`users` not `user`) — pick one, be consistent.
- Foreign keys: `<singular_table>_id` (`user_id`), not `uid` or `user`.
- Junction tables: alphabetical (`order_products`, not `products_orders`).
## Avoid (P2 Normalization Discipline)
- Computed columns that duplicate derivable data. Use a view or compute on read.
- `created_by_name` (denormalized) when `created_by_id` + JOIN suffices. Denormalize only with evidence.
- Soft-delete columns (`is_deleted`) without a corresponding constraint/behavior. Soft delete is a lifecycle decision (P8).
## Soft Delete vs Hard Delete (P8 Lifecycle Awareness)
- Soft delete (`deleted_at TIMESTAMP`) preserves auditability but complicates every query.
- Hard delete loses history. Choose based on the domain's legal/audit requirements.
- If soft delete: every query filters `WHERE deleted_at IS NULL` by default. A missing filter is a bug.
## What Violates Schema Design
| Violation | Principle |
|-----------|-----------|
| `VARCHAR` for a UUID | P7 Type Fidelity |
| No FOREIGN KEY on a relationship | P3, P9 Referential Integrity |
| `FLOAT` for money | P7, P1 Truth |
| `is_deleted` without consistent filtering | P8 Lifecycle |
| A column named after a feature, not a domain concept | P1 Truth |
+50
View File
@@ -0,0 +1,50 @@
# CI/CD — Derived Rules
> Derives from `domains/devops/first-principles.md` P2 (Automation), P4 (Rollback First), P5 (Progressive Delivery).
## The Pipeline is the Process (P2)
- If it is not in the pipeline, it does not happen. Manual deploys are a bug.
- The pipeline: lint → test → build → deploy → verify.
- Every step is scripted, versioned, and reproducible. No "run this command on the server."
## Lint (P2, core C2 Clarity)
- Run the linter on every commit. Fail the build on lint errors.
- Format check (prettier, gofmt, rustfmt). Format is not a debate; it is automated.
- Security lint (eslint-plugin-security, bandit, gosec). Catch the obvious ones.
## Test (P2, domains/testing)
- Unit tests in the pipeline. Fast. Every commit.
- Integration tests on merge to main. Slower. Every merge.
- E2E tests before deploy. Slowest. Every deploy candidate.
## Build (P7 Immutability)
- Build once. The artifact is immutable. The same artifact goes to every environment.
- The build is reproducible: same commit → same artifact (modulo timestamps, which are stripped).
- Build artifacts are signed and stored. A deploy is a reference to an artifact, not a rebuild.
## Deploy (P4 Rollback First, P5 Progressive Delivery)
- Every deploy has a rollback. The rollback is tested before the deploy.
- Progressive: canary (1% → 10% → 100%), blue-green, or feature flags.
- No big-bang deploys. A big-bang deploy is a rollback with no rehearsal.
## Verify (P3 Observability)
- After deploy, verify: health checks, smoke tests, metric watching.
- A deploy is not "done" when the code is on the server. It is done when the metrics are healthy.
- Auto-rollback on metric regression. The pipeline watches; humans sleep.
## What Violates CI/CD Discipline
| Violation | Principle |
|-----------|-----------|
| Manual deploy script | P2 Automation |
| No rollback path | P4 Rollback First |
| Big-bang deploy to prod | P5 Progressive Delivery |
| Rebuild per environment | P7 Immutability |
| Deploy without health check | P3 Observability |
| No lint in CI | P2, C2 |
+44
View File
@@ -0,0 +1,44 @@
# Environments — Derived Rules
> Derives from `domains/devops/first-principles.md` P1 (Reproducibility), P6 (Configuration as Code), P7 (Immutability).
## Environment Parity (P1 Reproducibility)
- Dev, staging, prod are the same system, different data.
- The same artifact runs in all three. The same config schema, different values.
- "Works on my machine" is a parity failure. The machine is the pipeline.
## Configuration (P6 Configuration as Code)
- Config is in the repo (default values) + environment overrides (secrets, endpoints).
- No snowflake servers. No "this one is different because we edited it in prod."
- Config changes are PRs, not SSH sessions.
## Secrets (P9 Secret Hygiene via security)
- Secrets are per-environment. Dev secrets ≠ prod secrets.
- Secrets come from a secrets manager (Vault, AWS Secrets Manager, Doppler), not env files in prod.
- `.env` files are for local dev only. Prod uses the manager.
## Promotion (P5 Progressive Delivery via devops)
- Code moves dev → staging → prod. Never the reverse.
- A hotfix to prod is backported to staging and dev. Don't let them diverge.
- Promotion is automated. The pipeline decides when code is ready, not a human.
## Data (P1, domains/data P8 Lifecycle Awareness)
- Prod data is sacred. Never copy prod to dev without anonymization.
- Staging uses prod-like data (anonymized, sampled). Dev uses synthetic data.
- A test that runs against prod data is a test that can destroy prod data. Don't.
## What Violates Environment Discipline
| Violation | Principle |
|-----------|-----------|
| "It works on my machine" | P1 Parity |
| Manual config edit in prod | P6 Configuration as Code |
| Dev secret reused in prod | P9 Secret Hygiene |
| Copy prod DB to dev | P1, data P8 |
| Hotfix in prod not backported | P5 (divergence) |
| A snowflake server | P1, P6 |
+43
View File
@@ -0,0 +1,43 @@
# DevOps — First Principles
## 1. The Principles
### P1. Reproducibility
Any environment can be rebuilt from source. Configuration is
declarative and versioned.
### P2. Automation
Manual steps are bugs waiting to happen. Everything is scripted,
testable, repeatable.
### P3. Observability
You cannot operate what you cannot see. Logs, metrics, and traces
are first-class.
### P4. Rollback First
Every deploy has a known-good rollback path. Rollback is rehearsed,
not improvised.
### P5. Progressive Delivery
Changes go out gradually — canary, blue-green, feature flags.
Big-bang deploys are for prototypes.
### P6. Configuration as Code
No snowflake servers. No "this one is different". Configuration is
in the repo.
### P7. Immutability
Build once, deploy many. Artifacts are immutable. Servers are not
mutated in place.
### P8. Security at Every Layer
Scanning, signing, SBOM, and supply chain integrity are part of the
pipeline, not bolt-ons.
### P9. Documentation in the Pipeline
The pipeline is the documentation. Reading the pipeline tells you
how the system ships.
### P10. Failure as Expected
Design for the failure mode, not the happy path. Chaos engineering
is a discipline.
+83
View File
@@ -0,0 +1,83 @@
# Doc Templates — Derived Rules
> Derives from `domains/documentation/first-principles.md` P7 (Structure), P2 (Audience Awareness), P3 (Examples are Mandatory).
## Document Structure (P7)
Every framework document follows a consistent structure:
```
# <Title>
> One-line purpose. Who reads this and when.
## 1. Manifesto (or Introduction)
Why this document exists. The core belief.
## 2. The Principles (or Rules)
The numbered, named, derivable rules. Each rule has:
- A name (P1, P2, ...)
- A one-line definition
- A "what it means" paragraph
- A "what violates it" entry
## 3. Conflict Resolution (for first-principles docs)
Precedence among the rules. Non-tradeable declarations.
## 4. What Violates These Principles
A table of violations and the principle they breach.
## 5. Relationship to Core
Derivation link. See `matrix/principles-matrix.md`.
```
- The structure is the contract. A reader can scan any framework doc and find the same sections.
- Derived (topic) docs simplify: drop §3, replace §5 with "Derives from `<domain>/first-principles.md`."
## Audience Templates (P2)
### For Agents
- Lead with what to check before completing a task.
- Bullet lists, not paragraphs.
- "Run this checklist" framing.
### For Humans (Onboarding)
- Lead with what this is and who it's for.
- Reading order. Quickstart.
- Conversational tone, not terse.
### For Humans (Reference)
- Lead with the rules, indexed.
- Tables for lookup.
- Cross-references to other docs.
## Example Template (P3 Examples are Mandatory)
Every rule includes an example. The template:
```
### P<n>. <Name>
<one-line definition>
<what it means>
Good:
<example>
Bad:
<counter-example> (violates P<n>)
```
- The "good" example is realistic, not a strawman.
- The "bad" example cites the principle it violates.
- Examples are code, not prose. Show, don't tell.
## What Violates Doc Templates
| Violation | Principle |
|-----------|-----------|
| A doc with no examples | P3 Examples are Mandatory |
| A first-principles doc with no "what violates" table | P7 Structure |
| A doc that does not link to its core derivation | P5 Discoverability |
| Inconsistent structure across domain docs | P7 |
| A doc with no audience statement | P2 Audience Awareness |
+37
View File
@@ -0,0 +1,37 @@
# Documentation — First Principles
## 1. The Principles
### P1. Documentation is Code
It is versioned, reviewed, tested, and owned. Unowned docs rot.
### P2. Audience Awareness
Different readers need different docs. A new user, an operator, and
a contributor are different audiences.
### P3. Examples are Mandatory
Code without examples is incomplete. Show, then explain.
### P4. Currency
Docs that lie are worse than no docs. Stale docs are technical debt.
### P5. Discoverability
The right doc is findable in under a minute. Structure, search, and
indexing are part of the doc.
### P6. Conciseness
Say what is needed, no more. Verbose docs are skimmed, then ignored.
### P7. Structure
Consistent structure aids scanning. Headings, ordering, and
formatting follow conventions.
### P8. Why Over What
Document intent, decisions, and tradeoffs. The "what" is in the code.
### P9. Living Documents
Docs evolve with code, not after. Doc PRs ship with code PRs.
### P10. Public by Default
If it is not documented, it does not exist. The absence of docs is
a feature gap.
+227
View File
@@ -0,0 +1,227 @@
# CDN — Derived Rules
> Derives from `domains/edge/first-principles.md`. Applies P1
> (Proximity is the Design Driver) and P6 (Cache Invalidation is
> Explicit) primarily, with P5 (idempotent cache fill), P8
> (geographic distribution), and P9 (identity at the edge). For the
> edge-cache-vs-origin decision, see the decision matrix below.
> Cross-links `domains/performance/frontend` for generic caching,
> `domains/security/input-validation` for cache poisoning, and
> `domains/observability/metrics` for cache-hit ratio.
## What a CDN Is (P1 Proximity is the Design Driver)
- A content delivery network is a fleet of PoPs (points of presence)
placed near users. The PoP serves cached content; the origin is
the authoritative source. The CDN's whole purpose is P1: compute
(the cache) is placed near the user so the round trip to the origin
does not bound latency. Latency is a correctness constraint at the
edge (C1), not a performance preference.
- The CDN is the canonical edge-cache architecture (Akamai,
Cloudflare, Fastly): PoPs near users, origin shielding, cache-key
normalization, purge APIs. Atelier derives the
placement/invalidation principles, not the vendor config.
- The boundary with `domains/performance/frontend` is per D-061:
performance owns *generic* caching and optimization (cache what is
expensive, stable, read often — `performance/P5 Caching with
Intent`); edge owns the *geographic, partition-aware* placement and
invalidation angle. A CDN is an edge concern because its defining
trait is geographic distribution (P8) and partition-aware
invalidation (P6), not measurement.
## Cache Key Design (P6 Cache Invalidation is Explicit)
- The cache key is the contract between the URL and the cached
representation. A key that varies on the wrong dimensions serves
the wrong content; a key that varies on too many dimensions
collapses the hit ratio. Key design *is* the invalidation
surface: a key that includes a content hash or version segment
makes invalidation explicit; a key that ignores `Vary` headers
serves stale variants.
- Normalize the key: lower-case the host, strip default ports,
sort query parameters, ignore tracking parameters. A
non-normalized key is a cache-poisoning vector (see
`domains/security/input-validation`) and a hit-ratio destroyer
(see `domains/observability/metrics`).
- A cache with no explicit key strategy is a TTL-less cache under
partition (P6 violation): staleness is silent and unbounded.
```http
# Cache key derivation: vary on what changes content, ignore what
# does not. The key is the tuple (host, normalized-path, sorted-
# query, Vary-headers); the cache entry is the representation + TTL.
Cache-Key: example.com /api/v1/products?sort=price&region=us Vary:Accept-Encoding
Cache-Control: public, max-age=60, s-maxage=600, stale-while-revalidate=300
Vary: Accept-Encoding
```
- `max-age` bounds the browser cache; `s-maxage` bounds the CDN
PoP; `stale-while-revalidate` allows serving stale while
refetching. Each is an explicit invalidation strategy (P6).
## TTL vs Explicit Invalidation (P6, C3 Simplicity)
- **TTL-based invalidation** (`max-age`, `s-maxage`): the cache entry
expires after a duration. Simple, no origin contact required to
invalidate, but bounded staleness is the contract — the entry may
be stale up to TTL. Fits content where eventual consistency is
acceptable (asset fingerprints, lists, derived images).
- **Explicit invalidation** (purge, surrogate keys): the operator
signals the cache to drop entries. Tighter staleness bounds, but
requires the origin or operator to know which entries to purge.
Fits content where staleness is a correctness defect (price
updates, availability, breaking news).
- A TTL-less cache with no explicit invalidation is a P6 violation:
stale-forever under partition. Every cache must have one or the
other (or both), and the choice is documented per content type.
## Cache-Hit / Miss / Origin-Fetch (P5, P6)
- **Hit**: the PoP serves from cache. Latency is PoP-local (P1).
- **Miss**: the PoP has no entry; it fetches from the origin (or an
origin-shield PoP). The fetch must be idempotent (P5) — a retried
miss must not corrupt the cache or double-write side effects.
- **Revalidate**: the PoP holds a stale entry and asks the origin
(`If-None-Match`, `If-Modified-Since`); a 304 refreshes the TTL
without re-fetching the body. Revalidation is the bandwidth-economical
middle ground (C8).
```http
# Conditional revalidation — the PoP asks the origin "is this still
# current?" The 304 response refreshes the TTL without a body.
GET /api/v1/products HTTP/1.1
Host: example.com
If-None-Match: "etag-7a3f"
HTTP/1.1 304 Not Modified
ETag: "etag-7a3f"
Cache-Control: s-maxage=600
```
- A cache-hit ratio that is not measured is a gate on noise — see
`domains/observability/metrics` for the SLI/SLO discipline that
makes the hit ratio a meaningful signal. A CDN with no hit-ratio
metric is operating blind (P10 analog).
## Origin Shielding (P1, P8, C8 Economy)
- Origin shielding routes all origin fetches through a single
shield PoP (or shield region). The shield absorbs the
thundering-herd: 10 000 PoPs missing the same URL fetch the origin
once, not 10 000 times. This is C8 Economy (origin bandwidth is
bounded) and P1 (the shield is itself a proximity layer for the
origin).
- Shielding is a geographic decision (P8): the shield sits in a
region close to the origin, not close to the user. The shield is
the inner ring of the CDN; the user-facing PoPs are the outer ring.
- A CDN without origin shielding under a stampede will overload the
origin; a shield that is itself partitioned from the origin must
degrade gracefully (P7) — serve stale per `stale-while-revalidate`
rather than 500.
## Purge Strategies (P6, C3 Simplicity)
| Strategy | Granularity | Latency to Invalidate | Cost | Best for |
|----------|-------------|-----------------------|------|----------|
| URL purge | One URL | Seconds | Low (one entry) | Surgical fixes, single-page corrections |
| Soft purge | One URL (mark stale, serve while refetch) | Seconds | Low | High-traffic URLs where a hard purge causes a stampede |
| Surrogate-key purge | A tag set (e.g., `product:123`, `category:shoes`) | Seconds | Medium (key indexing) | Related-content invalidation (a product update purges all its category pages) |
| Wildcard purge | A path prefix or pattern | Seconds to minutes | High (scan) | Site-wide template changes |
| All-cache purge | Everything | Seconds | Very high (origin stampede) | Disaster recovery only; never the steady-state invalidation path |
- Surrogate-key purge (Fastly, Akamai) is the highest-value
strategy: tag cache entries with content keys, then purge by tag.
This is explicit invalidation at scale (P6) without the origin
stampede of an all-cache purge.
- An all-cache purge as the steady-state invalidation path is a P6
violation dressed as a feature — it pushes the origin load back to
100% miss, defeating the CDN's purpose (P1).
## Cache Poisoning Prevention (P9, cross-link security/input-validation)
- A cache poisoned by a crafted request (a URL with a malicious
header that gets cached and served to others) is a correctness
defect (C1) and a security breach (P9 — the edge node is
exploited). Prevent poisoning by:
- Normalizing the cache key (strip untrusted query parameters,
ignore unknown headers, lower-case the host).
- Validating `Vary` against an allow-list; never `Vary: *` on a
shared cache (poisonable via header injection).
- Treating uncacheable responses (`Set-Cookie`,
`Cache-Control: private`) as never-stored.
- See `domains/security/input-validation` for the general
input-validation discipline the cache key must follow. The cache
key is a validation surface; a non-validated key is an attack
surface.
## Multi-CDN Routing (P8 Geographic Distribution)
- A multi-CDN strategy routes each request to the best PoP across
providers (Akamai + Cloudflare + Fastly). Routing is
location-aware (P8): latency, cost, and availability vary by
region and provider. The DNS layer (or a client-side router)
selects the CDN per request.
- Multi-CDN is a P8 decision, not a vendor-management decision:
geographic distribution is the first-class constraint. A
single-CDN deployment routes everything to one provider's PoPs;
a multi-CDN deployment routes by region, latency, and cost.
- Invalidation across multiple CDNs is harder (P6): each provider
has its own purge API and surrogate-key scheme. A multi-CDN purge
must fan out to all providers; a purge that reaches only one CDN
leaves the others stale. Track purge completion per provider —
see `domains/observability/metrics` for the per-CDN hit-ratio and
purge-latency signals.
```http
# A CDN config example: cache-control headers + a purge rule.
# Origin response: declare the cache contract (P6).
HTTP/1.1 200 OK
Cache-Control: public, max-age=60, s-maxage=600, stale-while-revalidate=300
Surrogate-Key: product:123 category:shoes
ETag: "etag-7a3f"
Vary: Accept-Encoding
# Purge rule (Fastly-style surrogate-key): when product 123
# updates, purge every cache entry tagged product:123 OR
# category:shoes. Explicit, bounded, no origin stampede (P6).
POST /service/svc1/purge
Surrogate-Key: product:123 category:shoes
# Returns: {"status": "ok", "id": "purge-abc"} — poll the purge
# status to confirm completion across all PoPs (P8, P10).
```
## Edge-Cache vs Origin — Decision Matrix (D-069)
| Strategy | When | Latency | Origin Load | Correctness Risk |
|----------|------|---------|-------------|------------------|
| Serve from PoP (cache hit) | The PoP holds a fresh entry (within TTL or revalidated) | Lowest (PoP-local, P1) | None | Low — bounded by TTL staleness (P6) |
| Serve stale while revalidate | The PoP holds a stale entry and `stale-while-revalidate` is set | Low (stale served immediately, refetch in background) | Background refetch (1 per entry) | Medium — stale served up to the revalidate window; acceptable for eventually-consistent content |
| Fetch fresh from origin (miss) | The PoP has no entry, or the content is non-cacheable | High (origin round trip) | Full fetch per miss | Low — fresh by construction; the miss is the correctness floor |
| Origin-shield fetch | Multiple PoPs miss the same URL; the shield collapses the herd | Medium (PoP → shield → origin) | Bounded to one origin fetch per shield (C8) | Low — shield is the inner ring; staleness bounded by shield TTL |
| Purge and serve fresh | Explicit invalidation received (surrogate-key or URL purge) | Medium (purge propagates, then fresh fetch) | Full fetch post-purge | Lowest — explicit invalidation is the tightest staleness bound (P6) |
| Serve from origin directly (bypass cache) | Content is non-cacheable (personalized, real-time) | Highest (every request hits origin) | Full fetch per request | Lowest for correctness, highest for origin load — use sparingly |
- The default is **serve from PoP** when fresh, **fetch fresh from
origin** on miss with **origin-shield** to bound origin load, and
**purge and serve fresh** when explicit invalidation is required.
Bypass-the-cache is for non-cacheable content only — bypassing for
cacheable content is a P1 violation (you have defeated the CDN).
- The correctness risk column is bounded by the invalidation
strategy (P6): every row except "bypass" carries staleness risk
that is bounded by TTL or explicit purge. A row with no
invalidation strategy is a P6 violation.
## What Violates CDN Discipline
| Violation | Principle |
|-----------|-----------|
| TTL-less edge cache under partition (stale-forever, no explicit invalidation) | P6 Cache Invalidation is Explicit |
| Cache key that varies on untrusted query parameters (poisonable) | P6, P9 (`domains/security/input-validation`) |
| All-cache purge as the steady-state invalidation path (origin stampede) | P6, C8 Economy |
| Non-normalized cache key (case-sensitive host, unsorted query) | P6, `domains/security/input-validation` |
| Bypass-the-cache for cacheable content | P1 Proximity is the Design Driver (defeats the CDN) |
| Multi-CDN with no per-CDN purge completion tracking | P8, P10 (stale cache invisible to the operator) |
| Origin fetch that is not idempotent under retry | P5 Edge Operations are Idempotent |
| Cache-hit ratio not measured | P10, `domains/observability/metrics` |
| Single-CDN deployed where geographic distribution requires multi-CDN | P8 Geographic Distribution |
| Shield PoP that 500s instead of serving stale under partition | P7 Partial Degradation is Engineered |
+218
View File
@@ -0,0 +1,218 @@
# Edge — First Principles
## 1. The Principles
### P1. Proximity is the Design Driver
Compute, storage, and data are placed near the user or the data
source. At the edge, latency is a correctness constraint (C1), not a
performance preference — a late answer is a wrong answer when the
round trip to a central region exceeds the user's or device's
tolerance. This is the geographic expression of `C4 Locality`:
performance's locality is algorithmic (data near compute); edge's
locality is geographic (compute near user/data source). Placement is
a design decision, not an accident of deployment, and it is
constrained by `P8 Geographic Distribution`. The proximity angle is
the distinguishing trait of the edge domain per D-061: this is what
separates edge from `domains/performance/` (which owns *generic*
measurement and optimization, not placement).
### P2. Offline is a First-Class State
The system continues to operate when disconnected from the center.
Partition is the norm, not the exception; reconciliation happens on
reconnect, never assumed to be instant. An app that crashes on
disconnect has no offline state and is unengineered. Offline
operation derives from `C5 Reversibility` — the disconnected state
is reversible back to consistency via reconciliation — and `C1
Correctness`, because correctness under partition is the contract,
not eventual correctness as a hedge. This is the foundation for
`domains/edge/offline-first.md` and the precondition for the
bounded-conflict discipline of `P4`.
### P3. Resources are Constrained and Declared
Edge nodes — IoT sensors, gateways, point-of-sale devices, CDN PoP
caches, 5G MEC nodes — have bounded CPU, memory, power, and
bandwidth. Constraints are declared per node class, never assumed
infinite. An undeclared budget is a defect: unbounded growth is a
bug, and a constrained device with no budget will OOM or exhaust
power. This derives from `C8 Economy` (use no more than the task
requires) and `C1 Correctness` (a node that exceeds its bounds has
failed). This is the edge-specific angle on `domains/performance/P4
Resource Bounds` — performance owns the generic principle; edge owns
the constrained-device reality. See `domains/edge/iot.md` for the
per-device-class application.
### P4. Sync Conflicts are Bounded, Not Infinite
Divergent state across partitioned nodes converges. Oscillation and
infinite sync loops are correctness failures, not eventual
consistency. A merge that never terminates is a livelock; a CRDT
without merge semantics or an LWW without a monotonic clock can
oscillate forever. This derives from `C1 Correctness` (convergence is
a correctness contract) and `C5 Reversibility` (divergent state is
reversible back to convergence). The bound may be eventual (CRDTs) or
arbitrated (LWW with vector clocks), but it must exist. This is the
foundation for `domains/edge/sync.md` and the rule the
`edge-sync-loop` chaos anti-pattern breaches.
### P5. Edge Operations are Idempotent
Sync, cache fill, and device commands are retried by nature — the
network is partition-prone and the operation will be re-attempted.
Idempotency keys (or deterministic operations) make retries safe. A
non-idempotent edge write retried with side effects doubles the
effect; a non-idempotent cache fill under retry corrupts the cache.
This derives from `C1 Correctness`: correctness under retry is the
contract, not a nice-to-have. This parallels `domains/messaging/P3
Consumers are Idempotent` (cross-process delivery) and is the edge's
device-and-cache-flavored analog — see `domains/edge/iot.md` for
device command idempotency and `domains/edge/cdn.md` for cache-fill
idempotency.
### P6. Cache Invalidation is Explicit
Edge caches carry a defined invalidation or TTL strategy. A
stale-forever cache under partition is a silent correctness defect;
a TTL-less cache with no explicit invalidation is a bug, not a
feature. This derives from `C1 Correctness` (cached state must be
correct) and `C3 Simplicity` (a defined invalidation strategy is
simpler and clearer than ad-hoc staleness). This is distinct from
`domains/performance/P5 Caching with Intent`, which owns *generic*
caching and optimization; edge owns the *geographic,
partition-aware* invalidation angle — when a PoP is partitioned from
the origin, the invalidation strategy is the correctness mechanism.
See `domains/edge/cdn.md` for purge strategies (URL vs soft vs
surrogate-key) and the edge-cache-vs-origin decision matrix.
### P7. Partial Degradation is Engineered
The system degrades gracefully when an edge node or link fails. A
partial service is a designed state with a defined contract, not a
crash. One node's failure must not collapse the whole fleet; the
degraded mode is documented, observable, and recoverable. This
derives from `C1 Correctness` (the degraded contract is a
correctness bound) and `C5 Reversibility` (recovery from degradation
is reversible by construction). A crash-on-node-failure system has
no degradation contract — it has an all-or-nothing failure mode that
violates the fleet assumption. See `domains/edge/iot.md` for
device-drop degradation and `domains/edge/offline-first.md` for
partition degradation.
### P8. Geographic Distribution is a First-Class Constraint
The fleet is geo-distributed; routing, fan-out, and data placement
are location-aware decisions, not accidents of deployment. The
system is many nodes across many locations, not a single deployment.
Data residency, regional latency, and PoP selection are engineered,
not discovered in production. This derives from `C4 Locality` (the
placement of data and compute is a locality decision) and `C6
Composability` (the fleet composes from location-aware parts, each
with its own contract). This is the structural companion to `P1
Proximity`: P1 says *where* compute should be (near the user); P8
says the *distribution* of compute across geographies is a
first-class constraint. See `domains/edge/cdn.md` for multi-CDN
routing.
### P9. Identity is Constrained at the Edge
Edge devices and nodes hold scoped, minimal credentials. No edge
node is a cluster-admin-equivalent; device identity is per-device,
not shared. One compromise must not equal a fleet compromise. This
derives from `C1 Correctness` (security is a subset of correctness —
an exploitable edge node does not do what it was supposed to do) and
`C8 Economy` of trust (the credential scope is minimal for the task).
A shared edge-device credential is the edge analog of a
cluster-admin GitOps robot — blast radius is unbounded. See
`domains/security/secrets.md` for the general secret-hygiene
principles and `domains/edge/iot.md` for device provisioning.
### P10. Edge Observability Survives Partition
Telemetry is local-first: buffered on the node and forwarded on
reconnect. Partition does not blind the operator. A fire-and-forget
telemetry pipeline loses data when the link drops; a local-first
buffer survives. This derives from `C7 Observability` (the fleet's
behavior is visible to the operator) and `C5 Reversibility` (the
buffered telemetry is reversible back to visibility on reconnect).
This is distinct from `domains/observability/P1 Structured by
Default`, which owns *generic* structured telemetry; edge owns the
*partition-survivable, local-first* angle. See
`domains/observability/metrics.md` and
`domains/observability/logging.md` for the generic structured-
telemetry foundations edge builds on.
## 2. Core Principle Trace
Each edge P-rule derives from one or more core C-rules (C1C8). The
matrix extension lands in P4 of the v0.4 plan; the traces below are
authoritative. Edge is a broad-derivation domain touching 7 of 8
core principles (C1, C3, C4, C5, C6, C7, C8); C2 (Clarity) is not a
primary derivation — edge clarity is indirect (a cache with explicit
invalidation is clearer than one without, but the primary trace is
C1/C3).
| P-rule | Core | Why |
|--------|------|-----|
| P1 Proximity is the Design Driver | C4, C1 | Locality of compute near user/data; correctness via latency |
| P2 Offline is a First-Class State | C1, C5 | Correctness under partition; reversibility of reconciliation |
| P3 Resources are Constrained and Declared | C8, C1 | Economy of constrained nodes; correctness of declared bounds |
| P4 Sync Conflicts are Bounded, Not Infinite | C1, C5 | Correctness of convergence; reversibility of divergent state |
| P5 Edge Operations are Idempotent | C1 | Correctness under retry |
| P6 Cache Invalidation is Explicit | C1, C3 | Correctness of cached state; simplicity of defined invalidation |
| P7 Partial Degradation is Engineered | C1, C5 | Correctness of degraded modes; reversibility of recovery |
| P8 Geographic Distribution is a First-Class Constraint | C4, C6 | Locality of placement; composability of the fleet |
| P9 Identity is Constrained at the Edge | C1, C8 | Correctness via security; economy of trust |
| P10 Edge Observability Survives Partition | C7, C5 | Observability of the fleet; reversibility of buffered telemetry |
## 3. What Violates These Principles
| Violation | Principle Breached |
|-----------|-------------------|
| Central-region-only deployment for a latency-bound workload | P1 Proximity is the Design Driver |
| App that crashes on disconnect (no offline state) | P2 Offline is a First-Class State |
| Undeclared edge-node resource budget (assumes infinite CPU/memory) | P3 Resources are Constrained and Declared |
| Sync loop that oscillates forever (CRDT without merge-semantics, LWW without monotonic clock) | P4 Sync Conflicts are Bounded, Not Infinite |
| Non-idempotent edge write (cache-fill or device command retried with side effects) | P5 Edge Operations are Idempotent |
| TTL-less edge cache under partition (stale-forever, no explicit invalidation) | P6 Cache Invalidation is Explicit |
| Crash-on-node-failure (no partial-degradation contract) | P7 Partial Degradation is Engineered |
| Random geographic placement (no location-aware routing) | P8 Geographic Distribution is a First-Class Constraint |
| Shared edge-device credential (one key for the whole fleet) | P9 Identity is Constrained at the Edge |
| Fire-and-forget telemetry (no on-node buffer; data lost on partition) | P10 Edge Observability Survives Partition |
| Blocking call on a constrained IoT device with no timeout | P3, P5 (blocks the node; retry unsafe without idempotency) |
| Multi-CDN routing with no PoP-selection logic (latency uncontrolled) | P8, P1 (placement not a design decision) |
## 4. Relationship to Other Domains
Edge computing is the engineering discipline of placing compute,
storage, and data **near the source of generation or consumption**
rather than in a centralized cloud. The distinguishing constraints are
latency-bound operation, resource-constrained nodes,
geo-distribution as a fleet, and partition-prone operation. Edge
overlaps three existing domains by *subject* but not by *angle*: per
D-061, edge owns the proximity/location/constraint/disconnection
concerns that only arise at the network edge. The C4 Locality
emphasis is the discriminator: performance's locality is algorithmic
(data near compute); edge's locality is geographic (compute near
user/data source). Cross-links are one-directional outward (per
D-026 extended); no back-link edits to v0.1/v0.2/v0.3 content.
- `domains/performance/frontend` ← P6 (edge owns geographic,
partition-aware cache invalidation; performance owns *generic*
caching and measurement — D-061 boundary)
- `domains/performance/P4 Resource Bounds` ← P3 (edge owns
constrained-device reality; performance owns the generic
unbounded-growth-is-a-bug principle)
- `domains/observability/metrics` ← P10 (cache-hit ratio, edge
telemetry aggregation; edge owns the local-first angle, observability
owns generic structured metrics)
- `domains/observability/logging` ← P10 (local-first logging buffered
on-node and forwarded on reconnect)
- `domains/concurrency/patterns` ← P5 (the offline write-queue is
the cross-partition analog of the in-process bounded buffer —
concurrency owns in-process; edge owns partition-survivable)
- `domains/security/secrets` ← P9 (device credentials are scoped,
per-device, never shared — edge owns the constrained-identity
angle; security owns the general secret hygiene)
- `domains/security/input-validation` ← P6 (cache poisoning
prevention — edge cache keys are a validation surface)
- `domains/data/migrations` ← P4 (schema migration under sync must
reconcile across partitioned nodes; data owns the generic migration
discipline, edge owns the partitioned-reconcile angle)
> Note: cross-links to `domains/messaging/` (e.g., MQTT QoS parallels
> for delivery semantics) are intentionally omitted here — the
> messaging domain is authored in P2. The intra-v0.4 edge↔messaging
> links are added in P5 (ATELIER-114 per IDEATE-40) once both
> domains exist; the dangling link is acceptable per D-053.
+258
View File
@@ -0,0 +1,258 @@
# IoT — Derived Rules
> Derives from `domains/edge/first-principles.md`. Applies P3
> (Resources are Constrained and Declared) primarily, with P5
> (command idempotency), P7 (partial degradation when devices drop),
> P9 (device identity and provisioning), and P10 (telemetry from
> devices). Cross-links `domains/security/secrets` for device
> credentials and `domains/messaging/queues` for the MQTT QoS
> parallels to delivery semantics.
## What IoT at the Edge Is (P3 Resources are Constrained and Declared)
- IoT at the edge is the engineering discipline of operating
constrained devices — sensors, actuators, gateways, microcontrollers
— as first-class participants in a distributed system. The
distinguishing constraint is per-device resource bounds (P3): a
battery-powered sensor has kilobytes of RAM, a constrained
protocol, and a multi-year sleep budget. These constraints are
declared per device class, never assumed infinite.
- The boundary is per D-061: edge owns the constrained-device
reality; performance owns the generic unbounded-growth-is-a-bug
principle (`performance/P4 Resource Bounds`); concurrency owns
in-process primitives. IoT is an edge concern because its defining
traits are constrained resources (P3), geographic distribution as
a fleet (P8), partition-prone operation (P2), and device-scoped
identity (P9) — concerns that only arise at the network edge.
- See `domains/edge/offline-first.md` for the partition-survival
discipline that constrained devices depend on, and
`domains/edge/sync.md` for the reconciliation of device state
across partitions.
## Device Resource Classes (P3, C8 Economy)
- A device resource class declares the bounds for a class of
devices: CPU (MHz, cores), memory (KB/MB), power (battery mAh,
duty-cycle budget), bandwidth (bytes/sec, latency budget), and
storage (KB/MB). Every device in the fleet is assigned to a class;
every operation is budgeted against its class.
- An undeclared budget is a defect (P3 violation): a sensor that
sends telemetry every second without a duty-cycle budget exhausts
its battery in days, not years. The budget is the correctness
bound (C1) and the economy bound (C8).
- A device class implies a protocol choice: a class-0 device
(constrained sensor, KB RAM) speaks CoAP; a class-1 device
(gateway, MB RAM) speaks MQTT; a class-2 device (edge compute
node, GB RAM) speaks HTTP. The protocol follows the constraint,
not the reverse.
| Class | RAM | Power | Protocol | Typical role |
|-------|-----|-------|----------|--------------|
| 0 (constrained sensor) | < 10 KB | Battery, multi-year | CoAP, LoRaWAN | Telemetry only, no inbound commands |
| 1 (actuator, gateway) | 10 KB 1 MB | Battery or wired, weeks-months | MQTT, CoAP | Telemetry + commands, queue-and-forward |
| 2 (edge compute) | > 1 MB | Wired, continuous | HTTP, MQTT | Local aggregation, gateway, edge inference |
## Constrained Protocols — MQTT and CoAP (P3, P5)
- **MQTT** is the canonical pub/sub protocol for constrained devices.
It is lightweight (2-byte header), broker-backed, and provides QoS
levels (0, 1, 2) that map to delivery semantics. MQTT is the
cross-process analog of message-queue delivery — see
`domains/messaging/queues` for the general queue/delivery-semantics
discipline; the cross-link is one-directional outward (edge →
messaging) per D-062 and D-026 extended.
- **CoAP** is the REST analog for constrained devices: UDP-based,
low-overhead, with confirmable (CON) and non-confirmable (NON)
message types. CoAP fits class-0 devices where TCP is too heavy.
- A blocking synchronous call on a constrained device with no
timeout is the `blocking-call-on-constrained-device` chaos
anti-pattern: it blocks the node, has no timeout (= hang), and
retries are unsafe without idempotency (P3 + P5 breach). Every
device operation must be async with a timeout, and every retried
operation must be idempotent.
```json
// MQTT publish/subscribe payload with QoS levels (P5 idempotency,
// P3 constrained protocol).
// QoS 0 — at-most-once: fire-and-forget, no ack. For telemetry
// where a dropped sample is acceptable (P3 economy of the
// constrained link).
{
"topic": "devices/sensor-7/temperature",
"qos": 0,
"payload": {
"device": "sensor-7",
"ts": 1700000000,
"value": 21.4,
"unit": "C"
}
}
// QoS 1 — at-least-once: acked, may duplicate. The consumer must
// be idempotent (P5) — dedup by (device, ts) or an idempotency key.
{
"topic": "devices/actuator-3/command",
"qos": 1,
"payload": {
"device": "actuator-3",
"idempotencyKey": "cmd-1700000000-1",
"command": "set-point",
"value": 22.0
}
}
// QoS 2 — exactly-once: four-step handshake, no duplication. The
// heaviest QoS; use only where duplicates are intolerable AND the
// device has the budget for the handshake (class-1+ only, P3).
{
"topic": "devices/actuator-3/irreversible-command",
"qos": 2,
"payload": {
"device": "actuator-3",
"idempotencyKey": "cmd-1700000000-2",
"command": "calibrate"
}
}
```
- The QoS choice is a P5 (idempotency) and P3 (resource) decision:
QoS 0 is cheapest (no ack) but lossy; QoS 1 requires consumer
idempotency (P5); QoS 2 is exactly-once but costs a four-step
handshake on a constrained link. The default for telemetry is QoS
0 or 1; the default for commands is QoS 1 with an idempotency key
(P5); QoS 2 is reserved for irreversible commands where the
device budget permits.
## Device Identity and Provisioning (P9 Identity is Constrained at the Edge)
- Every device holds a per-device identity: a unique device ID and a
scoped credential (X.509 certificate, API token, or rotating
key). No shared fleet credential — one compromise must not equal
a fleet compromise (P9). The credential scope is minimal: a
device can publish to `devices/<its-id>/+` and subscribe to
`devices/<its-id>/commands`, nothing else.
- Provisioning is the act of assigning a device identity at
enrollment time. The provisioning manifest declares the device,
its class, its allowed topics, and its credential. The manifest is
the P9 contract — a device operating outside its manifest scope is
a violation.
- A device that is provisioned with a shared fleet key (the
`shared-edge-device-credential` anti-pattern) is a P9 violation:
blast radius is unbounded. See `domains/security/secrets` for the
general secret-hygiene discipline (per-identity credentials,
rotation, minimal scope) that device provisioning builds on.
```yaml
# Device provisioning manifest (P9 per-device identity + scoped
# credentials). The manifest is the contract; the device operates
# only within its declared scope.
device:
id: sensor-7
class: 0 # P3 resource class
model: temp-sensor-v2
firmware: 1.4.2
identity:
cert: "sha256-of-device-cert"
credentialScope:
publish:
- "devices/sensor-7/temperature"
- "devices/sensor-7/status"
subscribe:
- "devices/sensor-7/commands"
# No wildcard, no fleet-wide topics (P9).
provisioning:
enrolledAt: 2024-01-15T00:00:00Z
rotatesEvery: 90d
# Per-device credential; never shared (P9, domains/security/secrets).
```
## Telemetry from Devices (P10 Edge Observability Survives Partition)
- Device telemetry is local-first (P10): the device buffers telemetry
on-node and forwards on reconnect. A fire-and-forget telemetry
pipeline loses data when the link drops; a buffered pipeline
survives. The buffer is bounded by the device class (P3): a
class-0 sensor buffers minutes of telemetry, not hours.
- Telemetry is observable in aggregate: the operator sees the fleet's
behavior, not just per-device. A device that has not reported in
its expected interval is itself a signal (a dead device, a
partitioned device, a drained battery). See
`domains/observability/metrics` for the generic structured-metrics
discipline; edge owns the partition-survivable, local-first angle.
- Telemetry must not be a secrets channel (P9 analog, see
`domains/observability/P6 No Secrets in Observability`): device
credentials, PII, and personally-identifying location must not
enter telemetry payloads.
## Command Idempotency (P5 Edge Operations are Idempotent)
- Device commands are retried by nature (the network is
partition-prone). Every command carries an idempotency key so a
retried command does not double-apply (P5). A `set-point` command
retried with the same idempotency key sets the point once, not
twice; an `open-valve` command retried is safe because the valve
is already open.
- Irreversible commands (a calibration burn-in, a firmware flash)
require stronger idempotency: the device tracks applied
idempotency keys and refuses re-application. A retried irreversible
command without idempotency tracking double-applies the effect
(P5 violation, possibly a physical-side-effect bug).
- The idempotency key is per-command, not per-device. A device that
dedups by device ID alone will drop distinct commands issued in
the same window. Use `(device, command-id, ts-window)` or a
UUID per command.
## Partial Degradation When Devices Drop (P7 Partial Degradation is Engineered)
- A fleet degrades when devices drop (battery exhaustion, partition,
hardware failure). The system must continue to operate with the
remaining devices; a whole-system crash on one device's failure is
a P7 violation. The degraded mode is documented: which
aggregations are valid with N-1 devices, which alerts fire, which
fallbacks engage.
- A device that drops is not an incident by itself — fleets expect
churn. The operator-facing signal is the *aggregate* health (X%
of devices reporting, Y% partitioned for >Z minutes), not the
per-device drop. Per-device drop alerts are noise; aggregate
degradation alerts are signal (see `domains/observability/metrics`).
- A command to a dropped device must time out (P5 — idempotent
retry) and degrade (P7 — the fleet continues without that
device). A command that blocks forever waiting for a dropped
device is the `blocking-call-on-constrained-device` chaos
anti-pattern (P3 + P5 breach).
## Cross-Link to Messaging (P5, cross-link messaging/queues)
- MQTT QoS 0/1/2 maps to at-most-once / at-least-once / exactly-once
delivery semantics — the same three-way tradeoff documented in
`domains/messaging/queues`. The cross-link is one-directional
outward (edge → messaging) per D-026 extended: edge owns the
constrained-device protocol angle; messaging owns the generic
cross-process delivery-semantics angle.
- This link dangles until P2 (the messaging domain is authored in
P2); it is verified bidirectional in P5 (ATELIER-114 per
IDEATE-40). Acceptable per D-053 (vertical-slice integrity — P1
ships the edge domain self-consistent; the messaging cross-link
resolves by the P6 ship).
- The parallel: a constrained device's QoS 1 publish is the
device-flavored instance of an at-least-once queue delivery — the
consumer (the broker or the downstream service) must be
idempotent (P5 here, `messaging/P3 Consumers are Idempotent`
there). The idempotency discipline is the same; the protocol and
failure model differ (constrained-device link vs broker-backed
network).
## What Violates IoT-at-the-Edge Discipline
| Violation | Principle |
|-----------|-----------|
| Undeclared device resource budget (assumes infinite battery/RAM) | P3 Resources are Constrained and Declared |
| Shared fleet credential (one key for all devices) | P9 Identity is Constrained at the Edge |
| Non-idempotent device command (retried command doubles the effect) | P5 Edge Operations are Idempotent |
| Blocking synchronous call on a constrained device with no timeout | P3, P5 (blocks the node; retry unsafe) |
| Fire-and-forget telemetry with no on-device buffer (lost on partition) | P10 Edge Observability Survives Partition |
| Whole-system crash on one device's failure (no degradation contract) | P7 Partial Degradation is Engineered |
| Device credential scope that includes fleet-wide topics (over-scoped) | P9, `domains/security/secrets` |
| QoS 2 used on a class-0 device (no budget for the handshake) | P3 Resources are Constrained and Declared |
| Per-device-drop alert (noise; aggregate degradation is the signal) | P7, `domains/observability/metrics` |
| Telemetry payload that includes device credentials or PII | P9, `domains/observability/P6 No Secrets in Observability` |
+354
View File
@@ -0,0 +1,354 @@
# Offline-First — Derived Rules
> Derives from `domains/edge/first-principles.md`. Applies P2
> (Offline is a First-Class State) primarily, with P5 (idempotent
> queue-and-forward), P4 (bounded sync conflicts on reconnect), P7
> (partial degradation), and P10 (local-first telemetry). Cross-links
> `domains/concurrency/patterns` for the in-process bounded-buffer
> analog and `domains/observability/logging` for local-first logging.
## What Offline-First Is (P2 Offline is a First-Class State)
- Offline-first is the design discipline in which the system
continues to operate when disconnected from the center. Partition
is the norm, not the exception; reconciliation happens on
reconnect. The offline state is engineered, not a degenerate mode
the app falls into by accident.
- The boundary is per D-061: edge owns the
proximity/location/disconnection angle. An offline-first web app
is an edge concern because its defining trait is partition-survival
(P2), not generic performance. The local-first storage is the
edge device's constrained-resource reality (P3).
- Offline-first is the precondition for the bounded-conflict
discipline of `P4 Sync Conflicts are Bounded, Not Infinite`:
without offline operation there is nothing to reconcile; with it,
the reconnect reconciliation is the correctness mechanism. See
`domains/edge/sync.md` for the conflict-resolution strategies.
## Local-First Storage (P2, P3)
- Local-first storage holds the working copy on the device:
IndexedDB (browser), SQLite (mobile, embedded), or on-device file
storage (desktop, IoT gateway). The local store is the authority
while offline; the server is reconciled later, not consulted per
read.
- The local store is bounded by the device (P3 — Resources are
Constrained and Declared). A local store that grows without bound
is a defect: declare a budget (e.g., a 50 MB IndexedDB quota, a
30-day rolling window), and evict outside the budget deterministically.
- The local store is the offline state; without it the app is
online-only and crashes on disconnect (P2 violation). The store is
the reversibility mechanism (C5): every local write is reversible
on reconcile.
```typescript
// Local-first store sketch (IndexedDB). The app reads from the
// local store, never the network, while offline. Writes queue
// locally and forward on reconnect (P2, P5).
const db = await openDB("atelier-offline", 1, {
upgrade(db) {
const store = db.createObjectStore("pending-writes", {
keyPath: "id",
});
store.createIndex("by-createdAt", "createdAt");
},
});
async function readRecord(id: string) {
// Read from local store first; the network is a reconcile path,
// not the read path.
return db.get("pending-writes", id);
}
```
## Queue-and-Forward for Writes (P5 Edge Operations are Idempotent)
- Every write while offline is queued locally and forwarded to the
server on reconnect. The queue is the offline write-queue; the
forward is the reconcile. Each queued write carries an idempotency
key so a retried forward (the network is partition-prone) does not
double-apply (P5).
- The queue is bounded (P3): a queue that grows without limit on a
constrained device will exhaust it. Declare a max-queue-depth and
a max-queue-bytes; reject or evict beyond the bound with a defined
policy (oldest-first, lowest-priority-first).
- The queue is the cross-partition analog of the in-process bounded
buffer — see `domains/concurrency/patterns` (bounded buffer,
backpressure). Concurrency owns the in-process analog; edge owns
the partition-survivable analog. The failure model differs: the
in-process buffer fails by OOM; the offline write-queue fails by
partition or device loss.
```typescript
// Offline write-queue sketch. Each entry carries an idempotency
// key (P5) so a retried forward is safe. The queue is bounded by
// maxDepth (P3).
interface PendingWrite {
id: string; // local id
idempotencyKey: string; // server-side dedup key (P5)
collection: string;
payload: unknown;
createdAt: number;
}
const MAX_DEPTH = 1000;
async function queueWrite(write: Omit<PendingWrite, "id" | "idempotencyKey" | "createdAt">) {
const depth = await db.count("pending-writes");
if (depth >= MAX_DEPTH) {
// P3: bounded queue. Evict the oldest pending write or reject.
// Rejecting is correct when the write is higher-priority than
// the oldest; evicting is correct when the newest is lowest.
throw new Error("offline-queue-full");
}
const entry: PendingWrite = {
...write,
id: crypto.randomUUID(),
idempotencyKey: `${write.collection}:${crypto.randomUUID()}`,
createdAt: Date.now(),
};
await db.put("pending-writes", entry);
// The forward loop picks this up when connectivity returns.
}
async function forwardPendingWrites(server: Server) {
const pending = await db.getAllFromIndex("pending-writes", "by-createdAt");
for (const write of pending) {
// P5: idempotent — the server dedups by idempotencyKey.
await server.apply(write, write.idempotencyKey);
await db.delete("pending-writes", write.id);
}
}
```
## Conflict Detection on Reconnect (P4 Sync Conflicts are Bounded)
- On reconnect, the queued writes are forwarded; the server may
have advanced while the device was offline. A conflict is when the
local write and the server state diverge. Conflict detection is
the precondition for bounded reconciliation (P4): a write forwarded
blindly (last-write-wins with no clock) is a P4 violation waiting
to happen.
- Conflict resolution strategies (CRDT, LWW with vector clocks,
application-specific merge) are the subject of
`domains/edge/sync.md` — the CRDT-vs-LWW decision matrix there
determines which applies. Offline-first owns the *detection*; sync
owns the *resolution*.
- A reconnect that detects no conflicts when conflicts exist is a
silent correctness defect (C1, P4). Detection must be conservative:
when in doubt, flag a conflict and surface it to the merge
function or the user.
## UI for Offline State (P7 Partial Degradation is Engineered)
- The UI must reflect the offline state visibly: a "you are offline,
changes will sync when connected" banner, a pending-writes counter,
a last-synced timestamp. A UI that hides the offline state
violates P7 — the degraded mode is a designed state with a defined
contract, not a silent fall-through.
- The UI must function while offline: reads from local-first
storage, writes to the queue, navigation that does not require the
network. An app that shows a blank screen or a spinner-forever when
offline has no offline state (P2 violation) and no degradation
contract (P7 violation).
- The pending-writes counter is the local-first analog of the
messaging consumer-lag metric — see
`domains/observability/metrics` for the lag-discipline parallel.
## Service Workers (P2, P6)
- A service worker is a client-side proxy that intercepts network
requests and serves from a local cache. It is the browser's
offline-first primitive: the service worker cache is the
offline-capable store for assets; the IndexedDB store is the
offline-capable store for data.
- The service worker cache is an edge cache (P6 — Cache Invalidation
is Explicit): it must carry a TTL or explicit invalidation
strategy. A service worker that caches forever and never
invalidates is a TTL-less edge cache under partition — a P6
violation (stale-forever).
- See `domains/edge/cdn.md` for the generic edge-cache invalidation
discipline; the service worker is the on-device instance of it.
```javascript
// Service worker cache strategy: stale-while-revalidate for
// assets, network-first for data, explicit version-bump for
// breaking changes (P6).
const CACHE = "atelier-v3"; // bump on deploy to invalidate (P6)
const ASSETS = ["/", "/app.js", "/styles.css"];
self.addEventListener("install", (event) => {
event.waitUntil(
caches.open(CACHE).then((cache) => cache.addAll(ASSETS))
);
});
self.addEventListener("fetch", (event) => {
const url = new URL(event.request.url);
if (url.pathname.startsWith("/api/")) {
// Network-first for data; fall back to cache on partition (P2).
event.respondWith(
fetch(event.request).catch(() => caches.match(event.request))
);
} else {
// Stale-while-revalidate for assets (P6 explicit invalidation).
event.respondWith(
caches.open(CACHE).then(async (cache) => {
const cached = await cache.match(event.request);
const network = fetch(event.request).then((resp) => {
cache.put(event.request, resp.clone());
return resp;
}).catch(() => cached);
return cached || network;
})
);
}
});
self.addEventListener("activate", (event) => {
// P6: explicit invalidation. Drop old caches on activate.
event.waitUntil(
caches.keys().then((keys) =>
Promise.all(keys.filter((k) => k !== CACHE).map((k) => caches.delete(k)))
)
);
});
```
## Offline Write-Queue and Conflict Detection Mapped to the Testing Pyramid (IDEATE-38, ATELIER-94)
The offline write-queue and conflict-detection patterns must be
tested at every tier of the testing pyramid. Each tier exercises a
different failure mode; skipping a tier leaves a correctness gap
(P2, P4 violations that surface only in production partitions).
| Pyramid Tier | What it exercises | What it proves |
|-------------|-------------------|----------------|
| **Unit** | Conflict detection on a merge function (pure inputs → expected merge result) | The merge logic is correct in isolation (P4) — given two divergent states, the merge returns the converged state |
| **Integration** | Reconnect reconcile against a local store (fake server, real IndexedDB/SQLite) | The queue-and-forward loop drains correctly; the local store and server converge after reconnect (P2, P5) |
| **E2e** | Partition simulation with a fake network (the app runs in a browser, the network is cut and restored) | The offline state, UI, and reconcile work end-to-end under partition (P2, P7) |
- **Unit — conflict detection on a merge function.** The merge
function is pure: given two divergent states and a clock, it
returns the converged state. Test every merge case (LWW, CRDT
register, set union, application-specific three-way merge) as a
pure function. This is the cheapest tier and the highest coverage
per test — see `domains/testing/pyramid`.
```typescript
// Unit test sketch: conflict detection on a merge function (P4).
// The merge function is pure; no network, no store. Test that
// divergent states converge and that the merge is bounded (no
// oscillation).
function mergeLWW(local: State, remote: State, clock: Clock): State {
// Last-write-wins: the state with the later vector-clock wins.
// Returns the converged state (P4).
return clock.compare(local.clock, remote.clock) >= 0 ? local : remote;
}
// Unit cases:
// - local ahead → local wins
// - remote ahead → remote wins
// - concurrent (clocks incomparable) → conflict flagged or LWW tiebreak
// - identical → no-op convergence (bounded, no oscillation)
test("mergeLWW converges when local is ahead", () => {
const local = { v: 2, clock: { a: 2 } };
const remote = { v: 1, clock: { a: 1 } };
expect(mergeLWW(local, remote, { compare: (a, b) => a.a - b.a })).toEqual(local);
});
```
- **Integration — reconnect reconcile against a local store.** A
fake server stands in for the network; the real IndexedDB (or
SQLite) holds the queue. The test fills the queue while offline,
reconnects, and asserts the queue drains and the server and local
store converge. This exercises the queue-and-forward loop (P5)
and the reconcile against real storage.
```typescript
// Integration test sketch: reconnect reconcile against a local
// store. A fake server; real IndexedDB. The queue drains; the
// server and local store converge after reconnect (P2, P5).
test("reconnect reconciles pending writes against the server", async () => {
const db = await openDB("test-offline", 1, { /* schema */ });
const server = new FakeServer();
await queueWrite(db, { collection: "docs", payload: { v: 1 } });
// Simulate offline: server is unreachable.
server.offline();
await queueWrite(db, { collection: "docs", payload: { v: 2 } });
expect(await db.count("pending-writes")).toBe(2);
// Simulate reconnect: server is reachable.
server.online();
await forwardPendingWrites(server, db);
expect(await db.count("pending-writes")).toBe(0);
expect(await server.latest("docs")).toEqual({ v: 2 });
});
```
- **E2e — partition simulation with a fake network.** The app runs
in a real browser; a fake network layer cuts and restores the
connection. The test asserts the UI shows the offline state, the
writes queue, the reconnect reconciles, and the UI returns to
online. This is the highest-fidelity tier and the lowest coverage
per test — run a small number of representative scenarios, not a
combinatorial matrix.
```typescript
// E2e test sketch: partition simulation with a fake network. The
// app runs in a browser; the network is cut and restored. Asserts
// the offline UI state, the queue, the reconcile, and the online
// recovery (P2, P7).
test("app survives a network partition and reconciles on reconnect", async () => {
await page.goto("https://app.example.com");
await page.click("text=Edit document");
await page.fill("textarea", "offline edit");
// Cut the network.
await page.setOffline(true);
await page.click("text=Save");
await expect(page.locator("text=You are offline")).toBeVisible();
await expect(page.locator("text=1 pending change")).toBeVisible();
// Restore the network.
await page.setOffline(false);
await expect(page.locator("text=All changes synced")).toBeVisible();
await expect(page.locator("text=0 pending changes")).toBeVisible();
});
```
- The three tiers are complementary: unit proves the merge logic,
integration proves the reconcile loop, e2e proves the partition
behavior. Skipping any tier leaves a correctness gap. See
`domains/testing/pyramid` for the pyramid discipline and
`domains/testing/fixtures` for the fake-server and fake-network
fixture patterns.
## Observability (P10 Edge Observability Survives Partition)
- The offline state is itself an observable signal: the
pending-writes count, the last-synced timestamp, the
reconcile-failure count. A device stuck offline for days with a
full queue is an incident; without local-first telemetry it is
invisible (P10 violation).
- Local-first logging (buffered on-device, forwarded on reconnect)
is the offline-first instance of `P10 Edge Observability Survives
Partition`. See `domains/observability/logging` for the generic
structured-logging discipline the local-first buffer builds on.
- A reconcile failure that is not logged locally is a silent defect
— the operator cannot debug what they cannot see (C7, P10).
## What Violates Offline-First Discipline
| Violation | Principle |
|-----------|-----------|
| App that crashes on disconnect (no offline state) | P2 Offline is a First-Class State |
| Unbounded offline write-queue (grows until device OOM) | P3 Resources are Constrained and Declared |
| Queued write forwarded without an idempotency key (retry doubles the effect) | P5 Edge Operations are Idempotent |
| Reconnect that detects no conflicts when conflicts exist | P4 Sync Conflicts are Bounded, Not Infinite |
| UI that hides the offline state (no banner, no pending counter) | P7 Partial Degradation is Engineered |
| Service worker cache with no TTL and no explicit invalidation | P6 Cache Invalidation is Explicit |
| Reconcile failure with no local log (silent under partition) | P10 Edge Observability Survives Partition |
| Merge function that oscillates (no convergence guarantee) | P4, `domains/edge/sync.md` |
| Local-first store with no declared budget (grows without bound) | P3, `domains/concurrency/patterns` (bounded buffer analog) |
| E2e tests that never simulate a partition (offline path untested) | P2, `domains/testing/pyramid` |
+315
View File
@@ -0,0 +1,315 @@
# Sync — Derived Rules
> Derives from `domains/edge/first-principles.md`. Applies P4 (Sync
> Conflicts are Bounded, Not Infinite) primarily, with P5
> (idempotent merge operations), P2 (offline as the precondition),
> and P10 (sync is observable). Cross-links `domains/data/migrations`
> for schema migration under sync and `domains/concurrency/patterns`
> for the immutability-aid-merge principle.
## What the Sync Problem Is (P4 Sync Conflicts are Bounded, Not Infinite)
- Sync is the discipline of reconciling divergent state across
partitioned nodes. While partitioned, each node accepts writes
independently; on reconnect, the divergent state must converge.
The correctness contract is that the merge terminates and
converges — oscillation and infinite sync loops are correctness
failures, not eventual consistency (P4).
- Sync is the edge domain's deepest problem: it is the
reconciliation layer above `P2 Offline is a First-Class State`.
Without offline operation there is nothing to sync; with it, the
reconnect reconciliation is the correctness mechanism. See
`domains/edge/offline-first.md` for the offline write-queue that
produces the divergent state to be reconciled.
- The boundary is per D-061: edge owns the partitioned-reconcile
angle; concurrency owns the in-process analog
(`concurrency/P1 Immutability by Default` — immutability aids
merge); data owns the generic migration discipline
(`data/migrations`). Sync is an edge concern because its defining
trait is partitioned divergence, a concern that only arises at the
network edge.
## Conflict Resolution Strategies (P4, C1 Correctness, C5 Reversibility)
- A conflict is when two nodes have divergent state for the same
logical entity and no total order determines which is correct.
Resolution strategies fall into two families:
- **Conflict-free**: the data type guarantees convergence by
construction (CRDTs). The merge is deterministic; no conflict
surfaces to the user or the application.
- **Conflict-tolerant**: the data type can conflict; the
resolution policy (last-write-win, three-way merge,
application-specific) arbitrates. Conflicts may surface to the
user or be silently resolved per a documented policy.
- The choice is a P4 decision: conflict-free types guarantee the
bound (convergence) but constrain the data model; conflict-tolerant
types are flexible but require the resolution policy to be correct
and bounded (no oscillation). See the decision matrix below.
## CRDTs — Conflict-Free Replicated Data Types (P4, C5, C6)
- A CRDT is a data type whose merge operation is associative,
commutative, and idempotent. Given any set of divergent replicas,
merging them in any order converges to the same state — the merge
is deterministic and terminating (P4 bound). CRDTs derive from C5
Reversibility (divergent state reverses to convergence) and C6
Composability (CRDTs compose: a CRDT map of CRDT registers is
itself a CRDT).
- **State-based (CvRDT — convergent):** each replica carries its
full state; merge is a least-upper-bound on a semi-lattice. The
payload is larger (full state per merge); the merge is simple
(one function). Fits small state and unreliable networks.
- **Operation-based (CmRDT — commutative):** each replica carries
operations; merge is applying the operations in causal order. The
payload is smaller (ops, not state); the delivery must be
reliable and causally ordered. Fits large state and reliable
transport.
- The tradeoff: state-based is simpler but heavier; operation-based
is lighter but requires causal delivery. Both guarantee
convergence (P4); the choice is a C8 Economy decision (bandwidth
vs delivery complexity).
```typescript
// CRDT register: LWW-element-set (state-based, CvRDT). The merge
// is deterministic — the register with the later timestamp wins.
// Convergence is guaranteed (P4); the merge is idempotent (P5).
interface LWWRegister<T> {
value: T;
timestamp: number; // monotonic clock; ties broken by node id
nodeId: string;
}
function mergeLWWRegister<T>(
local: LWWRegister<T>,
remote: LWWRegister<T>,
): LWWRegister<T> {
// The merge is associative, commutative, idempotent (P4, P5).
// (local.timestamp, local.nodeId) > (remote.timestamp, remote.nodeId)
// is a total order — no oscillation, no infinite loop.
if (local.timestamp > remote.timestamp) return local;
if (local.timestamp < remote.timestamp) return remote;
// Tie: break by node id for a deterministic total order.
return local.nodeId > remote.nodeId ? local : remote;
}
// The register is a CRDT: merge(merge(a, b), c) === merge(a, merge(b, c))
// for any replicas a, b, c. Convergence is guaranteed (P4).
```
```typescript
// CRDT set: add-wins last-write-wins element set (state-based).
// Each element carries a timestamp; remove only wins if the
// remove-timestamp is later than the add-timestamp. This avoids
// the remove-wins-vs-add race (P4) without surfacing a conflict.
interface AWLWWSet<T> {
adds: Map<T, number>; // element -> add-timestamp
removes: Map<T, number>; // element -> remove-timestamp
}
function mergeAWLWWSet<T>(a: AWLWWSet<T>, b: AWLWWSet<T>): AWLWWSet<T> {
const adds = new Map<T, number>(a.adds);
const removes = new Map<T, number>(a.removes);
for (const [el, ts] of b.adds) {
adds.set(el, Math.max(adds.get(el) ?? 0, ts)); // add-wins union
}
for (const [el, ts] of b.removes) {
removes.set(el, Math.max(removes.get(el) ?? 0, ts));
}
return { adds, removes };
}
function contains<T>(set: AWLWWSet<T>, el: T): boolean {
const addTs = set.adds.get(el) ?? 0;
const rmTs = set.removes.get(el) ?? 0;
return addTs > rmTs; // add wins on equal timestamp (P4 bounded)
}
```
## Last-Write-Win (LWW) with Vector Clocks (P4, C1, C5)
- LWW is the simplest conflict-tolerant strategy: the write with the
latest timestamp wins. It is cheap, but it silently discards
concurrent writes — the "lost update" is the correctness cost. LWW
is correct only when the timestamp is a total order (a monotonic
clock, not wall time), and when lost concurrent writes are
acceptable (e.g., caching, presence, ephemeral state).
- **Vector clocks** are the timestamp that knows about concurrency.
A vector clock records the logical time of each node; two writes
are concurrent iff neither vector dominates the other. LWW with
vector clocks: a write that is causally later wins; a write that
is concurrent conflicts and is resolved by a tiebreak (node id,
wall time, or application policy).
- The tiebreak is the P4 bound: the conflict must be resolved
deterministically (no oscillation) and the resolution must be
documented. A tiebreak by wall time alone (no vector clock) is a
P4 violation waiting to happen — wall time skews across nodes,
and a clock skew can flip the tiebreak, oscillating the merge.
```typescript
// LWW with vector clocks (conflict-tolerant, P4 bounded). The
// vector clock records causal order; concurrent writes conflict;
// the conflict is tiebroken deterministically (no oscillation).
type VectorClock = Record<string, number>; // nodeId -> counter
function compareClock(a: VectorClock, b: VectorClock): "before" | "after" | "equal" | "concurrent" {
let aBefore = false, bBefore = false;
const keys = new Set([...Object.keys(a), ...Object.keys(b)]);
for (const k of keys) {
const av = a[k] ?? 0;
const bv = b[k] ?? 0;
if (av < bv) aBefore = true;
if (av > bv) bBefore = true;
}
if (aBefore && bBefore) return "concurrent";
if (aBefore) return "before";
if (bBefore) return "after";
return "equal";
}
interface LWWVectorState<T> {
value: T;
clock: VectorClock;
writerId: string; // tiebreak: deterministic, no oscillation (P4)
}
function mergeLWWVector<T>(
local: LWWVectorState<T>,
remote: LWWVectorState<T>,
): LWWVectorState<T> {
const order = compareClock(local.clock, remote.clock);
if (order === "before") return remote; // remote causally later
if (order === "after" || order === "equal") return local;
// Concurrent: tiebreak by writer id (deterministic, P4 bounded).
return local.writerId > remote.writerId ? local : remote;
}
```
- The merge is idempotent (P5): merging the same two replicas twice
yields the same result. The tiebreak by `writerId` is a total
order, so the merge cannot oscillate (P4 bound).
- A vector-clock merge that surfaces the concurrent conflict to the
application (instead of tiebreaking) is also valid — the
application resolves per its own policy. The P4 bound is that the
resolution terminates; the policy determines whether the user sees
the conflict or the system silences it.
## Merge Semantics (P4, P5, cross-link concurrency/patterns)
- The merge function is the heart of sync. Its properties (P4):
- **Associative**: `merge(merge(a, b), c) === merge(a, merge(b, c))`.
- **Commutative**: `merge(a, b) === merge(b, a)`.
- **Idempotent**: `merge(a, a) === a` (P5 — retried merges are safe).
- Immutability aids merge: an immutable state representation (the
CRDT payload, the LWW register with a clock) makes the merge a
pure function of two inputs, with no in-place mutation race. See
`domains/concurrency/patterns` (`concurrency/P1 Immutability by
Default`) for the in-process immutability principle; sync is the
cross-partition instance of it.
- A merge that mutates in place is a P5 violation waiting to
happen: a retried merge mutates the same state twice, and the
result is not idempotent. Always merge into a new state; never
mutate the inputs.
## Conflict-Free vs Conflict-Tolerant Data Types (P4, C3 Simplicity)
- **Conflict-free (CRDTs):** the data type guarantees convergence.
The application never sees a conflict; the merge is deterministic.
The cost: the data model is constrained (counters, sets, registers,
maps of these). A conflict-free type for arbitrary JSON is hard;
a conflict-free type for a counter is a PN-counter.
- **Conflict-tolerant (LWW, three-way merge, application policy):**
the data type can conflict; the resolution policy arbitrates. The
cost: the policy must be correct and bounded (no oscillation), and
the conflict may surface to the user. The benefit: any data model
can be made conflict-tolerant (just pick a tiebreak).
- The choice is the decision matrix below. It is a P4 decision
(which bound), a C1 decision (which correctness cost is
acceptable), and a C3 decision (which simplicity is affordable).
See also `domains/data/migrations` for the schema-evolution angle
— a schema change under sync must be compatible with both
replicas, or the merge fails on the new shape.
## Schema Migration Under Sync (P4, cross-link data/migrations)
- A schema migration under sync is harder than a single-node
migration: both replicas must understand the new shape, or the
merge fails. The migration must be forward-and-backward compatible
across all replicas that may still hold the old shape — see
`domains/data/migrations` for the generic compatibility discipline.
- A breaking schema change under sync requires a staged migration:
deploy the new-shape-aware merge first (it accepts both shapes),
then deploy the new shape, then deploy the old-shape-removing
merge. A big-bang schema change under sync is a P4 violation: the
replicas that have not yet upgraded will fail the merge, and the
sync will not converge.
- The merge function's version awareness is the P4 bound: the merge
must handle every shape version that may exist in the fleet, or
reject (and surface) the merge rather than silently corrupting.
## CRDT vs Last-Write-Win — Decision Matrix (D-069)
| Strategy | When | Correctness Guarantee | Operational Cost | Failure Mode |
|----------|------|------------------------|-------------------|--------------|
| CRDT (state-based, CvRDT) | The data model fits a CRDT (counter, set, register, map of these); convergence must be guaranteed without surfacing conflicts; the network is unreliable (full-state merge tolerates dropped ops) | Strong eventual convergence — `merge(a, b) === merge(b, a)` for any replicas (P4 bound by construction) | Medium — full state per merge (bandwidth); semi-lattice merge function per type; CRDT library or hand-rolled | A bug in the merge function = silent divergence (C1); large state = bandwidth cost on constrained links (P3) |
| CRDT (operation-based, CmRDT) | The data model fits a CRDT; bandwidth is constrained (ops are smaller than state); the transport is reliable and causally ordered | Strong eventual convergence — same guarantee, smaller payload | High — requires causal delivery (vector clock or broker with ordering); op transform must be idempotent (P5) | Causal-delivery violation = lost ops = divergence; op-transform bug = silent divergence |
| Last-Write-Win (LWW) with vector clocks | The data model is arbitrary (any JSON, any record); concurrent writes are acceptable to discard or tiebreak; a total order tiebreak (node id) is acceptable | Bounded convergence — causally-later writes win; concurrent writes are tiebroken deterministically (P4 bound via tiebreak) | Low — simple merge (compare clocks, pick winner); no CRDT library; small payload | Concurrent writes are silently discarded (lost update); tiebreak by wall time = clock-skew oscillation (P4 violation); no vector clock = no concurrent-write detection = silent loss |
| LWW with wall-clock timestamp only | The data model is ephemeral (cache, presence); lost updates are acceptable; the clock is roughly synchronized (NTP) | Weak — convergence eventually, but concurrent writes may oscillate with clock skew; no concurrent-write detection | Lowest — one timestamp per write; no clock vector | Clock skew = oscillation (P4 violation); concurrent writes silently lost; not a correctness-safe strategy for durable state |
| Three-way merge (application-specific) | The data model is structured (documents, forms); conflicts should surface to the user or a domain-specific resolver; the merge is field-level | Bounded if the merge function is correct (associative, commutative, idempotent — P4, P5); conflicts surface per field | High — application-specific merge function per type; UI for conflict resolution; user-facing conflict surface | Merge-function bug = silent divergence or oscillation; unbounded conflict UI = user fatigue |
- The default for structured state that must converge silently is a
**CRDT** (state-based for unreliable networks, operation-based for
bandwidth-constrained reliable transport). The default for
arbitrary JSON where lost concurrent updates are acceptable is
**LWW with vector clocks** (never wall-clock-only for durable
state). The default for user-facing documents where conflicts
should surface is **three-way merge** with a documented resolution
policy.
- The failure-mode column is the P4 check: every row except
wall-clock-only LWW carries a bounded failure mode (the bug is in
the implementation, not the strategy). Wall-clock-only LWW carries
an unbounded failure mode (clock skew = oscillation) and is a P4
violation for durable state. Use it only for ephemeral state
where lost updates are acceptable.
- The choice is a P4 decision (which bound) and a C1 decision
(which correctness cost). A CRDT guarantees convergence but
constrains the data model; LWW is flexible but discards concurrent
writes. Neither is universally correct; the matrix is the
decision tool.
## Observability of Sync (P10 Edge Observability Survives Partition)
- Sync is itself an observable operation: the merge count, the
conflict count, the convergence lag (time from reconnect to
convergence), and the divergent-replica count are first-class
signals. A sync that runs forever without converging is the
`edge-sync-loop` chaos anti-pattern (P4 breach); without
observability it is invisible until the user notices the stale
state.
- A conflict that is silently resolved should be logged (the
resolution policy applied, the discarded write's idempotency key,
the winning write's clock). A conflict that surfaces to the user
should be metricated (the conflict rate, the resolution time).
See `domains/observability/metrics` for the generic discipline.
- A divergent replica that has not converged after the expected
window is an incident; without a metric it is invisible (P10
breach). Wire sync convergence to an alert — the
`divergent-replica-count` is the sync analog of the messaging
`consumer-lag` metric.
## What Violates Sync Discipline
| Violation | Principle |
|-----------|-----------|
| Sync loop that oscillates forever (CRDT without merge-semantics, LWW without monotonic clock) | P4 Sync Conflicts are Bounded, Not Infinite |
| LWW with wall-clock timestamp only on durable state (clock skew = oscillation) | P4, C1 (no concurrent-write detection) |
| Merge function that mutates inputs in place (retried merge is not idempotent) | P5 Edge Operations are Idempotent, `domains/concurrency/patterns` |
| Big-bang schema change under sync (replicas fail the merge) | P4, `domains/data/migrations` |
| Conflict silently resolved with no log (the policy is invisible) | P10 Edge Observability Survives Partition |
| Divergent replica with no convergence-lag metric (invisible stale state) | P10, `domains/observability/metrics` |
| Three-way merge with an unbounded conflict UI (user fatigue, no termination) | P4 (the merge must terminate) |
| Operation-based CRDT without causal delivery (lost ops = divergence) | P4, C1 (the delivery contract is the bound) |
| Merge that surfaces every concurrent conflict to the user (no default policy) | P4, C3 (the default policy is the simplicity bound) |
| Sync with no convergence test (the merge is untested under partition) | P4, `domains/testing/pyramid` |
+41
View File
@@ -0,0 +1,41 @@
# Error Handling — First Principles
## 1. The Principles
### P1. Errors are Data
Errors are structured, typed, and intentional. They are values, not
exceptions to the flow of code.
### P2. Fail Loudly
Never swallow an error. Silent failure is worse than visible failure.
### P3. Fail Specifically
Generic errors are debugging enemies. "Something went wrong" is
never acceptable.
### P4. Preserve Context
Errors carry where (file, line, function), when (timestamp, request),
why (cause), and what (user-facing message).
### P5. Recoverable When Possible
Retry, fallback, or degrade. Do not crash what can be salvaged.
### P6. Unrecoverable Means Stop
When recovery is impossible or unsafe, fail fast. Do not limp on
after fatal errors.
### P7. Errors are Boundaries
Define how errors cross API, service, and module boundaries. Translation
is explicit, not accidental.
### P8. User-Facing Errors are UX
Error messages are a feature. They are written for the user, not the
developer.
### P9. Errors are Logged
Even when handled, errors are recorded. The handling is the recovery;
the log is the memory.
### P10. Errors Don't Lie
Never catch what you cannot handle. Never claim success on failure.
Never claim failure on success.
+94
View File
@@ -0,0 +1,94 @@
# Error Patterns — Derived Rules
> Derives from `domains/errors/first-principles.md`. Common error-handling patterns and when to use them.
## Pattern 1: Result Type (P1 Errors are Data)
```typescript
type Result<T, E> = { ok: true; value: T } | { ok: false; error: E };
function divide(a, b): Result<number, string> {
if (b === 0) return { ok: false, error: "division by zero" };
return { ok: true, value: a / b };
}
```
- Errors are values, not exceptions. The caller handles them explicitly.
- Use when errors are expected (parsing, validation, fallible operations).
- Avoid when errors are truly exceptional (out of memory, programmer error) — use exceptions/panics.
## Pattern 2: Sentinel Error (P4 Preserve Context)
```go
var ErrNotFound = errors.New("not found")
if errors.Is(err, ErrNotFound) { ... }
```
- A sentinel is a known error value the caller checks against.
- Use for a small, known set of error conditions.
- Wrap with context: `fmt.Errorf("load user %d: %w", id, ErrNotFound)`.
## Pattern 3: Typed Error (P1, P3 Fail Specifically)
```rust
enum AppError {
NotFound(String),
Invalid(String),
Internal(String),
}
```
- A typed error carries the kind and the detail.
- The caller matches on kind; the detail is for logging/display.
- Use when there are distinct error categories the caller handles differently.
## Pattern 4: Error Wrapping (P4 Preserve Context)
```go
return fmt.Errorf("query users: %w", err)
```
- Wrap errors as they cross boundaries. The outer error says "what was happening"; the inner says "what went wrong."
- The error chain is the stack trace of intent. Read it top-down: "I was doing X, which failed because Y, which was caused by Z."
- Never wrap with a generic message ("operation failed"). Wrap with the specific operation.
## Pattern 5: Fail Fast (P6 Unrecoverable Means Stop)
```typescript
if (config.secret === undefined) throw new Error("config.secret is required");
```
- For unrecoverable conditions, fail immediately. Do not limp on.
- Use at startup: missing required config, missing database, missing secrets.
- Do not use for recoverable conditions (a 404 is recoverable; a missing secret is not).
## Pattern 6: Retry with Backoff (P5 Recoverable When Possible)
```python
for attempt in range(3):
try:
return do_thing()
except TransientError:
sleep(2 ** attempt)
raise PermanentError()
```
- Retry transient errors (network, 429, 5xx). Do not retry permanent errors (400, 401).
- Exponential backoff with jitter. A retry storm is worse than the original failure.
- Bounded retries. Infinite retry is infinite hang (P8 Timeout Discipline).
## Pattern 7: Circuit Breaker (P3 Defense in Depth via errors P7)
- After N consecutive failures, stop trying. Return a fallback or error immediately.
- Use for external dependencies (a downstream service, an API).
- The breaker resets after a cooldown. Protects the system and the downstream.
## What Violates Error Patterns
| Violation | Pattern |
|-----------|---------|
| `catch (e) { return null }` | (anti-pattern, P2 Fail Loudly) |
| `throw new Error("error")` | P3 Fail Specifically |
| Retry without backoff | P5, P8 |
| `return null` for "not found" | P1 (errors are data, not absence) |
| `throw` in a recovery path | P6 (fail fast in the wrong place) |
+176
View File
@@ -0,0 +1,176 @@
# ArgoCD — Derived Rules
> Derives from `domains/gitops-operators/first-principles.md`.
> Applies P1P10 to ArgoCD specifically. For the ArgoCD-vs-Flux
> decision, see the decision matrix at the end of this doc and in
> `flux.md`.
## What ArgoCD Is (P1 Git is the Source of Truth, P3 Pull, Don't Push)
- ArgoCD is a pull-based GitOps controller for Kubernetes. It runs
inside the target cluster, pulls desired state from git, and
reconciles the cluster to match. CI never holds `kubectl` rights
against the cluster (P3).
- An Application is a declarative binding of "this git path" to
"this cluster destination." The Application CRD is the unit of
reconciliation. The cluster is a derivative of git, never the
authority (P1).
- ArgoCD supports Helm charts, Kustomize overlays, ksonnet, and raw
manifests as source formats — see `domains/kubernetes/helm.md`
and `domains/kubernetes/kustomize.md`.
## Application CRD (P2 Declarative Over Imperative, P4 Continuous Reconciliation)
- An Application declares `source` (repo, path, revision, chart),
`destination` (server, namespace), and `syncPolicy`. The
reconciler loops continuously; drift is corrected automatically,
not on-demand (P4).
```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: payments-api
namespace: argocd
spec:
source:
repoURL: https://git.example.com/platform/payments
targetRevision: 1.2.3
path: manifests/prod
destination:
server: https://kubernetes.default.svc
namespace: payments
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=false
```
- `automated.prune: true` deletes resources removed from git.
`selfHeal: true` corrects hand-edited drift back to git (P8).
Disable both for workloads that need manual approval gates.
## App-of-Apps (P6 Operators Encode Domain Knowledge, C6 Composability)
- The App-of-Apps pattern: one root Application points at a git
directory of child Application manifests. The root app reconciles
the children; the children reconcile the workloads. This is the
ArgoCD expression of composition — a fleet of apps as a tree of
Applications.
- Use App-of-Apps for cluster bootstrapping (one repo, many
clusters, many apps). Do not use it as a substitute for a package
manager; if you are templating hundreds of near-identical
Applications, use a generator (ApplicationSet) instead.
## Sync Waves and Hooks (P4 Continuous Reconciliation, P7 Reversibility)
- Sync waves order resources within a sync: `PreSync``Sync`
`PostSync`. Use waves to run a job before a Deployment, or a
migration before the app that depends on it.
- Sync hooks (`PreSync`, `Sync`, `PostSync`, `SyncFail`) are
Resources annotated to execute at a wave boundary. A `SyncFail`
hook runs on sync failure — the abort path (P7).
- Wave ordering is a correctness mechanism, not a performance one.
Mis-ordered waves (e.g., app starts before its migration job)
are a correctness bug.
## Health and Status (P9 Failure is Observable and Surfaced)
- ArgoCD assesses every resource's health (`Healthy`, `Progressing`,
`Degraded`, `Missing`, `Suspended`) and surfaces the aggregate as
Application status. Sync status (`Synced`, `OutOfSync`) reports
drift against git.
- Health checks are pluggable via Lua scripts for custom CRDs. An
Operator-managed CRD without a health check reads as `Progressing`
forever — write one (see `operators.md`).
- Out-of-sync or degraded status must emit a notification (Slack,
PagerDuty, webhook). Silent drift is the bug (P9). Wire status to
`domains/observability/metrics.md`.
## Diff and Drift (P8 Reconcile, Don't Mutate by Hand, P4)
- `argocd app diff` shows the diff between git and live cluster.
A non-empty diff on a synced app is hand-edit drift — the
recovery is `selfHeal`, not a manual `kubectl apply` (P8).
- Drift detection runs continuously (P4). The gap between "git
changed" and "cluster matches git" is observable, not assumed.
## RBAC and SSO (P10 Least Privilege Reconciliation)
- ArgoCD's own RBAC governs who can view, sync, and admin
Applications. Bind to SSO (OIDC, SAML) for human identity; bind
the controller's service account to a Role scoped to the
namespaces it reconciles.
- The controller's credentials must not be `cluster-admin` (P10).
Use namespace-scoped Roles via `ApplicationSet` namespaces or
cluster-wide AppProject restrictions. See
`domains/kubernetes/rbac.md` and `domains/security/authorization.md`.
- AppProjects bound the blast radius of what an Application can
deploy (allowed repos, destinations, roles). One AppProject per
team or environment; the default project is for nothing in
production.
## Multi-Cluster (P4 Locality, P10)
- ArgoCD registers external clusters by secret. The controller
pulls from git and pushes to the registered cluster's API server.
The "pull, don't push" boundary (P3) is between the target
cluster's reconciler and CI — the controller-to-apiserver hop is
internal to the platform.
- Scope each registered cluster's credentials to the namespaces
ArgoCD manages there. Do not register a cluster with cluster-admin
and call it done (P10).
## Sync Windows (P5 Reversibility, P7)
- Sync windows restrict when automated sync runs (e.g., no syncs
during business hours, or syncs only in a maintenance window).
They are a reversibility mechanism: a bad commit lands in git,
but the sync window holds it until review.
- Sync windows do not replace health monitoring (P9). A degraded
app inside a window is still an incident.
## Secrets (P10, cross-link security/secrets)
- Do not store raw Secrets in the GitOps repo. Use a sealed-secret
controller (Bitnami Sealed Secrets, SOPS, External Secrets
Operator) so the git store holds encrypted material only. See
`domains/security/secrets.md` for the general secret-hygiene
principles.
## ArgoCD vs Flux — Decision Matrix (IDEATE-21, D-039)
| Axis | ArgoCD | Flux |
|------|--------|------|
| Architecture | Monolithic controller + Application CRD | Composable GitOps Toolkit controllers (source, kustomize, helm, notification) |
| Reconciliation unit | Application (one CRD per app) | Kustomization / HelmRelease (one per deploy unit) |
| UI | Web UI + CLI (full dashboard, tree view, diff viewer) | CLI-first; UI via Weave GitOps or FluxUI (add-on) |
| Sync model | Periodic poll or webhook; sync waves + hooks | Poll + webhook; runs continuously, no explicit sync waves |
| Multi-cluster | One ArgoCD manages many clusters (hub-and-spoke) | One Flux per cluster (per-cluster autonomy) |
| Templating in repo | Helm, Kustomize, ksonnet, raw manifests, Jsonnet | Helm, Kustomize, raw manifests |
| RBAC | Built-in RBAC + SSO + AppProjects | Kubernetes RBAC (no built-in RBAC layer) |
| Progressive delivery | Argo Rollouts (sister project, tight integration) | Flagger (sister project, tight integration) |
| Best for | Teams wanting a UI, multi-cluster from one pane, App-of-Apps bootstrapping | Teams wanting composable controllers, per-cluster autonomy, minimal footprint |
| Watch out for | Monolithic controller scaling, UI as ops crutch, AppProject sprawl | No native UI, steeper learning curve, manual multi-cluster orchestration |
- Use ArgoCD when you want a UI, central multi-cluster management,
and sync-wave ordering. Use Flux when you want composable
controllers, per-cluster autonomy, and a minimal footprint.
- Both are CNCF graduated and both implement the OpenGitOps
principles. The choice is architectural fit, not correctness. See
`flux.md` for the Flux-side perspective.
## What Violates ArgoCD Discipline
| Violation | Principle |
|-----------|-----------|
| CI pipeline with `kubectl` rights pushing to the cluster | P3 Pull, Don't Push |
| `argocd app set` used as the steady state instead of git | P1 Git is the Source of Truth |
| `selfHeal: false` on a prod app with no manual gate | P8 Reconcile, Don't Mutate by Hand |
| Controller ServiceAccount bound to `cluster-admin` | P10 Least Privilege Reconciliation |
| Sync failure with no notification wired | P9 Failure is Observable and Surfaced |
| AppProject with no destination restrictions in prod | P10 Least Privilege Reconciliation |
| Raw Secret in the GitOps repo | P10, `domains/security/secrets.md` |
| Manual `kubectl edit` on an ArgoCD-managed resource | P8 Reconcile, Don't Mutate by Hand |
@@ -0,0 +1,131 @@
# GitOps + Operators — First Principles
## 1. The Principles
### P1. Git is the Source of Truth
Desired state lives in a versioned, immutable git store. The
cluster is a derivative of git, never the authority. If a state
exists only in the cluster and not in git, it is drift, not truth.
The commit history is the audit trail and the rollback path.
### P2. Declarative Over Imperative
Express the desired cluster state, not the commands to reach it.
A manifest says what should exist; the reconciler makes it so.
Imperative `kubectl` is for inspection and incident response, not
for the steady state. This is the GitOps expression of
`domains/kubernetes/P1 Declarative Desired State` and
`domains/infrastructure-as-code/P1 Declarative Intent`.
### P3. Pull, Don't Push
Agents running inside the target pull desired state from git; the
target never accepts outside push credentials. No CI pipeline holds
`kubectl` rights against the production cluster. The cluster reaches
out to git, not the other way around. This is the security primitive
of GitOps: the blast radius of a compromised CI is bounded by what CI
can push, and a pull model gives CI nothing to push.
### P4. Continuous Reconciliation
The reconciliation loop is the primitive. Drift is detected and
corrected automatically, not on-demand. A manual `apply` is an
exception, not the workflow. The loop runs continuously; the gap
between "git changed" and "cluster matches git" is measured in
seconds, not tickets.
### P5. State is Immutable and Versioned
Every change to desired state is a commit. History is the audit
trail and the rollback path. A revert is a rollback; a force-push is
history deletion. The git store is treated like
`domains/infrastructure-as-code/P3 State is Truth` — lose it or
tamper with it, and you lose the ability to reason about the system.
### P6. Operators Encode Domain Knowledge
Operational expertise lives as CRDs plus controllers, not as
runbooks that humans must remember. An operator is a control loop
that encodes how to reconcile a specific domain (a database, a
message queue, a certificate). The operator is the deepest
expression of `domains/kubernetes/P1 Declarative Desired State`
the domain knowledge is the desired state.
### P7. Progressive Delivery is Reversible by Construction
Canary and blue-green are staged, metric-gated, and one-command
abortable. Promotion without a rollback path is a violation. A
rollout that cannot be aborted is a deploy, not a progressive
delivery. This is the GitOps extension of
`domains/devops/P5 Progressive Delivery` and
`domains/kubernetes/P10 Roll Forward, Roll Back`.
### P8. Reconcile, Don't Mutate by Hand
Manual `kubectl apply` or `kubectl edit` on a GitOps-managed
resource is an incident. The reconciler will overwrite the hand
edit on the next loop; the hand edit was never truth. Drift back to
git is the recovery, not the failure. This is the GitOps angle on
`domains/infrastructure-as-code/P9 Drift is Recoverable`.
### P9. Failure is Observable and Surfaced
Sync failures, health degradation, and rollout-stall events emit
status and notifications. Silent drift is the bug. A GitOps
controller that fails to sync without surfacing the failure has
violated the contract — you cannot fix what you cannot see
(`domains/observability/metrics.md`).
### P10. Least Privilege Reconciliation
The controller's credentials are scoped to the namespaces and
resources it reconciles. No `cluster-admin` GitOps robots. One
credential set per boundary; the reconciler sees only what it
reconciles. This is the GitOps angle on
`domains/kubernetes/P7 RBAC by Intent, Not Identity` and
`domains/security/authorization.md`.
## 2. Core Principle Trace
Each GitOps + Operators P-rule derives from one or more core
C-rules (C1C8). The matrix extension lands in P4 of the v0.3
plan; the traces below are authoritative.
| P-rule | Core | Why |
|--------|------|-----|
| P1 Git is the Source of Truth | C1, C5 | Correctness of state; reversibility via history |
| P2 Declarative Over Imperative | C2, C3 | Clarity of intent; simplicity of mental model |
| P3 Pull, Don't Push | C1, C4 | Correctness via security; locality of credentials |
| P4 Continuous Reconciliation | C7, C1 | Observability of drift; correctness of convergence |
| P5 State is Immutable and Versioned | C5 | Reversibility via version history |
| P6 Operators Encode Domain Knowledge | C6, C2 | Composability of expertise; clarity of operational intent |
| P7 Progressive Delivery is Reversible | C5, C1 | Reversibility of promotion; correctness of abort |
| P8 Reconcile, Don't Mutate by Hand | C1, C7 | Correctness of single source; observability of drift |
| P9 Failure is Observable and Surfaced | C7 | Observability of reconciliation |
| P10 Least Privilege Reconciliation | C1, C8 | Correctness via security; economy of trust |
## 3. What Violates These Principles
| Violation | Principle Breached |
|-----------|-------------------|
| CI pipeline pushes manifests to the cluster | P3 Pull, Don't Push |
| A resource exists in the cluster but not in git | P1 Git is the Source of Truth |
| `kubectl edit` on a GitOps-managed resource | P8 Reconcile, Don't Mutate by Hand |
| Reconciler with `cluster-admin` ClusterRoleBinding | P10 Least Privilege Reconciliation |
| Sync failure with no status or notification | P9 Failure is Observable and Surfaced |
| Canary with no abort/rollback path | P7 Progressive Delivery is Reversible |
| Operator runbook that exists only in a wiki | P6 Operators Encode Domain Knowledge |
| Reconciler that applies on a cron, not continuously | P4 Continuous Reconciliation |
| Force-push rewrites GitOps repo history | P5 State is Immutable and Versioned |
| Imperative deploy script as the steady state | P2 Declarative Over Imperative |
## 4. Relationship to Other Domains
GitOps + Operators is the deployment-automation layer above
`domains/kubernetes/` and `domains/infrastructure-as-code/`. It
borrows their declarative-reconciliation model and adds the
git-as-source-of-truth and pull-based credential boundaries. Cross
links are one-directional (per D-026 extended):
- `domains/kubernetes/P1 Declarative Desired State` ← P2
- `domains/kubernetes/P10 Roll Forward, Roll Back` ← P7
- `domains/infrastructure-as-code/P1 Declarative Intent` ← P2
- `domains/infrastructure-as-code/P3 State is Truth` ← P1, P5
- `domains/infrastructure-as-code/P9 Drift is Recoverable` ← P4, P8
- `domains/devops/P4 Rollback First` ← P5, P7
- `domains/devops/P5 Progressive Delivery` ← P7
- `domains/devops/P6 Configuration as Code` ← P1, P2
- `domains/security/secrets.md` ← P3, P10 (reconciliation credentials)
- `domains/security/supply-chain.md` ← P5 (signed, immutable provenance)
- `domains/observability/metrics.md` ← P4, P9 (reconciliation + rollout metrics)
+159
View File
@@ -0,0 +1,159 @@
# Flux — Derived Rules
> Derives from `domains/gitops-operators/first-principles.md`.
> Applies P1P10 to Flux specifically. For the ArgoCD-vs-Flux
> decision, see the decision matrix at the end of this doc and in
> `argocd.md`.
## What Flux Is (P1 Git is the Source of Truth, P3 Pull, Don't Push)
- Flux is a set of composable controllers — the GitOps Toolkit —
that run inside the target cluster, pull desired state from git
or OCI registries, and reconcile the cluster to match. CI never
holds `kubectl` rights against the cluster (P3).
- The composable-controller architecture is a C6 (Composability)
exemplar: each controller does one thing (source, kustomize, helm,
notification) and the controllers compose into a full GitOps
system.
- Flux supports Helm releases, Kustomize overlays, and raw
manifests — see `domains/kubernetes/helm.md` and
`domains/kubernetes/kustomize.md`.
## GitOps Toolkit Controllers (P6 Composability, P4 Continuous Reconciliation)
- **source-controller** — pulls git, Helm, OCI, and bucket sources;
emits artifacts (tarballs) with a digest. The source is the
pinned input to reconciliation (P5 versioning by digest).
- **kustomize-controller** — reconciles Kustomization CRDs against
the artifacts from source-controller. Runs continuously (P4).
- **helm-controller** — reconciles HelmRelease CRDs against Helm
charts from source-controller.
- **notification-controller** — emits events and notifications for
sync, health, and source-readiness events (P9).
- **image-automation-controller** (optional) — updates git with new
image tags when a policy matches, closing the "latest image"
loop declaratively.
## Kustomization CRD (P2 Declarative Over Imperative, P4)
- A Kustomization binds "this source" to "this target namespace"
with a reconciliation interval. The reconciler loops
continuously; drift is corrected automatically (P4).
```yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: payments-api
namespace: flux-system
spec:
sourceRef:
kind: GitRepository
name: platform
namespace: flux-system
path: ./manifests/prod
targetNamespace: payments
interval: 1m
prune: true
wait: true
healthChecks:
- apiVersion: apps/v1
kind: Deployment
name: payments-api
namespace: payments
```
- `prune: true` deletes resources removed from git. `wait: true`
waits for health checks before declaring the Kustomization ready.
Disable prune for workloads that need manual removal gates.
## HelmRelease CRD (P6 Composability, cross-link helm.md)
- A HelmRelease binds a Helm chart (from a HelmRepository or OCI
source) to target values and a target namespace. helm-controller
renders and applies it. See `domains/kubernetes/helm.md` for the
chart model.
- Pin the chart version in the HelmRepository or the HelmRelease.
Never float `latest` — unversioned charts drift (P5).
## OCI Sources (P5 State is Immutable and Versioned)
- source-controller can pull from OCI registries (Helm charts as
OCI artifacts, or generic OCI repositories). The digest is the
version — immutable by construction (P5).
- OCI sources close the supply-chain loop: the manifest is signed
and immutable in the registry, and Flux pulls it by digest. Cross-
link `domains/security/supply-chain.md` for signed-provenance
principles.
## Reconciliation and Drift (P4 Continuous Reconciliation, P8)
- Flux reconciles on `interval` (default 1m) and on webhook event.
Drift between git and cluster is detected each interval and
corrected (with `prune` + `selfHeal` semantics).
- Hand-edited drift on a Flux-managed resource is overwritten on the
next loop — the hand edit was never truth (P8). The recovery is
to fix git, not to `kubectl apply`.
## Notifications and Events (P9 Failure is Observable and Surfaced)
- notification-controller emits events for source readiness, sync
success/failure, and health transitions. Wire them to Slack,
PagerDuty, or a webhook. Silent drift is the bug (P9).
- Events flow to `domains/observability/metrics.md` via the
notification controller's provider model — sync and health as
first-class signals.
## RBAC and Multi-Cluster (P10 Least Privilege Reconciliation, P4)
- Flux's controllers run with a ServiceAccount in `flux-system`.
Scope that account to the namespaces Flux reconciles. Do not bind
it to `cluster-admin` (P10). See `domains/kubernetes/rbac.md` and
`domains/security/authorization.md`.
- Flux is per-cluster by design (one Flux install per cluster). For
multi-cluster, use one repo with per-cluster paths, or a fleet
tool that bootstraps Flux per cluster. Per-cluster autonomy is a
feature, not a limitation — it bounds the blast radius of a
compromised controller (P4 locality, P10).
## Secrets (P10, cross-link security/secrets)
- Do not store raw Secrets in the GitOps repo. Use the
SOPS-compatible decryption in kustomize-controller, or External
Secrets Operator, so the git store holds encrypted material only.
See `domains/security/secrets.md`.
## ArgoCD vs Flux — Decision Matrix (IDEATE-21, D-039)
| Axis | ArgoCD | Flux |
|------|--------|------|
| Architecture | Monolithic controller + Application CRD | Composable GitOps Toolkit controllers (source, kustomize, helm, notification) |
| Reconciliation unit | Application (one CRD per app) | Kustomization / HelmRelease (one per deploy unit) |
| UI | Web UI + CLI (full dashboard, tree view, diff viewer) | CLI-first; UI via Weave GitOps or FluxUI (add-on) |
| Sync model | Periodic poll or webhook; sync waves + hooks | Poll + webhook; runs continuously, no explicit sync waves |
| Multi-cluster | One ArgoCD manages many clusters (hub-and-spoke) | One Flux per cluster (per-cluster autonomy) |
| Templating in repo | Helm, Kustomize, ksonnet, raw manifests, Jsonnet | Helm, Kustomize, raw manifests |
| RBAC | Built-in RBAC + SSO + AppProjects | Kubernetes RBAC (no built-in RBAC layer) |
| Progressive delivery | Argo Rollouts (sister project, tight integration) | Flagger (sister project, tight integration) |
| Best for | Teams wanting a UI, multi-cluster from one pane, App-of-Apps bootstrapping | Teams wanting composable controllers, per-cluster autonomy, minimal footprint |
| Watch out for | Monolithic controller scaling, UI as ops crutch, AppProject sprawl | No native UI, steeper learning curve, manual multi-cluster orchestration |
- Use Flux when you want composable controllers, per-cluster
autonomy, and a minimal footprint. Use ArgoCD when you want a UI,
central multi-cluster management, and sync-wave ordering.
- Both are CNCF graduated and both implement the OpenGitOps
principles. The choice is architectural fit, not correctness. See
`argocd.md` for the ArgoCD-side perspective.
## What Violates Flux Discipline
| Violation | Principle |
|-----------|-----------|
| CI pipeline with `kubectl` rights pushing to the cluster | P3 Pull, Don't Push |
| HelmRelease with no pinned chart version | P5 State is Immutable and Versioned |
| Flux ServiceAccount bound to `cluster-admin` | P10 Least Privilege Reconciliation |
| Kustomization with no `healthChecks` on a prod app | P9 Failure is Observable and Surfaced |
| No notification provider wired for sync failures | P9 Failure is Observable and Surfaced |
| Raw Secret in the GitOps repo | P10, `domains/security/secrets.md` |
| Manual `kubectl edit` on a Flux-managed resource | P8 Reconcile, Don't Mutate by Hand |
| `interval: 24h` on a prod Kustomization (drift window too wide) | P4 Continuous Reconciliation |
+140
View File
@@ -0,0 +1,140 @@
# Operators — Derived Rules
> Derives from `domains/gitops-operators/first-principles.md`.
> Applies P6 (Operators Encode Domain Knowledge) primarily, with
> P1, P4, P8, P9, P10. Cross-links `domains/kubernetes/workloads.md`
> and `domains/kubernetes/rbac.md` for the underlying controller
> model, and `domains/infrastructure-as-code/modules.md` for the
> module-vs-operator boundary.
## What an Operator Is (P6 Operators Encode Domain Knowledge)
- An Operator is a Kubernetes controller that encodes human
operational knowledge as CRDs plus a control loop. The operator
reconciles a domain-specific resource (a database, a message
queue, a certificate, a ML model) to a desired state.
- The operator is the deepest expression of
`domains/kubernetes/P1 Declarative Desired State`: the domain
knowledge itself is the desired state. A runbook that lives only
in a wiki is operational knowledge that has not been encoded —
the operator is the encoding (P6).
- An operator runs inside the cluster, observes its CRDs, and acts.
It is a pull-based reconciler by construction — see
`domains/gitops-operators/first-principles.md` P3.
## CRDs and Controllers (P2 Declarative Over Imperative, P4 Continuous Reconciliation)
- A CustomResourceDefinition (CRD) defines the schema of the
domain resource. The controller watches instances of that CRD
and reconciles current → desired (P4).
- The CRD is the public contract of the operator. Version it
(`v1alpha1``v1beta1``v1`) and preserve backward
compatibility — see `domains/api/versioning.md` for the general
API-evolution principles. A CRD is an API surface, not an
internal type.
```yaml
apiVersion: postgres.example.com/v1
kind: PostgresCluster
metadata:
name: payments-db
namespace: payments
spec:
replicas: 3
version: "16"
storage:
size: 100Gi
storageClass: fast-ssd
backup:
schedule: "0 2 * * *"
retention: 7d
```
- The controller reconciles this spec: creates StatefulSets, PVCs,
Services, backup CronJobs. The user declares intent; the operator
makes it so (P2, P6).
## The Control Loop (P4 Continuous Reconciliation, P8)
- The loop watches CRD instances, compares current vs desired, and
acts to converge. Drift (a hand-deleted pod, a failed backup) is
detected and corrected each loop (P4).
- An operator-managed resource should not be hand-edited (P8). The
operator owns the subordinate resources (StatefulSets, PVCs); a
manual `kubectl edit` on a subordinate is drift the operator will
overwrite.
## Operator SDK and OLM (P6 Composability, C6)
- The Operator SDK scaffolds a controller from a CRD (Go, Ansible,
Helm). Use it to avoid re-implementing the controller boilerplate.
- Operator Lifecycle Manager (OLM) installs, updates, and manages
operators as first-class cluster components. OLM is the package
manager for operators — the operator analogue of
`domains/kubernetes/helm.md` for workloads.
- An operator published via OLM is a versioned, catalog-tracked
artifact. Pin the operator version; do not float `latest` (P5
applies to operators as much as to manifests).
## When to Write an Operator vs a Helm Chart (P6, C6 Composability)
| Axis | Helm chart | Operator |
|------|-----------|----------|
| Day-2 operations | None — chart installs, you operate | Encoded — operator reconciles lifecycle (backup, resize, failover, upgrade) |
| State | Static manifests | Live control loop watching CRDs |
| Day-1 install | Strong fit — package and install | Overkill if install is all you need |
| Day-2 reconcile | None — drift is manual | Continuous — drift corrected each loop |
| Domain knowledge | Lives in runbooks + on-call | Lives in the controller code |
| Best for | Off-the-shelf apps, stateless services, one-shot deploys | Stateful apps, complex lifecycles, day-2 automation (backup, scale, failover, version upgrades) |
| Watch out for | Templating complexity, no day-2 reconcile | Controller complexity, multi-team maintenance burden, scope creep |
- Write an operator when the day-2 operations (backup, failover,
resize, version upgrade) are non-trivial and repeated. Write a
Helm chart when install is all you need and day-2 is run by a
human or a separate tool.
- Do not write an operator to wrap a Helm chart and call it day-2
automation — that is a Helm chart with extra steps. See
`domains/infrastructure-as-code/modules.md` for the
module-vs-copy boundary (the operator-vs-chart boundary is its
analogue).
## Scope and Responsibility Boundaries (P10 Least Privilege, C6)
- An operator owns one domain. An operator that manages databases
and message queues and certificates is doing three jobs — split
it. Scope creep is the most common operator failure mode (P6
violation: the encoded knowledge is no longer coherent).
- The operator's ServiceAccount must be scoped to the resources it
manages (P10). A database operator that needs `cluster-admin` to
create a StatefulSet has the wrong RBAC — see
`domains/kubernetes/rbac.md` and `domains/security/authorization.md`.
- One operator per CRD family; one ServiceAccount per operator; one
namespace per operator (or a shared `operators` namespace with
strict RoleBindings). Default namespace is for nothing in
production.
## Failure and Observability (P9 Failure is Observable and Surfaced)
- An operator must surface its reconcile status on the CRD
(`status.conditions`, `status.observedGeneration`). A CRD with no
status is an operator that fails silently (P9).
- Wire operator events to notifications and metrics. A failed
backup, a stuck failover, a version-upgrade stall must emit a
signal — see `domains/observability/metrics.md`.
- An operator that reconciles but does not report health is a
black box. The GitOps controller (ArgoCD/Flux) will read it as
`Progressing` forever — write the health check (see `argocd.md`
"Health and Status").
## What Violates Operator Discipline
| Violation | Principle |
|-----------|-----------|
| Operator that manages databases + queues + certs | P6 Operators Encode Domain Knowledge (scope creep) |
| Operator ServiceAccount bound to `cluster-admin` | P10 Least Privilege Reconciliation |
| CRD with no `status.conditions` | P9 Failure is Observable and Surfaced |
| Operator with no health check wired to GitOps | P9, `argocd.md` Health and Status |
| Unversioned CRD (`v1` shipped without alpha/beta) | P5, `domains/api/versioning.md` |
| Manual `kubectl edit` on an operator-managed subordinate | P8 Reconcile, Don't Mutate by Hand |
| Operator that wraps a Helm chart and adds no day-2 logic | P6 (no knowledge encoded) |
| Operator runbook that exists only in a wiki | P6 Operators Encode Domain Knowledge |
@@ -0,0 +1,177 @@
# Progressive Delivery — Derived Rules
> Derives from `domains/gitops-operators/first-principles.md`.
> Applies P7 (Progressive Delivery is Reversible by Construction)
> primarily, with P4, P9. Cross-links `domains/devops/first-principles.md`
> P4 Rollback First and P5 Progressive Delivery, and
> `domains/observability/metrics.md` for the analysis signals.
## What Progressive Delivery Is (P7 Reversible by Construction)
- Progressive delivery shifts traffic in stages (canary, blue-green)
gated by analysis (metrics, counters, error rates). Each stage is
metric-checked; a failed gate aborts the rollout and reverts to
the prior stable version. Promotion without a rollback path is a
violation (P7).
- Progressive delivery is the GitOps extension of
`domains/devops/P5 Progressive Delivery` and
`domains/kubernetes/P10 Roll Forward, Roll Back`. The k8s rolling
update is the floor; progressive delivery adds metric-gated
promotion and one-command abort.
- Two sister projects dominate: **Argo Rollouts** (Argo ecosystem)
and **Flagger** (Flux ecosystem). Both implement the same pattern
— a Rollout CRD replaces a Deployment, an analysis drives the
gates, an abort reverts traffic.
## The Rollout CRD (P2 Declarative Over Imperative, P7)
- A Rollout (Argo Rollouts) or Canary/Flag (Flagger) is a CRD that
replaces the Deployment as the reconciled resource. It declares
the strategy (canary, blue-green), the traffic split, and the
analysis gates. The controller reconciles traffic and pods to
match.
```yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: payments-api
namespace: payments
spec:
replicas: 10
selector:
matchLabels:
app: payments-api
template:
metadata:
labels:
app: payments-api
spec:
containers:
- name: api
image: registry.example.com/payments-api:1.2.3
strategy:
canary:
trafficRouting:
istio:
virtualService:
name: payments-vs
routes: [primary]
steps:
- setWeight: 5
- pause: { duration: 2m }
- analysis:
templates:
- templateName: success-rate
- setWeight: 25
- pause: { duration: 5m }
- analysis:
templates:
- templateName: success-rate
- setWeight: 50
- pause: { duration: 5m }
- setWeight: 100
```
- Each `setWeight` shifts traffic; each `pause` holds for
observation; each `analysis` runs a metric gate. A failed
analysis aborts the rollout and reverts traffic to the stable
ReplicaSet (P7).
## Canary vs Blue-Green (P7, C3 Simplicity)
| Strategy | Mechanism | Cost | Best for |
|----------|-----------|------|----------|
| Canary | Shift a small % of traffic to the new version; increase on gate success | Low (few new pods) | Most production rollouts; metric-gated, gradual |
| Blue-Green | Run two full environments; switch traffic all-at-once | High (2× capacity) | Schema-breaking changes, instant rollback, low-frequency deploys |
- Canary is the default — it is reversible by construction (P7)
and economical (C8). Blue-green is for changes that cannot be
partial (a breaking schema migration, a full cutover).
- A canary with no analysis gate is a slow blue-green — it is not
progressive delivery. The gate is what makes it progressive (P7).
## Analysis Templates (P9 Failure is Observable and Surfaced, P7)
- An AnalysisTemplate declares the metric query, the success
threshold, and the count of samples. The rollout controller runs
the analysis at each gate; a failed analysis aborts the rollout.
```yaml
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
name: success-rate
namespace: payments
spec:
metrics:
- name: success-rate
interval: 1m
successCondition: result[0] >= 0.99
failureLimit: 2
provider:
prometheus:
address: http://prometheus.observability:9090
query: |
sum(rate(http_requests_total{job="payments-api",code!~"5.."}[2m]))
/
sum(rate(http_requests_total{job="payments-api"}[2m]))
```
- `successCondition` is the gate; `failureLimit` is the tolerance
for transient blips. A single failed sample aborts immediately if
`failureLimit: 0`; tolerate noise with `failureLimit: 2`.
- The metric is the abort signal — see `domains/observability/metrics.md`
for the SLI/SLO discipline that makes the gate meaningful. A gate
on an undefined SLO is a gate on noise.
## Argo Rollouts vs Flagger (P6 Composability, P7)
| Axis | Argo Rollouts | Flagger |
|------|---------------|---------|
| Ecosystem | Argo (ArgoCD sister project) | Flux (Flux sister project) |
| CRD | `Rollout` (replaces `Deployment`) | `Canary` / `Flag` (wraps a `Deployment`) |
| Traffic providers | Istio, NGINX, ALB, SMI, Traefik, Ambassador | Istio, NGINX, Linkerd, SMI, App Mesh, Gloo, Contour |
| Analysis sources | Prometheus, Datadog, Wavefront, NewRelic, CloudWatch, Graphite, Kayenta | Prometheus, Datadog, CloudWatch, Stackdriver, Elasticsearch, Graphite |
| Integration | Tight with ArgoCD (UI shows rollout) | Tight with Flux (events via notification-controller) |
| Learning curve | Rollout CRD replaces Deployment (migration cost) | Wraps existing Deployment (lower migration cost) |
| Best for | ArgoCD shops wanting rollout in the Argo UI | Flux shops wanting progressive delivery with minimal migration |
- Both implement the same pattern. The choice follows your GitOps
controller — Argo Rollouts with ArgoCD, Flagger with Flux. Mixing
is possible but not idiomatic.
## Abort and Rollback (P7 Reversible by Construction, P5)
- An abort reverts traffic to the stable ReplicaSet immediately. A
rollout without a tested abort is a prototype (P7).
- The abort must be one-command (or one-gate-failure). A
progressive delivery that requires manual rollback steps has
lost the "reversible by construction" property — it is a deploy
with extra steps.
- Test the abort path in staging. An abort that has never been
exercised will fail when you need it most — see
`domains/devops/first-principles.md` P4 Rollback First.
## Observability (P9 Failure is Observable and Surfaced)
- Progressive delivery is only as good as its metrics. A rollout
gated on a metric that is not tracked is ungated — the gate is
theater (P9).
- Wire rollout status (phase, weight, analysis result) to
notifications and dashboards. A stalled rollout with no signal is
silent drift (P9). See `domains/observability/metrics.md`.
- Cross-link `domains/kubernetes/workloads.md` for the underlying
Deployment/ReplicaSet model that progressive delivery replaces.
## What Violates Progressive Delivery Discipline
| Violation | Principle |
|-----------|-----------|
| Canary with no analysis gate | P7 Progressive Delivery is Reversible by Construction |
| Rollout with no tested abort path | P7, `domains/devops/P4 Rollback First` |
| Analysis gate on an undefined SLO | P9 Failure is Observable and Surfaced |
| Blue-green with no 2× capacity budget | C8 Economy (blue-green is a cost decision) |
| Rollout stalled with no notification | P9 Failure is Observable and Surfaced |
| Manual `kubectl` traffic shift on a Rollout-managed service | P8 Reconcile, Don't Mutate by Hand |
| `failureLimit: 0` on a noisy metric (constant false aborts) | P4 Continuous Reconciliation (gate noise tolerance) |
+173
View File
@@ -0,0 +1,173 @@
# Internationalization (i18n) — First Principles
> Grounded in Unicode ICU + CLDR, W3C i18n WG, BCP 47 / RFC 5646,
> ICU MessageFormat / FormatJS / i18next / Mozilla Fluent, the
> JavaScript `Intl` API, and WCAG 2.1 AA. The developer's language is
> one locale among many, not the neutral form.
## 1. The Principles
### P1. Source Language is a Locale, Not the Default
The developer's own language is one locale among many — it is not the
"neutral" or "unlocalized" form of the product. Strings are extracted
from day one, addressed by key, and routed through a locale resource
layer even when only one locale is populated. Treating the source
language as the default produces hidden concatenations, hardcoded
grammar assumptions, and a translation debt that compounds until the
first second locale arrives — at which point the fix is a rewrite, not
a patch. The source locale is `en-US` (or whatever the team writes in);
it is not `null`. This is the i18n angle on `domains/uiux/copywriting.md`:
copy lives in resources, not in code.
### P2. Locale Identifiers are Standardized
Use BCP 47 language tags (`en-US`, `ar-EG`, `zh-Hans-CN`, `pt-BR`).
No ad-hoc locale codes, no two-letter-only hacks, no invented keys.
The tag carries language, script (when needed), and region (when
needed); it is the contract between the resource layer, the
formatting layer, and the runtime. A locale identifier that is not
BCP 47 is a key that cannot be resolved by any standard tool, which
is a correctness violation. Cross `domains/data/schema-design.md`:
locale identifiers are a data shape with a defined vocabulary.
### P3. Resources are External, Not Inline
User-facing strings live in locale resource files (`.po`, JSON,
Fluent `.ftl`, ICU Resource Bundle), never concatenated inline in
code. Inline strings are invisible to the translation pipeline,
unversionable as a unit, and untestable for completeness. String
concatenation in code (`"Welcome, " + name + "!"`) is the cardinal
violation: it bakes in source-language grammar and breaks for every
locale with different word order. Resources are the boundary; code
addresses strings by key, the resource layer resolves the key to the
locale. This is the i18n angle on C4 Locality: strings and their
locale-specific consequences live together in the resource, not
scattered across code.
### P4. Plural and Gender are Parameterized
Plural forms, gender, and select are expressed with ICU MessageFormat
(or an equivalent parameterized formatter), never with `if (n == 1)`
branching in code. Plural rules are locale-specific — English has
one/other, Arabic has six categories, Russian has three — and a
hand-rolled branch encodes exactly one locale's rules while pretending
to be universal. The formatter is the contract; the resource carries
the variants; the code passes the count and lets the formatter choose.
A `if (n == 1)` plural is a C1 (Correctness) violation masquerading as
a shortcut.
### P5. Formatting is Locale-Aware
Dates, times, numbers, currencies, units, and relative time are
formatted via ICU / CLDR / the JavaScript `Intl` API — never
hand-rolled. A hand-rolled date formatter encodes one locale's
conventions and silently produces wrong output for every other locale
(mm/dd/yyyy vs dd/mm/yyyy is the canonical failure). CLDR is the
source of truth for locale data; `Intl` is the runtime that exposes
it. Formatting correctness is observable: a misformatted date is a
wrong answer in the user's locale, even if it is "right" in the
developer's. Cross `domains/api/error-responses.md` for localized
error messages at API boundaries.
### P6. Text Direction is a Layout Primitive
RTL and bidi are first-class layout concerns, not a CSS afterthought.
Logical CSS properties (`margin-inline-start`, `padding-block-end`,
`inset-inline-end`) over physical (`margin-left`, `padding-top`). The
`dir` attribute is set on the document and on subtrees; the bidi
algorithm (UAX #9) handles inline reordering. A layout that assumes
LTR is a layout that is wrong for `ar`, `he`, `fa`, `ur`, and any
RTL-mixed context. Text direction is not a skin — it is a structural
property of the layout, and fixing it late is a rewrite. This is the
i18n angle on `domains/uiux/accessibility.md`: RTL support is an
accessibility concern for non-Latin-script users.
### P7. Layout Accommodates Expansion
Translated text expands and contracts — German is ~30% longer than
English, Japanese often shorter, RTL mirroring shifts every visual
anchor. Layouts are flexible: no fixed pixel widths for text, no
truncation without an ellipsis-and-title strategy, no
`white-space: nowrap` on translatable strings. A layout that breaks
on a 30% expansion is a layout that is wrong for most of the world's
locales. Designing for the worst-case expansion up front is cheaper
than reworking every screen when the first long-form locale ships.
### P8. Pseudo-Locales Test Early
Test with pseudo-locales (accented, lengthened, RTL-mirrored, brack-
enclosed) before real translations arrive. A pseudo-locale run
surfaces hardcoded strings, layout overflow, broken concatenation,
and LTR assumptions while the fix is still cheap — the translator
hasn't been paid yet, and the string freeze hasn't happened. Finding
these bugs after real translation is a C5 (Reversibility) violation:
the cost of undoing is now a re-translation. Cross
`domains/testing/fixtures.md` and `domains/testing/pyramid.md` for
where pseudo-locales sit in the testing pyramid.
### P9. Images and Icons are Cultural
Icons, colors, gestures, and imagery are locale-sensitive. A
mailbox icon means "email" in the US and "mail" in Japan — but a
green checkmark means "correct" in the West and "incorrect" in some
East Asian contexts. A thumbs-up is positive in much of the world
and an insult in parts of the Middle East. Avoid locale-bound symbols
as universal; parameterize imagery per locale where the symbol is not
globally neutral. Icons are not a universal language; they are a
locale with a picture. This is a C2 (Clarity) concern: an icon whose
meaning changes by locale is unclear to the reader it was not drawn
for.
### P10. Translation is Reversible and Versioned
Resource files are versioned alongside code; a bad translation is a
rollback, not a hot-patch. Every locale resource has a history
(what shipped when), a provenance (which translator / which service),
and a rollback path. A translation that breaks the UI is reverted to
the prior resource version, the same way a code regression is
reverted to the prior commit. Translations without version history
are anecdote, not artifact — you cannot tell what changed, when, or
why. This is the i18n angle on C5 Reversibility applied to the
resource layer.
## 2. Core Principle Trace
Each i18n P-rule derives from one or more core C-rules (C1C8). The
matrix extension lands in P4 of the v0.3 plan; the traces below are
authoritative.
| P-rule | Core | Why |
|--------|------|-----|
| P1 Source Language is a Locale, Not the Default | C2, C1 | Clarity of locale intent; correctness of treating source as one-of-many |
| P2 Locale Identifiers are Standardized | C2, C6 | Clarity of a standard vocabulary; composability with standard tools |
| P3 Resources are External, Not Inline | C4, C6 | Locality of strings and their locale consequences; composability of the resource layer |
| P4 Plural and Gender are Parameterized | C1, C6 | Correctness of locale-specific plural rules; composability of the formatter contract |
| P5 Formatting is Locale-Aware | C1, C7 | Correctness of formatted output; observability of format correctness |
| P6 Text Direction is a Layout Primitive | C1, C4 | Correctness of layout for RTL; locality of direction with the text it governs |
| P7 Layout Accommodates Expansion | C8, C3 | Economy of rework; simplicity of flexible layouts over per-locale overrides |
| P8 Pseudo-Locales Test Early | C7, C5 | Observability of i18n defects early; reversibility of fixing before translation |
| P9 Images and Icons are Cultural | C1, C2 | Correctness of locale-appropriate symbols; clarity of meaning across locales |
| P10 Translation is Reversible and Versioned | C5 | Reversibility of the resource layer |
## 3. What Violates These Principles
| Violation | Principle Breached |
|-----------|-------------------|
| A user-facing string hardcoded in source | P3 Resources are External, Not Inline |
| `"Welcome, " + name + "!"` string concatenation | P3 Resources are External, Not Inline |
| `if (n == 1) { return "item"; } else { return "items"; }` | P4 Plural and Gender are Parameterized |
| A locale code like `en_us` or `english` instead of `en-US` | P2 Locale Identifiers are Standardized |
| A hand-rolled date formatter (`getMonth() + 1 + "/" + getDay()`) | P5 Formatting is Locale-Aware |
| `margin-left: 10px` on a translatable layout | P6 Text Direction is a Layout Primitive |
| A fixed-width text container that overflows on German | P7 Layout Accommodates Expansion |
| First i18n test runs against real translations, not pseudo-locales | P8 Pseudo-Locales Test Early |
| A thumbs-up icon shipped as universally positive | P9 Images and Icons are Cultural |
| Resource files with no git history or no rollback path | P10 Translation is Reversible and Versioned |
| The source language treated as the "unlocalized" default | P1 Source Language is a Locale, Not the Default |
## 4. Relationship to Other Domains
i18n is the locale-awareness layer that `domains/uiux/` consumes and
that `domains/api/` surfaces at boundaries. It borrows the testing
discipline of `domains/testing/` and the data-shape discipline of
`domains/data/`. Cross-links are one-directional (per D-026 extended):
- `domains/uiux/copywriting.md` ← P1, P3 (strings live in resources)
- `domains/uiux/accessibility.md` ← P6 (RTL is an a11y concern for non-Latin users)
- `domains/uiux/components.md` ← P6, P7 (layout primitives that survive direction and expansion)
- `domains/api/error-responses.md` ← P5 (localized error messages)
- `domains/data/schema-design.md` ← P2, P3 (locale data shapes)
- `domains/testing/fixtures.md` ← P8 (pseudo-locale fixtures)
- `domains/testing/pyramid.md` ← P8 (pseudo-locale tier mapping)
- `domains/testing/first-principles.md` ← P8 (testing discipline for locale)
+141
View File
@@ -0,0 +1,141 @@
# Formatting — Derived Rules
> Derives from `domains/i18n/first-principles.md`. Covers P2 (Locale
> Identifiers Standardized), P4 (Plural/Gender Parameterized), and P5
> (Formatting is Locale-Aware). Referenced by `locale-resources.md`
> (the formatter resolves the messages) and `testing-i18n.md` (the
> formatted output is what snapshots assert).
## Formatting is Locale-Aware (P5 Formatting is Locale-Aware)
- Dates, times, numbers, currencies, units, and relative time are
formatted via ICU / CLDR / the JavaScript `Intl` API — never
hand-rolled. CLDR is the source of truth for locale data; `Intl`
is the runtime that exposes it.
- A hand-rolled formatter encodes one locale's conventions and
silently produces wrong output for every other locale. The
canonical failure is date format: `mm/dd/yyyy` (US) vs
`dd/mm/yyyy` (most of the world) vs `yyyy-mm-dd` (ISO, sortable).
Picking one and calling it done is a correctness violation in
every locale it is wrong for.
## BCP 47 Tags Drive Formatting (P2 Locale Identifiers Standardized)
- Every formatter takes a BCP 47 locale tag. The tag is the contract
between the resource layer and the formatting layer: the same tag
that selects the resource selects the formatter.
- A locale tag that is not BCP 47 cannot be resolved by `Intl`, ICU,
or CLDR — the formatter returns the runtime default, which is the
developer's locale, not the user's. This is why P2 is a
prerequisite of P5: you cannot format for a locale you cannot name.
## The Intl Surface (ICU/CLDR in the Browser and Node)
| API | Formats | Notes |
|-----|---------|-------|
| `Intl.DateTimeFormat` | Dates, times, date+time, time zones | Calendar (`buddhist`, `hebrew`, `islamic`), numbering system (`arab`, `hanidec`) via locale tag extensions |
| `Intl.NumberFormat` | Numbers, currencies, units, percent | Notation (`compact`, `scientific`), grouping, sign display |
| `Intl.RelativeTimeFormat` | "3 days ago", "in 2 months" | Locale-specific phrasing; numeric vs auto |
| `Intl.PluralRules` | Plural category for a count | `one`, `few`, `many`, `other`, `zero`, `two` per CLDR — the engine ICU MessageFormat uses |
| `Intl.ListFormat` | "a, b, and c" | Conjunction / disjunction / unit lists, locale-specific separators |
| `Intl.Collator` | Locale-aware string sorting | Strength (`base`, `accent`, `case`); numeric collation |
- All of these are built on ICU/CLDR; they are the runtime baseline.
Use them. A `moment.js`-style hand-rolled format string
(`"MM/DD/YYYY"`) is a relic of the pre-`Intl` era and a P5
violation in any locale-aware code path.
## Dates and Times
```
// Correct — Intl, locale-aware
new Intl.DateTimeFormat("ar-EG", {
dateStyle: "full",
timeStyle: "short",
}).format(new Date());
// "الأربعاء، ٧ نوفمبر ٢٠٢٤، ٣:١٥ م"
// Wrong — hand-rolled, source-locale only
const d = new Date();
const s = (d.getMonth() + 1) + "/" + d.getDate() + "/" + d.getFullYear();
// "11/7/2024" — meaningless in most locales
```
- Time zones are not locales. A locale tells you *how to format* a
timestamp; a time zone tells you *what instant* it refers to. Do
not derive one from the other (`ar-EG` is not a time zone).
Format with the user's locale; render in the user's time zone;
store in UTC.
## Numbers, Currencies, Units
```
new Intl.NumberFormat("de-DE", { style: "currency", currency: "EUR" })
.format(1234.56); // "1.234,56 €"
new Intl.NumberFormat("ar-EG", { style: "currency", currency: "EGP" })
.format(1234.56); // "١٬٢٣٤٫٥٦ ج.م."
new Intl.NumberFormat("en-US", { style: "unit", unit: "kilometer-per-hour" })
.format(100); // "100 km/h"
```
- The currency code (`EUR`, `EGP`, `USD`) is ISO 4217; the locale
determines the symbol, grouping, and placement. A hand-rolled
`"$" + amount` is wrong for `de-DE` (symbol, grouping, placement
all differ).
## Plural Rules (P4 Plural/Gender Parameterized)
- `Intl.PluralRules` returns the CLDR plural category for a count in
a given locale. ICU MessageFormat uses this category to select the
variant from the resource (`locale-resources.md`).
- Never branch on the raw count in code. The count goes to the
formatter; the formatter consults `PluralRules` for the locale;
the resource carries the variant for that category.
```
// ICU MessageFormat (FormatJS)
new Intl.MessageFormat(
"{count, plural, one {# item} other {# items}}",
"en-US"
).format({ count: 1 }); // "1 item"
// ar-EG — six categories; the code is identical, only the
// resource differs.
```
## Gender and Select
- ICU MessageFormat also supports `{gender, select, male {...} female {...} other {...}}`
for gendered agreement and `{case, select, ...}` for general
disjunction. These live in the resource, not in code branches.
- A `switch (gender)` in code that picks a string is the same
violation as `if (n == 1)`: it encodes one locale's grammar in
code and breaks for every locale with different agreement rules.
## What Violates Formatting Discipline
| Violation | Principle |
|-----------|-----------|
| `getMonth() + 1 + "/" + getDay()` hand-rolled date | P5 Formatting is Locale-Aware |
| `"$" + amount` hand-rolled currency | P5 Formatting is Locale-Aware |
| `if (n === 1) "item" else "items"` plural branch | P4 Plural and Gender are Parameterized |
| `moment("MM/DD/YYYY")` format string in locale-aware code | P5 Formatting is Locale-Aware |
| Deriving time zone from locale tag | P2 Locale Identifiers are Standardized |
| A non-BCP-47 tag passed to `Intl` (silently falls back) | P2 Locale Identifiers are Standardized |
| `switch (gender)` selecting strings in code | P4 Plural and Gender are Parameterized |
| Storing timestamps in local time, not UTC | P5 Formatting is Locale-Aware |
## Relationship to Other Domains
- `domains/api/error-responses.md` — API error messages are
formatted for the requesting locale; the error code is stable, the
message is locale-formatted.
- `domains/data/schema-design.md` — locale identifiers, currency
codes, and time zones are data contracts; treat them as schema
(`en-US`, `EUR`, `UTC`), not free text.
- `domains/i18n/locale-resources.md` — the resource layer carries
the parameterized messages this formatter resolves.
- `domains/testing/fixtures.md` — formatted output per locale is the
fixture; snapshot tests assert against it.
+138
View File
@@ -0,0 +1,138 @@
# Locale Resources — Derived Rules
> Derives from `domains/i18n/first-principles.md`. Covers P1 (Source
> Language is a Locale), P2 (Locale Identifiers Standardized), P3
> (Resources External, Not Inline), P4 (Plural/Gender Parameterized),
> and P10 (Translation Reversible and Versioned). Referenced by
> `formatting.md` (strings the formatter resolves) and
> `rtl-bidi.md` (the `dir` the resource layer carries).
## Resources are the Boundary (P3 Resources are External, Not Inline)
- User-facing strings live in locale resource files, addressed by
key. Code references a key; the resource layer resolves the key to
the active locale. The source language is itself a locale
(`en-US`), not a fallback baked into code.
- String concatenation in code (`"Welcome, " + name + "!"`) is the
cardinal violation: it bakes in source-language word order and
breaks for every locale with different grammar. Replace every
concatenation with a parameterized message:
`t("welcome", { name })`.
- The resource is the single place a string lives. Editing a string
in code instead of the resource is a locality violation (C4): the
string and its locale consequences now live apart.
## Resource File Formats
| Format | Shape | When | Notes |
|--------|-------|------|-------|
| `.po` / `.pot` | gettext; msgid → msgstr, plural headers | Server-side, GNU ecosystem, PHP/Python/C | Mature tooling (`xgettext`, `msgmerge`); supports plural categories via header |
| JSON (flat or namespaced) | `{ "key": "value" }` per locale | JS/web, i18next, FormatJS | Simple, machine-readable, but no native plural support — wrap with ICU MessageFormat |
| Fluent `.ftl` | Mozilla FTL; asymmetric, resolver-driven | Browser-grade l10n, asymmetric translations | One message can resolve differently per locale without code changes; supports attributes, selectors |
| ICU Resource Bundle | ICU binary/text resources | ICU-native, JVM, C++ | Tightest integration with ICU formatting/CLDR; steeper tooling |
- None is advocated over the others. The choice is ecosystem fit,
not correctness. All four satisfy P3/P4 when used as the boundary.
- A custom format (a hand-rolled `.csv` of strings) is a violation:
it is unsupported by standard tooling, has no plural grammar, and
cannot compose with `formatting.md`'s ICU layer.
## Key Naming and Namespaces (P2 Locale Identifiers Standardized)
- Locale identifiers are BCP 47 tags (`en-US`, `ar-EG`, `zh-Hans-CN`).
No ad-hoc codes. The resource file is named for its locale:
`en-US.json`, `ar-EG.po`, `ftl/ar-EG/main.ftl`.
- Message keys are stable, semantic, and structured — not prose.
`checkout.cart.item_count` not `"You have 3 items in your cart"`.
A key that is the source string (`t("You have items")`) breaks the
moment the source copy is edited; the key must outlive the copy.
- Namespaces segment by surface (`checkout.*`, `errors.*`, `onboarding.*`)
so that a locale can be loaded incrementally and so that key
collisions across surfaces are impossible. A flat namespace with
thousands of keys is a C2 (Clarity) violation waiting to happen.
## Fallback Chains
- The fallback chain is explicit: requested locale → language-only
(`en` from `en-GB`) → default locale → key itself (last resort).
The default locale is declared once, not re-derived in every call
site.
- A missing key in the requested locale falling back silently to the
source locale is a P3 violation: the user is silently shown the
developer's locale, which is not the locale they asked for. Missing
keys must be observable (see `testing-i18n.md`).
- Fallback is a property of the resource layer, not of individual
components. A component that re-implements fallback is duplicating
a contract (C6 Composability violation).
## Plural and Gender in Resources (P4 Plural/Gender Parameterized)
- Plural variants live in the resource, selected by the formatter,
parameterized by the count. The code passes the count; the resource
carries the variants; the formatter picks the right one per the
locale's CLDR plural rules.
```
// JSON + ICU MessageFormat (FormatJS / i18next)
{
"cart.item_count": "{count, plural, one {# item} other {# items}}"
}
// ar-EG.json — six plural categories per CLDR
{
"cart.item_count": "{count, plural, zero {لا عناصر} one {عنصر واحد} two {عنصران} few {# عناصر} many {# عنصرًا} other {# عنصر}}"
}
```
- `if (n == 1)` branching in code is a violation regardless of
language. Arabic has six plural categories; Russian has three;
English has two. A two-branch `if` encodes exactly one locale's
rules and is wrong for every other.
## Extraction Tooling (P1, P3)
- Strings are extracted mechanically (e.g. `xgettext`, `i18next-
parser`, FormatJS babel plugin), not by hand-tagging. Mechanical
extraction produces a `.pot` template that translators work from;
the template is regenerated on every build.
- A string that cannot be extracted (built at runtime from
fragments) is a P3 violation: it is invisible to the pipeline. If
the extractor cannot see it, neither can the translator.
- The extracted template is versioned (`P10`): the diff between
templates is the change in translatable surface. A template that
is not committed is a contract that is not reviewable.
## Versioning and Rollback (P10 Translation Reversible and Versioned)
- Resource files are committed to git alongside code. A bad
translation is a `git revert` of the resource, not a hot-patch over
the translator's work. Every locale resource has history,
provenance (which translator / which service produced which
commit), and a rollback path.
- A locale resource that is generated by a translation service and
committed without review is a P10 violation: the resource is
versioned but the provenance is opaque. Review the diff the same
way you review a code diff.
## What Violates Locale-Resource Discipline
| Violation | Principle |
|-----------|-----------|
| `t("You have " + n + " items")` concatenation | P3 Resources are External, Not Inline |
| A custom `.csv` string store instead of a standard format | P3 Resources are External, Not Inline |
| Locale file named `english.json` not `en-US.json` | P2 Locale Identifiers are Standardized |
| `if (n == 1) { t("item") } else { t("items") }` in code | P4 Plural and Gender are Parameterized |
| A key equal to the source string (`t("Welcome back")`) | P2 / P10 — keys must outlive copy |
| Silent fallback to the source locale with no signal | P3 Resources are External, Not Inline |
| A runtime-built string the extractor cannot see | P3 Resources are External, Not Inline |
| Resource files committed by a bot with no human review | P10 Translation is Reversible and Versioned |
## Relationship to Other Domains
- `domains/uiux/copywriting.md` — copy lives in resources; UI
microcopy is the source content the resource layer carries.
- `domains/api/error-responses.md` — API error messages are locale-
resource keys resolved at the boundary, not inline strings.
- `domains/data/schema-design.md` — locale identifiers and resource
shapes are a data contract; treat them as schema.
- `domains/i18n/formatting.md` — the formatter resolves the
parameterized message this layer produces.
+117
View File
@@ -0,0 +1,117 @@
# RTL and Bidi — Derived Rules
> Derives from `domains/i18n/first-principles.md`. Covers P6 (Text
> Direction is a Layout Primitive) and P7 (Layout Accommodates
> Expansion). Referenced by `testing-i18n.md` (RTL coverage is an
> e2e tier). Grounded in W3C i18n bidi authoring, UAX #9, and
> `domains/uiux/accessibility.md`.
## Text Direction is a Layout Primitive (P6 Text Direction is a Layout Primitive)
- RTL and bidi are first-class layout concerns, not a CSS
afterthought. The layout is designed for both directions from the
first commit, not retrofitted when an RTL locale ships.
- Logical CSS properties over physical properties, always. The
browser resolves logical → physical from the `dir` attribute; the
code never has to.
| Physical (LTR-only) | Logical (dir-aware) | Resolves to in RTL |
|---------------------|---------------------|--------------------|
| `margin-left` | `margin-inline-start` | `margin-right` |
| `margin-right` | `margin-inline-end` | `margin-left` |
| `padding-left` | `padding-inline-start` | `padding-right` |
| `left: 0` | `inset-inline-start: 0` | `right: 0` |
| `text-align: left` | `text-align: start` | `text-align: right` |
| `float: left` | use flexbox/grid + `inline-start` where supported | mirrored |
- The `dir` attribute is set on the document root (`<html dir="rtl">`)
and on subtrees whose direction differs from the document
(`<span dir="ltr">` for an embedded Latin run). `dir` is the
contract the bidi algorithm (UAX #9) reads; do not fake direction
with `text-align` alone.
## The Bidi Algorithm (UAX #9)
- The Unicode bidi algorithm resolves inline reordering of mixed-
direction runs. The browser applies it; the author's job is to
mark direction correctly, not to reorder by hand.
- A string like `"The price is 15 USD"` in an RTL context renders
with the Latin run `"15 USD"` in LTR within the RTL line — the
algorithm handles it *if* the container's `dir` is set. Without
`dir`, numbers and Latin fragments drift to the wrong edge.
- `dir="auto"` on a container infers direction from the first strong
directional character of its content — useful for user-generated
content whose direction is unknown. `dir="auto"` is not a
replacement for `dir="rtl"` on a known-RTL document.
## Mirroring (Icons, Controls, Diagrams)
- Direction-aware icons mirror in RTL: a "back" arrow pointing left
in LTR points right in RTL. A "refresh" circular arrow does not
mirror. The rule: icons that imply direction mirror; icons that
imply time or rotation do not.
- Use `[dir="rtl"]` selectors or logical icon variants — never
`transform: scaleX(-1)` as a one-off hack scattered across
components. Centralize the mirroring rule (a token, a component
prop) so it is auditable.
- Numbers do not mirror. `"15 USD"` in an RTL line is still
`"15 USD"` left-to-right inside the bidi run; mirroring it to
`"DSU 51"` is a correctness violation.
- Diagrams and flowcharts: a left-to-right process flow in LTR is a
right-to-left flow in RTL. Decide per diagram whether the flow
mirrors (most do) or is direction-neutral (some scientific
schematics).
## Layout Accommodates Expansion (P7 Layout Accommodates Expansion)
- Translated text expands. German is ~30% longer than English;
Japanese is often shorter but taller; RTL mirroring shifts every
visual anchor. Layouts are flexible:
- No fixed pixel widths on translatable text containers.
- No `white-space: nowrap` on translatable strings.
- No `text-overflow: ellipsis` without a `title` carrying the full
string.
- Buttons sized to fit their longest locale variant, not the
source.
- A layout that breaks at +30% width is a layout that is wrong for
most of the world's locales. Designing for the worst case up front
is cheaper than reworking every screen when the first long-form
locale ships.
## Common Pitfalls
| Pitfall | Why it breaks | Fix |
|---------|---------------|-----|
| `margin-left` everywhere | In RTL the start is the right; `margin-left` leaves the right side unstyled | `margin-inline-start` |
| `text-align: left` for "default" alignment | In RTL the default is right; `left` pins content to the wrong edge | `text-align: start` |
| Icons hardcoded to LTR orientation | "Back" arrow points the wrong way in RTL | Mirror direction-implying icons via `[dir="rtl"]` |
| Numbers mirrored with the layout | Numbers are LTR inside RTL; mirroring produces garbage | Leave number runs LTR; the bidi algorithm handles embedding |
| Fixed `width: 120px` on a button | German button label overflows and truncates | `min-width` + `max-width` + flex; let content size |
| `position: absolute; left: 0` | Pins to the physical left in both directions | `inset-inline-start: 0` |
| Fake direction with `text-align` only | The bidi algorithm reads `dir`, not `text-align`; mixed runs reorder wrong | Set `dir` on the container |
## What Violates RTL/Bidi Discipline
| Violation | Principle |
|-----------|-----------|
| A layout with no `dir` attribute, assuming LTR | P6 Text Direction is a Layout Primitive |
| `margin-left` / `left: 0` / `text-align: left` throughout | P6 Text Direction is a Layout Primitive |
| A "back" arrow that points left in the RTL build | P6 Text Direction is a Layout Primitive |
| Numbers mirrored to read right-to-left | P6 Text Direction is a Layout Primitive |
| `width: 100px` on a text container that overflows in German | P7 Layout Accommodates Expansion |
| `white-space: nowrap` on a translated label | P7 Layout Accommodates Expansion |
| `dir` faked with `text-align` and no `dir` attribute | P6 Text Direction is a Layout Primitive |
| No RTL build until the first RTL locale ships | P6 Text Direction is a Layout Primitive |
## Relationship to Other Domains
- `domains/uiux/accessibility.md` — RTL support is an accessibility
concern for non-Latin-script users; WCAG 2.1 AA requires that
direction be set correctly.
- `domains/uiux/components.md` — components are built with logical
properties so they survive direction and expansion without per-
locale overrides.
- `domains/i18n/testing-i18n.md` — RTL coverage is an e2e-tier
test; pseudo-locale mirroring surfaces direction bugs early.
- `domains/i18n/locale-resources.md` — the `dir` is part of the
locale's metadata, carried alongside the resource bundle.
+142
View File
@@ -0,0 +1,142 @@
# Testing i18n — Derived Rules
> Derives from `domains/i18n/first-principles.md`. Covers P8
> (Pseudo-Locales Test Early) and the testing-discipline angle on
> P3 (Resources External), P5 (Formatting Locale-Aware), P6 (Text
> Direction), and P10 (Translation Versioned). Referenced by
> `locale-resources.md` (missing-key detection) and `rtl-bidi.md`
> (RTL coverage tier).
## Pseudo-Locales Test Early (P8 Pseudo-Locales Test Early)
- A pseudo-locale is a synthetic locale that transforms the source
strings to surface i18n defects before real translations arrive.
Three transforms cover the three defect classes:
| Pseudo-locale | Transform | Surfaces |
|---------------|-----------|----------|
| `en-XA` (accented) | `Wêlcômê tô thê çhêckôût` | Strings not extracted (raw source appears), encoding bugs |
| `en-XB` (lengthened / "long") | `Wᴇʟᴄᴏᴍᴇ ᴛᴏ ᴛʜᴇ ᴄʜᴇᴄᴋᴏᴜᴛ──────` (~30% longer, bracketed) | Layout overflow, fixed widths, truncation |
| `en-XC` (RTL-mirrored) | Source rendered with `dir="rtl"` and a Latin-in-RTL run | LTR-only layout assumptions, physical CSS properties |
- Pseudo-locale tests are cheap: they run against source strings, no
translator involved, no string freeze required. A failing pseudo-
locale run is a bug found at the cheapest possible point in the
pipeline. Finding the same bug after real translation is a C5
(Reversibility) violation: the fix now costs a re-translation.
## Pseudo-Locale → Testing Pyramid Mapping (IDEATE-28)
- The testing pyramid (`domains/testing/pyramid.md`) has three tiers;
i18n tests map to each tier with a distinct signal. The mapping is
deliberate: each tier catches a different class of defect, and
skipping a tier leaves a blind spot.
| Pyramid Tier | i18n Test | Defect Caught | Tooling Shape |
|--------------|-----------|---------------|---------------|
| **Unit** | Missing-key detection | A key referenced in code but absent from the resource bundle; a key present in the source locale but missing from a target locale | Static scan over the resource bundle + code AST; runs per file, no runtime |
| **Integration** | Snapshot per locale | Formatted output for a fixture input differs across locales in a way that breaks the contract (wrong plural, wrong date, overflow) | Render a known fixture through the formatter per locale; snapshot-diff against the recorded baseline |
| **e2e** | RTL coverage | The app renders and is navigable in `dir="rtl"`; no layout overflow, no off-screen controls, no LTR-pinned anchors | Browser-driven run against the `en-XC` pseudo-locale (or a real RTL locale); assert on layout, not just text |
- Unit is the broad base (fast, runs on every commit), e2e is the
narrow top (slow, runs on PR merge). Integration sits between.
This mirrors `domains/testing/pyramid.md` exactly — i18n is not a
special case; it is a domain that uses the same tiers.
## Unit Tier — Missing-Key Detection (P3 Resources External)
- A static scan compares the set of keys referenced in code against
the keys present in each locale bundle. A key in code but not in
`en-US` is a P3 violation (the string is not in the resource
layer). A key in `en-US` but not in `ar-EG` is a coverage gap —
the missing-key scan flags it before the locale ships.
- Missing keys fail the build, not the runtime. A missing key that
surfaces only when a user switches locale is a defect found in
production, which is the most expensive place to find it.
```
// tool output (illustrative)
// missing-key scan
[FAIL] ar-EG: key "checkout.cart.item_count" referenced in code,
absent from ar-EG.json
[FAIL] en-US: key "checkout.cart.total" referenced in Checkout.tsx:42,
absent from en-US.json (not extracted)
[PASS] en-US, ar-EG, de-DE, zh-Hans-CN: all other keys present
```
## Integration Tier — Snapshot per Locale (P5 Formatting Locale-Aware)
- For a fixed fixture input, render the formatted output per locale
and snapshot it. A change in the snapshot is either an intended
change (new CLDR data, new copy) or a regression.
- The snapshot is per locale, not per format string. The same
fixture (`{ count: 1, currency: "EUR", date: 2024-11-07 }`)
produces different snapshots for `en-US`, `de-DE`, `ar-EG` — and
that difference is the assertion. A locale whose snapshot matches
the source locale's is a red flag: the formatter is not actually
locale-aware.
```
// snapshot — checkout.cart (fixture: count=1, currency=EUR, date=2024-11-07)
// en-US
"1 item · €1,234.56 · 11/7/2024"
// de-DE
"1 Artikel · 1.234,56 € · 07.11.2024"
// ar-EG
"عنصر واحد · ١٬٢٣٤٫٥٦ € · ٧/١١/٢٠٢٤"
```
- Snapshots are reviewed, not rubber-stamped. A snapshot diff that
changes the plural form for `ar-EG` is either a CLDR update (verify)
or a regression (revert).
## e2e Tier — RTL Coverage (P6 Text Direction is a Layout Primitive)
- A browser-driven run against `en-XC` (or a real RTL locale like
`ar-EG`) asserts that the app is navigable in RTL: no overflow, no
off-screen controls, no LTR-pinned anchors. The assertion is on
layout, not on text — text correctness is the integration tier's
job.
- RTL e2e is the narrow top of the i18n pyramid: it is slow, it
requires a browser, and it catches the defects the lower tiers
cannot (the interaction of `dir` with the real layout engine). It
runs on PR merge, not on every commit.
## Snapshot Discipline (P10 Translation Reversible and Versioned)
- Snapshots are versioned in git. A snapshot that changes because of
a real translation update is a committed diff, reviewed like a
code change. A snapshot that changes because of a regression is a
`git revert`.
- A snapshot that is regenerated and committed without review is a
P10 violation: the snapshot is versioned but the provenance is
opaque. The same discipline applies to snapshots as to resources
(`locale-resources.md`).
## What Violates i18n Testing Discipline
| Violation | Principle |
|-----------|-----------|
| First i18n test runs against real translations, not pseudo-locales | P8 Pseudo-Locales Test Early |
| No missing-key scan — gaps surface only at runtime in production | P3 Resources are External, Not Inline |
| Snapshot per locale that matches the source locale's snapshot | P5 Formatting is Locale-Aware |
| No RTL e2e — "we'll test RTL when we ship an RTL locale" | P6 Text Direction is a Layout Primitive |
| Snapshots regenerated and committed without review | P10 Translation is Reversible and Versioned |
| i18n tests only at e2e (no unit/integration tier) | pyramid inversion — `domains/testing/pyramid.md` |
| Pseudo-locale run skipped because "it's not a real locale" | P8 Pseudo-Locales Test Early |
## Relationship to Other Domains
- `domains/testing/pyramid.md` — the pseudo-locale → pyramid mapping
mirrors this domain's unit / integration / e2e tiers exactly.
- `domains/testing/fixtures.md` — locale fixtures (a fixed input
rendered per locale) are the fixture shape for the integration
tier.
- `domains/i18n/locale-resources.md` — missing-key detection is the
unit-tier scan over the resource bundle this doc defines.
- `domains/i18n/formatting.md` — the integration-tier snapshot
asserts against the formatter's output.
- `domains/i18n/rtl-bidi.md` — the e2e tier exercises the layout
rules this doc establishes.
- `domains/uiux/accessibility.md` — RTL coverage is an a11y
concern; an untested RTL build is an untested a11y surface.
@@ -0,0 +1,55 @@
# Infrastructure as Code — First Principles
## 1. The Principles
### P1. Declarative Intent
Describe the desired state, not the steps to reach it. The tool
reconciles current → desired. Imperative scripts describe how;
declarative config describes what.
### P2. Idempotence
Applying the same configuration twice yields the same result. A
second `apply` with no changes is a no-op, not an error. Idempotence
is what makes `plan` trustworthy.
### P3. State is Truth
The state file is the authoritative record of what the tool believes
exists. Drift between state and reality is a bug to be reconciled,
not tolerated. Lose state, lose the ability to reason about
infrastructure.
### P4. Plan Before Apply
Preview every change before mutating real infrastructure. `plan` is
the contract review; `apply` is the signature. No `apply` without a
read `plan`. The plan is the rollback rehearsal.
### P5. Version Everything
Configuration, state, providers, and modules are all versioned and
reproducible. A commit pins a complete, rebuildable world. Pin
providers; pin module sources; never `latest`.
### P6. Modules Compose
Encapsulate repeatable patterns as versioned modules. Compose
modules; do not copy them. A module is the unit of reuse, review,
and versioning — the IaC expression of composition.
### P7. Least Privilege Providers
Provider credentials are scoped to the minimum needed for the
declared resources. No account-wide admin keys in CI. One credential
per environment, per boundary.
### P8. Remote State with Locking
State is stored remotely with locking. Local state is for a single
developer on a throwaway sandbox. Concurrent `apply` without a lock
is data corruption waiting to happen.
### P9. Drift is Recoverable
`plan` detects drift; `apply` reconciles it. Manual mutation of
managed infrastructure is an incident, not a shortcut. Drift is
expected; unreconciled drift is the bug.
### P10. Secrets Never in Code
Secrets come from providers, external secret stores, or environment
variables — never hardcoded in HCL, never committed to the repo,
never written to state in plaintext. State is a secret-bearing
artifact; treat it accordingly.
+65
View File
@@ -0,0 +1,65 @@
# Modules — Derived Rules
> Derives from `domains/infrastructure-as-code/first-principles.md`. P6 (Modules Compose) lives here. Referenced by `terraform.md` and `opentofu.md`.
## Why Modules (P6 Modules Compose)
- A module is the unit of reuse, review, and versioning in IaC. It encapsulates a repeatable pattern behind a typed interface.
- Composition — building large from small — is the IaC expression of core C6 Composability. Without modules, every stack is a one-off; with modules, a stack is an assembly of reviewed parts.
- A good module has one job (a VPC, a database, a load balancer), a small typed surface, and no hidden side effects.
- A versioned module is the IaC expression of `domains/devops/first-principles.md` P1 (Reproducibility) and P6 (Configuration as Code): a module pins a reusable, rebuildable pattern that any environment can call.
## Module Structure (P1 Declarative Intent, C2 Clarity)
- The conventional layout: `main.tf` (resources), `variables.tf` (inputs), `outputs.tf` (outputs), `versions.tf` (provider/version pins). A `README.md` is required for any published module.
- Inputs are typed and validated: `variable "name" { type = string, description = "...", validation { ... } }`. The description is the contract.
- Outputs are the module's interface to consumers. Mark sensitive outputs `sensitive = true`. Document non-obvious outputs in the description.
- A module does not declare a provider configuration unless it owns the provider. Most modules declare only `required_providers` (the constraint) and let the consumer configure the provider.
## Versioning (P5 Version Everything)
- Modules are versioned. The registry expects SemVer tags (`v1.0.0`). A consumer pins to a version or a range (`~> 1.0`).
- A breaking change bumps the major. An additive change bumps the minor. A fix bumps the patch. No silent breaking changes within a minor.
- Tag the module repo; the tag IS the version. Never `source = "git::...?ref=main"` in prod — unversioned modules drift.
## Source Patterns (P5 Version Everything)
| Source | When | Risk |
|--------|------|------|
| Registry (`<ns>/<name>/<provider>`) | Public, versioned, signed | Verify the publisher; pin the version |
| Git (`git::https://...?ref=v1.0.0`) | Private modules across repos | Pin to a tag, not a branch |
| Local (`./modules/networking`) | Monorepo, single repo | Re-reviewed on every change; no independent version |
| Inline (no module) | Trivial one-off | Becomes a copy-paste anti-pattern at scale |
- Local modules in a monorepo are fine — they trade independent versioning for co-evolution. The boundary is the review unit: if the module and the consumer always change together, local is correct.
- Cross-repo modules must be versioned via git tags; unversioned cross-repo modules are the worst case (drift without a version to pin).
## The Module-vs-Copy Boundary (P6 Modules Compose)
- If a block is used more than once, it is a module. If it is used once and will never be reused, inline is acceptable.
- If two copies differ in one attribute, that is a module with a variable, not two copies. The variable is the difference; the shared body is the module.
- If you find yourself copy-pasting a block and editing it, stop. The edit is a variable. The copy is a module call.
- A module that has grown to do many jobs should be split. A module with 20 variables is two modules.
## Composition (P6 Modules Compose, C6 Composability)
- Compose by calling modules from a root configuration: `module "vpc" { source = "...", version = "..." }`. The root is the assembly; the modules are the parts.
- Outputs of one module feed inputs of another: `module "app" { vpc_id = module.vpc.vpc_id }`. This is the composition edge.
- Avoid hidden coupling: a module should not reach into another module's state. If two modules must share state, promote the shared concern to the root or a parent module.
## Reviewing Modules (P4 Plan Before Apply)
- A module is reviewed once, at its version. Consumers trust the version pin. A module change requires a new version and a review of the diff.
- When a module changes, every consumer that bumps the version gets the change. Treat a module version bump as a real change: review the module diff, run the consumer's `plan`.
- A module with a breaking change must not auto-bump in consumers. Pin consumers to the old major until they explicitly migrate.
## What Violates Module Discipline
| Violation | Principle |
|-----------|-----------|
| Copy-pasted block with a one-line difference | P6 Modules Compose |
| `source = "git::...?ref=main"` in prod | P5 Version Everything |
| Module with 20 variables | P6 Modules Compose (split it) |
| Silent breaking change within a minor | P5 Version Everything |
| Module reaching into another module's state | C6 Composability, P1 Declarative Intent |
| Unpublished module with no README | C2 Clarity |
@@ -0,0 +1,50 @@
# OpenTofu — Derived Rules
> Derives from `domains/infrastructure-as-code/first-principles.md`. OpenTofu is the open-source fork of Terraform; this doc covers fork-specific governance, license, and migration. The shared HCL/state/module model is documented in `terraform.md`. See also `state.md` and `modules.md`.
## Fork Lineage (P5 Version Everything)
- OpenTofu is a 2023 fork of Terraform, created when HashiCorp switched Terraform from MPL-2.0 to the Business Source License (BUSL), which is not open source.
- OpenTofu is stewarded by the Linux Foundation under a genuinely open-source license. The fork's reason for existing is license neutrality.
- Both tools implement the same HCL configuration language, the same provider protocol, and the same state model. Configuration written for one runs on the other at the fork point; divergence accrues slowly over time.
## When to Choose OpenTofu (P7 Least Privilege Providers, supply-chain)
- **License neutrality matters:** if your organization cannot accept BUSL's "competitive use" ambiguity, OpenTofu removes it.
- **Supply-chain provenance:** Linux Foundation stewardship means no single vendor can relicense the tool out from under you.
- **Community governance:** features and fixes are accepted on merit, not vendor strategy.
- **When NOT to switch:** if you depend on HCP Terraform (HashiCorp's managed platform), BUSL-licensed providers, or provider features that have diverged since the fork, stay on Terraform. The decision is supply-chain, not syntax.
## CLI Parity (P1 Declarative Intent)
- `tofu init`, `tofu plan`, `tofu apply`, `tofu destroy` mirror `terraform init/plan/apply/destroy`.
- The lock file (`.terraform.lock.hcl``.tofu.lock.hcl`) is committed; it makes `init` reproducible.
- Workspaces, state backends, and module sources behave as in Terraform — see `terraform.md` and `state.md`.
## Registry Parity (P6 Modules Compose)
- OpenTofu can consume the Terraform Registry and the OpenTofu Registry. Module version pinning works identically.
- Some providers have BUSL licenses that OpenTofu cannot ship; verify a provider's license before adopting it. An MPL or Apache provider is portable; a BUSL provider is not.
- See `modules.md` for module structure, which is unchanged from Terraform.
## Migration from Terraform (P5 Version Everything, P9 Drift is Recoverable)
- `terraform state pull > state.json``tofu state push state.json` carries state across. Validate with `tofu plan` after the push — the plan should be empty.
- Rename the binary in CI: replace `terraform` with `tofu` in scripts. The lock file may need regeneration.
- Migrate one workspace at a time. Do not big-bang a migration; rehearse on a non-prod workspace first (P4 Plan Before Apply applies to the migration itself).
- Pin the OpenTofu version in CI. A migration is a versioned, reviewed change, not a quiet swap.
## Governance and Community (cross-link `domains/security/supply-chain.md`)
- OpenTofu's governance model — impartial, community-driven, layered, modular, backwards-compatible — is itself a supply-chain principle. A tool you cannot trust to remain open is a tool you cannot build on.
- This is the OpenTofu angle on `security/supply-chain.md`: license is a supply-chain property, not a legal footnote.
## What Violates OpenTofu Discipline
| Violation | Principle |
|-----------|-----------|
| Assuming OpenTofu == latest Terraform (unverified parity) | P5 Version Everything |
| Migrating prod state without a non-prod rehearsal | P4 Plan Before Apply |
| Adopting a BUSL-licensed provider into OpenTofu CI | P7 Least Privilege Providers, supply-chain |
| Quiet swap of `terraform` for `tofu` without a versioned change | P5 Version Everything |
| Losing state during migration | P3 State is Truth |
+77
View File
@@ -0,0 +1,77 @@
# State — Derived Rules
> Derives from `domains/infrastructure-as-code/first-principles.md`. State is the cross-cutting IaC concern: P3 (State is Truth) and P8 (Remote State with Locking) live here. Referenced by `terraform.md` and `opentofu.md`.
## Why State Matters (P3 State is Truth)
- The state file is the tool's memory. It records every resource it has claimed, every attribute it has set, and every dependency it has inferred.
- Without state, `plan` cannot compute a diff — it would have nothing to diff against. Lose state, lose the ability to reason about infrastructure safely.
- State can contain plaintext secrets (any sensitive resource attribute). Treat state as a secret-bearing artifact: encrypt at rest, restrict access, never commit it.
## Remote State is Mandatory (P8 Remote State with Locking)
- Local state (`terraform.tfstate` on disk) is acceptable only for a single developer on a throwaway sandbox. Any shared or production environment uses a remote backend.
- A remote backend provides: durability (state survives workstation loss), shared access (team members and CI read the same state), and locking (concurrent `apply` is serialized).
- No locking = data corruption. Two `apply` runs against the same unlocked state race; the loser's changes are silently overwritten.
## Backend Comparison (P8, C4 Locality)
| Backend | Locking | Encryption | Best for | Notes |
|---------|---------|------------|----------|-------|
| S3 + DynamoDB | DynamoDB | SSE-KMS | AWS-hosted | The canonical AWS backend; DynamoDB provides the lock |
| GCS | Built-in | CMEK | GCP-hosted | Native locking via GCS object versioning |
| Azure Blob | Lease | Customer key | Azure-hosted | Lease-based locking |
| HTTP (remote) | Server-side | Server-side | Self-hosted / on-prem | Requires a backend server (e.g., `terraform-backend`) |
| Local | None | None | Single-dev sandbox | Never for shared or prod |
| Consul | KV lock | — | Consul shops | Locking via Consul sessions |
| Postgres | TX | DB encryption | DBA-owned infra | Row-level locking |
- Pick one backend per environment family. Mixing backends across environments fragments operational knowledge (C4 Locality).
- The backend config is part of the configuration, not a runtime secret. Credentials for the backend are runtime secrets. The state file itself is a secret-bearing artifact — treat it per `domains/security/secrets.md`: encrypt at rest, restrict access, never commit it.
## State Isolation per Environment (P4 Plan Before Apply, C4 Locality)
- One state per environment. Never share a single state file across dev, staging, and prod. A `plan` against a shared state crosses environment boundaries — a prod change could appear in a dev plan.
- Isolation patterns: separate workspaces, separate state keys in the same backend, or separate backends entirely. Stricter isolation = safer (separate backends for prod vs non-prod).
- Name state keys by environment and stack: `env:/prod/Networking`, not `prod` or `state`.
## Locking Discipline (P8 Remote State with Locking)
- `terraform force-unlock` is for a stuck lock after a crashed run, not for impatience. Verify the run is actually dead before forcing.
- A forced unlock without verifying the other run is dead causes the corruption the lock prevents.
- In CI, set a lock timeout so a wedged job fails rather than hanging.
## Sensitive Values in State (P10 Secrets Never in Code)
- Any `sensitive = true` attribute is hidden from plan output but stored in state in plaintext (unless the provider encrypts it).
- Backends with at-rest encryption (S3 SSE-KMS, GCS CMEK) protect state at rest. Access to the state file itself is the boundary.
- Never log, print, or commit state. Never pipe `terraform show` to a public channel.
## State Commands (P3 State is Truth)
- `terraform state list` — enumerate resources in state. First step of any state investigation.
- `terraform state show <addr>` — inspect one resource's recorded attributes.
- `terraform state mv` — rename a resource's address without destroying and recreating it. Use when refactoring module structure.
- `terraform state rm` — stop managing a resource without destroying it. Use when handing a resource to another configuration.
- `terraform state pull` / `push` — export and import state. Used in migrations (see `opentofu.md`).
- `terraform import` — bring an existing resource under management by recording its state. The resource must already exist; `import` does not create.
- All `state` subcommands except `list` and `show` mutate state. Treat them as changes: review the intent, run in CI where possible, and commit the resulting config change that justifies the state move.
## Drift and Reconciliation (P9 Drift is Recoverable)
- `terraform plan` reports drift: resources that exist in state but were changed out-of-band, or resources in state that no longer exist in the provider.
- `terraform apply` reconciles drift by bringing reality back to the declared state.
- Manual changes to managed resources are the cause of drift. Treat a drift report as an incident: find who made the manual change and why, then close the access path or the gap that allowed it.
- `terraform plan -refresh=false` skips drift detection. Use only when you know state is current and you want a fast plan; never use it to hide drift.
## What Violates State Discipline
| Violation | Principle |
|-----------|-----------|
| Committed `terraform.tfstate` | P3 State is Truth, P10 Secrets |
| Local state in prod | P8 Remote State with Locking |
| `force-unlock` without verifying the dead run | P8 Remote State with Locking |
| Shared state across environments | P4 Plan Before Apply, C4 Locality |
| Unnamed state keys (`env:/prod`) | C4 Locality |
| Manual change to a managed resource | P9 Drift is Recoverable |
| `state rm` to "fix" a stuck resource | P3 State is Truth |
@@ -0,0 +1,58 @@
# Terraform — Derived Rules
> Derives from `domains/infrastructure-as-code/first-principles.md`. Applies P1P10 to Terraform specifically. See also `opentofu.md` (the open-source fork), `state.md`, and `modules.md`.
## HCL Structure (P1 Declarative Intent)
- Resources are declared, not scripted. A resource block states what should exist; Terraform reconciles it.
- `resource "aws_s3_bucket" "logs" { ... }` — the type and name are the identity; the body is the desired state.
- Data sources read existing state without claiming ownership: `data "aws_caller_identity" "current" {}`.
- Variables are the input contract; outputs are the interface to consumers. Both are typed.
## Providers (P5 Version Everything, P7 Least Privilege)
- Pin the provider version: `required_providers { aws = { source = "hashicorp/aws", version = "~> 5.0" } }`.
- A provider block configures credentials and region. Credentials come from environment, files, or a secrets manager — never inline.
- One credential set per environment. Do not reuse a prod credential in a dev workspace.
## The Core Workflow (P4 Plan Before Apply)
- `terraform init` — resolve providers and modules. Reproducible from the lock file (`.terraform.lock.hcl`), which is committed.
- `terraform plan` — preview the diff. Read it. Every line. The plan is the contract review.
- `terraform apply` — execute the plan. Requires a reviewed plan in CI; in interactive use, requires typing `yes`.
- `terraform destroy` — tear down. Treat `destroy` as a first-class operation with its own plan review; prod destroys are a change event, not a keystroke.
## Workspaces (P4 Locality of Environments)
- Workspaces separate state for the same configuration across environments (dev, staging, prod).
- Do not use workspaces to separate unrelated stacks — use separate configurations. A workspace is an environment axis, not a project axis.
- State is isolated per workspace (see `state.md`).
## State Backends (P3 State is Truth, P8 Remote State with Locking)
- Remote state is mandatory for any shared or production environment. See `state.md` for backend selection and locking.
- Never commit `terraform.tfstate` to the repo. It is a secret-bearing artifact and a source of drift.
- `terraform state` subcommands inspect and manipulate state directly — use sparingly, only for recovery.
## Registry and Modules (P6 Modules Compose)
- The Terraform Registry hosts versioned, signed modules. Reference modules by version: `source = "terraform-aws-modules/vpc/aws"`, `version = "5.x"`.
- Compose modules rather than copy-pasting blocks. A module is reviewed once and reused many times.
- See `modules.md` for module structure, versioning, and the module-vs-copy boundary.
## Secrets (P10 Secrets Never in Code)
- Secrets via provider data sources (`aws_secretsmanager_secret_version`), environment variables, or a dedicated secrets provider. Never a literal string in a resource block.
- State may contain plaintext secrets if a resource attribute is sensitive. Mark attributes `sensitive = true` to keep them out of plan output; use a backend that encrypts state at rest (see `state.md`). This is the IaC angle on `domains/security/secrets.md` — secret hygiene is non-tradeable.
## What Violates Terraform Discipline
| Violation | Principle |
|-----------|-----------|
| Unpinned provider (`source` without `version`) | P5 Version Everything |
| `terraform apply` without a read `plan` | P4 Plan Before Apply |
| Local state in a shared environment | P8 Remote State with Locking |
| Hardcoded secret in HCL | P10 Secrets Never in Code |
| Copy-pasted resource blocks instead of a module | P6 Modules Compose |
| Manual change to a managed resource | P9 Drift is Recoverable |
| Admin credentials in CI | P7 Least Privilege Providers |
+67
View File
@@ -0,0 +1,67 @@
# Kubernetes — First Principles
## 1. The Principles
### P1. Declarative Desired State
You declare the desired state; controllers reconcile current →
desired. Imperative `kubectl` is for inspection and incident
response, not for the steady state. The cluster's job is to make
reality match the manifest.
### P2. Pods are Mortal
A pod is born, runs, and dies. Never assume its identity, its IP,
or its lifetime. Use controllers (Deployment, StatefulSet,
DaemonSet), not bare pods. A bare pod has no recovery, no
scaling, no rollback.
### P3. Labels Select
Labels and selectors are the join mechanism of the platform —
workloads to services, policies to workloads, workloads to nodes.
Label by intent (`app`, `tier`, `env`), not by infrastructure
(`node-3`, `ip-10.0.0.5`). Selectors compose; ad-hoc naming does
not.
### P4. Requests and Limits are Contracts
Resource requests drive scheduling; limits drive quality of
service. A workload with no requests is `BestEffort` — first
evicted under pressure. A workload with no limits is unbounded.
Specifying requests is not optional in production.
### P5. Probes Drive Health
Liveness, readiness, and startup probes are how the platform
sees your workload. Without a readiness probe, traffic routes to
a pod that is not ready. Without a liveness probe, a wedged
container runs forever. The platform cannot heal what it cannot
see.
### P6. Namespaces Bound Blast Radius
Namespaces are the unit of quota, RBAC, network policy, and
cleanup. A namespace is the boundary of "this thing and all its
parts." Default namespace is for nothing in production; every
workload gets a named namespace sized to its blast radius.
### P7. RBAC by Intent, Not Identity
Bind roles to service accounts by the workload's purpose, not to
user identities. Least privilege: the role grants the minimum
the workload needs. `cluster-admin` is a smell, not a shortcut.
Cross-link `domains/security/authorization.md`.
### P8. Storage is Explicit
Storage is ephemeral by default. Persistence requires a
PVC, a StorageClass, and a reclaim policy decision. `emptyDir`
for state that must survive is a bug. The choice of
reclaim policy (`Retain`, `Delete`) is a data-safety decision,
not a default.
### P9. Config and Secrets are Separate
ConfigMaps are non-sensitive configuration; Secrets are
sensitive configuration. Both are injected at runtime, never
baked into the image. A configuration change should not require
a rebuild; a secret rotation should not require a redeploy of the
image. Cross-link `domains/security/secrets.md`.
### P10. Roll Forward, Roll Back
Every Deployment has a rolling update strategy and a rollout
history. A deploy is reversible: `kubectl rollout undo`. A deploy
without a tested rollback is a prototype. Canary and blue-green
are the k8s expression of `domains/devops/P5 Progressive Delivery`.
+68
View File
@@ -0,0 +1,68 @@
# Helm — Derived Rules
> Derives from `domains/kubernetes/first-principles.md`. Applies P1, P3, P6. For the Helm-vs-Kustomize decision, see the decision matrix at the end of this doc and in `kustomize.md`.
## What Helm Is (P6 Modules Compose)
- Helm is a package manager for Kubernetes. A chart is a versioned package of templated manifests. `helm install` renders the templates against `values.yaml` and applies the result.
- A chart encapsulates a reusable deployment (an application, a database, a full stack). It is the k8s analogue of an IaC module — see `domains/infrastructure-as-code/modules.md`.
- Charts live in registries (Helm registry via OCI, or the classic chart repos) and are versioned per SemVer.
## Chart Structure (P1 Declarative Desired State, C2 Clarity)
- `Chart.yaml` — metadata (name, version, appVersion, dependencies).
- `values.yaml` — default inputs; the chart's public interface.
- `templates/` — Go-templated manifests. `templates/_helpers.tpl` holds reusable template partials.
- `values.schema.json` — optional schema for values, giving type checking on inputs. Use it for published charts.
- A chart should have one logical purpose. A chart that deploys an app and a database and an ingress and an observability stack has too many jobs — split it.
## Values (P5 Version Everything, C2 Clarity)
- `values.yaml` holds defaults. Override per release: `helm install --set key=value` or `helm install -f my-values.yaml`.
- Pin values files in git per environment. A release is reproducible from the chart version + the values file.
- Sensitive values do not belong in `values.yaml`. Inject via Secrets (see `rbac.md` P9 and `domains/security/secrets.md`). Some charts accept `existingSecret` to reference a pre-created Secret.
## Release Management (P5 Version Everything, P10 Roll Forward Roll Back)
- A release is a named instantiation of a chart. `helm upgrade` applies a new chart version or new values. `helm rollback` reverts to the previous release revision.
- `helm history <release>` lists revisions; `helm rollback <release> <revision>` is the rollback. The rollback must be tested like any deploy (P10).
- Pin the chart version: `helm install --version 1.2.3`. Never `--version latest` in production — unversioned charts drift (same anti-pattern as unpinned IaC modules).
## Templating Discipline (P1 Declarative Desired State, C2 Clarity)
- Templates render to valid manifests. The chart author's job is that the rendered output is correct k8s, not that the template is clever.
- Keep `templates/` readable. Heavy logic belongs in `_helpers.tpl` or in a values structure that the template merely projects.
- `helm template` renders to stdout without applying — use it to review what a release will create before installing it.
## Registries (P5 Version Everything)
- OCI registries are the modern chart distribution (same registry as container images, charts as OCI artifacts). Classic chart repos are legacy.
- Pull from a pinned registry reference: `oci://registry/chart:1.2.3`. The digest + tag is the version.
## Helm vs Kustomize — Decision Matrix (IDEATE-10)
| Axis | Helm | Kustomize |
|------|------|----------|
| Mechanism | Templating (Go templates) | Overlays (base + patches) |
| Reuse unit | Chart (versioned package) | Base directory (kustomization.yaml) |
| Distribution | Registry (OCI, chart repo) | Git (base dir in a repo) |
| Values | `values.yaml` + overrides | `kustomization.yaml` + patches |
| Release mgmt | `helm` tracks releases, history, rollback | None native — apply with `kubectl apply -k` |
| Learning curve | Template language to learn | YAML patching, no DSL |
| Blast radius | One chart, many resources, templated | One base, many overlays, patched |
| Best for | Off-the-shelf apps, packaged stacks, multi-env via values | Internal apps, patching upstream manifests, env-specific deltas |
| Watch out for | Template complexity, `latest` chart drift, secrets in values | No release tracking, manual rollback, patch sprawl |
- Use Helm when you distribute a reusable app or consume third-party charts. Use Kustomize when you patch existing manifests or keep env deltas in one repo.
- Mixing both is fine and common: Helm for the packaged parts, Kustomize for the last-mile per-env patching. Do not fight the tool that fits the job.
## What Violates Helm Discipline
| Violation | Principle |
|-----------|-----------|
| `helm install --version latest` in prod | P5 Version Everything |
| Secrets in `values.yaml` | P9 Config and Secrets are Separate, security |
| Chart with 15 subcharts doing unrelated things | P6 Modules Compose (split it) |
| No `values.schema.json` on a published chart | C2 Clarity |
| `helm upgrade` without reviewing `helm template` output | P1 Declarative Desired State, P4 Plan Before Apply |
| Untested `helm rollback` | P10 Roll Forward Roll Back |
+63
View File
@@ -0,0 +1,63 @@
# Kustomize — Derived Rules
> Derives from `domains/kubernetes/first-principles.md`. Applies P1, P3, P6. For the Helm-vs-Kustomize decision, see the decision matrix at the end of this doc and in `helm.md`.
## What Kustomize Is (P1 Declarative Desired State)
- Kustomize customizes manifests without templating. A base directory holds the canonical manifests; overlays hold the deltas. The result is plain YAML applied with `kubectl apply -k`.
- No DSL, no template language, no rendering step hidden from `kubectl`. The patch is a YAML file; the result is inspectable.
- Kustomize is built into `kubectl` (`kubectl apply -k`, `kubectl diff -k`). No separate runtime is required to apply.
## Base and Overlays (P6 Namespaces Bound Blast Radius, C4 Locality)
- A `kustomization.yaml` in a base directory lists the resources (Deployment, Service, etc.) the application needs. It is the canonical manifest.
- An overlay is a directory with its own `kustomization.yaml` that references the base (`resources: - ../../base`) and applies patches or additional resources.
- Typical structure: `base/`, `overlays/dev/`, `overlays/staging/`, `overlays/prod/`. The overlay is the environment axis; the base is the shared truth.
## Patches (P1 Declarative Desired State, C2 Clarity)
- Strategic merge patches — a YAML document that overrides matching fields. Simple for single-resource changes.
- JSON patches (RFC 6902) — precise operations (`add`, `replace`, `remove`) on a path. Use when a strategic merge is ambiguous (e.g., list operations).
- `patches` field (modern) takes a list of patch files with targets, replacing the older `patchesStrategicMerge` and `patchesJson6902`. Prefer it.
- A patch is a delta. It is reviewed as "what changes from base," which is exactly the diff a reviewer wants to see.
## Generators and Transformers (P3 Labels Select)
- `configMapGenerator` and `secretGenerator` create ConfigMaps and Secrets from files or literals, with content hashes in the names. A change to the source file changes the hash, which changes the name, which rolls the workload. This is the kustomize pattern for "config change = redeploy."
- `namePrefix`, `nameSuffix`, and `namespace` transformers rewrite names across the base. Use for namespace isolation (P6) or to run the same base multiple times in one cluster without collisions.
- `commonLabels` and `commonAnnotations` stamp labels onto everything in the base — the kustomize-native way to enforce the labelling discipline of P3.
## No Release Tracking (P10 Roll Forward Roll Back)
- Kustomize has no release object, no history, no built-in rollback. `kubectl apply -k` is a one-shot apply; the previous state is in git, not in a Helm-style release record.
- Rollback is `git revert` + `kubectl apply -k`. The git history IS the release history. This is fine — and arguably cleaner — but it means rollback is a git operation, not a `helm rollback` command.
- Use a GitOps tool (ArgoCD, Flux) on top of Kustomize for automated reconciliation and rollback tracking. The tool watches the git ref; rollback is a git revert.
## Helm vs Kustomize — Decision Matrix (IDEATE-10)
| Axis | Kustomize | Helm |
|------|----------|------|
| Mechanism | Overlays (base + patches) | Templating (Go templates) |
| Reuse unit | Base directory (kustomization.yaml) | Chart (versioned package) |
| Distribution | Git (base dir in a repo) | Registry (OCI, chart repo) |
| Values | `kustomization.yaml` + patches | `values.yaml` + overrides |
| Release mgmt | None native — `kubectl apply -k` | `helm` tracks releases, history, rollback |
| Learning curve | YAML patching, no DSL | Template language to learn |
| Blast radius | One base, many overlays, patched | One chart, many resources, templated |
| Best for | Internal apps, patching upstream manifests, env-specific deltas | Off-the-shelf apps, packaged stacks, multi-env via values |
| Watch out for | No release tracking, manual rollback, patch sprawl | Template complexity, `latest` chart drift, secrets in values |
- Use Kustomize when you patch existing manifests or keep env deltas in one repo. Use Helm when you distribute a reusable app or consume third-party charts.
- Mixing both is fine and common: Kustomize for the internal apps, Helm for the packaged parts. The decision is per-workload, not per-cluster.
- `commonLabels` is the kustomize-native enforcement of P3 (Labels Select); see `domains/devops/first-principles.md` P6 (Configuration as Code) for the upstream principle that the rendered manifest — not a console click — is the source of truth.
## What Violates Kustomize Discipline
| Violation | Principle |
|-----------|-----------|
| Duplicated base instead of an overlay | P6 Modules Compose (use an overlay) |
| Patch that overrides most of the base | C3 Simplicity (the base is wrong — fix the base) |
| No `commonLabels` on a multi-team base | P3 Labels Select |
| No git-based rollback strategy | P10 Roll Forward Roll Back |
| Hand-edited rendered output instead of `apply -k` | P1 Declarative Desired State |
| Patch sprawl (10 overlays each patching 15 fields) | C3 Simplicity (refactor the base) |
+49
View File
@@ -0,0 +1,49 @@
# Networking — Derived Rules
> Derives from `domains/kubernetes/first-principles.md`. Covers Service, Ingress, Gateway API, EndpointSlices, NetworkPolicy, and DNS. Applies P1, P3, P6.
## The Service (P3 Labels Select)
- A Service routes traffic to pods selected by a label selector. The selector is the join between the network abstraction and the workloads.
- Service types: `ClusterIP` (in-cluster only, default), `NodePort` (exposed on every node's IP at a fixed port), `LoadBalancer` (cloud-managed LB points to the Service). Default to `ClusterIP`; expose only what must be exposed.
- A Service fronts a Deployment (or other controller), never a bare pod. The controller keeps pods available; the Service routes to whichever are ready (per the readiness probe — see `workloads.md`).
## EndpointSlices (P3 Labels Select, P5 Probes Drive Health)
- An EndpointSlice lists the pod IPs currently backing a Service. Only pods passing their readiness probe appear.
- The Service routes by EndpointSlice, not by selector directly. A pod with the right labels but a failed readiness probe is not in the Service.
## Ingress and Gateway API (P6 Namespaces Bound Blast Radius)
- Ingress routes HTTP/HTTPS traffic from outside the cluster to Services. It is L7 routing by host and path.
- Gateway API is the successor to Ingress: more expressive (TCP, UDP, TLS passthrough), role-oriented (GatewayClass → Gateway → Route), and cross-platform. Prefer Gateway API for new L7 needs.
- Both Ingress and Gateway API are implemented by a controller (nginx-ingress, Traefik, Istio, Envoy Gateway). Pick one; mixing ingress controllers in a cluster is operational debt.
## NetworkPolicy (P6 Namespaces Bound Blast Radius, P7 RBAC by Intent)
- A NetworkPolicy is a firewall rule for pods. Default-deny ingress; allow by namespace and pod selector.
- Without a default-deny NetworkPolicy, every pod can reach every other pod. In production, default-deny is the baseline; allows are the exceptions.
- NetworkPolicy is the network-layer expression of zero-trust authorization — see `domains/security/authorization.md`. RBAC (see `rbac.md`) governs the API; NetworkPolicy governs the network; together they bound blast radius (P6).
- NetworkPolicy is enforced by the CNI plugin (Calico, Cilium, etc.). A NetworkPolicy with no supporting CNI is a no-op. Verify the CNI enforces before relying on it.
## DNS (P3 Labels Select)
- Every Service gets a DNS record: `<service>.<namespace>.svc.cluster.local`. Pods get `pod-ip-address.<namespace>.pod.cluster.local` (with dots replaced).
- Headless Services (`clusterIP: None`) resolve directly to pod IPs — use for StatefulSet peer discovery (`<statefulset>-0.<service>`).
- DNS is how workloads find each other without hardcoded IPs. Use the DNS name, not the ClusterIP.
## Dual-Stack (C4 Locality)
- IPv4/IPv6 dual-stack is opt-in per cluster. Services can be single-stack or dual-stack per Service.
- Decide at cluster creation. Migrating a single-stack cluster to dual-stack is disruptive and rarely worth it.
## What Violates Networking Discipline
| Violation | Principle |
|-----------|-----------|
| `LoadBalancer` on an internal-only Service | P6 Namespaces Bound Blast Radius |
| No default-deny NetworkPolicy | P6 Namespaces Bound Blast Radius, P7 RBAC by Intent |
| Hardcoded pod IP in config | P3 Labels Select (use DNS) |
| Service pointing at a bare pod | P3 Labels Select (point at a controller) |
| Multiple ingress controllers in one cluster | C3 Simplicity (operational debt) |
| No readiness probe on a Service-backed workload | P5 Probes Drive Health (empty EndpointSlices) |
+45
View File
@@ -0,0 +1,45 @@
# RBAC and Pod Security — Derived Rules
> Derives from `domains/kubernetes/first-principles.md`. P7 (RBAC by Intent, Not Identity) lives here. Cross-link `domains/security/authorization.md` for the general authorization principles and `domains/security/secrets.md` for secret handling.
## RBAC Objects (P7 RBAC by Intent, Not Identity)
- **Role** — permissions within a namespace (verb on resource). **ClusterRole** — permissions cluster-wide or usable across namespaces.
- **RoleBinding** — binds a Role to a subject (ServiceAccount, User, Group) within a namespace. **ClusterRoleBinding** — binds a ClusterRole cluster-wide.
- Prefer Role + RoleBinding per namespace over ClusterRole + ClusterRoleBinding. Cluster-level is the broad axe; namespace-level is the scalpel.
## Bind to Service Accounts, Not Users (P7 RBAC by Intent, Not Identity)
- A workload authenticates as a ServiceAccount. Bind the Role to the ServiceAccount, scoped to the workload's namespace.
- The Role encodes the workload's intent: "this workload reads ConfigMaps in this namespace." Not "this user is an admin."
- One ServiceAccount per workload (or workload family). Do not reuse the `default` ServiceAccount for production workloads; it is a shared identity.
## Least Privilege (P7 RBAC by Intent, C1 Correctness via security)
- Grant the minimum verbs on the minimum resources. `get, list, watch` on `pods` is fine for a monitoring sidecar; `*` on `*` is not.
- `cluster-admin` is a smell. If a workload "needs" `cluster-admin`, the workload is either doing something it should not, or it is a cluster operator that should be reviewed as such.
- Audit `ClusterRoleBindings` regularly. They are the broadest grant in the system and the easiest to leave behind.
## Pod Security Standards and Admission (P7 RBAC by Intent, security)
- Pod Security Standards (PSS) define three profiles: `privileged` (unrestricted), `baseline` (some restrictions), `restricted` (hardened).
- Pod Security Admission (built-in) enforces a PSS profile per namespace via labels: `pod-security.kubernetes.io/enforce: restricted`. It replaces the deprecated PodSecurityPolicy.
- Map namespaces to profiles: `restricted` for prod workloads, `baseline` for most, `privileged` only for system add-ons (CNI, CSI, node agents) that need it. A workload in `privileged` is a security event, not a default.
## Service Accounts and Token Automation (P9 Config and Secrets are Separate)
- ServiceAccount tokens are auto-mounted into pods unless `automountServiceAccountToken: false`. For workloads that do not call the API, disable auto-mount.
- Long-lived ServiceAccount tokens are deprecated. Use projected tokens (bound to the pod, time-limited) via `TokenRequest`.
- A workload that does not need API access should not have a token. A workload that needs API access should have a Role scoped to its intent.
## What Violates RBAC Discipline
| Violation | Principle |
|-----------|-----------|
| `cluster-admin` bound to a workload | P7 RBAC by Intent, Not Identity |
| Reused `default` ServiceAccount for prod | P7 RBAC by Intent, Not Identity |
| `automountServiceAccountToken: true` on a non-API workload | P9 Config and Secrets are Separate |
| `privileged` PSS on an application namespace | P7 RBAC by Intent, security |
| ClusterRoleBinding where a RoleBinding would suffice | P6 Namespaces Bound Blast Radius, P7 |
| Long-lived static token instead of projected | P9 Config and Secrets are Separate |
| Leftover ClusterRoleBindings after a workload is removed | P7 RBAC by Intent (audit) |
+59
View File
@@ -0,0 +1,59 @@
# Storage — Derived Rules
> Derives from `domains/kubernetes/first-principles.md`. P8 (Storage is Explicit) lives here. Covers Volumes, PV/PVC, StorageClass, CSI, snapshots, and reclaim policies. Cross-link `domains/data/` for the data-model angle.
## Ephemeral by Default (P8 Storage is Explicit)
- A container's filesystem is ephemeral. When the pod dies, the filesystem dies with it. This is the design, not a flaw.
- `emptyDir` is an ephemeral volume scoped to the pod's lifetime (survives container restarts within the pod, dies with the pod). It is scratch space, never durable storage.
- Any data that must survive a pod restart requires a PersistentVolumeClaim (PVC). The choice of "must survive" is the data-safety decision at the heart of P8.
## PersistentVolume and PersistentVolumeClaim (P8 Storage is Explicit)
- A PersistentVolume (PV) is a piece of storage in the cluster. A PersistentVolumeClaim (PVC) is a request for that storage by a workload.
- The PV is the resource; the PVC is the consumer. A workload mounts the PVC, not the PV directly.
- For StatefulSets, use `volumeClaimTemplates` so each replica gets its own PVC with a stable name (`data-<statefulset>-0`). Do not share one PVC across replicas of a stateful workload.
## StorageClass and Dynamic Provisioning (P8 Storage is Explicit, P5 Version Everything)
- A StorageClass describes the "flavour" of storage (e.g., `fast-ssd`, `cold-hdd`, `encrypted`). A PVC names a StorageClass or gets the cluster default.
- Dynamic provisioning creates the PV on demand when the PVC is created, via the CSI driver. Manual PV creation is for specific cases (a pre-existing disk, a static NFS export).
- Mark a default StorageClass only if the default is safe for all workloads. A fast-but-expensive default can cause cost surprises; a slow default can cause performance surprises.
## CSI (P5 Version Everything)
- The Container Storage Interface (CSI) is the standard driver interface. Each storage backend ships a CSI driver. Pin the CSI driver version in the cluster; treat it as infrastructure.
- CSI enables features beyond mount/unmount: snapshots, cloning, volume expansion, and topology-aware provisioning. Not all drivers implement all features; verify before relying.
## Volume Snapshots (P5 Reversibility, P8 Storage is Explicit)
- A VolumeSnapshot is a point-in-time copy of a PVC, taken by the CSI driver. Restore creates a new PVC from the snapshot.
- Snapshots are not backups. They are local to the storage backend and may share blocks with the source. An off-cluster backup is still required for disaster recovery.
- Snapshot scheduling is a workload concern (use a CronJob or a tool like Velero), not a k8s-native feature.
## Reclaim Policies (P8 Storage is Explicit, P5 Reversibility)
| Policy | On PVC delete | When |
|--------|---------------|------|
| `Retain` | PV and its data persist; PV must be manually reclaimed | Production, data-safety default |
| `Delete` | PV and the underlying storage are deleted | Ephemeral, dev, scratch |
| `Recycle` (deprecated) | PV scrubbed and made available again | Do not use — use dynamic provisioning |
- The reclaim policy is a data-safety decision. `Delete` on a production PVC is a footgun: deleting the PVC destroys the data. Default to `Retain` for prod, `Delete` for dev.
- For StatefulSet PVCs, the reclaim policy on the StorageClass governs what happens when the PVC is deleted (which happens when the StatefulSet is scaled down or deleted, depending on the policy).
## Ephemeral Volumes (P8 Storage is Explicit)
- `configMap`, `secret`, `downwardAPI` volumes are read-only (by default) projections injected at pod start. They are configuration, not storage.
- `emptyDir` with `medium: Memory` is a tmpfs — fast, ephemeral, memory-charged. Use for scratch that must be fast and never persist.
## What Violates Storage Discipline
| Violation | Principle |
|-----------|-----------|
| `emptyDir` for data that must survive pod restart | P8 Storage is Explicit |
| Shared PVC across StatefulSet replicas | P8 Storage is Explicit (use `volumeClaimTemplates`) |
| `Delete` reclaim policy on production storage | P8 Storage is Explicit, P5 Reversibility |
| Snapshot treated as a backup | P5 Reversibility (snapshots are local, not DR) |
| No default StorageClass decision (accidental default) | P8 Storage is Explicit |
| Manual PV creation when dynamic provisioning exists | C3 Simplicity |
+54
View File
@@ -0,0 +1,54 @@
# Workloads — Derived Rules
> Derives from `domains/kubernetes/first-principles.md`. Covers Pod, ReplicaSet, Deployment, StatefulSet, DaemonSet, Job, and CronJob. Applies P1P10.
## The Pod (P2 Pods are Mortal)
- A pod is the smallest deployable unit: one or more containers sharing network and storage namespaces.
- Never deploy a bare pod. A bare pod has no controller to restart, scale, or replace it. Use a controller.
- Pods are replaceable by design. Do not store state in a pod's filesystem (`emptyDir` is scratch, not storage — see `storage.md`).
## Controllers (P1 Declarative Desired State)
| Controller | When | Identity | Ordering |
|------------|------|----------|----------|
| Deployment | Stateless workloads | None (pods interchangeable) | No ordering |
| StatefulSet | Stateful workloads (databases, queues) | Stable name (`pod-0`, `pod-1`) + stable PVC | Ordered, sequential |
| DaemonSet | One pod per node (logging, monitoring, CNI) | Per-node | — |
| Job | Run to completion (batch) | — | — |
| CronJob | Scheduled batch | — | — |
- A Deployment manages a ReplicaSet, which manages pods. You interact with the Deployment; the ReplicaSet is an implementation detail except during rollouts.
- StatefulSet gives stable network identity and stable persistent storage per replica. Use it when the workload needs a stable name or per-replica data (databases, distributed systems). Do not use StatefulSet for stateless workloads — the ordering is overhead.
## Probes (P5 Probes Drive Health)
- **Readiness probe** — is the pod ready to serve traffic? Failing readiness removes the pod from the Service's endpoints but does not restart it. Use for "warm-up" and transient unavailability.
- **Liveness probe** — is the pod alive? Failing liveness restarts the container. Use for "wedged but running." Do not use liveness to check dependencies (it will cascade-restart on a dependency blip).
- **Startup probe** — has the pod finished starting? Disables liveness/readiness until it succeeds. Use for slow-starting workloads so liveness does not kill them before they are ready.
- Probes must check the workload's own health, not the health of its dependencies. A readiness probe that fails on a downstream outage causes the Service to drain all pods simultaneously.
## Lifecycle and Disruption (P2 Pods are Mortal, P10 Roll Forward Roll Back)
- `kubectl rollout status` watches a Deployment's rollout to completion. `kubectl rollout undo` reverts to the previous ReplicaSet.
- PodDisruptionBudgets (PDBs) protect voluntary disruptions (node drain, cluster autoscaler). An involuntary disruption (node failure) ignores the PDB. Set a PDB on every workload that must keep a minimum available.
- Rolling update strategy: `maxUnavailable` and `maxSurge` control the speed of rollout. Slow rollouts (low `maxSurge`) are safer; fast rollouts (high `maxUnavailable`) risk availability.
## Resource Contracts (P4 Requests and Limits are Contracts)
- Every container in production has a CPU request, a memory request, and a memory limit. CPU limits are optional but recommended to bound noisy neighbours.
- QoS classes: `Guaranteed` (requests == limits), `Burstable` (requests < limits), `BestEffort` (no requests). `BestEffort` is first evicted under node pressure — never for prod.
- A workload without requests is an unbounded gamble on the scheduler. Set them.
- The rolling update + rollout history described below is the k8s expression of `domains/observability/metrics.md` for health and `domains/devops/first-principles.md` P5 (Progressive Delivery): the platform observes the rollout via probes and metrics and can stop or reverse it.
## What Violates Workload Discipline
| Violation | Principle |
|-----------|-----------|
| Bare pod (no controller) | P2 Pods are Mortal |
| StatefulSet for a stateless workload | P1 Declarative Desired State (overhead) |
| No probes | P5 Probes Drive Health |
| Liveness probe checks a dependency | P5 Probes Drive Health |
| No PDB on a critical workload | P10 Roll Forward Roll Back |
| No resource requests in prod | P4 Requests and Limits are Contracts |
| `emptyDir` for data that must persist | P8 Storage is Explicit |
+379
View File
@@ -0,0 +1,379 @@
# Delivery Semantics — Derived Rules
> Derives from `domains/messaging/first-principles.md`. Applies P4
> (Delivery Semantics are Explicit), P3 (Consumers are Idempotent),
> P2 (Ordering is a Property, Not an Assumption), P5 (Dead-Letter
> Handling is Defined), and P6 (Backpressure is Bounded) primarily,
> with P10 (DLQ depth as an alert). For the dead-letter strategy
> decision, see the comparison table below. Includes a fenced
> idempotency-key dedup-store example (IDEATE-39). Cross-links
> `domains/concurrency/patterns` for the in-process retry/backoff
> analog, `domains/errors/patterns` for errors as data for message
> failures, and `domains/observability/metrics` for DLQ depth as an
> alert.
## The Three Delivery Semantics (P4 Delivery Semantics are Explicit)
- **At-most-once**: a message is delivered 0 or 1 times; loss is
possible, duplication is not. The producer fires-and-forgets; the
broker does not ack; the consumer does not dedup. Lowest latency,
lowest implementation cost, lossy. Fits telemetry where a dropped
sample is acceptable (MQTT QoS 0, fire-and-forget metrics).
- **At-least-once**: a message is delivered 1 or more times;
duplication is possible, loss is not. The producer sends and
waits for a broker ack; the consumer processes and acks; a crash
before the consumer's ack triggers redelivery. The consumer must
be idempotent (P3). The default for side-effecting operations
(orders, payments, commands). The vast majority of broker-backed
queues (SQS standard, RabbitMQ ack, MQTT QoS 1).
- **Exactly-once**: a message is delivered exactly 1 time; no
loss, no duplication. In practice, this is at-least-once plus
idempotency (P3), or a transactional consume-process-produce loop
(P4 — see `domains/messaging/streams.md`). Jepsen analyses verify
broker claims: "exactly-once" requires independent verification;
the durable engineering practice is at-least-once with idempotent
consumers (P3), which collapses to exactly-once under correct
dedup.
- The semantic is declared per channel (P4), not emergent. An
unstated semantic is a defect: the consumer guesses, and the
guess is wrong under the first failure. See
`domains/messaging/queues.md` for the three-semantics comparison
table (semantics, latency cost, implementation cost, when each
fits).
## Idempotency (P3 Consumers are Idempotent)
- Idempotency is the correctness property that makes at-least-once
safe. A consumer that processes the same message twice has the
same effect as processing it once. The mechanism is the
idempotency key: a per-message unique identifier the consumer
uses to dedup redeliveries.
- The idempotency key is per-message, not per-producer or
per-session. A consumer that dedups by producer alone drops
distinct messages issued in the same window. Use a UUID per
message, or a deterministic key derived from the message content
(e.g., `(entity, operation, version)`).
- The dedup store is bounded (P6 — Backpressure is Bounded): a
dedup store that grows without bound is a memory leak. Use a TTL
window longer than the broker's max-redelivery window, or a
bounded LRU. The TTL is the P6 bound: a key seen within the TTL
is a redelivery; a key older than the TTL is expired (the broker
has given up redelivering it).
## Idempotency-Key Dedup Store (IDEATE-39, P3 — fenced example)
The dedup store is the concrete mechanism that makes a consumer
idempotent under at-least-once delivery. This is the fenced
example required by IDEATE-39 (parallel to the v0.3 IDEATE-29
signed-attestation fenced example): it is NOT prose-only — the
consumer-with-dedup-store demonstrates P3 concretely.
```python
# Idempotent consumer with a dedup store (P3 Consumers are
# Idempotent, P6 Backpressure is Bounded — the dedup store is
# TTL-bounded). The consumer dedups by idempotency key before
# processing; a redelivered message is a no-op, not a double-apply.
import time
# P6: the dedup store is bounded by a TTL window. The TTL must
# exceed the broker's max-redelivery window; beyond the TTL, the
# key is expired (the broker has given up). A dedup store with no
# TTL is a memory leak (P6 violation).
DEDUP_TTL_SECONDS = 24 * 3600 # longer than max-redelivery window
class DedupStore:
"""A TTL-bounded idempotency-key dedup store (P3, P6).
seen(key): True if the key was processed within the TTL.
mark(key): Record the key as processed (with a timestamp).
"""
def __init__(self, backend):
# backend is a Redis, a DB, or an in-process LRU. The
# backend must be shared across consumer instances if the
# subscription is shared (P2/P3 — see domains/messaging/
# pubsub.md on shared vs independent subscriptions).
self.backend = backend
def seen(self, key: str) -> bool:
ts = self.backend.get(key)
if ts is None:
return False
if time.time() - ts > DEDUP_TTL_SECONDS:
# P6: expired. The broker has given up redelivering;
# this key is no longer a redelivery signal.
self.backend.delete(key)
return False
return True
def mark(self, key: str):
self.backend.set(key, time.time(), ttl=DEDUP_TTL_SECONDS)
# The idempotent consumer: dedup before process, mark after
# process, ack after mark. A crash before mark re-processes (the
# dedup store does not have the key); a crash before ack
# redelivers (the broker did not see the ack) and the dedup store
# makes the redelivery a no-op (P3).
def consume_idempotent(broker, dedup: DedupStore, process):
for message in broker.receive():
# P3: dedup BEFORE process. A redelivered message is a
# no-op, not a double-apply.
if dedup.seen(message["idempotencyKey"]):
broker.ack(message) # already processed; skip
continue
try:
process(message["payload"])
# P3: mark AFTER process succeeds. A crash between
# process and mark re-processes (acceptable: the
# process must be idempotent OR the mark must be
# transactional with the process — see below).
dedup.mark(message["idempotencyKey"])
broker.ack(message)
except TransientError as exc:
# P6: bounded retry with backoff. Nack for redelivery;
# the broker redelivers after exponential backoff.
broker.nack(message, delay=backoff(message["attempt"]))
except PoisonError as exc:
# P5: poison message. Route to DLQ, do NOT retry
# forever (see the DLQ routing rule below).
route_to_dlq(broker, message, exc)
broker.ack(message) # remove from the origin queue
```
- The order `process → mark → ack` gives at-least-once with
idempotent dedup: a crash before `mark` re-processes (the dedup
store does not have the key), and a crash before `ack`
redelivers (the dedup store makes the redelivery a no-op). If
`process` is not itself idempotent, the `process → mark` window
must be transactional (e.g., process and mark in one DB
transaction) — otherwise a crash in the window double-applies.
- For a non-idempotent `process` (e.g., a payment that must not
double-charge), use a transactional dedup: process and mark in
one DB transaction, so the mark commits iff the process
commits. This is the "exactly-once via idempotency" pattern
(P4): at-least-once delivery plus a transactional
process-and-mark collapses to exactly-once under correct
transactional semantics.
## Ordering (P2 Ordering is a Property, Not an Assumption)
- The delivery semantic interacts with ordering (P2). At-least-once
with per-partition ordering: a redelivery within a partition
preserves order (the redelivered message re-appears in its
original position relative to other messages the consumer has
not yet seen). At-least-once with no ordering: a redelivery may
appear out of order relative to messages delivered after it.
- The consumer must not assume an ordering property the broker
does not provide (P2). A standard queue delivers per-receive-node
arrival order but no global order and no order across redeliveries;
a FIFO queue delivers strict per-group order including across
redeliveries; a partitioned stream delivers strict per-partition
order across redeliveries. Document the property; do not assume
it. See `domains/messaging/queues.md` (FIFO vs standard) and
`domains/messaging/streams.md` (per-partition order).
## Dead-Letter Strategies (P5 Dead-Letter Handling is Defined)
- A poison message (unparseable, repeatedly failing, or exhausting
the retry budget) must be routed to a dead-letter queue, not
retried forever or silently dropped (P5). The DLQ is observable
(P10 — depth is an alert) and drainable (an operator can
inspect, replay, or discard with audit).
- The dead-letter strategy determines when a message is
dead-lettered and what the operator sees. The choice is the
decision matrix below (D-069).
## Dead-Letter Strategy Comparison (D-069)
| Strategy | When It Applies | Failure Visibility | Operational Cost |
|----------|-----------------|-------------------|------------------|
| **Retry-count-limit** | A fixed max-redeliveries count (e.g., 5). After N redeliveries, route to DLQ. Simple, predictable. | The redelivery count is visible in the DLQ entry; the operator sees how many times it was retried. | Low — a counter per message; no backoff tuning. Risk: retries fire as fast as the broker redelivers, hammering a downstream that is already failing (P6 — no backoff = no backpressure escape). |
| **TTL-with-backoff** | A max time-to-live for redelivery (e.g., 30 minutes) with exponential backoff between retries. After the TTL, route to DLQ. | The TTL and the backoff schedule are visible; the operator sees the retry timeline. | Medium — backoff tuning per message type. Benefit: backoff gives the downstream time to recover (P6 — the retry rate is bounded); fits transient failures (a downstream that is briefly unavailable). |
| **Poison-queue** | A separate queue for messages that fail a specific check (unparseable, schema-invalid, unknown type) before any processing retry. Routed immediately, not retried. | The poison queue is a separate signal from the DLQ; the operator sees parse-vs-process failures distinctly. | Low — a routing rule per check. Benefit: distinguishes "never going to succeed" (poison) from "might succeed on retry" (DLQ). Use for unparseable messages that no retry will fix. |
| **DLQ + alert** | Any of the above strategies, plus an alert on DLQ depth. The DLQ is observable (P10) — depth, age, and rate are alerted. | Highest — the operator is paged on DLQ growth; the DLQ is a first-class signal, not a graveyard. | Medium — alerting setup per DLQ. This is the P5/P10 floor: a DLQ without an alert is a silent correctness defect (poison messages accumulate invisibly). |
- The default for transient failures is **TTL-with-backoff + DLQ
+ alert**: backoff gives the downstream time to recover (P6),
the TTL bounds the retry budget (P5), the DLQ captures the
unprocessable, and the alert makes it visible (P10). The default
for unparseable messages is **poison-queue + alert**: route
immediately, do not retry a message no retry will fix.
- **Retry-count-limit alone** (no backoff, no alert) is the
`messaging-unbounded-retry` chaos anti-pattern's cousin: it
caps the count but hammers the downstream at full retry rate,
and the DLQ grows silently if no alert is wired. Always pair
a retry budget with backoff (P6) and an alert (P10).
- The failure-visibility column is the P10 check: every strategy
must surface the failure to the operator. A strategy with no
visibility is a P10 violation regardless of its retry semantics.
- The operational-cost column is the C8 tradeoff: more visibility
and more backoff cost more to set up but pay back in operational
stability. The P5/P10 floor is "DLQ + alert"; below that, the
strategy is a silent defect waiting to grow.
## DLQ Routing Rule (P5, P10)
```python
# DLQ routing rule (P5 dead-letter handling, P6 bounded retry with
# backoff, P10 DLQ depth alert). Combines TTL-with-backoff for
# transient failures and poison-queue for unparseable messages,
# with an alert on DLQ depth.
import json, time
MAX_RETRY_TTL_SECONDS = 30 * 60 # 30 min total retry window
DLQ = "orders-dlq"
POISON = "orders-poison" # unparseable; never retried
def route_to_dlq(broker, message, reason):
"""Route a message to the DLQ with audit metadata (P5)."""
broker.send(DLQ, body=json.dumps({
"original": message,
"reason": str(reason),
"deadLetteredAt": now_iso(),
"redeliveryCount": message.get("attempt", 0),
}))
# P10: emit a metric so DLQ depth alerts fire. A DLQ that
# grows with no alert is a silent correctness defect (P5/P10).
metrics.increment("dlq.depth", tags={"queue": "orders"})
# The ack removes the message from the origin queue; the DLQ
# is the durable record (P5 — observable and drainable).
def consume_with_dlq(broker, dedup, process):
for message in broker.receive():
# P1: parse first. An unparseable message is poison —
# route immediately, do NOT retry (no retry will fix it).
try:
payload = json.loads(message["body"])
except (ValueError, SchemaError) as exc:
broker.send(POISON, body=json.dumps({
"original": message["body"],
"reason": f"parse-failed: {exc}",
"poisonedAt": now_iso(),
}))
metrics.increment("poison.depth", tags={"queue": "orders"})
broker.ack(message) # remove from origin; poison queue holds it
continue
# P3: dedup before process.
if dedup.seen(payload["idempotencyKey"]):
broker.ack(message); continue
attempt = payload.get("attempt", 0)
first_attempt_ts = payload.get("firstAttemptTs", time.time())
try:
process(payload)
dedup.mark(payload["idempotencyKey"])
broker.ack(message)
except TransientError as exc:
# P6: TTL-with-backoff. If the retry window is
# exhausted, route to DLQ; otherwise redeliver with
# exponential backoff.
if time.time() - first_attempt_ts > MAX_RETRY_TTL_SECONDS:
route_to_dlq(broker, message, exc) # P5
broker.ack(message)
else:
broker.nack(message, delay=backoff(attempt))
except PermanentError as exc:
# A permanent error (e.g., a not-found dependency)
# does not benefit from retry — route to DLQ now.
route_to_dlq(broker, message, exc)
broker.ack(message)
```
- The routing rule distinguishes three failure modes: **poison**
(unparseable — route immediately, no retry), **transient**
(retry with backoff until the TTL, then DLQ), and **permanent**
(a retry will not fix it — DLQ now). This distinction is the P5
discipline: not every failure is a retry; some are immediate
DLQs.
- The DLQ entry carries `reason`, `deadLetteredAt`, and
`redeliveryCount` — it is auditable (the operator knows why each
message was dead-lettered and how many times it was retried).
This is the errors-as-data discipline — see
`domains/errors/patterns` for the general principle a DLQ entry
instantiates.
## Retry Budgets and Backoff (P6 Backpressure is Bounded)
- The retry budget is the cap on redelivery: a count, a TTL, or
both. After the budget, the message routes to the DLQ (P5). An
unbounded retry budget is the `messaging-unbounded-retry` chaos
anti-pattern (P5 breach): the consumer never makes progress past
the poison message.
- **Exponential backoff** spaces retries: 1s, 2s, 4s, 8s, ... with
a jitter to avoid thundering-herd synchrony. Backoff gives the
downstream time to recover (P6 — the retry rate is bounded,
giving the downstream a chance to catch up). A retry with no
backoff hammers the downstream at full rate, making the failure
worse.
- The retry budget × the backoff schedule is the P6 bound: the
consumer's retry load is bounded by design, not by luck. See
`domains/concurrency/patterns` Pattern 6 (Timeout on Every
Block) for the in-process retry/backoff analog; messaging owns
the broker-backed instance where the redelivery comes from the
broker across a network, not an in-process loop (D-062).
## Poison Messages (P5, P1)
- A poison message is one no retry will fix: unparseable (the
schema is wrong, P1), unknown type (the consumer does not handle
this version, P9), or a permanent failure (a not-found
dependency). Retrying a poison message wastes resources and
blocks the queue (P6 — the consumer never makes progress).
- Poison messages route to the poison queue immediately (no
retry), distinct from the DLQ (which holds messages that
exhausted their retry budget on transient failures). The
distinction is the P5 discipline: a poison queue is for
"never going to succeed"; a DLQ is for "might have succeeded
but didn't within the budget."
- A poison queue without an alert is the same defect as a DLQ
without an alert (P10): the operator cannot see the poison
accumulating. Wire both to `domains/observability/metrics`.
## Cross-Link to Concurrency (P6, cross-link concurrency/patterns)
- The retry/backoff discipline here is the cross-process analog
of `domains/concurrency/patterns` Pattern 6 (Timeout on Every
Block) for in-process retry. Concurrency owns the in-process
analog (a retry loop within one program, with a timeout per
attempt); messaging owns the broker-backed instance (the broker
redelivers across a network, the consumer applies backoff via
nack-with-delay). The failure model differs: in-process retry
fails by a thread crash or a timeout; broker-backed retry fails
by network partition, broker restart, or consumer crash-and-retry
(D-062).
- The cross-link is one-directional outward (messaging →
concurrency) per D-026 extended: messaging references concurrency
as the in-process foundation; concurrency does not back-link to
messaging.
## Cross-Link to Errors (P5, cross-link errors/patterns)
- A poison message is an errors-as-data instance: the failure is
captured as a DLQ entry (with `reason`, `redeliveryCount`,
`deadLetteredAt`), not swallowed. See `domains/errors/patterns`
for the general errors-as-data discipline a DLQ entry
instantiates. The DLQ is the async-messaging instance of an
error log — observable, auditable, drainable.
- The cross-link is one-directional outward (messaging → errors):
messaging references errors for the errors-as-data pattern;
errors does not back-link to messaging.
## What Violates Delivery-Semantics Discipline
| Violation | Principle |
|-----------|-----------|
| Unstated delivery semantic (at-least-once vs exactly-once guessed) | P4 Delivery Semantics are Explicit |
| Non-idempotent consumer under at-least-once delivery | P3 Consumers are Idempotent |
| Dedup store with no TTL (memory leak; grows without bound) | P6 Backpressure is Bounded |
| Dedup key per-producer (distinct messages in the same window deduped) | P3 Consumers are Idempotent |
| Retry with no backoff (hammers the downstream at full rate) | P6 Backpressure is Bounded |
| Unbounded retry budget (consumer never progresses past the poison) | P5 Dead-Letter Handling is Defined |
| DLQ with no depth alert (poison messages accumulate invisibly) | P10, `domains/observability/metrics` |
| Poison message retried forever (no poison queue, no immediate DLQ) | P5 Dead-Letter Handling is Defined |
| Process-not-idempotent with non-transactional mark (crash in window double-applies) | P3, P4 |
| DLQ entry with no reason/audit metadata (uninspectable failure) | P5, `domains/errors/patterns` |
| Ordering assumption the broker does not provide (FIFO assumed on standard queue) | P2 Ordering is a Property, Not an Assumption |
+307
View File
@@ -0,0 +1,307 @@
# Messaging — First Principles
## 1. The Principles
### P1. Messages are Contracts
A message has an explicit, versioned schema. Producer and consumer
agree on shape before exchange; the schema is the boundary, not a
guess. A schemaless message — a free-form JSON blob the consumer
parses by hope — is a defect: the consumer breaks silently on the
next shape change, and the producer has no contract to evolve
against. This derives from `C1 Correctness` (the exchange must
carry what the parties agreed to) and `C2 Clarity` (the schema
makes the boundary obvious to both sides). This is the
cross-process expression of the contract discipline that
`domains/api/rest` owns for synchronous request/response: the
message schema is to async exchange what the API contract is to
sync exchange. It is distinct from
`domains/concurrency/patterns` Pattern 1 (Message Passing), which
owns the *in-process* channel primitive — here the contract spans
separate systems and survives network failure (D-062). See
`domains/messaging/queues.md` for the queue-flavored application
and `domains/messaging/streams.md` for the durable-log-flavored
application.
### P2. Ordering is a Property, Not an Assumption
Ordering guarantees — per-partition strict, global, or none — are
explicit and documented. "It's FIFO" is a claim that must be backed
by the broker's partitioning contract, not an assumption the
consumer makes and the broker may not honor. A standard queue
delivers in arrival order per receive-node but offers no global
ordering across shards; a FIFO queue delivers strict per-message-
group order but at a latency cost; a partitioned stream delivers
strict per-partition order but only within a partition. Each is a
distinct, declared property. This derives from `C1 Correctness`
(order is a correctness property — a consumer that assumes order
the broker does not provide is wrong) and `C2 Clarity` (the
ordering guarantee is documented, not discovered in production).
This is distinct from in-process ordering, which
`domains/concurrency/patterns` Pattern 1 owns for channels within
one program: messaging ordering survives network failure, broker
restart, and consumer crash-and-retry — a stronger failure model
than thread-local channels (D-062). The `messaging-shared-
subscription` chaos anti-pattern breaches this rule: two consumers
sharing one subscription break per-consumer ordering because the
broker dispatches each message to an arbitrary consumer. See
`domains/messaging/streams.md` for the partition-order contract
and `domains/messaging/delivery-semantics.md` for the interaction
of ordering with the three delivery semantics.
### P3. Consumers are Idempotent
Delivery is at-least-once by default across the network; a
consumer deduplicates via idempotency keys or deterministic
processing. "Exactly-once" is idempotency plus at-least-once, not a
broker guarantee — Jepsen analyses of Kafka, RabbitMQ, and NATS
establish that exactly-once claims require independent
verification, and the durable engineering practice is to make
consumers idempotent under redelivery. A non-idempotent consumer
under at-least-once delivery doubles the effect on every retry; a
non-idempotent consumer under a claimed exactly-once broker is a
bug waiting for the broker's exactly-once invariant to break. This
derives from `C1 Correctness`: correctness under redelivery is the
contract, not a nice-to-have. This parallels
`domains/edge/P5 Edge Operations are Idempotent` (the
cross-partition device-and-cache-flavored analog) and is the
cross-process instance of the retry-safety discipline that
`domains/concurrency/patterns` Pattern 6 (Timeout on Every Block)
implies for in-process retry. It is distinct from in-process
retry because the redelivery comes from the broker across a
network, not from an in-process loop (D-062). See
`domains/messaging/delivery-semantics.md` for the idempotency-key
dedup-store pattern.
### P4. Delivery Semantics are Explicit
At-least-once / at-most-once / exactly-once is a declared choice
per channel, not an emergent behavior. The tradeoff — latency cost,
implementation complexity, operational cost — is made consciously
and documented. At-most-once is fire-and-forget (low latency, lossy);
at-least-once is acked with possible duplication (the default,
requires idempotent consumers per P3); exactly-once is at-least-once
plus idempotency or a transactional two-phase commit (highest cost,
narrowest fit). An unstated semantic is a defect: the consumer
guesses, and the guess is wrong under the first failure. This
derives from `C1 Correctness` (the chosen semantic must hold) and
`C2 Clarity` (the tradeoff is visible to the reader and the
operator). This is the cross-process analog of the explicit-failure-
mode discipline that `domains/errors/patterns` owns for synchronous
code — messaging makes the delivery-mode choice as explicit as an
error-handling choice. See `domains/messaging/queues.md` for the
three-semantics comparison table and `domains/messaging/delivery-
semantics.md` for the correctness properties of each.
### P5. Dead-Letter Handling is Defined
Poison messages — unparseable, repeatedly failing, or exhausting
the retry budget — are routed to a dead-letter queue, not retried
forever or silently dropped. The DLQ is observable and drainable: an
operator can inspect it, replay from it, or discard with audit. An
unbounded retry loop is a livelock: the consumer never makes
progress past the poison message. A silent drop is a correctness
defect: the message vanished with no record. This derives from `C1
Correctness` (poison messages must not livelock the consumer or
silently disappear) and `C5 Reversibility` (the DLQ is the
reversibility mechanism — a dead-lettered message can be reprocessed
after the bug is fixed). This is the cross-process analog of the
bounded-error discipline that `domains/errors/patterns` owns for
synchronous code: a poison message is an error-as-data instance
that must be observable and recoverable, not swallowed. It is
distinct from in-process error handling because the failure spans a
network and a consumer restart (D-062). See
`domains/messaging/delivery-semantics.md` for the dead-letter
strategy comparison table and the DLQ routing rule pattern.
### P6. Backpressure is Bounded
A slow consumer cannot unbounded-buffer the broker or the
producer. Backpressure is explicit: consumer lag is visible,
max-unacked is bounded, the retry budget is capped. A consumer
that falls behind without a visible signal is a silent backlog —
the operator cannot fix what they cannot see, and the broker's
memory grows without bound until it fails. This derives from `C1
Correctness` (a backlog that grows until OOM is a correctness
failure) and `C8 Economy` (the broker's memory is bounded by
design, not by luck). This is distinct from
`domains/concurrency/P9 Bounded Queues` and
`domains/concurrency/patterns` Pattern 5 (Bounded Queue with
Backpressure), which own the *in-process* analog: concurrency's
bounded queue fails by OOM or thread crash; messaging's bounded
backpressure fails by network partition, broker restart, or
consumer crash-and-retry (D-062). The Reactive Streams
specification (`request(n)`, `onNext` bounded) is the in-process
instance; messaging's broker-backed backpressure is the
cross-process instance above it. See `domains/messaging/queues.md`
for prefetch and max-unacked and `domains/observability/metrics`
for consumer-lag as an alert.
### P7. Partitioning is Intentional
The partition key determines ordering, parallelism, and hotspots.
Key choice is a design decision with documented rationale, not a
default. A key that hashes unevenly creates a hot partition that
limits throughput; a key that does not match the ordering need
breaks per-key semantics; a key that is too coarse (one partition
for the whole topic) serializes all the traffic. The partition
count is a capacity bound: too few partitions cap parallelism, too
many partition overhead the broker. This derives from `C4 Locality`
(ordering and parallelism are co-located with the partition) and
`C6 Composability` (the partition is the unit of parallelism and
scaling — consumer groups compose from per-partition workers).
This is the cross-process analog of the locality discipline that
`domains/performance/` owns for generic data-near-compute
optimization: performance's locality is algorithmic (data near
compute); messaging's locality is partitional (order and
parallelism near the partition). See `domains/messaging/streams.md`
for the partitioned-log model and consumer-group rebalance
strategies.
### P8. Replay and Retention are Configured
Retention windows and replay-from-offset are explicit. A message
is not ephemeral by default; the broker is a durable log, not a
pipe. A topic with no retention is a fire-and-forget stream — a
consumer that falls behind loses data permanently; a topic with
infinite retention is an unbounded log — the broker grows until
disk exhaustion. Both are defects: the retention window is a
declared bound, and replay-from-offset is the mechanism that makes
the log durable (re-consumable) rather than ephemeral. This derives
from `C5 Reversibility` (a retained message is reversible — it can
be re-consumed; an ephemeral message is not) and `C7 Observability`
(the durable log is itself an observable record of what happened —
the offset is the position from which to replay). This is the
foundation for `domains/messaging/streams.md` and the rule that
distinguishes a stream from a queue (a queue deletes on ack; a
stream retains for replay). See `domains/messaging/pubsub.md` for
the pub/sub-vs-stream durability boundary.
### P9. Schemas Evolve Compatibly
Schema changes are backward- and forward-compatible by
construction. Breaking changes are versioned migrations, not
silent shape edits. A producer that ships a new field the old
consumer ignores is backward-compatible; a consumer that handles a
missing field the new producer omits is forward-compatible. A
silent schema change — the producer renames a field and the
consumer parses `undefined` — is a P1 violation (the contract was
broken) compounded here as an evolution defect. This derives from
`C5 Reversibility` (a schema change is reversible by versioning —
the old shape is still readable) and `C6 Composability` (producers
and consumers of different versions compose because the schema
evolves compatibly). This parallels `domains/data/migrations`
(schema migration for databases) and `domains/api/versioning`
(API contract evolution): messaging's schema evolution is the
async instance of the same compatibility discipline. See
`domains/messaging/streams.md` for the stream-schema-evolution
angle.
### P10. Messaging is Observable
Consumer lag, DLQ depth, throughput, and consumer-group health are
first-class signals. Silent backlog is a bug, not a feature: a
consumer that falls behind with no lag metric is invisible until
the downstream effect surfaces — by which time the backlog may be
hours or days. A DLQ that grows without an alert is a silent
correctness defect: poison messages are accumulating and no one
knows. This derives from `C7 Observability` (the broker's behavior
is visible to the operator) and `C1 Correctness` (backlog
detection is a correctness bound — unbounded lag is a failure).
This is distinct from `domains/observability/metrics`, which owns
*generic* structured metrics; messaging owns the *broker-specific*
signals — lag, DLQ depth, partition imbalance, consumer-group
rebalance events. See `domains/observability/metrics` for the
generic SLI/SLO discipline and `domains/observability/tracing` for
cross-partition traces.
## 2. Core Principle Trace
Each messaging P-rule derives from one or more core C-rules
(C1C8). The matrix extension lands in P4 of the v0.4 plan; the
traces below are authoritative. Messaging is a broad-derivation
domain touching 7 of 8 core principles (C1, C2, C4, C5, C6, C7,
C8); C3 (Simplicity) is not a primary derivation — messaging is
inherently a tradeoff domain where simplicity yields to the
correctness of delivery guarantees (a simpler-than-necessary
delivery model does not handle the failure cases, per C3's
"simpler than necessary is also a violation").
| P-rule | Core | Why |
|--------|------|-----|
| P1 Messages are Contracts | C1, C2 | Correctness of the exchange; clarity of the schema boundary |
| P2 Ordering is a Property, Not an Assumption | C1, C2 | Correctness of order; clarity of the guarantee |
| P3 Consumers are Idempotent | C1 | Correctness under redelivery |
| P4 Delivery Semantics are Explicit | C1, C2 | Correctness of the chosen semantic; clarity of the tradeoff |
| P5 Dead-Letter Handling is Defined | C1, C5 | Correctness of poison-message routing; reversibility of reprocessing |
| P6 Backpressure is Bounded | C1, C8 | Correctness of bounded backlog; economy of broker memory |
| P7 Partitioning is Intentional | C4, C6 | Locality of order; composability of parallelism |
| P8 Replay and Retention are Configured | C5, C7 | Reversibility of replay; observability of the durable log |
| P9 Schemas Evolve Compatibly | C5, C6 | Reversibility of schema changes; composability of versions |
| P10 Messaging is Observable | C7, C1 | Observability of lag/DLQ; correctness of backlog detection |
## 3. What Violates These Principles
| Violation | Principle Breached |
|-----------|-------------------|
| Schemaless message (no versioned contract; consumer parses by guess) | P1 Messages are Contracts |
| "It's FIFO" with no documented partition contract | P2 Ordering is a Property, Not an Assumption |
| Non-idempotent consumer under at-least-once delivery | P3 Consumers are Idempotent |
| Unstated delivery semantic (at-least-once vs exactly-once guessed) | P4 Delivery Semantics are Explicit |
| No dead-letter queue (poison message retried forever or silently dropped) | P5 Dead-Letter Handling is Defined |
| Unbounded retry budget (no cap; slow consumer stalls the partition) | P6 Backpressure is Bounded |
| Default partition key (no rationale; hotspot or wrong-order) | P7 Partitioning is Intentional |
| Ephemeral broker (no retention; no replay) | P8 Replay and Retention are Configured |
| Silent schema change (producer breaks consumers with no version bump) | P9 Schemas Evolve Compatibly |
| Silent backlog (no lag metric; consumer falls behind invisibly) | P10 Messaging is Observable |
| Shared subscription (two consumers share one subscription; per-consumer ordering breaks) | P2 Ordering is a Property, Not an Assumption (P3 compounding) |
| Blocking consumer (slow downstream call with no timeout; broker redelivers to the stuck consumer) | P6 Backpressure is Bounded |
## 4. Relationship to Other Domains
Messaging systems are the engineering discipline of
**cross-process, cross-system asynchronous communication via
brokers**. Producer and consumer are separate systems; the broker
is the intermediary that brokers delivery, ordering, retention,
and failure semantics. The distinguishing constraints are a
cross-process failure model (network, not crash), explicit
delivery semantics, decoupled producer/consumer lifecycle, and
replay-and-retention as a durable-log property. Messaging overlaps
`domains/concurrency/` by *subject* (messages, queues,
backpressure) but not by *failure model*: per D-062, messaging
owns the cross-process/network-failure-model angle; concurrency
owns the in-process/crash-failure-model angle. The discriminator
is the failure model: concurrency's queue fails by OOM or thread
crash; messaging's queue fails by network partition, broker
restart, or consumer crash-and-retry. Messaging extends
concurrency's bounded-queue/backpressure model to the network-
partition regime. Cross-links are one-directional outward (per
D-026 extended); no back-link edits to v0.1/v0.2/v0.3 content.
- `domains/concurrency/patterns` ← P6 (the broker-backed bounded
queue is the cross-process analog of the in-process bounded
buffer — concurrency Pattern 5 owns in-process; messaging owns
the network-failure-model instance above it, per D-062)
- `domains/concurrency/patterns` ← P3 (idempotent retry is the
cross-process analog of in-process retry-safety — the failure
model differs: broker redelivery across a network vs in-process
loop)
- `domains/observability/metrics` ← P10 (consumer lag and DLQ
depth as alerts; observability owns the generic SLI/SLO
discipline, messaging owns the broker-specific signals)
- `domains/observability/tracing` ← P10 (cross-partition traces
for stream processing; observability owns the generic tracing
discipline, messaging owns the cross-partition propagation)
- `domains/data/schema-design` ← P1, P9 (message schema design
and evolution; data owns the generic schema discipline,
messaging owns the cross-process message-shape instance)
- `domains/errors/patterns` ← P5 (errors as data for message
failures; a poison message is an error-as-data instance that must
be observable and recoverable, not swallowed)
- `domains/edge/iot` ← P4 (the edge↔messaging cross-link
resolves bidirectionally here: edge/iot.md links outward to
messaging/queues for MQTT QoS parallels to delivery semantics;
this first-principles doc acknowledges the back-link — the
edge/iot.md → messaging/queues link from P1 now resolves because
messaging/queues.md exists, completing the bidirectionality per
IDEATE-40)
> Note: the edge/iot.md → messaging/queues cross-link (MQTT QoS
> parallels for delivery semantics) was authored in P1 with a
> dangling reference; this P2 authorship of messaging/queues.md
> resolves it. The bidirectionality is verified in P5
> (ATELIER-114 per IDEATE-40). The cross-link is one-directional
> outward from edge/iot.md; this first-principles doc
> acknowledges the resolution without editing edge/iot.md (per
> D-026 extended — no back-link edits to v0.1/v0.2/v0.3 or to
> P1-authored edge content).
+269
View File
@@ -0,0 +1,269 @@
# Pub/Sub — Derived Rules
> Derives from `domains/messaging/first-principles.md`. Applies P1
> (Messages are Contracts), P2 (Ordering is a Property, Not an
> Assumption), P3 (Consumers are Idempotent), and P4 (Delivery
> Semantics are Explicit) primarily, with P7 (partitioning), P10
> (per-subscription lag). The `messaging-shared-subscription` chaos
> anti-pattern lives here (pre-specified in P4 ATELIER-110).
> Cross-links `domains/messaging/streams` for the pub/sub-vs-stream
> durability boundary and `domains/observability/metrics` for
> per-subscription lag.
## What Pub/Sub Is (P1 Messages are Contracts)
- Pub/sub is the fan-out primitive: a producer publishes a message
to a topic; N independent subscriptions each receive a copy. The
message has an explicit, versioned schema (P1): the topic's
schema is the contract every subscription agrees to before
subscribing. A schemaless topic is a defect — every subscriber
breaks silently on the next shape change.
- The boundary with queues is the fan-out ratio. A queue is
point-to-point (one producer, one consumer); pub/sub is
one-to-many (one producer, N consumers, each with its own
subscription). The boundary with streams is the durability model
— see the cross-link below. Pub/sub is an async concern because
producer and consumers are separate systems and the failure model
is network, not crash (D-062).
- See `domains/messaging/queues.md` for the point-to-point variant
and `domains/messaging/streams.md` for the durable-log variant.
## Topic / Subscription Model (P1, P3, P4)
- A **topic** is the named stream of messages. A **subscription**
is a durable cursor over the topic: each subscription receives
every message published after it was created (subject to
retention and filtering). The subscription is independent — its
ack, redelivery, and DLQ are per-subscription, not shared.
- Each subscription is a consumer under at-least-once by default
(P4): the broker redelivers until the subscription acks, and the
subscriber must be idempotent (P3). A subscription with no
idempotency dedup duplicates every redelivered message.
- The topic's schema evolves compatibly (P9 — Schemas Evolve
Compatibly): a new field the old subscriber ignores is
backward-compatible; a renamed field the old subscriber parses
as `undefined` is a P1 violation.
```python
# Publish + two independent subscriptions (P1 contract, P3
# idempotency, P4 at-least-once per subscription). Each
# subscription is an independent durable cursor; acking one does
# not affect the other.
import json, uuid
# --- Publisher ---
def publish(topic, event, broker):
# P1: versioned schema on the topic. All subscribers must
# understand this schema (or a compatible superset — P9).
message = {
"schema": "user.signed-up.v1",
"id": str(uuid.uuid4()),
"idempotencyKey": f"user:{event['userId']}:signup",
"payload": event,
}
broker.publish(topic=topic, body=json.dumps(message))
# --- Subscription A: welcome-email service ---
def subscribe_welcome(broker, dedup_store, send_email):
sub = broker.subscribe(topic="users", subscription="welcome-email")
for message in sub.receive():
# P3: idempotent per subscription. A redelivered message is
# a no-op for THIS subscription, not for the others.
if dedup_store.seen(("welcome", message["idempotencyKey"])):
sub.ack(message)
continue
send_email(message["payload"]["email"], "Welcome!")
dedup_store.mark(("welcome", message["idempotencyKey"]))
sub.ack(message)
# --- Subscription B: analytics-ingest service ---
def subscribe_analytics(broker, dedup_store, ingest):
# Independent subscription: its own cursor, its own dedup,
# its own ack. Welcome-email acking does NOT advance this.
sub = broker.subscribe(topic="users", subscription="analytics")
for message in sub.receive():
if dedup_store.seen(("analytics", message["idempotencyKey"])):
sub.ack(message)
continue
ingest(message["payload"])
dedup_store.mark(("analytics", message["idempotencyKey"]))
sub.ack(message)
```
- The dedup key is scoped per subscription: `(subscription,
idempotencyKey)`. A redelivery to subscription A that was already
processed by A is a no-op for A; the same message delivered to
subscription B is processed by B independently. Scoping the dedup
key by subscription prevents one subscription's dedup from
masking another's redelivery.
## Fan-Out Semantics (P4, P7)
- Fan-out means every subscription receives every published message
(subject to filtering — see below). The broker duplicates the
message per subscription; each subscription's delivery is
independent. The fan-out ratio is the number of subscriptions; the
broker's cost scales with fan-out × message size.
- Partitioning (P7) applies to topics that are partitioned for
throughput: a partitioned topic delivers per-partition order, and
each subscription receives from every partition. A subscription
that consumes partitions in parallel must handle per-partition
ordering and cross-partition non-ordering (P2 — document the
property, do not assume global order).
- The delivery semantic is per-subscription (P4): subscription A
may be at-least-once, subscription B may be at-most-once (for a
loss-tolerant analytics feed). The choice is per subscription,
declared, not emergent.
## Shared vs Independent Subscriptions (P2, P3 — the chaos anti-pattern)
- An **independent subscription** is one durable cursor per
consumer group: each subscription receives every message in
topic order (per partition, P2) and acks independently. This is
the correct default: per-consumer ordering and per-consumer
idempotency hold.
- A **shared subscription** is one subscription shared by multiple
consumers: the broker dispatches each message to an arbitrary
consumer in the shared group. This breaks per-consumer ordering
(P2 — consumer A sees message 3 before consumer B sees message
1) and complicates idempotency (P3 — the dedup state must be
shared across consumers, not per-consumer). This is the
`messaging-shared-subscription` chaos anti-pattern
(pre-specified in P4 ATELIER-110): the primary breach is P2
(ordering); P3 (idempotency) is the compounding consequence.
- A shared subscription is correct ONLY when the consumers are
stateless, the per-message processing is order-independent, and
the dedup store is shared (a shared Redis, a shared DB). A shared
subscription for order-dependent or per-consumer-stateful
processing is the chaos anti-pattern: the broker's arbitrary
dispatch breaks the order the consumer assumes.
```python
# The messaging-shared-subscription chaos anti-pattern (P2
# ordering breach, P3 idempotency compounding). Two consumers
# share one subscription; the broker dispatches each message to
# an arbitrary consumer. Per-consumer ordering breaks; dedup
# must be shared (and often is not).
# BAD — shared subscription, per-consumer dedup (chaos):
def shared_subscription_bad(broker, send_email):
# Both consumers call subscribe with the SAME subscription
# name. The broker round-robins; consumer A gets msg 1, msg 3;
# consumer B gets msg 2, msg 4. Per-consumer order is broken.
# If each consumer has its OWN dedup store, a redelivery to
# the OTHER consumer re-processes (P3 breach).
sub = broker.subscribe(topic="users", subscription="shared")
for message in sub.receive():
# Per-consumer dedup — WRONG. A redelivered message may
# land on the other consumer, which has not seen it.
if local_dedup.seen(message["idempotencyKey"]): # per-consumer
sub.ack(message); continue
send_email(message["payload"]["email"], "Welcome!")
local_dedup.mark(message["idempotencyKey"])
sub.ack(message)
# CORRECT — independent subscriptions (per-consumer ordering,
# per-subscription dedup):
def independent_subscriptions_good(broker, send_email):
sub = broker.subscribe(topic="users", subscription="welcome-email")
for message in sub.receive():
if dedup_store.seen(("welcome", message["idempotencyKey"])):
sub.ack(message); continue
send_email(message["payload"]["email"], "Welcome!")
dedup_store.mark(("welcome", message["idempotencyKey"]))
sub.ack(message)
```
- If a shared subscription is genuinely required (stateless,
order-independent, shared dedup), document the choice and the
shared-dedup requirement (P2 — the ordering property is "none
across consumers"; P3 — the dedup is shared). The default is
independent subscriptions; shared is an opt-in for the narrow case.
## Filtering (P4, C8 Economy)
- **Subscription filtering** lets a subscription receive only
messages matching a filter (e.g., `event.type == "order"`).
Filtering at the broker saves bandwidth (C8 — the subscriber
does not receive and discard) and reduces subscriber load.
- **Server-side filtering** (broker evaluates the filter before
delivery) is more efficient than **client-side filtering**
(subscriber receives and discards). Server-side filtering is the
default where the broker supports it (GCP Pub/Sub, SNS filtering,
NATS subject filtering); client-side is the fallback.
- A filter that is too broad wastes bandwidth; a filter that is
too narrow drops messages the subscriber needed. The filter is
a P1 (contract) and P4 (semantic) decision: the subscription's
filter is part of its declared contract.
## Ordering Across Subscriptions (P2)
- A topic with per-partition ordering delivers per-partition order
to each subscription. Across subscriptions, there is no ordering
guarantee: subscription A may ack message 3 while subscription B
is still on message 1. This is correct and expected — each
subscription is independent.
- Within a subscription, ordering holds per partition (P2 — the
documented property). A subscription that processes partitions
in parallel must not assume cross-partition order. A subscription
that needs global order must use a single partition (sacrificing
parallelism, P7) or an external sequencing mechanism.
- The `messaging-shared-subscription` anti-pattern breaks even
per-partition order within a subscription: the broker's arbitrary
dispatch to consumers in the shared group breaks the per-
partition sequence each consumer sees.
## Pub/Sub vs Stream — The Durability Boundary (cross-link messaging/streams)
- Pub/sub and streams are both fan-out or one-to-many primitives,
but their durability model differs. Pub/sub is a
**push-to-subscription** model: each subscription is a cursor,
retention is short (the subscription's unacked window), and
replay is limited to the unacked messages. A subscription that
falls behind beyond the retention window loses messages
permanently.
- A stream is a **durable-log** model: messages are retained by
the log for a configured window (P8 — Replay and Retention are
Configured), and any consumer group can replay from any offset
within the window. A stream consumer that falls behind can
catch up by replaying; a pub/sub subscription that falls behind
beyond retention cannot.
- The choice is the durability requirement: if the consumer must
be able to replay (reprocessing, backfill, new consumer starting
from the beginning), use a stream. If the consumer only needs
the live feed (and can tolerate loss on a long fall-behind),
pub/sub is lighter. See `domains/messaging/streams.md` for the
durable-log model, offsets, and consumer groups.
## Observability — Per-Subscription Lag (P10)
- Per-subscription lag (messages published minus messages acked
for each subscription, or the age of the oldest unacked message
per subscription) is the primary pub/sub health signal. Each
subscription has its own lag — a fast subscription and a slow
subscription on the same topic are independent signals.
- A subscription whose lag grows beyond the retention window is a
silent data-loss risk: the broker will drop the oldest messages,
and the subscription will never see them. Alert on lag relative
to retention — lag approaching retention is the loss threshold.
- Wire per-subscription lag to `domains/observability/metrics` as
an SLI per subscription. A topic with N subscriptions has N lag
metrics; a single aggregate hides the slow one. See
`domains/observability/metrics` for the generic SLI/SLO
discipline.
## What Violates Pub/Sub Discipline
| Violation | Principle |
|-----------|-----------|
| Shared subscription for order-dependent processing (broker dispatch breaks per-consumer order) | P2 Ordering is a Property, Not an Assumption |
| Shared subscription with per-consumer dedup (redelivery to the other consumer re-processes) | P3 Consumers are Idempotent |
| Schemaless topic (no versioned contract; subscribers parse by guess) | P1 Messages are Contracts |
| Subscription with no idempotency dedup (redelivered message duplicates the effect) | P3 Consumers are Idempotent |
| Subscription whose lag approaches retention (silent data loss) | P10, `domains/observability/metrics` |
| Unstated delivery semantic per subscription (at-least-once vs at-most-once guessed) | P4 Delivery Semantics are Explicit |
| Filter that is too narrow (drops messages the subscriber needed) | P1, P4 |
| Partitioned topic with no documented per-partition ordering contract | P2 Ordering is a Property, Not an Assumption |
| No per-subscription lag metric (slow subscription invisible) | P10 Messaging is Observable |
| Cross-partition order assumption within a subscription (no global order guarantee) | P2, P7 |
+318
View File
@@ -0,0 +1,318 @@
# Queues — Derived Rules
> Derives from `domains/messaging/first-principles.md`. Applies P1
> (Messages are Contracts), P3 (Consumers are Idempotent), P4
> (Delivery Semantics are Explicit), P5 (Dead-Letter Handling is
> Defined), and P6 (Backpressure is Bounded) primarily, with P2
> (ordering), P7 (partitioning), and P10 (observable lag). For the
> at-least-once / at-most-once / exactly-once decision, see the
> comparison table below. Cross-links `domains/concurrency/patterns`
> for the in-process bounded-queue analog and
> `domains/observability/metrics` for consumer lag.
## What a Queue Is (P1 Messages are Contracts)
- A queue is a point-to-point async delivery primitive. A producer
enqueues a message; exactly one consumer dequeues and processes
it. The message has an explicit, versioned schema (P1): the
producer and consumer agree on shape before exchange, and the
schema is the boundary — a schemaless message is a defect (the
consumer breaks silently on the next shape change).
- The boundary is per D-062: messaging owns the cross-process /
network-failure-model angle; concurrency owns the in-process
analog. A queue is a messaging concern because producer and
consumer are separate systems, the broker is the intermediary,
and the failure model is network (the message can be lost,
duplicated, reordered, or delayed by the broker or the network,
not by a thread crash). The in-process bounded buffer
(`domains/concurrency/patterns` Pattern 5) is the analog below
this boundary — it fails by OOM; a broker-backed queue fails by
partition, broker restart, or consumer crash-and-retry.
- See `domains/messaging/pubsub.md` for the fan-out (one-to-many)
variant and `domains/messaging/streams.md` for the durable-log
(replay-from-offset) variant. A queue deletes on ack; a stream
retains for replay — the durability boundary is the
distinguishing trait.
## Producer / Consumer Model (P1, P3)
- The producer enqueues a message with an idempotency key (P3).
The consumer dequeues, processes, and acks. If the consumer
crashes before acking, the broker redelivers; the idempotency
key makes the redelivery safe (the consumer dedups, not the
broker).
- The idempotency key is per-message, not per-producer or
per-session. A consumer that dedups by producer alone will drop
distinct messages issued in the same window. Use a UUID per
message, or a deterministic key derived from the message content
(e.g., `(entity, operation, version)`).
```python
# Producer/consumer pair with idempotency key (P1 contract, P3
# idempotency). The producer tags each message with a versioned
# schema and a unique idempotency key; the consumer dedups by the
# key so a redelivered message is processed once (P3).
# --- Producer ---
import json, uuid
def enqueue(order, broker):
# P1: versioned schema. The message carries its schema version
# so the consumer can route by shape (P9 evolution discipline).
message = {
"schema": "order.created.v1",
"id": str(uuid.uuid4()),
"idempotencyKey": f"order:{order['id']}:{order['version']}",
"payload": order,
}
broker.send(queue="orders", body=json.dumps(message))
# At-least-once by default (P4): the broker acks the send; the
# consumer may see this message more than once under retry.
# --- Consumer ---
def consume(broker, dedup_store, process_order):
for message in broker.receive(queue="orders"):
# P3: idempotent consumer. Dedup by idempotency key before
# processing; a redelivered message is a no-op, not a
# double-apply.
if dedup_store.seen(message["idempotencyKey"]):
broker.ack(message) # already processed; skip
continue
try:
process_order(message["payload"])
dedup_store.mark(message["idempotencyKey"])
broker.ack(message) # success; broker drops it
except Exception:
broker.nack(message) # redeliver (at-least-once, P4)
```
- The dedup store is bounded (P6 — Backpressure is Bounded): a
dedup store that grows without bound is a memory leak. Use a TTL
window longer than the broker's max-redelivery window, or a
bounded LRU. See `domains/messaging/delivery-semantics.md` for
the full idempotency-key dedup-store pattern.
## Ack / Nack (P4 Delivery Semantics are Explicit)
- **Ack** tells the broker the message was processed; the broker
drops it. **Nack** (negative ack) tells the broker the
processing failed; the broker redelivers (at-least-once) or
routes to a DLQ (after the retry budget — P5).
- A consumer that neither acks nor nacks within the visibility
timeout causes the broker to redeliver (the broker assumes the
consumer died). This is the at-least-once default: the broker
prefers duplication to loss.
- The semantic is explicit (P4): at-least-once is the default; the
consumer must be idempotent (P3). At-most-once is fire-and-forget
(no ack; the broker drops on send) — lossy but lowest latency.
Exactly-once is at-least-once plus idempotency, or a
transactional two-phase commit — see the comparison table below.
## Visibility Timeouts and Redelivery (P4, P5)
- The visibility timeout is the window the broker hides a message
after delivery, waiting for the ack. If the consumer does not
ack within the window, the broker makes the message visible
again and redelivers it (to the same consumer or another). This
is the at-least-once mechanism: the broker assumes a
no-ack-in-time consumer is dead.
- The timeout must be longer than the processing time, or the
broker redelivers a message the consumer is still processing —
causing duplicate processing (which P3 idempotency makes safe,
but which wastes resources). A timeout shorter than processing
time is a P6 (backpressure) smell: the consumer is too slow for
the configured timeout.
- Redelivery has a budget (P5): after N redeliveries or a TTL, the
message routes to the DLQ. An unbounded retry budget is the
`messaging-unbounded-retry` chaos anti-pattern: the consumer
never makes progress past the poison message.
## FIFO vs Standard Queues (P2 Ordering is a Property, Not an Assumption)
- A **standard queue** delivers in arrival order per receive-node
but offers no global ordering across shards, no per-message-
group ordering, and may redeliver out of order under retry. It
is the high-throughput default; ordering is *not* guaranteed
(P2: the ordering property is "none" — explicitly documented).
- A **FIFO queue** delivers strict per-message-group order: all
messages with the same group ID are delivered to one consumer
in send order. The cost is throughput (FIFO queues cap at lower
TPS) and latency (the broker must sequence per group). The
ordering property is "per-group strict" — explicitly documented
(P2).
- The choice is a P2 decision (which ordering guarantee) and a C8
decision (throughput cost). A consumer that assumes FIFO on a
standard queue is a P2 violation: the broker does not provide
the guarantee the consumer assumes. Document the property; do
not assume it.
## Prefetch and Concurrency (P6 Backpressure is Bounded)
- **Prefetch** (or max-unacked) bounds how many messages the
broker delivers to one consumer without an ack. A prefetch of 1
is strict stop-and-wait (lowest throughput, tightest backpressure);
a prefetch of N allows the consumer to process N in flight
(higher throughput, more memory). An unbounded prefetch is a P6
violation: the broker floods the consumer's memory.
- **Consumer concurrency** is the number of parallel workers
processing from the queue. More workers increase throughput up to
the downstream's limit; beyond that, the workers stall the
downstream (P6 — the backpressure propagates to the
downstream, not the broker).
- The prefetch × concurrency product is the in-flight cap. Declare
it (P6): an undeclared cap is a defect — the consumer either
underutilizes the broker (prefetch too low) or OOMs under load
(prefetch too high). This is the cross-process analog of
`domains/concurrency/patterns` Pattern 5 (bounded queue with
backpressure): concurrency owns the in-process analog;
messaging owns the broker-backed instance.
## Long Polling (P6, C8 Economy)
- Long polling (or `ReceiveMessage` with a wait-time-seconds)
holds the receive request open until a message arrives or the
wait expires. This reduces empty-receive round trips (C8
economy of the constrained link) and reduces latency-to-first-
message (the message is delivered when it arrives, not on the
next poll cycle).
- Long polling is the default for low-throughput queues: short
polling burns CPU on empty receives; long polling waits for
work. For high-throughput queues, the broker is usually full
enough that long polling adds no latency; for low-throughput
queues, long polling is the difference between 20ms and 20s
latency-to-first-message.
## Redelivery + DLQ Flow (P5 Dead-Letter Handling is Defined)
- A poison message (unparseable, repeatedly failing, or exhausting
the retry budget) routes to the dead-letter queue. The DLQ is
observable (P10 — DLQ depth is an alert) and drainable (an
operator can inspect, replay, or discard with audit).
- The retry budget is bounded (P6): N redeliveries, or a TTL with
exponential backoff. After the budget is exhausted, the message
is moved to the DLQ, not retried forever. An unbounded retry is
the `messaging-unbounded-retry` chaos anti-pattern (P5 breach).
- The DLQ routing rule is a redelivery-count or TTL threshold
plus a target queue. See `domains/messaging/delivery-semantics.md`
for the dead-letter strategy comparison table.
```python
# Redelivery + DLQ flow (P5 dead-letter handling, P6 bounded
# retry budget). The consumer tracks redelivery count; after the
# budget, the message routes to the DLQ. The DLQ is observable
# (P10 — depth is an alert) and drainable.
MAX_REDELIVERIES = 5
DLQ = "orders-dlq"
def consume_with_dlq(broker, dedup_store, process_order):
for message in broker.receive(queue="orders"):
# P3 idempotency: a redelivered, already-processed message
# is acked and skipped (not re-processed, not DLQ'd).
if dedup_store.seen(message["idempotencyKey"]):
broker.ack(message)
continue
try:
process_order(message["payload"])
dedup_store.mark(message["idempotencyKey"])
broker.ack(message)
except Exception as exc:
# P5: bounded retry budget. After MAX_REDELIVERIES,
# route to DLQ — do NOT retry forever.
count = message.get("redeliveryCount", 0) + 1
if count >= MAX_REDELIVERIES:
broker.send(DLQ, body=json.dumps({
"original": message,
"reason": str(exc),
"deadLetteredAt": now_iso(),
"redeliveryCount": count,
}))
broker.ack(message) # remove from the origin queue
# P10: the DLQ depth must alert. A DLQ that grows
# with no alert is a silent correctness defect.
else:
# Nack with backoff: the broker redelivers after a
# delay. The backoff caps the retry rate (P6).
broker.nack(message, delay=exponential_backoff(count))
```
- The `deadLetteredAt` and `reason` fields make the DLQ entry
observable and auditable: an operator inspecting the DLQ sees
why each message was dead-lettered and when. See
`domains/errors/patterns` for the errors-as-data discipline the
DLQ entry follows.
## Delivery Semantics Comparison (D-069)
| Semantic | Guarantee | Latency Cost | Implementation Cost | When It Fits |
|----------|-----------|-------------|---------------------|--------------|
| **At-most-once** | A message is delivered 0 or 1 times; loss is possible, duplication is not | Lowest (no ack; fire-and-forget) | Lowest (no ack, no dedup) | Telemetry where a dropped sample is acceptable; high-throughput metrics; MQTT QoS 0; logs where a lost line is tolerable. Never for billing, orders, or any side-effecting operation. |
| **At-least-once** | A message is delivered 1 or more times; duplication is possible, loss is not | Low (one ack round-trip) | Medium (consumer must be idempotent — P3; dedup store required) | The default for side-effecting operations: orders, payments, commands. The consumer dedups via idempotency keys (P3); the broker guarantees delivery. Fits the vast majority of broker-backed queues (SQS standard, RabbitMQ ack, MQTT QoS 1). |
| **Exactly-once** | A message is delivered exactly 1 time; no loss, no duplication | Highest (two-phase commit or transactional producer+consumer) | Highest (requires transactions, a transactional producer, and a transactional consumer — or at-least-once plus idempotency, which collapses to at-least-once with dedup) | Rare. Kafka transactions (consume-process-produce in one transaction); MQTT QoS 2 (four-step handshake). In practice, "exactly-once" is usually at-least-once plus idempotency (P3) — the broker does not guarantee it; the consumer enforces it. Jepsen analyses verify broker claims. |
- The default for side-effecting operations is **at-least-once with
idempotent consumers** (P3). At-most-once is for loss-tolerant
telemetry. Exactly-once is reserved for the narrow case where
the consume-process-produce loop must be transactional (Kafka
transactions) — and even then, the consumer should be idempotent
as defense-in-depth.
- The latency cost column is the C8 tradeoff: at-most-once is
cheapest, exactly-once is most expensive. The implementation
cost column is the C1/C3 tradeoff: at-most-once is simplest,
exactly-once is most complex (and most fragile — a transactional
consumer that partially fails is a bug source). The "when it
fits" column is the P4 decision: declare the semantic per
channel, do not let it emerge.
- This is the decision matrix required by D-069 for queues; see
`domains/messaging/delivery-semantics.md` for the correctness
properties of each semantic and the dead-letter strategy
comparison.
## Observability — Consumer Lag (P10 Messaging is Observable)
- Consumer lag (messages enqueued minus messages acked, or the
age of the oldest unacked message) is the primary queue health
signal. A lag that grows without bound is a P6 violation (the
consumer is slower than the producer) and a P10 violation if it
is not alerted.
- DLQ depth is the secondary signal: a DLQ that grows is a P5
signal (poison messages are accumulating) and a P10 signal if
not alerted. Wire both to `domains/observability/metrics` as
SLIs with SLOs (e.g., lag < 1000 messages, DLQ depth < 10).
- A queue with no lag metric is operating blind (P10 violation):
the operator cannot see the consumer falling behind until the
downstream effect surfaces.
## Cross-Link to Concurrency (P6, cross-link concurrency/patterns)
- The broker-backed queue is the cross-process analog of the
in-process bounded buffer. `domains/concurrency/patterns`
Pattern 5 (Bounded Queue with Backpressure) owns the in-process
instance ("producer is blocked or signaled" within one program);
messaging owns the broker-backed instance above it (the producer
is the broker's enqueue, the consumer is the broker's dequeue, the
backpressure is the prefetch cap and the lag signal). The
failure model differs: in-process fails by OOM; broker-backed
fails by network partition, broker restart, or consumer
crash-and-retry (D-062).
- The cross-link is one-directional outward (messaging →
concurrency) per D-026 extended: messaging references concurrency
as the in-process foundation; concurrency does not back-link to
messaging.
## What Violates Queue Discipline
| Violation | Principle |
|-----------|-----------|
| Schemaless message (no versioned contract; consumer parses by guess) | P1 Messages are Contracts |
| Non-idempotent consumer under at-least-once delivery (redelivery doubles the effect) | P3 Consumers are Idempotent |
| Unstated delivery semantic (at-least-once vs exactly-once guessed) | P4 Delivery Semantics are Explicit |
| No DLQ (poison message retried forever or silently dropped) | P5 Dead-Letter Handling is Defined |
| Unbounded prefetch (broker floods consumer memory) | P6 Backpressure is Bounded |
| Unbounded retry budget (no cap; consumer never progresses past the poison) | P5, P6 |
| Consumer that assumes FIFO on a standard queue (ordering not guaranteed) | P2 Ordering is a Property, Not an Assumption |
| Visibility timeout shorter than processing time (redeliver while still processing) | P4, P6 |
| DLQ with no depth alert (poison messages accumulate invisibly) | P10, `domains/observability/metrics` |
| No consumer-lag metric (consumer falls behind invisibly) | P10, `domains/observability/metrics` |
| Dedup store that grows without bound (memory leak) | P6 Backpressure is Bounded |
| Default prefetch with no rationale (underutilizes or OOMs) | P6, `domains/concurrency/patterns` |
+329
View File
@@ -0,0 +1,329 @@
# Streams — Derived Rules
> Derives from `domains/messaging/first-principles.md`. Applies P8
> (Replay and Retention are Configured) primarily, with P2
> (per-partition ordering), P3 (idempotent consumers), P4 (exactly-
> once via transactions), P7 (partitioning), and P10 (observable
> consumer-group health). For the Kafka / Kinesis / Pulsar /
> NATS JetStream decision, see the stream-platform comparison
> table below. For the consumer-group rebalance strategy choice,
> see the rebalance enumeration below (IDEATE-41). Cross-links
> `domains/messaging/delivery-semantics` for exactly-once via
> transactions, `domains/data/schema-design` for stream schema,
> and `domains/observability/tracing` for cross-partition traces.
## What a Stream Is (P8 Replay and Retention are Configured)
- A stream is a durable-log messaging primitive. Messages are
appended to a partitioned, replicated log; consumers read from an
offset and advance at their own pace. The log is retained for a
configured window (P8) — a stream is a durable log, not a pipe.
A consumer that falls behind can catch up by replaying from an
earlier offset; a consumer that starts fresh can replay from the
beginning (within retention).
- The boundary with queues and pub/sub is the durability model. A
queue deletes on ack; a pub/sub subscription retains only its
unacked window; a stream retains the whole log for the configured
retention. This makes a stream replayable (P8 — the
reversibility mechanism) and observable as a record (P10 — the
log is itself an audit of what happened). See
`domains/messaging/pubsub.md` for the pub/sub-vs-stream
durability boundary discussion.
- The boundary with concurrency is per D-062: messaging owns the
cross-process/network-failure-model angle. A stream is a
messaging concern because the log spans brokers and consumers
across a network, and the failure model is partition, broker
restart, or consumer crash-and-retry — not in-process OOM or
thread crash.
## Partitioned Log Model (P2, P7)
- A stream is partitioned for throughput and parallelism. Each
partition is an ordered, append-only log; messages within a
partition are strictly ordered (P2 — per-partition strict order
is the documented property). Across partitions, there is no
ordering guarantee: partition 0 and partition 1 are independent
logs.
- The partition key (P7 — Partitioning is Intentional) determines
which partition a message lands on. A key that hashes evenly
spreads load; a key that matches the per-entity ordering need
(e.g., `userId` for user events) keeps a user's events on one
partition in order; a key that is too coarse (one partition for
the whole topic) serializes all traffic.
- The partition count is a capacity bound: it caps the parallelism
(one consumer per partition per consumer group) and the
throughput (each partition has a write-throughput limit). Too
few partitions cap parallelism; too many partition overhead the
broker (file handles, replication, rebalance cost). The choice
is documented (P7), not defaulted.
## Offsets (P2, P8)
- An offset is a consumer's position in a partition. The consumer
reads from its last committed offset; acking (committing the
offset) advances it. A consumer that crashes before committing
re-reads from the last committed offset (at-least-once by
default, P4) — the consumer must be idempotent (P3).
- The offset is per-partition (P2): each partition has its own
position, and the consumer commits them independently (or
atomically across partitions in a transaction — see below).
- Replay (P8) is resetting the offset backward: a consumer can
replay from the beginning of retention, from a timestamp, or
from a specific offset. This is the durable-log property that
distinguishes a stream from a queue.
## Consumer Groups (P3, P7, P10)
- A consumer group is a set of consumers sharing the stream's
partitions: each partition is assigned to exactly one consumer
in the group. The group is the unit of parallelism and the unit
of offset tracking. Within a group, each consumer handles its
assigned partitions; across groups, each group independently
reads the whole stream (the pub/sub fan-out property, per
subscription/group).
- A consumer in the group is idempotent (P3): under at-least-once
(the default), a redelivery after a crash-and-retry re-processes
messages. The consumer dedups by idempotency key, or processes
deterministically (e.g., a stateful aggregation that overwrites
with the latest value).
- The consumer group's health is observable (P10): per-partition
lag (offset of the consumer vs the log's head), the group's
consumption rate, and rebalance events are first-class signals.
A group whose lag grows without bound is a P6 (backpressure)
smell and a P10 (observability) violation if not alerted.
```python
# Consumer-group reading from offsets (P2 per-partition order,
# P3 idempotent under at-least-once, P7 partition assignment,
# P8 replay from offset). Each consumer in the group handles its
# assigned partitions; the group commits offsets atomically or
# per-partition.
def consume_stream(stream, group, dedup_store, process_event):
# Assign partitions to this consumer by the group's
# rebalance strategy (see the enumeration below).
for partition in stream.assigned_partitions(group, consumer=ME):
# Read from the last committed offset (P8 — replay by
# resetting this offset).
offset = stream.committed_offset(group, partition)
for message in stream.read(partition, from_offset=offset):
# P3: idempotent under at-least-once. A redelivery
# after a crash-and-retry re-processes; dedup by key.
if dedup_store.seen(message["idempotencyKey"]):
stream.commit(group, partition, message["offset"])
continue
process_event(message["payload"])
dedup_store.mark(message["idempotencyKey"])
# Commit the offset to advance (P8 — the position is
# the replay pointer).
stream.commit(group, partition, message["offset"])
```
- The commit-after-process order gives at-least-once (a crash
before commit re-reads); the commit-before-process order gives
at-most-once (a crash after commit loses the unprocessed
message). The default is at-least-once with idempotent consumers
(P3, P4).
## Consumer-Group Rebalance Strategies (IDEATE-41, ATELIER-100 refinement)
When a consumer joins or leaves the group, the broker must
reassign partitions. The rebalance strategy determines the cost
and the use-case fit. This enumeration parallels the v0.3
IDEATE-30 drift-type enumeration (each strategy with its
stop-the-world cost and use-case fit).
| Strategy | Mechanism | Partition Stop-the-World Cost | Use-Case Fit |
|----------|-----------|-------------------------------|--------------|
| **Eager rebalance** (stop-the-world) | Every consumer in the group revokes ALL its partitions, the broker reassigns the full partition set, then consumers resume. Every rebalance pauses the whole group. | High — every partition pauses for every rebalance; the whole group stops processing during the revocation+reassignment window. Throughput drops to zero during rebalance. | Simple brokers, small groups, or rarely-rebalancing groups where the simplicity of full revocation outweighs the pause cost. Kafka's legacy protocol (pre-2.4). Avoid for large groups or frequent scale events. |
| **Sticky (incremental cooperative) rebalance** | The broker reassigns only the partitions that must move (the joining/leaving consumer's share); existing partitions stay assigned. The rebalance is incremental and cooperative — no full revocation. | Low — only the moving partitions pause; the rest of the group continues processing. The pause is proportional to the changed partition count, not the total. | The default for large groups, frequent scale events, and rolling deploys. Kafka's CooperativeStickyAssignor (2.4+), Pulsar, NATS JetStream. Prefer for any group where a full stop-the-world on every deploy is unacceptable. |
| **Cooperative (no-revoke) rebalance** | A subset of sticky where no partition is revoked unless the consumer leaves; only additions are incremental. The strictest minimization of stop-the-world. | Lowest — only added partitions pause; existing assignments are untouched. | Groups where partition assignment is append-only (consumers join but rarely leave). Useful for long-lived consumers with incremental scaling. |
- The default for any non-trivial group is **sticky/cooperative**:
a rolling deploy that triggers an eager rebalance pauses the
whole group on every pod restart, which is unacceptable at
scale. The eager strategy is a legacy default that survives
because it is simple; prefer sticky where the broker supports
it.
- The "partition stop-the-world cost" column is the P6
(backpressure) angle: a full stop-the-world during rebalance
causes lag to spike (the consumer is paused, the producer is
not). Sticky rebalance bounds the spike to the moving partitions.
- A rebalance that pauses without a lag alert is a P10 violation:
the operator cannot see the rebalance-induced lag. Wire rebalance
events to `domains/observability/metrics` as an event signal.
## Replay and Retention Windows (P8, C5 Reversibility)
- Retention is the configured window the log keeps messages: time-
based (e.g., 7 days), size-based (e.g., 10 GB per partition), or
compacted (keep the latest value per key — a changelog). A
stream with no retention is a pipe, not a log (P8 violation); a
stream with infinite retention grows until disk exhaustion (P6
violation — backpressure on the broker).
- Replay (P8) is resetting a consumer's offset to re-read from
within the retention window. Use cases: reprocessing after a
consumer bug (replay from the timestamp of the buggy deploy),
backfilling a new consumer (replay from the beginning), or
reindexing (replay to rebuild a derived store).
- Compacted topics (Kafka log-compaction, Pulsar compaction) keep
the latest value per key and discard older values for the same
key. This turns the log into a changelog — a durable
materialized view that replays to the current state. Compaction
is a P8 (retention) and C5 (reversibility) mechanism: the log
retains the current state per key and is replayable to it.
## Stream Processing (P2, P3, P4)
- Stream processing is computing over the stream as it arrives:
windowing (tumbling, sliding, session windows), joins (stream-
stream, stream-table), aggregations (count, sum, per-key
windows), and stateful transformations. The processing is
per-partition ordered (P2 — a window over a partition is
deterministic; a window across partitions is not unless the
window is global).
- Stream processing consumers are idempotent (P3): a redelivery
after a crash re-processes a window; the aggregation must
tolerate re-application (e.g., a sum is idempotent under replay
if the window is keyed by offset range, not by wall time).
- Exactly-once stream processing (P4) requires transactions: the
consume-process-produce loop is one transaction — the input
offset commit and the output produce are atomic. See the
transactional exactly-once producer below.
## Exactly-Once via Transactions (P4, P3)
- Exactly-once stream processing is at-least-once plus a
transaction: the consumer commits the input offset and produces
the output in one transactional operation. If the consumer
crashes mid-transaction, neither the offset commit nor the
output produce happens — the consumer re-reads from the last
committed offset and re-processes (at-least-once), but the
transaction ensures the output is produced exactly once.
- This is NOT a broker guarantee of exactly-once delivery; it is
at-least-once delivery plus idempotent/transactional processing
(P3, P4). Jepsen analyses of Kafka transactions confirm the
boundaries: the transaction is atomic within the broker, but
the downstream sink must be transactional or idempotent too.
- See `domains/messaging/delivery-semantics.md` for the full
exactly-once-via-idempotency discussion.
```python
# Transactional exactly-once producer (P4 exactly-once via
# transactions, P3 idempotent produce). The consume-process-
# produce loop is one transaction: the input offset commit and
# the output produce are atomic. A crash mid-transaction rolls
# both back; the consumer re-reads and re-processes.
def consume_transform_produce(stream, group, txn_producer):
# Begin a transaction. All produces and the offset commit in
# this block are atomic (P4).
with txn_producer.transaction() as txn:
for partition in stream.assigned_partitions(group, ME):
offset = stream.committed_offset(group, partition)
for message in stream.read(partition, from_offset=offset):
output = transform(message["payload"])
# P3: idempotent produce. The txn producer
# dedups by an epoch+sequence so a retried
# transaction does not double-produce.
txn.produce(
topic="enriched-events",
key=message["key"],
value=output,
idempotencyKey=message["idempotencyKey"],
)
# Commit the input offset within the same
# transaction (P4 atomicity). A crash before
# txn.commit() rolls this back; the consumer
# re-reads from the prior offset.
txn.commit_offset(group, partition, message["offset"])
# txn.commit() makes the produces and the offset commit
# visible atomically. A crash before this point aborts
# both; a crash after is safe (idempotent produce — P3).
```
- The transactional producer's idempotency (P3) is the defense
against a retried transaction: the broker dedups the output by
the producer's epoch and sequence so a re-commit does not
double-produce. The transaction (P4) is the defense against a
partial failure: the offset and the output commit together.
## Stream Schema (P1, P9, cross-link data/schema-design)
- A stream's messages carry a versioned schema (P1). The schema
evolves compatibly (P9): a new field the old consumer ignores is
backward-compatible; a renamed field the old consumer parses as
`undefined` is a P1 violation compounded as an evolution defect.
- Stream schemas are often registered in a schema registry
(Confluent, Apicurio) that enforces compatibility on produce.
A producer that tries to publish an incompatible schema is
rejected; the registry is the P1/P9 enforcement point.
- See `domains/data/schema-design` for the generic schema-design
discipline (Avro, Protobuf, JSON Schema); messaging owns the
stream-specific instance — the registry, the per-topic
compatibility mode, the consumer-side routing by schema
version.
## Stream-Platform Comparison (D-069)
| Axis | Apache Kafka | AWS Kinesis | Apache Pulsar | NATS JetStream |
|------|--------------|-------------|---------------|----------------|
| **Ordering** | Per-partition strict (P2); global only via single-partition topic | Per-shard strict; global only via single shard | Per-partition strict; global via single partition; also supports shared (out-of-order) subscriptions | Per-stream strict; per-subject ordering; global via single stream |
| **Partitioning model** | Partitions (immutable count post-creation; increase requires recreate); key→partition by hash | Shards (reshardable: split/merge at runtime); key→shard by hash | Partitions (resizable; Pulsar's layered architecture separates compute from storage); key→partition by hash | Streams (subject-based; republish to resize); key→stream by subject |
| **Replay / retention** | Time- or size-based retention; compaction (latest-per-key); replay from offset or timestamp | Time-based retention (24h365d); replay from sequence number or timestamp; no compaction | Time- or size-based; compaction; replay from offset or timestamp; tiered storage (hot/warm/cold) | Time- or size-based; per-stream max-age; replay from sequence; no native compaction |
| **Consumer groups** | Group-coordinated; offsets stored in an internal topic; eager (legacy) or sticky/cooperative (2.4+) rebalance | Enhanced fan-out consumers (per-shard HTTP/2 push); KCL for group coordination; no native group rebalance (shard is the unit) | Group-coordinated; shared or failover subscription modes; cooperative rebalance | Per-stream consumers; durable cursors; no native group rebalance (stream is the unit) |
| **Exactly-once** | Transactions (KIP-98): consume-process-produce atomic; idempotent producer (KIP-516) | No native exactly-once; at-least-once with consumer-side dedup (P3) | Transactions: produce-ack atomic; idempotent producer | At-least-once by default; dedup window per stream (P3 idempotency) |
| **Use-case fit** | High-throughput durable logs, stream processing (Kafka Streams, Flink), event sourcing, multi-consumer replay | AWS-native streaming, log ingestion, simple ETL within AWS; low operational burden | Cloud-native, geo-replication, tiered storage, mixed pub/sub + streaming; multi-tenant | Lightweight, low-latency, edge-friendly; NATS ecosystem; simpler ops than Kafka |
| **Watch out for** | Partition count is fixed at creation (resize requires recreate + republish); rebalance cost on large groups; operational complexity | Shard limits per account; no compaction; retention cap at 365 days; AWS lock-in | Two-arch (BookKeeper + Brokers) operational complexity; smaller ecosystem | Smaller ecosystem; no native compaction; fewer stream-processing libraries |
- The default for high-throughput durable logs with multi-consumer
replay is **Kafka**; for AWS-native streaming, **Kinesis**; for
geo-replicated multi-tenant or mixed pub/sub + streaming,
**Pulsar**; for lightweight low-latency edge-friendly streaming,
**NATS JetStream** (which also cross-links `domains/edge/iot`
via MQTT parallels — see the edge↔messaging bidirectionality in
`domains/messaging/first-principles.md` §4).
- The ordering column is the P2 check: every platform provides
per-partition/per-shard strict order; none provides global order
across partitions except by single-partition. The replay/
retention column is the P8 check: every platform retains for a
configured window; replay is from offset or timestamp. The
exactly-once column is the P4 check: Kafka and Pulsar provide
transactions; Kinesis and JetStream rely on at-least-once plus
consumer-side idempotency (P3).
## Cross-Partition Traces (P10, cross-link observability/tracing)
- A stream-processing pipeline that fans out across partitions
must propagate a trace context per event: the trace ID follows
the event from source to processed output, even as the event
crosses partition boundaries. Without cross-partion traces, a
downstream error cannot be traced back to its source event.
- See `domains/observability/tracing` for the generic distributed-
tracing discipline (trace context propagation, span
correlation). Messaging owns the stream-specific instance: the
trace context is a message header, the span boundary is the
consume-process-produce edge, and the cross-partition
correlation is by trace ID (not by partition — partitions are
independent logs, P2).
- A stream processor with no trace propagation is a P10
violation: the operator cannot trace a processed event back to
its source. Wire the trace context into every produce and every
consume.
## What Violates Stream Discipline
| Violation | Principle |
|-----------|-----------|
| Stream with no retention (pipe, not log; no replay) | P8 Replay and Retention are Configured |
| Infinite retention (grows until disk exhaustion) | P8, P6 |
| Consumer group with eager rebalance at scale (full stop-the-world per deploy) | P6, IDEATE-41 (use sticky/cooperative) |
| Non-idempotent stream consumer under at-least-once (redelivery re-processes the window) | P3 Consumers are Idempotent |
| Default partition key (no rationale; hotspot or wrong-order) | P7 Partitioning is Intentional |
| Partition count too low (caps parallelism) or too high (overhead) | P7 |
| Cross-partition order assumption (no global order guarantee) | P2 Ordering is a Property, Not an Assumption |
| Exactly-once claimed without transactional consume-process-produce (P4 violation) | P4 Delivery Semantics are Explicit |
| Stream schema with no registry / no compatibility enforcement (silent shape break) | P1, P9, `domains/data/schema-design` |
| No per-partition lag metric (consumer falls behind invisibly) | P10, `domains/observability/metrics` |
| No cross-partition trace propagation (downstream error untraceable) | P10, `domains/observability/tracing` |
| Compacted topic treated as a full log (old values already discarded) | P8, C1 (compaction is a retention mode, not a full log) |
+43
View File
@@ -0,0 +1,43 @@
# Observability — First Principles
## 1. The Principles
### P1. Structured by Default
Logs, metrics, and traces are structured. Free-form text is for
humans; machines need fields.
### P2. Correlation
Every event is traceable to a request, a user, an action, a trace ID.
Context flows through the system.
### P3. Sufficient Context
The information needed to debug is in the event itself, not in tribal
knowledge. "What was the user doing?" is answerable from logs.
### P4. Cardinality Discipline
Labels and tags have bounded cardinality. Unbounded labels are an
unbounded bill.
### P5. Sampling with Intent
Sampling is deliberate, documented, and consistent. Head-based,
tail-based, or none — chosen with reason.
### P6. No Secrets in Observability
Observability data is not a secrets channel. Tokens, passwords, and
PII do not enter logs, metrics, or traces.
### P7. Actionable Alerts
Alerts are for things humans must act on. Every alert has a runbook.
Alert fatigue is a defect.
### P8. SLI/SLO Awareness
"Good enough" is defined. SLOs are targets, not aspirations.
Error budgets are real.
### P9. Cost Awareness
Observability has a cost — storage, compute, attention. Spend it on
what earns it.
### P10. Debuggability Over Coverage
A few high-cardinality traces beat millions of low-context logs.
Signal beats volume.
+49
View File
@@ -0,0 +1,49 @@
# Logging — Derived Rules
> Derives from `domains/observability/first-principles.md` P1 (Structured by Default), P3 (Sufficient Context), P6 (No Secrets in Observability).
## Structured by Default (P1)
- Logs are JSON (or structured key-value). Free-form text is for humans; machines need fields.
- Every log entry has: `timestamp`, `level`, `message`, `request_id`, plus domain-specific fields.
- A log you cannot query is a log you cannot use. Structure is the query API.
## Levels (P3 Sufficient Context)
| Level | When |
|-------|------|
| ERROR | Something failed; an operator must look |
| WARN | Something unexpected; not a failure but notable |
| INFO | Significant application events (start, stop, deploy, user signup) |
| DEBUG | Diagnostic detail; off in production by default |
- ERROR is not for "this branch ran." ERROR is for "this failed and someone should know."
- Logging everything at ERROR means nothing is an error. Alert fatigue is a defect (observability P7).
## Context (P3)
- Every log in a request includes `request_id` (correlation ID). Trace the request across services.
- Include the user ID, the action, the resource. "What was the user doing?" is answerable.
- A log that says `"failed"` with no context is worse than no log. It is noise.
## No Secrets (P6, domains/security P9)
- Never log tokens, passwords, API keys, session IDs, PII.
- Redact: replace the secret with `[REDACTED]` or a hash. Log the hash, not the value.
- Never log the full request body. It may contain a token, a password, or PII.
## Volume (P4 Cardinality Discipline, core C8 Economy)
- Don't log every request at INFO. Log significant events.
- A million logs a minute is not "good observability"; it is a storage bill and a signal-to-noise problem.
- Sample high-volume logs (P5 Sampling with Intent). Sample deliberately, not randomly.
## What Violates Logging Discipline
| Violation | Principle |
|-----------|-----------|
| `console.log("here")` | P1 (not structured) |
| `logger.error("done")` | P3 (wrong level) |
| `logger.info(req.body)` | P6 (secrets), volume |
| A log with no `request_id` | P3 (no correlation) |
| 10M logs/day at INFO | P4, C8 |
+50
View File
@@ -0,0 +1,50 @@
# Metrics — Derived Rules
> Derives from `domains/observability/first-principles.md` P1 (Structured by Default), P4 (Cardinality Discipline), P8 (SLI/SLO Awareness).
## The Four Golden Signals
| Signal | What |
|--------|------|
| Latency | Time to serve a request (p50, p95, p99) |
| Traffic | Request rate (req/s) |
| Errors | Error rate (errors/s, or % of traffic) |
| Saturation | How full is the system (CPU, memory, queue depth) |
- All four are needed. Missing one is a blind spot.
- Latency is percentiles, not average. Average hides the long tail.
## Cardinality (P4)
- Labels have bounded cardinality. `user_id` as a label = unbounded cardinality = unbounded bill.
- High-cardinality dimensions belong in traces, not metrics.
- A metric with `user_id` as a label is a 1M-series metric. That is a budget bomb.
## Counter vs Gauge vs Histogram
| Type | What | Example |
|------|------|---------|
| Counter | Monotonically increasing | `http_requests_total` |
| Gauge | A value at a point in time | `active_connections` |
| Histogram | Distribution of values | `http_request_duration_seconds` |
- A counter never decreases. Use `rate()` over time to get the rate.
- A gauge can go up and down. Use it for saturation.
- A histogram gives percentiles. Use it for latency.
## SLI/SLO (P8)
- SLI (Service Level Indicator): a metric of good/total (e.g., 99.9% of requests < 500ms).
- SLO (Service Level Objective): the target for the SLI (e.g., 99.9% over 30 days).
- Error budget: 1 - SLO. If SLO is 99.9%, error budget is 0.1%. Spend it on feature risk, not bugs.
- When the error budget is exhausted, freeze features. Fix reliability.
## What Violates Metrics Discipline
| Violation | Principle |
|-----------|-----------|
| `user_id` as a label | P4 Cardinality |
| Average latency only | P8 (hides the tail) |
| No error rate metric | P8 (no SLI) |
| 1000 metrics, no SLO | P8 (no objective) |
| A counter that decreases | (type error) |
+45
View File
@@ -0,0 +1,45 @@
# Tracing — Derived Rules
> Derives from `domains/observability/first-principles.md` P2 (Correlation), P5 (Sampling with Intent), P10 (Debuggability Over Coverage).
## Distributed Tracing (P2)
- A trace is a tree of spans. Each span is a unit of work with a start, end, and context.
- `trace_id` ties spans across services. `span_id`/`parent_span_id` form the tree.
- Every request has a `trace_id`. Propagate it in headers (`traceparent`, W3C standard).
## Sampling (P5)
- Head-based: sample at the start. Simple, but you miss the interesting failures.
- Tail-based: sample at the end. Keep all errors, sample the successs. Better signal, harder to build.
- A 100% trace rate is too expensive. 1% is often enough for debugging.
- Sample deliberately: keep all errors, all slow requests, and a fraction of the rest.
## Context (P3 Sufficient Context, P10 Debuggability)
- A span has: name, start time, duration, attributes (key-value), events, status.
- Attributes: `http.method`, `http.url`, `db.statement`, `user.id`. The fields you need to debug.
- Events: notable points within a span (e.g., "cache miss", "retry").
- A span with no attributes is a span that tells you nothing.
## Where to Span (P4 Locality)
- Span at service boundaries (HTTP in/out, DB query, queue send/receive).
- Span at significant internal operations (a long computation, a batch step).
- Don't span every function call. Span the meaningful units.
## Traces vs Logs (P10)
- Logs are events; traces are causality. Logs answer "what happened"; traces answer "why it was slow."
- A trace contains log events (span events). They are not separate systems.
- Use traces for the request flow; use logs for the details.
## What Violates Tracing Discipline
| Violation | Principle |
|-----------|-----------|
| No `trace_id` propagation | P2 Correlation |
| 100% trace rate | P5 Sampling |
| Span per function | P4 (too noisy) |
| Span with no attributes | P10 (no debug value) |
| Traces for successful requests only | P5 (miss the failures) |
+51
View File
@@ -0,0 +1,51 @@
# Backend Performance — Derived Rules
> Derives from `domains/performance/first-principles.md` P1 (Measure First), P3 (Complexity Awareness), P4 (Resource Bounds).
## Measure First (P1)
- p50, p95, p99 latencies. The average hides the long tail.
- Throughput (req/s) under load. Saturation point (where latency rises).
- Resource utilization: CPU, memory, I/O, network. Each is a budget.
## N+1 Queries (P3 Complexity Awareness)
- A query in a loop is an N+1. It is O(N) queries instead of O(1).
- Detect with a query counter in tests. A test that issues 100 queries is failing.
- Fix with a JOIN, a batch load, or a dataloader. Never "we'll fix it later."
## Caching (P5 Caching with Intent)
- Cache what is: expensive to compute, stable, read often.
- Invalidation is designed: TTL, event-based, or version-based. Never "we'll just clear it."
- A cache without an invalidation strategy is a cache that serves stale data forever.
- Multi-level: HTTP cache → CDN → app cache → DB. Each layer has its own rules.
## Async and Concurrency (P7 Async When Independent, see `domains/concurrency/`)
- I/O-bound work is async. Don't block a thread on a network call.
- CPU-bound work is in a worker, not the request path.
- Bounded queues everywhere (P9 Bounded Queues). Unbounded = OOM.
## Database (P4 Resource Bounds, see `domains/data/indexing.md`)
- Connection pool: bounded. The DB has a connection limit; the pool respects it.
- Slow queries: logged, explained, fixed. A 10-second query is a bug.
- Pagination on large tables: cursor, not offset. Offset scans rows.
## Resource Bounds (P4)
- Memory: bounded. A request that allocates unbounded memory is a DoS vector.
- Timeouts: every external call has one. A call without a timeout is a call that can hang forever (P8 Timeout Discipline).
- File handles, DB connections, HTTP connections: all bounded, all pooled.
## What Violates Backend Performance
| Violation | Principle |
|-----------|-----------|
| N+1 query in a loop | P3 |
| No timeout on an HTTP call | P4, P8 (concurrency) |
| Unbounded in-memory sort | P4 |
| Cache with no invalidation | P5 |
| `SELECT *` | P4 (data P10) |
| Connection pool size = 1000 | P4 (DB limit) |
+40
View File
@@ -0,0 +1,40 @@
# Performance — First Principles
## 1. The Principles
### P1. Measure First
No optimization without measurement. Intuition about performance is
usually wrong.
### P2. Critical Path Focus
Optimize what users actually wait for. The 95th percentile matters
more than the average.
### P3. Complexity Awareness
Algorithmic cost is known. Big-O is a design conversation, not an
afterthought.
### P4. Resource Bounds
Memory, CPU, I/O, network — all bounded. Unbounded growth is a bug.
### P5. Caching with Intent
Cache what is expensive, stable, and read often. Invalidation is
designed, not bolted on.
### P6. Lazy by Default
Compute only when needed. Pay only for what is used.
### P7. Async When Independent
Work that does not depend on other work runs in parallel.
### P8. Budget Discipline
Performance is a design constraint. The budget is set, not negotiated
after the fact.
### P9. Perceived Performance
What the user feels is what matters. A 200ms perceived response beats
a 50ms measured one with no feedback.
### P10. Regression Prevention
Performance tests catch what functional tests miss. The slow path
is tested as a path.
+55
View File
@@ -0,0 +1,55 @@
# Frontend Performance — Derived Rules
> Derives from `domains/performance/first-principles.md` P1 (Measure First), P9 (Perceived Performance).
## Measure First (P1)
- Lighthouse, Core Web Vitals (LCP, FID/INP, CLS), RUM (Real User Monitoring).
- A performance claim without a measurement is an opinion.
- Measure the 75th percentile (P75), not the average. The average hides the long tail.
## The Three Core Web Vitals
| Vital | What | Target (P75) |
|-------|------|--------------|
| LCP (Largest Contentful Paint) | When the main content loads | ≤ 2.5s |
| INP (Interaction to Next Paint) | When input is responded to | ≤ 200ms |
| CLS (Cumulative Layout Shift) | Visual stability | ≤ 0.1 |
- LCP > 4s is poor. INP > 500ms is poor. CLS > 0.25 is poor.
- Measure on mobile, not just desktop. Mobile is the long tail.
## Perceived Performance (P9)
- A skeleton screen beats a spinner. A spinner beats nothing.
- Optimistic UI updates: the click responds immediately; the server confirms later.
- Prefetch the next page on hover (if cheap). Prefetch is a bet, not a certainty.
## Bundle Size (P4 Resource Bounds, core C8 Economy)
- Ship less JavaScript. Every KB is parsed, compiled, and executed on the client.
- Code-split routes. Lazy-load below-the-fold. Don't ship the admin bundle to the user bundle.
- A 500KB JS bundle is large. A 2MB JS bundle is a defect.
## Images (P4, P6 Lazy by Default)
- WebP/AVIF, not JPEG/PNG. Modern formats are 3050% smaller.
- `loading="lazy"` on below-the-fold images. `width`/`height` to prevent CLS.
- `srcset` for responsive images. Ship the right size to the right device.
- Never ship a 4K image to a 360px screen.
## Rendering (P7 Async When Independent)
- Server-render the first paint (SSR/SSG). Hydrate after.
- Avoid hydration waterfalls: a 3-second hydration is a 3-second blank page with a "loaded" script.
- Defer non-critical hydration. Interactive above the fold first; below the fold later.
## What Violates Frontend Performance
| Violation | Principle |
|-----------|-----------|
| 3MB JS bundle | P4, C8 |
| LCP > 4s on mobile | P1 (measured) |
| Layout shift on image load | CLS |
| Synchronous hydration of a 50KB page | P7 |
| No image optimization | P4 |
+62
View File
@@ -0,0 +1,62 @@
# Authentication — Derived Rules
> Derives from `domains/security/first-principles.md` P1 (Zero Trust), P2 (Least Privilege), P6 (Crypto Correctness).
## The Default: Authenticated
- Every endpoint is authenticated unless explicitly public.
- "Public" is an explicit declaration, not a default.
- A missing auth check is a bug, not an oversight.
## Authentication Methods
### Session-based (browser)
- Server-side session, cookie-borne session ID.
- Cookie: `HttpOnly`, `Secure`, `SameSite=Lax` (or `Strict`).
- Session ID: cryptographically random, ≥ 128 bits.
- Session timeout: bounded. Idle timeout + absolute timeout.
### Token-based (API, SPA)
- Bearer token in `Authorization: Bearer <token>`.
- Token: JWT (signed) or opaque (server-stored).
- JWT: signed (HS256/RS256), never `none`. Short TTL (≤ 1 hour). Refresh token for long sessions.
- Opaque: server-stored, revocable. Use when revocation matters.
### API Keys (service-to-service)
- Long-lived, scoped, rotatable.
- Sent in header (`X-API-Key`), not query string (logged in URLs).
- Stored in a secrets manager, never in code.
## What Never to Do (P6 Crypto Correctness)
- Never roll your own auth. Use a vetted library or framework.
- Never store passwords in plaintext. Use bcrypt/scrypt/argon2 with a work factor.
- Never use MD5 or SHA1 for password hashing.
- Never put a token in a URL. URLs are logged.
- Never accept `alg: none` in a JWT.
- Never trust a token without verifying its signature.
## Password Rules (P4 Input Validation)
- Minimum length: 12 characters (NIST 800-63B). No maximum (don't prevent long passwords).
- No composition rules (no "must contain a symbol"). They don't help and frustrate users.
- Check against a breach corpus (HIBP API or similar).
- Rate limit login attempts. Lockout after N failures (with exponential backoff, not a hard lock).
## Multi-Factor (P3 Defense in Depth)
- MFA is the default for privileged accounts.
- TOTP (RFC 6238) or WebAuthn. SMS is deprecated (SIM swapping).
- MFA is a layer, not a replacement for strong primary auth.
## Session Lifecycle (P2 Least Privilege, P5 Reversibility)
- Sessions are revocable. A logout invalidates the session server-side, not just client-side.
- Tokens are revocable. A refresh token revocation list is maintained.
- "Remember me" extends the session, it does not make it permanent.
## Audit (P7 Auditability)
- Every auth event is logged: login (success/fail), logout, token issuance, token revocation.
- Logs include: user ID, timestamp, IP, user agent, outcome.
- Logs do not include: passwords, tokens, session IDs (use a hash).
+61
View File
@@ -0,0 +1,61 @@
# Authorization — Derived Rules
> Derives from `domains/security/first-principles.md` P2 (Least Privilege), P3 (Defense in Depth), P1 (Zero Trust).
## The Default: Deny
- Every request is denied unless explicitly authorized.
- "Authorized by default" is an anti-pattern. The absence of a rule means denial.
- A missing authz check is a bug, not a feature gap.
## Authorization Models
### RBAC (Role-Based)
- Users have roles; roles have permissions.
- Roles are coarse: `admin`, `editor`, `viewer`. Permissions are fine: `post:create`, `post:delete`.
- Check permissions, not roles: `can(user, 'post:create')`, not `user.role === 'admin'`.
- Roles can change; permission checks are stable.
### ABAC (Attribute-Based)
- Authorization based on attributes of the user, resource, and context.
- More expressive: "user can edit a post if user.department == post.department and post.status == 'draft'".
- Use when RBAC is too coarse. Beware: complex ABAC is hard to audit.
### ReBAC (Relationship-Based)
- Authorization based on relationships (e.g., Zanzibel).
- "user:alice is editor of document:42" — check the relationship graph.
- Scales for fine-grained, resource-specific access (Google Docs-style).
## Where to Check (P4 Locality)
- Check at the boundary: the API endpoint, the resolver, the controller.
- Check at the data layer: defense in depth. A query that bypasses the controller still respects row-level security.
- Never check only in the UI. The UI is a convenience, not a security boundary.
## Principle of Least Privilege (P2)
- A token/role gets the minimum permissions to do its job.
- No "admin" role for daily work. Admin is for administration; daily work uses a scoped role.
- Service tokens are scoped to one service's resources, not "all resources."
## IDOR (Insecure Direct Object Reference) (P1 Zero Trust)
- `/api/users/123` — does the requester own 123? Check.
- Never assume the user can access any ID they request. The ID is input; inputs are untrusted.
- Use scoped queries: `User.find({ id, owner: userId })`, not `User.find(id)`.
## Caching and Authz (P3 Defense in Depth)
- Authorization is not cached across users. A cached response for user A is not served to user B.
- Cache keys include the user/role, not just the resource.
- "Cache it as public if anyone can see it" — only if truly anyone (no auth).
## What Violates Authorization
| Violation | Principle |
|-----------|-----------|
| `/admin` endpoint with no authz check | P1 Zero Trust |
| `user.role === 'admin'` instead of permission check | P2 (roles change) |
| IDOR: `User.find(req.params.id)` with no ownership check | P1 Zero Trust |
| Cached authz decision reused across users | P3 Defense in Depth |
| Service token with "all resources" scope | P2 Least Privilege |
+102
View File
@@ -0,0 +1,102 @@
# Security — First Principles
**Version:** 1.0.0
**Status:** Foundational
**Audience:** AI agents and humans handling authentication, data,
trust boundaries, or any security-relevant code.
## 1. Manifesto
Security is not a feature. It is a property of correct code. The
highest quality code is code that does what it is supposed to do —
and nothing else, no matter who asks.
An AI agent using this framework does not "add security". It writes
secure code by default. There is no version of correct code that is
insecure.
## 2. The Principles
### P1. Zero Trust
No request, user, system, or input is trusted by default. Trust is
earned at every boundary, every time.
### P2. Least Privilege
Every actor — user, service, process — gets the minimum access
required to do its job, for the minimum time required.
### P3. Defense in Depth
Security is layered. No single control is load-bearing. The failure
of one control does not compromise the system.
### P4. Input Validation
All input is untrusted until proven otherwise. Validation happens at
the boundary, against a schema, with explicit failure modes.
### P5. Output Safety
All output is encoded, escaped, or filtered for its destination
context. The system never trusts its callers, including itself.
### P6. Cryptographic Correctness
Crypto is hard. Use vetted, maintained libraries. Never roll your
own. Never invent your own primitives. Never bypass a primitive to
"make it work".
### P7. Auditability
Security-relevant events — auth attempts, authz decisions, data
access, configuration changes — are logged with sufficient context
to investigate.
### P8. Fail Securely
When security fails, it fails closed. The default state is denied,
disabled, or safe. Errors never grant access by accident.
### P9. Secret Hygiene
Secrets are not in code, configs, logs, error messages, URLs, or
screenshots. Secrets are loaded from a secrets manager and treated
as transient.
### P10. Surface Minimization
The smaller the attack surface, the smaller the risk. Dependencies
are minimized. Endpoints are minimized. Features are minimized. Code
that does not exist cannot be exploited.
## 3. Conflict Resolution
1. Zero Trust — never sacrificed.
2. Least Privilege — never sacrificed.
3. Defense in Depth — never sacrificed.
4. Input Validation — never sacrificed.
5. Output Safety — never sacrificed.
6. Cryptographic Correctness — never sacrificed.
7. Fail Securely — never sacrificed.
8. Auditability — sacrificed only when logging itself is the threat.
9. Secret Hygiene — never sacrificed.
10. Surface Minimization — sacrificed only when a feature is required.
Eight of ten principles are non-tradeable. Security does not
trade-off. It is either present or it is not.
## 4. What Violates These Principles
| Violation | Principle Breached |
|------------------------------------|----------------------|
| `SELECT *` from user input | P4 Input Validation |
| `eval()` of any string | P4, P5 |
| Hardcoded API key in source | P9 Secret Hygiene |
| Catch-all `catch (e) {}` | P7 Auditability, P8 Fail Securely |
| `md5` or `sha1` for security | P6 Crypto Correctness |
| Open CORS to `*` in production | P1 Zero Trust, P10 |
| Detailed error to end user | P7 Auditability, P5 |
| `chmod 777` | P2 Least Privilege |
| Long-lived session token | P1, P2 |
| Logging the request body | P9 Secret Hygiene |
These are never acceptable. They are not "to be reviewed later".
They are rejected on sight.
## 5. Relationship to Core
Subordinate to `core/first-principles.md`. Note: security principles
overlap heavily with core Correctness (C1) and Observability (C7).
See `matrix/principles-matrix.md`.

Some files were not shown because too many files have changed in this diff Show More