From cc4c27c8abc1b4c18a6b7de4dd6a6c6293b087d3 Mon Sep 17 00:00:00 2001 From: Jon Chery Date: Tue, 21 Jul 2026 19:31:42 +0000 Subject: [PATCH] docs(P10): create Phase 10 plan (v1-spike-l2-and-contract-e2e) ---ci--- project: acdl phase: 10 milestone: v1.1 status: plan plan: waves: 5 tasks: 9 requirements: [REQ-25, REQ-27, REQ-28] ---/ci--- Phase 10 plan (milestone capstone). 5 waves, 9 files: - Wave 1 (platform): L2 composition + registry + adapter L2 extension (D-P10-1) - Wave 2 (backend): contract spike.yaml + contract_resolver.py + outbox_writer.py - Wave 3 (backend+security+platform): run_spike_e2e.sh (10-step pipeline) - Wave 4 (lead): verify_phase10.sh (incl. REQ-28 substrate-agnostic grep) - Wave 5 (EXECUTE-only): run e2e against real AWS + verify + ship Key decisions: D-P10-1 (adapter consumes resolved IR instance; L2 root module = L1 resource for depth-1), D-P10-2 (YAML contract + JSON Schema validation), D-P10-3 (single CONFIDENCE_COMPUTED event; GENESIS chain). REQ-28 binding claim: only adapters/ files contain aws_s3_bucket / aws_ Terraform-specific terms; modules-ir/ schemas/ contracts/ acdl_platform/ are substrate-agnostic. --- .ciagent/PLAN.md | 1553 +++------------------------------------------- 1 file changed, 88 insertions(+), 1465 deletions(-) diff --git a/.ciagent/PLAN.md b/.ciagent/PLAN.md index 5bda38e..f295ee2 100644 --- a/.ciagent/PLAN.md +++ b/.ciagent/PLAN.md @@ -1,1522 +1,145 @@ --- -phase: 9 -name: v1-spike-ir-and-l1-and-adapter +phase: 10 +name: v1-spike-l2-and-contract-e2e milestone: v1.1 milestone_type: feature status: planned -requirements: [REQ-24, REQ-26] +requirements: [REQ-25, REQ-27, REQ-28] must_haves: - # REQ-24 — IR-typed L1 interface + registry + doc - - "modules-ir/l1/l1-s3/interface.json exists and is a valid JSON *contract* (typed input/output declarations) with stack.name=l1-s3, stack.kind=l1, stack.depth=1, one resource id=s3, type=aws:s3:bucket, module=l1-s3@1.0.0, inputs={bucket_name:{type:string,...}, region:{type:string,...}}, outputs={bucket_arn:{type:arn,...}}. Per D-P09-2 this file is a CONTRACT (typed declarations), NOT an IR-schema instance — it does NOT validate against schemas/ir.schema.json (which is for stack instances with concrete values). A separate minimal stack instance (modules-ir/l1/l1-s3/spike_instance.json) with concrete values DOES validate against schemas/ir.schema.json." - - "modules-ir/registry.json exists and is valid JSON with exactly one entry: {\"l1-s3\":{\"1.0.0\":{\"interface\":\"modules-ir/l1/l1-s3/interface.json\",\"published_at\":\"\",\"deprecated\":false}}} (D-P09-3: registry lives under modules-ir/ for the spike, co-located with the L1 modules it catalogs per §12.7)." - - "modules-ir/l1/l1-s3/README.md exists and documents: type aws:s3:bucket (substrate-agnostic IR type — NOT aws_s3_bucket); inputs bucket_name:string + region:string; outputs bucket_arn:arn; the IR→Terraform mapping the adapter performs (type→aws_s3_bucket resource, inputs→resource args, outputs→output blocks); cites ARCHITECTURE.md §12.1/§12.2/§12.7 + the IR schema." - - "modules-ir/l1/l1-s3/spike_instance.json exists and is a minimal IR *stack instance* with CONCRETE values (bucket_name=acdl-spike-bucket, region=us-east-1) that VALIDATES against schemas/ir.schema.json (the IR schema is for instances, not contracts per D-P09-2)." - # REQ-26 — Terraform adapter (thin layer; l1-s3 only) - - "adapters/terraform/adapter.py exists, py_compiles, and exposes adapt(ir_instance, out_dir) + a __main__ CLI (adapter.py ). Reads an IR stack INSTANCE (concrete values) + emits main.tf (resource blocks), terraform.tf (required_version + required_providers + S3 backend, NO dynamodb_table per D-P09-1), providers.tf (aws provider, region from IR). For the spike: handles l1-s3 only — IR type aws:s3:bucket → resource \"aws_s3_bucket\" \"\" { bucket = }; output bucket_arn → output \"bucket_arn\" { value = aws_s3_bucket..arn }. The adapter does NOT own L1/L2 content (§12.2 — thin layer). Backend config: bucket=acdl-tfstate-581513795199-us-east-1, key=spike/l1-s3/terraform.tfstate, region=us-east-1, NO dynamodb_table (D-P09-1)." - # REQ-26 — generated spike Terraform config (committed; D-P09-4) - - "terraform/spike/main.tf exists and contains resource \"aws_s3_bucket\" \"s3\" { bucket = \"acdl-spike-bucket\" } + output \"bucket_arn\" { value = aws_s3_bucket.s3.arn } (generated by running adapter.py against modules-ir/l1/l1-s3/spike_instance.json; committed so verify_phase09.sh can terraform validate/plan without regenerating — D-P09-4)." - - "terraform/spike/terraform.tf exists and contains terraform { required_version = \">= 1.9, < 1.10\"; required_providers { aws = { source = \"hashicorp/aws\", version = \"~> 5.0\" } }; backend \"s3\" { bucket = \"acdl-tfstate-581513795199-us-east-1\", key = \"spike/l1-s3/terraform.tfstate\", region = \"us-east-1\" } } (NO dynamodb_table — D-P09-1)." - - "terraform/spike/providers.tf exists and contains provider \"aws\" { region = \"us-east-1\" }." - # REQ-26 — spike runner (plan-only, rotated key, no long-lived credential in workflow) - - "scripts/run_spike_plan.sh exists, bash -n passes, loads the rotated spike key from gitignored .env.secrets (NEVER committed), exports AWS_ACCESS_KEY_ID/SECRET_ACCESS_KEY/DEFAULT_REGION, cd terraform/spike, runs terraform init -lock=false, terraform validate, terraform plan -lock=false -out=tfplan (plan-only; no apply per Out of Scope + D-P09-1 no-lock), echoes 'spike plan OK'. Uses -lock=false because plan does not write state (D-P09-1) and the outbox table's PK contractId does not match Terraform's expected LockID." - # verify - - "scripts/verify_phase09.sh exists, bash -n passes, and asserts: (a) modules-ir/l1/l1-s3/interface.json exists + has the expected typed-input shape (NOT IR-schema validation per D-P09-2); (b) modules-ir/l1/l1-s3/spike_instance.json validates against schemas/ir.schema.json via python -m jsonschema; (c) modules-ir/registry.json has the l1-s3@1.0.0 entry; (d) adapters/terraform/adapter.py py_compiles + when run against spike_instance.json emits a main.tf containing 'resource \"aws_s3_bucket\"'; (e) terraform/spike/main.tf exists + is the committed generated file; (f) terraform validate passes in terraform/spike; (g) terraform plan -lock=false succeeds (real AWS, rotated key from .env.secrets); (h) no long-lived credential in any committed file (rg AKIA terraform/spike/ adapters/ modules-ir/ is empty); (i) scripts/run_spike_plan.sh exits 0; prints 'VERIFIED — Phase 09: IR + l1-s3 + Terraform adapter; terraform plan OK (real AWS, D-P09-1 no-lock)'." - # gitignore - - ".gitignore has entries for terraform/spike/.terraform/, terraform/spike/tfplan, terraform/spike/*.tfstate* (Terraform working artifacts — appended, not duplicated)." - # traceability - - "Traceability updated: REQUIREMENTS.md REQ-24 + REQ-26 → covered/complete; ROADMAP.md Phase 09 → complete; PROJECT.md records D-P09-1..D-P09-4." + - "modules-ir/l2/l2-static-asset/composition.json exists with kind=l2, depth=1, one child l1-s3@1.0.0, wires passthrough" + - "modules-ir/registry.json extended with l2-static-asset@1.0.0 entry" + - "modules-ir/l2/l2-static-asset/README.md documents the thin-composition" + - "contracts/spike.yaml exists (stack: l2-static-asset, environment: dev, inputs: bucket_name + region)" + - "acdl_platform/contract_resolver.py exists, py_compiles, loads YAML contract -> validates against contract.schema.json -> resolves L2 composition -> emits IR instance validating against ir.schema.json" + - "adapters/terraform/adapter.py extended to handle kind=l2 IR instances (D-P10-1: shallow L2 root module = the L1 resource)" + - "acdl_platform/outbox_writer.py exists, py_compiles, writes a DynamoDB outbox item (PK contractId, SK eventType#eventTs, prev_event_hash=GENESIS, hash=SHA-256 canonical JSON, expire_at TTL)" + - "scripts/run_spike_e2e.sh exists, bash -n passes, orchestrates the 10-step end-to-end pipeline" + - "scripts/verify_phase10.sh exists, bash -n passes, asserts all success criteria + REQ-28 (grep: only adapters/ files contain aws_s3_bucket; modules-ir/ schemas/ contracts/ acdl_platform/ are substrate-agnostic)" + - "Evidence event is written to the DynamoDB outbox (verified by querying the table)" + - "Confidence band is 'pass' for dev (score >= 0.50)" verification: - typecheck: "python3 -m py_compile adapters/terraform/adapter.py && bash -n scripts/run_spike_plan.sh scripts/verify_phase09.sh" - test: "scripts/verify_phase09.sh" + typecheck: "python3 -m py_compile acdl_platform/contract_resolver.py acdl_platform/outbox_writer.py adapters/terraform/adapter.py && bash -n scripts/run_spike_e2e.sh scripts/verify_phase10.sh" + test: "scripts/verify_phase10.sh" build: "terraform -chdir=terraform/spike init -lock=false" --- -# Phase 09 — v1-spike-ir-and-l1-and-adapter PLAN +# Phase 10 — v1-spike-l2-and-contract-e2e PLAN ## Goal -Materialize the **Target Stack IR**, one real L1 module (`l1-s3`) with an -IR-typed interface, the L1 registry, and the **Terraform adapter** — the -only substrate-specific code (§12.2) — that compiles the IR to a real -`terraform plan` against AWS using the **per-run-rotated spike key** -(waiver D-039; OIDC deferred to v1.2, blocked on go-gitea/gitea#36988). +The milestone capstone. Implement `l2-static-asset` (thin-composition +referencing `l1-s3` only, depth 1), the contract→IR resolution, and one +end-to-end contract submission flowing through: contract schema +validation → IR resolution → `terraform plan` (real AWS) → Checkov +`PolicyCheckResult` → confidence signal → evidence event to the DynamoDB +outbox. Verify the IR commitments hold (REQ-28: the adapter is the only +substrate-specific code; no polyglot mess). -This phase proves the binding constraint of ARCHITECTURE.md §12: the IR -is substrate-agnostic; the adapter is the only substrate-specific code; -the L1 content is IR-typed, not Terraform-typed. The deliverable is a -single `terraform plan` that reads real AWS state (the spike user has S3 -read on the state bucket + STS) and produces a valid plan for one -`aws_s3_bucket` resource — without a long-lived credential in any -committed file and without `terraform apply` (Out of Scope). - -**Key design frictions resolved during planning (D-P09-1..D-P09-4):** - -- **D-P09-1 — Terraform locking:** the Phase 08 DynamoDB table `acdl-outbox` - has PK `contractId` (D-044 + D-P08-1 consolidated table), but Terraform's - S3 backend expects a `LockID` partition key. The spike runs - `terraform plan -lock=false` + **omits `dynamodb_table`** from the S3 - backend config entirely: plan-only does not write state, so locking is - unnecessary; this avoids a root-key escalation + a second table. The - v1.2 real-apply path creates a proper `LockID`-keyed `acdl-tflock` table. -- **D-P09-2 — L1 interface vs IR schema:** the IR schema - (`schemas/ir.schema.json`) defines `resource.inputs` as - `additionalProperties: {type: [string, number, boolean]}` — i.e. - *concrete values*, not typed declarations. An L1 *interface* declares - input *types* (`bucket_name: {type: string, description: ...}`). The - resolution: the L1 interface file (`interface.json`) is a **contract** - (typed input/output declarations) that does **NOT** validate against - the IR schema; a separate minimal **stack instance** - (`spike_instance.json`) with concrete values validates against the IR - schema. The IR schema is for *instances*; the L1 interface is for - *contracts*. This cleanly separates declaration (L1 catalog) from - materialization (the adapter's input). -- **D-P09-3 — registry location:** the L1 registry lives at - `modules-ir/registry.json` (not `platform/registry/`) for the spike — - co-located with the L1 modules it catalogs (§12.7); v1.2 may move it. -- **D-P09-4 — generated TF committed:** the spike generates - `terraform/spike/*.tf` by running the adapter (Wave 3) and commits the - generated files so `verify_phase09.sh` can `terraform validate/plan` - without regenerating; the generation step is separately verified by - running the adapter in `verify_phase09.sh` + diffing. - -The deliverables, mapped to requirements + decisions: - -| REQ / decision | File | Owner persona (PERSONAS.md territory) | -|----------------|------|---------------------------------------| -| REQ-24 (L1 interface contract) | `modules-ir/l1/l1-s3/interface.json` (typed input/output declarations; NOT IR-schema validation per D-P09-2) | platform-engineer (owns `modules-ir/**`, the IR) | -| REQ-24 (L1 stack instance) | `modules-ir/l1/l1-s3/spike_instance.json` (concrete values; validates against `schemas/ir.schema.json`) | platform-engineer | -| REQ-24 (L1 registry) | `modules-ir/registry.json` (one entry: l1-s3@1.0.0; D-P09-3 location) | platform-engineer | -| REQ-24 (L1 doc) | `modules-ir/l1/l1-s3/README.md` (type, inputs, outputs, IR→TF mapping) | platform-engineer | -| REQ-26 (adapter) | `adapters/terraform/adapter.py` (IR instance → TF root module; thin layer; l1-s3 only) | platform-engineer (owns `adapters/terraform/**`) | -| REQ-26 (spike TF config) | `terraform/spike/{main.tf,terraform.tf,providers.tf}` (generated + committed; D-P09-4) | platform-engineer | -| REQ-26 (spike runner) | `scripts/run_spike_plan.sh` (rotated key from .env.secrets; plan -lock=false) | platform-engineer | -| (verify) | `scripts/verify_phase09.sh` (the gate) | lead-developer (owns `scripts/verify_phase*.sh`) | -| (gitignore) | `.gitignore` (add `terraform/spike/.terraform/` + `tfplan` + `*.tfstate*`) | lead-developer (owns `.gitignore`) | +After Phase 10 ships + verifies: the COMPLETE gate (review → ship v1.2.0 +→ audit). ## Requirements covered -- **REQ-24** — "One real L1 module `l1-s3` exists under `modules-ir/l1/l1-s3/` - with an IR-typed interface (typed inputs/outputs/NFRs) registered in the - L1 registry." - - **REQ-24a (interface contract)** → T-9.1 (`interface.json`). - - **REQ-24b (stack instance)** → T-9.2 (`spike_instance.json` — the - IR-schema-valid materialization). - - **REQ-24c (registry)** → T-9.3 (`registry.json` — the IR-typed - contract catalog per §12.7). - - **REQ-24d (doc)** → T-9.4 (`README.md`). -- **REQ-26** — "The Terraform adapter (`adapters/terraform/`) compiles the - IR-typed L1 interface to Terraform variable/output blocks and the L2 - thin-composition tree to a Terraform root module; it emits a real - `terraform plan` against AWS via OIDC; state is stored in S3 + - DynamoDB." - - **REQ-26a (adapter)** → T-9.5 (`adapter.py` — IR instance → TF root - module; thin layer per §12.2; l1-s3 only in the spike; L2 - thin-composition is Phase 10). - - **REQ-26b (generated TF)** → T-9.6 (`terraform/spike/*.tf` — - generated + committed per D-P09-4). - - **REQ-26c (spike runner)** → T-9.7 (`run_spike_plan.sh` — rotated - key, plan-only, -lock=false per D-P09-1). - - **REQ-26d (verify)** → T-9.8 (`verify_phase09.sh` — the gate). - - **REQ-26e (gitignore)** → T-9.9 (`.gitignore` — TF working artifacts). - -REQ-24 and REQ-26 are the only requirements assigned to Phase 09 per -REQUIREMENTS.md traceability. Phase 09 depends on Phase 08 (complete, -v1.1.3): the S3 state bucket (`acdl-tfstate-581513795199-us-east-1`) + -DynamoDB outbox table (`acdl-outbox`) + rotated spike key in -gitignored `.env.secrets` all exist and are verified. +- **REQ-25** → T-10.1 (composition.json), T-10.2 (registry extend), T-10.3 (README) +- **REQ-27** → T-10.4 (spike.yaml), T-10.5 (contract_resolver.py), T-10.6 (adapter L2 extension), T-10.7 (outbox_writer.py), T-10.8 (run_spike_e2e.sh) +- **REQ-28** → T-10.9 (verify_phase10.sh with the substrate-agnostic grep check) ## Waves -Domain priority from PERSONAS.md: `coordination → security → platform → -backend → frontend`. Phase 09 is **platform-engineer-heavy** (per -PERSONAS.md "Phase-specific overrides" Phase 09 row: "platform-engineer -(lead), backend-engineer (IR schema co-author) — but the schema is -already authored in Phase 07; backend-engineer owns contract→IR -resolution which is Phase 10, NOT Phase 09"). security/frontend idle. - -Dependency order (platform layer materialization): -- The **L1 interface + registry + doc** (Wave 1) define the IR-typed - contract that the adapter reads. -- The **adapter** (Wave 2) reads an IR stack *instance* (Wave 1's - `spike_instance.json`) + emits Terraform → depends on Wave 1's shape. -- The **generated spike TF config + runner** (Wave 3) are produced by - running the adapter (Wave 2) against the Wave 1 instance. -- The **verify script** (Wave 4) asserts all prior + runs the runner. -- **Wave 5 is EXECUTE-only** — runs `run_spike_plan.sh` (real - `terraform plan` against AWS) + `verify_phase09.sh`. This is the only - wave that touches real AWS with the spike key. - -Five waves, strictly ordered: - ``` -Wave 1 (platform-engineer) — T-9.1 interface.json (L1 contract) - T-9.2 spike_instance.json (IR-schema instance) - T-9.3 registry.json (L1 catalog) - T-9.4 README.md (L1 doc) -Wave 2 (platform-engineer) — T-9.5 adapter.py (IR instance → TF; reads Wave 1 shape) -Wave 3 (platform-engineer) — T-9.6 terraform/spike/*.tf (generated from Wave 2 + Wave 1) - T-9.7 run_spike_plan.sh (runner) - T-9.9 .gitignore (TF artifacts) -Wave 4 (lead-developer) — T-9.8 verify_phase09.sh (asserts all prior) -Wave 5 (lead-developer, EXECUTE) — run run_spike_plan.sh (real terraform plan) - → verify_phase09.sh - → traceability update +Wave 1 (platform-engineer) — T-10.1 composition.json, T-10.2 registry, T-10.3 README, T-10.6 adapter L2 extension +Wave 2 (backend-engineer) — T-10.4 spike.yaml, T-10.5 contract_resolver.py, T-10.7 outbox_writer.py +Wave 3 (backend+security) — T-10.8 run_spike_e2e.sh +Wave 4 (lead-developer) — T-10.9 verify_phase10.sh +Wave 5 (lead, EXECUTE-only) — run e2e + verify + traceability + ship ``` -**Dependency graph (explicit):** -- Wave 2 → Wave 1 (the adapter reads an IR stack instance; its input - shape — `stack`, `resources[].type=aws:s3:bucket`, - `resources[].inputs.{bucket_name,region}` — is defined by Wave 1's - `spike_instance.json`; the adapter's IR-type→TF-type mapping - (`aws:s3:bucket` → `aws_s3_bucket`) is driven by the L1 interface's - `type` field from Wave 1's `interface.json`). -- Wave 3 → Wave 2 (the generated TF is produced by running the adapter; - the runner script references the generated files). -- Wave 4 → all prior (`verify_phase09.sh` asserts the interface, the - instance, the registry, the adapter, the generated TF, the runner, + - runs `terraform validate/plan`). -- Wave 5 → Wave 4 (EXECUTE-only; runs the runner + verify script against - real AWS; the verify script is the last file Wave 5 invokes). - -**Parallelizable:** Wave 1's four files (T-9.1..T-9.4) are independent -files owned by the same persona — can be authored in parallel. Wave 3's -three files (T-9.6 generated TF, T-9.7 runner, T-9.9 gitignore) are -independent files owned by the same persona — can be authored in -parallel (T-9.6 requires running the adapter, which is a Wave 2 -deliverable; T-9.7 and T-9.9 only reference the generated path). All -other waves are sequential within the wave. +Dependencies: Wave 2 → Wave 1 (resolver reads composition.json); Wave 3 → Wave 1+2; Wave 4 → all prior; Wave 5 EXECUTE-only. --- -### Wave 1 — platform-engineer: REQ-24 L1 interface + registry + doc +### Wave 1 — platform-engineer: L2 composition + registry + adapter extension -Wave 1 defines the IR-typed L1 contract (the substrate-agnostic -declaration) + the IR-schema-valid stack instance (the materialization -the adapter consumes) + the registry entry (§12.7) + the L1 doc. This is -the substrate-agnostic layer — no Terraform keywords here (the IR schema -is substrate-agnostic per Phase 07; the L1 interface must not introduce -Terraform block keywords). +#### T-10.1 — Author `modules-ir/l2/l2-static-asset/composition.json` +- **Owner:** platform-engineer +- **Content:** `{name: l2-static-asset, version: 1.0.0, kind: l2, depth: 1, children: [{id: s3, module: l1-s3@1.0.0}], wires: {bucket_name: {target: s3, input: bucket_name}, region: {target: s3, input: region}}}` -#### T-9.1 — Author `modules-ir/l1/l1-s3/interface.json` +#### T-10.2 — Extend `modules-ir/registry.json` with l2-static-asset@1.0.0 +- **Owner:** platform-engineer +- **Content:** add `"l2-static-asset": {"1.0.0": {"composition": "modules-ir/l2/l2-static-asset/composition.json", "published_at": "", "deprecated": false}}` -- **Owner:** platform-engineer (territory: `modules-ir/**` per - PERSONAS.md). -- **Files owned:** `modules-ir/l1/l1-s3/interface.json` (new); create - `modules-ir/l1/l1-s3/` directory + a `modules-ir/l1/l1-s3/.gitkeep` if - needed for the dir to track cleanly. -- **What the file must contain** (cite sources): - 1. **JSON L1 interface contract** (NOT an IR-schema instance — D-P09-2). - Shape: - ```json - { - "name": "l1-s3", - "kind": "l1", - "version": "1.0.0", - "description": "S3 bucket primitive (substrate-agnostic IR-typed L1).", - "type": "aws:s3:bucket", - "module": "l1-s3@1.0.0", - "inputs": { - "bucket_name": { - "type": "string", - "description": "Globally-unique S3 bucket name.", - "required": true - }, - "region": { - "type": "string", - "description": "AWS region for the bucket.", - "required": true - } - }, - "outputs": { - "bucket_arn": { - "type": "arn", - "description": "The S3 bucket ARN." - } - }, - "nfrs": {} - } - ``` - 2. **The `type` is the IR type** `aws:s3:bucket` (substrate-agnostic, - colon-separated), NOT the Terraform resource type `aws_s3_bucket` - (the adapter translates IR type → TF type per §12.1/§12.2). This is - the binding constraint: the L1 interface is substrate-agnostic. - 3. **The `module` is the registry reference** `l1-s3@1.0.0` - (name@semver per W3.D + the IR schema's `resource.module` pattern - `^l1-[a-z][a-z0-9-]*@\d+\.\d+\.\d+$`). - 4. **Inputs are *typed declarations*** (`{type, description, required}`), - NOT concrete values. This is the friction D-P09-2 resolves: the IR - schema's `resource.inputs` is `additionalProperties: - {type:[string,number,boolean]}` (concrete values); the L1 - *interface* declares types, the L2 *contract* / stack *instance* - provides values. The interface file therefore does NOT validate - against `schemas/ir.schema.json`; it has its own shape (the - contract shape above). - 5. **Outputs are *typed declarations*** (`{type, description}`) per - the IR schema's `outputSpec` (`type` required, `description` - optional). The `type: arn` is an IR output type (the adapter - translates to a Terraform output value reference). - 6. **NFRs** is an empty object for the spike (`l1-s3` declares no - NFRs; the confidence signal's NFR input is "present + neutral 0.5" - per RESEARCH TARGET 6 A-6.2). - 7. **NO Terraform block keywords** (`variable`, `output`, `resource`, - `aws_s3_bucket`, `provider`) in this file — the IR is - substrate-agnostic (Phase 07 IR schema `$comment`; §12.1). The - `type: aws:s3:bucket` is an IR type, not a Terraform type. -- **Source citations:** - - `schemas/ir.schema.json` (Phase 07 — the IR schema; `resource.type` - is "IR-typed resource identifier (substrate-agnostic), e.g. - 'aws:s3:bucket'. NOT a Terraform resource type ('aws_s3_bucket'); - the adapter translates IR type -> substrate type"; `resource.module` - pattern `^l1-[a-z][a-z0-9-]*@\d+\.\d+\.\d+$`; `outputSpec.type`). - - `ARCHITECTURE.md` §12.1 (IR: resources with typed inputs/outputs/ - NFRs, single parent per child, composition tree max-depth-5, - substrate-agnostic). - - `ARCHITECTURE.md` §12.7 (registry maintenance: L1 publication - updates the registry in the same PR; IR-typed contract). - - `PROJECT.md` operational parameters (spike L1 inputs: bucket_name - string, region string; depth 1). - - `PROJECT.md` D-036 (spike picks l1-s3 + l2-static-asset). - - `RESEARCH.md` TARGET 3 (v1.0 demo L1 manifest shape — the v1.1 L1 - interface upgrades the demo's `manifest.yaml` to a typed JSON - contract). -- **Commit message:** - `phase: 9, status: plan-as-execute, persona: platform-engineer, task: T-9.1` - (then `feat(P09): author l1-s3 interface.json (REQ-24a; IR-typed contract; D-P09-2)`) -- **Self-verify:** - - [ ] `python3 -c "import json; json.load(open('modules-ir/l1/l1-s3/interface.json'))"` - exits 0 (valid JSON). - - [ ] `name == "l1-s3"`, `kind == "l1"`, `version == "1.0.0"`. - - [ ] `type == "aws:s3:bucket"` (IR type, NOT `aws_s3_bucket`). - - [ ] `module == "l1-s3@1.0.0"` (matches the registry pattern). - - [ ] `inputs` has `bucket_name` + `region`, each with `type: string`. - - [ ] `outputs.bucket_arn.type == "arn"`. - - [ ] NO occurrence of `aws_s3_bucket`, `variable`, `output "`, `resource "`, - `provider` in the file (substrate-agnostic — grep should be empty). - - [ ] (D-P09-2) The file does NOT validate against - `schemas/ir.schema.json` (it's a contract, not an instance — - `inputs` are typed declarations, not concrete values). +#### T-10.3 — Author `modules-ir/l2/l2-static-asset/README.md` +- **Owner:** platform-engineer +- **Content:** thin-composition doc (references l1-s3 only, depth 1, wires passthrough, the adapter's L2→root-module translation, D-P10-1) -#### T-9.2 — Author `modules-ir/l1/l1-s3/spike_instance.json` - -- **Owner:** platform-engineer (territory: `modules-ir/**`). -- **Files owned:** `modules-ir/l1/l1-s3/spike_instance.json` (new). -- **What the file must contain** (cite sources): - 1. **A minimal IR *stack instance* with CONCRETE values** that - validates against `schemas/ir.schema.json` (D-P09-2: the IR schema - is for instances). Shape: - ```json - { - "version": "1.0.0", - "stack": { - "name": "l1-s3", - "kind": "l1", - "depth": 1 - }, - "resources": [ - { - "id": "s3", - "type": "aws:s3:bucket", - "module": "l1-s3@1.0.0", - "inputs": { - "bucket_name": "acdl-spike-bucket", - "region": "us-east-1" - }, - "outputs": { - "bucket_arn": { - "type": "arn", - "description": "The S3 bucket ARN." - } - } - } - ] - } - ``` - 2. **`version`** = `"1.0.0"` (matches the IR schema's pattern - `^\d+\.\d+\.\d+$`). - 3. **`stack`** = `{name: "l1-s3", kind: "l1", depth: 1}` (matches the - schema's `stack.name` pattern `^l[12]-[a-z][a-z0-9-]*$`, `kind` enum - `[l1, l2]`, `depth` int 1-5; depth 1 per the operational parameter - "Spike L2 l2-static-asset thin-composition referencing l1-s3 only; - depth 1" — the L1 instance itself is depth 1). - 4. **`resources[0]`** = the single S3 resource. `id: "s3"` (matches - `^[a-z][a-z0-9-]*$`); `type: "aws:s3:bucket"` (IR type); `module: - "l1-s3@1.0.0"` (registry ref); `inputs: {bucket_name: - "acdl-spike-bucket", region: "us-east-1"}` (**concrete string - values** — the IR schema's `resource.inputs.additionalProperties` - is `{type: [string, number, boolean]}`); `outputs: {bucket_arn: - {type: "arn", description: "..."}}` (matches `outputSpec`). - 5. **No `parent`** (the root resource has no parent per the schema). - 6. **No `relationships`** (the spike's L1 has a single resource; the - L2 thin-composition in Phase 10 will add the parent relationship). - 7. **The concrete `bucket_name` value** `"acdl-spike-bucket"` matches - the operational parameter "Spike contract inputs: - bucket_name=acdl-spike-bucket" (PROJECT.md operational parameters). -- **Source citations:** - - `schemas/ir.schema.json` (the schema this instance validates against). - - `PROJECT.md` operational parameters (spike L1 inputs bucket_name + - region; spike contract inputs bucket_name=acdl-spike-bucket, - region=us-east-1). - - `RESEARCH.md` TARGET 3 (the IR instance shape; the v1 IR round-trips - to Terraform cleanly — `resource.inputs` → Terraform args). -- **Commit message:** - `phase: 9, status: plan-as-execute, persona: platform-engineer, task: T-9.2` - (then `feat(P09): author l1-s3 spike_instance.json (REQ-24b; IR-schema-valid instance)`) -- **Self-verify:** - - [ ] `python3 -c "import json; json.load(open('modules-ir/l1/l1-s3/spike_instance.json'))"` - exits 0. - - [ ] `python3 -m jsonschema -i modules-ir/l1/l1-s3/spike_instance.json schemas/ir.schema.json` - exits 0 (validates against the IR schema — D-P09-2). - - [ ] `resources[0].inputs.bucket_name == "acdl-spike-bucket"` (concrete value). - - [ ] `resources[0].inputs.region == "us-east-1"` (concrete value). - - [ ] `resources[0].type == "aws:s3:bucket"` (IR type, not TF type). - - [ ] `stack.depth == 1`. - -#### T-9.3 — Author `modules-ir/registry.json` - -- **Owner:** platform-engineer (territory: `modules-ir/**` — D-P09-3: - the registry lives under `modules-ir/` for the spike, co-located with - the L1 modules it catalogs per §12.7; v1.2 may move it to - `platform/registry/`). -- **Files owned:** `modules-ir/registry.json` (new); remove the empty - `modules-ir/.gitkeep` if the registry file makes it redundant (leave - it if other files still need the dir — the dir now has - `l1/l1-s3/interface.json` etc., so the .gitkeep is no longer needed; - delete it to keep the tree clean). -- **What the file must contain** (cite sources): - 1. **JSON L1 registry** — one entry for the spike: - ```json - { - "l1-s3": { - "1.0.0": { - "interface": "modules-ir/l1/l1-s3/interface.json", - "published_at": "", - "deprecated": false - } - } - } - ``` - 2. **Shape:** keyed by L1 name → keyed by semver → `{interface, - published_at, deprecated}`. The `interface` is a relative path to - the L1 interface contract (T-9.1). `published_at` is ISO-8601 UTC - (e.g. `2026-07-21T12:00:00Z`). `deprecated: false` (the spike - publishes a fresh entry; W3.D deprecation window starts on MAJOR - bump). - 3. **The registry is the IR-typed contract catalog** (§12.7: "the - registry is the IR-typed contract, not a Terraform-specific - variable schema"). It references the interface file, which is - substrate-agnostic. - 4. **Exactly one entry** for the spike (REQ-24: "one real L1 module - `l1-s3`"). v1.2 will add the full L1 catalog. -- **Source citations:** - - `ARCHITECTURE.md` §12.7 (registry maintenance: L1 publication - updates the registry in the same PR; IR-typed contract). - - `ARCHITECTURE.md` §12.1 (the IR is substrate-agnostic; the registry - references IR-typed interfaces). - - `PROJECT.md` D-036 (spike picks l1-s3). -- **Commit message:** - `phase: 9, status: plan-as-execute, persona: platform-engineer, task: T-9.3` - (then `feat(P09): author modules-ir/registry.json (REQ-24c; L1 catalog; D-P09-3)`) -- **Self-verify:** - - [ ] `python3 -c "import json; json.load(open('modules-ir/registry.json'))"` - exits 0. - - [ ] Exactly one top-level key: `l1-s3`. - - [ ] `registry["l1-s3"]["1.0.0"]["interface"] == - "modules-ir/l1/l1-s3/interface.json"`. - - [ ] `registry["l1-s3"]["1.0.0"]["deprecated"] == false`. - - [ ] `published_at` parses as ISO-8601 (e.g. - `datetime.fromisoformat(...)`). - -#### T-9.4 — Author `modules-ir/l1/l1-s3/README.md` - -- **Owner:** platform-engineer (territory: `modules-ir/**`; this is a - module-level doc, NOT a meta doc — lead-developer owns `docs/**` meta - only, not module docs). -- **Files owned:** `modules-ir/l1/l1-s3/README.md` (new). -- **What the file must contain** (cite sources): - 1. **Title + purpose** — "# l1-s3 — S3 bucket primitive (IR-typed L1)". - A short doc describing the L1: it is substrate-agnostic, IR-typed; - the Terraform adapter (`adapters/terraform/adapter.py`) translates - it to a Terraform root module. - 2. **IR type** — `aws:s3:bucket` (substrate-agnostic; NOT - `aws_s3_bucket` — the adapter translates). - 3. **Inputs** — `bucket_name: string` (globally-unique S3 bucket name), - `region: string` (AWS region). Both required. - 4. **Outputs** — `bucket_arn: arn` (the S3 bucket ARN). - 5. **IR → Terraform mapping** (the adapter performs this; cite §12.2): - - IR `resource.type = aws:s3:bucket` → Terraform - `resource "aws_s3_bucket" "" { ... }`. - - IR `resource.inputs.bucket_name` → Terraform `bucket = ` - arg. - - IR `resource.inputs.region` → Terraform provider `region = `. - - IR `resource.outputs.bucket_arn` → Terraform - `output "bucket_arn" { value = aws_s3_bucket..arn }`. - 6. **Files in this directory** — a table: - - `interface.json` — the typed L1 interface contract (T-9.1). - - `spike_instance.json` — the IR-schema-valid stack instance with - concrete values (T-9.2; the adapter's input for the spike). - - `README.md` — this doc. - 7. **Spike scope vs v1.2 boundary** — short note: the spike handles - `l1-s3` only; v1.2 adds the full L1 catalog; L2 thin-composition - (`l2-static-asset` referencing `l1-s3`) is Phase 10. - 8. **Registry entry** — `modules-ir/registry.json` records - `l1-s3@1.0.0` → this interface (§12.7). -- **Source citations:** - - `ARCHITECTURE.md` §12.1 (IR substrate-agnostic) + §12.2 (adapter - translates IR-typed L1 interface → TF variable/output blocks) + - §12.7 (registry is IR-typed contract). - - `PROJECT.md` operational parameters (spike L1 inputs). - - `schemas/ir.schema.json` (the IR schema the instance validates - against). -- **Commit message:** - `phase: 9, status: plan-as-execute, persona: platform-engineer, task: T-9.4` - (then `docs(P09): author l1-s3 README.md (REQ-24d; IR→TF mapping)`) -- **Self-verify:** - - [ ] `modules-ir/l1/l1-s3/README.md` exists, non-empty. - - [ ] Mentions `aws:s3:bucket` (IR type) + `aws_s3_bucket` (TF type, - only in the mapping section). - - [ ] The IR→TF mapping table is present. - - [ ] Cites ARCHITECTURE.md §12.1/§12.2/§12.7. +#### T-10.6 — Extend `adapters/terraform/adapter.py` for kind=l2 +- **Owner:** platform-engineer +- **Content:** D-P10-1: the adapter consumes the *resolved IR instance* (which has kind=l2 + the L1 resources as its resources array). For a depth-1 thin-composition, the L2 root module IS the L1's resource — no separate module block. The existing `adapt()` + TYPE_MAP + resource emission handle both l1 and l2 instances (the resources array is the same shape). The only L2-specific bit: the `relationships` array is ignored at TF level for the spike. Update the backend key to `spike/l2-static-asset/terraform.tfstate` for the L2 spike. --- -### Wave 2 — platform-engineer: REQ-26 Terraform adapter +### Wave 2 — backend-engineer: contract + resolver + outbox writer -Wave 2 authors the adapter — the only substrate-specific code (§12.2). -It reads an IR stack *instance* (Wave 1's `spike_instance.json` with -concrete values) + emits a Terraform root module. The adapter is a THIN -LAYER; it does not own L1/L2 content. For the spike it handles `l1-s3` -only (IR type `aws:s3:bucket` → `aws_s3_bucket`); L2 thin-composition is -Phase 10. +#### T-10.4 — Author `contracts/spike.yaml` +- **Owner:** backend-engineer +- **Content:** `stack: l2-static-asset\nenvironment: dev\ninputs:\n bucket_name: acdl-spike-bucket\n region: us-east-1\n` -#### T-9.5 — Author `adapters/terraform/adapter.py` +#### T-10.5 — Author `acdl_platform/contract_resolver.py` +- **Owner:** backend-engineer +- **Content:** `resolve(contract_path) -> ir_instance` dict. Steps: (1) load YAML; (2) validate against contract.schema.json; (3) look up the L2 in registry.json; (4) load composition.json; (5) map contract inputs through wires to child L1 inputs; (6) emit IR instance; (7) validate IR against ir.schema.json. CLI: `contract_resolver.py `. stdlib + jsonschema + yaml only. -- **Owner:** platform-engineer (territory: `adapters/terraform/**` per - PERSONAS.md — owns the Terraform adapter; the binding constraint per - §12: the adapter is the only substrate-specific code). NOTE: - `adapters/terraform/policy/checkov_adapter.py` already exists from - Phase 07 (owned by security-engineer) — this adapter is a separate - file at `adapters/terraform/adapter.py` (the substrate adapter, NOT - the policy adapter). -- **Files owned:** `adapters/terraform/adapter.py` (new); create - `adapters/terraform/__init__.py` (empty) if it doesn't exist (so the - dir is a package; check first — `adapters/terraform/policy/` has an - `__init__.py` but `adapters/terraform/` itself may not). -- **What the file must contain** (cite sources): - 1. **Module docstring** — "ACDL Terraform adapter (REQ-26; §12.2 — - the only substrate-specific code). Translates an IR stack - *instance* (validated against schemas/ir.schema.json; concrete - values, NOT the L1 interface contract per D-P09-2) to a Terraform - root module: main.tf (resources), terraform.tf (required_version + - required_providers + S3 backend, NO dynamodb_table per D-P09-1), - providers.tf (aws provider, region from the IR). For the spike: - handles l1-s3 only (IR type aws:s3:bucket → aws_s3_bucket resource; - inputs → resource args; outputs → output blocks). Thin layer: does - NOT own L1/L2 content. L2 thin-composition is Phase 10." - 2. **Imports** — `import json, os, sys, datetime`. No third-party deps - (pure stdlib; the adapter emits HCL text, no HCL parser needed for - the spike's single resource). - 3. **Constants** — - - `IR_TYPE_TO_TF = {"aws:s3:bucket": "aws_s3_bucket"}` (IR type → - Terraform resource type map; spike has one entry; v1.2 extends). - - `STATE_BUCKET = "acdl-tfstate-581513795199-us-east-1"` (Phase 08 - bootstrap state bucket; concrete — the adapter emits the backend - config with this bucket). - - `STATE_KEY = "spike/l1-s3/terraform.tfstate"` (spike state key). - - `REGION = "us-east-1"` (operational parameter; the default - region; the adapter reads `inputs.region` from the IR resource - if present, else falls back to this). - 4. **`adapt(ir_instance, out_dir)`** — the main function. Takes an IR - instance dict (or a path to a JSON file — accept both: if `str`, - `json.load(open(...)); if `dict`, use as-is) + an output directory - path. Creates the dir if it doesn't exist. Emits three files: - - `main.tf` — the resources + outputs. - - `terraform.tf` — the `terraform {}` block (required_version + - required_providers + S3 backend, NO dynamodb_table). - - `providers.tf` — the `provider "aws" {}` block. - 5. **`_emit_main_tf(ir)`** — generate HCL for the resources + outputs. - For the spike (one `aws:s3:bucket` resource): - - Iterate `ir["resources"]`. For each resource: - - `tf_type = IR_TYPE_TO_TF[resource["type"]]` (e.g. - `aws_s3_bucket`). If the type is not in the map, raise a clear - `ValueError("unsupported IR type: ; spike handles - aws:s3:bucket only")`. - - `rid = resource["id"]` (e.g. `s3`). - - `inputs = resource["inputs"]` (concrete values, e.g. - `{bucket_name: "acdl-spike-bucket", region: "us-east-1"}`). - - Emit a `resource "" "" { ... }` block. For - `aws_s3_bucket`: `bucket = ` (string → - HCL quoted; numbers/booleans → HCL literal). Skip `region` - from the resource body — `region` goes to the provider - (T-9.5 step 7). (For other inputs, map by name → HCL arg; the - spike only has `bucket_name` + `region`, so `bucket_name` → - `bucket` is the only resource arg.) - - Emit an `output "" { value = .. }` - block for each output. For `bucket_arn`: emit - `output "bucket_arn" { value = aws_s3_bucket.s3.arn }`. The - adapter knows `bucket_arn` → `.arn` attribute (a small - `OUTPUT_ATTR_MAP = {"bucket_arn": "arn"}` for the spike; v1.2 - generalizes). - - Return the HCL string. Indent with 2 spaces. Quote string values - with double quotes. - 6. **`_emit_terraform_tf(ir)`** — generate the `terraform {}` block: - ```hcl - terraform { - required_version = ">= 1.9, < 1.10" - required_providers { - aws = { - source = "hashicorp/aws" - version = "~> 5.0" - } - } - backend "s3" { - bucket = "acdl-tfstate-581513795199-us-east-1" - key = "spike/l1-s3/terraform.tfstate" - region = "us-east-1" - } - } - ``` - **NO `dynamodb_table`** in the backend block (D-P09-1: the outbox - table's PK `contractId` does not match Terraform's expected - `LockID`; the spike runs `terraform plan -lock=false`, so no - locking; v1.2 creates a proper `LockID`-keyed `acdl-tflock` table). - The `required_version = ">= 1.9, < 1.10"` matches D-045 (terraform - pinned 1.9.*). The `aws` provider `~> 5.0` is a stable major. - 7. **`_emit_providers_tf(ir)`** — generate the `provider "aws" {}` - block. Read the region from the IR: find the first resource with - `inputs.region`; if present, use it; else fall back to `REGION` - constant. Emit: - ```hcl - provider "aws" { - region = "us-east-1" - } - ``` - (The spike's `spike_instance.json` has `inputs.region = "us-east-1"`, - so this reads it; if absent, the constant is used.) - 8. **`_hcl_value(v)`** — helper: convert a Python value to an HCL - literal. `str` → `""` (double-quoted, escape `"` + `\`); - `bool` → `true`/`false`; `int`/`float` → `str(v)`. The spike only - uses strings, but be defensive. - 9. **`__main__` CLI** — `if __name__ == "__main__":` parse argv: - `adapter.py `. Call `adapt(ir_path, out_dir)`. - Print "adapter: emitted /main.tf, terraform.tf, - providers.tf" on success. Exit 0. Exit 1 with a clear message on - error (missing args, file not found, unsupported IR type). - 10. **Thin layer — no L1/L2 content ownership:** the adapter does NOT - define L1 inputs/outputs (those live in `interface.json`); it does - NOT define L2 composition (Phase 10); it only *translates* an IR - instance to HCL. The IR-type→TF-type map (`IR_TYPE_TO_TF`) is the - only substrate-specific knowledge; it is the binding constraint - (§12.2: "the adapter is a thin layer; it does not own L1/L2 - content"). - 11. **No secrets in the adapter** — the adapter does NOT read - `.env.secrets` or set AWS env vars (that's `run_spike_plan.sh`'s - job). It only emits HCL text. The backend config has the state - bucket name (non-secret) but NOT any credentials. -- **Source citations:** - - `ARCHITECTURE.md` §12.2 (Terraform adapter: translates IR-typed L1 - interface → TF variable/output blocks; IR-typed L2 thin-composition - tree → TF root module; IR-typed relationships → module references; - emits terraform plan from IR; thin layer; does not own L1/L2 - content). - - `ARCHITECTURE.md` §12.1 (IR substrate-agnostic; nearly isomorphic to - TF in v1). - - `PROJECT.md` D-045 (terraform pinned 1.9.* → `required_version`). - - `PROJECT.md` D-039 (rotated key; OIDC deferred; the adapter emits - static-cred-compatible TF — no OIDC provider block). - - `PROJECT.md` D-P08-1 + the orchestrator's environment facts (the - outbox table PK is `contractId`, NOT `LockID` → D-P09-1: omit - `dynamodb_table`). - - `schemas/ir.schema.json` (the IR instance shape the adapter reads). - - `RESEARCH.md` TARGET 3 (the IR→TF round-trip mapping: `resource.module` - → `module` block; `resource.inputs` → args; `resource.outputs` → - outputs; `relationship.kind=uses_output` → interpolation; for the - spike's single-resource L1 there are zero relationships). -- **Commit message:** - `phase: 9, status: plan-as-execute, persona: platform-engineer, task: T-9.5` - (then `feat(P09): author adapters/terraform/adapter.py (REQ-26a; IR→TF; thin layer; D-P09-1)`) -- **Self-verify:** - - [ ] `python3 -m py_compile adapters/terraform/adapter.py` exits 0. - - [ ] No third-party imports (stdlib only). - - [ ] `adapt(ir_instance, out_dir)` is the public function. - - [ ] `__main__` CLI: `adapter.py `. - - [ ] `IR_TYPE_TO_TF["aws:s3:bucket"] == "aws_s3_bucket"`. - - [ ] Running `python3 adapters/terraform/adapter.py - modules-ir/l1/l1-s3/spike_instance.json /tmp/adapter_test` - produces `/tmp/adapter_test/main.tf` containing - `resource "aws_s3_bucket" "s3"` + `bucket = "acdl-spike-bucket"` - + `output "bucket_arn" { value = aws_s3_bucket.s3.arn }`. - - [ ] `/tmp/adapter_test/terraform.tf` contains - `backend "s3"` with `bucket = "acdl-tfstate-581513795199-us-east-1"` - + `key = "spike/l1-s3/terraform.tfstate"` + `region = "us-east-1"` - + NO `dynamodb_table` (D-P09-1). - - [ ] `/tmp/adapter_test/providers.tf` contains - `provider "aws" { region = "us-east-1" }`. - - [ ] `required_version = ">= 1.9, < 1.10"` (D-045 pin). - - [ ] The adapter does NOT read `.env.secrets` or set AWS env vars - (grep for `env` / `secrets` / `AWS_` should be empty). +#### T-10.7 — Author `acdl_platform/outbox_writer.py` +- **Owner:** backend-engineer +- **Content:** `write_event(event, outbox_table="acdl-outbox")` — compute SHA-256 over canonical JSON, set prev_event_hash="GENESIS" for the first event, build DynamoDB item, boto3 put_item. CLI: `outbox_writer.py `. stdlib + boto3. --- -### Wave 3 — platform-engineer: REQ-26 generated spike TF + runner + gitignore +### Wave 3 — backend+security+platform: e2e runner -Wave 3 runs the adapter (Wave 2) against the IR instance (Wave 1) to -generate the spike's Terraform root module, commits the generated files -(D-P09-4), and authors the spike runner + gitignore edits. - -#### T-9.6 — Generate `terraform/spike/{main.tf,terraform.tf,providers.tf}` - -- **Owner:** platform-engineer (territory: `terraform/**` per - PERSONAS.md + the generated output of the adapter owned by - platform-engineer). -- **Files owned:** `terraform/spike/main.tf` (new); - `terraform/spike/terraform.tf` (new); `terraform/spike/providers.tf` - (new); create `terraform/spike/` directory. -- **What the files must contain** (D-P09-4: generated by running the - adapter + committed so verify_phase09.sh can validate/plan without - regenerating): - 1. **Generation command** (run by the executor in Wave 3): - ```bash - python3 adapters/terraform/adapter.py modules-ir/l1/l1-s3/spike_instance.json terraform/spike - ``` - This emits the three files into `terraform/spike/`. - 2. **`terraform/spike/main.tf`** — must contain (exact, after - generation): - ```hcl - resource "aws_s3_bucket" "s3" { - bucket = "acdl-spike-bucket" - } - - output "bucket_arn" { - value = aws_s3_bucket.s3.arn - } - ``` - (`bucket = "acdl-spike-bucket"` comes from the IR instance's - `inputs.bucket_name`; `region` is NOT in the resource body — it - goes to the provider.) - 3. **`terraform/spike/terraform.tf`** — must contain: - ```hcl - terraform { - required_version = ">= 1.9, < 1.10" - required_providers { - aws = { - source = "hashicorp/aws" - version = "~> 5.0" - } - } - backend "s3" { - bucket = "acdl-tfstate-581513795199-us-east-1" - key = "spike/l1-s3/terraform.tfstate" - region = "us-east-1" - } - } - ``` - **NO `dynamodb_table`** (D-P09-1). - 4. **`terraform/spike/providers.tf`** — must contain: - ```hcl - provider "aws" { - region = "us-east-1" - } - ``` - 5. **Commit the generated files** (D-P09-4): the executor runs the - adapter, verifies the output matches the expected shape above, - then commits the three files. `verify_phase09.sh` (Wave 4) will - run the adapter again into a temp dir + `diff` against the - committed files to prove the generation step is reproducible. - 6. **No credentials in the generated TF** — the backend config has - the state bucket name (non-secret) but NO access keys. The spike - runner (`run_spike_plan.sh`) loads credentials from `.env.secrets` - at runtime. -- **Source citations:** - - `adapters/terraform/adapter.py` T-9.5 (the generator). - - `modules-ir/l1/l1-s3/spike_instance.json` T-9.2 (the IR instance). - - `PROJECT.md` operational parameters (spike contract inputs - bucket_name=acdl-spike-bucket, region=us-east-1; state bucket - acdl-tfstate-581513795199-us-east-1). - - `PROJECT.md` D-P09-1 (no dynamodb_table; -lock=false). - - `PROJECT.md` D-045 (terraform 1.9.* pin → required_version). -- **Commit message:** - `phase: 9, status: plan-as-execute, persona: platform-engineer, task: T-9.6` - (then `feat(P09): generate terraform/spike/*.tf (REQ-26b; adapter output; D-P09-4)`) -- **Self-verify:** - - [ ] `terraform/spike/main.tf` exists + contains - `resource "aws_s3_bucket" "s3"` + `bucket = "acdl-spike-bucket"` - + `output "bucket_arn"`. - - [ ] `terraform/spike/terraform.tf` exists + contains - `backend "s3"` + `bucket = "acdl-tfstate-581513795199-us-east-1"` - + NO `dynamodb_table`. - - [ ] `terraform/spike/providers.tf` exists + contains - `provider "aws" { region = "us-east-1" }`. - - [ ] `rg AKIA terraform/spike/` is empty (no credentials). - - [ ] Regeneration check: `python3 adapters/terraform/adapter.py - modules-ir/l1/l1-s3/spike_instance.json /tmp/p9_regen && diff -r - terraform/spike /tmp/p9_regen` (ignoring the .terraform dir + - tfplan) shows no difference in the three .tf files. - -#### T-9.7 — Author `scripts/run_spike_plan.sh` - -- **Owner:** platform-engineer (territory: the spike runner is infra - tooling under `scripts/`; lead-developer owns `scripts/verify_phase*.sh` - only, not the runner — the runner is platform tooling). -- **Files owned:** `scripts/run_spike_plan.sh` (new). -- **What the file must contain** (cite sources): - 1. **Shebang + strict mode** — `#!/usr/bin/env bash`, - `set -euo pipefail`. - 2. **Header comment** — "Run the ACDL v1.1 spike terraform plan - (REQ-26c; D-039 rotated key; D-P09-1 no-lock). Loads the rotated - spike key from gitignored .env.secrets (NEVER committed), exports - AWS env vars, runs terraform init -lock=false + validate + plan - -lock=false -out=tfplan in terraform/spike/. Plan-only: NO apply - (Out of Scope). Uses -lock=false because plan does not write state - + the outbox table's PK contractId does not match Terraform's - expected LockID (D-P09-1). Real OIDC is v1.2 (D-039, blocked on - go-gitea/gitea#36988)." - 3. **Constants** — - - `REPO_ROOT="$(git rev-parse --show-toplevel)"`. - - `SECRETS_FILE="$REPO_ROOT/.env.secrets"`. - - `SPIKE_DIR="$REPO_ROOT/terraform/spike"`. - 4. **Pre-flight checks** — - - `.env.secrets` exists; if not, exit "FAIL: .env.secrets missing - — run scripts/rotate_spike_key.sh first (Phase 08)". - - `.env.secrets` is gitignored: `git check-ignore -q - "$SECRETS_FILE"` → if it fails, exit "FAIL: .env.secrets is NOT - gitignored (security violation)". - - `terraform` is installed: `command -v terraform >/dev/null` → - if not, exit "FAIL: terraform not installed (D-045: install via - HashiCorp apt pinned 1.9.*)". - - `terraform/spike/main.tf` exists (the generated config from - T-9.6); if not, exit "FAIL: terraform/spike/main.tf missing — - run the adapter first (T-9.6)". - 5. **Load the rotated key** — parse `.env.secrets` into env vars: - ```bash - set -a - . "$SECRETS_FILE" - set +a - ``` - The .env.secrets file has `ACDL_AWS_ACCESS_KEY_ID=<...>` + - `ACDL_AWS_SECRET_ACCESS_KEY=<...>` + `AWS_DEFAULT_REGION=<...>` - (Phase 08 format). Export the AWS env vars terraform expects: - ```bash - export AWS_ACCESS_KEY_ID="$ACDL_AWS_ACCESS_KEY_ID" - export AWS_SECRET_ACCESS_KEY="$ACDL_AWS_SECRET_ACCESS_KEY" - export AWS_DEFAULT_REGION="${AWS_DEFAULT_REGION:-us-east-1}" - export AWS_REGION="$AWS_DEFAULT_REGION" - ``` - Do NOT echo these values. - 6. **Confirm the caller is the spike user (not root)** — run - `aws sts get-caller-identity` (via `aws` CLI if available, else a - `python3 -c` with boto3). Assert the Arn is - `arn:aws:iam::581513795199:user/acdl-spike-runner` (NOT root). If - it's root, exit "FAIL: caller is root — the spike key was not - rotated; D-034 not closed". Print "caller-identity: ". - 7. **terraform init -lock=false** — `cd "$SPIKE_DIR"` then - `terraform init -lock=false -input=false`. The `-lock=false` is - D-P09-1 (no DynamoDB lock table; the backend omits - `dynamodb_table`). `-input=false` for non-interactive use. - 8. **terraform validate** — `terraform validate`. Assert exit 0. - Print "terraform validate: OK". - 9. **terraform plan -lock=false** — - `terraform plan -lock=false -input=false -out=tfplan`. Assert - exit 0. The `-out=tfplan` saves the plan to - `terraform/spike/tfplan` (gitignored per T-9.9). `-lock=false` is - D-P09-1. `-input=false` for non-interactive use. Print - "terraform plan: OK (saved tfplan)". - 10. **Success message** — print "spike plan OK" + exit 0. - 11. **No apply** — the script NEVER runs `terraform apply` (Out of - Scope; `apply` is HITL-gated in v1.2). Add a comment: "DO NOT add - terraform apply here — the spike is plan-only (REQUIREMENTS.md - Out of Scope; PROJECT.md operational parameter 'Spike terraform - command = plan only')." - 12. **No secrets to stdout** — do NOT echo `AWS_SECRET_ACCESS_KEY` - or the `.env.secrets` contents. The `aws sts - get-caller-identity` output shows the Arn (non-secret) but NOT - the key. -- **Source citations:** - - `ROADMAP.md` Phase 09 success criteria (terraform validate + plan - succeed; no long-lived credential in the workflow). - - `PROJECT.md` D-039 (per-run-rotated key waiver; OIDC deferred). - - `PROJECT.md` D-045 (terraform 1.9.* via HashiCorp apt). - - `PROJECT.md` operational parameters (spike terraform command = plan - only; state bucket acdl-tfstate-581513795199-us-east-1). - - `PROJECT.md` D-P09-1 (this plan — no-lock; no dynamodb_table). - - `REQUIREMENTS.md` Out of Scope (terraform apply — out of scope; plan - only). - - `ARCHITECTURE.md` §12.3 (state S3 + DynamoDB single-region) + §12.5 - (execution layer; long-lived creds forbidden — D-039 waiver). -- **Commit message:** - `phase: 9, status: plan-as-execute, persona: platform-engineer, task: T-9.7` - (then `feat(P09): author run_spike_plan.sh (REQ-26c; rotated key; plan-only; D-P09-1)`) -- **Self-verify:** - - [ ] `bash -n scripts/run_spike_plan.sh` exits 0. - - [ ] `set -euo pipefail` is present. - - [ ] The script checks `.env.secrets` is gitignored BEFORE loading - it (security). - - [ ] The script loads `.env.secrets` + exports `AWS_ACCESS_KEY_ID` etc. - - [ ] `terraform init -lock=false` + `terraform validate` + - `terraform plan -lock=false -out=tfplan` all appear (in order). - - [ ] `terraform apply` does NOT appear anywhere in the script. - - [ ] The caller-identity assertion checks for the user ARN, NOT root. - - [ ] The script does NOT echo `AWS_SECRET_ACCESS_KEY`. - -#### T-9.9 — Update `.gitignore` for Terraform working artifacts - -- **Owner:** lead-developer (territory: `.gitignore` per PERSONAS.md) — - but this is a Wave 3 task alongside T-9.6/T-9.7 (platform-engineer's - wave); lead-developer edits `.gitignore` here as a cross-territory - edit (territory enforcement mode is `warn` per PERSONAS.md — log in - the commit message, do not fail). Alternatively platform-engineer - edits `.gitignore` as the Wave 3 owner; either is acceptable under - `warn` mode. **Decision:** platform-engineer edits `.gitignore` in - Wave 3 (it's a Wave 3 artifact alongside the generated TF + runner); - lead-developer reviews in Wave 4. -- **Files owned:** `.gitignore` (edit — append Terraform artifact - entries). -- **What `.gitignore` must add** (append, do not duplicate if present): - ``` - # Phase 09 — Terraform spike working artifacts (never commit) - terraform/spike/.terraform/ - terraform/spike/tfplan - terraform/spike/*.tfstate* - ``` - Check first with `grep` that none of the three lines exist; append - only if missing. Do not remove existing entries (`.env.secrets` + - `terraform/bootstrap/.bootstrap_state.json` from Phase 08 must remain). -- **Source citations:** - - Terraform working artifacts: `.terraform/` (the provider plugins + - state lock dir), `tfplan` (the saved plan from `terraform plan - -out`), `*.tfstate*` (state files, if any — the spike's backend is - S3, so local state files shouldn't appear, but gitignore them - defensively). - - Phase 08 `.gitignore` entries (`.env.secrets` + - `.bootstrap_state.json`) — do not duplicate. -- **Commit message:** - `phase: 9, status: plan-as-execute, persona: platform-engineer, task: T-9.9` - (then `chore(P09): gitignore terraform/spike working artifacts (.terraform/, tfplan, *.tfstate*)`) -- **Self-verify:** - - [ ] `.gitignore` contains `terraform/spike/.terraform/` + - `terraform/spike/tfplan` + `terraform/spike/*.tfstate*`. - - [ ] `git check-ignore terraform/spike/.terraform/` exits 0. - - [ ] `git check-ignore terraform/spike/tfplan` exits 0. - - [ ] Phase 08 entries (`.env.secrets` + - `terraform/bootstrap/.bootstrap_state.json`) are still present. +#### T-10.8 — Author `scripts/run_spike_e2e.sh` +- **Owner:** backend-engineer (orchestration) + platform-engineer (TF) + security-engineer (Checkov) +- **Content:** the 10-step orchestrator: (1) load .env.secrets; (2) resolve contract→IR (contract_resolver.py validates as step 1); (3) adapter compiles IR→terraform/spike/*.tf; (4) terraform init -lock=false + validate + plan -lock=false -out=tfplan; (5) run Checkov on terraform/spike/main.tf → /tmp/checkov.json; (6) checkov_adapter.py → /tmp/pcr.json (PolicyCheckResult list); (7) build confidence inputs (policy = PCR list, validation all true, freshness/source/history/nfrs cold-start) + confidence_signal.py → /tmp/signal.json; (8) assert band=="pass" for dev; (9) write evidence event to outbox (outbox_writer.py); (10) print summary + Signal + outbox item. Exit 0 only if all steps succeed + band==pass. --- ### Wave 4 — lead-developer: verify script -Wave 4 authors the phase gate. It asserts all prior waves' deliverables -+ runs `terraform validate/plan` + the runner + greps for credentials. -It depends on all prior waves. - -#### T-9.8 — Author `scripts/verify_phase09.sh` - -- **Owner:** lead-developer (territory: `scripts/verify_phase*.sh` per - PERSONAS.md). -- **Files owned:** `scripts/verify_phase09.sh` (new). -- **What the file must contain** (cite sources): - 1. **Shebang + strict mode** — `#!/usr/bin/env bash`, - `set -euo pipefail`. - 2. **Header comment** — "Verify Phase 09 (REQ-24 + REQ-26): the IR + - l1-s3 + the Terraform adapter + a real terraform plan against AWS - using the rotated spike key (D-039; OIDC deferred to v1.2). Asserts: - (a) interface.json exists + has the typed-input shape (NOT - IR-schema validation per D-P09-2); (b) spike_instance.json - validates against schemas/ir.schema.json; (c) registry.json has - the l1-s3@1.0.0 entry; (d) adapter.py py_compiles + emits a main.tf - with aws_s3_bucket; (e) terraform/spike/main.tf exists + is the - committed generated file; (f) terraform validate passes; (g) - terraform plan -lock=false succeeds (real AWS); (h) no AKIA in - committed files; (i) run_spike_plan.sh exits 0. Prints VERIFIED on - success." - 3. **Constants** — `REPO_ROOT="$(git rev-parse --show-toplevel)"`; - all paths relative to `$REPO_ROOT`. - 4. **Pre-flight checks** — - - `.env.secrets` exists + is gitignored (same as - `run_spike_plan.sh`). - - `terraform` is installed (`command -v terraform`). - - `python3 -m jsonschema` is available (or `ajv`); if neither, - exit "FAIL: jsonschema validator missing (pip install jsonschema - OR apt install ajv)". - 5. **Check (a) — interface.json exists + has the typed-input shape** - (D-P09-2: NOT IR-schema validation): - ```bash - python3 - "$REPO_ROOT" <<'PYEOF' - import json, sys, os - root = sys.argv[1] - iface = json.load(open(os.path.join(root, "modules-ir/l1/l1-s3/interface.json"))) - assert iface["name"] == "l1-s3", f"name: {iface.get('name')}" - assert iface["kind"] == "l1" - assert iface["version"] == "1.0.0" - assert iface["type"] == "aws:s3:bucket", f"type: {iface.get('type')} (must be IR type, NOT aws_s3_bucket)" - assert iface["module"] == "l1-s3@1.0.0" - assert "bucket_name" in iface["inputs"] and iface["inputs"]["bucket_name"]["type"] == "string" - assert "region" in iface["inputs"] and iface["inputs"]["region"]["type"] == "string" - assert "bucket_arn" in iface["outputs"] and iface["outputs"]["bucket_arn"]["type"] == "arn" - # substrate-agnostic: no TF block keywords in the interface - s = json.dumps(iface) - for kw in ['aws_s3_bucket', 'variable "', 'output "', 'resource "', 'provider "']: - assert kw not in s, f"substrate keyword {kw!r} found in interface.json (IR must be substrate-agnostic)" - print("check-a: interface.json OK (typed contract, substrate-agnostic)") - PYEOF - ``` - 6. **Check (b) — spike_instance.json validates against the IR schema** - (D-P09-2: the instance, NOT the interface, is IR-schema-valid): - ```bash - python3 -m jsonschema -i "$REPO_ROOT/modules-ir/l1/l1-s3/spike_instance.json" \ - "$REPO_ROOT/schemas/ir.schema.json" \ - || { echo "FAIL: spike_instance.json does not validate against ir.schema.json"; exit 1; } - echo "check-b: spike_instance.json validates against ir.schema.json" - ``` - 7. **Check (c) — registry.json has the l1-s3@1.0.0 entry**: - ```bash - python3 - "$REPO_ROOT" <<'PYEOF' - import json, sys, os - root = sys.argv[1] - reg = json.load(open(os.path.join(root, "modules-ir/registry.json"))) - assert "l1-s3" in reg and "1.0.0" in reg["l1-s3"], "l1-s3@1.0.0 entry missing" - entry = reg["l1-s3"]["1.0.0"] - assert entry["interface"] == "modules-ir/l1/l1-s3/interface.json" - assert entry["deprecated"] == False - print("check-c: registry.json l1-s3@1.0.0 OK") - PYEOF - ``` - 8. **Check (d) — adapter.py py_compiles + emits a main.tf with - aws_s3_bucket**: - ```bash - python3 -m py_compile "$REPO_ROOT/adapters/terraform/adapter.py" \ - || { echo "FAIL: adapter.py py_compile failed"; exit 1; } - echo "check-d1: adapter.py py_compile OK" - # Run the adapter into a temp dir + assert the output - TMPD=$(mktemp -d) - python3 "$REPO_ROOT/adapters/terraform/adapter.py" \ - "$REPO_ROOT/modules-ir/l1/l1-s3/spike_instance.json" "$TMPD" \ - || { echo "FAIL: adapter.py failed to run"; exit 1; } - grep -q 'resource "aws_s3_bucket"' "$TMPD/main.tf" \ - || { echo "FAIL: adapter main.tf missing aws_s3_bucket resource"; exit 1; } - grep -q 'bucket = "acdl-spike-bucket"' "$TMPD/main.tf" \ - || { echo "FAIL: adapter main.tf missing bucket arg"; exit 1; } - grep -q 'output "bucket_arn"' "$TMPD/main.tf" \ - || { echo "FAIL: adapter main.tf missing bucket_arn output"; exit 1; } - grep -q 'backend "s3"' "$TMPD/terraform.tf" \ - || { echo "FAIL: adapter terraform.tf missing s3 backend"; exit 1; } - # D-P09-1: NO dynamodb_table in the backend - if grep -q 'dynamodb_table' "$TMPD/terraform.tf"; then - echo "FAIL: adapter terraform.tf contains dynamodb_table (D-P09-1 forbids it)"; exit 1 - fi - grep -q 'provider "aws"' "$TMPD/providers.tf" \ - || { echo "FAIL: adapter providers.tf missing aws provider"; exit 1; } - echo "check-d2: adapter emits correct TF (main.tf + terraform.tf + providers.tf)" - ``` - 9. **Check (e) — terraform/spike/main.tf exists + is the committed - generated file** (D-P09-4: diff the committed file against a - fresh adapter run): - ```bash - test -f "$REPO_ROOT/terraform/spike/main.tf" \ - || { echo "FAIL: terraform/spike/main.tf missing"; exit 1; } - test -f "$REPO_ROOT/terraform/spike/terraform.tf" \ - || { echo "FAIL: terraform/spike/terraform.tf missing"; exit 1; } - test -f "$REPO_ROOT/terraform/spike/providers.tf" \ - || { echo "FAIL: terraform/spike/providers.tf missing"; exit 1; } - # Diff committed vs fresh generation (D-P09-4 reproducibility) - TMPD2=$(mktemp -d) - python3 "$REPO_ROOT/adapters/terraform/adapter.py" \ - "$REPO_ROOT/modules-ir/l1/l1-s3/spike_instance.json" "$TMPD2" >/dev/null - for f in main.tf terraform.tf providers.tf; do - if ! diff -q "$REPO_ROOT/terraform/spike/$f" "$TMPD2/$f" >/dev/null; then - echo "FAIL: terraform/spike/$f differs from adapter output (not reproducible — re-run adapter)"; exit 1 - fi - done - echo "check-e: terraform/spike/*.tf present + reproducible (D-P09-4)" - ``` - 10. **Check (h) — no AKIA in committed files** (run BEFORE the plan, - so a credential leak fails fast): - ```bash - if rg -q 'AKIA' "$REPO_ROOT/terraform/spike/" "$REPO_ROOT/adapters/" "$REPO_ROOT/modules-ir/" 2>/dev/null; then - echo "FAIL: AKIA (AWS access key id prefix) found in committed files (security violation)"; exit 1 - fi - echo "check-h: no AKIA in terraform/spike/ + adapters/ + modules-ir/" - ``` - (Use `rg` if available, else `grep -rl 'AKIA' ...`.) - 11. **Check (i) — run_spike_plan.sh exits 0** (this runs the real - `terraform plan` against AWS — checks (f) + (g) are subsumed by - this, but the script runs them explicitly for clearer failure - messages): - ```bash - bash "$REPO_ROOT/scripts/run_spike_plan.sh" \ - || { echo "FAIL: run_spike_plan.sh exited non-zero"; exit 1; } - echo "check-i: run_spike_plan.sh OK (terraform init + validate + plan -lock=false all passed)" - ``` - `run_spike_plan.sh` internally runs `terraform init -lock=false` + - `terraform validate` + `terraform plan -lock=false -out=tfplan` + - asserts the caller is `acdl-spike-runner`. So check (i) covers - checks (f) + (g) too; the script's own assertions provide the - detail. - 12. **Success message** — print: - ``` - VERIFIED — Phase 09: IR + l1-s3 + Terraform adapter; terraform plan OK (real AWS, D-P09-1 no-lock) - ``` - Exit 0. - 13. **Failure mode** — any assertion failure: print `FAIL: ` - to stderr + exit 1. Do NOT print any secret in any error message. -- **Source citations:** - - `ROADMAP.md` Phase 09 success criteria (schemas/ir.schema.json - satisfied by the l1-s3 instance; adapter translates l1-s3 to a - valid terraform plan; terraform validate + plan succeed; no - long-lived credential). - - `PROJECT.md` D-P09-1 (no dynamodb_table; -lock=false). - - `PROJECT.md` D-P09-2 (interface is a contract, not an IR-schema - instance; the instance is IR-schema-valid). - - `PROJECT.md` D-P09-4 (generated TF committed + reproducible). - - `REQUIREMENTS.md` REQ-24 + REQ-26. - - `PERSONAS.md` verification_toolchain (typecheck = py_compile + - bash -n; test = verify_phaseNN.sh; build = terraform init). -- **Commit message:** - `phase: 9, status: plan-as-execute, persona: lead-developer, task: T-9.8` - (then `feat(P09): author verify_phase09.sh (asserts IR + adapter + terraform plan + no AKIA)`) -- **Self-verify:** - - [ ] `bash -n scripts/verify_phase09.sh` exits 0. - - [ ] `set -euo pipefail` is present. - - [ ] Check (a) asserts interface.json shape + substrate-agnosticism - (no TF keywords). - - [ ] Check (b) runs `python3 -m jsonschema` against the IR schema. - - [ ] Check (d) runs the adapter into a temp dir + greps for - `aws_s3_bucket` + asserts NO `dynamodb_table` (D-P09-1). - - [ ] Check (e) diffs committed TF against fresh adapter output - (D-P09-4). - - [ ] Check (h) greps for `AKIA` in the three dirs. - - [ ] Check (i) runs `run_spike_plan.sh` (which runs the real plan). - - [ ] No secret is echoed in any error path. +#### T-10.9 — Author `scripts/verify_phase10.sh` +- **Owner:** lead-developer +- **Content:** 8 checks: (a) composition.json exists + shape; (b) spike.yaml validates against contract schema; (c) resolver py_compiles + emits IR validating against ir.schema.json; (d) adapter py_compiles + emits main.tf with aws_s3_bucket; (e) run_spike_e2e.sh exits 0; (f) confidence band is "pass" for dev; (g) outbox item exists (query DynamoDB); (h) REQ-28: grep for aws_s3_bucket + aws_ — only adapters/ match; modules-ir/ schemas/ contracts/ acdl_platform/ do NOT match (substrate-agnostic). --- -### Wave 5 — lead-developer (EXECUTE-only): run the spike plan + verify +### Wave 5 — EXECUTE-only -**This wave is the only wave that touches real AWS with the spike key.** -It runs the spike runner (real `terraform plan` against AWS) + the -verify script, then updates traceability. It does NOT author any files -(except the traceability update — lead-developer territory). - -#### T-10.0 — Execute the spike plan + verify + traceability - -- **Owner:** lead-developer (orchestrates; the execution runs - platform-engineer's adapter + runner + verify script). -- **Files owned:** none authored (this wave runs files from prior - waves); the only file mutation is the traceability update - (`REQUIREMENTS.md`, `ROADMAP.md`, `PROJECT.md` — lead-developer - territory). -- **Execution sequence** (the orchestrator runs these in a shell; the - rotated spike key is in `.env.secrets`, loaded by the scripts): - 1. `bash scripts/verify_phase09.sh` — runs all checks (a)-(i), - including `run_spike_plan.sh` (which runs `terraform init -lock=false` - + `validate` + `plan -lock=false -out=tfplan` against real AWS). - Assert exit 0 + the "VERIFIED — Phase 09" message. The `terraform - plan` reads the existing state bucket (the spike user has S3 read - on `acdl-tfstate-581513795199-us-east-1`) + calls STS - (`GetCallerIdentity`) + computes the diff for one - `aws_s3_bucket` resource (`acdl-spike-bucket`). Plan-only: NO - apply, NO state write, NO resource mutation. - 2. **Traceability update** — edit: - - `REQUIREMENTS.md`: REQ-24 → `complete (v1.1.4)`; REQ-26 → - `complete (v1.1.4)` (in the traceability table). - - `ROADMAP.md`: Phase 09 → `complete (v1.1.4)`. - - `PROJECT.md`: add D-P09-1..D-P09-4 to the Key Decisions table. - 3. **Commit + tag** — the executor commits the traceability update - with message `docs(P09): post-ship traceability + roadmap update (v1.1.4)` - + a ship commit `ship: phase-09 v1-spike-ir-and-l1-and-adapter (v1.1.4)`, - then tags `v1.1.4` (per `config.json` git.auto_commit + auto_push - + the Phase 08 precedent `v1.1.3`). -- **No AWS mutation** — the spike is plan-only (Out of Scope: terraform - apply). The spike user's policy (Phase 08) grants S3 read on the state - bucket + DynamoDB on the outbox + STS GetCallerIdentity + explicit - Deny everything else — the user CANNOT mutate AWS resources even if - `terraform apply` were run (it's not). The plan reads state + computes - a diff; it does not provision. -- **D-039 rotation after the run** — per D-039's per-run rotation - discipline, the executor runs `bash scripts/rotate_spike_key.sh` - after the verify script succeeds (rotates the spike key: creates a - fresh key, deactivates + deletes the one just used, writes the new - key to `.env.secrets`). This is the spike's "no persistently long-lived - key" guarantee. (Phase 08 established this; Phase 09 continues it.) - Document this in the traceability update. -- **Commit message:** - `phase: 9, status: execute, persona: lead-developer, task: T-10.0` - (then `docs(P09): post-ship traceability + roadmap update (v1.1.4)`) -- **Self-verify:** - - [ ] `bash scripts/verify_phase09.sh` exits 0 + prints "VERIFIED — - Phase 09". - - [ ] `terraform/spike/tfplan` exists (the saved plan; gitignored). - - [ ] `terraform/spike/.terraform/` exists (the init dir; gitignored). - - [ ] `git status` shows the traceability edits staged. - - [ ] `git tag --list | grep v1.1.4` shows the new tag. - - [ ] `bash scripts/rotate_spike_key.sh` runs after + `.env.secrets` - has a fresh key (the AccessKeyId changed). - ---- - -## Wave ordering + dependencies - -``` -Wave 1 (platform-engineer) T-9.1 interface.json ┐ - T-9.2 spike_instance.json ├ independent (parallel) - T-9.3 registry.json │ - T-9.4 README.md ┘ - │ - ▼ -Wave 2 (platform-engineer) T-9.5 adapter.py (reads Wave 1 instance shape) - │ - ▼ -Wave 3 (platform-engineer) T-9.6 terraform/spike/*.tf (generated from Wave 2 + Wave 1) - T-9.7 run_spike_plan.sh (references Wave 3 generated TF) - T-9.9 .gitignore (TF artifacts) - │ - ▼ -Wave 4 (lead-developer) T-9.8 verify_phase09.sh (asserts all prior + runs runner) - │ - ▼ -Wave 5 (lead-developer, EXECUTE) T-10.0 run verify_phase09.sh (real terraform plan vs AWS) - + rotate_spike_key.sh (D-039) - + traceability update + tag v1.1.4 -``` - -**Dependencies:** -- Wave 2 → Wave 1 (adapter reads the IR instance; its IR-type→TF-type - mapping is driven by the L1 interface's `type` field). -- Wave 3 → Wave 2 (generated TF is the adapter's output; the runner - references the generated files). -- Wave 4 → all prior (verify asserts interface + instance + registry + - adapter + generated TF + runner). -- Wave 5 → Wave 4 (EXECUTE-only; runs verify against real AWS; the - verify script is the last file Wave 5 invokes). - -**Parallelizable:** -- Wave 1: T-9.1..T-9.4 are independent files (same persona) — parallel. -- Wave 3: T-9.6 (generated TF), T-9.7 (runner), T-9.9 (gitignore) are - independent files (same persona) — parallel (T-9.6 requires running - the adapter first, which is a Wave 2 deliverable; T-9.7 + T-9.9 only - reference the generated path). - ---- - -## Per-file content checklist - -### `modules-ir/l1/l1-s3/interface.json` (T-9.1) -- JSON L1 interface *contract* (typed declarations; NOT an IR-schema - instance — D-P09-2). -- Keys: `name=l1-s3`, `kind=l1`, `version=1.0.0`, `description`, - `type=aws:s3:bucket` (IR type, NOT `aws_s3_bucket`), `module=l1-s3@1.0.0`. -- `inputs`: `bucket_name={type:string, description, required:true}`, - `region={type:string, description, required:true}`. -- `outputs`: `bucket_arn={type:arn, description}`. -- `nfrs`: `{}` (empty for the spike). -- NO `aws_s3_bucket` / `variable "..."` / `output "..."` / `resource "..."` - / `provider "..."` (substrate-agnostic; grep should be empty). -- Does NOT validate against `schemas/ir.schema.json` (it's a contract, - not an instance — `inputs` are typed declarations, not concrete values). - -### `modules-ir/l1/l1-s3/spike_instance.json` (T-9.2) -- Minimal IR *stack instance* with CONCRETE values; validates against - `schemas/ir.schema.json` (D-P09-2). -- `version=1.0.0`; `stack={name:l1-s3, kind:l1, depth:1}`. -- `resources[0]={id:s3, type:aws:s3:bucket, module:l1-s3@1.0.0, - inputs:{bucket_name:"acdl-spike-bucket", region:"us-east-1"}, - outputs:{bucket_arn:{type:arn, description}}}`. -- No `parent` (root resource). No `relationships` (single resource). -- `inputs.bucket_name` + `inputs.region` are concrete strings (NOT - typed declarations — the IR schema's `resource.inputs.additionalProperties` - is `{type:[string,number,boolean]}`). - -### `modules-ir/registry.json` (T-9.3) -- One entry: `{"l1-s3":{"1.0.0":{"interface":"modules-ir/l1/l1-s3/interface.json","published_at":"","deprecated":false}}}`. -- The `interface` path is relative to the repo root (NOT to the - registry file). -- D-P09-3: lives under `modules-ir/` (co-located with the L1 modules; - v1.2 may move to `platform/registry/`). - -### `modules-ir/l1/l1-s3/README.md` (T-9.4) -- Short doc: type `aws:s3:bucket` (IR type); inputs `bucket_name`/`region`; - outputs `bucket_arn`; the IR→Terraform mapping table (type→`aws_s3_bucket`, - `inputs.bucket_name`→`bucket` arg, `inputs.region`→provider `region`, - `outputs.bucket_arn`→`output ... .arn`). -- Cites ARCHITECTURE.md §12.1/§12.2/§12.7. -- Files-in-directory table (interface.json, spike_instance.json, README.md). - -### `adapters/terraform/adapter.py` (T-9.5) -- Python module: `adapt(ir_instance, out_dir)` + `__main__` CLI - (`adapter.py `). -- Stdlib only (no third-party deps). -- `IR_TYPE_TO_TF = {"aws:s3:bucket": "aws_s3_bucket"}` (the only - substrate-specific knowledge; the binding constraint per §12.2). -- Emits `main.tf` (resources + outputs), `terraform.tf` (required_version - `>= 1.9, < 1.10` + required_providers aws ~> 5.0 + S3 backend with - `bucket=acdl-tfstate-581513795199-us-east-1`, - `key=spike/l1-s3/terraform.tfstate`, `region=us-east-1`, **NO - `dynamodb_table`** per D-P09-1), `providers.tf` (`provider "aws" - { region = }`). -- For `aws:s3:bucket`: `resource "aws_s3_bucket" "" { bucket = - }` + `output "bucket_arn" { value = - aws_s3_bucket..arn }`. -- Does NOT read `.env.secrets` / set AWS env vars (that's the runner). -- Does NOT own L1/L2 content (thin layer; §12.2). - -### `terraform/spike/main.tf` (T-9.6, generated) -- Exact content (after running the adapter against spike_instance.json): - ```hcl - resource "aws_s3_bucket" "s3" { - bucket = "acdl-spike-bucket" - } - - output "bucket_arn" { - value = aws_s3_bucket.s3.arn - } - ``` - -### `terraform/spike/terraform.tf` (T-9.6, generated) -- Exact content: - ```hcl - terraform { - required_version = ">= 1.9, < 1.10" - required_providers { - aws = { - source = "hashicorp/aws" - version = "~> 5.0" - } - } - backend "s3" { - bucket = "acdl-tfstate-581513795199-us-east-1" - key = "spike/l1-s3/terraform.tfstate" - region = "us-east-1" - } - } - ``` - NO `dynamodb_table` (D-P09-1). - -### `terraform/spike/providers.tf` (T-9.6, generated) -- Exact content: - ```hcl - provider "aws" { - region = "us-east-1" - } - ``` - -### `scripts/run_spike_plan.sh` (T-9.7) -- `#!/usr/bin/env bash` + `set -euo pipefail`. -- Loads `.env.secrets` (gitignored); exports `AWS_ACCESS_KEY_ID` etc. -- Asserts caller is `acdl-spike-runner` (not root) via STS. -- `cd terraform/spike`; `terraform init -lock=false -input=false`; - `terraform validate`; `terraform plan -lock=false -input=false -out=tfplan`. -- Prints "spike plan OK". NO `terraform apply` anywhere. -- Does NOT echo `AWS_SECRET_ACCESS_KEY`. - -### `scripts/verify_phase09.sh` (T-9.8) -- `#!/usr/bin/env bash` + `set -euo pipefail`. -- Checks (a)-(i) per the must_haves in frontmatter: - - (a) interface.json typed shape + substrate-agnostic (no TF keywords). - - (b) spike_instance.json validates against ir.schema.json. - - (c) registry.json l1-s3@1.0.0 entry. - - (d) adapter.py py_compile + emits correct main.tf/terraform.tf/providers.tf - (NO dynamodb_table — D-P09-1). - - (e) terraform/spike/*.tf present + reproducible (diff vs fresh - adapter run — D-P09-4). - - (f) terraform validate passes (subsumed by check (i) running - run_spike_plan.sh). - - (g) terraform plan -lock=false succeeds (subsumed by check (i)). - - (h) no `AKIA` in `terraform/spike/` + `adapters/` + `modules-ir/`. - - (i) `run_spike_plan.sh` exits 0. -- Prints "VERIFIED — Phase 09: IR + l1-s3 + Terraform adapter; terraform - plan OK (real AWS, D-P09-1 no-lock)". -- No secret echoed in any error path. - -### `.gitignore` (T-9.9) -- Appends (does not duplicate): - ``` - # Phase 09 — Terraform spike working artifacts (never commit) - terraform/spike/.terraform/ - terraform/spike/tfplan - terraform/spike/*.tfstate* - ``` -- Phase 08 entries (`.env.secrets` + `terraform/bootstrap/.bootstrap_state.json`) - remain. - ---- +Run `scripts/run_spike_e2e.sh` against real AWS + `scripts/verify_phase10.sh` + traceability + ship. ## Decisions made during planning -### D-P09-1 — Terraform locking: `-lock=false` + omit `dynamodb_table` -- **Decision:** the spike runs `terraform plan -lock=false` and the - S3 backend config **omits `dynamodb_table`** entirely. -- **Rationale:** Phase 08's DynamoDB table `acdl-outbox` (D-044 + - D-P08-1 consolidated table) has PK `contractId` + SK `eventType#eventTs`, - but Terraform's S3 backend expects a DynamoDB table whose primary key - attribute is named `LockID` (a string). The consolidated table does - NOT match. Options evaluated: - - (a) create a separate `acdl-tflock` table (PK `LockID`) — requires a - root-key escalation (the spike user's policy does NOT grant - `dynamodb:CreateTable`) OR granting CreateTable to the spike user - (broadens the attack surface); - - (b) use Terraform's S3-native locking via `-lock=true` + a `tflock` - table — same CreateTable problem as (a); - - (c) **disable locking (`-lock=false`)** — chosen. - - (c) is correct because (i) `terraform plan` does NOT write state, so - locking is unnecessary for plan-only; (ii) it avoids a root-key - escalation + a second table; (iii) the v1.2 real-apply path will - create the proper `LockID`-keyed `acdl-tflock` table (the - operational parameter "Terraform lock table = acdl-tflock" is - realized in v1.2, NOT in the spike). -- **Confidence:** 0.90 (plan-only semantics are well-documented; the - outbox PK mismatch is a verified fact from Phase 08). -- **Alternatives:** (a) separate `acdl-tflock` table (root-key - escalation); (b) S3-native locking (same CreateTable problem). - -### D-P09-2 — L1 interface vs IR schema: contract ≠ instance -- **Decision:** the L1 interface file (`modules-ir/l1/l1-s3/interface.json`) - is a **contract** (typed input/output declarations) that does NOT - validate against `schemas/ir.schema.json`; a separate minimal stack - instance (`spike_instance.json`) with concrete values validates - against the IR schema. The IR schema is for *instances*; the L1 - interface is for *contracts*. -- **Rationale:** the IR schema's `resource.inputs` is - `additionalProperties: {type: [string, number, boolean]}` — i.e. - *concrete values*, not typed declarations. An L1 *interface* declares - input *types* (`bucket_name: {type: string, description: ...}`). The - two shapes are incompatible: the interface cannot be an IR-schema - instance (its `inputs` are typed declarations, not values). The clean - separation: the IR schema validates *stack instances* (with values, - the adapter's input); the L1 interface is a *contract* (the L1 - catalog entry, the registry's referenced artifact). Phase 10's - contract→IR resolution will produce the instance from the contract + - the contract's input values. -- **Confidence:** 0.85 (the friction is real — the schema's - `additionalProperties` is values-only; the separation is the cleanest - resolution; Phase 10 will confirm the contract→instance flow). -- **Alternatives:** (a) extend the IR schema with a `type` field for - inputs (would make the schema validate both contracts + instances — - but muddies the schema's purpose); (b) make the L1 interface a - superset of the IR instance (rejected — the instance has concrete - values, the interface has types; they are different shapes). - -### D-P09-3 — L1 registry location: `modules-ir/registry.json` -- **Decision:** the L1 registry lives at `modules-ir/registry.json` - (not `platform/registry/`) for the spike. -- **Rationale:** the registry is co-located with the L1 modules it - catalogs (§12.7: "L1 publication updates the registry in the same - PR"). For the spike's single L1, `modules-ir/registry.json` is the - simplest location. v1.2 may move it to `platform/registry/` when the - L1 catalog grows + the registry gains a publication workflow. -- **Confidence:** 0.80 (the location is a spike convenience; the §12.7 - "same PR" constraint is satisfied by either location; v1.2 may - relocate). -- **Alternatives:** `platform/registry/registry.json` (the PERSONAS.md - platform-engineer territory lists `platform/registry/**` — but that's - the v1.2 location; for the spike, `modules-ir/` is co-located). - -### D-P09-4 — Generated TF committed + reproducibility-verified -- **Decision:** the spike generates `terraform/spike/*.tf` by running - the adapter (Wave 3) and commits the generated files; the verify - script regenerates into a temp dir + diffs against the committed - files to prove reproducibility. -- **Rationale:** committing the generated files lets `verify_phase09.sh` - run `terraform validate/plan` without regenerating (the generated - files are the artifact under test). Separately regenerating + diffing - in the verify script proves the generation step is reproducible (no - manual edits to the generated TF; the adapter is the single source). - This matches the Phase 08 precedent (`.bootstrap_state.json` is - committed as bookkeeping; the verify script asserts its shape). -- **Confidence:** 0.90 (the diff-in-verify pattern is standard for - generated artifacts; the adapter is deterministic for a fixed IR - instance). -- **Alternatives:** (a) do NOT commit the generated TF; regenerate in - the verify script (rejected — the verify script would then test the - adapter's output, not the committed artifact; a manual edit to the - committed TF would be invisible); (b) commit + do NOT verify - reproducibility (rejected — drift between the adapter + the committed - TF would be invisible). - ---- +| ID | Decision | Rationale | +|----|----------|-----------| +| D-P10-1 | The adapter handles kind=l2 by consuming the resolved IR instance (L1 resources as the resources array); for depth-1, the L2 root module IS the L1's resource — no separate module block. Relationships ignored at TF level for the spike. | The adapter is a thin layer; the composition is shallow. v1.2 may emit `module "l1_s3" { source = "..." }` when L1s become real TF modules. | +| D-P10-2 | The contract is authored as YAML + the resolver parses YAML→dict→validates against the JSON contract schema. | YAML is the consumer surface; JSON Schema validates the parsed dict. PyYAML required. | +| D-P10-3 | The evidence event is a single CONFIDENCE_COMPUTED event (spike writes ONE event; chain = GENESIS→this event). | The spike proves the outbox write path; v1.2 writes the full event stream. | ## Spike scope vs v1.2 boundary -| Concern | Spike (Phase 09) | v1.2 | +| Concern | Spike (Phase 10) | v1.2 | |---------|------------------|------| -| Terraform locking | `-lock=false` (no DynamoDB lock table; D-P09-1) | proper `LockID`-keyed `acdl-tflock` table | -| Adapter scope | `l1-s3` only (IR type `aws:s3:bucket` → `aws_s3_bucket`) | full L1 catalog + L2 thin-composition (Phase 10 extends) | -| L1 interface vs IR schema | separate files (contract `interface.json` vs instance `spike_instance.json`; D-P09-2) | unified IR-typed interface (the contract→IR resolution produces the instance from the contract) | -| Registry location | `modules-ir/registry.json` (one entry; D-P09-3) | `platform/registry/` + central L1 catalog + publication workflow | -| AWS auth | rotated long-lived key (D-039 waiver; `.env.secrets`) | real OIDC federation (blocked on go-gitea/gitea#36988) | -| `terraform` command | `plan` only (Out of Scope: apply) | HITL-gated `apply` (prod/dr environments) | -| State backend | S3 bucket (no DynamoDB lock; D-P09-1) | S3 + DynamoDB lock (`acdl-tflock`) | -| IR-type→TF-type map | one entry (`aws:s3:bucket` → `aws_s3_bucket`) | full map (all L1 types) | -| L2 thin-composition | NOT in Phase 09 (Phase 10) | `l2-static-asset` → root module (Phase 10) | -| Relationships | zero (single resource) | `parent` + `uses_output` (Phase 10) | +| L2 composition | depth-1, one child, wires passthrough | multi-child, complex wires, module blocks | +| Evidence events | ONE CONFIDENCE_COMPUTED event | full stream (submission, validation, plan, policy, confidence, promotion) | +| Audit chain | GENESIS → one event | full chain + JWS + Object Lock | +| HITL | dev-only (not exercised) | qa/prod/dr gates wired | +| Checkov | terraform framework + ACDL_TAG_NAMING SKIPPED | custom Checkov YAML rule + Kyverno + OPA | ---- +## REQ-28 verification (the binding spike claim) -## Security constraints - -- **No long-lived credential in any committed file.** The rotated spike - key lives in `.env.secrets` (gitignored; Phase 08). The spike runner - loads it at runtime. `verify_phase09.sh` greps for `AKIA` (the AWS - access key id prefix) in `terraform/spike/` + `adapters/` + - `modules-ir/` — must be empty. -- **`.env.secrets` is gitignored.** Phase 08 added it; Phase 09 does - not touch the gitignore entry. The verify script asserts - `git check-ignore .env.secrets` succeeds before loading it. -- **The spike key is least-privilege.** Phase 08's - `spike_runner_policy.json` grants S3 read/write on the state bucket + - DynamoDB on the outbox table + STS GetCallerIdentity + explicit Deny - everything else. The user CANNOT mutate AWS resources (no - `terraform apply` permission, no IAM/EC2/etc.). `terraform plan` only - reads + computes a diff. -- **`terraform plan` does not mutate AWS.** Plan-only (Out of Scope: - apply). The spike user's policy does not grant any resource-creation - permission; even if `apply` were run, it would fail. The plan reads - the state bucket (S3 GetObject/ListBucket granted) + calls STS - (GetCallerIdentity granted) + queries the AWS provider (which reads - existing resources — the spike user has no read on arbitrary AWS - resources, so the provider's refresh reads return empty/forbidden, - which is fine for the spike's single new `aws_s3_bucket` that does - not exist yet). -- **D-039 per-run rotation.** After the spike run (Wave 5), the - executor runs `scripts/rotate_spike_key.sh` to rotate the key - (creates a fresh key, deactivates + deletes the one just used). This - is the spike's "no persistently long-lived key" guarantee. -- **No OIDC in the spike.** Real OIDC federation is deferred to v1.2 - (D-039; blocked on go-gitea/gitea#36988). The spike uses static AWS - creds from `.env.secrets` (the rotated key). The adapter emits - static-cred-compatible Terraform (no OIDC provider block); the runner - sets `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY` env vars. - ---- - -## Phase 10 downstream contract (informs Phase 09 boundary) - -Phase 10 (`v1-spike-l2-and-contract-e2e`) will: -- Author `modules-ir/l2/l2-static-asset/` (thin-composition referencing - `l1-s3` only, depth 1) + its interface + instance. -- Author `platform/contract_resolver.py` (contract→IR resolution: takes - a `contracts/spike.yaml` + the L1 registry + the L2 thin-composition - → produces an IR stack instance with concrete values — the adapter's - input). This is where D-P09-2's contract→instance separation is - exercised: the resolver reads `interface.json` (the contract) + - the contract's input values → produces an IR instance (validates - against `schemas/ir.schema.json`). -- Author `contracts/spike.yaml` (one end-to-end submission). -- Run the full pipeline: contract schema validation → contract→IR - resolution → `terraform plan` (via the Phase 09 adapter) → Checkov - `PolicyCheckResult` → confidence signal → evidence event to the - DynamoDB outbox. - -Phase 09's adapter must be reusable for Phase 10: the adapter takes any -IR instance (not just `spike_instance.json`) + emits TF. Phase 10's -contract→IR resolution produces a `l2-static-asset` IR instance (with -the `l1-s3` resource as a child + a `parent` relationship); the adapter -will need to handle `parent` relationships (Phase 10 extends the -adapter; the spike's single-resource L1 has zero relationships, so -Phase 09 does not exercise relationships — but the adapter's -`adapt(ir_instance, out_dir)` signature is generic enough to accept a -multi-resource instance in Phase 10). - -**Phase 09 boundary:** the adapter handles `l1-s3` only (one IR type, -one resource, zero relationships). L2 thin-composition, relationships, -and contract→IR resolution are Phase 10. The adapter's `IR_TYPE_TO_TF` -map has one entry; Phase 10 + v1.2 extend it. The adapter's -`adapt(ir_instance, out_dir)` signature is the stable contract Phase -10 builds on. \ No newline at end of file +verify_phase10.sh's Check (h) greps the repo for `aws_s3_bucket` + +`aws_` (Terraform-specific terms). ONLY `adapters/terraform/adapter.py` ++ `adapters/terraform/policy/checkov_adapter.py` should match. +`modules-ir/`, `schemas/`, `contracts/`, `acdl_platform/confidence_signal.py`, +`acdl_platform/contract_resolver.py`, `acdl_platform/outbox_writer.py` +should NOT match (substrate-agnostic). This proves the IR commitments +hold: the adapter is the only substrate-specific code; the L1 content, +contract YML, resolver, confidence signal, and outbox writer are all +substrate-neutral. \ No newline at end of file