From 1efb44444ab426cf3a1ab5fa8f0c3db98efd4836 Mon Sep 17 00:00:00 2001 From: Jon Chery Date: Tue, 28 Jul 2026 15:01:45 +0000 Subject: [PATCH] =?UTF-8?q?docs(P56a):=20create=20v1.11=20RESTART=20phase?= =?UTF-8?q?=20plan=20=E2=80=94=20stateless=20adapter=20+=20s3=20reference?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PLAN stage. P56a is the first phase of the v1.11 restart: rewrite the 918-line adapter monolith to a ~80-line stateless assembler, prove the design with the s3 reference module. 6 tasks, single wave (no parallelization — one cohesive change): 1. s3 reference terraform module (versions/variables/locals/main/outputs) 2. Registry extension (terraform_dir field) 3. Stateless adapter rewrite (delete TYPE_MAP/INPUT_MAP/OUTPUT_MAP + 39 branches) 4. STANDARDS.md §8 rewrite (stateless assembler pattern) 5. test_adapter.py rewrite (assert assembly, not HCL strings) 6. Offline regression (pytest + run_primitive_plan.sh --check-only s3) Success gate: adapter < 100 lines, no type-specific logic, s3 module validates standalone, adapter-emitted root main.tf validates, offline suite green. ---ci--- project: acdl phase: P56a milestone: v1.11 status: plan ---/ci--- --- .ciagent/PLAN.md | 256 +++++++++++++++++------------------------------ 1 file changed, 92 insertions(+), 164 deletions(-) diff --git a/.ciagent/PLAN.md b/.ciagent/PLAN.md index cf27d9a..6ea2b55 100644 --- a/.ciagent/PLAN.md +++ b/.ciagent/PLAN.md @@ -1,194 +1,122 @@ --- -phase: 52-55 -name: v1.10-pipeline-regression-fix-and-capability-reverification -milestone: v1.10 -requirements: [REQ-112, REQ-113, REQ-114, REQ-115] -type: fix/test/docs +phase: P56a +name: stateless-adapter-rewrite +milestone: v1.11 +requirements: [REQ-123] +wave: 1 --- -# ACDL v1.10 — Pipeline Regression Fix + Capability Re-Verification +# P56a — Stateless Adapter Rewrite + s3 Reference Module -> Milestone v1.10. Generated at PLAN stage. Autonomy: full. -> Requirements: REQ-112..REQ-115 (see REQUIREMENTS.md). -> Decisions: D-090..D-094 (see PROJECT.md). -> Versioning: NFR/fix milestone — progressive patch versions per phase -> (v1.9.9..v1.9.12), tag `v1.10.0` at milestone COMPLETE (next minor; -> this is fix/test/docs, not a breaking schema change). +**Phase:** P56a +**Milestone:** v1.11 (RESTART) +**Requirement:** REQ-123 +**Wave:** 1 (foundation — sequential prerequisite for P56b–P58) +**Branch:** `milestone/v1.11-restart` → `phase/p56a-stateless-adapter` -## Context +## Goal -The CLARIFY/RESEARCH stages (this run, 2026-07-27) surfaced a structural -defect and a credibility gap: +Rewrite `adapters/terraform/adapter.py` from a 918-line monolith (3 constant +tables + 39 type-specific branches) to a ~80-line stateless assembler. Author +`modules/l1/s3/terraform/` as the reference module proving the assembly path +end-to-end. Extend the registry. Rewrite STANDARDS.md §8. Rewrite +`tests/test_adapter.py` to assert assembly, not HCL string matching. -1. **VERIFY is diff-scoped (D-091).** The CIAgent VERIFY stage checks the - phase diff only; it never re-runs underlying platform capability. The - pipeline has no regression memory. As a result, 8 NFR-patch phases - (v1.9.1→v1.9.8, deck rework) passed VERIFY while the platform they - described decayed underneath them. -2. **Advertised capability is not currently reproducible.** The v1.2 ECS - Fargate E2E and v1.7 pipelines ran once historically (tags true at the - time) but are not reproducible today without revival work. The decks - present this capability as current without disclosing the decay. -3. **Decks froze critical-path work but were sequenced backwards.** Deck - rework (v1.9.1→v1.9.8) was justified by real incremental exec viewings, - but the feedback signal was mixed/ambiguous (thesis-not-landing + - demand-proof + needs-polish). The honest sequencing is re-verify → - rewrite docs/decks to match reality → polish. This was done backwards - for 8 phases. +This phase proves the stateless adapter design with ONE module (s3) before +P56b authors the remaining 11. If the assembly path works for s3, it works +for all L1s (the adapter is generic; only the module terraform differs). -User decisions (this run): -- **D-090:** No cap on the re-verification sweep. Fix every advertised - capability in-sweep; all must end Verified. Unbounded-risk trade-off - accepted for full integrity. Decks stay frozen until the sweep completes. -- **D-091:** Add a regression-class VERIFY that re-runs capability checks - (not just diff checks), at minimum on milestone completion. -- **D-092:** Build local emulating adapters (flat-file outbox, local ECS - emulator, local S3 state, local Lambda stub) so the platform is fully - locally testable without cloud credentials. -- **D-093:** Re-verify every v1.1→v1.8 advertised capability. v1.0 demo - excluded as archived/superseded. Headline E2E runs both live-AWS and - local-emulator tiers (both must pass); all other capabilities run - locally via emulating adapters. -- **D-094:** Rewrite PROJECT/ROADMAP/decks to match verified reality; - decks unfrozen only after this lands. +## Vertical Slice — Single Wave (no parallelization; one cohesive change) -## Wave ordering +### Task 1 — s3 reference terraform module (data-engineer) -- **Wave 1 (sequential):** Phase 52 — pipeline regression-VERIFY fix. - Must land first; the sweep runs through the fixed pipeline. -- **Wave 2 (sequential):** Phase 53 — local emulating adapters. The - sweep's local tier depends on these. -- **Wave 3 (sequential):** Phase 54 — v1.1→v1.8 capability re-verification - sweep. Fix in-sweep per D-090 (no cap). Tag each capability - Verified/Decayed/Broken; repair Decayed/Broken in-phase; all must end - Verified. -- **Wave 4 (sequential):** Phase 55 — rewrite PROJECT/ROADMAP/decks to - verified reality; unfreeze decks. +Author `modules/l1/s3/terraform/`: +- `versions.tf` — `terraform { required_version = ">= 1.9, < 1.10"; required_providers { aws = { source = "hashicorp/aws", version = "~> 5.0" } } }` +- `variables.tf` — one `variable {}` per s3 interface.json input: `bucket_name` (string, required), `region` (string, required), `kms_key_arn` (string, optional, default null). Plus `tags` (map(string), optional, default {}) for tag merge. +- `locals.tf` — heavy interpolation: + - `sse_algorithm` — "aws:kms" if `var.kms_key_arn != null`, else "AES256" (SSE-S3). This is the default the adapter currently hardcodes (adapter.py:225-228, 289-297). + - `tags` — merge `var.tags` with `acdl:owner`/`acdl:environment` defaults. +- `main.tf` — `resource "aws_s3_bucket" "this" { bucket = var.bucket_name; tags = local.tags }` + `resource "aws_s3_bucket_versioning" "this" { bucket = aws_s3_bucket.this.id; versioning { enabled = true } }` + `resource "aws_s3_bucket_server_side_encryption_configuration" "this" { bucket = aws_s3_bucket.this.id; rule { apply_server_side_encryption_by_default { sse_algorithm = local.sse_algorithm; kms_master_key_id = var.kms_key_arn } } }` (conditional on kms_key_arn). +- `outputs.tf` — `output "bucket_arn" { value = aws_s3_bucket.this.arn }`, `output "bucket_name" { value = aws_s3_bucket.this.id }`, `output "bucket_regional_domain_name" { value = aws_s3_bucket.this.bucket_regional_domain_name }`. ---- +**Must-have:** `terraform init + validate` passes standalone in `modules/l1/s3/terraform/`. -## Phase 52 — pipeline-regression-verify-fix +### Task 2 — Registry extension (backend-engineer) -**Requirements:** REQ-112 -**Personas:** backend-engineer (lead: VERIFY stage), ci-verifier (review) -**Branch:** `phase/52-pipeline-regression-verify-fix` +Update `modules/registry.json`: add `"terraform_dir": "modules/l1/s3/terraform"` to the s3 1.0.0 entry. This is the field the adapter reads to find the module source path. -### Task 52.1 — Add regression-class VERIFY (REQ-112, backend-engineer) -- Extend the VERIFY stage to support a `regression` mode that re-runs - capability checks (not just diff checks). Triggered at minimum on - milestone completion; may also be invoked per-phase when a phase - touches platform code (not docs-only NFR patches). -- The regression run executes the local-emulator tier (Phase 53) for - every capability marked Verified in prior milestones. Any capability - that fails the regression run blocks milestone completion. -- Record the regression result in `---ci---` blocks as - `regression: { capability: , status: Verified|Decayed|Broken }`. -- Verify: a regression run against the current codebase surfaces at - least one Decayed/Broken capability (proving the gate catches decay, - not just passes). +**Must-have:** `registry.json` validates as JSON; the s3 entry has `terraform_dir`. -### Success Criteria -- VERIFY supports `regression` mode; milestone completion requires a - clean regression run. -- A regression run against current code surfaces decay (fails closed). -- `tests/test_verify_regression_mode.py` passes. -- Existing diff-scoped VERIFY behavior preserved for non-regression - invocations. +### Task 3 — Stateless adapter rewrite (backend-engineer) ---- +Rewrite `adapters/terraform/adapter.py` (918 → ~80 lines): +- Delete `TYPE_MAP`, `INPUT_MAP`, `OUTPUT_MAP` (lines 26-94). +- Delete all 39 type-specific branches in `_emit_resource` + `_emit_igw`, `_container_definitions`, `_resource_block`, `_emit_output`. +- New `adapt(stack_instance, out_dir)`: + 1. `os.makedirs(out_dir, exist_ok=True)`. + 2. Read `registry.json` → build a `module_name → terraform_dir` map. + 3. For each resource in `stack_instance["resources"]`: + - Look up the module name from the resource's `module` field (e.g. `s3@1.0.0` → `s3`). + - Get `terraform_dir` from the registry. + - Emit a `module "" { source = "" ... }` block, passing each input as a module argument. For `ref:` values, emit `module "".` interpolations. Skip `region` (provider-level, not a module arg). + 4. Emit root `output {}` blocks wiring module outputs to stack outputs. + 5. Emit `providers.tf` (aws provider, region from first resource) + `terraform.tf` (required_version + required_providers + S3 backend, state key `spike/{stack_name}/terraform.tfstate` — env-aware key fix is P58, not this phase). +- The adapter owns NO resource shape, NO nested blocks, NO defaults, NO type-specific logic. It only assembles module instantiations and wires refs. -## Phase 53 — local-emulating-adapters +**Must-have:** +- `grep -n "TYPE_MAP\|INPUT_MAP\|OUTPUT_MAP\|rtype ==" adapters/terraform/adapter.py` returns nothing. +- `wc -l adapters/terraform/adapter.py` < 100. +- `python3 adapters/terraform/adapter.py modules/l1/s3/instance.json /tmp/p56a-test/` emits a root `main.tf` containing `module "s3" { source = "modules/l1/s3/terraform" ... }`. +- `terraform init + validate` passes in the emitted output dir (with the s3 module present). -**Requirements:** REQ-113 -**Personas:** backend-engineer (lead: adapters), data-engineer (flat-file -outbox), ci-verifier (review) -**Branch:** `phase/53-local-emulating-adapters` +### Task 4 — STANDARDS.md §8 rewrite (backend-engineer) -### Task 53.1 — Flat-file DynamoDB outbox emulator (REQ-113, data-engineer) -- A local adapter that writes evidence events to flat files in a temp - folder instead of DynamoDB. Same write/read interface as the live - DynamoDB outbox adapter. -- Verify: a contract submission through the local tier writes an - evidence event to the flat-file outbox with a valid hash chain. +Rewrite `modules/STANDARDS.md` §8 "Adapter Extension Pattern" (lines 448-504): +- Delete the "three tables" + "specialized emit branch" documentation (it described the drift). +- New §8: "Stateless Assembler Pattern". The adapter is a ~80-line assembler reading `terraform_dir` from the registry. Adding a new L1 = author the `terraform/` subdir (versions/variables/locals/main/outputs.tf) + register in `registry.json`. No adapter code changes. Document the file-split standard (versions/variables/locals/main/outputs) + the heavy-locals.tf convention for default interpolation. -### Task 53.2 — Local ECS emulator (REQ-113, backend-engineer) -- A local adapter that emulates ECS Fargate: records the service - definition, returns a synthetic HTTP 200 from a local shell process - instead of a real ECS service. Same interface as the live ECS adapter. -- Verify: the headline E2E against the local tier returns HTTP 200 from - the emulator. +**Must-have:** STANDARDS.md §8 documents the stateless assembler + per-module terraform dir standard. -### Task 53.3 — Local S3 state + Lambda stub (REQ-113, backend-engineer) -- Local S3 state backend (flat-file tfstate in temp folder) + local - Lambda stub (invokes the handler in-process, no AWS Lambda call). -- Verify: `terraform plan` runs against the local state backend; the - Lambda stub executes the contract-ingestion handler locally. +### Task 5 — test_adapter.py rewrite (backend-engineer) -### Success Criteria -- All three local adapters exist; the headline E2E runs end-to-end - against the local tier with no cloud credentials. -- `tests/test_local_emulating_adapters.py` passes. -- `run_platform.sh --local` (or equivalent) runs the full pipeline - locally. +Rewrite `tests/test_adapter.py` (667 lines → focused assembly assertions): +- Delete the HCL string-matching tests (TestTypeMap, TestInputMap, etc. — they test the deleted constants). +- New tests assert the adapter ASSEMBLES module instantiations: + - `test_adapt_emits_module_block` — given the s3 instance, the root main.tf contains `module "s3" { source = "modules/l1/s3/terraform" ... }`. + - `test_adapt_passes_inputs` — the module block passes `bucket_name` and `region` as arguments. + - `test_adapt_wires_refs` — given a 2-resource stack with a `ref:` input, the root main.tf wires `module "".` as the argument value. + - `test_adapt_emits_outputs` — root outputs wire module outputs to stack outputs. + - `test_adapt_emits_providers_and_terraform_tf` — providers.tf + terraform.tf present with correct region + backend. + - `test_no_type_specific_logic` — `grep` assertion: adapter.py contains no `TYPE_MAP`/`INPUT_MAP`/`OUTPUT_MAP`/`rtype ==`. + - `test_adapter_under_100_lines` — `wc -l` assertion. +- Keep the TestRegistry class (registry still has 14 entries; now with `terraform_dir`). ---- +**Must-have:** `pytest tests/test_adapter.py` passes (offline). -## Phase 54 — v1.1-v1.8 capability-reverification-sweep +### Task 6 — Offline regression (backend-engineer) -**Requirements:** REQ-114 -**Personas:** ci-verifier (lead: sweep), ci-debugger (in-sweep fixes), -backend-engineer (in-sweep fixes) -**Branch:** `phase/54-capability-reverification-sweep` +Run the full offline test suite to confirm no regression from the adapter rewrite: +- `pytest tests/ -m "not slow"` — all offline tests pass (the adapter change is the largest; other tests that import the adapter must still work). +- `bash scripts/run_primitive_plan.sh --check-only s3` — the plan-only pipeline still works against the new module-assembled output. -### Task 54.1 — Capability inventory (REQ-114, ci-verifier) -- Enumerate every capability advertised in v1.1→v1.8 PROJECT/ROADMAP: - IR + L1 + adapter, ECS Fargate E2E, contract ingestion Lambda, 3 - platform pipelines, CloudFront/WAF, uptime-kuma, decommission mode, - 8 P1 remediations, etc. Write the inventory to - `.ciagent/CAPABILITY_INVENTORY.md` with a unique ID per capability. +**Must-have:** offline pytest suite green + `run_primitive_plan.sh --check-only s3` exits 0. -### Task 54.2 — Re-verify each capability (REQ-114, ci-verifier + ci-debugger) -- Headline E2E: run both tiers (live AWS + local emulator). Both must - pass. -- All other capabilities: run the local tier via emulating adapters. -- Tag each capability Verified / Decayed / Broken in - `CAPABILITY_INVENTORY.md`. -- For each Decayed/Broken capability: fix in-sweep (D-090, no cap) until - Verified. Commit per capability: - `verify(P54): — Verified|Decayed|Broken` then - `fix(P54): — ` as needed. +## Success Criteria (phase gate) -### Success Criteria -- Every v1.1→v1.8 advertised capability is tagged Verified in - `CAPABILITY_INVENTORY.md`. (D-090: no cap; all must end Verified.) -- Headline E2E passes at both tiers. -- Regression run (Phase 52) is clean against the re-verified state. +1. `grep -n "TYPE_MAP\|INPUT_MAP\|OUTPUT_MAP\|rtype ==" adapters/terraform/adapter.py` returns nothing. +2. `wc -l adapters/terraform/adapter.py` < 100. +3. `modules/l1/s3/terraform/` passes `terraform init + validate` standalone. +4. Adapter, given the s3 instance, emits a root `main.tf` that `terraform init + validate` accepts. +5. `pytest tests/test_adapter.py` passes (offline). +6. Full offline pytest suite green (no regression). +7. `run_primitive_plan.sh --check-only s3` exits 0. +8. STANDARDS.md §8 documents the stateless assembler + per-module terraform dir standard. +9. `registry.json` s3 entry has `terraform_dir`. ---- +## Out of Scope (deferred to P56b–P58) -## Phase 55 — rewrite-to-verified-reality - -**Requirements:** REQ-115 -**Personas:** ci-doc-writer (lead: docs/decks), ci-doc-verifier (review) -**Branch:** `phase/55-rewrite-to-verified-reality` - -### Task 55.1 — Rewrite PROJECT/ROADMAP (REQ-115, ci-doc-writer) -- Add a "Capability Status (Re-Verified 2026-07-27)" section to - PROJECT.md listing every v1.1→v1.8 capability with its Verified tag - and the tier(s) tested. -- Add a decay disclosure: capabilities marked complete in v1.1–v1.8 ran - at the time of tagging; as of 2026-07-27 they were not reproducible - and were re-verified in v1.10. -- Update ROADMAP.md v1.9.x entries to note deck-freeze and - superseded-by-reverification status. - -### Task 55.2 — Rewrite decks (REQ-115, ci-doc-writer) -- Update both leadership decks so every capability claim reflects the - re-verified status. Remove any claim that cannot be demonstrated - live. -- Re-render HTML; upload PPTX to the v1.10.0 release. - -### Success Criteria -- PROJECT/ROADMAP/decks match `CAPABILITY_INVENTORY.md` exactly. -- `ci-doc-verifier` confirms no stale capability claims remain. -- Decks unfrozen; v1.10.0 tagged; Gitea release published. \ No newline at end of file +- Authoring the remaining 11 L1 module terraform subdirs (P56b). +- The `--apply`/`--destroy` shell lifecycle modes (P57). +- The single platform VPC + env-aware state key fix (P58). +- The modules-lifecycle pipeline (P59). +- Live AWS apply/modify/destroy (P60). \ No newline at end of file