docs(P56a): create v1.11 RESTART phase plan — stateless adapter + s3 reference
PLAN stage. P56a is the first phase of the v1.11 restart: rewrite the 918-line adapter monolith to a ~80-line stateless assembler, prove the design with the s3 reference module. 6 tasks, single wave (no parallelization — one cohesive change): 1. s3 reference terraform module (versions/variables/locals/main/outputs) 2. Registry extension (terraform_dir field) 3. Stateless adapter rewrite (delete TYPE_MAP/INPUT_MAP/OUTPUT_MAP + 39 branches) 4. STANDARDS.md §8 rewrite (stateless assembler pattern) 5. test_adapter.py rewrite (assert assembly, not HCL strings) 6. Offline regression (pytest + run_primitive_plan.sh --check-only s3) Success gate: adapter < 100 lines, no type-specific logic, s3 module validates standalone, adapter-emitted root main.tf validates, offline suite green. ---ci--- project: acdl phase: P56a milestone: v1.11 status: plan ---/ci---
This commit is contained in:
Jon Chery
+92
-164
@@ -1,194 +1,122 @@
|
||||
---
|
||||
phase: 52-55
|
||||
name: v1.10-pipeline-regression-fix-and-capability-reverification
|
||||
milestone: v1.10
|
||||
requirements: [REQ-112, REQ-113, REQ-114, REQ-115]
|
||||
type: fix/test/docs
|
||||
phase: P56a
|
||||
name: stateless-adapter-rewrite
|
||||
milestone: v1.11
|
||||
requirements: [REQ-123]
|
||||
wave: 1
|
||||
---
|
||||
|
||||
# ACDL v1.10 — Pipeline Regression Fix + Capability Re-Verification
|
||||
# P56a — Stateless Adapter Rewrite + s3 Reference Module
|
||||
|
||||
> Milestone v1.10. Generated at PLAN stage. Autonomy: full.
|
||||
> Requirements: REQ-112..REQ-115 (see REQUIREMENTS.md).
|
||||
> Decisions: D-090..D-094 (see PROJECT.md).
|
||||
> Versioning: NFR/fix milestone — progressive patch versions per phase
|
||||
> (v1.9.9..v1.9.12), tag `v1.10.0` at milestone COMPLETE (next minor;
|
||||
> this is fix/test/docs, not a breaking schema change).
|
||||
**Phase:** P56a
|
||||
**Milestone:** v1.11 (RESTART)
|
||||
**Requirement:** REQ-123
|
||||
**Wave:** 1 (foundation — sequential prerequisite for P56b–P58)
|
||||
**Branch:** `milestone/v1.11-restart` → `phase/p56a-stateless-adapter`
|
||||
|
||||
## Context
|
||||
## Goal
|
||||
|
||||
The CLARIFY/RESEARCH stages (this run, 2026-07-27) surfaced a structural
|
||||
defect and a credibility gap:
|
||||
Rewrite `adapters/terraform/adapter.py` from a 918-line monolith (3 constant
|
||||
tables + 39 type-specific branches) to a ~80-line stateless assembler. Author
|
||||
`modules/l1/s3/terraform/` as the reference module proving the assembly path
|
||||
end-to-end. Extend the registry. Rewrite STANDARDS.md §8. Rewrite
|
||||
`tests/test_adapter.py` to assert assembly, not HCL string matching.
|
||||
|
||||
1. **VERIFY is diff-scoped (D-091).** The CIAgent VERIFY stage checks the
|
||||
phase diff only; it never re-runs underlying platform capability. The
|
||||
pipeline has no regression memory. As a result, 8 NFR-patch phases
|
||||
(v1.9.1→v1.9.8, deck rework) passed VERIFY while the platform they
|
||||
described decayed underneath them.
|
||||
2. **Advertised capability is not currently reproducible.** The v1.2 ECS
|
||||
Fargate E2E and v1.7 pipelines ran once historically (tags true at the
|
||||
time) but are not reproducible today without revival work. The decks
|
||||
present this capability as current without disclosing the decay.
|
||||
3. **Decks froze critical-path work but were sequenced backwards.** Deck
|
||||
rework (v1.9.1→v1.9.8) was justified by real incremental exec viewings,
|
||||
but the feedback signal was mixed/ambiguous (thesis-not-landing +
|
||||
demand-proof + needs-polish). The honest sequencing is re-verify →
|
||||
rewrite docs/decks to match reality → polish. This was done backwards
|
||||
for 8 phases.
|
||||
This phase proves the stateless adapter design with ONE module (s3) before
|
||||
P56b authors the remaining 11. If the assembly path works for s3, it works
|
||||
for all L1s (the adapter is generic; only the module terraform differs).
|
||||
|
||||
User decisions (this run):
|
||||
- **D-090:** No cap on the re-verification sweep. Fix every advertised
|
||||
capability in-sweep; all must end Verified. Unbounded-risk trade-off
|
||||
accepted for full integrity. Decks stay frozen until the sweep completes.
|
||||
- **D-091:** Add a regression-class VERIFY that re-runs capability checks
|
||||
(not just diff checks), at minimum on milestone completion.
|
||||
- **D-092:** Build local emulating adapters (flat-file outbox, local ECS
|
||||
emulator, local S3 state, local Lambda stub) so the platform is fully
|
||||
locally testable without cloud credentials.
|
||||
- **D-093:** Re-verify every v1.1→v1.8 advertised capability. v1.0 demo
|
||||
excluded as archived/superseded. Headline E2E runs both live-AWS and
|
||||
local-emulator tiers (both must pass); all other capabilities run
|
||||
locally via emulating adapters.
|
||||
- **D-094:** Rewrite PROJECT/ROADMAP/decks to match verified reality;
|
||||
decks unfrozen only after this lands.
|
||||
## Vertical Slice — Single Wave (no parallelization; one cohesive change)
|
||||
|
||||
## Wave ordering
|
||||
### Task 1 — s3 reference terraform module (data-engineer)
|
||||
|
||||
- **Wave 1 (sequential):** Phase 52 — pipeline regression-VERIFY fix.
|
||||
Must land first; the sweep runs through the fixed pipeline.
|
||||
- **Wave 2 (sequential):** Phase 53 — local emulating adapters. The
|
||||
sweep's local tier depends on these.
|
||||
- **Wave 3 (sequential):** Phase 54 — v1.1→v1.8 capability re-verification
|
||||
sweep. Fix in-sweep per D-090 (no cap). Tag each capability
|
||||
Verified/Decayed/Broken; repair Decayed/Broken in-phase; all must end
|
||||
Verified.
|
||||
- **Wave 4 (sequential):** Phase 55 — rewrite PROJECT/ROADMAP/decks to
|
||||
verified reality; unfreeze decks.
|
||||
Author `modules/l1/s3/terraform/`:
|
||||
- `versions.tf` — `terraform { required_version = ">= 1.9, < 1.10"; required_providers { aws = { source = "hashicorp/aws", version = "~> 5.0" } } }`
|
||||
- `variables.tf` — one `variable {}` per s3 interface.json input: `bucket_name` (string, required), `region` (string, required), `kms_key_arn` (string, optional, default null). Plus `tags` (map(string), optional, default {}) for tag merge.
|
||||
- `locals.tf` — heavy interpolation:
|
||||
- `sse_algorithm` — "aws:kms" if `var.kms_key_arn != null`, else "AES256" (SSE-S3). This is the default the adapter currently hardcodes (adapter.py:225-228, 289-297).
|
||||
- `tags` — merge `var.tags` with `acdl:owner`/`acdl:environment` defaults.
|
||||
- `main.tf` — `resource "aws_s3_bucket" "this" { bucket = var.bucket_name; tags = local.tags }` + `resource "aws_s3_bucket_versioning" "this" { bucket = aws_s3_bucket.this.id; versioning { enabled = true } }` + `resource "aws_s3_bucket_server_side_encryption_configuration" "this" { bucket = aws_s3_bucket.this.id; rule { apply_server_side_encryption_by_default { sse_algorithm = local.sse_algorithm; kms_master_key_id = var.kms_key_arn } } }` (conditional on kms_key_arn).
|
||||
- `outputs.tf` — `output "bucket_arn" { value = aws_s3_bucket.this.arn }`, `output "bucket_name" { value = aws_s3_bucket.this.id }`, `output "bucket_regional_domain_name" { value = aws_s3_bucket.this.bucket_regional_domain_name }`.
|
||||
|
||||
---
|
||||
**Must-have:** `terraform init + validate` passes standalone in `modules/l1/s3/terraform/`.
|
||||
|
||||
## Phase 52 — pipeline-regression-verify-fix
|
||||
### Task 2 — Registry extension (backend-engineer)
|
||||
|
||||
**Requirements:** REQ-112
|
||||
**Personas:** backend-engineer (lead: VERIFY stage), ci-verifier (review)
|
||||
**Branch:** `phase/52-pipeline-regression-verify-fix`
|
||||
Update `modules/registry.json`: add `"terraform_dir": "modules/l1/s3/terraform"` to the s3 1.0.0 entry. This is the field the adapter reads to find the module source path.
|
||||
|
||||
### Task 52.1 — Add regression-class VERIFY (REQ-112, backend-engineer)
|
||||
- Extend the VERIFY stage to support a `regression` mode that re-runs
|
||||
capability checks (not just diff checks). Triggered at minimum on
|
||||
milestone completion; may also be invoked per-phase when a phase
|
||||
touches platform code (not docs-only NFR patches).
|
||||
- The regression run executes the local-emulator tier (Phase 53) for
|
||||
every capability marked Verified in prior milestones. Any capability
|
||||
that fails the regression run blocks milestone completion.
|
||||
- Record the regression result in `---ci---` blocks as
|
||||
`regression: { capability: <id>, status: Verified|Decayed|Broken }`.
|
||||
- Verify: a regression run against the current codebase surfaces at
|
||||
least one Decayed/Broken capability (proving the gate catches decay,
|
||||
not just passes).
|
||||
**Must-have:** `registry.json` validates as JSON; the s3 entry has `terraform_dir`.
|
||||
|
||||
### Success Criteria
|
||||
- VERIFY supports `regression` mode; milestone completion requires a
|
||||
clean regression run.
|
||||
- A regression run against current code surfaces decay (fails closed).
|
||||
- `tests/test_verify_regression_mode.py` passes.
|
||||
- Existing diff-scoped VERIFY behavior preserved for non-regression
|
||||
invocations.
|
||||
### Task 3 — Stateless adapter rewrite (backend-engineer)
|
||||
|
||||
---
|
||||
Rewrite `adapters/terraform/adapter.py` (918 → ~80 lines):
|
||||
- Delete `TYPE_MAP`, `INPUT_MAP`, `OUTPUT_MAP` (lines 26-94).
|
||||
- Delete all 39 type-specific branches in `_emit_resource` + `_emit_igw`, `_container_definitions`, `_resource_block`, `_emit_output`.
|
||||
- New `adapt(stack_instance, out_dir)`:
|
||||
1. `os.makedirs(out_dir, exist_ok=True)`.
|
||||
2. Read `registry.json` → build a `module_name → terraform_dir` map.
|
||||
3. For each resource in `stack_instance["resources"]`:
|
||||
- Look up the module name from the resource's `module` field (e.g. `s3@1.0.0` → `s3`).
|
||||
- Get `terraform_dir` from the registry.
|
||||
- Emit a `module "<rid>" { source = "<terraform_dir>" ... }` block, passing each input as a module argument. For `ref:` values, emit `module "<ref_rid>".<output>` interpolations. Skip `region` (provider-level, not a module arg).
|
||||
4. Emit root `output {}` blocks wiring module outputs to stack outputs.
|
||||
5. Emit `providers.tf` (aws provider, region from first resource) + `terraform.tf` (required_version + required_providers + S3 backend, state key `spike/{stack_name}/terraform.tfstate` — env-aware key fix is P58, not this phase).
|
||||
- The adapter owns NO resource shape, NO nested blocks, NO defaults, NO type-specific logic. It only assembles module instantiations and wires refs.
|
||||
|
||||
## Phase 53 — local-emulating-adapters
|
||||
**Must-have:**
|
||||
- `grep -n "TYPE_MAP\|INPUT_MAP\|OUTPUT_MAP\|rtype ==" adapters/terraform/adapter.py` returns nothing.
|
||||
- `wc -l adapters/terraform/adapter.py` < 100.
|
||||
- `python3 adapters/terraform/adapter.py modules/l1/s3/instance.json /tmp/p56a-test/` emits a root `main.tf` containing `module "s3" { source = "modules/l1/s3/terraform" ... }`.
|
||||
- `terraform init + validate` passes in the emitted output dir (with the s3 module present).
|
||||
|
||||
**Requirements:** REQ-113
|
||||
**Personas:** backend-engineer (lead: adapters), data-engineer (flat-file
|
||||
outbox), ci-verifier (review)
|
||||
**Branch:** `phase/53-local-emulating-adapters`
|
||||
### Task 4 — STANDARDS.md §8 rewrite (backend-engineer)
|
||||
|
||||
### Task 53.1 — Flat-file DynamoDB outbox emulator (REQ-113, data-engineer)
|
||||
- A local adapter that writes evidence events to flat files in a temp
|
||||
folder instead of DynamoDB. Same write/read interface as the live
|
||||
DynamoDB outbox adapter.
|
||||
- Verify: a contract submission through the local tier writes an
|
||||
evidence event to the flat-file outbox with a valid hash chain.
|
||||
Rewrite `modules/STANDARDS.md` §8 "Adapter Extension Pattern" (lines 448-504):
|
||||
- Delete the "three tables" + "specialized emit branch" documentation (it described the drift).
|
||||
- New §8: "Stateless Assembler Pattern". The adapter is a ~80-line assembler reading `terraform_dir` from the registry. Adding a new L1 = author the `terraform/` subdir (versions/variables/locals/main/outputs.tf) + register in `registry.json`. No adapter code changes. Document the file-split standard (versions/variables/locals/main/outputs) + the heavy-locals.tf convention for default interpolation.
|
||||
|
||||
### Task 53.2 — Local ECS emulator (REQ-113, backend-engineer)
|
||||
- A local adapter that emulates ECS Fargate: records the service
|
||||
definition, returns a synthetic HTTP 200 from a local shell process
|
||||
instead of a real ECS service. Same interface as the live ECS adapter.
|
||||
- Verify: the headline E2E against the local tier returns HTTP 200 from
|
||||
the emulator.
|
||||
**Must-have:** STANDARDS.md §8 documents the stateless assembler + per-module terraform dir standard.
|
||||
|
||||
### Task 53.3 — Local S3 state + Lambda stub (REQ-113, backend-engineer)
|
||||
- Local S3 state backend (flat-file tfstate in temp folder) + local
|
||||
Lambda stub (invokes the handler in-process, no AWS Lambda call).
|
||||
- Verify: `terraform plan` runs against the local state backend; the
|
||||
Lambda stub executes the contract-ingestion handler locally.
|
||||
### Task 5 — test_adapter.py rewrite (backend-engineer)
|
||||
|
||||
### Success Criteria
|
||||
- All three local adapters exist; the headline E2E runs end-to-end
|
||||
against the local tier with no cloud credentials.
|
||||
- `tests/test_local_emulating_adapters.py` passes.
|
||||
- `run_platform.sh --local` (or equivalent) runs the full pipeline
|
||||
locally.
|
||||
Rewrite `tests/test_adapter.py` (667 lines → focused assembly assertions):
|
||||
- Delete the HCL string-matching tests (TestTypeMap, TestInputMap, etc. — they test the deleted constants).
|
||||
- New tests assert the adapter ASSEMBLES module instantiations:
|
||||
- `test_adapt_emits_module_block` — given the s3 instance, the root main.tf contains `module "s3" { source = "modules/l1/s3/terraform" ... }`.
|
||||
- `test_adapt_passes_inputs` — the module block passes `bucket_name` and `region` as arguments.
|
||||
- `test_adapt_wires_refs` — given a 2-resource stack with a `ref:` input, the root main.tf wires `module "<rid>".<output>` as the argument value.
|
||||
- `test_adapt_emits_outputs` — root outputs wire module outputs to stack outputs.
|
||||
- `test_adapt_emits_providers_and_terraform_tf` — providers.tf + terraform.tf present with correct region + backend.
|
||||
- `test_no_type_specific_logic` — `grep` assertion: adapter.py contains no `TYPE_MAP`/`INPUT_MAP`/`OUTPUT_MAP`/`rtype ==`.
|
||||
- `test_adapter_under_100_lines` — `wc -l` assertion.
|
||||
- Keep the TestRegistry class (registry still has 14 entries; now with `terraform_dir`).
|
||||
|
||||
---
|
||||
**Must-have:** `pytest tests/test_adapter.py` passes (offline).
|
||||
|
||||
## Phase 54 — v1.1-v1.8 capability-reverification-sweep
|
||||
### Task 6 — Offline regression (backend-engineer)
|
||||
|
||||
**Requirements:** REQ-114
|
||||
**Personas:** ci-verifier (lead: sweep), ci-debugger (in-sweep fixes),
|
||||
backend-engineer (in-sweep fixes)
|
||||
**Branch:** `phase/54-capability-reverification-sweep`
|
||||
Run the full offline test suite to confirm no regression from the adapter rewrite:
|
||||
- `pytest tests/ -m "not slow"` — all offline tests pass (the adapter change is the largest; other tests that import the adapter must still work).
|
||||
- `bash scripts/run_primitive_plan.sh --check-only s3` — the plan-only pipeline still works against the new module-assembled output.
|
||||
|
||||
### Task 54.1 — Capability inventory (REQ-114, ci-verifier)
|
||||
- Enumerate every capability advertised in v1.1→v1.8 PROJECT/ROADMAP:
|
||||
IR + L1 + adapter, ECS Fargate E2E, contract ingestion Lambda, 3
|
||||
platform pipelines, CloudFront/WAF, uptime-kuma, decommission mode,
|
||||
8 P1 remediations, etc. Write the inventory to
|
||||
`.ciagent/CAPABILITY_INVENTORY.md` with a unique ID per capability.
|
||||
**Must-have:** offline pytest suite green + `run_primitive_plan.sh --check-only s3` exits 0.
|
||||
|
||||
### Task 54.2 — Re-verify each capability (REQ-114, ci-verifier + ci-debugger)
|
||||
- Headline E2E: run both tiers (live AWS + local emulator). Both must
|
||||
pass.
|
||||
- All other capabilities: run the local tier via emulating adapters.
|
||||
- Tag each capability Verified / Decayed / Broken in
|
||||
`CAPABILITY_INVENTORY.md`.
|
||||
- For each Decayed/Broken capability: fix in-sweep (D-090, no cap) until
|
||||
Verified. Commit per capability:
|
||||
`verify(P54): <capability-id> — Verified|Decayed|Broken` then
|
||||
`fix(P54): <capability-id> — <fix-summary>` as needed.
|
||||
## Success Criteria (phase gate)
|
||||
|
||||
### Success Criteria
|
||||
- Every v1.1→v1.8 advertised capability is tagged Verified in
|
||||
`CAPABILITY_INVENTORY.md`. (D-090: no cap; all must end Verified.)
|
||||
- Headline E2E passes at both tiers.
|
||||
- Regression run (Phase 52) is clean against the re-verified state.
|
||||
1. `grep -n "TYPE_MAP\|INPUT_MAP\|OUTPUT_MAP\|rtype ==" adapters/terraform/adapter.py` returns nothing.
|
||||
2. `wc -l adapters/terraform/adapter.py` < 100.
|
||||
3. `modules/l1/s3/terraform/` passes `terraform init + validate` standalone.
|
||||
4. Adapter, given the s3 instance, emits a root `main.tf` that `terraform init + validate` accepts.
|
||||
5. `pytest tests/test_adapter.py` passes (offline).
|
||||
6. Full offline pytest suite green (no regression).
|
||||
7. `run_primitive_plan.sh --check-only s3` exits 0.
|
||||
8. STANDARDS.md §8 documents the stateless assembler + per-module terraform dir standard.
|
||||
9. `registry.json` s3 entry has `terraform_dir`.
|
||||
|
||||
---
|
||||
## Out of Scope (deferred to P56b–P58)
|
||||
|
||||
## Phase 55 — rewrite-to-verified-reality
|
||||
|
||||
**Requirements:** REQ-115
|
||||
**Personas:** ci-doc-writer (lead: docs/decks), ci-doc-verifier (review)
|
||||
**Branch:** `phase/55-rewrite-to-verified-reality`
|
||||
|
||||
### Task 55.1 — Rewrite PROJECT/ROADMAP (REQ-115, ci-doc-writer)
|
||||
- Add a "Capability Status (Re-Verified 2026-07-27)" section to
|
||||
PROJECT.md listing every v1.1→v1.8 capability with its Verified tag
|
||||
and the tier(s) tested.
|
||||
- Add a decay disclosure: capabilities marked complete in v1.1–v1.8 ran
|
||||
at the time of tagging; as of 2026-07-27 they were not reproducible
|
||||
and were re-verified in v1.10.
|
||||
- Update ROADMAP.md v1.9.x entries to note deck-freeze and
|
||||
superseded-by-reverification status.
|
||||
|
||||
### Task 55.2 — Rewrite decks (REQ-115, ci-doc-writer)
|
||||
- Update both leadership decks so every capability claim reflects the
|
||||
re-verified status. Remove any claim that cannot be demonstrated
|
||||
live.
|
||||
- Re-render HTML; upload PPTX to the v1.10.0 release.
|
||||
|
||||
### Success Criteria
|
||||
- PROJECT/ROADMAP/decks match `CAPABILITY_INVENTORY.md` exactly.
|
||||
- `ci-doc-verifier` confirms no stale capability claims remain.
|
||||
- Decks unfrozen; v1.10.0 tagged; Gitea release published.
|
||||
- Authoring the remaining 11 L1 module terraform subdirs (P56b).
|
||||
- The `--apply`/`--destroy` shell lifecycle modes (P57).
|
||||
- The single platform VPC + env-aware state key fix (P58).
|
||||
- The modules-lifecycle pipeline (P59).
|
||||
- Live AWS apply/modify/destroy (P60).
|
||||
Reference in New Issue
Block a user