acdl/.ciagent/PLAN.md

phase, name, milestone, requirements, type

phase	name	milestone	requirements	type
39-43	v1.9-design-doc-interpolation-per-env-ci-stubs-p1-1	v1.9	REQ-100 REQ-101 REQ-102 REQ-103 REQ-104 REQ-105 REQ-106 REQ-107 REQ-108 REQ-109 REQ-110 REQ-111	feat/docs/fix

ACDL v1.9 — Phase Plans

Milestone v1.9. Generated at PLAN stage. Autonomy: full. Requirements: REQ-100..REQ-111 (see REQUIREMENTS.md). Decisions: D-080..D-089 (see PROJECT.md + RESEARCH.md RA section). Versioning: feature milestone — progressive patch versions per phase (v1.8.1..v1.8.5), tag v1.9.0 at milestone COMPLETE.

Wave ordering

• Wave 1 (parallel, 2 tasks): Phase 39 — design-doc refresh (security-engineer) + P1-1 adapter parameterization (platform-engineer). Disjoint file sets; no merge conflict.
• Wave 2 (sequential): Phase 40 — contract interpolation. Depends on Phase 39's design-doc context (lightweight).
• Wave 3 (sequential): Phase 41 — per-env CI jobs. Depends on Phase 40's interpolation + env schema.
• Wave 4 (sequential): Phase 42 — stub implementation. Depends on Phase 41's HITL job structure.
• Wave 5 (sequential): Phase 43 — verify + review + audit + complete.

---

Phase 39 — design-doc-refresh-and-p1-1-parameterization

Requirements: REQ-100, REQ-101, REQ-102 Personas: security-engineer (lead: design docs), platform-engineer (lead: P1-1), backend-engineer (review) Branch: phase/39-design-doc-refresh-and-p1-1

Task 39.1 — Refresh hitl_matrix_design.md (REQ-100, security-engineer)

• Rewrite the status block: "v1.2 wires the gates" → "v1.9 wires the gates (Phase 42)".
• Update "Spike scope note" → "v1.9 scope note": qa/prod/dr now exercised (Phase 41 wires the job structure; Phase 42 wires the attestation gates); dev remains autonomous.
• Update §10.4 matrix: mark the offline-testable concerns (contract NFRs, schema validity, policy pass) as implemented in v1.9 (core/attestation_matrix.py); mark operator-supplied concerns as accept signed evidence artifacts (D-084).
• Add a "v1.9 wiring" section: cross-reference Phase 41's per-env jobs + Phase 42's hitl_gates.py + attestation_matrix.py + the outbox-based SoD check.
• Preserve D-042 (approver identity = gitea.actor / github.actor) — still accurate.
• Verify: grep -i "dev-only spike" core/hitl_matrix_design.md returns 0 hits; grep -i "v1.2 wires" core/hitl_matrix_design.md returns 0 hits.

Task 39.2 — Refresh audit_ledger_design.md (REQ-101, security-engineer)

• Mark the "Spike scope (D-041)" section as shipped + production since v1.8 (hash chain + DynamoDB outbox + acdl-evidence mirror).
• Move the "v1.2 build-out" section (S3 Object Lock + JWS + async worker + DLQ + daily checkpoints) under a clearly-labeled "Deferred to a future milestone (D-083)" heading. Keep the content (it's the design for when it ships) but mark it not-v1.9.
• Update the RPO/RTO table: spike row → "v1.8+ (production): RPO=0 (sync outbox), RTO=workflow re-run"; v1.2 row → "Future milestone (D-083): RPO=0, RTO=DLQ replay".
• Update the outbox item shape: note approver_qa/approver_prod/approver_dr are populated by v1.9's hitl_gates.attest (Phase 42).
• Verify: grep -i "Phases 08-10 implement" core/audit_ledger_design.md returns 0 hits; the deferred section is clearly labeled.

Task 39.3 — P1-1 adapter parameterization (REQ-102, platform-engineer)

• modules/l1/ecs-service/interface.json: add inputs desired_count (integer, default 1), launch_type (string, default "FARGATE"), family (string, default "app").
• modules/l1/alb/interface.json: add inputs load_balancer_type (string, default "application"), target_type (string, default "ip").
• modules/l1/vpc/interface.json: add input name (string, default "app") for the VPC/IGW/RT Name tag prefix.
• adapters/terraform/adapter.py: change hardcoded defaults to inputs.get("<name>", "<default>") where the default matches the interface default (safety fallback; the resolver populates from the interface). Remove the hardcoded Name = "acdl-microservice-rt" (line 283) → use inputs.get("name", "app")-derived tag.
• Preserve the v1.1 S3 regression (S3 has none of these inputs → no change).
• Tests: tests/test_p1_1_adapter_parameterization.py — (a) desired_count: 3 in contract inputs emits desired_count = 3; (b) absent desired_count emits desired_count = 1 via interface default; (c) target_type: "instance" emits target_type = "instance"; (d) v1.1 S3 regression still passes (byte-identical main.tf).
• Verify: pytest tests/test_p1_1_adapter_parameterization.py passes; run_platform.sh --check-only exits 0; pytest total count increases; v1.1 S3 regression test passes.

Task 39.4 — Design doc test (REQ-100/101, backend-engineer)

• tests/test_design_docs_current.py: assert (a) no stale "dev-only spike" / "v1.2 wires the gates" / "Phases 08-10 implement" framing in either design doc; (b) audit_ledger_design.md has a "Deferred to a future milestone" section referencing D-083; (c) hitl_matrix_design.md references the v1.9 implementation (attestation_matrix.py, hitl_gates.py).
• Verify: pytest tests/test_design_docs_current.py passes.

Must-haves (Phase 39)

• core/hitl_matrix_design.md refreshed (no stale framing).
• core/audit_ledger_design.md refreshed (S3 Object Lock marked deferred D-083).
• Adapter has no hardcoded ECS/ALB/VPC defaults (read from inputs).
• tests/test_p1_1_adapter_parameterization.py + tests/test_design_docs_current.py pass.
• run_ci.sh exits 0; run_platform.sh --check-only exits 0; v1.1 S3 regression passes.

---

Phase 40 — contract-interpolation

Requirements: REQ-103, REQ-104 Personas: backend-engineer (lead), platform-engineer (review) Branch: phase/40-contract-interpolation

Task 40.1 — Environment JSON schema (REQ-104, backend-engineer)

• schemas/environment.schema.json (draft 2020-12): required name (string), account_id (string), region (string), state_backend (object: bucket, lock_table), network (object: vpc_cidr, azs array), runner_role_arn (string), autonomy (enum: full/attested), confidence_threshold (number).
• core/environments/dev.json validates against it.
• Add core/environments/qa.json, prod.json, dr.json: account_id: "000000000000", autonomy: "attested", confidence_threshold 0.75/0.90/0.95, regions us-east-1, state_backend buckets acdl-qa-state/acdl-prod-state/acdl-dr-state.
• core/environment_check.py: add load(env_name, root=None) returning the parsed env dict; check() stays. Add a stderr warning when account_id == "000000000000" and env_name != "dev" (prompts real binding).
• tests/test_environment_schema.py: all 4 env files validate; load("dev") returns the dict; warning emitted for qa/prod/dr placeholders.
• Verify: pytest tests/test_environment_schema.py passes.

Task 40.2 — Interpolation expansion in the resolver (REQ-103, backend-engineer)

• core/contract_resolver.py: add _expand_vars(value, context) — recursively walks dicts/lists/strings; replaces ${env.<dotted.path>} and ${contract.<dotted.path>} tokens by looking up the dotted path in the context dict. Unknown token → ValueError(f"unresolved interpolation token: {token}").
• resolve(): after schema validation, load the env via environment_check.load(contract["environment"]), build context = {"env": env, "contract": contract}, expand all string values in contract["inputs"] (recursively, per D-087), then proceed to IR resolution.
• The expansion is post-schema-validation (schema sees the raw tokens, which are valid strings) and pre-IR-resolution (the resolver sees concrete values).
• tests/test_interpolation.py: (a) ${env.region} expands to us-east-1; (b) ${env.state_backend.bucket} expands to acdl-dev-state; (c) ${contract.module} expands to static-assets; (d) unknown token raises ValueError; (e) nested map value env: { DB_URL: "acdl-${env.environment}-db" } expands recursively; (f) resolve("contracts/static-assets.yaml") succeeds with expanded values.
• Verify: pytest tests/test_interpolation.py passes.

Task 40.3 — Sample contracts use naming patterns (REQ-103, backend-engineer)

• contracts/static-assets.yaml: bucket_name: acdl-${env.environment}-${contract.module}-${env.account_id}-${env.region} (the naming pattern the requirement calls out: region + account id + environment).
• contracts/microservice.yaml: same pattern for bucket_name.
• Keep region: us-east-1 as a literal (or ${env.region} — both valid; use ${env.region} to demonstrate).
• tests/test_sample_contracts_interpolate.py: resolving the sample contracts produces concrete bucket names like acdl-dev-static-assets-000000000000-us-east-1.
• Verify: pytest tests/test_sample_contracts_interpolate.py passes; run_platform.sh --check-only exits 0 (resolver expands before adapter).

Must-haves (Phase 40)

• schemas/environment.schema.json exists; 4 env files validate.
• _expand_vars in resolver; unknown tokens raise.
• Sample contracts use ${env.*} + ${contract.*} naming patterns.
• tests/test_environment_schema.py + tests/test_interpolation.py + tests/test_sample_contracts_interpolate.py pass.
• run_ci.sh exits 0; run_platform.sh --check-only exits 0.

---

Phase 41 — per-environment-ci-jobs

Requirements: REQ-105, REQ-106 Personas: backend-engineer (lead), security-engineer (HITL gate review) Branch: phase/41-per-environment-ci-jobs

Task 41.1 — Per-env contract files (REQ-105, backend-engineer)

• contracts/static-assets.dev.yaml, .qa.yaml, .prod.yaml, .dr.yaml — each sets environment: to its own name; inputs.bucket_name uses ${env.environment}-${contract.module}-${env.account_id}-${env.region} interpolation (so the file content is near-identical; only environment: differs).
• contracts/microservice.{dev,qa,prod,dr}.yaml — same pattern.
• Keep contracts/static-assets.yaml + contracts/microservice.yaml as the dev default (backwards compat).
• tests/test_per_env_contracts.py: all 8 per-env files validate against schemas/contract.schema.json; each resolves to a stack with the correct environment.
• Verify: pytest tests/test_per_env_contracts.py passes.

Task 41.2 — Deploy workflow environment input (REQ-106, backend-engineer)

• .github/workflows/deploy.yml + .gitea/workflows/deploy.yml (byte-identical): add environment input (type: string, default "", description "Target environment override (dev/qa/prod/dr); when empty, the contract's environment field is used").
• scripts/run_platform.sh: add --environment <name> flag. When set, override the contract's environment field at load time (before schema validation per D-088, so interpolation context is consistent). Re-run the onboarding check against the supplied env.
• The workflow's "Run the platform pipeline" step passes --environment ${{ inputs.environment }} when non-empty.
• tests/test_deploy_workflow_env_input.py: both deploy workflows declare the environment input; byte-identical; run_platform.sh --environment qa contracts/static-assets.yaml produces a stack whose env is qa (tested via the resolver directly since run_platform.sh needs AWS for full mode — test the override logic in the resolver).
• core/contract_resolver.py resolve(): accept optional environment_override arg; when set, set contract["environment"] = override before schema validation + interpolation.
• Verify: pytest tests/test_deploy_workflow_env_input.py passes; both deploy workflows byte-identical.

Task 41.3 — Per-env caller workflow docs + HITL gate structure (REQ-106, security-engineer review)

• docs/CONSUMER_GUIDE.md: add a "Per-environment deployment" section with 4 caller-workflow examples (.github/workflows/deploy-dev.yml, deploy-qa.yml, deploy-prod.yml, deploy-dr.yml), each uses: acdl/.github/workflows/deploy.yml@v1.9 with environment: <env> + contract: .acdl/<module>.<env>.yaml. Document: "Promotion = running the matching job; no environment: field editing."
• HITL gate structure (wired in Phase 42, documented here): qa/prod/dr caller workflows use workflow_dispatch with approval inputs (approve_qa, approve_prod, approve_dr) per hitl_matrix_design.md D-042; gitea.actor / github.actor is the approver of record. dev is autonomous (no gate).
• tests/test_consumer_guide_per_env_section.py: the consumer guide has the per-env section with 4 caller examples.
• Verify: pytest tests/test_consumer_guide_per_env_section.py passes.

Must-haves (Phase 41)

• 8 per-env contract files exist + validate + resolve.
• Deploy workflow has environment input (byte-identical Gitea + GitHub).
• run_platform.sh --environment <name> overrides; resolver supports environment_override.
• Consumer guide documents per-env caller workflows + promotion-without-editing.
• tests/test_per_env_contracts.py + tests/test_deploy_workflow_env_input.py + tests/test_consumer_guide_per_env_section.py pass.
• run_ci.sh exits 0; both deploy workflows byte-identical.

---

Phase 42 — stub-implementation

Requirements: REQ-107, REQ-108, REQ-109, REQ-110, REQ-111 Personas: security-engineer (lead), backend-engineer (run_platform wiring), lambda-engineer (SNS topic Terraform) Branch: phase/42-stub-implementation

Task 42.1 — route_halt_artifact real (REQ-107, security-engineer + lambda-engineer)

• core/separation_of_duties.py route_halt_artifact: when ACDL_SOD_HALT_TOPIC_ARN set, publish to SNS via boto3 (sns.publish(TopicArn=arn, Message=..., Subject="ACDL SoD halt")); when unset, fall back to structured stderr emission + a SEPARATION_OF_DUTIES_VIOLATION event write via outbox_writer.write_event (so the halt is in the audit chain). No silent print-only stub.
• terraform/platform/main.tf: add aws_sns_topic.acdl-sod-halt + a basic access policy (allow the platform Lambda / runner role to publish). Output the topic ARN.
• tests/test_route_halt_artifact.py: (a) with ACDL_SOD_HALT_TOPIC_ARN set, moto-mocked SNS receives the publish; (b) without it, a SEPARATION_OF_DUTIES_VIOLATION event is written to the outbox (moto-mocked DynamoDB); (c) stderr emission occurs in both cases.
• Verify: pytest tests/test_route_halt_artifact.py passes.

Task 42.2 — HITL attestation gates (REQ-108, security-engineer + backend-engineer)

• core/hitl_gates.py: attest(contract_id, env, approver, evidence, outbox_client=None) → records approver_qa/approver_prod/approver_dr to the outbox item for contract_id; runs separation_of_duties.check(outbox_client, contract_id, approver) on prod; invokes the attestation matrix (Task 42.3) for the target env; returns (ok, reason). Dev skips (returns (True, "dev autonomous")).
• scripts/run_platform.sh: before apply (for qa/prod/dr), call hitl_gates.attest with the approver from GITHUB_ACTOR/GITEA_ACTOR env. Block on (ok=False).
• tests/test_hitl_gates.py: (a) dev skips; (b) qa records approver_qa (moto outbox); (c) prod records approver_prod + SoD blocks when approver_qa == approver_prod; (d) prod passes when approvers differ.
• Verify: pytest tests/test_hitl_gates.py passes.

Task 42.3 — 8-concern attestation matrix (REQ-109, security-engineer)

• core/attestation_matrix.py: check(env, evidence_bundle) → runs the 8 concerns. Offline-testable concerns (contract NFRs, schema validity, policy pass) run for real. Operator-supplied concerns accept an uploaded signed evidence artifact (JSON with timestamp, type, payload, optional signature); validate freshness (within the declared window from hitl_matrix_design.md §10.4) + schema (per-concern). Signature verification via KMS when ACDL_ATTESTATION_SIGNING_KEY_ID set; skipped + logged when unset (D-089). Fail loud if missing/expired for prod/dr.
• hitl_gates.attest calls attestation_matrix.check(env, evidence) and blocks on any failing concern.
• tests/test_attestation_matrix.py: (a) offline concerns pass for a valid contract; (b) operator-supplied concern missing → block for prod; (c) operator-supplied concern present + fresh → pass; (d) expired artifact → block; (e) signature skip when key unset (logged).
• Verify: pytest tests/test_attestation_matrix.py passes.

Task 42.4 — Wiz real API client (REQ-110, security-engineer)

• adapters/wiz/wiz_adapter.py: add WizClient class — __init__ reads WIZ_API_TOKEN + WIZ_API_URL; fetch_issues(filter_by) queries the Wiz GraphQL API (<url>/graphql, Bearer auth, issues query). Translate results → PolicyCheckResult records (engine: "wiz", ruleId: <control.name>, severity: <lowercased>, status: FAIL, message: <title>, resource: <entity.name>). Graceful degrade: when WIZ_API_TOKEN or WIZ_API_URL unset → emit the existing single SKIPPED WIZ_NOT_CONFIGURED record (no network call). Pagination handled via pageInfo.hasNextPage.
• tests/test_wiz_adapter_real_client.py: (a) with a recorded GraphQL fixture, WizClient translates issues → PolicyCheckResult records; (b) graceful degrade when env unset; (c) pagination follows endCursor.
• Verify: pytest tests/test_wiz_adapter_real_client.py passes.

Task 42.5 — Kyverno translator fleshed out (REQ-111, security-engineer)

• adapters/kyverno/kyverno_adapter.py: full PolicyReport → PolicyCheckResult mapping — handle pass/fail/skip/warn results, severity mapping (critical/high/medium/low/info), resource extraction, skip-with-reason handling. Keep the inactive-for-Terraform guard (emits a single SKIPPED KYVERNO_INACTIVE_TF_STACK record when no K8s manifests). Add a --kube-version stub (parsed but not yet used — for future GitOps).
• tests/test_kyverno_adapter.py: expand — (a) pass result → PolicyCheckResult with status: PASS; (b) fail with severity → correct severity mapping; (c) skip with reason → SKIPPED record; (d) inactive-for-TF guard emits the KYVERNO_INACTIVE_TF_STACK record.
• Verify: pytest tests/test_kyverno_adapter.py passes.

Must-haves (Phase 42)

• route_halt_artifact real (SNS + outbox fallback); SNS topic in Terraform.
• hitl_gates.py attests qa/prod/dr; SoD blocks on identity equality.
• attestation_matrix.py implements 8 concerns (offline-testable + signed artifacts).
• Wiz adapter real client + graceful degrade.
• Kyverno translator fleshed out + inactive guard preserved.
• All 5 new test files pass; run_ci.sh exits 0.

---

Phase 43 — verify-review-audit-complete

Requirements: — (milestone gate) Personas: lead-developer (lead), all personas (review participation) Branch: phase/43-verify-review-audit-complete

Task 43.1 — 4-layer verify

• Structural: all new files present (environment.schema.json, 4 env files, 8 per-env contracts, hitl_gates.py, attestation_matrix.py, SNS topic in main.tf, 5+ new test files).
• Behavioral: pytest passes (count increases from v1.8's 350 by ~30+ new tests); run_ci.sh exits 0; run_platform.sh --check-only exits 0.
• Security: no hardcoded adapter defaults; HITL gates block on SoD violation; attestation matrix fails loud on missing evidence for prod/dr; Wiz degrades gracefully.
• Quality: each new feature has dedicated tests (interpolation, per-env jobs, SoD, HITL gates, attestation matrix, Wiz, Kyverno).

Task 43.2 — Multi-persona review

• ciagent-review across the v1.9 diff (phases 39–42). Auto-apply P0; flag P1+ for post-hoc.
• Reconstruct .ciagent/REVIEW.md with v1.9 content (D-086). Note that v1.3–v1.8 reviews were not persisted (no git-history rewrite).

Task 43.3 — Audit

• Reconstruction: git log matches .ciagent/ files.
• File discipline: all .ciagent/ files valid.
• Branch hygiene: stale branches cleaned.
• Commit discipline: all commits have ---ci--- blocks.

Task 43.4 — Complete

• Update .ciagent/REQUIREMENTS.md: mark REQ-100..REQ-111 complete; add v1.9 traceability table.
• Update .ciagent/ROADMAP.md: add v1.9 milestone section (complete).
• Update .ciagent/PROJECT.md: v1.9 status → complete.
• Tag v1.9.0; update floating v1.9 + v1 tags.
• Bump uses:/ref: from @v1.6 → @v1.9 in contracts/*.yaml, deploy.yml checkout ref:, docs/CONSUMER_GUIDE.md (D-071 successor).
• Commit: docs(milestone): complete v1.9.

Must-haves (Phase 43)

• 4-layer verify PASS.
• Review: 0 new P0; P1+ flagged for post-hoc.
• Audit: clean.
• Tag v1.9.0 created; floating tags updated.
• uses:/ref: bumped to @v1.9.
• REQUIREMENTS.md + ROADMAP.md + PROJECT.md updated.

---

End of PLAN.md.