From 772ac721b089e9d8020081483bf2f31686c7e406 Mon Sep 17 00:00:00 2001 From: Jon Chery Date: Mon, 27 Jul 2026 17:10:09 +0000 Subject: [PATCH] =?UTF-8?q?docs(P52):=20create=20v1.10=20milestone=20plan?= =?UTF-8?q?=20=E2=80=94=20pipeline=20regression=20fix=20+=20capability=20r?= =?UTF-8?q?e-verification?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ---ci--- project: acdl phase: 52 milestone: v1.10 status: plan decisions: [D-090, D-091, D-092, D-093, D-094] requirements: [REQ-112, REQ-113, REQ-114, REQ-115] ---/ci--- --- .ciagent/PLAN.md | 352 ++++++++++++++++++--------------------- .ciagent/PROJECT.md | 5 + .ciagent/REQUIREMENTS.md | 22 +++ .ciagent/ROADMAP.md | 64 +++++++ .ciagent/config.json | 4 +- 5 files changed, 252 insertions(+), 195 deletions(-) diff --git a/.ciagent/PLAN.md b/.ciagent/PLAN.md index 227d0e4..cf27d9a 100644 --- a/.ciagent/PLAN.md +++ b/.ciagent/PLAN.md @@ -1,228 +1,194 @@ --- -phase: 39-43 -name: v1.9-design-doc-interpolation-per-env-ci-stubs-p1-1 -milestone: v1.9 -requirements: [REQ-100, REQ-101, REQ-102, REQ-103, REQ-104, REQ-105, REQ-106, REQ-107, REQ-108, REQ-109, REQ-110, REQ-111] -type: feat/docs/fix +phase: 52-55 +name: v1.10-pipeline-regression-fix-and-capability-reverification +milestone: v1.10 +requirements: [REQ-112, REQ-113, REQ-114, REQ-115] +type: fix/test/docs --- -# ACDL v1.9 — Phase Plans +# ACDL v1.10 — Pipeline Regression Fix + Capability Re-Verification -> Milestone v1.9. Generated at PLAN stage. Autonomy: full. -> Requirements: REQ-100..REQ-111 (see REQUIREMENTS.md). -> Decisions: D-080..D-089 (see PROJECT.md + RESEARCH.md RA section). -> Versioning: feature milestone — progressive patch versions per phase -> (v1.8.1..v1.8.5), tag `v1.9.0` at milestone COMPLETE. +> Milestone v1.10. Generated at PLAN stage. Autonomy: full. +> Requirements: REQ-112..REQ-115 (see REQUIREMENTS.md). +> Decisions: D-090..D-094 (see PROJECT.md). +> Versioning: NFR/fix milestone — progressive patch versions per phase +> (v1.9.9..v1.9.12), tag `v1.10.0` at milestone COMPLETE (next minor; +> this is fix/test/docs, not a breaking schema change). + +## Context + +The CLARIFY/RESEARCH stages (this run, 2026-07-27) surfaced a structural +defect and a credibility gap: + +1. **VERIFY is diff-scoped (D-091).** The CIAgent VERIFY stage checks the + phase diff only; it never re-runs underlying platform capability. The + pipeline has no regression memory. As a result, 8 NFR-patch phases + (v1.9.1→v1.9.8, deck rework) passed VERIFY while the platform they + described decayed underneath them. +2. **Advertised capability is not currently reproducible.** The v1.2 ECS + Fargate E2E and v1.7 pipelines ran once historically (tags true at the + time) but are not reproducible today without revival work. The decks + present this capability as current without disclosing the decay. +3. **Decks froze critical-path work but were sequenced backwards.** Deck + rework (v1.9.1→v1.9.8) was justified by real incremental exec viewings, + but the feedback signal was mixed/ambiguous (thesis-not-landing + + demand-proof + needs-polish). The honest sequencing is re-verify → + rewrite docs/decks to match reality → polish. This was done backwards + for 8 phases. + +User decisions (this run): +- **D-090:** No cap on the re-verification sweep. Fix every advertised + capability in-sweep; all must end Verified. Unbounded-risk trade-off + accepted for full integrity. Decks stay frozen until the sweep completes. +- **D-091:** Add a regression-class VERIFY that re-runs capability checks + (not just diff checks), at minimum on milestone completion. +- **D-092:** Build local emulating adapters (flat-file outbox, local ECS + emulator, local S3 state, local Lambda stub) so the platform is fully + locally testable without cloud credentials. +- **D-093:** Re-verify every v1.1→v1.8 advertised capability. v1.0 demo + excluded as archived/superseded. Headline E2E runs both live-AWS and + local-emulator tiers (both must pass); all other capabilities run + locally via emulating adapters. +- **D-094:** Rewrite PROJECT/ROADMAP/decks to match verified reality; + decks unfrozen only after this lands. ## Wave ordering -- **Wave 1 (parallel, 2 tasks):** Phase 39 — design-doc refresh (security-engineer) + P1-1 adapter parameterization (platform-engineer). Disjoint file sets; no merge conflict. -- **Wave 2 (sequential):** Phase 40 — contract interpolation. Depends on Phase 39's design-doc context (lightweight). -- **Wave 3 (sequential):** Phase 41 — per-env CI jobs. Depends on Phase 40's interpolation + env schema. -- **Wave 4 (sequential):** Phase 42 — stub implementation. Depends on Phase 41's HITL job structure. -- **Wave 5 (sequential):** Phase 43 — verify + review + audit + complete. +- **Wave 1 (sequential):** Phase 52 — pipeline regression-VERIFY fix. + Must land first; the sweep runs through the fixed pipeline. +- **Wave 2 (sequential):** Phase 53 — local emulating adapters. The + sweep's local tier depends on these. +- **Wave 3 (sequential):** Phase 54 — v1.1→v1.8 capability re-verification + sweep. Fix in-sweep per D-090 (no cap). Tag each capability + Verified/Decayed/Broken; repair Decayed/Broken in-phase; all must end + Verified. +- **Wave 4 (sequential):** Phase 55 — rewrite PROJECT/ROADMAP/decks to + verified reality; unfreeze decks. --- -## Phase 39 — design-doc-refresh-and-p1-1-parameterization +## Phase 52 — pipeline-regression-verify-fix -**Requirements:** REQ-100, REQ-101, REQ-102 -**Personas:** security-engineer (lead: design docs), platform-engineer (lead: P1-1), backend-engineer (review) -**Branch:** `phase/39-design-doc-refresh-and-p1-1` +**Requirements:** REQ-112 +**Personas:** backend-engineer (lead: VERIFY stage), ci-verifier (review) +**Branch:** `phase/52-pipeline-regression-verify-fix` -### Task 39.1 — Refresh hitl_matrix_design.md (REQ-100, security-engineer) -- Rewrite the status block: "v1.2 wires the gates" → "v1.9 wires the gates (Phase 42)". -- Update "Spike scope note" → "v1.9 scope note": qa/prod/dr now exercised (Phase 41 wires the job structure; Phase 42 wires the attestation gates); dev remains autonomous. -- Update §10.4 matrix: mark the offline-testable concerns (contract NFRs, schema validity, policy pass) as **implemented in v1.9** (`core/attestation_matrix.py`); mark operator-supplied concerns as **accept signed evidence artifacts** (D-084). -- Add a "v1.9 wiring" section: cross-reference Phase 41's per-env jobs + Phase 42's `hitl_gates.py` + `attestation_matrix.py` + the outbox-based SoD check. -- Preserve D-042 (approver identity = `gitea.actor` / `github.actor`) — still accurate. -- Verify: `grep -i "dev-only spike" core/hitl_matrix_design.md` returns 0 hits; `grep -i "v1.2 wires" core/hitl_matrix_design.md` returns 0 hits. +### Task 52.1 — Add regression-class VERIFY (REQ-112, backend-engineer) +- Extend the VERIFY stage to support a `regression` mode that re-runs + capability checks (not just diff checks). Triggered at minimum on + milestone completion; may also be invoked per-phase when a phase + touches platform code (not docs-only NFR patches). +- The regression run executes the local-emulator tier (Phase 53) for + every capability marked Verified in prior milestones. Any capability + that fails the regression run blocks milestone completion. +- Record the regression result in `---ci---` blocks as + `regression: { capability: , status: Verified|Decayed|Broken }`. +- Verify: a regression run against the current codebase surfaces at + least one Decayed/Broken capability (proving the gate catches decay, + not just passes). -### Task 39.2 — Refresh audit_ledger_design.md (REQ-101, security-engineer) -- Mark the "Spike scope (D-041)" section as **shipped + production since v1.8** (hash chain + DynamoDB outbox + `acdl-evidence` mirror). -- Move the "v1.2 build-out" section (S3 Object Lock + JWS + async worker + DLQ + daily checkpoints) under a clearly-labeled "**Deferred to a future milestone (D-083)**" heading. Keep the content (it's the design for when it ships) but mark it not-v1.9. -- Update the RPO/RTO table: spike row → "v1.8+ (production): RPO=0 (sync outbox), RTO=workflow re-run"; v1.2 row → "Future milestone (D-083): RPO=0, RTO=DLQ replay". -- Update the outbox item shape: note `approver_qa`/`approver_prod`/`approver_dr` are populated by v1.9's `hitl_gates.attest` (Phase 42). -- Verify: `grep -i "Phases 08-10 implement" core/audit_ledger_design.md` returns 0 hits; the deferred section is clearly labeled. - -### Task 39.3 — P1-1 adapter parameterization (REQ-102, platform-engineer) -- `modules/l1/ecs-service/interface.json`: add inputs `desired_count` (integer, default 1), `launch_type` (string, default "FARGATE"), `family` (string, default "app"). -- `modules/l1/alb/interface.json`: add inputs `load_balancer_type` (string, default "application"), `target_type` (string, default "ip"). -- `modules/l1/vpc/interface.json`: add input `name` (string, default "app") for the VPC/IGW/RT `Name` tag prefix. -- `adapters/terraform/adapter.py`: change hardcoded defaults to `inputs.get("", "")` where the default matches the interface default (safety fallback; the resolver populates from the interface). Remove the hardcoded `Name = "acdl-microservice-rt"` (line 283) → use `inputs.get("name", "app")`-derived tag. -- Preserve the v1.1 S3 regression (S3 has none of these inputs → no change). -- Tests: `tests/test_p1_1_adapter_parameterization.py` — (a) `desired_count: 3` in contract inputs emits `desired_count = 3`; (b) absent `desired_count` emits `desired_count = 1` via interface default; (c) `target_type: "instance"` emits `target_type = "instance"`; (d) v1.1 S3 regression still passes (byte-identical `main.tf`). -- Verify: `pytest tests/test_p1_1_adapter_parameterization.py` passes; `run_platform.sh --check-only` exits 0; `pytest` total count increases; v1.1 S3 regression test passes. - -### Task 39.4 — Design doc test (REQ-100/101, backend-engineer) -- `tests/test_design_docs_current.py`: assert (a) no stale "dev-only spike" / "v1.2 wires the gates" / "Phases 08-10 implement" framing in either design doc; (b) `audit_ledger_design.md` has a "Deferred to a future milestone" section referencing D-083; (c) `hitl_matrix_design.md` references the v1.9 implementation (`attestation_matrix.py`, `hitl_gates.py`). -- Verify: `pytest tests/test_design_docs_current.py` passes. - -### Must-haves (Phase 39) -- [ ] `core/hitl_matrix_design.md` refreshed (no stale framing). -- [ ] `core/audit_ledger_design.md` refreshed (S3 Object Lock marked deferred D-083). -- [ ] Adapter has no hardcoded ECS/ALB/VPC defaults (read from inputs). -- [ ] `tests/test_p1_1_adapter_parameterization.py` + `tests/test_design_docs_current.py` pass. -- [ ] `run_ci.sh` exits 0; `run_platform.sh --check-only` exits 0; v1.1 S3 regression passes. +### Success Criteria +- VERIFY supports `regression` mode; milestone completion requires a + clean regression run. +- A regression run against current code surfaces decay (fails closed). +- `tests/test_verify_regression_mode.py` passes. +- Existing diff-scoped VERIFY behavior preserved for non-regression + invocations. --- -## Phase 40 — contract-interpolation +## Phase 53 — local-emulating-adapters -**Requirements:** REQ-103, REQ-104 -**Personas:** backend-engineer (lead), platform-engineer (review) -**Branch:** `phase/40-contract-interpolation` +**Requirements:** REQ-113 +**Personas:** backend-engineer (lead: adapters), data-engineer (flat-file +outbox), ci-verifier (review) +**Branch:** `phase/53-local-emulating-adapters` -### Task 40.1 — Environment JSON schema (REQ-104, backend-engineer) -- `schemas/environment.schema.json` (draft 2020-12): required `name` (string), `account_id` (string), `region` (string), `state_backend` (object: `bucket`, `lock_table`), `network` (object: `vpc_cidr`, `azs` array), `runner_role_arn` (string), `autonomy` (enum: full/attested), `confidence_threshold` (number). -- `core/environments/dev.json` validates against it. -- Add `core/environments/qa.json`, `prod.json`, `dr.json`: `account_id: "000000000000"`, `autonomy: "attested"`, `confidence_threshold` 0.75/0.90/0.95, regions us-east-1, state_backend buckets `acdl-qa-state`/`acdl-prod-state`/`acdl-dr-state`. -- `core/environment_check.py`: add `load(env_name, root=None)` returning the parsed env dict; `check()` stays. Add a stderr warning when `account_id == "000000000000"` and `env_name != "dev"` (prompts real binding). -- `tests/test_environment_schema.py`: all 4 env files validate; `load("dev")` returns the dict; warning emitted for qa/prod/dr placeholders. -- Verify: `pytest tests/test_environment_schema.py` passes. +### Task 53.1 — Flat-file DynamoDB outbox emulator (REQ-113, data-engineer) +- A local adapter that writes evidence events to flat files in a temp + folder instead of DynamoDB. Same write/read interface as the live + DynamoDB outbox adapter. +- Verify: a contract submission through the local tier writes an + evidence event to the flat-file outbox with a valid hash chain. -### Task 40.2 — Interpolation expansion in the resolver (REQ-103, backend-engineer) -- `core/contract_resolver.py`: add `_expand_vars(value, context)` — recursively walks dicts/lists/strings; replaces `${env.}` and `${contract.}` tokens by looking up the dotted path in the context dict. Unknown token → `ValueError(f"unresolved interpolation token: {token}")`. -- `resolve()`: after schema validation, load the env via `environment_check.load(contract["environment"])`, build `context = {"env": env, "contract": contract}`, expand all string values in `contract["inputs"]` (recursively, per D-087), then proceed to IR resolution. -- The expansion is post-schema-validation (schema sees the raw tokens, which are valid strings) and pre-IR-resolution (the resolver sees concrete values). -- `tests/test_interpolation.py`: (a) `${env.region}` expands to `us-east-1`; (b) `${env.state_backend.bucket}` expands to `acdl-dev-state`; (c) `${contract.module}` expands to `static-assets`; (d) unknown token raises `ValueError`; (e) nested map value `env: { DB_URL: "acdl-${env.environment}-db" }` expands recursively; (f) `resolve("contracts/static-assets.yaml")` succeeds with expanded values. -- Verify: `pytest tests/test_interpolation.py` passes. +### Task 53.2 — Local ECS emulator (REQ-113, backend-engineer) +- A local adapter that emulates ECS Fargate: records the service + definition, returns a synthetic HTTP 200 from a local shell process + instead of a real ECS service. Same interface as the live ECS adapter. +- Verify: the headline E2E against the local tier returns HTTP 200 from + the emulator. -### Task 40.3 — Sample contracts use naming patterns (REQ-103, backend-engineer) -- `contracts/static-assets.yaml`: `bucket_name: acdl-${env.environment}-${contract.module}-${env.account_id}-${env.region}` (the naming pattern the requirement calls out: region + account id + environment). -- `contracts/microservice.yaml`: same pattern for `bucket_name`. -- Keep `region: us-east-1` as a literal (or `${env.region}` — both valid; use `${env.region}` to demonstrate). -- `tests/test_sample_contracts_interpolate.py`: resolving the sample contracts produces concrete bucket names like `acdl-dev-static-assets-000000000000-us-east-1`. -- Verify: `pytest tests/test_sample_contracts_interpolate.py` passes; `run_platform.sh --check-only` exits 0 (resolver expands before adapter). +### Task 53.3 — Local S3 state + Lambda stub (REQ-113, backend-engineer) +- Local S3 state backend (flat-file tfstate in temp folder) + local + Lambda stub (invokes the handler in-process, no AWS Lambda call). +- Verify: `terraform plan` runs against the local state backend; the + Lambda stub executes the contract-ingestion handler locally. -### Must-haves (Phase 40) -- [ ] `schemas/environment.schema.json` exists; 4 env files validate. -- [ ] `_expand_vars` in resolver; unknown tokens raise. -- [ ] Sample contracts use `${env.*}` + `${contract.*}` naming patterns. -- [ ] `tests/test_environment_schema.py` + `tests/test_interpolation.py` + `tests/test_sample_contracts_interpolate.py` pass. -- [ ] `run_ci.sh` exits 0; `run_platform.sh --check-only` exits 0. +### Success Criteria +- All three local adapters exist; the headline E2E runs end-to-end + against the local tier with no cloud credentials. +- `tests/test_local_emulating_adapters.py` passes. +- `run_platform.sh --local` (or equivalent) runs the full pipeline + locally. --- -## Phase 41 — per-environment-ci-jobs +## Phase 54 — v1.1-v1.8 capability-reverification-sweep -**Requirements:** REQ-105, REQ-106 -**Personas:** backend-engineer (lead), security-engineer (HITL gate review) -**Branch:** `phase/41-per-environment-ci-jobs` +**Requirements:** REQ-114 +**Personas:** ci-verifier (lead: sweep), ci-debugger (in-sweep fixes), +backend-engineer (in-sweep fixes) +**Branch:** `phase/54-capability-reverification-sweep` -### Task 41.1 — Per-env contract files (REQ-105, backend-engineer) -- `contracts/static-assets.dev.yaml`, `.qa.yaml`, `.prod.yaml`, `.dr.yaml` — each sets `environment:` to its own name; `inputs.bucket_name` uses `${env.environment}-${contract.module}-${env.account_id}-${env.region}` interpolation (so the file content is near-identical; only `environment:` differs). -- `contracts/microservice.{dev,qa,prod,dr}.yaml` — same pattern. -- Keep `contracts/static-assets.yaml` + `contracts/microservice.yaml` as the dev default (backwards compat). -- `tests/test_per_env_contracts.py`: all 8 per-env files validate against `schemas/contract.schema.json`; each resolves to a stack with the correct environment. -- Verify: `pytest tests/test_per_env_contracts.py` passes. +### Task 54.1 — Capability inventory (REQ-114, ci-verifier) +- Enumerate every capability advertised in v1.1→v1.8 PROJECT/ROADMAP: + IR + L1 + adapter, ECS Fargate E2E, contract ingestion Lambda, 3 + platform pipelines, CloudFront/WAF, uptime-kuma, decommission mode, + 8 P1 remediations, etc. Write the inventory to + `.ciagent/CAPABILITY_INVENTORY.md` with a unique ID per capability. -### Task 41.2 — Deploy workflow `environment` input (REQ-106, backend-engineer) -- `.github/workflows/deploy.yml` + `.gitea/workflows/deploy.yml` (byte-identical): add `environment` input (`type: string`, default `""`, description "Target environment override (dev/qa/prod/dr); when empty, the contract's environment field is used"). -- `scripts/run_platform.sh`: add `--environment ` flag. When set, override the contract's `environment` field at load time (before schema validation per D-088, so interpolation context is consistent). Re-run the onboarding check against the supplied env. -- The workflow's "Run the platform pipeline" step passes `--environment ${{ inputs.environment }}` when non-empty. -- `tests/test_deploy_workflow_env_input.py`: both deploy workflows declare the `environment` input; byte-identical; `run_platform.sh --environment qa contracts/static-assets.yaml` produces a stack whose env is qa (tested via the resolver directly since run_platform.sh needs AWS for full mode — test the override logic in the resolver). -- `core/contract_resolver.py` `resolve()`: accept optional `environment_override` arg; when set, set `contract["environment"] = override` before schema validation + interpolation. -- Verify: `pytest tests/test_deploy_workflow_env_input.py` passes; both deploy workflows byte-identical. +### Task 54.2 — Re-verify each capability (REQ-114, ci-verifier + ci-debugger) +- Headline E2E: run both tiers (live AWS + local emulator). Both must + pass. +- All other capabilities: run the local tier via emulating adapters. +- Tag each capability Verified / Decayed / Broken in + `CAPABILITY_INVENTORY.md`. +- For each Decayed/Broken capability: fix in-sweep (D-090, no cap) until + Verified. Commit per capability: + `verify(P54): — Verified|Decayed|Broken` then + `fix(P54): — ` as needed. -### Task 41.3 — Per-env caller workflow docs + HITL gate structure (REQ-106, security-engineer review) -- `docs/CONSUMER_GUIDE.md`: add a "Per-environment deployment" section with 4 caller-workflow examples (`.github/workflows/deploy-dev.yml`, `deploy-qa.yml`, `deploy-prod.yml`, `deploy-dr.yml`), each `uses: acdl/.github/workflows/deploy.yml@v1.9` with `environment: ` + `contract: .acdl/..yaml`. Document: "Promotion = running the matching job; no `environment:` field editing." -- HITL gate structure (wired in Phase 42, documented here): qa/prod/dr caller workflows use `workflow_dispatch` with approval inputs (`approve_qa`, `approve_prod`, `approve_dr`) per `hitl_matrix_design.md` D-042; `gitea.actor` / `github.actor` is the approver of record. dev is autonomous (no gate). -- `tests/test_consumer_guide_per_env_section.py`: the consumer guide has the per-env section with 4 caller examples. -- Verify: `pytest tests/test_consumer_guide_per_env_section.py` passes. - -### Must-haves (Phase 41) -- [ ] 8 per-env contract files exist + validate + resolve. -- [ ] Deploy workflow has `environment` input (byte-identical Gitea + GitHub). -- [ ] `run_platform.sh --environment ` overrides; resolver supports `environment_override`. -- [ ] Consumer guide documents per-env caller workflows + promotion-without-editing. -- [ ] `tests/test_per_env_contracts.py` + `tests/test_deploy_workflow_env_input.py` + `tests/test_consumer_guide_per_env_section.py` pass. -- [ ] `run_ci.sh` exits 0; both deploy workflows byte-identical. +### Success Criteria +- Every v1.1→v1.8 advertised capability is tagged Verified in + `CAPABILITY_INVENTORY.md`. (D-090: no cap; all must end Verified.) +- Headline E2E passes at both tiers. +- Regression run (Phase 52) is clean against the re-verified state. --- -## Phase 42 — stub-implementation +## Phase 55 — rewrite-to-verified-reality -**Requirements:** REQ-107, REQ-108, REQ-109, REQ-110, REQ-111 -**Personas:** security-engineer (lead), backend-engineer (run_platform wiring), lambda-engineer (SNS topic Terraform) -**Branch:** `phase/42-stub-implementation` +**Requirements:** REQ-115 +**Personas:** ci-doc-writer (lead: docs/decks), ci-doc-verifier (review) +**Branch:** `phase/55-rewrite-to-verified-reality` -### Task 42.1 — route_halt_artifact real (REQ-107, security-engineer + lambda-engineer) -- `core/separation_of_duties.py` `route_halt_artifact`: when `ACDL_SOD_HALT_TOPIC_ARN` set, publish to SNS via boto3 (`sns.publish(TopicArn=arn, Message=..., Subject="ACDL SoD halt")`); when unset, fall back to structured stderr emission + a `SEPARATION_OF_DUTIES_VIOLATION` event write via `outbox_writer.write_event` (so the halt is in the audit chain). No silent print-only stub. -- `terraform/platform/main.tf`: add `aws_sns_topic.acdl-sod-halt` + a basic access policy (allow the platform Lambda / runner role to publish). Output the topic ARN. -- `tests/test_route_halt_artifact.py`: (a) with `ACDL_SOD_HALT_TOPIC_ARN` set, moto-mocked SNS receives the publish; (b) without it, a `SEPARATION_OF_DUTIES_VIOLATION` event is written to the outbox (moto-mocked DynamoDB); (c) stderr emission occurs in both cases. -- Verify: `pytest tests/test_route_halt_artifact.py` passes. +### Task 55.1 — Rewrite PROJECT/ROADMAP (REQ-115, ci-doc-writer) +- Add a "Capability Status (Re-Verified 2026-07-27)" section to + PROJECT.md listing every v1.1→v1.8 capability with its Verified tag + and the tier(s) tested. +- Add a decay disclosure: capabilities marked complete in v1.1–v1.8 ran + at the time of tagging; as of 2026-07-27 they were not reproducible + and were re-verified in v1.10. +- Update ROADMAP.md v1.9.x entries to note deck-freeze and + superseded-by-reverification status. -### Task 42.2 — HITL attestation gates (REQ-108, security-engineer + backend-engineer) -- `core/hitl_gates.py`: `attest(contract_id, env, approver, evidence, outbox_client=None)` → records `approver_qa`/`approver_prod`/`approver_dr` to the outbox item for `contract_id`; runs `separation_of_duties.check(outbox_client, contract_id, approver)` on prod; invokes the attestation matrix (Task 42.3) for the target env; returns `(ok, reason)`. Dev skips (returns `(True, "dev autonomous")`). -- `scripts/run_platform.sh`: before apply (for qa/prod/dr), call `hitl_gates.attest` with the approver from `GITHUB_ACTOR`/`GITEA_ACTOR` env. Block on `(ok=False)`. -- `tests/test_hitl_gates.py`: (a) dev skips; (b) qa records `approver_qa` (moto outbox); (c) prod records `approver_prod` + SoD blocks when `approver_qa == approver_prod`; (d) prod passes when approvers differ. -- Verify: `pytest tests/test_hitl_gates.py` passes. +### Task 55.2 — Rewrite decks (REQ-115, ci-doc-writer) +- Update both leadership decks so every capability claim reflects the + re-verified status. Remove any claim that cannot be demonstrated + live. +- Re-render HTML; upload PPTX to the v1.10.0 release. -### Task 42.3 — 8-concern attestation matrix (REQ-109, security-engineer) -- `core/attestation_matrix.py`: `check(env, evidence_bundle)` → runs the 8 concerns. Offline-testable concerns (contract NFRs, schema validity, policy pass) run for real. Operator-supplied concerns accept an uploaded signed evidence artifact (JSON with `timestamp`, `type`, `payload`, optional `signature`); validate freshness (within the declared window from `hitl_matrix_design.md` §10.4) + schema (per-concern). Signature verification via KMS when `ACDL_ATTESTATION_SIGNING_KEY_ID` set; skipped + logged when unset (D-089). Fail loud if missing/expired for prod/dr. -- `hitl_gates.attest` calls `attestation_matrix.check(env, evidence)` and blocks on any failing concern. -- `tests/test_attestation_matrix.py`: (a) offline concerns pass for a valid contract; (b) operator-supplied concern missing → block for prod; (c) operator-supplied concern present + fresh → pass; (d) expired artifact → block; (e) signature skip when key unset (logged). -- Verify: `pytest tests/test_attestation_matrix.py` passes. - -### Task 42.4 — Wiz real API client (REQ-110, security-engineer) -- `adapters/wiz/wiz_adapter.py`: add `WizClient` class — `__init__` reads `WIZ_API_TOKEN` + `WIZ_API_URL`; `fetch_issues(filter_by)` queries the Wiz GraphQL API (`/graphql`, Bearer auth, `issues` query). Translate results → `PolicyCheckResult` records (`engine: "wiz"`, `ruleId: `, `severity: `, `status: FAIL`, `message: `, `resource: <entity.name>`). Graceful degrade: when `WIZ_API_TOKEN` or `WIZ_API_URL` unset → emit the existing single `SKIPPED` `WIZ_NOT_CONFIGURED` record (no network call). Pagination handled via `pageInfo.hasNextPage`. -- `tests/test_wiz_adapter_real_client.py`: (a) with a recorded GraphQL fixture, `WizClient` translates issues → `PolicyCheckResult` records; (b) graceful degrade when env unset; (c) pagination follows `endCursor`. -- Verify: `pytest tests/test_wiz_adapter_real_client.py` passes. - -### Task 42.5 — Kyverno translator fleshed out (REQ-111, security-engineer) -- `adapters/kyverno/kyverno_adapter.py`: full `PolicyReport` → `PolicyCheckResult` mapping — handle `pass`/`fail`/`skip`/`warn` results, severity mapping (critical/high/medium/low/info), resource extraction, skip-with-reason handling. Keep the inactive-for-Terraform guard (emits a single `SKIPPED` `KYVERNO_INACTIVE_TF_STACK` record when no K8s manifests). Add a `--kube-version` stub (parsed but not yet used — for future GitOps). -- `tests/test_kyverno_adapter.py`: expand — (a) `pass` result → `PolicyCheckResult` with `status: PASS`; (b) `fail` with severity → correct severity mapping; (c) `skip` with reason → `SKIPPED` record; (d) inactive-for-TF guard emits the `KYVERNO_INACTIVE_TF_STACK` record. -- Verify: `pytest tests/test_kyverno_adapter.py` passes. - -### Must-haves (Phase 42) -- [ ] `route_halt_artifact` real (SNS + outbox fallback); SNS topic in Terraform. -- [ ] `hitl_gates.py` attests qa/prod/dr; SoD blocks on identity equality. -- [ ] `attestation_matrix.py` implements 8 concerns (offline-testable + signed artifacts). -- [ ] Wiz adapter real client + graceful degrade. -- [ ] Kyverno translator fleshed out + inactive guard preserved. -- [ ] All 5 new test files pass; `run_ci.sh` exits 0. - ---- - -## Phase 43 — verify-review-audit-complete - -**Requirements:** — (milestone gate) -**Personas:** lead-developer (lead), all personas (review participation) -**Branch:** `phase/43-verify-review-audit-complete` - -### Task 43.1 — 4-layer verify -- Structural: all new files present (environment.schema.json, 4 env files, 8 per-env contracts, hitl_gates.py, attestation_matrix.py, SNS topic in main.tf, 5+ new test files). -- Behavioral: `pytest` passes (count increases from v1.8's 350 by ~30+ new tests); `run_ci.sh` exits 0; `run_platform.sh --check-only` exits 0. -- Security: no hardcoded adapter defaults; HITL gates block on SoD violation; attestation matrix fails loud on missing evidence for prod/dr; Wiz degrades gracefully. -- Quality: each new feature has dedicated tests (interpolation, per-env jobs, SoD, HITL gates, attestation matrix, Wiz, Kyverno). - -### Task 43.2 — Multi-persona review -- `ciagent-review` across the v1.9 diff (phases 39–42). Auto-apply P0; flag P1+ for post-hoc. -- Reconstruct `.ciagent/REVIEW.md` with v1.9 content (D-086). Note that v1.3–v1.8 reviews were not persisted (no git-history rewrite). - -### Task 43.3 — Audit -- Reconstruction: git log matches `.ciagent/` files. -- File discipline: all `.ciagent/` files valid. -- Branch hygiene: stale branches cleaned. -- Commit discipline: all commits have `---ci---` blocks. - -### Task 43.4 — Complete -- Update `.ciagent/REQUIREMENTS.md`: mark REQ-100..REQ-111 complete; add v1.9 traceability table. -- Update `.ciagent/ROADMAP.md`: add v1.9 milestone section (complete). -- Update `.ciagent/PROJECT.md`: v1.9 status → complete. -- Tag `v1.9.0`; update floating `v1.9` + `v1` tags. -- Bump `uses:`/`ref:` from `@v1.6` → `@v1.9` in `contracts/*.yaml`, `deploy.yml` checkout `ref:`, `docs/CONSUMER_GUIDE.md` (D-071 successor). -- Commit: `docs(milestone): complete v1.9`. - -### Must-haves (Phase 43) -- [ ] 4-layer verify PASS. -- [ ] Review: 0 new P0; P1+ flagged for post-hoc. -- [ ] Audit: clean. -- [ ] Tag `v1.9.0` created; floating tags updated. -- [ ] `uses:`/`ref:` bumped to `@v1.9`. -- [ ] REQUIREMENTS.md + ROADMAP.md + PROJECT.md updated. - ---- - -*End of PLAN.md.* \ No newline at end of file +### Success Criteria +- PROJECT/ROADMAP/decks match `CAPABILITY_INVENTORY.md` exactly. +- `ci-doc-verifier` confirms no stale capability claims remain. +- Decks unfrozen; v1.10.0 tagged; Gitea release published. \ No newline at end of file diff --git a/.ciagent/PROJECT.md b/.ciagent/PROJECT.md index adbb32c..ed31b53 100644 --- a/.ciagent/PROJECT.md +++ b/.ciagent/PROJECT.md @@ -632,6 +632,11 @@ D-080+ to avoid collision with v1.8 research decisions D-073..D-077): | D-084 | 8-concern attestation matrix: offline-testable concerns (contract NFRs, schema validity, policy pass) run for real; operator-supplied concerns (k6 load test, DR drill, FinOps forecast) accept signed evidence artifacts validated for freshness + schema, failing loud if missing/expired for prod/dr. | The platform cannot run live load tests / DR drills / FinOps forecasts inline. Accepting signed evidence artifacts with freshness + schema validation is the regulatorily-defensible middle ground. | Phase 42 implements `core/attestation_matrix.py`. | | D-085 | P1-1 closure: adapter ECS/ALB/VPC hardcoded defaults (`desired_count = 1`, `launch_type = "FARGATE"`, `target_type = "ip"`, `load_balancer_type = "application"`, `family = "app"`, `Name = ...`) move into L1 `interface.json` inputs with defaults. The adapter reads inputs (falling back to interface defaults) and is a thin translator. | P1-1 was flagged in the v1.2 review (deferred to v1.3, never implemented). Defaults belong in the L1 interface, not the adapter. | Phase 39 closes P1-1. | | D-086 | `.ciagent/REVIEW.md` reconstructed at v1.9 complete; v1.3–v1.8 reviews noted as not-persisted (no git-history rewrite). | REVIEW.md still holds v1.2 content — later milestone reviews were not persisted or were overwritten. The v1.9 review overwrites it with current content; a note records the historical gap. | Phase 43 reconstructs REVIEW.md. | +| D-090 | No cap on the v1.1→v1.8 capability re-verification sweep. Fix every advertised capability in-sweep; all must end Verified. | The user rejected a phase cap. Unbounded-risk trade-off accepted for full integrity: decks stay frozen until every advertised capability is Verified. Recorded as a traceable decision, not silent scope creep. | Phase 54 executes the sweep under D-090. | +| D-091 | Add a regression-class VERIFY that re-runs capability checks (not just diff checks), at minimum on milestone completion. | VERIFY is currently diff-scoped (structural defect); 8 NFR-patch phases passed while the platform decayed. Without regression memory the pipeline cannot keep the sweep honest. | Phase 52 implements the regression-class VERIFY. | +| D-092 | Build local emulating adapters (flat-file outbox, local ECS emulator, local S3 state, local Lambda stub) so the platform is fully locally testable without cloud credentials. | Required for the sweep's local tier and for durable regression testing without AWS access. Cloud interactions are emulated with flat files in temp folders + local shell. | Phase 53 builds the local emulating adapters. | +| D-093 | Re-verify every v1.1→v1.8 advertised capability. v1.0 demo excluded as archived/superseded. Headline E2E runs both live-AWS and local-emulator tiers (both must pass); all other capabilities run locally via emulating adapters. | Tiered verification: live for cloud-backed headline, local for the rest. The bar is what an exec could see demonstrated. | Phase 54 executes the re-verification sweep. | +| D-094 | Rewrite PROJECT/ROADMAP/decks to match verified reality; decks unfrozen only after this lands. | Decks were sequenced backwards for 8 phases (polish before re-verify). The honest order is re-verify → rewrite → unfreeze. | Phase 55 rewrites docs/decks to verified reality. | ### CLARIFY auto-resolved parameters (full autonomy) diff --git a/.ciagent/REQUIREMENTS.md b/.ciagent/REQUIREMENTS.md index 82fa1d6..df1bcd6 100644 --- a/.ciagent/REQUIREMENTS.md +++ b/.ciagent/REQUIREMENTS.md @@ -264,6 +264,20 @@ - **REQ-110:** The Wiz adapter (`adapters/wiz/wiz_adapter.py`) is a real API client: a `WizClient` queries the Wiz GraphQL API (`WIZ_API_TOKEN` + `WIZ_API_URL`) and translates issues → `PolicyCheckResult` records. It degrades gracefully (existing `WIZ_NOT_CONFIGURED` SKIPPED record) when env unset. Offline tests use a recorded GraphQL fixture. - **REQ-111:** The Kyverno adapter (`adapters/kyverno/kyverno_adapter.py`) translator is fleshed out: full `PolicyReport` → `PolicyCheckResult` mapping with severity + skip handling. It remains inactive for Terraform-only stacks (guard preserved); a `--kube-version` stub is added for future GitOps. Sample policies already exist. +## v1.10 (active — pipeline regression fix + capability re-verification + verified-reality rewrite, tag `v1.10.0`) + +### Category: Pipeline Regression Fix +- **REQ-112:** The CIAgent VERIFY stage supports a `regression` mode that re-runs capability checks (not just diff checks), triggered at minimum on milestone completion. The regression run executes the local-emulator tier (REQ-113) for every capability marked Verified in prior milestones; any capability that fails the regression run blocks milestone completion. Regression results are recorded in `---ci---` blocks as `regression: { capability: <id>, status: Verified|Decayed|Broken }`. Existing diff-scoped VERIFY behavior is preserved for non-regression invocations. A regression run against the current codebase surfaces at least one Decayed/Broken capability (proving the gate catches decay, not just passes). `tests/test_verify_regression_mode.py` passes. + +### Category: Local Emulating Adapters +- **REQ-113:** Local emulating adapters exist so the platform is fully locally testable without cloud credentials: (a) a flat-file DynamoDB outbox adapter that writes evidence events to flat files in a temp folder with a valid hash chain, same write/read interface as the live DynamoDB outbox adapter; (b) a local ECS Fargate emulator that records the service definition and returns a synthetic HTTP 200 from a local shell process, same interface as the live ECS adapter; (c) a local S3 state backend (flat-file tfstate in a temp folder); (d) a local Lambda stub that invokes the handler in-process with no AWS Lambda call. The headline E2E (contract submission → service live → evidence event) runs end-to-end against the local tier with no cloud credentials. `tests/test_local_emulating_adapters.py` passes. `run_platform.sh --local` (or equivalent) runs the full pipeline locally. + +### Category: Capability Re-Verification Sweep +- **REQ-114:** Every capability advertised in v1.1→v1.8 PROJECT/ROADMAP is enumerated in `.ciagent/CAPABILITY_INVENTORY.md` with a unique ID per capability (v1.0 demo excluded as archived/superseded). Each capability is re-verified: the headline E2E (contract → ECS Fargate → evidence event) runs both live-AWS and local-emulator tiers, both must pass; all other capabilities run the local tier via emulating adapters (REQ-113). Each capability is tagged Verified / Decayed / Broken in `CAPABILITY_INVENTORY.md`. Every Decayed/Broken capability is fixed in-sweep (D-090: no cap) until Verified, with per-capability commits `verify(P54): <id> — <status>` and `fix(P54): <id> — <summary>`. All v1.1→v1.8 advertised capabilities end Verified. The regression run (REQ-112) is clean against the re-verified state. + +### Category: Verified-Reality Rewrite +- **REQ-115:** PROJECT.md, ROADMAP.md, and both leadership decks are rewritten to match `CAPABILITY_INVENTORY.md` exactly. PROJECT.md gains a "Capability Status (Re-Verified 2026-07-27)" section listing every v1.1→v1.8 capability with its Verified tag and the tier(s) tested, plus a decay disclosure: capabilities marked complete in v1.1–v1.8 ran at the time of tagging; as of 2026-07-27 they were not reproducible and were re-verified in v1.10. ROADMAP.md v1.9.x entries note deck-freeze and superseded-by-reverification status. Both leadership decks reflect the re-verified status; any claim that cannot be demonstrated live is removed. HTML is re-rendered; PPTX is uploaded to the v1.10.0 release. Decks are unfrozen only after this lands. `ci-doc-verifier` confirms no stale capability claims remain. v1.10.0 is tagged; the Gitea release is published. + ## Out of Scope (v1.9) | Feature | Reason | @@ -433,3 +447,11 @@ | REQ-109 | 42 | complete (v1.9.0) | | REQ-110 | 42 | complete (v1.9.0) | | REQ-111 | 42 | complete (v1.9.0) | +### v1.10 (active — pipeline regression fix + capability re-verification + verified-reality rewrite, tag `v1.10.0`) + +| Requirement | Phase | Status | +|-------------|-------|--------| +| REQ-112 | 52 | pending | +| REQ-113 | 53 | pending | +| REQ-114 | 54 | pending | +| REQ-115 | 55 | pending | diff --git a/.ciagent/ROADMAP.md b/.ciagent/ROADMAP.md index 7fd12ba..b3658e9 100644 --- a/.ciagent/ROADMAP.md +++ b/.ciagent/ROADMAP.md @@ -641,3 +641,67 @@ also closes P1-1 (adapter hardcoded defaults, deferred from v1.2). - Tag `v1.9.0` created; floating tags updated; `uses:` bumped to `@v1.9`. After Phase 43: COMPLETE gate — review → ship `v1.9.0` → audit. **DONE.** + +--- + +## v1.10 (active — pipeline regression fix + capability re-verification + verified-reality rewrite, tag `v1.10.0`) + +The v1.10 milestone corrects a structural defect and a credibility gap +surfaced in the 2026-07-27 CLARIFY/RESEARCH stages: + +1. **VERIFY is diff-scoped** — it checks the phase diff only, never + re-runs underlying capability. 8 NFR-patch phases (v1.9.1→v1.9.8) + passed VERIFY while the platform decayed underneath. +2. **Advertised capability is not currently reproducible** — v1.2 ECS + E2E and v1.7 pipelines ran once historically but decayed; decks + presented them as current without disclosing the decay. +3. **Deck work was sequenced backwards** — re-verify → rewrite → polish + is the honest order; v1.9.x did it backwards for 8 phases. + +User decisions: D-090 (no cap on sweep; fix everything; unbounded risk +accepted), D-091 (regression-class VERIFY), D-092 (local emulating +adapters), D-093 (re-verify v1.1→v1.8; v1.0 demo excluded), D-094 +(rewrite docs/decks to verified reality; unfreeze decks). + +### Phase 52 — pipeline-regression-verify-fix +- **Description:** Add a regression-class VERIFY that re-runs capability checks (not just diff checks), at minimum on milestone completion. Regression run executes the local-emulator tier for every capability marked Verified in prior milestones; any failure blocks milestone completion. Records `regression: { capability, status }` in `---ci---` blocks. +- **Status:** pending (v1.9.9) +- **Depends on:** — +- **Requirements:** REQ-112 +- **Success Criteria:** + - VERIFY supports `regression` mode; milestone completion requires a clean regression run. + - A regression run against current code surfaces decay (fails closed). + - `tests/test_verify_regression_mode.py` passes. + +### Phase 53 — local-emulating-adapters +- **Description:** Build local emulating adapters so the platform is fully locally testable without cloud credentials: flat-file DynamoDB outbox, local ECS emulator (synthetic HTTP 200 from local shell), local S3 state backend (flat-file tfstate), local Lambda stub (in-process handler invocation). Same interfaces as the live adapters. +- **Status:** pending (v1.9.10) +- **Depends on:** [52] +- **Requirements:** REQ-113 +- **Success Criteria:** + - All local adapters exist; headline E2E runs end-to-end against the local tier with no cloud credentials. + - `tests/test_local_emulating_adapters.py` passes. + - `run_platform.sh --local` runs the full pipeline locally. + +### Phase 54 — v1.1-v1.8 capability-reverification-sweep +- **Description:** Enumerate every capability advertised in v1.1→v1.8 PROJECT/ROADMAP to `.ciagent/CAPABILITY_INVENTORY.md`. Re-verify each: headline E2E at both tiers (live AWS + local emulator, both must pass); all other capabilities at the local tier via emulating adapters. Tag each Verified/Decayed/Broken. Fix every Decayed/Broken capability in-sweep (D-090: no cap; all must end Verified) until Verified. v1.0 demo excluded as archived/superseded. +- **Status:** pending (v1.9.11) +- **Depends on:** [53] +- **Requirements:** REQ-114 +- **Success Criteria:** + - Every v1.1→v1.8 advertised capability is tagged Verified in `CAPABILITY_INVENTORY.md`. + - Headline E2E passes at both tiers. + - Regression run (Phase 52) is clean against the re-verified state. + +### Phase 55 — rewrite-to-verified-reality +- **Description:** Rewrite PROJECT.md (add "Capability Status (Re-Verified 2026-07-27)" section + decay disclosure), ROADMAP.md (v1.9.x entries noted as deck-freeze / superseded-by-reverification), and both leadership decks so every capability claim reflects the re-verified status. Remove any claim that cannot be demonstrated live. Re-render HTML; upload PPTX to the v1.10.0 release. Decks unfrozen only after this lands. +- **Status:** pending (v1.9.12) +- **Depends on:** [54] +- **Requirements:** REQ-115 +- **Success Criteria:** + - PROJECT/ROADMAP/decks match `CAPABILITY_INVENTORY.md` exactly. + - `ci-doc-verifier` confirms no stale capability claims remain. + - Decks unfrozen; v1.10.0 tagged; Gitea release published. + +After Phase 55: COMPLETE gate — review → ship `v1.10.0` (next minor; +fix/test/docs, not a breaking schema change) → audit. diff --git a/.ciagent/config.json b/.ciagent/config.json index e2a3539..38e15ad 100644 --- a/.ciagent/config.json +++ b/.ciagent/config.json @@ -4,8 +4,8 @@ { "slug": "acdl", "name": "Agentic Cloud Delivery Platform", - "milestone": "v1.9", - "status": "complete" + "milestone": "v1.10", + "status": "active" } ], "active_project": "acdl",