acdl/.ciagent/ROADMAP.md

# ACDL — Roadmap

## Overview

- **v1.0 (demo):** complete — tag `v1.1.0`, 2026-07-21. All 5 phases shipped + audited PASS.
- **v1.1 (complete):** architecture finalization + v1 spike. 5 phases (06–10). Tag `v1.2.0`, 2026-07-21. All 5 phases shipped + verified; review READY TO SHIP (0 P0); audit CLEAN. Gitea release id 202.
- **v1.2 (active):** platform hardening + first real consumer deployment. 6 phases (11–16). Ship tag `v1.3.0`.
- **v1.0 demo URL:** https://git.cloudinit.dev/continuous-intelligence/acdl-evidence/raw/branch/main/index.html

---

## v1.0 (Prior — the demo, complete)

Five-phase breakdown that took ACDL from empty repo to a reproducible 4-act
executive demo. Milestone `v1.0-initial` covered the full demo build. Each
phase produced a runnable increment and ended with a phase-completion commit
+ tag. All phases complete; demo archived to `demo/` in v1.1 Phase 06.

## Phases

### Phase 01 — repo-scaffolding
- **Description:** Create the three repos under `continuous-intelligence` (`acdl-contracts`, `acdl-evidence`; `acdl` already exists), seed directory layouts, configure Pages on `acdl-evidence`, add environment protection for `qa` and `prod` on `acdl-contracts`.
- **Status:** complete (v1.0.1)
- **Depends on:** —
- **Requirements:** REQ-01, REQ-09, REQ-10
- **Success Criteria:**
  - `acdl-contracts` and `acdl-evidence` exist and are pushable.
  - `acdl-evidence` Pages returns 200 with placeholder `index.html`.
  - `qa` and `prod` environments exist on `acdl-contracts`.

### Phase 02 — l1-modules
- **Description:** Create all 8 L1 module folders under `acdl/modules/l1/`, each with `manifest.yaml` (declared inputs) and `mock_apply.sh` (uniform echo + 1s sleep + exit 0).
- **Status:** complete (v1.0.2)
- **Depends on:** [1]
- **Requirements:** REQ-02, REQ-03
- **Success Criteria:**
  - All 8 L1s present; `mock_apply.sh` runs and exits 0 for each.
  - `manifest.yaml` validates against the L1 schema.

### Phase 03 — l2-modules-and-core-scripts
- **Description:** Create the 4 L2 compositions under `acdl/modules/l2/` referencing L1s, plus the 5 core scripts in `acdl/scripts/` (`mock_executor.sh`, `policy_checker.py`, `confidence_signal.py`, `evidence_writer.py`, `l3b_agent_stub.py`).
- **Status:** complete (v1.0.3)
- **Depends on:** [2]
- **Requirements:** REQ-04, REQ-05, REQ-06, REQ-07
- **Success Criteria:**
  - `mock_executor.sh` applies each L1 in an L2 and writes `state.json`.
  - `policy_checker.py` fails on `public-ingress: true` with `POLICY_VIOLATION:PUBLIC_INGRESS`.
  - `confidence_signal.py` returns 0.90 (pass) / 0.40 (fail).
  - `evidence_writer.py` appends an event with a valid hash chain.
  - `l3b_agent_stub.py` maps the Act 3 example issue to `l2-commodity-price-feed`.

### Phase 04 — pipeline-and-approval-gates
- **Description:** Build the reusable pipeline workflow in `acdl/.gitea/workflows/` (Dev → QA → Prod → Finalize) plus the issue-triggered L3B workflow in `acdl-contracts/.gitea/workflows/`. Wire environment protection for QA and Prod.
- **Status:** complete (v1.0.4)
- **Depends on:** [3]
- **Requirements:** REQ-08, REQ-09, REQ-10, REQ-12
- **Success Criteria:**
  - Pushing a valid `contract.yaml` runs Dev automatically and pauses at QA.
  - Approving QA moves to Prod; approving Prod finalizes.
  - Opening an Issue with the Act 3 text generates a `contract.yaml` commit and triggers the pipeline.

### Phase 05 — evidence-ui-and-demo-dry-run
- **Description:** Build `index.html` (vanilla JS, fetches `audit.json`, renders timeline) and run all four acts end-to-end as a dry run.
- **Status:** complete (v1.0.5)
- **Depends on:** [4]
- **Requirements:** REQ-11, REQ-13, REQ-14, REQ-15
- **Success Criteria:**
  - Pages timeline renders events from `audit.json`.
  - Act 2: valid contract passes through all gates; timeline shows the full flow.
  - Act 3: Issue text produces the expected `l2-commodity-price-feed` contract and triggers the pipeline.
  - Act 4: malicious `public-ingress: true` contract halts in Dev with confidence < 0.50 and a visible rejection reason on the timeline.

---

## v1.1 (Complete — architecture finalization + v1 spike, 2026-07-21, tag `v1.2.0`)

Five-phase breakdown to finalize the architecture to v1.0 and prove the
locked commitments with one end-to-end implementation spike. Milestone
`v1.1-spike` covered the real platform's first materialization. Ship tag
at milestone COMPLETE: **`v1.2.0`** (feature milestone, next minor per
ship.md). **Status: COMPLETE — all 5 phases shipped (v1.1.1..v1.1.5) +
verified; review READY TO SHIP (0 P0); audit CLEAN; Gitea release id 202.
D-034 closed (root key deactivated by user).**

### Phase 06 — archive-demo-and-reorient
- **Description:** Move the v1.0 demo (`modules/`, `scripts/`, `evidence-ui/`, `contracts/`, demo `.gitea/workflows/`) to `demo/`. Establish the new repo layout (`platform/`, `schemas/`, `adapters/`, `terraform/`, `modules-ir/`). Rewrite README to reflect the real platform. Verify the demo still runs from `demo/` (regression check).
- **Status:** complete (v1.1.1)
- **Depends on:** —
- **Requirements:** (no new REQ; repo hygiene)
- **Success Criteria:**
  - `demo/` contains the full v1.0 demo; `demo/scripts/run_demo.sh --no-upload` still exits 0.
  - New top-level dirs exist and are empty-but-scaffolded: `platform/`, `schemas/`, `adapters/`, `terraform/`, `modules-ir/`.
  - README reflects the real platform (vision + architecture links, new layout).

### Phase 07 — architecture-v1-finalization
- **Description:** Resolve the 11 open decisions in `docs/architecture.md` §13 (already recorded in `PROJECT.md`). Author the locked schemas + designs: `schemas/ir.schema.json` (REQ-17), `schemas/policy_check_result.schema.json` (REQ-18), `schemas/contract.schema.json` (REQ-22), `platform/confidence_signal.py` spec (REQ-19), `platform/audit_ledger_design.md` (REQ-20), `platform/hitl_matrix_design.md` (REQ-21). Mark architecture v1.0.
- **Status:** complete (v1.1.2)
- **Depends on:** [06]
- **Requirements:** REQ-16, REQ-17, REQ-18, REQ-19, REQ-20, REQ-21, REQ-22
- **Success Criteria:**
  - All 11 open decisions resolved and recorded in `PROJECT.md`.
  - All 6 schema/design files exist and validate (`ajv` / `python -m jsonschema`).
  - `docs/architecture.md` status note updated to v1.0 (or a `docs/architecture-v1.0.md` snapshot).

### Phase 08 — aws-oidc-bootstrap
- **Description:** **Re-scoped per RESEARCH TARGET 1 + D-039.** Gitea Actions does not support `id-token: write` (conf 0.95), so real OIDC is deferred to v1.2. This phase instead: uses the temporary long-lived key (waiver D-034) once to create an S3 state bucket, a DynamoDB lock/outbox table, and an IAM user with a minimal scoped policy (S3 + DynamoDB + plan-only); stores the key as a Gitea Actions secret; implements `scripts/rotate_spike_key.sh` to rotate the key after each spike run. Real OIDC federation is tracked via go-gitea/gitea#36988 for v1.2.
- **Status:** complete (v1.1.3)
- **Depends on:** [07]
- **Requirements:** REQ-23 (re-interpreted: AWS auth bootstrap + state backend; OIDC deferred to v1.2 per D-039)
- **Success Criteria:**
  - S3 state bucket + DynamoDB lock/outbox table exist.
  - An IAM user with a minimal scoped policy exists; its access key is stored as a Gitea Actions secret.
  - `scripts/rotate_spike_key.sh` rotates the key (deactivates old, creates new, updates the secret) and is idempotent.
  - A workflow step authenticates to AWS with the rotated secret and runs `aws sts get-caller-identity` successfully.
  - D-034 is closed: the bootstrap long-lived key is rotated/deactivated (logged in `PROJECT.md`).

### Phase 09 — v1-spike-ir-and-l1-and-adapter
- **Description:** Implement the Target Stack IR, one real L1 `l1-s3` (IR-typed interface, registered), and the Terraform adapter that compiles the IR → Terraform `variable`/`output` + root module and emits a real `terraform plan` against AWS (via the rotated-key secret per D-039; OIDC is v1.2). State in S3 + DynamoDB.
- **Status:** complete (v1.1.4)
- **Depends on:** [08]
- **Requirements:** REQ-24, REQ-26
- **Success Criteria:**
  - `schemas/ir.schema.json` is satisfied by `modules-ir/l1/l1-s3/` interface.
  - The Terraform adapter translates `l1-s3` to a valid `terraform plan` (real AWS).
  - `terraform validate` + `terraform plan` succeed; no long-lived credential in the workflow.

### Phase 10 — v1-spike-l2-and-contract-e2e
- **Description:** Implement `l2-static-asset` (thin-composition referencing `l1-s3`), the contract schema + contract→IR resolution, and one end-to-end contract submission (`contracts/spike.yaml` for `l2-static-asset`) flowing through schema validation → IR resolution → `terraform plan` → Checkov `PolicyCheckResult` → confidence signal → evidence event to the DynamoDB outbox. Verify the IR commitments hold (no polyglot mess).
- **Status:** complete (v1.1.5)
- **Depends on:** [09]
- **Requirements:** REQ-25, REQ-27, REQ-28
- **Success Criteria:**
  - `l2-static-asset` references `l1-s3` only (depth 1).
  - One contract submission completes the full pipeline end-to-end.
  - `scripts/verify_phase10.sh` proves the adapter is the only substrate-specific code.
  - Evidence event is written to the DynamoDB outbox.

After Phase 10: COMPLETE gate — review → ship `v1.2.0` → audit. **DONE.**

---

## v1.2 (Active — platform hardening + first real consumer deployment)

Six-phase breakdown to harden the v1.1 spike, simplify the setup, update
the docs, and prove the platform delivers real value by deploying a basic
microservice to AWS ECS Fargate end-to-end. Ship tag at milestone COMPLETE:
**`v1.3.0`** (feature milestone, next minor per ship.md — v1.1 shipped
`v1.2.0`). Phase patches `v1.2.1`..`v1.2.6`.

### Phase 11 — v1.2-research-and-readme
- **Description:** Re-evaluate go-gitea/gitea#36988 (OIDC for Gitea Actions) — confirm still open (re-checked 2026-07-21: open, last updated 2026-05-27, not merged) and record the decision to extend D-039 as D-047. Audit the v1.1 spike for NFR gaps (least-privilege IAM, idempotency, error handling, rotation hygiene) and simplification opportunities (script consolidation, dead code, stale paths). Rewrite `README.md` to reflect v1.1 complete + the actual spike flow + how to run + the real repo layout + the v1.2 objective.
- **Status:** complete (v1.2.1)
- **Depends on:** —
- **Requirements:** REQ-29
- **Success Criteria:**
  - `RESEARCH.md` has a v1.2 addendum with the #36988 re-check + NFR audit + simplification findings.
  - `README.md` reflects v1.1 complete; documents the spike flow, `scripts/run_platform.sh`, the repo layout, and the v1.2 objective; no stale "v1.1 (active)" framing.
  - D-047 is recorded in `PROJECT.md`.

### Phase 12 — nfr-harden-and-simplify
- **Description:** Apply Phase 11's findings. Tighten `terraform/bootstrap/spike_runner_policy.json` to least-privilege (add ECS + ECR + ELB + IAM plan-only permissions for v1.2; audit for wildcards). Make `create_state_backend.py` and `create_iam_user.py` idempotent. Consolidate `run_spike_plan.sh` + `run_spike_e2e.sh` into a single `scripts/run_platform.sh` with proper exit codes and error handling. Redact P1-1 (the two AWS access key IDs in `.ciagent/VERIFY.md` Phase 09 narrative). Fix any remaining stale `platform/` paths in `.ciagent/`. The v1.1 spike still runs e2e after the refactor.
- **Status:** complete (v1.2.2)
- **Depends on:** [11]
- **Requirements:** REQ-30
- **Success Criteria:**
  - `scripts/run_platform.sh` runs the full v1.1 spike e2e and exits 0.
  - `create_state_backend.py` / `create_iam_user.py` re-runs are idempotent (no duplicate resources; exit 0).
  - `spike_runner_policy.json` passes a least-privilege audit (no `*` actions beyond documented exceptions).
  - `.ciagent/VERIFY.md` Phase 09 narrative has no live AWS access key IDs.
  - No stale `platform/` paths remain in `.ciagent/`.

### Phase 13 — l1-catalog-for-ecs
- **Description:** Author six IR-typed L1 modules for an ECS Fargate microservice: `l1-vpc` (VPC + subnets + route tables), `l1-ecs-cluster` (ECS Fargate cluster), `l1-ecs-service` (ECS service + task definition), `l1-iam-role` (task execution + task role), `l1-alb` (ALB + listener + target group), `l1-ecr` (ECR repository). Each has an `interface.json` valid against `schemas/ir.schema.json`. Register all six in `modules-ir/registry.json`. Expand the Terraform adapter `TYPE_MAP` to cover the new IR resource types. Each L1 produces a valid `terraform plan` fragment.
- **Status:** complete (v1.2.3)
- **Depends on:** [12]
- **Requirements:** REQ-31
- **Success Criteria:**
  - All six L1s exist under `modules-ir/l1/` with `interface.json` valid against `schemas/ir.schema.json`.
  - `modules-ir/registry.json` lists all six.
  - The adapter `TYPE_MAP` covers all six IR resource types.
  - Each L1 produces a valid `terraform plan` fragment.

### Phase 14 — l2-microservice-and-contract-schema
- **Description:** Author `l2-microservice` thin-composition under `modules-ir/l2/l2-microservice/` referencing the six ECS L1s (depth ≤ 5). Extend `schemas/contract.schema.json` with microservice inputs (`image: string`, `port: integer`, `env: map`, `healthcheck: object`). Verify contract→IR resolution yields a complete target stack.
- **Status:** complete (v1.2.4)
- **Depends on:** [13]
- **Requirements:** REQ-32
- **Success Criteria:**
  - `l2-microservice` references the six ECS L1s only (depth ≤ 5).
  - `schemas/contract.schema.json` validates a `contracts/microservice.yaml` with the new inputs.
  - Contract→IR resolution yields a complete target stack (all six L1 instances + relationships).

### Phase 15 — consumer-repo-and-terraform-apply
- **Description:** Create a new Gitea repo `acdl-consumer-microservice` under the `continuous-intelligence` org containing a basic HTTP microservice (tiny Python/Go server returning 200), a `Dockerfile`, an ECR push step, and a `contracts/microservice.yaml` submission for `l2-microservice` (dev environment). Lift the platform from `plan` to **`apply`** for the `dev` environment (autonomous per §10, confidence ≥ 0.50, no HITL). Submit the contract → pipeline → IR → plan → apply → a real ECS Fargate service running.
- **Status:** complete (v1.2.5, PARTIAL — terraform apply blocked by IAM P0)
- **Depends on:** [14]
- **Requirements:** REQ-33 (partial), REQ-34
- **Success Criteria:**
  - `acdl-consumer-microservice` repo exists under `continuous-intelligence`.
  - The microservice builds into a Docker image and is pushed to ECR.
  - `terraform apply` (dev) creates real AWS resources (VPC, ECS cluster, ECR repo, ALB, ECS service).
  - The apply result is captured in the evidence stream.

### Phase 16 — v1.2-capstone-e2e
- **Description:** End-to-end verification: consumer commit to `acdl-consumer-microservice` triggers the pipeline → contract→IR resolution → `terraform plan` → `terraform apply` (dev) → a live ECS Fargate service serving HTTP 200 on its ALB → evidence event written to the DynamoDB outbox → the event renders on the `acdl-evidence` timeline. Verify the NFR improvements from Phase 12 hold, the setup is simpler (one `scripts/run_platform.sh`), and the README is accurate. `scripts/verify_phase16.sh` proves the full flow green.
- **Status:** complete (v1.2.6, capstone — terraform apply blocked by IAM P0, verified up to plan)
- **Depends on:** [15]
- **Requirements:** REQ-35 (partial — IAM-blocked)
- **Success Criteria:**
  - One consumer commit produces a live ECS service serving HTTP 200.
  - An evidence event for the apply is in the DynamoDB outbox and renders on the timeline.
  - `scripts/verify_phase16.sh` exits 0.
  - README accurately documents the v1.2 platform flow.

After Phase 16: COMPLETE gate — review → ship `v1.3.0` → audit.