acdl/.ciagent/REQUIREMENTS.md

# ACDL — Requirements

## v1

### Category: Repos & Org
- **REQ-01:** All demo code lives under the `continuous-intelligence` Gitea org at `https://git.cloudinit.dev`.
- **REQ-09:** Three repos exist: `acdl` (platform + stubs + reusable workflows), `acdl-contracts` (developer surface), `acdl-evidence` (Pages audit timeline).

### Category: L1 Modules
- **REQ-02:** 8 L1 module folders exist under `acdl/modules/l1/`: `l1-eks-fargate`, `l1-iam-role`, `l1-lambda`, `l1-api-gateway`, `l1-eventbridge`, `l1-sqs`, `l1-s3`, `l1-cloudwatch`.
- **REQ-03:** Each L1 module has a `manifest.yaml` (declaring inputs) and a `mock_apply.sh` that echoes success, sleeps 1s, and exits 0.

### Category: L2 Modules
- **REQ-04:** 4 L2 modules exist under `acdl/modules/l2/`: `l2-invoice-service`, `l2-commodity-price-feed`, `l2-energy-analytics-api`, `l2-regulatory-reporting`, each composing the specified L1s.
- **REQ-05:** L2 modules compose L1 primitives into deployable shapes with a maximum depth of 5.

### Category: Core Scripts
- **REQ-06:** `mock_executor.sh` reads an L2 composition, invokes each L1 `mock_apply.sh`, and writes `state.json`.
- **REQ-07:** `policy_checker.py` reads `contract.yaml` and fails with `POLICY_VIOLATION:PUBLIC_INGRESS` on `public-ingress: true`; otherwise passes.
- **REQ-08:** `confidence_signal.py` returns a base score of 0.90 and drops to 0.40 (with reason code) when policy fails; gate threshold is ≥ 0.50.

### Category: Evidence Stream
- **REQ-11:** `evidence_writer.py` appends events to `audit.json` and links each event to the previous via a SHA-256 hash chain (`prev_hash` + own `hash`).
- **REQ-13:** `acdl-evidence` is Pages-enabled and serves `audit.json` plus `index.html`.

### Category: Pipeline
- **REQ-10:** The reusable pipeline runs Dev (autonomous), pauses at QA (manual approval), pauses at Prod (manual approval), then finalizes by committing `audit.json` to `acdl-evidence`.
- **REQ-12:** Opening an Issue in `acdl-contracts` runs `l3b_agent_stub.py`, commits a generated `contract.yaml` to a new branch, closes the Issue, and triggers the main pipeline.

### Category: Demo Acts
- **REQ-14:** `index.html` uses vanilla JS to fetch `audit.json` from the Pages URL and render events as a timeline.
- **REQ-15:** All four demo acts (Friction, Developer Self-Service, Citizen Developer, Safety Net) reproduce deterministically in a dry run.

## v2

(None — v1 covers the complete demo.)

## v1.1 (Prior milestone — architecture finalization + v1 spike, complete)

### Category: Architecture Finalization
- **REQ-16:** Architecture reaches v1.0 — all 11 open decisions in `docs/architecture.md` §13 are resolved and recorded in `PROJECT.md` (W1.A, W1.B, W2.A, W3.D, W3.E, BA.A–F, OpenTofu timing).
- **REQ-17:** Target Stack IR is defined as a JSON Schema under `schemas/ir.schema.json`; substrate-agnostic (resources, relationships, composition max-depth-5, policy hooks).
- **REQ-18:** `PolicyCheckResult` normalized schema is defined under `schemas/policy_check_result.schema.json`; a Checkov adapter translates Checkov JSON to this schema.
- **REQ-19:** Six-input confidence signal is specified under `platform/confidence_signal.py` with per-env thresholds (dev 0.50 / qa 0.75 / prod 0.90 / dr 0.95) and severity→penalty mapping (critical=hard override, high=-0.2, medium=-0.05, low=-0.01, info=0.0).
- **REQ-20:** Tiered audit ledger design is authored: S3 Object Lock (compliance mode, 7-yr) + DynamoDB outbox (RPO=0, JWS detached signatures, `prev_event_hash` chain, daily checkpoints).
- **REQ-21:** Full 8-concern HITL matrix + separation-of-duties design is authored (CODEOWNERS routing + DynamoDB identity-distinctness check; pre-execution gate model; 1d warn / 2d freeze timeout).
- **REQ-22:** Contract schema (JSON Schema draft 2020-12) is defined under `schemas/contract.schema.json` with per-env mandatory/optional inputs (W3.E) and `profile: agentic` marker for L3B fields.

### Category: AWS OIDC Bootstrap
- **REQ-23:** AWS auth bootstrap + state backend for the spike: an S3 state bucket + DynamoDB lock/outbox table + an IAM user with a minimal scoped policy (S3 + DynamoDB + plan-only). The temporary long-lived key is used once (waiver D-034) then rotated via `scripts/rotate_spike_key.sh` after each spike run (D-039). **Real OIDC federation is deferred to v1.2** — Gitea Actions does not support `id-token: write` (RESEARCH TARGET 1, conf 0.95), blocked on go-gitea/gitea#36988.

### Category: v1 Spike — IR, L1, Adapter
- **REQ-24:** One real L1 module `l1-s3` exists under `modules-ir/l1/l1-s3/` with an IR-typed interface (typed inputs/outputs/NFRs) registered in the L1 registry.
- **REQ-25:** One real L2 thin-composition `l2-static-asset` exists under `modules-ir/l2/l2-static-asset/` referencing `l1-s3` only (depth 1, within max-depth-5).
- **REQ-26:** The Terraform adapter (`adapters/terraform/`) compiles the IR-typed L1 interface to Terraform `variable`/`output` blocks and the L2 thin-composition tree to a Terraform root module; it emits a real `terraform plan` against AWS via OIDC; state is stored in S3 + DynamoDB.

### Category: v1 Spike — End-to-End
- **REQ-27:** One end-to-end contract submission (`contracts/spike.yaml` for `l2-static-asset`) flows through: contract schema validation → contract→IR resolution → `terraform plan` (real AWS) → Checkov `PolicyCheckResult` → confidence signal → evidence event written to the DynamoDB outbox.
- **REQ-28:** Spike verification (`scripts/verify_phase10.sh`) proves the IR-shaped commitments hold: the adapter is the only substrate-specific code; no polyglot mess; the L1 content, contract YML, and thin-composition tree are substrate-agnostic.

## Out of Scope (v1.1)

| Feature | Reason |
|---------|--------|
| Full HITL matrix wiring (qa/prod/dr) | Spike is dev-only (`terraform plan`); HITL wiring is v1.2. |
| Kyverno + OPA policy engines | Spike uses Checkov only; Kyverno/OPA are v1.2. |
| MCP skill catalog + real L3B agent | L3B spike = a single stub contract submission; the 5-skill catalog is v1.2. |
| GitOps reconciler (ArgoCD/Flux) | v1.2. |
| Multi-region state / outbox | Single-region in v1 (§9, §12.3). |
| Prod/dr environments | v1.2. |
| Terraform `apply` (real provisioning) | Spike runs `plan` only; `apply` is gated by HITL in v1.2. |

## v1.2 (Prior milestone — platform hardening + first real consumer deployment, complete, tag `v1.3.0`)

### Category: Documentation & Simplification
- **REQ-29:** `README.md` is fully rewritten to reflect the v1.1-complete platform: the actual spike flow (contract → IR → `terraform plan` → Checkov → confidence signal → outbox), how to run it (`scripts/run_platform.sh`), the real repo layout (`acdl_platform/`, `schemas/`, `adapters/`, `terraform/`, `modules-ir/`, `contracts/`, `demo/`), and the v1.2 objective. No stale "v1.1 (active)" framing.
- **REQ-30:** NFR hardening of the v1.1 spike: (a) `terraform/bootstrap/spike_runner_policy.json` audited to least-privilege (S3 + DynamoDB + ECS + ECR + ELB + IAM plan-only, no wildcards beyond the documented exceptions); (b) `create_state_backend.py` and `create_iam_user.py` are idempotent (re-running exits 0 without duplicating resources); (c) `run_spike_plan.sh` + `run_spike_e2e.sh` consolidated into a single `scripts/run_platform.sh` with proper exit codes and error handling; (d) P1-1 carried forward from the v1.1 audit — the two AWS access key IDs in `.ciagent/VERIFY.md` Phase 09 narrative are redacted to placeholders; (e) any remaining stale `platform/` paths in `.ciagent/` are corrected to `acdl_platform/`.

### Category: L1 Catalog Expansion (ECS Fargate)
- **REQ-31:** Six new IR-typed L1 modules exist under `modules-ir/l1/` and are registered in `modules-ir/registry.json`: `l1-vpc` (VPC + subnets + route tables), `l1-ecs-cluster` (ECS Fargate cluster), `l1-ecs-service` (ECS service + task definition), `l1-iam-role` (task execution + task role), `l1-alb` (application load balancer + listener + target group), `l1-ecr` (ECR repository). Each has an `interface.json` valid against `schemas/ir.schema.json` and produces a valid `terraform plan` fragment via the Terraform adapter. The adapter `TYPE_MAP` is expanded to cover all six IR resource types.

### Category: L2 Composition & Contract Schema
- **REQ-32:** `l2-microservice` thin-composition exists under `modules-ir/l2/l2-microservice/` referencing the six ECS L1s (depth ≤ 5, within max-depth-5). `schemas/contract.schema.json` is extended with microservice inputs (`image: string`, `port: integer`, `env: map`, `healthcheck: object`) and validates a `contracts/microservice.yaml` submission. Contract→IR resolution (`acdl_platform/contract_resolver.py`) yields a complete target stack for `l2-microservice`.

### Category: Real Provisioning
- **REQ-33:** The platform runs `terraform apply` (not just `plan`) for the `dev` environment, autonomous per §10 (confidence ≥ 0.50, no HITL). The apply creates real AWS resources (VPC, ECS cluster, ECR repo, ALB, ECS service) and the result is captured in the evidence stream. `apply` for qa/prod/dr remains HITL-gated and out of scope for v1.2.

### Category: Consumer Repo
- **REQ-34:** A new Gitea repo `acdl-consumer-microservice` exists under the `continuous-intelligence` org, containing: a basic HTTP microservice (e.g., a tiny Python/Go server returning 200), a `Dockerfile`, an ECR push step, and a `contracts/microservice.yaml` submission for `l2-microservice` (dev environment).

### Category: End-to-End Verification
- **REQ-35:** One end-to-end flow: consumer commit to `acdl-consumer-microservice` → pipeline triggered → contract→IR resolution → `terraform plan` → `terraform apply` (dev) → a live ECS Fargate service serving HTTP 200 on its ALB → evidence event written to the DynamoDB outbox → the event renders on the `acdl-evidence` timeline. `scripts/verify_phase16.sh` proves the full flow green.

## v1.3 (Prior — module documentation + thin-composition removal, complete)

### Category: Thin-Composition Removal
- **REQ-36:** The L2 thin-composition layer is removed completely: `composition.json` files, `acdl_platform/contract_resolver.py`, `schemas/contract.schema.json`, `contracts/spike.yaml`, `contracts/microservice.yaml`, and L2 entries in `modules-ir/registry.json` are deleted. The L2 directories are kept as placeholders with READMEs. The downstream pipeline (adapter → checkov → confidence → outbox) is patched to load a pre-existing IR instance instead of resolving a contract.
- **REQ-37:** A `modules-ir/README-TEMPLATE.md` exists that works for both L1 and L2 modules, written in plain language (no jargon), with sections for Overview, Resources, Inputs, Outputs, Usage, Compliance extension points, and Versioning.
- **REQ-38:** Every module has a `README.md`: the 7 L1 modules have full READMEs with Resources/Inputs/Outputs/Usage/Compliance-extension-points/Versioning sections derived from their `interface.json`; the 2 L2 modules have placeholder READMEs noting the composition is under redesign. A `modules-ir/README.md` catalog index lists all modules with one-line descriptions and links.

### Category: Testing
- **REQ-39:** A pytest test suite exists under `tests/` covering the platform components offline (no AWS, no Checkov, no DynamoDB): the Terraform adapter (`adapters/terraform/adapter.py`), the confidence signal (`acdl_platform/confidence_signal.py`), the Checkov adapter (`adapters/terraform/policy/checkov_adapter.py`), and the outbox writer (`acdl_platform/outbox_writer.py`). The suite validates the IR schema, registry, spike_instance, and adapter output structure. `pyproject.toml` + `requirements-test.txt` pin test dependencies (pytest, jsonschema, pyyaml, boto3-stubs or moto for outbox mocking).

### Category: Shell Reproducibility
- **REQ-40:** `scripts/run_platform.sh` has a `--check-only` mode that runs offline: loads the pre-existing IR instance, runs the adapter to emit Terraform, validates the JSON structure — without AWS credentials, Checkov, or DynamoDB. The existing `--plan-only` and full modes continue to require AWS. The `--check-only` mode is what CI pipelines run.

### Category: CI/CD Pipelines
- **REQ-41:** Identical CI/CD pipelines exist for both Gitea Actions (`.gitea/workflows/ci.yml`, dev environment) and GitHub Actions (`.github/workflows/ci.yml`, production). Both run the same three stages: (1) lint — `py_compile` all Python files, (2) test — `pytest`, (3) check-only — `bash scripts/run_platform.sh --check-only`. Both trigger on push to main + pull request. Both use `ubuntu-latest`. Identical outcomes — the only difference is the runner environment.

- **REQ-42:** `pyproject.toml` exists at the repo root with pytest configuration (testpaths, markers) and the project metadata. `requirements-test.txt` pins test-only dependencies separate from runtime dependencies.

## v1.4 (Active — central pipeline contract + shell reproducibility + streaming)

### Category: Central Pipeline Contract
- **REQ-43:** A central pipeline contract exists as `schemas/pipeline.schema.json` (JSON Schema draft 2020-12) + `pipelines/ci.yaml` (YAML instance). The contract declares the pipeline name, triggers (push/PR branches), runner, Python version, and stages (name + command + required + install + description). Both `.gitea/workflows/ci.yml` (Gitea Actions, dev) and `.github/workflows/ci.yml` (GitHub Actions, production) implement the same stages, commands, triggers, and runner as declared in the contract. A test (`tests/test_pipeline_contract.py`) validates the contract against the schema and asserts both workflows conform (same jobs, same commands, same triggers, same runner, byte-identical).

### Category: Shell Reproducibility
- **REQ-44:** `scripts/run_ci.sh` reproduces the CI pipeline locally — runs the same 3 stages (lint, test, check-only) in sequence with proper exit codes, failing on first error. The script exits 0 with "CI PIPELINE OK" on success. A `--quiet` flag suppresses per-stage banners. The script mirrors the central pipeline contract (`pipelines/ci.yaml`) so the shell and CI environments produce identical outcomes.

### Category: Pipeline Streaming
- **REQ-45:** `scripts/run_platform.sh` streams output by default: terraform init/validate/plan output is piped to stdout via `tee` (visible to the user and logged), Checkov results are printed in human-readable form, and PolicyCheckResult records are displayed with severity, rule ID, and pass/fail status per record. The `--check-only` mode streams the emitted Terraform file content. A `--quiet` flag suppresses streaming (output to log files only) for backwards compatibility. Both gitea and github workflows are byte-identical (identical outcomes — the only difference is the forge runtime).

## Out of Scope (v1.2)

| REQ | Original criterion | Clarified criterion (effective) | Decision |
|-----|--------------------|----------------------------------|----------|
| REQ-09 | Three repos exist | Three repos exist (`acdl`, `acdl-contracts`, `acdl-evidence`) under `continuous-intelligence`; new repos use `default_branch: "main"`, `auto_init: true` | D-015 |
| REQ-10 | "Pages returns 200 with placeholder `index.html`" on `acdl-evidence` | Gitea has no Pages; substitute: an HTTP GET against the raw file URL `https://git.cloudinit.dev/continuous-intelligence/acdl-evidence/raw/branch/main/index.html` returns 200 with the placeholder HTML body | D-012, D-016 |
| REQ-10 | "`qa` and `prod` environments exist on `acdl-contracts`" | Gitea has no environments API and ignores `environment:` blocks; substitute: the reusable workflow defines `qa-gate` and `prod-gate` jobs gated by `workflow_dispatch` approval inputs (D-004 fallback); a `qa` and `prod` branch may be created on `acdl-contracts` as a visible stand-in for environments | D-013 |

## Out of Scope (v1.0 demo — retained for history)

| Feature | Reason |
|---------|--------|
| Real cloud provisioning (AWS/GCP/Azure) | Demo explicitly stubs all infrastructure; no cloud access available. |
| Real LLM inference / external AI APIs | Spec forbids external AI; L3B is a keyword parser. |
| Production-grade infrastructure | Demo target is a 30-minute executive show, not a production system. |
| Adversarial tamper-proofing of evidence | Hash chain is demonstrative; not cryptographically secure against a determined attacker. |
| Multi-tenant isolation | Out of demo scope. |

## Traceability

### v1.0 (prior — demo)

| Requirement | Phase | Status |
|-------------|-------|--------|
| REQ-01 | 1 | complete (v1.0.1) |
| REQ-02 | 2 | complete (v1.0.2) |
| REQ-03 | 2 | complete (v1.0.2) |
| REQ-04 | 3 | complete (v1.0.3) |
| REQ-05 | 3 | complete (v1.0.3) |
| REQ-06 | 3 | complete (v1.0.3) |
| REQ-07 | 3 | complete (v1.0.3) |
| REQ-08 | 3 | complete (v1.0.3) |
| REQ-09 | 1 | complete (v1.0.1) |
| REQ-10 | 4 | complete (v1.0.4) |
| REQ-11 | 3 | complete (v1.0.3) |
| REQ-12 | 4 | complete (v1.0.4) |
| REQ-13 | 5 | complete (v1.0.5) |
| REQ-14 | 5 | complete (v1.0.5) |
| REQ-15 | 5 | complete (v1.0.5) |

### v1.1 (prior — architecture finalization + v1 spike, complete)

| Requirement | Phase | Status |
|-------------|-------|--------|
| REQ-16 | 07 | complete (v1.1.2) |
| REQ-17 | 07 | complete (v1.1.2) |
| REQ-18 | 07 | complete (v1.1.2) |
| REQ-19 | 07 | complete (v1.1.2) |
| REQ-20 | 07 | complete (v1.1.2) |
| REQ-21 | 07 | complete (v1.1.2) |
| REQ-22 | 07 | complete (v1.1.2) |
| REQ-23 | 08 | complete (v1.1.3) |
| REQ-24 | 09 | complete (v1.1.4) |
| REQ-25 | 10 | complete (v1.1.5) |
| REQ-26 | 09 | complete (v1.1.4) |
| REQ-27 | 10 | complete (v1.1.5) |
| REQ-28 | 10 | complete (v1.1.5) |

### v1.2 (prior — platform hardening + first real consumer deployment, complete)

| Requirement | Phase | Status |
|-------------|-------|--------|
| REQ-29 | 11 | complete (v1.2.1) |
| REQ-30 | 12 | complete (v1.2.2) |
| REQ-31 | 13 | complete (v1.2.3) |
| REQ-32 | 14 | complete (v1.2.4) |
| REQ-33 | 15 | partial (v1.2.5, IAM-blocked) |
| REQ-34 | 15 | complete (v1.2.5) |
| REQ-35 | 16 | partial (v1.2.6, IAM-blocked) |

### v1.3 (prior — module documentation + thin-composition removal, complete)

| Requirement | Phase | Status |
|-------------|-------|--------|
| REQ-36 | 17 | complete (v1.3.1) |
| REQ-37 | 17 | complete (v1.3.1) |
| REQ-38 | 17 | complete (v1.3.1) |
| REQ-39 | 18 | complete (v1.3.2) |
| REQ-40 | 18 | complete (v1.3.2) |
| REQ-41 | 18 | complete (v1.3.2) |
| REQ-42 | 18 | complete (v1.3.2) |

### v1.4 (active — central pipeline contract + shell reproducibility + streaming)

| Requirement | Phase | Status |
|-------------|-------|--------|
| REQ-43 | 19 | complete (v1.4.1) |
| REQ-44 | 19 | complete (v1.4.1) |
| REQ-45 | 19 | complete (v1.4.1) |