acdl/.ciagent/ROADMAP.md

# Nova — Roadmap

> **Compressed.** The full v1.0–v1.24 phase-by-phase breakdown is preserved
> verbatim at `.ciagent/archive/ROADMAP-v1.0-v1.24.md`. This file retains
> the milestone Overview (one-line-per-milestone summary + archive
> pointers), the v1.25 phase detail (immediate predecessor, still
> load-bearing for v1.26's policy-engine substrate), and the active
> v1.26 phase detail. The active phase plan (task-level, wave-ordered,
> persona-assigned) lives in `.ciagent/PLAN.md`.

## Overview

- **v1.0 (demo):** complete — tag `v1.1.0`, 2026-07-21. 5 phases (01–05)
  shipped + audited PASS. Demo URL:
  https://git.cloudinit.dev/continuous-intelligence/acdl-evidence/raw/branch/main/index.html
- **v1.1:** complete — architecture finalization + v1 spike. Tag `v1.2.0`.
  5 phases (06–10). Gitea release id 202.
- **v1.2:** complete — platform hardening + first real consumer
  deployment. Tag `v1.3.0`. 6 phases (11–16).
- **v1.3:** complete — module documentation + thin-composition removal.
  Tag `v1.3.2`.
- **v1.4:** complete — central pipeline contract + shell reproducibility
  + output streaming.
- **v1.5:** complete (tag `v1.5.0`) — consumer happy path + zero-trust
  docs + reusable deploy workflow.
- **v1.6:** complete (tag `v1.6.0`) — consumer-facing docs restructure
  + terminology normalization + environments concept. `acdl_platform/` →
  `core/`; L2 → "modules", L1 → "primitives", "composition" → "pattern".
- **v1.7:** complete (tag `v1.7.0`) — production platform + contract
  ingestion + pipeline maturation. `cloudfront` + `waf` primitives;
  tagging-standard enforcement; Wiz stub + Kyverno K8s adapter (D-053);
  Platform Lambda + DynamoDB `acdl-contracts`; deploy outputs via SSM
  SecureString + GitHub PR comment; uniform error reporting via Lambda
  `report_error`; release job with semver + floating tag maintenance.
- **v1.8:** complete (tag `v1.8.0`) — P1 remediation + uptime monitoring
  + engineering standards + encryption/deletion-protection by default +
  decommission alias + path documentation. Per-stack CMK;
  deletion-protection-by-default; `uptime-kuma` primitive; decommission
  mode; `modules/STANDARDS.md`.
- **v1.9.1–v1.9.8:** complete (tags `v1.9.1`..`v1.9.8`) — leadership
  presentation decks (PW + DX), progressively refined. **Superseded by
  v1.10 re-verification** — the decks presented advertised capability as
  current without disclosing that the platform had decayed.
- **v1.10:** complete (tag `v1.10.0`) — pipeline regression fix +
  capability re-verification + verified-reality rewrite. Re-verified
  every advertised capability, fixed all 7 adapter defects in-sweep
  (D-090: no cap), rewrote PROJECT/ROADMAP/decks to match verified reality.
- **v1.10.1:** complete (tag `v1.10.1`) — post-v1.10 NFR patch
  (adversarial grill, 4-layer verify PASS, multi-persona code review).
  ACDL reclassified as OSS reference implementation (G-003).
- **v1.10.2:** complete (tag `v1.10.2`) — contract surface redesign +
  rename + `.yml` repo-wide + deck polish. Breaking contract schema
  change: new top-level `id`/`name`/`infrastructure`; dropped
  `uses:`/`module:`/`inputs:`.
- **v1.11:** complete (tag `v1.11.0`) — RESTART: stateless adapter +
  pipeline-driven module lifecycle testing. The terraform adapter
  becomes a stateless assembler (~80 lines, emits `module "x" { source }`
  blocks); lifecycle owned by terraform via the shell orchestrator;
  testing is pipeline-driven (a `modules-lifecycle` pipeline
  matrix-runs each L1 module's examples through apply→modify→destroy
  against live AWS). A single platform VPC is shared by all stacks.
  13 phases (P56a–P65).
- **v1.12:** complete (tag `v1.12.0`) — Presentation Refinement
  (decks synced to v1.11-verified reality). 6 phases (P66–P70).
- **v1.13:** complete (tag `v1.13.0`) — Presentation Polish (both decks
  polished across all 4 pipeline layers). 1 phase (P71).
- **v1.13.1:** complete (tag `v1.13.1`) — `config.json` schema migration
  to CIAgent v2 config structure.
- **v1.13.2:** complete (tag `v1.13.2`) — presentation badge cleanup +
  platform architecture diagram (Slide 3 "The platform at a glance").
- **v1.23:** complete (tag `v1.22.6`) — Nova Deck Cleanup & Python
  PPTX. Consolidated the deck to a single source-of-truth `*-marp.md`;
  restored the clean S&P visual style; base64-inlined images;
  `scripts/render_pptx.py` (python-pptx); removed the term "penetrate"
  repo-wide. 13 requirements (REQ-263..275), 6 phases.
- **v1.24:** complete (tag `v1.23.4`) — Consumer Guide Accuracy +
  Env-Promotion Lifecycle Enforcement. Fixes 5 consumer-guide accuracy
  issues + adds platform-enforced destroy-on-environment-change (the
  platform detects `environment:` edits on a stable `contract.id` via
  the `nova-contracts` DynamoDB table, destroys the prior env's state
  before building the new env). New `core/env_transition.py` module.
  15 requirements (REQ-276..290), 4 phases.

> **Full v1.0–v1.24 phase detail, wave ordering, success criteria, and
> decision cross-references:** `.ciagent/archive/ROADMAP-v1.0-v1.24.md`.

---

## v1.25 (complete, tag `v1.24.5`): kyverno-json Unified Policy Engine

`kyverno-json` — a Kyverno-ecosystem runtime that applies Kyverno policies
to **any** JSON/YAML payload — becomes Nova's **primary compliance /
policy tool**, implemented behind a swappable `PolicyEngine` adapter so
OPA (or any other engine) can replace it one day. The unified-orchestrator
model: Checkov and Wiz remain as raw-finding adapters feeding *into*
kyverno-json meta-policies; the confidence signal is untouched (it already
consumes `list[PolicyCheckResult]` engine-agnostically). Policies cover
all four Nova artifacts: consumer contract JSON, resolved Stack IR,
Terraform plan JSON, and the merged PCR list itself (meta-validation).
The K8s-only Kyverno adapter stays documentation-only (D-053); the
kyverno-json engine and the K8s adapter are siblings, not replacements.
Quality improvement from the IDEATE pass: capability regression checks
(`core/regression_verify.py` CAP-013/023/024) become declarative
kyverno-json policies. New `policy-engineer` persona owns the policy
territory. 19 requirements (REQ-291..309), 6 phases (P0 + P1..P4 + P5
final). Tags: `v1.24.0` (P0) → `v1.24.5` (P5 = milestone release).

### Phase P1 — engine-core (complete, tag v1.24.1)
- REQ-291: `core/policy_engine.py` — `PolicyEngine` Protocol +
  `PolicyEngineRegistry` (selects engine from `config.json.policy.engine`).
- REQ-292: `config.json` gains `policy` object
  (`engine: "kyverno-json"`, `policy_root`).
- REQ-293: `adapters/kyverno-json/kyverno_json_engine.py` —
  `KyvernoJsonEngine` (shells to `kj scan`; translates native output →
  PCR; `is_configured()` guards on `which kj`).
- REQ-294: `adapters/kyverno-json/__init__.py` + `_smoke.json` policy +
  `scripts/install-kyverno-json.sh` + CI image install.
- REQ-308: `tests/test_policy_engine.py` — protocol conformance,
  registry, NullEngine fallback.
- REQ-309: `tests/test_kyverno_json_engine.py` — PCR schema validity,
  defensive parsing, `pytest.skip` when kj absent.

### Phase P2 — contract + stack-IR policies (complete, tag v1.24.2)
- REQ-295: `adapters/kyverno-json/policies/contract/` — 4 policies over
  consumer contract JSON (id-pattern, env-enum, infra-min-1,
  forbid-unknown-fields).
- REQ-296: `core/contract_resolver.py` invokes the engine pre-resolve
  (contract policies) — early-fail, confidence signal decides the gate.
- REQ-297: `adapters/kyverno-json/policies/stack-ir/` — 3 policies over
  resolved Stack IR (tagging-standard, public-ingress, encryption-by-
  default — ports of v1.0/v1.8 imperative rules).
- REQ-298: `core/contract_resolver.py` invokes the engine post-resolve
  (stack-IR policies); additive — existing tests pass.
- REQ-299: `tests/test_stack_ir_policies.py` + fixtures (passing + failing
  IR; skip when kj absent).

### Phase P3 — plan-JSON policies + meta-orchestration + pipeline wiring (complete, tag v1.24.3)
- REQ-300: `adapters/kyverno-json/policies/plan-json/` — 3 policies over
  `terraform show -json` (plaintext-secrets, iam-wildcard, kms-reference
  — ports of `checkov_adapter.py:RULE_MAP`).
- REQ-301: `run_platform.sh` Step 5 gains a parallel kyverno-json pass;
  both PCR lists (checkov/wiz + kj) concatenate into the confidence
  signal's `policy` input; skips gracefully when `which kj` is false.
- REQ-302: `tests/test_plan_json_policies.py` + fixtures;
  `tests/test_run_platform_plan_json_policies.py` (script-substring
  assertion).
- REQ-303: `adapters/kyverno-json/policies/meta/` —
  `block-on-any-critical.json` (declarative critical-block; the
  `confidence_signal.py` hard-override stays as defense-in-depth) +
  `tagging-rules-agree.json` (asserts Checkov + kj agree on tagging).
  `tests/test_meta_policies.py`.

### Phase P4 — regression-gate policies + docs (complete, tag v1.24.4)
- REQ-304: `adapters/kyverno-json/policies/regression/` — 3 policies over
  capability-inventory JSON (CAP-013/023/024) — declarative mirrors of
  `core/regression_verify.py` checks.
- REQ-305: `tests/test_regression_policies.py` + fixtures (clean +
  drifted inventory); regression gate still 287/287 baseline.
- REQ-306: `adapters/README.md` (new adapter row + PolicyEngine Protocol
  section) + `adapters/kyverno-json/README.md`.
- REQ-307: `.ciagent/ARCHITECTURE.md` §12.7 (Policy Engine Registry) +
  `schemas/README.md` + `modules/STANDARDS.md` (policy-authoring
  standard) + `docs/METRICS.md` (swappable engine narrative).

### Phase P5 — final review + audit + milestone ship (Final Phase, complete, tag v1.24.5)
- Multi-persona code review across P1..P4 (lead-developer, backend-
  engineer, data-engineer, policy-engineer). Auto-fix P0; flag P1+.
- Audit: reconstruction test (git log ↔ `.ciagent/`), branch hygiene,
  commit discipline.
- Milestone ship: merge `phase/05-final-review-ship` →
  `milestone/v1.25-kyverno-json` → `main`; tag `v1.24.5` (= the v1.25
  release per prev-minor tagging rule); Gitea release with full milestone
  summary; delete all milestone branches.
- Updated `REQUIREMENTS.md` (mark REQ-291..309 complete), `ROADMAP.md`
  (mark v1.25 complete), `NORTH_STAR.md` (note Strategic Objective #2 —
  provable trust via a replaceable policy-engine substrate).
- **Requirements:** REQ-291..309 (19 requirements).

---

## v1.26 (active, tag line `v1.25.x`): Live Pilot Estate Activation

`D-096` lifts. The first real consumer estate — a stock exchange on a
homegrown Proof-of-Authority blockchain (equities only, single
validator, T+1 settlement finality = block commit) — is activated
against live AWS account `581513795199`. The consumer repo
(`nova-blockchain-exchange`) owns the app code + `contract.yaml`; the
platform repo (`acdl`) provides the deploy workflow (`deploy.yml@v1.25`),
the policy engine (kyverno-json, swappable per v1.25), the confidence
signal, and the HITL attestation gates. The milestone grounds the three
Post-Pilot targets in NORTH_STAR.md (Touchless Resolution ≥99%, Human
Escalation <0.1%, AI Decision Accuracy ≥99.5%) — the denominators
activate when the pilot runs. Three kyverno-json policies extend v1.25:
settlement-finality (securities-specific), pilot-readiness (no
placeholder account), and the existing meta-policies (block-on-any-
critical, tagging-rules-agree) apply over the pilot's PCRs. The
env-JSON `state_backend` wiring gap is closed (adapter reads the env
JSON's bucket). Multi-project mode activates (`nova-blockchain-exchange`
is the 2nd tracked project). Pre-run (Workstream A) re-created the S3
state bucket + DynamoDB outbox table (bootstrap). 13 requirements
(REQ-310..322), 6 phases (P0 pre-execution + 4 execution + 1 final).
Tags: `v1.25.0` (P0) → `v1.25.5` (P5 = milestone release).

### Phase P0 — pre-execution (complete, tag v1.25.0)
SPECIFY → CLARIFY → RESEARCH → IDEATE → PLAN → GRILL. Pre-run
Workstream A: flaky-test fix (`8c68d68`), ACDL_*→NOVA_* bootstrap
migration (`f844fea`), AWS bootstrap (S3 `nova-tfstate-581513795199-us-east-1`
+ DynamoDB outbox), `nova-blockchain-exchange` Gitea repo created + cloned.
10 ambiguities resolved (D-200..D-213). 7 ideas accepted (I1..I7 →
REQ-315..322), 3 deferred. Adversarial grill: PROCEED 0.84.

### Phase P1 — blockchain-core (complete, tag v1.25.1)
- REQ-310: `nova-blockchain-exchange` repo — homegrown PoA blockchain
  core (`chain/block.py`, `chain/ledger.py`, `chain/validator.py`).
  Append-only blocks, single validator, SHA-256 hash chain,
  deterministic block production, genesis block.
- REQ-311: Order-matching engine (`engine/order_book.py`,
  `engine/order.py`) — limit order book, price-time priority, partial
  fills.
- REQ-312: Settlement service (`settlement/service.py`) — T+1,
  idempotent, finality = block commit.

### Phase P2 — consumer-contract-and-deploy (complete, tag v1.25.2)
- REQ-322: `modules/l1/dynamodb/` — new L1 primitive (interface.json +
  terraform/main.tf + README.md + instance.json + registry.json entry).
  The single platform-side module build-out (ECS + S3 already exist;
  the adapter is stateless/registry-driven). Landed in P2 W0 (before
  the contract) so the contract's `dynamodb` block resolves at registry
  time.
- REQ-313: `nova-blockchain-exchange/contract.yaml` + per-env variants
  (dev/qa/prod) — validated against `schemas/contract.schema.json`.
- REQ-314: `nova-blockchain-exchange/.github/workflows/deploy.yml` +
  `.gitea/workflows/deploy.yml` — `uses: acdl/.github/workflows/deploy.yml@v1.25`
  with `mode: full`.
- Cross-cutting: `v1.25` floating tag → `v1.25.0` (Phase 0 ship) on the
  platform repo.

### Phase P3 — pilot-metrics-and-policies (planned, tag v1.25.3)
- REQ-315: `adapters/kyverno-json/policies/settlement-finality.json` —
  kyverno-json policy asserting all matches in the promotion window have
  committed blocks (securities-specific). Authored + tested in v1.26;
  *enforcement* is deferred to the milestone that binds qa/prod/dr
  (D-208 — the policy gates promotions, not dev applies).
- REQ-316: `core/regression_verify.py` gains CAP-025
  (live-pilot-apply) — the round-trip assertion (contract resolve →
  adapter compile → terraform plan → policy scan → confidence signal →
  attestation → outbox record) against `581513795199`.
- REQ-317: `core/metrics/outcome_backfill.py` — wire
  `apply.completed`/`apply.failed` → `fact_decision.outcome` (grounds AI
  Decision Accuracy; today `outcome` is stuck `pending`).
- REQ-318: `core/confidence_signal.py` — `ai.decision.made` gains
  `escalation_reason: 'confidence'` when `band == 'block'` (grounds
  Human Escalation Frequency numerator).
- REQ-319: `adapters/terraform/adapter.py` — reads
  `env.state_backend.bucket` from the env JSON (closing the wiring gap);
  `core/environments/*.json` `state_backend.bucket` →
  `nova-tfstate-581513795199-us-east-1`.
- REQ-320: `adapters/kyverno-json/policies/pilot-readiness/no-placeholder-account.json`
  — declarative gate preventing apply against a placeholder account.

### Phase P4 — pilot-run-and-docs (planned, tag v1.25.4)
- REQ-321: `adapters/README.md` (new consumer row) +
  `docs/METRICS.md` (Post-Pilot metrics grounded note) +
  `.ciagent/ARCHITECTURE.md` §12.8 (Pilot Estate) +
  `.ciagent/nova-blockchain-exchange/README.md` (onboarding guide).
- Live pilot end-to-end run: `nova-blockchain-exchange` contract →
  `deploy.yml@v1.25` mode=full → apply → attest → record against
  `581513795199`. The run's `ai.decision.made` + `attestation.recorded`
  events land in the Decision Ledger; the regression gate (CAP-025)
  verifies the round-trip.

### Phase P5 — final review + audit + milestone ship (Final Phase, planned, tag v1.25.5)
- Multi-persona code review across P1..P4 (lead-developer, backend-
  engineer, data-engineer, policy-engineer, blockchain-engineer).
  Auto-fix P0; flag P1+.
- Audit: reconstruction test (git log ↔ `.ciagent/`), branch hygiene,
  commit discipline.
- Milestone ship: merge `phase/05-final-review-ship` →
  `milestone/v1.26-pilot-activation` → `main`; tag `v1.25.5` (= the
  v1.26 release per prev-minor tagging rule); create Gitea release with
  full milestone summary; delete all milestone branches.
- Update `REQUIREMENTS.md` (mark REQ-310..322 complete), `ROADMAP.md`
  (mark v1.26 complete), `NORTH_STAR.md` (note Strategic Objectives #1
  + #3 — first real consumer estate; Post-Pilot denominators activated).

> **Phase task-level breakdown, wave ordering, and persona
> assignments:** `.ciagent/PLAN.md` (the active phase plan, retained in
> full). **Authoritative resume state:** `.ciagent/CHECKPOINT.json`.