diff --git a/.ciagent/CHECKPOINT.json b/.ciagent/CHECKPOINT.json index d0b393e..861fb51 100644 --- a/.ciagent/CHECKPOINT.json +++ b/.ciagent/CHECKPOINT.json @@ -1,12 +1,12 @@ { "phase": 0, - "stage": "ideate", + "stage": "plan", "milestone": "v1.17", "phase_role": "pre_execution", "attempts": 0, - "updated_at": "2026-08-04T20:30:00Z", + "updated_at": "2026-08-04T20:45:00Z", "milestone_complete": false, "tag": null, "requirements": ["REQ-185"], - "notes": "IDEATE complete. 15 enhancement ideas accepted (REQ-199..213), all additive within D-120..D-132. ~12-14% scope addition. REQUIREMENTS.md updated with new REQs + traceability. No locked decisions re-opened." + "notes": "PLAN complete. 8 phases (P1..P7 + P8 final), 4 waves. Deck rebuild plan: 15 main + 2 appendix slides, 5-act arc, per-slide benefit callouts, fluid transitions. 29 requirements (REQ-185..213) covered. Wave dependency graph + critical path documented." } \ No newline at end of file diff --git a/.ciagent/PLAN.md b/.ciagent/PLAN.md index f602e44..6602182 100644 --- a/.ciagent/PLAN.md +++ b/.ciagent/PLAN.md @@ -1,420 +1,1114 @@ --- phase: P0 name: pre-execution -milestone: v1.16 -requirements: [REQ-165, REQ-166, REQ-167, REQ-168, REQ-169, REQ-170, REQ-171, REQ-172, REQ-173, REQ-174, REQ-175, REQ-176, REQ-177, REQ-178, REQ-179, REQ-180, REQ-181, REQ-182, REQ-183, REQ-184] +milestone: v1.17 +requirements: [REQ-185, REQ-186, REQ-187, REQ-188, REQ-189, REQ-190, REQ-191, REQ-192, REQ-193, REQ-194, REQ-195, REQ-196, REQ-197, REQ-198, REQ-199, REQ-200, REQ-201, REQ-202, REQ-203, REQ-204, REQ-205, REQ-206, REQ-207, REQ-208, REQ-209, REQ-210, REQ-211, REQ-212, REQ-213] wave: 0 depends_on: [] --- -# v1.16 — Nova Simplification Plan (20 execution phases + 1 final) +# v1.17 — Strategic Direction, Leadership Metrics & Unified Story (Plan) -**Milestone:** v1.16 (Nova Simplification — NFR) -**Type:** NFR (all phases fix/chore/docs/refactor/test). The final -phase's patch IS the deliverable — no separate milestone tag. Tags run -on the v1.15.x line: `v1.15.5` (P0) → `v1.15.6..v1.15.25` (P1–P20) → -`v1.15.26` (P21 final = milestone release). +**Milestone:** v1.17 — Strategic Direction, Leadership Metrics & Unified Story +**Type:** Feature (P1–P3 feat; P4 docs; P5 docs+test; P6 test; P7 review+audit+ship; +P8 final). Progressive patches; the final phase's patch IS the milestone +release. Tags run on the v1.16.x line: `v1.16.0` (P0) → `v1.16.1..v1.16.7` +(P1–P7) → `v1.16.8` (P8 final = milestone release). +**Branch:** `milestone/v1.17-strategic-metrics-deck` (branched off the v1.16 +complete merge). Execution phases branch `phase/NN-*` → merge to milestone +branch → tag patch on the v1.16.x line. +**Tags:** `metrics`, `telemetry`, `decision-ledger`, `powerbi`, `deck`, +`north-star`, `no-humans-thesis`, `regression-capability` +**Decisions (locked, D-120..D-132 — do NOT re-open):** +D-120 Nova-native + Infracost, drift deferred · D-121 Decision Ledger = +outbox_writer → SQLite hash-chain · D-122 AI decision = confidence_signal + +HITL gate · D-123 8 deferred metrics ship as empty placeholder views · +D-125 Hybrid events/files · D-126 Cold-only SQLite · D-127 Per-KPI +definition-of-success docs · D-128 metrics/ at repo root · D-129 PowerBI = +CSV/JSON folder connector · D-130 Deck arc Problem→Vision→How→Proof→Roadmap, +both old decks retired · D-131 MTTR = platform-run only · D-132 Attestation +instrumentation = emit attestation.recorded events. -**Objective:** A 20-phase NFR sweep (no new features) themed around five -user-directed axes: Simplify without regressions, Security, -Maintainability, User/Developer Experience, No Humans Onboarding Flow. -Clears the fresh debt the v1.15 rebrand left, delivers genuine -simplification, and implements the first self-service onboarding -request path (request-path only; real AWS provisioning deferred, D-113). +**Objective (three pillars):** +- **(A) Strategic Direction** — encode the PO's strategic direction in a + durable `NORTH_STAR.md` read by CIAgent in every future `/ci-run`. +- **(B) Leadership Metrics + PowerBI** — instrument Nova to collect, + aggregate, and surface leadership-grade metrics that prove the "no-humans" + autonomous-infrastructure value proposition — grounded in signals Nova + actually emits, derived via documented formulas, or explicitly deferred + with a decision ID — flowing into PowerBI-ready views. +- **(C) Unified Narrative Deck** — merge the two existing decks into one + unified narrative deck with the "tell them x3" arc at deck + slide level, + per-slide benefit callouts, and fluid transitions. -## Wave ordering +**Hard constraint:** DO NOT make anything up. Every metric carries a +`grounded` / `derived` / `deferred` status with a source file or decision +ID. Deferred metrics ship as empty PowerBI placeholder views with +documented schemas. -- **Wave 1 (P1–P4): correctness + brand regression fixes.** P1 first — - the state-bucket drift (`adapter.py:117` emits `acdl-tfstate-*` while - the live bucket is `nova-tfstate-*`) and the Kyverno policy - contradiction (enforces `acdl:*` labels that `nova_tagging.py` hard- - fails) are the highest-severity findings, both correctness regressions - left by the rebrand. P2–P4 independent brand/dead-code/except work. -- **Wave 2 (P5–P9): simplify without regressions.** P5 before P6/P9 - (regression-verify dedup is independent; P6/P9 both touch - `run_platform.sh`). P8 changes the workflow byte-identity test → - generator (D-115). P9 must run the regression gate (D-118) at the end - of Wave 2 — 22/22 capabilities must stay Verified. -- **Wave 3 (P10–P14): security + maintainability.** P10 before P11 - (identity enforcement before payload validation). P12/P13 independent - file splits. P14 mid-milestone checkpoint (offline) at end of Wave 3. -- **Wave 4 (P15–P17): developer experience.** Independent; P17 last - (reflects the consolidated path after P15/P16 land). -- **Wave 5 (P18–P20): no-humans onboarding (request-path only).** P18 - (schema + Lambda action) before P19 (env-file autogen consumes the - schema) before P20 (cross-account role, offline-proven per D-114). -- **Final (P21): review + audit + milestone ship.** +--- + +## Wave Overview + +| Wave | Phases | Theme | Dependency rationale | +|------|--------|-------|----------------------| +| **Wave 1** | P1 | Event emitters — the foundation | Everything depends on events being emitted. P1 establishes the CloudEvents envelope, per-run manifests, Decision Ledger, Infracost adapter, and attestation/confidence/policy event emission. P2/P3/P4/P5 all consume P1's event formats. | +| **Wave 2** | P2, P3 | Collector + PowerBI export | P2 (collector) reads P1's events/files → SQLite cold store. P3 (export) reads P2's SQLite → CSV/JSON views. P2 + P3 can partially parallelize: P3's view schemas can be authored against P2's schemas before P2's collector code is complete, but P3's export code needs P2's SQLite to exist. | +| **Wave 3** | P4, P5 | Metrics catalog + deck rebuild | P4 (catalog + NORTH_STAR integration) depends on metrics being grounded (P2/P3 done). P5 (deck) depends on P4's `METRICS.md` for the Proof section's grounded citations. P4 + P5 can partially parallelize: P5's Problem/Vision/How acts don't need P4; P5's Proof act needs P4's catalog. | +| **Wave 4** | P6, P7 | Regression capability + final review/ship | P6 (CAP-023/024) depends on P2/P3 (collector) + P5 (deck) existing. P7 (review/audit/ship) depends on all prior phases. | +| **Final** | P8 | Milestone ship | Merge to main, tag `v1.16.8`, Gitea release, delete milestone branches. | + +**Dependency chain (critical path):** +P1 → P2 → P3 → P4 → P5(Proof) → P6 → P7 → P8 + +**Parallelization opportunities:** +- P2 schemas + P3 view schemas can be authored concurrently (Wave 2 entry). +- P4 `docs/metrics/*.md` per-KPI docs + P5 Problem/Vision/How acts can be + authored concurrently (Wave 3 entry); P5 Proof act waits for P4 `METRICS.md`. +- P6 CAP-023 (collector) test can be drafted while P5 finishes (the test + needs P2's collector to exist, which it does by Wave 4). + +--- + +## Per-Phase Vertical-Slice Plans + +### Phase P1 — event-emitters (Wave 1, feat) + +**Goal:** Instrument every Nova decision point to emit structured CloudEvents +1.0 events + persist ephemeral `$WORK/*.json` as durable artifacts + extend +`outbox_writer.py` into the SQLite Decision Ledger. After P1, the metrics +layer has all the raw signals it needs — no downstream phase invents new +signals. + +**Requirements covered:** REQ-187, REQ-188, REQ-205 (emitter half), +REQ-206 (emitter half). + +**Primary persona:** backend-engineer. **Supporting:** data-engineer +(event schemas). + +**Tasks (vertical slices):** + +1. **CloudEvents envelope + schemas** — `core/metrics/event_envelope.py` + defines the CloudEvents 1.0 envelope + `platform.*` semantic conventions + (specversion, id, source, type, time, subject, datacontenttype, platform + block, data). `schemas/metrics_event.schema.json` validates the envelope. + `schemas/metrics_run_manifest.schema.json` validates per-run manifests. + - *Acceptance:* `python -m jsonschema` validates a sample event against + the schema; `tests/test_metrics_emitters.py::test_envelope` passes. + +2. **Per-run manifest writer** — `core/metrics/run_manifest.py` emits + `nova.run.started`, `nova.run.completed`, `nova.run.failed` events with + (run_id, contractId, env, stages×durations, exit, confidence, HITL block + count). Writes `metrics/runs/.json`. `scripts/run_platform.sh` + invokes the writer at run start + run end. + - *Acceptance:* a `--check-only` run produces `metrics/runs/.json` + with a valid manifest; `test_run_manifest` passes. + +3. **Persist ephemeral `$WORK/*.json`** — `run_platform.sh` copies + `$WORK/pcr.json`, `signal.json`, `event.json`, `outbox_item.json`, + `stack.json` to `metrics/runs//` as durable artifacts (the + ephemeral `$WORK` copies remain for the running pipeline; the persisted + copies are the metrics source of truth). + - *Acceptance:* after a run, `metrics/runs//pcr.json` exists and + matches `$WORK/pcr.json`; a test asserts the copy. + +4. **pytest addopts** — `pyproject.toml` `addopts` gains + `--junitxml=metrics/test-results.xml --json-report --cov=core --cov=adapters + --cov-report=json:metrics/coverage.json`. CAP-009 (offline pytest suite + passes) must remain Verified (assumption A5 — additive flags). + - *Acceptance:* `bash scripts/run_ci.sh` exits 0; `metrics/test-results.xml` + + `metrics/coverage.json` exist; regression gate 22/22 (run at P6, but + P1 must not break any cap locally). + +5. **Infracost post-processor** — `core/metrics/infracost_adapter.py` runs + Infracost on `terraform show -json plan.tfplan` (offline, reads plan JSON, + no live AWS). Emits `nova.cost.estimated{delta_usd}`. Degrades gracefully + (omits the event, logs a warning) when Infracost CLI is absent (A6). + `run_platform.sh` invokes it after the plan stage. + - *Acceptance:* when Infracost is available, `metrics/runs//` + contains a `cost_estimate.json`; when absent, the run still exits 0; + `test_infracost_adapter` passes (mock the CLI). + +6. **Decision Ledger (SQLite hash-chain)** — `core/metrics/decision_ledger.py` + extends `outbox_writer.py` to emit to a SQLite append-only table + (`metrics/decision_ledger.db`) with a hash chain (`prev_hash` + own + `hash`, SHA-256). Emits `ai.decision.made` events (decision_id=run_id, + chosen_action=band outcome, confidence=score, alternatives=perInput + breakdown, human_override=HITL block) with outcome backfill from + `apply.completed`. Honors D-083 (no S3 Object Lock/JWS — local SQLite + hash-chain only). + - *Acceptance:* `metrics/decision_ledger.db` exists after a run; the + hash chain verifies (`verify-chain` returns 0 broken); `test_decision_ledger` + passes. + +7. **Attestation event emission** — `core/hitl_gates.py` emits + `attestation.recorded` events to the Decision Ledger on qa/prod/dr gates + (approver, env, concerns, result). D-132. (Dev skips — autonomous.) + - *Acceptance:* a mocked qa gate produces an `attestation.recorded` row + in the Decision Ledger; `test_attestation_event` passes. + +8. **Confidence decision event emission** — `core/confidence_signal.py` + emits `nova.confidence.computed` + `nova.ai.decision.made` events (D-122: + the "AI decision" is the confidence-gated policy engine, not an LLM). + - *Acceptance:* a confidence computation produces both events in + `metrics/events.jsonl`; `test_confidence_event` passes. + +9. **Policy event emission** — `adapters/terraform/policy/checkov_adapter.py` + emits `nova.policy.evaluated` events (rule count, pass/fail/skipped, + severity breakdown). + - *Acceptance:* a Checkov run produces a `nova.policy.evaluated` event; + `test_policy_event` passes. + +10. **Lifecycle success-rate emitter** — each lifecycle run writes + `metrics/lifecycle/-.json` (module, env, phase + apply/modify/destroy, result, duration_ms). REQ-205 emitter half. + - *Acceptance:* a mocked lifecycle run produces the JSON; the emitter + test passes. + +11. **Capability event emission** — `core/regression_verify.py` emits + `nova.capability.verified` events (capability ID, status, tier, duration). + - *Acceptance:* a regression run produces `nova.capability.verified` + events; `test_capability_event` passes. + +**Must-haves (phase ships only if ALL true):** +- `core/metrics/event_envelope.py`, `run_manifest.py`, + `infracost_adapter.py`, `decision_ledger.py` exist and are tested. +- `metrics/events.jsonl` is appended to on every run (CloudEvents 1.0 + envelope, valid against `schemas/metrics_event.schema.json`). +- `metrics/runs/.json` manifest exists after every run. +- `metrics/decision_ledger.db` exists with a verified hash chain. +- `outbox_writer.py` extended to write to the SQLite Decision Ledger. +- `hitl_gates.py` emits `attestation.recorded` (D-132). +- `confidence_signal.py` emits `nova.confidence.computed` + + `nova.ai.decision.made` (D-122). +- `checkov_adapter.py` emits `nova.policy.evaluated`. +- `pyproject.toml` addopts include `--junitxml` + `--json-report` + `--cov`. +- `bash scripts/run_ci.sh` exits 0. +- No existing capability regresses (22/22 locally). + +**Risks + mitigations:** +- *Risk:* `--junitxml`/`--cov` addopts break the existing test suite. + *Mitigation:* A5 (additive flags); verify CAP-009 stays Verified locally + before merging. +- *Risk:* Infracost CLI not available in CI. *Mitigation:* A6 — degraded + mode (omit event, log warning, don't fail the run). +- *Risk:* SQLite hash-chain corruption on concurrent writes. *Mitigation:* + single-writer model (the run manifest writer is the only writer per run); + WAL mode + `BEGIN IMMEDIATE`. +- *Risk:* Event schema drift between emitters and collector. *Mitigation:* + schemas authored first (task 1); all emitters validate against the schema + before writing. + +--- + +### Phase P2 — metrics-collector (Wave 2, feat) + +**Goal:** Read all grounded signals (files + events) into a normalized +SQLite cold store at `metrics/nova_metrics.db` with idempotent re-runs. +After P2, the metrics layer has a queryable store — P3 exports it, P4 +catalogs it. + +**Requirements covered:** REQ-189, REQ-200, REQ-201, REQ-205 (collector +half), REQ-206 (collector half), REQ-207. + +**Primary persona:** data-engineer. **Supporting:** backend-engineer +(event formats). + +**Tasks (vertical slices):** + +1. **Fact/dimension schemas** — `schemas/metrics_fact_run.schema.json`, + `schemas/metrics_fact_capability.schema.json`, + `schemas/metrics_fact_policy_check.schema.json`, + `schemas/metrics_fact_confidence.schema.json`, + `schemas/metrics_fact_test.schema.json`, + `schemas/metrics_fact_decision.schema.json`, + `schemas/metrics_fact_cost_estimate.schema.json`, + `schemas/metrics_fact_lifecycle.schema.json`, + `schemas/metrics_dim_capability.schema.json`, + `schemas/metrics_dim_milestone.schema.json`. Schema-first (data-engineer + constraint): all schemas exist before any collector code. + - *Acceptance:* all schemas validate sample rows; `python -m jsonschema` + passes for each. + +2. **Collector core** — `core/metrics/collector.py` reads: + - `REGRESSION_REPORT.json` → `fact_capability` + `dim_capability`. + - `metrics/runs/*.json` → `fact_run`. + - `metrics/test-results.xml` (junit) → `fact_test`. + - `metrics/coverage.json` → `fact_test.coverage` column. + - `metrics/runs//pcr.json` → `fact_policy_check`. + - `metrics/runs//signal.json` → `fact_confidence`. + - `metrics/decision_ledger.db` → `fact_decision`. + - `metrics/runs//cost_estimate.json` → `fact_cost_estimate`. + - `metrics/lifecycle/*.json` → `fact_lifecycle`. + - `CHECKPOINT.json` → `dim_milestone`. + Writes to `metrics/nova_metrics.db` (SQLite cold store, D-126). + - *Acceptance:* after a run + collector invocation, + `metrics/nova_metrics.db` has all fact/dim tables populated; + `test_metrics_collector` passes. + +3. **Idempotent re-runs** — the collector is idempotent: re-running it + produces identical row counts + a verified chain. REQ-200. + - *Acceptance:* `test_metrics_collector_idempotent` passes (two runs → + identical row counts + chain verified). + +4. **Decision Ledger CLI** — `core/metrics/decision_ledger_cli.py` supports + `query`, `verify-chain`, `stats`, `export`, `replay`. `verify-chain` + detects broken hashes; `replay` prints ordered events. REQ-207. + - *Acceptance:* `decision_ledger_cli.py verify-chain` exits 0 on a clean + chain, exits 1 on a tampered chain; `test_decision_ledger_cli` passes. + +5. **Metrics README** — `metrics/README.md` documents regenerable vs + append-only artifacts + the restore procedure (the cold store is + regenerable from the raw signals; the Decision Ledger is append-only). + REQ-201. + - *Acceptance:* `metrics/README.md` exists with the two categories + a + restore procedure section. + +**Must-haves:** +- `core/metrics/collector.py` exists and is tested. +- `metrics/nova_metrics.db` is produced with all fact/dim tables. +- Idempotent re-runs (REQ-200) verified by test. +- `core/metrics/decision_ledger_cli.py` exists with all 5 subcommands. +- `metrics/README.md` documents regenerable vs append-only + restore. +- `bash scripts/run_ci.sh` exits 0. + +**Risks + mitigations:** +- *Risk:* Schema drift between P1's event formats and P2's fact schemas. + *Mitigation:* data-engineer authors both; backend-engineer reviews the + event-format alignment. +- *Risk:* Junit XML parsing edge cases (test names with special chars). + *Mitigation:* use `xml.etree.ElementTree` with XPath; test with a fixture + containing edge-case names. + +--- + +### Phase P3 — powerbi-export (Wave 2, feat) + +**Goal:** Emit CSV/JSON views from the SQLite cold store to +`metrics/powerbi/` — fact + dimension views + 8 empty placeholder views +for deferred metrics. After P3, a PowerBI folder-connector dashboard can +be built. + +**Requirements covered:** REQ-190, REQ-199, REQ-208, REQ-209 (P3 half), +REQ-205 (view half). + +**Primary persona:** data-engineer. + +**Tasks (vertical slices):** + +1. **PowerBI export core** — `core/metrics/powerbi_export.py` reads + `metrics/nova_metrics.db` and emits CSV/JSON views to `metrics/powerbi/`: + `fact_run.csv`, `fact_capability.csv`, `fact_policy_check.csv`, + `fact_confidence.csv`, `fact_test.csv`, `fact_decision.csv`, + `fact_cost_estimate.csv`, `fact_lifecycle.csv`, `dim_capability.csv`, + `dim_milestone.csv`. D-129 (CSV/JSON folder connector). + - *Acceptance:* after `powerbi_export.py` runs, all 10 CSV files exist + in `metrics/powerbi/` with non-empty content (given a populated cold + store); `test_powerbi_export` passes. + +2. **8 deferred placeholder views** — empty CSV files with documented + schemas (headers only, no data rows) for the 8 deferred metrics: + (1) Live Infrastructure Health, (2) Live Outbox Write Rate, + (3) Tamper-Evident Ledger Checkpoints, (4) Onboarding Funnel + (requested→granted), (5) Drift Auto-Reversal Rate, (6) Live CUR + Reconciliation, (7) SLA / Unplanned Downtime, (8) Predictive vs Reactive + Ratio. D-123. Each has a header row documenting the columns + a comment + row citing the blocking decision ID. + - *Acceptance:* all 8 placeholder CSVs exist with header rows + a + decision-ID comment; `test_placeholder_views` passes. + +3. **METRICS_VIEWS.md data dictionary** — `docs/METRICS_VIEWS.md` has a + per-column data-dictionary table (column, type, source/formula, unit, + grounded/derived/deferred status) for every view. REQ-209 (P3 half). + - *Acceptance:* `docs/METRICS_VIEWS.md` exists with a complete + per-column table covering all 18 views (10 fact/dim + 8 placeholder). + +4. **NOVA_DASHBOARD_README.md** — `metrics/powerbi/NOVA_DASHBOARD_README.md` + documents the folder-connector import path + a starter visual model + + a reference screenshot placeholder. REQ-208. + - *Acceptance:* the README exists with import steps + visual model + description. + +5. **Schema validation in CI** — `run_ci.sh` validates + `metrics/powerbi/*.json` + a sample `metrics/events.jsonl` against their + schemas; exits 0. REQ-199. + - *Acceptance:* `bash scripts/run_ci.sh` validates the PowerBI JSON + exports + a sample events file; exits 0. + +**Must-haves:** +- `core/metrics/powerbi_export.py` exists and is tested. +- `metrics/powerbi/` contains all 10 fact/dim CSVs + 8 placeholder CSVs. +- `docs/METRICS_VIEWS.md` has the per-column data dictionary. +- `metrics/powerbi/NOVA_DASHBOARD_README.md` exists. +- `run_ci.sh` schema validation (REQ-199) passes. +- `bash scripts/run_ci.sh` exits 0. + +**Risks + mitigations:** +- *Risk:* Placeholder view schemas diverge from what the future emitter + will produce. *Mitigation:* the schema is documented in the header row + + METRICS_VIEWS.md; the future emitter must conform to the documented + schema. +- *Risk:* PowerBI folder connector quirks (CSV encoding, delimiters). + *Mitigation:* UTF-8 + comma-delimited; documented in the README. + +--- + +### Phase P4 — metrics-catalog + north-star-integration (Wave 3, docs) + +**Goal:** Catalog every executive KPI in `docs/METRICS.md` with +grounded/derived/deferred status + per-KPI definition-of-success docs. +Wire `NORTH_STAR.md` into CIAgent context-loading so every future +`/ci-run` reads it. Produce the trust-snapshot report, the deferred-metrics +roadmap, the confidence-gate halt rate metric, and the no-humans thesis +brief. After P4, the metrics layer is fully documented and the strategic +direction is durable. + +**Requirements covered:** REQ-186, REQ-191, REQ-192, REQ-193, REQ-194, +REQ-195, REQ-204, REQ-209 (P4 half), REQ-210, REQ-211, REQ-212, REQ-213 +(P4 half). + +**Primary persona:** lead-developer. **Supporting:** data-engineer +(metric definitions). + +**Tasks (vertical slices):** + +1. **METRICS.md catalog** — `docs/METRICS.md` catalogs every executive KPI + with: name, NORTH_STAR target, `grounded`/`derived`/`deferred` status, + source file or decision ID, and a link to the per-KPI definition doc. + REQ-195. Covers all metrics from the scorecard (RESEARCH.md §3): + Touchless Resolution Rate, Human Escalation Frequency, MTTR (platform-run), + AI Decision Accuracy, Decision Ledger Coverage, Attestation Coverage, + Capability Health, Confidence Distribution, Policy Pass Rate, Test + Count/Pass Rate, Provisioning Lead Time, Deployment Frequency, + Cost Estimates (Infracost), FTE Hours Saved, Platform ROI, + Confidence-Gate Halt Rate, + the 8 deferred metrics. + - *Acceptance:* `docs/METRICS.md` exists; every KPI has a status badge + + a source link; a grep confirms no KPI is missing a status. + +2. **Per-KPI definition-of-success docs** — `docs/metrics/.md` for + every KPI (D-127). Each doc defines: the metric, the formula, the + grounding status, the source file, the definition of success (what + number = "won"), and the deferred dependency (if applicable). + - *Acceptance:* `docs/metrics/` contains one `.md` per KPI; each doc + has all 5 sections. + +3. **Zero-touch efficiency metrics docs** — REQ-191: Autonomous Resolution + Rate, Human Escalation Frequency, AI Decision Accuracy, MTTD/MTTR + (platform-run, D-131). Documented in METRICS.md + per-KPI docs with + the attestation exclusion clarification (attestation gates are designed + controls, not escalations). + - *Acceptance:* the 4 metrics have per-KPI docs with the correct + formulas + attestation exclusion language. + +4. **Velocity metrics docs** — REQ-192: Provisioning Lead Time + (apply.completed.time − intent.received.time), Deployment Frequency + (count(apply.completed) per day). Self-Healing Velocity deferred. + - *Acceptance:* the 2 metrics have per-KPI docs; the deferral is + documented. + +5. **Financial & cost-ROI metrics docs** — REQ-193: FTE Hours Saved + (derived), Cost Savings via Infracost (grounded), Cost Efficiency Ratio + (derived), Platform ROI (derived formula). Live CUR deferred (D-096). + - *Acceptance:* the 4 metrics have per-KPI docs with formulas; the CUR + deferral cites D-096. + +6. **Reliability, security & compliance metrics docs** — REQ-194: + Zero-Trust Policy Compliance Rate (from pcr.json), Attestation Coverage + (prod/dr promotions attested by a human ÷ total prod/dr promotions; + grounded in `hitl_gates.py` + outbox `approver_*` attributes). Uptime, + Patch Remediation, SLA/downtime deferred (D-096). **Attestation Coverage + is canonically owned here (REQ-194), not in REQ-191.** + - *Acceptance:* the 2 grounded metrics have per-KPI docs; the 3 deferred + metrics have deferral docs citing D-096. + +7. **NORTH_STAR integration** — REQ-186: `NORTH_STAR.md` is referenced from + `PROJECT.md` (a "Strategic Direction" section pointing to it) + + `ARCHITECTURE.md` (the v1.17 addendum already references it). `config.json` + gains `strategic_direction_file: ".ciagent/NORTH_STAR.md"` so the run + workflow reads it at SPECIFY. + - *Acceptance:* `PROJECT.md` has a Strategic Direction section; + `config.json` has the `strategic_direction_file` key; a test confirms + the file is readable. + +8. **NORTH_STAR diff-check in CI** — REQ-204: `run_ci.sh` includes + `check_north_star_diff` that fails when Vision/Objectives/Anti-Goals/ + Targets sections change without a `NORTH_STAR-CHANGE:` commit trailer. + - *Acceptance:* a test commit changing a Target without the trailer + fails the check; a commit with the trailer passes. + +9. **Deferred-metrics activation roadmap** — `docs/METRICS_DEFERRED_ROADMAP.md` + lists 8 deferred metrics + onboarding-grant half with {blocking decision, + unblock requirement, candidate milestone} + a "Hot-Path Activation + (post-D-096)" section (Nova-native only, D-120) + "Re-evaluation + Triggers" section. REQ-210. + - *Acceptance:* the roadmap exists with all 8 + the onboarding-grant + half + the 2 sections. + +10. **Trust-snapshot report** — `core/metrics/trust_snapshot.py` emits + `metrics/TRUST_SNAPSHOT.md` with 5 trust metrics (Decision Ledger + Coverage, Attestation Coverage, Capability Health, AI Decision + Accuracy, Confidence-Gate Halt Rate) + chain-integrity verdict + + snapshot hash. Runs offline. REQ-211. + - *Acceptance:* `metrics/TRUST_SNAPSHOT.md` exists after running + `trust_snapshot.py`; the 5 metrics + verdict + hash are present; + `test_trust_snapshot` passes. + +11. **Confidence-Gate Halt Rate metric** — REQ-212: `docs/METRICS.md` + + trust snapshot include "Confidence-Gate Halt Rate" (signal.json + band=halt ÷ total runs). PowerBI view includes it (added to + `fact_confidence` projection in P3's export — coordinate with P3). + - *Acceptance:* METRICS.md has the metric; the trust snapshot includes + it; the PowerBI export includes a column for it. + +12. **No-humans thesis brief** — `docs/NO_HUMANS_THESIS.md` defines the + thesis, grounded proof metrics, deferred proof metrics, and explicit + anti-claims (incl. D-122 honesty: the "AI" is the confidence-gated + policy engine, not an LLM). REQ-213 (P4 half). The unified deck's + Vision act cites it (P5). + - *Acceptance:* `docs/NO_HUMANS_THESIS.md` exists with all 4 sections; + the anti-claims section explicitly addresses D-122. + +**Must-haves:** +- `docs/METRICS.md` catalogs every KPI with status + source. +- `docs/metrics/*.md` per-KPI docs exist for every KPI. +- `NORTH_STAR.md` referenced from PROJECT.md + ARCHITECTURE.md + config.json. +- `run_ci.sh` includes `check_north_star_diff` (REQ-204). +- `docs/METRICS_DEFERRED_ROADMAP.md` exists (REQ-210). +- `core/metrics/trust_snapshot.py` + `metrics/TRUST_SNAPSHOT.md` (REQ-211). +- Confidence-Gate Halt Rate in METRICS.md + trust snapshot + PowerBI (REQ-212). +- `docs/NO_HUMANS_THESIS.md` exists (REQ-213 P4 half). +- `bash scripts/run_ci.sh` exits 0. + +**Risks + mitigations:** +- *Risk:* KPI definitions drift from NORTH_STAR targets. *Mitigation:* + the catalog cross-references NORTH_STAR target rows; the diff-check + (REQ-204) catches NORTH_STAR changes. +- *Risk:* The no-humans thesis overclaims. *Mitigation:* D-122 honesty + constraint — the anti-claims section explicitly states the "AI" is the + confidence-gated policy engine; A3. + +--- + +### Phase P5 — deck-rebuild (Wave 3, docs+test) + +**Goal:** Merge the two existing decks into one unified narrative deck +"Nova — The No-Humans Infrastructure Platform" with the 5-act arc +(Problem → Vision → How → Proof → Roadmap), x3 structure at deck + slide +level, per-slide benefit callouts, fluid transitions, a metrics glossary +appendix slide, a "what's deferred" slide, and the no-humans thesis cited +in the Vision act. Retire both old decks. Re-run the 4-step deck process +(source `.md` → Marp → HTML → talking-points). + +**Requirements covered:** REQ-196, REQ-197, REQ-202, REQ-203, REQ-213 +(P5 half). + +**Primary persona:** lead-developer. + +**Tasks (vertical slices):** + +1. **Unified deck source markdown** — `docs/presentations/nova-no-humans-platform.md` + is the single source of truth (the full slide-by-slide plan is in the + "Deck Rebuild Plan" section below). The 5-act arc with x3 at deck level + (opening = arc preview, body = tell them, closing = recap + ask) + x3 + per slide (opens with what it covers, delivers, closes with benefit + callout). Fluid transitions written into each slide's opening line. + REQ-196, REQ-197. + - *Acceptance:* the source `.md` exists with all slides from the deck + plan below; each slide has the 3-part structure; transitions are + written. + +2. **Marp deck** — `docs/presentations/nova-no-humans-platform-marp.md` + (Marp-formatted with the S&P visual theme, `sp-theme.json` unchanged). + - *Acceptance:* the Marp deck renders to HTML with the correct slide + count + theme. + +3. **HTML render** — `docs/presentations/nova-no-humans-platform.html` + (re-rendered from the Marp deck). + - *Acceptance:* the HTML exists and opens with the correct title slide. + +4. **Talking points** — `docs/presentations/nova-no-humans-platform-talking-points.md` + (distilled from the Marp deck, one section per slide with speaker notes). + - *Acceptance:* the talking-points file exists with one section per + slide. + +5. **Metrics glossary appendix slide** — REQ-202: the deck has a + "Metrics Glossary" appendix slide with one-line KPI definitions + + grounding badges (grounded/derived/deferred). + - *Acceptance:* the glossary slide exists with all KPIs + badges. + +6. **"What's Deferred — and Why" slide** — REQ-203: the deck has a slide + pairing each of 8 deferred metrics with its blocking decision ID. + - *Acceptance:* the deferred slide exists with all 8 + decision IDs. + +7. **No-humans thesis cited in Vision act** — REQ-213 (P5 half): the + Vision act cites `docs/NO_HUMANS_THESIS.md` (the thesis, grounded proof, + deferred proof, anti-claims). + - *Acceptance:* the Vision act slides reference the thesis brief. + +8. **Retire both old decks** — delete `how-the-platform-works.md` + + `-marp.md` + `.html` + `-talking-points.md` + `the-developer-experience.md` + + `-marp.md` + `.html` + `-talking-points.md`. D-130. + - *Acceptance:* a grep confirms the old deck files are deleted; no + references to them remain in the repo. + +**Must-haves:** +- `docs/presentations/nova-no-humans-platform.md` (+ marp + html + + talking-points) exists with the full slide plan. +- x3 structure at deck + slide level (REQ-197). +- Per-slide benefit callouts (REQ-197). +- Fluid transitions written into each slide (REQ-197). +- Metrics glossary appendix slide (REQ-202). +- "What's Deferred" slide (REQ-203). +- No-humans thesis cited in Vision act (REQ-213 P5 half). +- Both old decks deleted (D-130). +- `bash scripts/run_ci.sh` exits 0. + +**Risks + mitigations:** +- *Risk:* The deck claims a metric that isn't grounded yet. *Mitigation:* + P5 Proof act depends on P4's METRICS.md; every cited metric has a + grounded source file verified by the catalog. +- *Risk:* The old decks are referenced by other docs. *Mitigation:* grep + for references before deletion; update or remove them. + +--- + +### Phase P6 — regression-capability (Wave 4, test) + +**Goal:** Add CAP-023 (metrics collector runs, emits expected schema) + +CAP-024 (deck structure: slide count, x3 present, per-slide benefit +present) to `core/regression_verify.py`. After P6, the regression gate +protects the metrics layer + the deck structure. + +**Requirements covered:** REQ-198. + +**Primary persona:** backend-engineer. **Supporting:** data-engineer +(CAP-023 schema). + +**Tasks (vertical slices):** + +1. **CAP-023 — metrics collector** — `core/regression_verify.py` gains a + `CAP-023` check: runs `core/metrics/collector.py` against a fixture + metrics dir, asserts the SQLite cold store has all fact/dim tables with + the expected schema, asserts idempotent re-run. Tags Verified/Decayed/Broken. + - *Acceptance:* `CAP-023` returns Verified when the collector produces + the correct schema; `test_regression_cap023` passes. + +2. **CAP-024 — deck structure** — `core/regression_verify.py` gains a + `CAP-024` check: parses `docs/presentations/nova-no-humans-platform.md`, + asserts (a) slide count is in the expected range (12–20), (b) the x3 + structure is present (opening arc preview + closing recap), (c) each + slide has a benefit callout. Tags Verified/Decayed/Broken. + - *Acceptance:* `CAP-024` returns Verified when the deck meets all 3 + criteria; `test_regression_cap024` passes. + +3. **Regression gate run** — `bash scripts/run_regression.sh` runs the + full gate (22 prior capabilities + CAP-023 + CAP-024 = 24 total). All + must pass (Verified or Skipped per D-118). + - *Acceptance:* the regression report shows 24 capabilities, all + Verified or Skipped, 0 Decayed/Broken. + +**Must-haves:** +- `CAP-023` + `CAP-024` in `core/regression_verify.py`. +- `bash scripts/run_regression.sh` passes (24 capabilities, 0 Broken). +- `bash scripts/run_ci.sh` exits 0. + +**Risks + mitigations:** +- *Risk:* CAP-024's slide-count range is too tight and breaks on minor + deck edits. *Mitigation:* the range is 12–20 (generous); the check + focuses on structure (x3 + benefit callouts), not exact count. + +--- + +### Phase P7 — final-review-ship (Wave 4, review+audit+ship) + +**Goal:** Multi-persona review (incl. deck story quality), audit, and +milestone ship. After P7, v1.17 is complete and ready for the final merge. + +**Requirements covered:** all (review gate). + +**Primary persona:** lead-developer. **Supporting:** all active personas +(review participation). + +**Tasks (vertical slices):** + +1. **Multi-persona review** — each active persona reviews their territory: + - backend-engineer: event emitters, Decision Ledger, Infracost adapter, + regression CAP-023/024 code. + - data-engineer: collector, PowerBI export, schemas, data dictionary. + - lead-developer: NORTH_STAR integration, METRICS.md catalog, deck + narrative, no-humans thesis. + - Deck story quality review: the lead-developer reviews the deck for + narrative coherence, fluidity, and benefit-callout quality. + - *Acceptance:* review findings recorded; P0/P1 findings fixed before + ship; P2 findings logged for future milestones. + +2. **Audit** — verify: + - All 29 requirements (REQ-185..213) have a status of `complete` in + the traceability table. + - No stale claims in the deck (every metric citation has a grounded + source). + - `NORTH_STAR.md` is readable + referenced. + - The regression gate passes (24 capabilities). + - `bash scripts/run_ci.sh` exits 0. + - *Acceptance:* audit PASS recorded in `---ci---` block. + +3. **Milestone completion** — update `PROJECT.md`, `ROADMAP.md`, + `REQUIREMENTS.md` traceability to mark v1.17 complete. Tag `v1.16.7` + (P7 patch on the v1.16.x line). + - *Acceptance:* `PROJECT.md` reflects v1.17 complete; tag `v1.16.7` + exists. + +**Must-haves:** +- All 29 requirements marked complete. +- Multi-persona review complete (incl. deck story quality). +- Audit PASS. +- Regression gate 24/24 (Verified or Skipped). +- `bash scripts/run_ci.sh` exits 0. +- Tag `v1.16.7` exists. + +**Risks + mitigations:** +- *Risk:* Review surfaces a P0 finding late. *Mitigation:* the review is + scoped to each persona's territory; findings are fixed before the audit + step. + +--- + +### Phase P8 — milestone-ship (Final) + +**Goal:** Merge the milestone branch to main, tag `v1.16.8` (the milestone +release), publish the Gitea release, and delete the milestone branches. + +**Requirements covered:** all (ship gate). + +**Primary persona:** lead-developer. + +**Tasks (vertical slices):** + +1. **Merge to main** — merge `milestone/v1.17-strategic-metrics-deck` → + `main`. + - *Acceptance:* `main` contains all v1.17 commits; `git log main` shows + the milestone merge. + +2. **Tag + release** — tag `v1.16.8` on main; publish the Gitea release + (`Nova v1.16.8 — Strategic Direction, Leadership Metrics & Unified Story`) + with the release notes summarizing the three pillars. + - *Acceptance:* tag `v1.16.8` exists; Gitea release published (release + ID recorded). + +3. **Delete milestone branches** — delete `milestone/v1.17-strategic-metrics-deck` + + all `phase/NN-*` branches. + - *Acceptance:* `git branch -r` shows no v1.17 milestone/phase branches. + +**Must-haves:** +- `main` has the v1.17 merge. +- Tag `v1.16.8` exists. +- Gitea release published. +- Milestone + phase branches deleted. + +**Risks + mitigations:** +- *Risk:* Merge conflicts on main. *Mitigation:* the milestone branch is + off the v1.16 complete merge; rebase before merge if needed. + +--- + +## Deck Rebuild Plan + +> The unified deck: **"Nova — The No-Humans Infrastructure Platform."** +> 5-act arc: Problem → Vision → How → Proof → Roadmap. x3 at deck level +> (opening = arc preview, body = tell them, closing = recap + ask) + x3 +> per slide (opens with what it covers, delivers, closes with benefit +> callout). Fluid transitions written into each slide's opening line. +> Act indicator in the Marp footer (`Act N/5: `). + +### Deck-level x3 structure + +| Level | "What I'm going to tell you" | "Tell them" | "What I told you" | +|-------|------------------------------|-------------|-------------------| +| **Deck** | Slide 1 (arc preview: Problem→Vision→How→Proof→Roadmap) | Slides 2–14 (the 5 acts) | Slide 15 (recap of 5 acts + the ask) | +| **Per slide** | Opening line: "This slide shows X" | Body: bullets/diagram/table | Closing line: "Benefit: you now know Y" | + +### Act 1 — Problem (2 slides) + +> **Transition into Act 1:** (none — this is the opening; the arc preview +> slide sets up all 5 acts). + +**Slide 1 — Arc Preview (the "what I'm going to tell you" deck-level opening)** +- *Opens:* "This deck proves Nova is the no-humans infrastructure platform — + and shows you the metrics that make the claim defensible." +- *Delivers:* The 5-act arc as a visual roadmap: Problem → Vision → How → + Proof → Roadmap. One-line summary per act. +- *Closes:* "Benefit: you now know the arc — the next 14 slides deliver + each act in turn." +- *Grounded metrics cited:* none (this is the preview). +- *Deferred metrics:* none. + +**Slide 2 — The No-Humans Imperative** +- *Opens:* "This slide shows why the operator is the bottleneck — and why + removing them from operations (not accountability) is the imperative." +- *Delivers:* The cost of humans-in-the-loop: L1/L2 ops hours, escalation + latency, the trust gap (autonomous claims without proof). Cites the + no-humans thesis (`docs/NO_HUMANS_THESIS.md`). +- *Closes:* "Benefit: you now know the problem framing — autonomy in + operations, human at stage gates, is the path forward." +- *Grounded metrics cited:* none (problem framing). +- *Deferred metrics:* none. +- *Transition into Act 2:* "Having defined the problem, here is Nova's + strategic direction toward solving it." + +### Act 2 — Vision/Direction (3 slides) + +> **Transition into Act 2:** "Having defined the problem, here is Nova's +> strategic direction toward solving it." + +**Slide 3 — Nova's Vision** +- *Opens:* "This slide states Nova's vision — infrastructure operations + become invisible, with provable trust." +- *Delivers:* The NORTH_STAR vision statement verbatim. The attestation + model: human attestation required at stage gates (QA for production, SRE + for operational readiness); autonomy in operations, not in + accountability. Cites `docs/NO_HUMANS_THESIS.md` (the thesis, grounded + proof, deferred proof, anti-claims incl. D-122 honesty). +- *Closes:* "Benefit: you now know the destination — invisible operations + with provable trust, not promised trust." +- *Grounded metrics cited:* none (vision). +- *Deferred metrics:* none. +- *Transition:* "The vision is ambitious — here are the 4 strategic + objectives that make it concrete." + +**Slide 4 — Strategic Objectives + Anti-Goals** +- *Opens:* "This slide pairs what Nova is building toward (4 objectives) + with what Nova refuses to build (5 anti-goals)." +- *Delivers:* The 4 strategic objectives (zero-touch ops, provable trust, + compounding ROI, default substrate for agentic consumption) + the 5 + anti-goals (not a hyperscaler competitor, not a general AI platform, not + removing humans from accountability, not for legacy infra, not sold to + operators). From `NORTH_STAR.md`. +- *Closes:* "Benefit: you now know the scope boundaries — what Nova is, + and what it refuses to be." +- *Grounded metrics cited:* none (direction). +- *Deferred metrics:* none. +- *Transition:* "The objectives are committed to measurable targets — + here is the 12–18 month scorecard." + +**Slide 5 — 12–18 Month Targets (the scorecard)** +- *Opens:* "This slide shows the committed targets — numbers a board + member can repeat back — with their grounding status." +- *Delivers:* The NORTH_STAR targets table with the grounding column: + Touchless Resolution Rate ≥99% (grounded), Human Escalation <0.1% + (grounded), MTTR <60s (grounded, platform-run), AI Decision Accuracy + ≥99.5% (grounded), Decision Ledger Coverage 100% (grounded), + Attestation Coverage 100% (grounded), Cloud Spend Reduction ≥25% + (partial — Infracost grounded, CUR deferred), Platform ROI ≥250% + (derived), + deferred targets (Predictive vs Reactive, Drift Auto-Reversal, + AI-Agent Intent Share) marked **Planned**. +- *Closes:* "Benefit: you now know the destination numbers — and which + ones are measurable today vs deferred honestly." +- *Grounded metrics cited:* Touchless Resolution Rate, Human Escalation + Frequency, MTTR, AI Decision Accuracy, Decision Ledger Coverage, + Attestation Coverage — all `grounded` with source files. +- *Deferred metrics marked Planned:* Predictive vs Reactive, Drift + Auto-Reversal, AI-Agent Intent Share. +- *Transition into Act 3:* "The targets are committed — here is how Nova + works to achieve them." + +### Act 3 — How it works (4 slides) + +> **Transition into Act 3:** "The targets are committed — here is how +> Nova works to achieve them." + +**Slide 6 — The Platform Pipeline** +- *Opens:* "This slide shows the contract-to-evidence pipeline — how + intent becomes verified infrastructure without an operator." +- *Delivers:* The pipeline flow: contract → resolver → adapter → terraform + plan → Checkov (policy) → confidence signal → HITL gate (dev autonomous; + qa/prod/dr attested) → apply → evidence. Mermaid diagram. Grounded in + `scripts/run_platform.sh` + `core/contract_resolver.py` + + `adapters/terraform/adapter.py` + `core/confidence_signal.py`. +- *Closes:* "Benefit: you now know the path from intent to evidence — + and where the human appears (stage gates only)." +- *Grounded metrics cited:* none (architecture). +- *Deferred metrics:* none. +- *Transition:* "The pipeline produces decisions — here is how every + decision is captured and made accountable." + +**Slide 7 — The Decision Ledger** +- *Opens:* "This slide shows the Decision Ledger — every AI decision + captured with confidence, alternatives, and outcome." +- *Delivers:* The Decision Ledger architecture: `outbox_writer.py` + extended → SQLite append-only hash-chain table. `ai.decision.made` + events (decision_id=run_id, chosen_action=band, confidence=score, + alternatives=perInput, human_override=HITL block) with outcome backfill + from `apply.completed`. `attestation.recorded` events for qa/prod/dr. + D-121, D-122, D-132. Honors D-083 (no S3 Object Lock/JWS — local + hash-chain this milestone). +- *Closes:* "Benefit: you now know why 'autonomous' is defensible — every + decision is immutable, queryable, and accountable." +- *Grounded metrics cited:* Decision Ledger Coverage 100% (source: + `core/metrics/decision_ledger.py` + `metrics/decision_ledger.db`). +- *Deferred metrics marked Planned:* Tamper-Evident Ledger Checkpoints + (D-083). +- *Transition:* "Decisions are captured — here is how stage-gate + attestation keeps humans in accountability." + +**Slide 8 — The 8-Concern Attestation Matrix** +- *Opens:* "This slide shows the 8-concern attestation matrix — the + designed controls that keep humans at stage gates." +- *Delivers:* The 8 concerns (functional, performance, security posture, + contract NFRs, operational readiness, incident response, capacity/cost, + resilience). Offline-testable concerns run for real; operator-supplied + concerns accept signed evidence artifacts. Separation-of-duties on prod. + Grounded in `core/attestation_matrix.py` + `core/hitl_gates.py`. +- *Closes:* "Benefit: you now know the gate model — autonomy in + operations, human in accountability, by design." +- *Grounded metrics cited:* Attestation Coverage 100% (source: + `core/hitl_gates.py` + outbox `approver_*` attributes). +- *Deferred metrics:* none. +- *Transition:* "The platform is built — here is the proof that it works." + +**Slide 9 — Telemetry Architecture** +- *Opens:* "This slide shows how Nova instruments itself — the + CloudEvents envelope, the cold store, and the PowerBI export." +- *Delivers:* The telemetry architecture diagram (from ARCHITECTURE.md + v1.17 addendum): platform components → CloudEvents 1.0 envelope → + `metrics/events.jsonl` + `metrics/runs/` + `metrics/decision_ledger.db` + → collector → `metrics/nova_metrics.db` (SQLite cold store) → + `metrics/powerbi/` (CSV/JSON views) → PowerBI. D-120 (Nova-native), + D-125 (hybrid events/files), D-126 (cold-only). +- *Closes:* "Benefit: you now know the metrics pipeline — every signal is + grounded in a real file, not fabricated." +- *Grounded metrics cited:* none (architecture). +- *Deferred metrics marked Planned:* Hot-path (live ops dashboard) — D-126. +- *Transition into Act 4:* "The architecture is sound — here is the + measured proof." + +### Act 4 — Proof (4 slides) + +> **Transition into Act 4:** "The architecture is sound — here is the +> measured proof." + +**Slide 10 — Capability Health + Confidence Distribution** +- *Opens:* "This slide shows the grounded proof: capability health and + confidence distribution from real runs." +- *Delivers:* Capability health: 18 Verified + 4 Skipped (post-D-096 + teardown) from `.ciagent/REGRESSION_REPORT.json`. Confidence + distribution: from `metrics/nova_metrics.db` `fact_confidence` — score + histogram, band breakdown (pass/halt). The honesty model: Skipped is + honest (resources torn down per D-096), not a failure. +- *Closes:* "Benefit: you now know the platform is verified — 18 + capabilities pass, 4 are honestly skipped, 0 broken." +- *Grounded metrics cited:* Capability Health (source: + `REGRESSION_REPORT.json`), Confidence Distribution (source: + `metrics/nova_metrics.db` `fact_confidence`). +- *Deferred metrics:* none. +- *Transition:* "Capability health is necessary — here is the trust + substrate that makes autonomy defensible." + +**Slide 11 — Decision Ledger + Attestation Coverage** +- *Opens:* "This slide shows the trust metrics — Decision Ledger coverage + and attestation coverage, both 100%." +- *Delivers:* Decision Ledger Coverage: 100% of platform runs emit + `ai.decision.made` with outcome backfill (source: + `metrics/decision_ledger.db`). Attestation Coverage: 100% of prod/dr + promotions attested by a human (source: `hitl_gates.py` + outbox + `approver_*` attributes). AI Decision Accuracy: decisions not followed + by apply.failed/incident within 5min. The trust-snapshot report + (`metrics/TRUST_SNAPSHOT.md`) with chain-integrity verdict. +- *Closes:* "Benefit: you now know the trust is provable — not a marketing + claim, a queryable record." +- *Grounded metrics cited:* Decision Ledger Coverage, Attestation + Coverage, AI Decision Accuracy (source: `metrics/decision_ledger.db` + + `metrics/TRUST_SNAPSHOT.md`). +- *Deferred metrics:* Tamper-Evident Ledger Checkpoints (D-083) — Planned. +- *Transition:* "Trust is provable — here is the operational efficiency + that makes the ROI real." + +**Slide 12 — Zero-Touch Efficiency + Cost** +- *Opens:* "This slide shows the efficiency metrics — touchless resolution, + human escalation, and cost estimates." +- *Delivers:* Touchless Resolution Rate (runs without operational HITL + block ÷ total; attestation gates excluded). Human Escalation Frequency + (operational HITL blocks only). MTTR (platform-run: apply.failed → + successful retry, D-131). Cost Estimates via Infracost (pre-apply, + grounded). FTE Hours Saved (derived). Platform ROI (derived formula). + The grounded/derived/deferred honesty model. +- *Closes:* "Benefit: you now know the ROI is quantifiable — each quarter + on Nova must reduce spend, free hours, and avoid downtime measurably." +- *Grounded metrics cited:* Touchless Resolution Rate, Human Escalation + Frequency, MTTR, Cost Estimates (source: `metrics/nova_metrics.db` + `fact_run` + `fact_cost_estimate`). +- *Derived metrics:* FTE Hours Saved, Platform ROI. +- *Deferred metrics marked Planned:* Live CUR Reconciliation (D-096), + Drift Auto-Reversal (D-096). +- *Transition:* "The proof is grounded — here is what is honestly + deferred." + +**Slide 13 — What's Deferred — and Why** +- *Opens:* "This slide pairs each deferred metric with its blocking + decision — honesty about what isn't measured yet." +- *Delivers:* The 8 deferred metrics + onboarding-grant half, each paired + with its blocking decision ID: (1) Live Infrastructure Health — D-096, + (2) Live Outbox Write Rate — D-096, (3) Tamper-Evident Ledger + Checkpoints — D-083, (4) Onboarding Funnel (granted) — D-113/D-114/D-119, + (5) Drift Auto-Reversal — D-096 + no scheduler, (6) Live CUR + Reconciliation — D-096, (7) SLA / Unplanned Downtime — D-096, + (8) Predictive vs Reactive — future emitter. From + `docs/METRICS_DEFERRED_ROADMAP.md`. +- *Closes:* "Benefit: you now know the boundaries — what Nova measures + today, and exactly what blocks the rest." +- *Grounded metrics cited:* none (deferral honesty). +- *Deferred metrics:* all 8 + onboarding-grant half, each with decision ID. +- *Transition into Act 5:* "The proof is honest — here is the roadmap + from here to the 12–18 month targets." + +### Act 5 — Roadmap/Ask (2 slides) + +> **Transition into Act 5:** "The proof is honest — here is the roadmap +> from here to the 12–18 month targets." + +**Slide 14 — Roadmap to the North Star** +- *Opens:* "This slide shows the path from v1.17's grounded metrics to + the 12–18 month targets — the unblock path for each deferred metric." +- *Delivers:* The deferred-metrics activation roadmap (from + `docs/METRICS_DEFERRED_ROADMAP.md`): each deferred metric → blocking + decision → unblock requirement → candidate milestone. The hot-path + activation section (post-D-096, Nova-native only, D-120). Re-evaluation + triggers. +- *Closes:* "Benefit: you now know the path — every deferred metric has + an unblock requirement and a candidate milestone." +- *Grounded metrics cited:* none (roadmap). +- *Deferred metrics:* all 8 referenced with unblock paths. +- *Transition:* "The roadmap is clear — here is the recap and the ask." + +**Slide 15 — Recap + Ask (the "what I told you" deck-level closing)** +- *Opens:* "This slide recaps the 5 acts and states the ask." +- *Delivers:* Recap: Problem (operator bottleneck) → Vision (invisible + ops, provable trust) → How (pipeline + Decision Ledger + attestation) → + Proof (18V+4S, 100% ledger coverage, 100% attestation, grounded ROI) → + Roadmap (deferred metrics have unblock paths). The ask: fund the + hot-path activation (post-D-096) + the tamper-evident ledger build-out + (D-083 lift). +- *Closes:* "Benefit: you now know the full arc — and what is needed to + close the gap from grounded to complete." +- *Grounded metrics cited:* Capability Health, Decision Ledger Coverage, + Attestation Coverage (recap). +- *Deferred metrics:* referenced as the ask. + +### Appendix slides (2 slides) + +**Slide A1 — Metrics Glossary** +- *Opens:* "This appendix defines every KPI in one line with its grounding + badge." +- *Delivers:* One-line definitions for all KPIs with grounded/derived/ + deferred badges. REQ-202. +- *Closes:* "Benefit: you now have a reference for every metric mentioned + in the deck." +- *Grounded metrics cited:* all (glossary). +- *Deferred metrics:* all (badged). + +**Slide A2 — Operating Model & Cost** +- *Opens:* "This appendix shows the real cost figures + the zero-cost + steady state." +- *Delivers:* `COST.md` figures ($0.001883 / 8 days, ~$0.007/mo, + S3-dominated, zero BAU compute) + the zero-cost-steady-state / D-096 + teardown claim. References the pre-mortem (`PRE_MORTEM.md`: v1.10 decay + root cause + four forward failure modes + structural mitigations). +- *Closes:* "Benefit: you now know the operating cost is negligible — and + the structural mitigation that prevents decay." +- *Grounded metrics cited:* Cost figures (source: `COST.md`). +- *Deferred metrics:* none. + +### Fluidity strategy + +1. **Every slide's opening line references the previous slide's close.** + Each slide above has an explicit transition sentence. No disjointed + jumps. +2. **Act indicator in the Marp footer.** `Act N/5: ` keeps the + audience oriented. Configured in the Marp theme. +3. **The arc is visible.** Slide 1 (arc preview) + slide 15 (recap) bookend + the deck. The audience always knows where they are in the 5-act + structure. +4. **Per-slide benefit callout is the last line.** Every slide closes with + "Benefit: ..." — the audience leaves each slide with a takeaway, not a + cliffhanger. +5. **The Proof act is the centerpiece.** It is 4 slides (the longest act) + because the PO's direction is "prove it, don't promise it." The + grounded/derived/deferred honesty model is the narrative spine of the + Proof act. +6. **Deferred metrics are shown, not hidden.** Slide 13 ("What's Deferred + — and Why") pairs each deferred metric with its blocking decision. + This is the honesty that makes the grounded claims credible. + +### Deck file inventory (after P5) + +| File | Status | +|------|--------| +| `docs/presentations/nova-no-humans-platform.md` | NEW (source of truth) | +| `docs/presentations/nova-no-humans-platform-marp.md` | NEW (Marp) | +| `docs/presentations/nova-no-humans-platform.html` | NEW (rendered) | +| `docs/presentations/nova-no-humans-platform-talking-points.md` | NEW (talking points) | +| `docs/presentations/how-the-platform-works.md` | DELETED (retired, D-130) | +| `docs/presentations/how-the-platform-works-marp.md` | DELETED | +| `docs/presentations/how-the-platform-works.html` | DELETED | +| `docs/presentations/how-the-platform-works-talking-points.md` | DELETED | +| `docs/presentations/the-developer-experience.md` | DELETED (retired, D-130) | +| `docs/presentations/the-developer-experience-marp.md` | DELETED | +| `docs/presentations/the-developer-experience.html` | DELETED | +| `docs/presentations/the-developer-experience-talking-points.md` | DELETED | + +--- + +## Wave Dependency Graph + +``` +Wave 1 Wave 2 Wave 3 Wave 4 Final + ┌──────────────────┐ ┌──────────────────┐ ┌──────────────┐ +P1 (event emitters)──┤P2 (collector) │ │P4 (catalog + │ │P6 (regression│ P8 + │ P3 (powerbi │──▶│ NORTH_STAR │──▶│ capability) │──▶(ship) + │ export) │ │ integration) │ │P7 (review + │ + └──────────────────┘ │P5 (deck rebuild) │ │ audit + ship)│ + └──────────────────┘ └──────────────┘ + +Critical path: +P1 ──▶ P2 ──▶ P3 ──▶ P4 ──▶ P5(Proof) ──▶ P6 ──▶ P7 ──▶ P8 + +Parallelization: + Wave 2: P2 schemas + P3 view schemas can be authored concurrently. + Wave 3: P4 docs/metrics/*.md + P5 Problem/Vision/How acts can be authored + concurrently; P5 Proof act waits for P4 METRICS.md. + Wave 4: P6 CAP-023 test can be drafted while P5 finishes. +``` + +**Dependency details:** + +| Phase | Depends on | Blocks | +|-------|------------|--------| +| P1 | (none — foundation) | P2, P3, P4, P5, P6 | +| P2 | P1 (event formats) | P3 (SQLite store), P4 (catalog sources), P6 (CAP-023) | +| P3 | P2 (SQLite store) | P4 (PowerBI view references), P6 (CAP-023 schema) | +| P4 | P2 + P3 (grounded metrics) | P5 (Proof act citations), P6 (CAP-024 deck structure) | +| P5 | P4 (METRICS.md for Proof act) | P6 (CAP-024 deck structure) | +| P6 | P2 + P3 (CAP-023) + P5 (CAP-024) | P7 (regression gate must pass) | +| P7 | P1–P6 (all prior phases) | P8 (audit must pass) | +| P8 | P7 (milestone complete) | (none — terminal) | + +--- ## Execution approach - **Per-phase ship:** each execution phase merges `phase/NN-*` → - `milestone/v1.16-nova-simplification` and tags a patch on the v1.15.x - line (`v1.15.6` = P1 ... `v1.15.26` = P21). -- **Verification:** 4-layer verify (structural/behavioral/security/ - quality) per phase; the regression gate (D-091, 22 capabilities) runs - at P9 (end of Wave 2) and P21 (milestone complete) per D-118. -- **No live AWS:** `NOVA_LIFECYCLE_MODE=plan` default; terraform changes - validated via `terraform validate` + `--check-only`. P20 cross-account - Terraform is offline-proven only (D-114). + `milestone/v1.17-strategic-metrics-deck` and tags a patch on the v1.16.x + line (`v1.16.1` = P1 ... `v1.16.7` = P7, `v1.16.8` = P8 final). +- **Verification:** 4-layer verify (structural/behavioral/security/quality) + per phase; the regression gate (D-091, 22 prior + CAP-023 + CAP-024 = 24 + capabilities) runs at P6 and P7. +- **No live AWS:** `NOVA_LIFECYCLE_MODE=plan` default; all metrics that + require live AWS ship as placeholder views (D-096). Infracost runs + offline (reads plan JSON, A6). - **Test discipline:** each phase that changes runtime code adds/updates tests; `bash scripts/run_ci.sh` exits 0 at every phase boundary. - -## Wave 1 — Correctness + Brand Regression Fixes (P1–P4) - -### Phase P1 — state-bucket-and-kyverno-rebrand-fix (REQ-165) -- **Lead:** backend-engineer; **Contributor:** data-engineer (kyverno) -- **Must-haves:** - - `adapters/terraform/adapter.py:117` `state_bucket = - f"acdl-tfstate-{account_id}-us-east-1"` → `f"nova-tfstate-{account_id}-us-east-1"`. - - `adapters/kyverno/policies/require-resource-labels.yml`: annotation - title `Require ACDL Resource Labels` → `Require Nova Resource Labels`; - rule names `require-acdl-owner-label`/`require-acdl-environment-label` - → `require-nova-owner-label`/`require-nova-environment-label`; - messages + patterns `acdl:owner`/`acdl:environment` → `nova:owner`/ - `nova:environment`. - - Update any test fixtures referencing the old bucket name / label keys. -- **Verify:** `terraform validate` (adapter-emitted); pytest passes; - `run_ci.sh` exits 0; regression gate 22/22 (run at P9, but P1 must not - break any cap locally). - -### Phase P2 — user-facing-acdl-to-nova-sweep (REQ-166) -- **Lead:** lead-developer; **Contributor:** backend-engineer -- **Must-haves:** - - `core/environment_check.py:59,61` onboarding message header/body - "ACDL" → "Nova". - - `core/lambda/contract_ingestor.py:145` alert title `[ACDL-ALERT]` → - `[NOVA-ALERT]`; `:191` issue body "ACDL platform Lambda" → "Nova - platform Lambda". - - `scripts/post_stage_comment.sh:39` PR comment header "ACDL Stage" → - "Nova Stage"; `:46` footer "ACDL deploy pipeline" → "Nova deploy - pipeline". - - `scripts/run_ci.sh:39` CI banner "ACDL CI Pipeline" → "Nova CI - Pipeline". - - Module docstrings: `core/contract_resolver.py:1,474`, - `core/confidence_signal.py:1`, `adapters/terraform/adapter.py:1`, - `adapters/kyverno/kyverno_adapter.py:1`, `adapters/wiz/wiz_adapter.py:1`, - `adapters/README.md:1`, `adapters/kyverno/README.md:4,18` → Nova. - - Update tests that assert these strings. -- **Verify:** pytest passes; `run_ci.sh` exits 0. - -### Phase P3 — dead-code-and-stale-prefix-cleanup (REQ-167) -- **Lead:** lead-developer -- **Must-haves:** - - `scripts/run_platform.sh:153` remove the dead - `export ACDL_ENVIRONMENT_OVERRIDE=...` line (comment says "removed - in P5" but the line is present). - - Stale dual-read comments: drop the "ACDL_* fallback until P5" / - "dual-read NOVA_* first, ACDL_* fallback per G-106" comments in - `core/local_emulators.py:15-16,503,505`, - `core/regression_verify.py:318-319,333`, and the lifecycle scripts - (the G-106 fallback is retired per `core/env.py:4-5`). - - `acdl_*` temp-dir prefixes → `nova_*`: `core/local_emulators.py:71,252` - (`acdl_outbox_`/`acdl_tfstate_`), `core/regression_verify.py:183,234` - (`acdl_regr_`/`acdl_outbox_`), `scripts/run_pattern_plan.sh:29`, - `scripts/run_primitive_plan.sh:29`, `scripts/run_lifecycle_test.sh:41`, - `scripts/run_lifecycle_destroy.sh:36`. - - `core/regression_verify.py:214` interpolation fixture `acdl-` → `nova-` - (or make it a clearly-generic token). -- **Verify:** pytest passes; `run_ci.sh` exits 0. - -### Phase P4 — migrate-ssm-except-narrowing (REQ-168) -- **Lead:** backend-engineer -- **Must-haves:** - - `scripts/migrate_ssm_paths.py:113` `except Exception: pass` → - narrow to `ParameterNotFound` + structured log on the non- - ParameterNotFound path. - - Narrow `core/output_publisher.py:112,182` `except Exception` → - specific `(ClientError, OSError)` + structured stderr log. - - Test that a non-ParameterNotFound error is raised (not swallowed). -- **Verify:** pytest passes; `run_ci.sh` exits 0. - -## Wave 2 — Simplify Without Regressions (P5–P9) - -### Phase P5 — regression-verify-dedup (REQ-169) -- **Lead:** backend-engineer -- **Must-haves:** - - Extract `_check_live_terraform_plan(contract_path, label)` from the - two ~95% identical methods `_check_live_terraform_plan_microservice` - + `_check_live_terraform_plan_static_assets` (~35 lines saved). - - Extract `_check_resolver(contract_path)` from - `_check_resolver_static_assets` + `_check_resolver_microservice`. - - Extract `_assert_contracts_resolve(module_dir)` from the duplicated - lifecycle-contract-resolve block in - `_check_lifecycle_module_terraform` + `_check_lifecycle_l2_module`. - - Behavior preserved (the regression gate output is unchanged). -- **Verify:** pytest passes; `run_ci.sh` exits 0. - -### Phase P6 — run-platform-deadcode-and-hitl-fn (REQ-170) -- **Lead:** lead-developer -- **Must-haves:** - - Extract the duplicated HITL attestation block (`:336-350` + `:452-466`) - into a shell function `run_hitl_gate()` invoked at both sites (~14 - lines saved). - - `scripts/run_platform.sh:145` hardcoded `CONTRACT_ID` UUID → - `NOVA_CONTRACT_ID` env with the existing UUID as default. - - `scripts/run_platform.sh:146` `WORK="/tmp/acdl_platform_run_v18"` → - `WORK="${NOVA_WORK_DIR:-/tmp/nova_platform_run}"` (drop the stale - `v18` stamp + `acdl_` prefix). - - Drop the stale brand comment `run_platform.sh:2` "the ACDL platform - pipeline" → "the Nova platform pipeline". -- **Verify:** pytest passes; `run_ci.sh` exits 0; `run_platform.sh - --check-only` exits 0. - -### Phase P7 — contract-resolver-envloader-and-kind (REQ-171) -- **Lead:** backend-engineer -- **Must-haves:** - - `core/contract_resolver.py:50-68` `_load_env` → import - `core/environment_check.py:load()` (dedup; both load + placeholder - warning). - - Add a `kind` field (`"l1"` / `"l2"`) to each `modules/registry.json` - entry; the resolver reads `kind` directly instead of the fragile - `is_l2 = "l2" in interface_path or "composition" in interface_path` - heuristic (`contract_resolver.py:540`). - - Collapse the redundant `kind` computation (`:584-589`) → - `kind = "l2" if (multi_module or any_l2) else "l1"` (after the - registry `kind` field is authoritative, simplify further). -- **Verify:** pytest passes; `run_ci.sh` exits 0; resolver behavior - unchanged (all contracts still resolve to the same stacks). - -### Phase P8 — workflow-generator-dedup (REQ-172) -- **Lead:** lead-developer; **Contributor:** backend-engineer (test) -- **Must-haves:** - - Author `scripts/sync_workflows.py` — reads one source workflow per - pair (e.g. `workflows-src/ci.yml`, `workflows-src/deploy.yml`, - `workflows-src/modules-lifecycle.yml`) and writes byte-identical - copies to both `.gitea/workflows/` and `.github/workflows/`. - Establish the `workflows-src/` dir as the single source. - - Replace the byte-identity assertions in - `tests/test_pipeline_contract.py` with a "generated outputs match - committed files" test (run `sync_workflows.py --check` → exit 0 if - the committed files match the generated output, non-zero + diff if - drift). - - Migrate the 3 existing pairs to the `workflows-src/` source; remove - the hand-maintained duplicates (the generator owns them). -- **Verify:** `python3 scripts/sync_workflows.py --check` exits 0; - pytest passes; `run_ci.sh` exits 0; the 4 GitHub-only workflows are - untouched (they have no pair). - -### Phase P9 — run-platform-split (REQ-173) -- **Lead:** lead-developer -- **Must-haves:** - - Extract the decommission block (`scripts/run_platform.sh:180-237`) - into `scripts/run_decommission.sh` (sourced or invoked). - - Extract the uptime block (`:520-606`) into `scripts/run_uptime.sh`. - - `run_platform.sh` invokes the helpers; behavior unchanged. - - **G-112 binding:** the helpers are **`source`d** (shared shell env), - not invoked as subshells — the extracted blocks reference - `run_platform.sh`-local vars (`NOVA_CONTRACT_ID`/`NOVA_WORK_DIR` from - P6); a subshell would not inherit them. - - **G-111 binding:** update `core/regression_verify.py` CAP-015/016 - checks — when the live resource is absent - (`ResourceNotFoundException`/`404`), mark `Skipped (post-teardown, - D-096)` not `Decayed`, so a clean local run reports 20/20 Verified + - 2 Skipped (not a strict-`all` failure on the known teardown state). - - **Run the regression gate (D-118, end of Wave 2):** **20/22 Verified** - is the passing bar (CAP-015/016 Skipped — post-v1.11-teardown steady - state, D-096; re-provisioning is a future feature, not an NFR). Any - non-Verified/non-Skipped capability halts Wave 3. -- **Verify:** pytest passes; `run_ci.sh` exits 0; `run_platform.sh - --check-only` exits 0; **regression gate 20/22 Verified + 2 Skipped**. - -## Wave 3 — Security + Maintainability (P10–P14) - -### Phase P10 — contract-ingestor-defense-in-depth (REQ-174) -- **Lead:** backend-engineer; **Contributor:** lead-developer (review) -- **Must-haves:** - - `core/lambda/contract_ingestor.py:251-252` `if not caller_arn: pass` - → fail closed: return a 401/403 with a clear message when IAM identity - is absent (defense-in-depth; ABAC layer still the primary control). - - `core/lambda/contract_ingestor.py:269` hardcoded - `valid_envs = {"dev","qa","prod","dr"}` → derive from the - `core/environments/` directory (list `*.json` filenames). - - Document the ABAC reliance explicitly in the function docstring + - ARCHITECTURE.md. - - Test: a request without IAM identity is rejected; a request with an - unknown environment is rejected. -- **Verify:** pytest passes; `run_ci.sh` exits 0. - -### Phase P11 — contract-ingestor-payload-validation (REQ-175) -- **Lead:** backend-engineer -- **Must-haves:** - - `submit_contract`: size-cap the `contract` blob (e.g. 256 KB) before - the DynamoDB write; reject oversized payloads with 413. - - Schema-validate the contract blob against `schemas/contract.schema.json` - before the write; reject invalid with 400. - - Consistent caps: `error` and `stackTrace` use the same cap (align the - 10k vs 2k inconsistency). - - Tests for size-limit + schema-rejection paths. -- **Verify:** pytest passes; `run_ci.sh` exits 0. - -### Phase P12 — split-contract-resolver (REQ-176) -- **Lead:** backend-engineer -- **Must-haves:** - - Split `core/contract_resolver.py` (638 lines) into: - `core/contract_resolve.py` (the resolve + interpolation core), - `core/decommission_transform.py` (the decommission zero-counts - transform), `core/contract_resolver_cli.py` (the `__main__` CLI). - - `core/contract_resolver.py` becomes a thin re-export shim for - backwards compat (existing imports keep working). - - **G-113 binding:** import direction is one-way — split modules - import only each other + stdlib; the re-export shim imports the - split modules; nothing imports the shim except external callers - (prevents the latent cycle shim → split → split → shim). - - Behavior unchanged; all tests pass without modification. -- **Verify:** pytest passes; `run_ci.sh` exits 0. - -### Phase P13 — split-regression-verify (REQ-177) -- **Lead:** backend-engineer -- **Must-haves:** - - Split `core/regression_verify.py` (670 lines) into: - `core/regression_capabilities.py` (the CAP-001..022 checks), - `core/regression_live_plan.py` (the shared live-plan helpers from - P5), `core/regression_verify_cli.py` (the `__main__` CLI + - `run_regression` orchestration). - - `core/regression_verify.py` becomes a thin re-export shim. - - Behavior unchanged; the regression gate output is identical. -- **Verify:** pytest passes; `run_ci.sh` exits 0. - -### Phase P14 — schema-driven-outputs-and-cache (REQ-178) -- **Lead:** backend-engineer; **Contributor:** data-engineer (interface.json) -- **Must-haves:** - - `core/output_publisher.py:38-55` `SAFE_OUTPUT_NAMES` hardcoded set → - derived from `modules/l1/*/interface.json` `outputs[].sensitive` - annotations (non-sensitive outputs are safe to publish). - - `core/contract_resolver.py:498,617` (now in the split module) — - cache loaded JSON schemas in a module-level dict (avoid re-reading - from disk each resolve call). - - **Mid-milestone checkpoint (offline):** regression gate spot-check - (not the full P9/P21 gate); confirm Wave 3 introduced no regressions. -- **Verify:** pytest passes; `run_ci.sh` exits 0. - -## Wave 4 — Developer Experience (P15–P17) - -### Phase P15 — run-platform-help-and-flags-doc (REQ-179) -- **Lead:** lead-developer -- **Must-haves:** - - `scripts/run_platform.sh` add a real `--help` / `-h` flag that - prints all flags + a one-line description each (`--check-only`, - `--plan-only`, `--apply`, `--destroy`, `--quiet`, `--deploy-uptime`, - `--decommission`, `--local`, `--environment`). The current `:82` - reject-unknown-flags path must allow `--help` to print + exit 0. - - Document `--deploy-uptime` in the header comment block (currently - used at `:532` but absent from the header). - - Surface `--local` (D-092 local emulating tier) in the README "How to - run" section. -- **Verify:** `run_platform.sh --help` exits 0 and lists all flags; - pytest passes; `run_ci.sh` exits 0. - -### Phase P16 — workflows-readme-catalog (REQ-180) -- **Lead:** lead-developer -- **Must-haves:** - - Author `.github/workflows/README.md` cataloging all 7 workflows: - `ci.yml`, `deploy.yml`, `platform-test.yml`, `primitives-plan.yml`, - `patterns-plan.yml`, `release.yml`, `modules-lifecycle.yml`. For - each: trigger (`on:`), inputs (reusable-workflow `workflow_call` - inputs), required secrets, and one-line purpose. - - Note which 3 are byte-identical Gitea mirrors (post-P8, generated by - `sync_workflows.py`) and which 4 are GitHub-only (Gitea act_runner - feature gaps). - - Add a `tests/test_docs_coverage.py` assertion that the README exists - + lists all 7 workflow filenames. -- **Verify:** pytest passes; `run_ci.sh` exits 0. - -### Phase P17 — getting-started-consolidation (REQ-181) -- **Lead:** lead-developer -- **Must-haves:** - - Consolidate the README "How to run" into a single getting-started - section: **offline happy path first** (`bash scripts/run_ci.sh` + - `bash scripts/run_platform.sh --check-only` / `--local` — no AWS - needed), then the **AWS path** (bootstrap + `--apply`). - - Remove the fragmented 3-step bootstrap as the lead; demote it to - the AWS-path subsection. - - Cross-link `docs/CONSUMER_GUIDE.md` for the consumer contract model. -- **Verify:** pytest passes; `run_ci.sh` exits 0. - -## Wave 5 — No Humans Onboarding Flow (P18–P20) - -### Phase P18 — onboarding-schema-and-lambda-action (REQ-182) -- **Lead:** backend-engineer; **Contributor:** lead-developer (schema) -- **Must-haves:** - - Author `schemas/onboarding.schema.json` (JSON Schema draft 2020-12): - required fields `consumerRepo` (string, format), `requestedEnvironment` - (string, enum from environments dir), `ownerId` (string), `billingTag` - (string); optional `notes`. - - `core/lambda/contract_ingestor.py` add an `onboard_consumer` action - (D-119): validates the payload against the onboarding schema, writes - a `pending` row to `nova-contracts` (PK `consumerRepo`, SK - `onboarding##`, status `pending`). - No AWS resources created (D-113). - - Tests: valid onboarding request writes a pending row; invalid request - rejected with 400; offline-testable via moto/local Lambda stub. -- **Verify:** pytest passes; `run_ci.sh` exits 0. - -### Phase P19 — onboarding-envfile-autogen (REQ-183) -- **Lead:** backend-engineer; **Contributor:** lead-developer (docs) -- **Must-haves:** - - Author `core/onboarding.py` with `generate_env_file(request, - template_env="dev")` — produces a `.json` from a consumer - onboarding request (fills `account_id` placeholder, `ownerId`, - `billingTag` into the env template). Emits the file + a git patch / - PR-branch instruction. - - Rebrand `core/environment_check.py:57-77` onboarding message to - Nova; replace the "1. Contact the platform team" handoff with the - self-service request path: "Run `nova onboard` (or POST to the - Lambda `onboard_consumer` action) to request an environment; the - platform generates a binding + opens a PR." - - Update `core/environments/README.md:34-37` — self-service request - path is now implemented (real provisioning still a future feature). - - Tests: `generate_env_file` produces a valid env JSON; the rebranded - message no longer says "contact the platform team". -- **Verify:** pytest passes; `run_ci.sh` exits 0. - -### Phase P20 — cross-account-role-automation-offline (REQ-184) -- **Lead:** data-engineer; **Contributor:** backend-engineer (ABAC) -- **Must-haves:** - - Author `terraform/onboarding/` (new dir): `main.tf` defining the - consumer deploy-role + `nova:owner` ABAC tag grant (cross-account - IAM role + trust policy + tag-based permission boundary). Variables - for `consumer_repo`, `owner_id`, `account_id`. - - `terraform validate` passes; `terraform plan` (offline / no live - apply per D-114) produces the expected role + policy. - - Document the onboarding Terraform in `docs/ONBOARDING.md` — the - request path (P18) → env-file autogen (P19) → role grant (P20, this - phase, offline-proven; live apply deferred). - - Tests: `terraform validate` for the onboarding module; a - `test_onboarding_terraform.py` asserting the module validates. -- **Verify:** `terraform validate` (onboarding module) passes; pytest - passes; `run_ci.sh` exits 0. - -## Final Phase — P21 — final-review-ship - -- **Lead:** lead-developer; **Contributors:** all active (review) -- **Must-haves:** - - Multi-persona code review across all v1.16 phases (ci-code-reviewer). - Auto-apply P0 fixes; flag P1+ for post-hoc review. If P1+ found, fix - in this phase (not loop back to EXECUTE). - - Audit (ciagent-audit): reconstruction test (git log matches - `.ciagent/` files), file discipline, branch hygiene, commit - discipline. Fix critical issues in this phase. - - **Run the regression gate (D-118, milestone complete):** **20/22 - Verified** (CAP-015/016 Skipped — post-teardown steady state, D-096). - - Update `.ciagent/REQUIREMENTS.md` — mark REQ-165..184 complete. - - Update `.ciagent/ROADMAP.md` — mark v1.16 complete. - - Update `.ciagent/PROJECT.md` — v1.16 complete summary. - - Ship: merge `phase/21-final-review-ship` → - `milestone/v1.16-nova-simplification`; merge milestone → `main`; - tag `v1.15.26` (= milestone release); create Gitea release with full - milestone summary. - - Clear CHECKPOINT.json (milestone complete). - -## Success Criteria (milestone gate) - -- All 20 requirements (REQ-165..184) satisfied; 0 partial. -- Regression gate **20/22 Verified + 2 Skipped** at P9 + P21 (D-118, - G-111; CAP-015/016 are the post-v1.11-teardown steady state, D-096). -- `bash scripts/run_ci.sh` exits 0 at every phase boundary. -- Review: 0 new P0; P1+ flagged or auto-fixed. -- Audit: clean; reconstruction test passes. -- Tag `v1.15.26` created; milestone merged to main. -- Onboarding request path implemented (P18–P20); real AWS provisioning - explicitly deferred (D-113, D-114). \ No newline at end of file +- **No fabrication:** every metric carries a grounded/derived/deferred + status with a source file or decision ID. No fabricated numbers in any + deck slide or METRICS.md entry. +- **Decision discipline:** D-120..D-132 are locked. This plan does not + re-open any locked decision. If a decision needs revisiting, it goes + through the GRILL, not the plan. \ No newline at end of file