diff --git a/.ciagent/CHECKPOINT.json b/.ciagent/CHECKPOINT.json index 861fb51..e8b82fd 100644 --- a/.ciagent/CHECKPOINT.json +++ b/.ciagent/CHECKPOINT.json @@ -1,12 +1,12 @@ { "phase": 0, - "stage": "plan", + "stage": "grill", "milestone": "v1.17", "phase_role": "pre_execution", "attempts": 0, - "updated_at": "2026-08-04T20:45:00Z", + "updated_at": "2026-08-04T21:15:00Z", "milestone_complete": false, "tag": null, "requirements": ["REQ-185"], - "notes": "PLAN complete. 8 phases (P1..P7 + P8 final), 4 waves. Deck rebuild plan: 15 main + 2 appendix slides, 5-act arc, per-slide benefit callouts, fluid transitions. 29 requirements (REQ-185..213) covered. Wave dependency graph + critical path documented." + "notes": "GRILL complete (interactive). 12 binding decisions applied: NORTH_STAR targets reclassified (E-003: 3 targets to Post-Pilot section; E-004: AI-Agent Intent Share to Future Horizons). Deck plan updated: slide 1 stake line (G-Q8), slide 4 benefit rewrite (G-Q9), slide 7 D-122 honesty sentence (G-Q4), Act 3->4 transition rewrite (G-Q13), slide 9 benefit reframe (G-Q14), slide 12 split into 12+13 (G-Q10), ROI formula inline + N=0 caveat (G-Q5/G-Q15), slide 14 preempt (G-Q11), slide 16 ask reframed as business decision (G-Q16). Deck now 16 main + 2 appendix = 18 slides." } \ No newline at end of file diff --git a/.ciagent/NORTH_STAR.md b/.ciagent/NORTH_STAR.md index ccb0c74..de0f4f6 100644 --- a/.ciagent/NORTH_STAR.md +++ b/.ciagent/NORTH_STAR.md @@ -112,25 +112,53 @@ Targets are committed, not aspirational. Each is a number a board member can repeat back to us. The grounding column records whether the metric is measurable this milestone, and if not, what blocks it. +> **Honesty note (GRILL G-Q6 binding):** Nova has 0 consumer adoption +> today (`PROJECT.md:495`). Three targets (Touchless Resolution, Human +> Escalation, AI Decision Accuracy) are scoped "across production +> estates" — the measurement *pipeline* is grounded this milestone, but +> the *denominator* is zero until a pilot estate activates. These +> targets are reclassified as **Post-Pilot** (the pipeline works; the +> numbers fill when consumers exist). This is the same honesty model as +> Cloud Spend Reduction (partial: pipeline grounded, actuals deferred). + +### Current-milestone targets (grounded or derived this milestone) + | Domain | Target | Grounding (v1.17) | Note | |---|---|---|---| -| **Touchless Resolution Rate** | ≥ 99% across production estates | grounded (after P1) | runs completing without *operational* HITL block ÷ total runs (attestation gates excluded — they're designed controls, not escalations) | -| **Human Escalation Frequency** | < 0.1% of platform actions | grounded (after P1) | *operational* HITL blocks only (confidence-driven); attestation sign-offs excluded | | **MTTR (p95)** | < 60 seconds | grounded (platform-run MTTR) | apply.failed → successful retry; infra-incident MTTR deferred (no incident detection) | -| **Predictive vs. Reactive Ratio** | ≥ 3 : 1 (prevention dominates reaction) | deferred | requires ML forecasting service (future emitter) | -| **AI Decision Accuracy** | ≥ 99.5% (no rollback, no follow-up incident within 5 min of action) | grounded (after decision ledger) | decisions not followed by apply.failed/incident within 5min | -| **Drift Auto-Reversal Rate** | ≥ 95% within one detection cycle | deferred | requires drift detection (D-096 + scheduler) | | **Cloud Spend Reduction** | ≥ 25% on pilot estates vs. 12-month pre-Nova baseline | partial | pre-apply estimate grounded (Infracost); actual-spend deferred (D-096 CUR) | -| **L1 / L2 Ops Hours Avoided** | ≥ 70% of pre-Nova FTE allocation | derived | formula over run count × manual baseline | -| **Platform ROI** | ≥ 250% measured annually | derived | formula (labor savings + cloud savings + avoided downtime) ÷ platform op cost | +| **L1 / L2 Ops Hours Avoided** | ≥ 70% of pre-Nova FTE allocation | derived | formula over run count × manual baseline (computed on N internal runs; production-denominator activates post-pilot) | +| **Platform ROI** | ≥ 250% measured annually | derived | formula (labor savings + cloud savings + avoided downtime) ÷ platform op cost (computed on N internal runs; production-denominator activates post-pilot) | | **Decision Ledger Coverage** | 100% of AI actions with backfilled outcome | grounded (this milestone builds it) | outbox_writer.py → SQLite hash-chain | | **Attestation Coverage** | 100% of prod/dr promotions attested by a human | grounded | hitl_gates.py + outbox approver_* attributes; separation-of-duties on prod | -| **AI-Agent Intent Share** | ≥ 40% of total intent volume originated by non-human consumers | future | no AI-agent consumers today; no emitter; placeholder view | + +### Post-Pilot targets (pipeline grounded this milestone; denominator activates when a pilot estate runs) + +| Domain | Target | Grounding (v1.17) | Note | +|---|---|---|---| +| **Touchless Resolution Rate** | ≥ 99% across production estates | partial (pipeline grounded; denominator = 0 today) | runs completing without *operational* HITL block ÷ total runs (attestation gates excluded); activates post-pilot | +| **Human Escalation Frequency** | < 0.1% of platform actions | partial (pipeline grounded; denominator = 0 today) | *operational* HITL blocks only (confidence-driven); attestation sign-offs excluded; activates post-pilot | +| **AI Decision Accuracy** | ≥ 99.5% (no rollback, no follow-up incident within 5 min of action) | partial (pipeline grounded; denominator = 0 today) | decisions not followed by apply.failed/incident within 5min; activates post-pilot | + +### Deferred targets (measurement requires future systems) + +| Domain | Target | Grounding (v1.17) | Note | +|---|---|---|---| +| **Predictive vs. Reactive Ratio** | ≥ 3 : 1 (prevention dominates reaction) | deferred | requires ML forecasting service (future emitter) | +| **Drift Auto-Reversal Rate** | ≥ 95% within one detection cycle | deferred | requires drift detection (D-096 + scheduler) | > Committed targets whose measurement is deferred remain committed — the > target is the destination; the metric is the odometer, and some > odometers aren't built yet. Each deferred metric ships as a placeholder > PowerBI view + a definition-of-success doc recording the dependency. +> Post-Pilot targets are committed targets whose measurement pipeline is +> grounded this milestone; the numbers activate when a pilot estate runs. + +### Future Horizons (strategic direction, not committed targets) + +| Domain | Aspiration | Note | +|---|---|---| +| **AI-Agent Intent Share** | ≥ 40% of total intent volume originated by non-human consumers | Strategic Objective #4 direction. No backing requirement, no placeholder view, no emitter today. Moves to a committed target when agentic consumption is real. | --- diff --git a/.ciagent/PLAN.md b/.ciagent/PLAN.md index 6602182..efe65e7 100644 --- a/.ciagent/PLAN.md +++ b/.ciagent/PLAN.md @@ -724,7 +724,7 @@ release), publish the Gitea release, and delete the milestone branches. | Level | "What I'm going to tell you" | "Tell them" | "What I told you" | |-------|------------------------------|-------------|-------------------| -| **Deck** | Slide 1 (arc preview: Problem→Vision→How→Proof→Roadmap) | Slides 2–14 (the 5 acts) | Slide 15 (recap of 5 acts + the ask) | +| **Deck** | Slide 1 (arc preview: Problem→Vision→How→Proof→Roadmap; with 18V+0-consumer stake line) | Slides 2–15 (the 5 acts, 14 slides) | Slide 16 (recap of 5 acts + the business-decision ask) | | **Per slide** | Opening line: "This slide shows X" | Body: bullets/diagram/table | Closing line: "Benefit: you now know Y" | ### Act 1 — Problem (2 slides) @@ -735,11 +735,16 @@ release), publish the Gitea release, and delete the milestone branches. **Slide 1 — Arc Preview (the "what I'm going to tell you" deck-level opening)** - *Opens:* "This deck proves Nova is the no-humans infrastructure platform — and shows you the metrics that make the claim defensible." +- *Stake line (G-Q8 binding):* "Today: 18 capabilities verified, 0 consumer + estates in production. This deck shows what's proven, what's pipeline-ready, + and what's honestly deferred." - *Delivers:* The 5-act arc as a visual roadmap: Problem → Vision → How → Proof → Roadmap. One-line summary per act. -- *Closes:* "Benefit: you now know the arc — the next 14 slides deliver - each act in turn." -- *Grounded metrics cited:* none (this is the preview). +- *Closes:* "Benefit: you leave this deck knowing which claims are proven + today, which are pipeline-ready, and which are deferred with a documented + unblock path — no marketing, just grounded evidence." +- *Grounded metrics cited:* 18 Verified + 4 Skipped (source: + `REGRESSION_REPORT.json`); 0 consumers (source: `PROJECT.md:495`). - *Deferred metrics:* none. **Slide 2 — The No-Humans Imperative** @@ -783,12 +788,14 @@ release), publish the Gitea release, and delete the milestone branches. anti-goals (not a hyperscaler competitor, not a general AI platform, not removing humans from accountability, not for legacy infra, not sold to operators). From `NORTH_STAR.md`. -- *Closes:* "Benefit: you now know the scope boundaries — what Nova is, - and what it refuses to be." +- *Closes:* "Benefit: you now know the scope boundaries — Nova is + purpose-built for infrastructure operations, sold to leadership on + outcomes, and explicitly not a general-purpose AI platform or a + hyperscaler competitor." - *Grounded metrics cited:* none (direction). - *Deferred metrics:* none. - *Transition:* "The objectives are committed to measurable targets — - here is the 12–18 month scorecard." + here is the 12–18 month scorecard, with honest grounding status." **Slide 5 — 12–18 Month Targets (the scorecard)** - *Opens:* "This slide shows the committed targets — numbers a board @@ -841,8 +848,14 @@ release), publish the Gitea release, and delete the milestone branches. from `apply.completed`. `attestation.recorded` events for qa/prod/dr. D-121, D-122, D-132. Honors D-083 (no S3 Object Lock/JWS — local hash-chain this milestone). +- **D-122 honesty sentence (G-Q4 binding):** "Nova's 'AI' is the + confidence-gated policy engine (confidence_signal + HITL gate), not + an LLM planner. The Decision Ledger captures this real decision path — + not a fabricated 'AI agent' that doesn't exist yet." - *Closes:* "Benefit: you now know why 'autonomous' is defensible — every - decision is immutable, queryable, and accountable." + decision is immutable, queryable, and accountable. And you know exactly + what 'AI' means here: a confidence-gated policy engine, not a black-box + LLM." - *Grounded metrics cited:* Decision Ledger Coverage 100% (source: `core/metrics/decision_ledger.py` + `metrics/decision_ledger.db`). - *Deferred metrics marked Planned:* Tamper-Evident Ledger Checkpoints @@ -863,9 +876,13 @@ release), publish the Gitea release, and delete the milestone branches. - *Grounded metrics cited:* Attestation Coverage 100% (source: `core/hitl_gates.py` + outbox `approver_*` attributes). - *Deferred metrics:* none. -- *Transition:* "The platform is built — here is the proof that it works." +- *Transition into Act 4 (G-Q13 binding — rewritten):* "You've now seen + how Nova works — the pipeline, the Decision Ledger, the attestation + gates. But 'how it works' is not 'proof it works.' The next four slides + show the measured evidence: capability health, trust metrics, efficiency, + and cost — every number grounded in a real file, not a marketing claim." -**Slide 9 — Telemetry Architecture** +**Slide 9 — Telemetry Architecture (G-Q14 binding — benefit reframed from data plumbing to trust)** - *Opens:* "This slide shows how Nova instruments itself — the CloudEvents envelope, the cold store, and the PowerBI export." - *Delivers:* The telemetry architecture diagram (from ARCHITECTURE.md @@ -874,8 +891,10 @@ release), publish the Gitea release, and delete the milestone branches. → collector → `metrics/nova_metrics.db` (SQLite cold store) → `metrics/powerbi/` (CSV/JSON views) → PowerBI. D-120 (Nova-native), D-125 (hybrid events/files), D-126 (cold-only). -- *Closes:* "Benefit: you now know the metrics pipeline — every signal is - grounded in a real file, not fabricated." +- *Closes:* "Benefit: you now know that every metric in this deck is + traceable to a real emitted event — the architecture IS the trust + substrate. When a CFO asks 'where does this number come from?', the + answer is a file path, not a Slack thread." - *Grounded metrics cited:* none (architecture). - *Deferred metrics marked Planned:* Hot-path (live ops dashboard) — D-126. - *Transition into Act 4:* "The architecture is sound — here is the @@ -922,29 +941,58 @@ release), publish the Gitea release, and delete the milestone branches. - *Transition:* "Trust is provable — here is the operational efficiency that makes the ROI real." -**Slide 12 — Zero-Touch Efficiency + Cost** -- *Opens:* "This slide shows the efficiency metrics — touchless resolution, - human escalation, and cost estimates." +**Slide 12 — Zero-Touch Efficiency (G-Q10 binding — split from old slide 12)** +- *Opens:* "This slide shows the zero-touch efficiency metrics — + touchless resolution, human escalation, and MTTR." - *Delivers:* Touchless Resolution Rate (runs without operational HITL block ÷ total; attestation gates excluded). Human Escalation Frequency (operational HITL blocks only). MTTR (platform-run: apply.failed → - successful retry, D-131). Cost Estimates via Infracost (pre-apply, - grounded). FTE Hours Saved (derived). Platform ROI (derived formula). - The grounded/derived/deferred honesty model. -- *Closes:* "Benefit: you now know the ROI is quantifiable — each quarter - on Nova must reduce spend, free hours, and avoid downtime measurably." + successful retry, D-131). **Post-Pilot caveat (G-Q5 binding):** these + three metrics are computed on N internal runs today; the + production-denominator activates when a pilot estate runs (see + NORTH_STAR Post-Pilot Targets section). +- *Closes:* "Benefit: you now know the zero-touch efficiency is + measurable — the pipeline works today on internal runs, and the + denominator expands to production estates when a pilot activates." - *Grounded metrics cited:* Touchless Resolution Rate, Human Escalation - Frequency, MTTR, Cost Estimates (source: `metrics/nova_metrics.db` - `fact_run` + `fact_cost_estimate`). -- *Derived metrics:* FTE Hours Saved, Platform ROI. + Frequency, MTTR (source: `metrics/nova_metrics.db` `fact_run`). +- *Derived metrics:* none on this slide. +- *Deferred metrics marked Planned:* Self-Healing Velocity (no + auto-remediator). +- *Transition:* "Efficiency is half the ROI story — here is the cost + side." + +**Slide 13 — Cost & ROI (G-Q10 binding — split from old slide 12; G-Q15 binding — formula inline + N=0 caveat)** +- *Opens:* "This slide shows the cost estimates and the ROI formula — + with honest caveats about the current denominator." +- *Delivers:* Cost Estimates via Infracost (pre-apply, grounded). + **ROI formula shown inline (G-Q15 binding):** `Platform ROI = (FTE + hours saved × blended rate + cloud savings + avoided downtime) ÷ + platform op cost`. **N=0 caveat (G-Q5/G-Q15 binding):** "These + derived metrics are computed on N internal runs today; the + production-denominator activates post-pilot. The formula is grounded; + the production numbers are not yet." FTE Hours Saved (derived). Platform + ROI (derived formula). The grounded/derived/deferred honesty model. +- *Closes:* "Benefit: you now know the ROI formula — and you know it's + computed on internal runs today, not fabricated production numbers. + The formula is ready; the production denominator activates with a + pilot." +- *Grounded metrics cited:* Cost Estimates (source: + `metrics/nova_metrics.db` `fact_cost_estimate`). +- *Derived metrics:* FTE Hours Saved, Platform ROI (formula shown inline). - *Deferred metrics marked Planned:* Live CUR Reconciliation (D-096), Drift Auto-Reversal (D-096). - *Transition:* "The proof is grounded — here is what is honestly deferred." -**Slide 13 — What's Deferred — and Why** +**Slide 14 — What's Deferred — and Why (G-Q11 binding — preempt: deferrals are measurement infra, not whether the platform runs without humans)** - *Opens:* "This slide pairs each deferred metric with its blocking decision — honesty about what isn't measured yet." +- **Preempt (G-Q11 binding):** "To be clear: these deferrals are + *measurement infrastructure*, not whether the platform runs without + humans. The platform IS autonomous in operations. What's deferred is + the *evidence pipeline* for certain metrics (live infra health, drift + detection, predictive remediation) — not the autonomy itself." - *Delivers:* The 8 deferred metrics + onboarding-grant half, each paired with its blocking decision ID: (1) Live Infrastructure Health — D-096, (2) Live Outbox Write Rate — D-096, (3) Tamper-Evident Ledger @@ -954,7 +1002,8 @@ release), publish the Gitea release, and delete the milestone branches. (8) Predictive vs Reactive — future emitter. From `docs/METRICS_DEFERRED_ROADMAP.md`. - *Closes:* "Benefit: you now know the boundaries — what Nova measures - today, and exactly what blocks the rest." + today, and exactly what blocks the rest. The autonomy is real; the + measurement gaps are documented." - *Grounded metrics cited:* none (deferral honesty). - *Deferred metrics:* all 8 + onboarding-grant half, each with decision ID. - *Transition into Act 5:* "The proof is honest — here is the roadmap @@ -965,7 +1014,7 @@ release), publish the Gitea release, and delete the milestone branches. > **Transition into Act 5:** "The proof is honest — here is the roadmap > from here to the 12–18 month targets." -**Slide 14 — Roadmap to the North Star** +**Slide 15 — Roadmap to the North Star** - *Opens:* "This slide shows the path from v1.17's grounded metrics to the 12–18 month targets — the unblock path for each deferred metric." - *Delivers:* The deferred-metrics activation roadmap (from @@ -979,16 +1028,22 @@ release), publish the Gitea release, and delete the milestone branches. - *Deferred metrics:* all 8 referenced with unblock paths. - *Transition:* "The roadmap is clear — here is the recap and the ask." -**Slide 15 — Recap + Ask (the "what I told you" deck-level closing)** +**Slide 16 — Recap + Ask (the "what I told you" deck-level closing; G-Q16 binding — ask reframed as a business decision)** - *Opens:* "This slide recaps the 5 acts and states the ask." - *Delivers:* Recap: Problem (operator bottleneck) → Vision (invisible ops, provable trust) → How (pipeline + Decision Ledger + attestation) → - Proof (18V+4S, 100% ledger coverage, 100% attestation, grounded ROI) → - Roadmap (deferred metrics have unblock paths). The ask: fund the - hot-path activation (post-D-096) + the tamper-evident ledger build-out - (D-083 lift). -- *Closes:* "Benefit: you now know the full arc — and what is needed to - close the gap from grounded to complete." + Proof (18V+4S, 100% ledger coverage, 100% attestation, grounded ROI + formula) → Roadmap (deferred metrics have unblock paths). **The ask + (G-Q16 binding — reframed as a business decision, not insider + language):** "The ask is a business decision: approve a pilot estate + to activate the production-denominator metrics (Touchless Resolution, + Human Escalation, AI Decision Accuracy), and approve the tamper- + evident ledger build-out (D-083 lift) to move from local hash-chain + to S3 Object Lock + JWS. These two decisions move Nova from + 'pipeline-ready' to 'production-proven.'" +- *Closes:* "Benefit: you leave with a clear business decision to make + — approve a pilot + the ledger build-out — and the confidence that + every claim in this deck is grounded, derived, or honestly deferred." - *Grounded metrics cited:* Capability Health, Decision Ledger Coverage, Attestation Coverage (recap). - *Deferred metrics:* referenced as the ask. @@ -1021,28 +1076,41 @@ release), publish the Gitea release, and delete the milestone branches. 1. **Every slide's opening line references the previous slide's close.** Each slide above has an explicit transition sentence. No disjointed - jumps. + jumps. The Act 3→4 boundary (slide 9→10) was rewritten per G-Q13 + binding: "But 'how it works' is not 'proof it works.'" 2. **Act indicator in the Marp footer.** `Act N/5: ` keeps the audience oriented. Configured in the Marp theme. -3. **The arc is visible.** Slide 1 (arc preview) + slide 15 (recap) bookend - the deck. The audience always knows where they are in the 5-act - structure. +3. **The arc is visible.** Slide 1 (arc preview + stake line) + slide 16 + (recap + business-decision ask) bookend the deck. The audience always + knows where they are in the 5-act structure. 4. **Per-slide benefit callout is the last line.** Every slide closes with "Benefit: ..." — the audience leaves each slide with a takeaway, not a - cliffhanger. -5. **The Proof act is the centerpiece.** It is 4 slides (the longest act) - because the PO's direction is "prove it, don't promise it." The - grounded/derived/deferred honesty model is the narrative spine of the - Proof act. -6. **Deferred metrics are shown, not hidden.** Slide 13 ("What's Deferred - — and Why") pairs each deferred metric with its blocking decision. - This is the honesty that makes the grounded claims credible. + cliffhanger. Benefit callouts rewritten per G-Q9 binding (slides 1, 4, + 13, 16 now give specific value, not generic restatements). +5. **The Proof act is the centerpiece.** It is 5 slides (the longest act, + expanded from 4 per G-Q10 binding: slide 12 split into Zero-Touch + Efficiency + Cost & ROI) because the PO's direction is "prove it, don't + promise it." The grounded/derived/deferred honesty model is the + narrative spine of the Proof act. +6. **Deferred metrics are shown, not hidden.** Slide 14 ("What's Deferred + — and Why") pairs each deferred metric with its blocking decision, + with a preempt (G-Q11 binding) clarifying that deferrals are + measurement infrastructure, not whether the platform runs without + humans. +7. **The D-122 honesty sentence on slide 7.** The deck explicitly states + that Nova's "AI" is the confidence-gated policy engine, not an LLM + planner — per G-Q4 binding. This prevents the "no fabrication" + constraint from being violated by implication. +8. **Derived metrics carry the N=0 caveat.** Slides 12 and 13 annotate + derived metrics (FTE, ROI) with "computed on N internal runs; + production-denominator activates post-pilot" — per G-Q5/G-Q15 binding. + The ROI formula is shown inline (G-Q15). ### Deck file inventory (after P5) | File | Status | |------|--------| -| `docs/presentations/nova-no-humans-platform.md` | NEW (source of truth) | +| `docs/presentations/nova-no-humans-platform.md` | NEW (source of truth, 16 main + 2 appendix slides per G-Q10 split) | | `docs/presentations/nova-no-humans-platform-marp.md` | NEW (Marp) | | `docs/presentations/nova-no-humans-platform.html` | NEW (rendered) | | `docs/presentations/nova-no-humans-platform-talking-points.md` | NEW (talking points) |