docs(P3): marp deck + talking points + README + theme CSS + tests (REQ-245,251,252)
Nova Slides Render / render (push) Failing after 58s
Nova Slides Render / render (push) Failing after 58s
Marp deck (nova-autonomous-cloud-delivery-marp.md): synthesize from updated
source-of-truth; 18 main + 1 appendix slides; frontmatter — title 'Nova —
The Autonomous Cloud Delivery Platform', footer without version + without
'Act %{page}/5', title-slide subtitle 'Product Development & Citizen
Developer Overview'; no badges; embedded PNGs.
Talking points (nova-autonomous-cloud-delivery-talking-points.md):
re-distilled to 18-slide + A1 structure.
README.md: update deck title, audience, slide count (18 main + 1 appendix),
directory layout, remove badge docs, update deck table + render commands +
filenames. Document the v1.21 rename + restructure.
Theme CSS (nova-sp-theme.css): fix Appendix A1 table readability — tables
now have explicit white body + black text on any slide background
(including dark/title slides). Item 32.
Tests (test_slides_pipeline.py): add v1.21 assertions — no badges; no
version in footer/title slide; 18 main + 1 appendix slides; no D-###/REQ-
###/.py paths in audience-facing Marp deck or source slide body; old deck
files removed; render script default renamed; README references new deck
name. Update deck path in test_regression_cap023_024.py +
core/regression_verify.py CAP-024 (filename + 18-19 slide range, drop 'Arc
Preview' check per item 3).
attach_release_asset.py: usage example filename updated.
---ci---
project: acdl
phase: 3
milestone: v1.21
status: execute
phase_role: execution
---/ci---
This commit is contained in:
Jon Chery
@@ -3,331 +3,303 @@ marp: true
|
||||
theme: nova-sp
|
||||
paginate: true
|
||||
size: 16x9
|
||||
header: 'Nova — The No-Humans Infrastructure Platform'
|
||||
footer: 'Act %{page}/5 — v1.20'
|
||||
header: 'Nova — The Autonomous Cloud Delivery Platform'
|
||||
footer: 'Nova — The Autonomous Cloud Delivery Platform'
|
||||
---
|
||||
|
||||
<!-- _class: title -->
|
||||
<!-- _paginate: false -->
|
||||
|
||||
# Nova — The No-Humans Infrastructure Platform
|
||||
# Nova — The Autonomous Cloud Delivery Platform
|
||||
|
||||
**Shifting from Operational Overhead to Strategic Value**
|
||||
|
||||
v1.18 — Citizen Developer & Production-Grade Guidance
|
||||
Product Development & Citizen Developer Overview
|
||||
|
||||
---
|
||||
|
||||
## Slide 1 — Arc Preview
|
||||
## Slide 1 — The Problem
|
||||
|
||||
**This deck proves Nova is the no-humans infrastructure platform — and shows you the metrics that make the claim defensible.**
|
||||
**Product teams now own their cloud infrastructure — but ownership without discipline is destroying value.**
|
||||
|
||||
**Today:** 18 capabilities verified, 0 consumer estates in production.
|
||||
- **No lifecycle planning.** Resources are authored for creation, not for patching, decommissioning, or rollback — so changes are destructive.
|
||||
- **Proactive scanning is not part of authoring.** AI-frontier models exploit zero-days at a rapid pace; teams cannot keep up by reacting. Modules must be scanned as code and at runtime — and remediated at the pace the threat moves.
|
||||
- **Bandwidth gaps in infrastructure operations.** Time spent on remediation + the push for innovation leaves operations chronically under-resourced; detections are missed, incidents grow.
|
||||
- **Tribal knowledge and the rockstar-operator problem.** Operations depend on a handful of administrators; when they leave, the knowledge leaves with them. The platform should encode the discipline, not the person.
|
||||
|
||||
**The 5-act arc:**
|
||||
1. **Problem** — why the operator is the bottleneck
|
||||
2. **Vision** — Nova's strategic direction (NORTH_STAR)
|
||||
3. **How** — the pipeline, Decision Ledger, attestation gates
|
||||
4. **Proof** — grounded metrics that make the claim defensible
|
||||
5. **Roadmap** — deferred metrics with unblock paths + the ask + scope + RACI
|
||||
Every hour a developer spends writing, deploying, fixing, or remediating infrastructure is an hour not spent releasing features to production.
|
||||
|
||||
**Benefit:** you leave knowing which claims are proven today, which are pipeline-ready, and which are deferred with a documented unblock path — no marketing, just grounded evidence.
|
||||
**Benefit:** the answer is an autonomous cloud delivery platform that encodes discipline as policy, scans proactively, remediates rapidly, and makes operations visible to leadership rather than hidden in tribal knowledge.
|
||||
|
||||
---
|
||||
|
||||
## Slide 2 — The No-Humans Imperative
|
||||
## Slide 2 — Nova's Vision
|
||||
|
||||
**Why the operator is the bottleneck — and why removing them from operations (not accountability) is the imperative.**
|
||||
> **Infrastructure operations become visible. Every environment provisioned, every incident healed, every risk remediated — by an autonomous system whose trustworthiness is provable, not promised. Human attestation remains required at stage gates; the operator is never in the loop of normal operations.**
|
||||
|
||||
- **The cost of humans-in-the-loop:** L1/L2 ops hours, escalation latency, the trust gap
|
||||
- **The operator is the bottleneck:** provisioning takes days, not minutes
|
||||
- **The attestation model:** autonomy in operations, human at stage gates
|
||||
- Cites `docs/NO_HUMANS_THESIS.md`
|
||||
- **Visibility is the recurring theme** — security posture, remediation velocity, reliability, and lead time as queryable signals
|
||||
- **Provable, not promised** — trust established by deterministic scripts that calculate a score; the platform functions without AI
|
||||
- **Autonomy in operations, human at stage gates** — QA signs off for production; SRE greenlights operational readiness
|
||||
|
||||
**Benefit:** you now know the problem framing — autonomy in operations, human at stage gates, is the path forward.
|
||||
**Benefit:** the destination is autonomous operations with provable trust — security, remediation velocity, reliability, and lead time made visible to leadership, not promised to them.
|
||||
|
||||
---
|
||||
|
||||
## Slide 3 — Nova's Vision
|
||||
|
||||
> **Infrastructure operations become invisible. Every environment provisioned, every incident healed, every risk remediated — by an autonomous system whose trustworthiness is provable, not promised. Human attestation remains required at stage gates — QA signs off for production, SRE greenlights based on operational readiness — but the operator is never in the loop of normal operations.**
|
||||
|
||||
- Autonomy in operations, not in accountability
|
||||
- Cites `docs/NO_HUMANS_THESIS.md`
|
||||
|
||||
**Benefit:** you now know the destination — invisible operations with provable trust, not promised trust.
|
||||
|
||||
---
|
||||
|
||||
## Slide 4 — Strategic Objectives + Anti-Goals
|
||||
## Slide 3 — Strategic Objectives + Anti-Goals
|
||||
|
||||
**4 Strategic Objectives:**
|
||||
1. **Zero-touch operations** — autonomy as the default, not the demo
|
||||
2. **Provable trust in AI decisions** — Decision Ledger, confidence scoring, circuit breakers
|
||||
3. **Compounding, quantifiable ROI** — each quarter must reduce spend, free hours, avoid downtime
|
||||
4. **Default substrate for agentic consumption** — the platform AI agents reach for first
|
||||
1. **Zero-touch operations** — autonomy as the default, not the demo; stage-gate attestation (QA, SRE) remains human by design
|
||||
2. **Provable trust in automated decisions** — deterministic scripts calculate a score; the platform functions without AI; Decision Ledger, confidence scoring, circuit breakers, blast-radius controls
|
||||
3. **Compounding, quantifiable ROI** — four CTO-grade metrics, all flowing into PowerBI:
|
||||
- **Lead Time** (PR → Production) · **Infrastructure Vulnerability Count** (trend) · **MTTR** · **Cloud Spend Reduction**
|
||||
4. **Integrate with externally owned development platforms — regardless of source** — PDLC, SDLC, Agentic, or Citizen Developer; Nova provides skills + MCP endpoints; all prod intents go through the same controls and quality gates
|
||||
|
||||
**5 Anti-Goals (what Nova is NOT):**
|
||||
1. Not a hyperscaler competitor
|
||||
2. Not a general-purpose AI platform
|
||||
3. Not removing humans from accountability
|
||||
4. Not for legacy, untagged, or freeform infrastructure
|
||||
5. Not sold to operators
|
||||
**4 Anti-Goals (what Nova is NOT):**
|
||||
1. Not a general-purpose AI agent platform
|
||||
2. Not a system that removes humans from accountability — only from normal operations
|
||||
3. Not an upstream development platform (no product backlogs, IDE, code authorship)
|
||||
4. Not a replacement for the Product Development Lifecycle (PDLC)
|
||||
|
||||
**Benefit:** you now know the scope boundaries — Nova is purpose-built for infrastructure operations, sold to leadership on outcomes.
|
||||
**Benefit:** the scope is explicit — Nova governs infrastructure and delivery, integrates with any upstream source through one validated contract, and measures success on four metrics a CTO can repeat back.
|
||||
|
||||
---
|
||||
|
||||
## Slide 5 — 12–18 Month Targets
|
||||
## Slide 4 — Scope: Downstream of PDLC
|
||||
|
||||
**Current-milestone targets (grounded/derived):**
|
||||
**Nova governs infrastructure and delivery. The PDLC is upstream — Nova never penetrates it. Integration is through one validated contract.**
|
||||
|
||||
| Domain | Target | Status |
|
||||
|---|---|---|
|
||||
| MTTR (p95) | < 60s | grounded |
|
||||
| Cloud Spend Reduction | ≥ 25% | partial (CUR deferred D-096) |
|
||||
| L1/L2 Ops Hours Avoided | ≥ 70% | derived (N internal runs) |
|
||||
| Platform ROI | ≥ 250% | derived (formula; N=0 caveat) |
|
||||
| Decision Ledger Coverage | 100% | grounded |
|
||||
| Attestation Coverage | 100% | grounded |
|
||||
- **The PDLC is upstream:** product backlog, code authorship (AI agent, IDE, agentic SDLC), sprint planning, application business logic
|
||||
- **Nova is downstream:** contract ingestion → submission-readiness gate → policy enforcement → cloud resource lifecycle → environment progression (dev → qa → prod → dr) → immutable audit + attestation
|
||||
- **The integration point is one contract** — any upstream source (AI agent, agentic SDLC, dev platform) produces submissions subject to the same compliance standards
|
||||
- **Nova validates the submission, not the author** — the audit trail, the policy envelope, and the evidence stream are the same regardless of source
|
||||
|
||||
**Post-Pilot targets (pipeline grounded; 0 consumers today):**
|
||||
**Benefit:** a clean scope boundary — Nova is purpose-built for infrastructure operations and integrates with any upstream source through one validated contract, so the platform team's surface area stays bounded.
|
||||
|
||||
| Domain | Target | Status |
|
||||
|---|---|---|
|
||||
| Touchless Resolution Rate | ≥ 99% | partial |
|
||||
| Human Escalation Frequency | < 0.1% | partial |
|
||||
| AI Decision Accuracy | ≥ 99.5% | partial |
|
||||
---
|
||||
|
||||
**Deferred:** Predictive vs Reactive ≥3:1 <span class="badge planned">Planned</span> · Drift Auto-Reversal ≥95% <span class="badge planned">Planned</span>
|
||||
## Slide 5 — RACI: Who Owns What
|
||||
|
||||
**Benefit:** you now know the destination numbers — and which are measurable today vs deferred honestly.
|
||||
**Four roles, one matrix — citizen developer owns FRs + UAT, platform owns NFRs + infra, quality engineering owns the gate evidence, SRE owns operational readiness.**
|
||||
|
||||
| Work Category | Citizen Dev | Platform | Quality Eng | SRE |
|
||||
|---|---|---|---|---|
|
||||
| Functional Requirements | **R/A** | C | I | I |
|
||||
| User Acceptance Testing | **R/A** | C | I | I |
|
||||
| Non-Functional Requirements | I | **R/A** | C | C |
|
||||
| Infrastructure (cloud, state, IAM) | I | **R/A** | I | C |
|
||||
| QA (policy, confidence, schema) | C | R | **R/A** | I |
|
||||
| Production deployment to cloud | I | **R/A** | C | C |
|
||||
| Quality attestation (QA sign-off) | **A** | R | **R** | I |
|
||||
| Production readiness (SRE sign-off) | **A** | R | C | **R** |
|
||||
|
||||
**R**=Responsible · **A**=Accountable (sign-off) · **C**=Consulted · **I**=Informed. Production readiness is co-owned: the platform runs attestations agentically; the citizen developer authorizes the promotion at the stage gate.
|
||||
|
||||
**Benefit:** every party knows what they bring, what the platform provides, what quality engineering guards, and where SRE signs off — accountability is explicit, never diffuse.
|
||||
|
||||
---
|
||||
|
||||
## Slide 6 — The Platform Pipeline
|
||||
|
||||
**How intent becomes verified infrastructure without an operator.**
|
||||
**How intent becomes verified infrastructure — fail-fast policy scanning before the plan, runtime scanning after it.**
|
||||
|
||||
Contract → Resolver → Adapter → Terraform Plan → Checkov (Policy) → Confidence Signal → HITL Gate → Apply → Evidence
|
||||
|
||||
|
||||
- Dev: autonomous (no HITL gate)
|
||||
- qa/prod/dr: attested (human sign-off required)
|
||||
- Grounded in `run_platform.sh` + `contract_resolver.py` + `confidence_signal.py`
|
||||
- **Contract → resolver → adapter → Checkov on static code (before plan) → terraform plan → Wiz on the plan → confidence signal → stage gate → apply → evidence + ledger**
|
||||
- **Fail-fast, quick feedback** — Checkov runs on the authored Terraform code before `terraform plan` so developers get immediate policy feedback
|
||||
- **Wiz on the plan when configured; Checkov as a drop-in otherwise** — Wiz scans the plan output; when Wiz credentials are absent, Checkov runs against the plan. **Wiz and Checkov are never both run on the plan.**
|
||||
- **Dev is autonomous** (no stage gate); **qa/prod/dr require human attestation** (QA for quality, SRE for production readiness)
|
||||
|
||||
**Benefit:** you now know the path from intent to evidence — and where the human appears (stage gates only).
|
||||
**Benefit:** two layers of scanning, zero operator involvement in normal operations — fast deterministic feedback at authoring time and a runtime scan on the resolved plan.
|
||||
|
||||
---
|
||||
|
||||
## Slide 7 — The Decision Ledger
|
||||
|
||||
**Every AI decision captured with confidence, alternatives, and outcome.**
|
||||
**Every automated decision is captured, immutable, queryable — and accountable.**
|
||||
|
||||
- `outbox_writer.py` → SQLite append-only hash-chain table
|
||||
- `ai.decision.made`: decision_id=run_id, chosen_action=band, confidence=score, alternatives=perInput, human_override=HITL block
|
||||
- `attestation.recorded`: qa/prod/dr sign-offs
|
||||
- D-121, D-122, D-132. Honors D-083 (no S3 Object Lock/JWS — local hash-chain)
|
||||
- **What is captured:** the chosen action, the confidence score, the alternatives considered, whether a human overrode it, and the outcome (backfilled once the apply completes). Every stage-gate attestation (QA, SRE) is captured with approver identity and the evidence presented.
|
||||
- **"AI decisions" are really automated decisions** — made by deterministic scripts that calculate a score and a band; the platform functions without AI. When an LLM planner is added later, it will emit richer alternatives without breaking the schema.
|
||||
- **The value is accountability, not the storage engine** — the ledger is append-only and tamper-evident; every decision is queryable for auditing, traceable to an outcome, and impossible to rewrite after the fact.
|
||||
|
||||
**D-122 honesty:** Nova's "AI" is the confidence-gated policy engine (confidence_signal + HITL gate), not an LLM planner. The Decision Ledger captures this real decision path — not a fabricated "AI agent."
|
||||
|
||||
**Benefit:** you now know why 'autonomous' is defensible — every decision is immutable, queryable, and accountable. And you know exactly what 'AI' means here: a confidence-gated policy engine, not a black-box LLM.
|
||||
**Benefit:** "autonomous" is defensible because every decision is immutable, queryable, and accountable — and the audience knows exactly what "automated" means here: deterministic scoring, not a black-box LLM.
|
||||
|
||||
---
|
||||
|
||||
## Slide 8 — The 8-Concern Attestation Matrix
|
||||
## Slide 8 — The Attestation Matrix
|
||||
|
||||
**Designed controls that keep humans at stage gates.**
|
||||
**The designed controls that keep humans at stage gates — structured, freshness-validated, separation-of-duties-enforced.**
|
||||
|
||||
| Concern | Env | Freshness | Type |
|
||||
|---------|-----|-----------|------|
|
||||
| functional_correctness | qa | 24h | operator-supplied |
|
||||
| performance_baseline | qa | 7d | operator-supplied |
|
||||
| security_posture | qa | 24h | operator-supplied |
|
||||
| operational_readiness | prod | 30d | operator-supplied |
|
||||
| incident_response | prod | 90d | operator-supplied |
|
||||
| capacity_cost | prod | 30d | operator-supplied |
|
||||
| resilience_dr_drill | prod | 180d | operator-supplied |
|
||||
| dr_region_deploy | dr | 180d | operator-supplied |
|
||||
| Concern | Env | Freshness | Description |
|
||||
|---------|-----|-----------|-------------|
|
||||
| Functional correctness | qa | 24h | The application behaves as specified; evidence accepted from the consumer's UAT. |
|
||||
| Performance baseline | qa | 7d | The deployment meets its performance envelope vs. the agreed baseline. |
|
||||
| Security posture | qa | 24h | The deployment's security findings have been reviewed and accepted. |
|
||||
| Operational readiness | prod | 30d | SRE confirms the deployment is operable: runbooks, dashboards, on-call. |
|
||||
| Incident response | prod | 90d | The on-call path has been exercised; a working incident-response plan exists. |
|
||||
| Capacity & cost | prod | 30d | Capacity headroom and monthly cost are within the agreed envelope. |
|
||||
| Resilience: DR drill | prod | 180d | A DR drill has been run and recovery met the RTO. |
|
||||
| Resilience: chaos | prod | 90d | A chaos exercise has been run and the deployment absorbed the failure. |
|
||||
| Resilience: backup | prod | 30d | Backups are restorable and tested within the freshness window. |
|
||||
| DR region deploy | dr | 180d | The DR region can be deployed and is reachable. |
|
||||
|
||||
- Offline-testable concerns run for real; operator-supplied concerns accept signed evidence
|
||||
- Separation-of-duties on prod
|
||||
- Grounded in `attestation_matrix.py` + `hitl_gates.py`
|
||||
Separation-of-duties on prod: the approver cannot be the same person who built the deployment.
|
||||
|
||||
**Benefit:** you now know the gate model — autonomy in operations, human in accountability, by design.
|
||||
**Benefit:** the gate model is explicit — autonomy in operations, human in accountability, by design. The matrix is what makes autonomous operations safe enough to trust in production.
|
||||
|
||||
---
|
||||
|
||||
## Slide 9 — Telemetry Architecture
|
||||
## Slide 9 — Telemetry & Live Ops
|
||||
|
||||
**How Nova instruments itself — CloudEvents envelope, cold store, PowerBI export.**
|
||||
**Every metric in this deck is traceable to a real emitted signal — the live-ops dashboard makes operations visible in PowerBI.**
|
||||
|
||||
Platform → CloudEvents 1.0 → `metrics/events.jsonl` + `metrics/decision_ledger.db` + `metrics/runs/` → Collector → `metrics/nova_metrics.db` (SQLite cold store) → `metrics/powerbi/` (CSV/JSON) → PowerBI
|
||||
|
||||
|
||||
- D-120 (Nova-native), D-125 (hybrid), D-126 (cold-only)
|
||||
- <span class="badge planned">Planned</span>: Hot-path (live ops dashboard) — D-126
|
||||
- **Platform components → CloudEvents envelope → event log + decision ledger + run records → collector → cold store → PowerBI views → live ops dashboard**
|
||||
- **The live ops dashboard (PowerBI)** surfaces the four CTO-grade metrics (Lead Time, Vulnerability Count, MTTR, Cloud Spend) alongside trust metrics (Decision Ledger coverage, Attestation coverage) and efficiency metrics (touchless resolution, escalation frequency)
|
||||
- **Deliberately minimal** — Nova-native envelopes; no Kafka, no Prometheus, no ClickHouse. The cold store handles batch and historical analysis; the live-ops surface is built in PowerBI on the exported views
|
||||
- **Every number is traceable to a signal** — when a CFO asks "where does this number come from?", the answer is a query against the cold store, not a Slack thread
|
||||
|
||||
**Benefit:** you now know that every metric in this deck is traceable to a real emitted event — the architecture IS the trust substrate. When a CFO asks 'where does this number come from?', the answer is a file path, not a Slack thread.
|
||||
**Benefit:** the architecture is the trust substrate — leadership sees the same numbers the platform produces, in PowerBI, with full traceability. Operations become visible.
|
||||
|
||||
---
|
||||
|
||||
## Slide 10 — Capability Health + Confidence Distribution
|
||||
## Slide 10 — Decision Ledger + Attestation Coverage
|
||||
|
||||
**Grounded proof: capability health and confidence distribution from real runs.**
|
||||
**By design, no change reaches production without a ledger entry and a human attestation — both queryable for auditing, with full traceability.**
|
||||
|
||||
| Status | Count |
|
||||
|--------|-------|
|
||||
| Verified | 18 |
|
||||
| Skipped | 4 |
|
||||
| Broken | 0 |
|
||||
| Decayed | 0 |
|
||||
- **Decision Ledger coverage: 100%** — every platform run emits a decision record with outcome backfill; no automated decision is ever lost
|
||||
- **Attestation coverage: 100%** — every prod/dr promotion is attested by a human (QA for quality, SRE for production readiness), recorded with approver identity, separation-of-duties check, and the evidence matrix
|
||||
- **No change to production without both** — the ledger entry and the human attestation are mandatory, enforced by the pipeline, not by policy
|
||||
- **Easily queried for auditing** — queryable by run, by environment, by approver, and by outcome; the audit trail is a query, not a forensic exercise
|
||||
- **Full traceability** — a production change is traceable from the contract that declared intent, through the policy scan, the confidence score, the attestation, to the applied outcome
|
||||
|
||||
- 4 Skipped = live-AWS caps (CAP-013..016), honestly skipped (D-096 teardown), not a failure
|
||||
- Source: `.ciagent/REGRESSION_REPORT.json`
|
||||
|
||||
**Benefit:** you now know the platform is verified — 18 capabilities pass, 4 are honestly skipped, 0 broken.
|
||||
**Benefit:** trust is provable — not a marketing claim, a queryable record. An auditor answers "who approved this, when, on what evidence?" in one query; a CTO answers "how many of last quarter's prod changes were touchless?" in one query.
|
||||
|
||||
---
|
||||
|
||||
## Slide 11 — Decision Ledger + Attestation Coverage
|
||||
## Slide 11 — Cost & ROI
|
||||
|
||||
**Trust metrics — both 100%.**
|
||||
**The ROI formula and the cost estimates — grounded, with the production denominator honestly flagged.**
|
||||
|
||||
- **Decision Ledger Coverage:** 100% of platform runs emit `ai.decision.made` with outcome backfill
|
||||
- **Attestation Coverage:** 100% of prod/dr promotions attested by a human
|
||||
- **AI Decision Accuracy:** decisions not followed by apply.failed/incident within 5min
|
||||
- Trust snapshot: `metrics/TRUST_SNAPSHOT.md` with chain-integrity verdict
|
||||
- <span class="badge planned">Planned</span>: Tamper-Evident Ledger Checkpoints (D-083)
|
||||
- **Cost estimates are pre-apply and offline** — the platform reads the terraform plan and estimates cost before anything is applied; a cost regression is caught before the spend happens
|
||||
- **The ROI formula:**
|
||||
`Platform ROI = (FTE hours saved × blended rate + cloud savings + avoided downtime) ÷ platform op cost`
|
||||
- **The four CTO-grade metrics are the ROI proof:** Lead Time (PR → Prod), Infrastructure Vulnerability Count (trend), MTTR, Cloud Spend Reduction — all flow into PowerBI
|
||||
- **Honest caveat:** derived metrics are computed on internal runs today; the production-denominator activates when a pilot estate runs. The formula is grounded; the production numbers are not yet.
|
||||
|
||||
**Benefit:** you now know the trust is provable — not a marketing claim, a queryable record.
|
||||
**Benefit:** the ROI is not a black box — the formula is shown, the four metrics are committed, and the production-denominator caveat is stated up front. The CFO sees exactly what is real today and what activates with a pilot.
|
||||
|
||||
---
|
||||
|
||||
## Slide 12 — Zero-Touch Efficiency
|
||||
## Slide 12 — What's Deferred — and Why
|
||||
|
||||
**Touchless resolution, human escalation, and MTTR.**
|
||||
**Honesty about what is not measured yet — and the blocking work for each.**
|
||||
|
||||
- **Touchless Resolution Rate:** runs without operational HITL block ÷ total (attestation gates excluded)
|
||||
- **Human Escalation Frequency:** operational HITL blocks only (confidence-driven; attestation sign-offs excluded)
|
||||
- **MTTR (platform-run):** apply.failed → successful retry (D-131)
|
||||
To be clear: these deferrals are *measurement infrastructure*, not the autonomy itself. The platform runs without an operator in the loop of normal operations. What is deferred is the evidence pipeline for certain metrics — not the autonomy.
|
||||
|
||||
**Post-Pilot caveat:** computed on N internal runs today; production-denominator activates when a pilot estate runs.
|
||||
| # | Deferred metric | Blocking work |
|
||||
|---|-----------------|---------------|
|
||||
| 1 | Live infrastructure health | Live AWS re-provisioning (currently torn down to zero-cost steady state) |
|
||||
| 2 | Live outbox write rate | Live AWS re-provisioning |
|
||||
| 3 | Tamper-evident ledger checkpoints | Audit-ledger build-out (Object Lock + signed checkpoints) |
|
||||
| 4 | Onboarding funnel (requested → granted) | Auto-grant implementation |
|
||||
| 5 | Drift auto-reversal | Drift-detection scheduler (not yet built) |
|
||||
| 6 | Live cost reconciliation | Live AWS re-provisioning + actual-spend feed |
|
||||
| 7 | SLA / unplanned downtime | Live AWS re-provisioning |
|
||||
| 8 | Predictive vs reactive ratio | ML anomaly-forecasting service (not yet built) |
|
||||
|
||||
**Benefit:** you now know the zero-touch efficiency is measurable — the pipeline works today on internal runs, and the denominator expands to production estates when a pilot activates.
|
||||
**Benefit:** the boundaries are explicit — what Nova measures today, and exactly what blocks the rest. The autonomy is real; the measurement gaps are documented with the work that unblocks each one.
|
||||
|
||||
---
|
||||
|
||||
## Slide 13 — Cost & ROI
|
||||
## Slide 13 — Roadmap to the North Star
|
||||
|
||||
**Cost estimates and the ROI formula — with honest caveats.**
|
||||
**The path from the grounded metrics to the 12–18 month targets — each deferred metric has an unblock path and a timeframe.**
|
||||
|
||||
- **Cost Estimates via Infracost:** pre-apply, grounded (reads plan JSON, offline)
|
||||
- **ROI formula:** `Platform ROI = (FTE hours saved × blended rate + cloud savings + avoided downtime) ÷ platform op cost`
|
||||
- **N=0 caveat:** "Computed on N internal runs today; production-denominator activates post-pilot. The formula is grounded; the production numbers are not yet."
|
||||
- <span class="badge planned">Planned</span>: Live CUR Reconciliation (D-096)
|
||||
| Timeframe | Work | Unblocks |
|
||||
|-----------|------|----------|
|
||||
| Near-term | Live AWS re-provisioning | Live infra health, outbox write rate, live cost reconciliation, SLA |
|
||||
| Near-term | Auto-grant implementation | Onboarding funnel (requested → granted) |
|
||||
| Mid-term | Drift-detection scheduler | Drift auto-reversal |
|
||||
| Mid-term | Audit-ledger build-out (Object Lock + signed checkpoints) | Tamper-evident ledger checkpoints |
|
||||
| Mid-term | Hot-path activation (batch → near-real-time) | Live-ops dashboard freshness |
|
||||
| Longer-term | ML anomaly-forecasting service | Predictive vs reactive ratio |
|
||||
|
||||
**Benefit:** you now know the ROI formula — and you know it's computed on internal runs today, not fabricated production numbers.
|
||||
Re-evaluation triggers: each blocking piece of work lifts on its own schedule; the metrics layer evolves as each one lands.
|
||||
|
||||
**Benefit:** every deferred metric has an unblock path — nothing is hand-waved; everything has a plan and a timeframe.
|
||||
|
||||
---
|
||||
|
||||
## Slide 14 — What's Deferred — and Why
|
||||
## Slide 14 — 12-Month Product Roadmap
|
||||
|
||||
**Honesty about what isn't measured yet.**
|
||||
**The product arc from pilot activation to integration — four quarters, four outcomes.**
|
||||
|
||||
**To be clear:** these deferrals are *measurement infrastructure*, not whether the platform runs without humans. The platform IS autonomous in operations. What's deferred is the *evidence pipeline* for certain metrics — not the autonomy itself.
|
||||
| Quarter | Theme | Board-level outcome |
|
||||
|---------|-------|---------------------|
|
||||
| **Q1** | Pilot Activation | Nova runs a real customer estate end-to-end, autonomously, with a measurable zero-touch rate. |
|
||||
| **Q2** | Provable Trust | Every automated decision lands in a tamper-evident ledger; the CFO sees real cloud-spend reconciliation. |
|
||||
| **Q3** | Compounding ROI | Quarter-over-quarter cloud spend drops; drift is detected and reversed without a human. |
|
||||
| **Q4** | Integration & Predictive | AI agents deploy through Nova by default; the ML anomaly-forecasting service goes live. |
|
||||
|
||||
| # | Deferred Metric | Blocking Decision |
|
||||
|---|----------------|-------------------|
|
||||
| 1 | Live Infrastructure Health | D-096 |
|
||||
| 2 | Live Outbox Write Rate | D-096 |
|
||||
| 3 | Tamper-Evident Ledger Checkpoints | D-083 |
|
||||
| 4 | Onboarding Funnel (granted) | D-113/D-114/D-119 |
|
||||
| 5 | Drift Auto-Reversal | D-096 + no scheduler |
|
||||
| 6 | Live CUR Reconciliation | D-096 |
|
||||
| 7 | SLA / Unplanned Downtime | D-096 |
|
||||
| 8 | Predictive vs Reactive | future emitter |
|
||||
Grounded in the four strategic objectives (autonomy, provable trust, ROI, integration) and the deferred-metric unblock paths.
|
||||
|
||||
**Benefit:** you now know the boundaries — what Nova measures today, and exactly what blocks the rest. The autonomy is real; the measurement gaps are documented.
|
||||
**Benefit:** the 12-month product arc — each quarter activates a strategic objective and its corresponding board-level metric, from pilot activation through integration leadership.
|
||||
|
||||
---
|
||||
|
||||
## Slide 15 — Roadmap to the North Star
|
||||
## Slide 15 — Quarter-by-Quarter Outcomes
|
||||
|
||||
**The path from v1.17's grounded metrics to the 12–18 month targets.**
|
||||
| Quarter | Product theme | Key deliverable | Target metric | Grounding |
|
||||
|---------|---------------|-----------------|---------------|-----------|
|
||||
| **Q1** | Pilot Activation | Re-provision live AWS; activate first pilot estate; onboarding auto-grant | Touchless ≥ 99% · Escalation < 0.1% · Accuracy ≥ 99.5% | Objective #1 — autonomy as the default |
|
||||
| **Q2** | Provable Trust | Tamper-evident ledger (Object Lock + signed checkpoints); daily checkpoints; live cost reconciliation | Decision Ledger Coverage 100% · Cost Savings ≥ 25% | Objective #2 — trust is the moat |
|
||||
| **Q3** | Compounding ROI + Drift | Drift-detection scheduler; auto-reversal; pre-apply → actual-spend reconciliation on the pilot estate | Drift Auto-Reversal ≥ 95% · Spend Reduction ≥ 25% | Objective #3 — CFO-pointable numbers |
|
||||
| **Q4** | Integration + Predictive | ML anomaly-forecasting; AI-agent intent surface; multi-cloud (Azure/GCP) preview | Predictive:Reactive ≥ 3:1 · AI-Agent Intent Share (first measurement) | Objective #4 — default substrate for agents |
|
||||
|
||||
- Each deferred metric → blocking decision → unblock requirement → candidate milestone
|
||||
- Hot-path activation (post-D-096, Nova-native only, D-120)
|
||||
- Re-evaluation triggers: D-096 lift, D-083 lift, onboarding-grant lift
|
||||
**Month-18 destination:** *"Nova is the layer enterprise leadership points to when they say 'we don't have an infrastructure ops team anymore, and the audit trail is stronger than it ever was.'"*
|
||||
|
||||
From `docs/METRICS_DEFERRED_ROADMAP.md`.
|
||||
|
||||
**Benefit:** you now know the path — every deferred metric has an unblock requirement and a candidate milestone. Nothing is hand-waved; everything has a plan.
|
||||
**Benefit:** each quarter has a concrete deliverable, a target metric grounded in a strategic objective, and a path from "honestly deferred" to "shipped and measured."
|
||||
|
||||
---
|
||||
|
||||
## Slide 16 — Recap + Ask
|
||||
## Slide 16 — Production-Grade Guidance via Atelier (1/2)
|
||||
|
||||
**The 5-act recap + the business decision.**
|
||||
**Nova instructs the citizen developer's AI agent on production-grade engineering — a set of skills and an MCP server.**
|
||||
|
||||
- **Skills** — markdown files keyed to production-grade engineering domains (API, security, data, testing, observability, errors, DevOps, infrastructure-as-code, compliance); the skills extend the baseline catalog with Nova-specific production-grade principles
|
||||
- **MCP server** — a plugin-registry, stdio server exposing four tools: `lookup_principle`, `list_domains`, `matrix_lookup`, `validate_against_principles`. The developer's AI agent (or any agentic SDLC platform) calls these tools to look up the principles that apply to its submission
|
||||
- **The integration point is the same regardless of source** — whether the submission comes from an AI coding agent, an agentic SDLC platform, or a traditional IDE, the same skills and MCP server apply. This is how Nova makes the citizen developer production-grade without owning the PDLC
|
||||
|
||||
**Benefit:** the citizen developer's AI agent is not unguided — Nova provides production-grade engineering principles as skills and as an MCP surface, so submissions arrive at the contract boundary already aligned with the platform's standards.
|
||||
|
||||
---
|
||||
|
||||
## Slide 17 — Production-Grade Guidance via Atelier (2/2)
|
||||
|
||||
**Agentic validation catches engineering-discipline gaps that deterministic scanners miss — and the validation is reproducible.**
|
||||
|
||||
- **Beyond deterministic scanners** — Wiz, Checkmarx, and Mend check policy and secrets; they do not check engineering discipline. The Atelier MCP server catches correctness, clarity, and observability gaps that deterministic tools cannot: "is this service observable?", "is this error path handled?", "is this API contract clear?"
|
||||
- **Agentic validation, not a second policy engine** — the MCP server gives the AI agent the principles to validate against; the agent does the validation. The agent reasons about the submission against the principles, not a second static scan
|
||||
- **Vendored for audit reproducibility** — Atelier is vendored at a pinned tag. A validation result is replayable against the exact principles that produced it, so an audit can reproduce a validation months later, not just trust a log line
|
||||
|
||||
**Benefit:** the citizen developer's submission is checked for engineering discipline, not just policy compliance — and the check is reproducible for audit. That is what makes the submission production-grade, regardless of which upstream platform produced it.
|
||||
|
||||
---
|
||||
|
||||
## Slide 18 — Recap + Ask
|
||||
|
||||
**The 4-beat recap + the business decision.**
|
||||
|
||||
**Recap:**
|
||||
- **Problem:** operator is the bottleneck; autonomy in operations, human at stage gates
|
||||
- **Vision:** invisible operations with provable trust (NORTH_STAR)
|
||||
- **How:** pipeline + Decision Ledger + 8-concern attestation matrix
|
||||
- **Proof:** 18V+4S, 100% ledger coverage, 100% attestation, grounded ROI formula
|
||||
- **Roadmap:** deferred metrics have unblock paths
|
||||
- **Problem:** product teams own infrastructure without the discipline and lifecycle planning it requires; bandwidth gaps and tribal knowledge leave operations exposed
|
||||
- **Solution:** autonomous cloud delivery — operations become visible, trust is provable (deterministic scoring), humans at stage gates
|
||||
- **Proof:** 100% ledger coverage, 100% attestation coverage, grounded ROI formula, four CTO-grade metrics flowing into PowerBI
|
||||
- **Roadmap:** deferred metrics have unblock paths; the 12-month product arc activates one strategic objective per quarter
|
||||
|
||||
**The ask:** "Approve a pilot estate to activate the production-denominator metrics (Touchless Resolution, Human Escalation, AI Decision Accuracy), and approve the tamper-evident ledger build-out (D-083 lift) to move from local hash-chain to S3 Object Lock + JWS. These two decisions move Nova from 'pipeline-ready' to 'production-proven.'"
|
||||
**The ask:** "Approve a pilot estate to activate the production-denominator metrics (Lead Time, Vulnerability Count, MTTR, Cloud Spend), and approve the tamper-evident ledger build-out to move from the local hash-chain to S3 Object Lock + signed checkpoints. These two decisions move Nova from 'pipeline-ready' to 'production-proven.'"
|
||||
|
||||
**Benefit:** you leave with a clear business decision to make — approve a pilot + the ledger build-out — and the confidence that every claim in this deck is grounded, derived, or honestly deferred.
|
||||
|
||||
---
|
||||
|
||||
## Slide 17 — Scope: Downstream of PDLC
|
||||
|
||||
**Nova governs infrastructure + delivery. The PDLC (product backlog, code authorship, IDE) is upstream — Nova never penetrates it.**
|
||||
|
||||
- **The PDLC is upstream:** product backlog, code authorship (AI agent / IDE / agentic SDLC), sprint planning, application business logic
|
||||
- **Nova is downstream:** contract ingestion → submission-readiness gate → policy → cloud lifecycle → environment progression → audit + attestation
|
||||
- **Integration is only through the contract boundary:** the citizen developer's AI coding agent, an upstream agentic SDLC, or any dev platform may all produce submissions — the source does not matter as all are subject to the same compliance standards
|
||||
- Nova validates the submission, not the author
|
||||
- Cites `docs/scope.md` + `PROJECT.md` § Scope
|
||||
|
||||
**Benefit:** you now know the scope boundary — Nova is purpose-built for infrastructure operations, not product development; integration is through one validated contract.
|
||||
|
||||
---
|
||||
|
||||
## Slide 18 — RACI: Who Owns What
|
||||
|
||||
**Three roles, one matrix — the citizen developer owns FRs + UAT, the platform owns NFRs + infra + QA + prod deploy, release management is co-owned.**
|
||||
|
||||
| Work Category | Citizen Dev | Platform | Release Mgmt |
|
||||
|---|---|---|---|
|
||||
| Functional Requirements (FRs) | **R/A** | C | I |
|
||||
| User Acceptance Testing (UAT) | **R/A** | C | I |
|
||||
| Non-Functional Requirements (NFRs) | I | **R/A** | C |
|
||||
| Infrastructure (cloud, state, IAM) | I | **R/A** | C |
|
||||
| QA (policy, confidence, schema) | C | **R/A** | I |
|
||||
| Production deployment to cloud | I | **R/A** | C |
|
||||
| Release attestation (QA + SRE) | **A** | R | **R** |
|
||||
|
||||
- **Compliance-standard equivalence:** FRs + UAT may come from any upstream source (AI agent, agentic SDLC, dev platform) — all pass the same submission-readiness gate
|
||||
- **Release co-ownership:** the platform runs the attestations agentically; the citizen developer oversees and triggers the actual release (human at the stage gate)
|
||||
- Cites `docs/raci.md` + `PROJECT.md` § RACI Matrix
|
||||
|
||||
**Benefit:** you now know exactly what you bring (FRs + UAT), what Nova provides (NFRs + infra + QA + prod deploy), and what you co-own (the release attestation).
|
||||
|
||||
---
|
||||
|
||||
## Slide 19 — Production-Grade Guidance via Atelier
|
||||
|
||||
**Nova instructs the citizen developer's AI agent on production-grade engineering — skills + an MCP server with agentic validation beyond deterministic scanners.**
|
||||
|
||||
- **Skills (9):** markdown files under `skills/` keyed to Atelier domain paths (api, security, data, testing, observability, errors, devops, infrastructure-as-code, compliance) — extending the BA.A 5-skill catalog
|
||||
- **MCP server:** `mcp/atelier/server.py` (plugin-registry, stdio) — 4 tools: `lookup_principle`, `list_domains`, `matrix_lookup`, `validate_against_principles`
|
||||
- **Agentic validation:** catches C1 correctness + C2 clarity + C7 observability gaps that Wiz/Checkmarx/Mend cannot — deterministic tools check policy/secrets; the MCP server checks engineering discipline
|
||||
- **Vendored Atelier** (pinned tag v0.3.6): audit reproducibility — a validation result is replayable against the exact principles that produced it
|
||||
- Cites `docs/skills.md` + `mcp/atelier/README.md`
|
||||
|
||||
**Benefit:** you now know the citizen developer is not unguided — Nova provides production-grade engineering principles via skills + an MCP server, so the AI agent's submissions meet the same standards regardless of upstream source.
|
||||
**Benefit:** a clear business decision — approve a pilot and the ledger build-out — with the confidence that every claim in this deck is grounded, derived, or honestly deferred.
|
||||
|
||||
---
|
||||
|
||||
@@ -338,66 +310,18 @@ From `docs/METRICS_DEFERRED_ROADMAP.md`.
|
||||
|
||||
| KPI | Definition | Status |
|
||||
|-----|-----------|--------|
|
||||
| Touchless Resolution Rate | runs without operational HITL block ÷ total | partial (Post-Pilot) |
|
||||
| Human Escalation Frequency | operational HITL blocks ÷ total | partial (Post-Pilot) |
|
||||
| AI Decision Accuracy | decisions not followed by failure within 5min | partial (Post-Pilot) |
|
||||
| Touchless Resolution Rate | runs without operational stage-gate block ÷ total | partial (Post-Pilot) |
|
||||
| Human Escalation Frequency | operational stage-gate blocks ÷ total | partial (Post-Pilot) |
|
||||
| Automated Decision Accuracy | decisions not followed by failure within 5min | partial (Post-Pilot) |
|
||||
| MTTR (p95) | apply.failed → successful retry | grounded |
|
||||
| Confidence-Gate Halt Rate | runs with band=block ÷ total | grounded |
|
||||
| Provisioning Lead Time | run.completed − run.started | grounded |
|
||||
| Deployment Frequency | count(run.completed) per day | grounded |
|
||||
| Cost Savings (Infracost) | sum(delta_usd where delta < 0) | partial (CUR deferred) |
|
||||
| Cost Savings (pre-apply) | sum(delta_usd where delta < 0) | partial (live reconciliation deferred) |
|
||||
| FTE Hours Saved | run count × manual baseline × rate | derived (N=0 caveat) |
|
||||
| Platform ROI | (labor + cloud + avoided downtime) ÷ op cost | derived (N=0 caveat) |
|
||||
| Decision Ledger Coverage | decisions with outcome ÷ total | grounded |
|
||||
| Attestation Coverage | prod/dr attested ÷ total prod/dr | grounded |
|
||||
| Policy Compliance Rate | 1 − failed_assets ÷ total | grounded |
|
||||
|
||||
---
|
||||
|
||||
<!-- _class: title -->
|
||||
<!-- _paginate: false -->
|
||||
|
||||
## Appendix A2 — Operating Model & Cost
|
||||
|
||||
- **Cost figures** from `COST.md`: $0.001883 over 8 days, ~$0.007/month, S3-dominated, zero BAU compute
|
||||
- **Zero-cost steady state:** all resources torn down post-v1.11 (D-096); the platform runs offline
|
||||
- References the pre-mortem (`PRE_MORTEM.md`: v1.10 decay root cause + structural mitigations)
|
||||
|
||||
**Benefit:** you now know the operating cost is negligible — and the structural mitigation that prevents decay.
|
||||
---
|
||||
|
||||
<!-- _class: title -->
|
||||
<!-- _paginate: false -->
|
||||
|
||||
## Slide 20 — 12-Month Product Roadmap
|
||||
|
||||
**The product arc from pilot activation to agentic substrate — four quarters, four outcomes.**
|
||||
|
||||
| Quarter | Theme | Board-level outcome |
|
||||
|---------|-------|---------------------|
|
||||
| **Q1** | <span class="badge planned">Pilot Activation</span> | Nova runs a real customer estate end-to-end, autonomously, with a measurable zero-touch rate |
|
||||
| **Q2** | <span class="badge planned">Provable Trust</span> | Every AI decision lands in a tamper-evident ledger; CFO sees real cloud-spend reconciliation |
|
||||
| **Q3** | <span class="badge planned">Compounding ROI</span> | Quarter-over-quarter cloud spend drops; drift is detected and reversed without a human |
|
||||
| **Q4** | <span class="badge planned">Agentic Substrate</span> | AI agents deploy through Nova by default; Nova is the substrate, not a vendor arriving late |
|
||||
|
||||
**Grounded in:** the 4 strategic objectives (autonomy, provable trust, ROI, agentic substrate) + the deferred-metric unblock paths.
|
||||
|
||||
**Benefit:** you now know the 12-month product arc — each quarter activates a strategic objective and its corresponding board-level metric, from pilot activation through agentic substrate leadership.
|
||||
|
||||
---
|
||||
|
||||
<!-- _class: title -->
|
||||
<!-- _paginate: false -->
|
||||
|
||||
## Slide 21 — Quarter-by-Quarter Outcomes
|
||||
|
||||
| Quarter | Product theme | Key deliverable | Target metric | Grounding |
|
||||
|---------|--------------|-----------------|---------------|-----------|
|
||||
| **Q1** | Pilot Activation | Re-provision live AWS; activate first pilot estate; onboarding auto-grant | Touchless Resolution ≥ 99% · Escalation < 0.1% · AI Accuracy ≥ 99.5% | Strategic Objective #1 — autonomy as the default |
|
||||
| **Q2** | Provable Trust | Tamper-evident ledger (Object Lock + JWS); daily checkpoints; live cost reconciliation (CUR) | Decision Ledger Coverage 100% · Cost Savings ≥ 25% | Strategic Objective #2 — trust is the moat |
|
||||
| **Q3** | Compounding ROI + Drift | Drift detection scheduler; auto-reversal; Infracost→CUR reconciliation on pilot estate | Drift Auto-Reversal ≥ 95% · Spend Reduction ≥ 25% | Strategic Objective #3 — CFO-pointable numbers |
|
||||
| **Q4** | Agentic Substrate + Predictive | ML anomaly-forecasting; AI-agent intent surface; multi-cloud (Azure/GCP) preview | Predictive:Reactive ≥ 3:1 · AI-Agent Intent Share ≥ 40% (first measurement) | Strategic Objective #4 — default substrate for agents |
|
||||
|
||||
**Month-18 destination:** *"Nova is the layer enterprise leadership points to when they say 'we don't have an infrastructure ops team anymore, and the audit trail is stronger than it ever was.'"*
|
||||
|
||||
**Benefit:** you now know the quarter-by-quarter detail — each quarter has a concrete deliverable, a target metric grounded in a strategic objective, and a path from "honestly deferred" to "shipped and measured."
|
||||
**Benefit:** a reference for every metric mentioned in the deck.
|
||||
Reference in New Issue
Block a user