Jon Chery
b054849a99
P4 (Wave 3, docs) — REQ-186, 191, 192, 193, 194, 195, 204, 210, 211, 212, 213 New docs: - docs/METRICS.md — canonical KPI catalog (grounded/derived/deferred) - docs/metrics/*.md — 13 per-KPI definition-of-success docs (D-127) - docs/METRICS_DEFERRED_ROADMAP.md — 8 deferred metrics + hot-path plan + re-eval triggers (REQ-210) - docs/NO_HUMANS_THESIS.md — thesis defensibility brief (REQ-213) New tools: - core/metrics/trust_snapshot.py — 5 trust metrics + chain-integrity verdict + snapshot hash (REQ-211) - scripts/check_north_star_diff.sh — CI check for NORTH_STAR strategic section changes (REQ-204) Modified: - .ciagent/config.json — strategic_direction_file: .ciagent/NORTH_STAR.md (REQ-186) ---ci--- project: acdl phase: 4 milestone: v1.17 status: execute ---/ci---
4.1 KiB
Nova Deferred Metrics Activation Roadmap
v1.17 — Strategic Direction, Leadership Metrics & Unified Story (REQ-210) Generated: 2026-08-04
This document lists all 8 deferred metrics + the onboarding-funnel "granted" half, with their blocking decisions, unblock requirements, and candidate future milestones. It also includes the hot-path activation plan (post-D-096) and the re-evaluation triggers.
Deferred metrics
| # | Metric | Blocking Decision | What's Needed to Unblock | Candidate Milestone |
|---|---|---|---|---|
| 1 | Live Infrastructure Health (ECS, ALB, RPS) | D-096 | Re-provision live AWS; deploy microservice/static-assets stacks; emit live health metrics | v1.18+ (live AWS re-provisioning) |
| 2 | Live Outbox Write Rate / Ledger Append Latency | D-096 | Re-provision DynamoDB outbox table; emit write-latency metrics | v1.18+ |
| 3 | Tamper-Evident Ledger Checkpoints / JWS Signature Rate | D-083 | Build S3 Object Lock + JWS signing + async worker + DLQ + daily checkpoints | v1.19+ (audit ledger build-out) |
| 4 | Onboarding Funnel (requested → granted) | D-113/D-114/D-119 | Implement auto-grant: Lambda provisions the cross-account role + ABAC tag + environment binding | v1.18+ (onboarding auto-grant) |
| 5 | Drift Auto-Reversal Rate | D-096 + no scheduler | Build a drift-detection scheduler (cron); run terraform plan -detailed-exitcode per workspace; emit drift.detected events |
v1.20+ (drift detection) |
| 6 | Live CUR Reconciliation | D-096 | Re-provision live AWS billing access; build CUR reconciler (6h schedule); match bill lines to resource addresses via tags | v1.18+ |
| 7 | SLA / Unplanned Downtime | D-096 | Deploy live services with SLOs; emit uptime metrics against SLO targets | v1.18+ |
| 8 | Predictive vs Reactive Ratio | future emitter | Build an ML anomaly-forecasting service; emit anomaly.predicted events with proactive label | v1.21+ (predictive ops) |
Onboarding-funnel "granted" half
The onboarding request path is grounded (REQ-182/183 from v1.16): a
consumer submits a request → the Lambda writes a pending CMDB row →
core/onboarding.py generates a binding file. The "granted" half
(actual AWS account/network/state provisioning) is deferred per
D-113/D-114/D-119. When a future milestone implements auto-grant, the
onboarding funnel metric activates: count(granted) ÷ count(requested).
Hot-Path Activation (post-D-096)
Current state (v1.17): SQLite cold store only (D-126). No hot path. The hot path activates when live AWS is re-provisioned (D-096 lift).
Nova-native hot-path candidates (D-120 — no Kafka/Prometheus/ClickHouse):
- SQLite read-replica: the cold store becomes a read-replica updated on each run; a lightweight file-watcher notifies the dashboard of changes. Freshness = "last run" (not 1-second, but sufficient for batch ops).
- JSONL tail + webhook: the events.jsonl log is tailed by a small daemon that pushes updates to a webhook (e.g., a PowerBI streaming dataset or a custom dashboard). Nova-native (no new infra).
- SQLite + Grafana SQLite datasource: Grafana can read SQLite directly via the SQLite datasource plugin. No TSDB needed.
Migration steps (when D-096 lifts):
- Re-provision live AWS (microservice + static-assets stacks).
- Add live-health emitters (ECS running count, ALB 5xx, RPS) to
run_platform.sh. - Choose a hot-path candidate (above) and implement it.
- Populate the 8 placeholder views with real data.
- Re-run the collector + PowerBI export.
Re-evaluation Triggers
A follow-up metrics ideation should be triggered when any of these events occurs:
- D-096 lift (live AWS re-provisioned) — triggers hot-path activation + placeholder view population for metrics 1, 2, 5, 6, 7.
- D-083 lift (S3 Object Lock + JWS build-out approved) — triggers tamper-evident ledger checkpoint metric (metric 3).
- Onboarding-grant lift (auto-grant implemented) — triggers onboarding funnel metric (metric 4).
When any trigger fires, re-run /ci-run with a metrics-focused milestone
to activate the corresponding placeholder views.