Jon Chery
b054849a99
P4 (Wave 3, docs) — REQ-186, 191, 192, 193, 194, 195, 204, 210, 211, 212, 213 New docs: - docs/METRICS.md — canonical KPI catalog (grounded/derived/deferred) - docs/metrics/*.md — 13 per-KPI definition-of-success docs (D-127) - docs/METRICS_DEFERRED_ROADMAP.md — 8 deferred metrics + hot-path plan + re-eval triggers (REQ-210) - docs/NO_HUMANS_THESIS.md — thesis defensibility brief (REQ-213) New tools: - core/metrics/trust_snapshot.py — 5 trust metrics + chain-integrity verdict + snapshot hash (REQ-211) - scripts/check_north_star_diff.sh — CI check for NORTH_STAR strategic section changes (REQ-204) Modified: - .ciagent/config.json — strategic_direction_file: .ciagent/NORTH_STAR.md (REQ-186) ---ci--- project: acdl phase: 4 milestone: v1.17 status: execute ---/ci---
70 lines
4.1 KiB
Markdown
70 lines
4.1 KiB
Markdown
# Nova Deferred Metrics Activation Roadmap
|
|
|
|
> v1.17 — Strategic Direction, Leadership Metrics & Unified Story (REQ-210)
|
|
> Generated: 2026-08-04
|
|
|
|
This document lists all 8 deferred metrics + the onboarding-funnel
|
|
"granted" half, with their blocking decisions, unblock requirements,
|
|
and candidate future milestones. It also includes the hot-path activation
|
|
plan (post-D-096) and the re-evaluation triggers.
|
|
|
|
## Deferred metrics
|
|
|
|
| # | Metric | Blocking Decision | What's Needed to Unblock | Candidate Milestone |
|
|
|---|--------|-------------------|-------------------------|---------------------|
|
|
| 1 | Live Infrastructure Health (ECS, ALB, RPS) | D-096 | Re-provision live AWS; deploy microservice/static-assets stacks; emit live health metrics | v1.18+ (live AWS re-provisioning) |
|
|
| 2 | Live Outbox Write Rate / Ledger Append Latency | D-096 | Re-provision DynamoDB outbox table; emit write-latency metrics | v1.18+ |
|
|
| 3 | Tamper-Evident Ledger Checkpoints / JWS Signature Rate | D-083 | Build S3 Object Lock + JWS signing + async worker + DLQ + daily checkpoints | v1.19+ (audit ledger build-out) |
|
|
| 4 | Onboarding Funnel (requested → granted) | D-113/D-114/D-119 | Implement auto-grant: Lambda provisions the cross-account role + ABAC tag + environment binding | v1.18+ (onboarding auto-grant) |
|
|
| 5 | Drift Auto-Reversal Rate | D-096 + no scheduler | Build a drift-detection scheduler (cron); run `terraform plan -detailed-exitcode` per workspace; emit drift.detected events | v1.20+ (drift detection) |
|
|
| 6 | Live CUR Reconciliation | D-096 | Re-provision live AWS billing access; build CUR reconciler (6h schedule); match bill lines to resource addresses via tags | v1.18+ |
|
|
| 7 | SLA / Unplanned Downtime | D-096 | Deploy live services with SLOs; emit uptime metrics against SLO targets | v1.18+ |
|
|
| 8 | Predictive vs Reactive Ratio | future emitter | Build an ML anomaly-forecasting service; emit anomaly.predicted events with proactive label | v1.21+ (predictive ops) |
|
|
|
|
## Onboarding-funnel "granted" half
|
|
|
|
The onboarding request path is grounded (REQ-182/183 from v1.16): a
|
|
consumer submits a request → the Lambda writes a `pending` CMDB row →
|
|
`core/onboarding.py` generates a binding file. The "granted" half
|
|
(actual AWS account/network/state provisioning) is deferred per
|
|
D-113/D-114/D-119. When a future milestone implements auto-grant, the
|
|
onboarding funnel metric activates: `count(granted) ÷ count(requested)`.
|
|
|
|
## Hot-Path Activation (post-D-096)
|
|
|
|
**Current state (v1.17):** SQLite cold store only (D-126). No hot path.
|
|
The hot path activates when live AWS is re-provisioned (D-096 lift).
|
|
|
|
**Nova-native hot-path candidates (D-120 — no Kafka/Prometheus/ClickHouse):**
|
|
1. **SQLite read-replica:** the cold store becomes a read-replica updated
|
|
on each run; a lightweight file-watcher notifies the dashboard of
|
|
changes. Freshness = "last run" (not 1-second, but sufficient for
|
|
batch ops).
|
|
2. **JSONL tail + webhook:** the events.jsonl log is tailed by a small
|
|
daemon that pushes updates to a webhook (e.g., a PowerBI streaming
|
|
dataset or a custom dashboard). Nova-native (no new infra).
|
|
3. **SQLite + Grafana SQLite datasource:** Grafana can read SQLite
|
|
directly via the SQLite datasource plugin. No TSDB needed.
|
|
|
|
**Migration steps (when D-096 lifts):**
|
|
1. Re-provision live AWS (microservice + static-assets stacks).
|
|
2. Add live-health emitters (ECS running count, ALB 5xx, RPS) to
|
|
`run_platform.sh`.
|
|
3. Choose a hot-path candidate (above) and implement it.
|
|
4. Populate the 8 placeholder views with real data.
|
|
5. Re-run the collector + PowerBI export.
|
|
|
|
## Re-evaluation Triggers
|
|
|
|
A follow-up metrics ideation should be triggered when any of these
|
|
events occurs:
|
|
|
|
1. **D-096 lift** (live AWS re-provisioned) — triggers hot-path
|
|
activation + placeholder view population for metrics 1, 2, 5, 6, 7.
|
|
2. **D-083 lift** (S3 Object Lock + JWS build-out approved) — triggers
|
|
tamper-evident ledger checkpoint metric (metric 3).
|
|
3. **Onboarding-grant lift** (auto-grant implemented) — triggers
|
|
onboarding funnel metric (metric 4).
|
|
|
|
When any trigger fires, re-run `/ci-run` with a metrics-focused milestone
|
|
to activate the corresponding placeholder views. |