Jon Chery
f8616b806e
P1 (Wave 1, feat) — REQ-187, REQ-188, REQ-205 (emitter), REQ-206 (emitter) New components: - core/metrics/event_envelope.py — CloudEvents 1.0 envelope + platform.* conventions - core/metrics/run_manifest.py — per-run manifest writer (nova.run.started/completed/failed) - core/metrics/decision_ledger.py — SQLite append-only hash-chain (ai.decision.made + attestation.recorded) - core/metrics/infracost_adapter.py — Infracost post-processor (degraded mode when CLI absent, A6) - schemas/metrics_event.schema.json — CloudEvents envelope schema - schemas/metrics_run_manifest.schema.json — per-run manifest schema - metrics/README.md — backup/restore doc (REQ-201) - tests/test_metrics_emitters.py — 16 tests (all pass) Modified components: - core/confidence_signal.py — emits nova.confidence.computed + nova.ai.decision.made (D-122) - core/hitl_gates.py — emits nova.attestation.recorded on qa/prod/dr gates (D-132) - adapters/terraform/policy/checkov_adapter.py — emits nova.policy.evaluated - pyproject.toml — addopts gains --junitxml + --json-report + --cov (REQ-206) - .gitignore — metrics runtime artifacts ignored D-120: Nova-native (JSONL + SQLite, no Kafka/OTel) D-121: Decision Ledger = outbox_writer extension → SQLite hash-chain D-122: AI decision = confidence_signal + HITL gate (not LLM) D-128: metrics/ at repo root D-132: Attestation instrumentation ---ci--- project: acdl phase: 1 milestone: v1.17 status: execute ---/ci---
51 lines
3.0 KiB
Markdown
51 lines
3.0 KiB
Markdown
# Nova Metrics Directory
|
|
|
|
> v1.17 — Strategic Direction, Leadership Metrics & Unified Story (D-128)
|
|
|
|
This directory holds Nova's telemetry/observability artifacts. The
|
|
metrics layer is **Nova-native** (D-120): JSONL event log + SQLite cold
|
|
store + hash-chained Decision Ledger. No Kafka, Prometheus, ClickHouse,
|
|
or QLDB.
|
|
|
|
## Artifact inventory
|
|
|
|
| Artifact | Type | Regenerable? | Description |
|
|
|----------|------|-------------|-------------|
|
|
| `events.jsonl` | Append-only event log | No (append-only state) | CloudEvents 1.0 envelopes from all emitters (REQ-187) |
|
|
| `decision_ledger.db` | SQLite append-only hash-chain | No (append-only state) | Decision Ledger: `ai.decision.made` + `attestation.recorded` events (REQ-188, D-121) |
|
|
| `nova_metrics.db` | SQLite cold store | Yes (regenerate via collector) | Normalized fact/dimension tables (REQ-189, P2) |
|
|
| `runs/<run_id>.json` | Per-run manifest | Yes (regenerate from events) | Run lifecycle: stages, durations, exit, confidence, HITL (REQ-187) |
|
|
| `runs/<run_id>/` | Durable run artifacts | Yes (regenerate from $WORK) | Persisted copies of pcr.json, signal.json, event.json, etc. (REQ-187) |
|
|
| `lifecycle/<module>-<env>.json` | Lifecycle report | Yes (regenerate from lifecycle runs) | Per-module apply/modify/destroy results (REQ-205) |
|
|
| `test-results.xml` | JUnit XML | Yes (regenerate via pytest) | Test results (REQ-187, P1 addopts) |
|
|
| `test-report.json` | JSON test report | Yes (regenerate via pytest) | Test results in JSON (REQ-187, P1 addopts) |
|
|
| `coverage.json` | Coverage report | Yes (regenerate via pytest) | Code coverage (REQ-206, P1 addopts) |
|
|
| `powerbi/` | PowerBI export | Yes (regenerate via powerbi_export) | CSV/JSON views for PowerBI ingestion (REQ-190, P3) |
|
|
| `TRUST_SNAPSHOT.md` | Trust snapshot report | Yes (regenerate via trust_snapshot) | 5 trust metrics + chain-integrity verdict (REQ-211, P4) |
|
|
|
|
## Backup + restore
|
|
|
|
**Append-only state** (`events.jsonl`, `decision_ledger.db`): these are
|
|
the source of truth. They should be committed to git (events.jsonl) or
|
|
snapshotted (decision_ledger.db). If lost, they CANNOT be regenerated —
|
|
the events they captured are gone.
|
|
|
|
**Regenerable artifacts** (`nova_metrics.db`, `runs/`, `lifecycle/`,
|
|
`test-results.xml`, `coverage.json`, `powerbi/`): these are derived from
|
|
the append-only state + the source signals (REGRESSION_REPORT.json,
|
|
$WORK/*.json, junit XML). If lost, re-run the collector
|
|
(`core/metrics/collector.py`, P2) to rebuild `nova_metrics.db`, then
|
|
re-run the PowerBI export (`core/metrics/powerbi_export.py`, P3) to
|
|
rebuild `powerbi/`.
|
|
|
|
**Restore procedure:**
|
|
1. Recover `events.jsonl` + `decision_ledger.db` from git/snapshot.
|
|
2. `python3 core/metrics/collector.py` → rebuilds `nova_metrics.db`.
|
|
3. `python3 core/metrics/powerbi_export.py` → rebuilds `powerbi/`.
|
|
4. `python3 core/metrics/trust_snapshot.py` → rebuilds `TRUST_SNAPSHOT.md`.
|
|
|
|
## Concurrency model
|
|
|
|
Single-writer per run: the run manifest writer is the only writer per
|
|
run. SQLite WAL mode + `BEGIN IMMEDIATE` prevents concurrent-write
|
|
corruption on the Decision Ledger (P1 risk mitigation). |