docs(P00): research findings — v1.27 staleness inventory + backfill sources

NFR milestone, no new domain. Research is a codebase-grounded inventory:
- 11 files to archive (4 pre-execution v1.26 artifacts + 3 phase
  verifications/review + 1 evidence snapshot + 3 durable refs superseded
  by STATE.md/NORTH_STAR/archive + 1 consumer ROADMAP).
- 12 files kept active (no-edit: live code paths, durable refs).
- 3 files kept active (fix-only: PROJECT.md, ROADMAP.md, archive/README.md).
- STATE.md backfill sources: regression_verify.py, registry.json,
  REQUIREMENTS traceability, CHECKPOINT, git log, PROJECT decisions.
- Persona roster: lead-developer only (docs/chore milestone).
- 4 risks, all Low-Medium with documented mitigations.

---ci---
project: acdl
phase: 0
milestone: v1.27
status: research
---ci---
This commit is contained in:
Jon Chery
2026-08-19 19:09:32 +00:00
parent 155963d40d
commit e9fbb44ad1
2 changed files with 104 additions and 210 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
{
"phase": 0,
"stage": "specify",
"stage": "clarify",
"milestone": "v1.27",
"phase_role": "pre_execution",
"attempts": 0,
+103 -209
View File
@@ -1,250 +1,144 @@
# Nova — v1.26 Research Findings
# Nova — v1.27 Research Findings
> Phase: research (pre-execution). Milestone: v1.26 (Live Pilot Estate
> Activation). Status: research. Researcher: ci-researcher.
> Phase: research (pre-execution). Milestone: v1.27 (PO State Catalog &
> Ciagent Compression). Status: research. Researcher: ci-researcher.
> Autonomy: full.
---
## 1. Domain — Homegrown PoA Blockchain for Securities Settlement
### 1.1 Why a homegrown chain (not Ethereum/Solana/Hyperledger)
The pilot's purpose is to exercise the Nova platform's deploy/policy/
attestation gates over a real consumer estate — not to build a
production blockchain. A homegrown PoA ledger is the minimal viable
chain: append-only blocks, single validator (pilot), SHA-256 hash chain,
deterministic block production. It records every order, match, and
settlement as transactions; settlement finality = block commit. This
is sufficient to demonstrate that Nova's policy engine (kyverno-json)
can assert settlement finality declaratively (REQ-315) and that the
Decision Ledger captures the apply decision.
A production chain (Ethereum/Solana/Hyperledger) would be the *consumer
app's* choice, not the platform's. The platform is chain-agnostic — it
deploys whatever the consumer's `contract.yaml` declares. For the pilot,
the homegrown chain is the simplest way to produce a real consumer
estate without a heavyweight external dependency.
### 1.2 PoA consensus — single validator (pilot)
Proof-of-Authority with a single validator is the minimal consensus
model: the validator proposes + commits blocks. No Byzantine fault
tolerance (single validator = no forks). Deterministic block
production: same ordered transactions → same block (same hash). This
makes the chain auditable (the hash chain is verifiable) and
reproducible (a replay produces the same chain). Multi-validator BFT
is a future milestone (D-201).
### 1.3 T+1 settlement finality
Equities settle T+1 (trade date + 1 business day). The pilot's
settlement service records matches as transactions on the chain; a
settlement is final when its block is committed. The settlement-finality
kyverno-json policy (REQ-315) asserts `all_committed: true` before any
promotion (qa→prod) — the declarative gate that turns settlement
finality into a policy artifact. This is the securities-specific
extension of v1.25's policy engine: the same `KyvernoJsonEngine`
evaluates a policy over a new payload shape (settlement-service status
JSON).
### 1.4 Equities-only scope (D-200)
Bonds (T+2), derivatives (varying), and options (exercise models) have
different settlement models. A pilot should demonstrate the Nova
platform's gates over the simplest case (equities T+1) before
expanding. "All types of securities" is the product vision; v1.26 is
the pilot (equities first). Future milestones add other security types
with their settlement models.
v1.27 is an NFR milestone (docs/chore only, no code, no schema). There
is no new domain to research. The research is a codebase-grounded
inventory of (a) the files to archive + their staleness evidence, and
(b) the sources backing the STATE.md capability backfill. This file
records the inventory for the v1.27 record; the active authoring used
these sources directly.
---
## 2. Nova Consumer Deploy Model
## 1. Files to archive (staleness inventory)
### 2.1 The reusable `deploy.yml@v1.25` workflow
### 1.1 Pre-execution artifacts (v1.26 — shipped, decisions folded)
The platform's `.github/workflows/deploy.yml` is a `workflow_call`
a reusable workflow that a consumer repo invokes via
`uses: acdl/.github/workflows/deploy.yml@v1.25`. Inputs: `contract`
(default `.nova/contract.yml`), `mode` (default `full`; enum
`full|plan-only|check-only|decommission`), `environment` (override).
The workflow checks out the consumer repo + the platform repo, runs
`scripts/run_platform.sh`, and records the apply decision +
attestation in the Decision Ledger. Secrets: `NOVA_AWS_*`
(account + access key + secret) + `NOVA_LAMBDA_URL` (error reporting).
| File | Lines | Staleness evidence | Decisions folded into |
|---|---|---|---|
| `CLARIFY.md` | 225 | v1.26 milestone shipped (`v1.25.5`); decisions D-200..D-213 | `PROJECT.md` load-bearing decisions |
| `GRILL.md` | 225 | v1.26 grill verdict PROCEED 0.84; binding revisions applied | `PLAN.md` revisions (G-Q4 REQ-322→P2 W0; G-Q6 enforcement deferred; G-Q9 key-split future) |
| `IDEATE.md` | 193 | all 7 accepted ideas → REQ-315..322 (shipped) | `REQUIREMENTS.md` v1.26 traceability |
| `RESEARCH.md` | 250 | v1.26 domain research (blockchain, deploy, modules, metrics) | `ARCHITECTURE.md` §12.8; shipped REQs |
The pilot consumer (`nova-blockchain-exchange`) invokes this workflow
with `mode: full` for `dev` (D-209). The `.gitea/workflows/deploy.yml`
mirror is byte-identical (the platform's deploy workflow is
forge-agnostic — Gitea + GitHub).
All four are pre-execution artifacts for a shipped milestone. The next
P0 writes fresh versions. Per D-219 (user-confirmed) + D-222: archive
all four with `-v1.26` suffixes.
### 2.2 `run_platform.sh --apply` path (confirmed)
### 1.2 Phase verifications + review (v1.26 — shipped, PASS)
`scripts/run_platform.sh:431-455` — the `--apply` (or `mode: full`)
path runs `terraform apply -auto-approve` after the HITL gate
(`:438`). For `dev` (autonomous, no HITL gate), the apply proceeds
directly. The apply records the env via `core/env_transition.py record`
(`:450`). The full pipeline (no `--apply` flag) continues to Step 7
(confidence signal) + Step 8 (outbox write).
| File | Lines | Staleness evidence |
|---|---|---|
| `VERIFY-P03.md` | 39 | v1.26 P3 verification — PASS; shipped `v1.25.3` |
| `VERIFY-P04.md` | 31 | v1.26 P4 verification — PASS; shipped `v1.25.4` |
| `REVIEW-AUDIT-P05.md` | 218 | v1.26 P5 final review + audit — PROCEED; shipped `v1.25.5`; 0 P0 remain; audit CLEAN |
| `P4-PILOT-RUN-EVIDENCE.md` | 46 | v1.26 live apply evidence (`blkex-pilot-apply-v0.2`); summarized in `nova-blockchain-exchange/README.md` §5 + REVIEW-AUDIT-P05 §2.2 |
**Gap (noted in RESEARCH §4):** the `--apply` path exits before the
outbox write (Step 8). The pilot runs the full pipeline (not `--apply`
alone), so the outbox write happens. The `run.completed` event lands in
the JSONL Decision Ledger (not the DynamoDB outbox) — this is by design
(the outbox is the platform-run evidence stream; the Decision Ledger is
the cold store for metrics).
### 1.3 Durable references (superseded or stale)
### 2.3 Contract schema — multi-module manifest
| File | Lines | Staleness evidence | Superseded by |
|---|---|---|---|
| `CAPABILITY_INVENTORY.md` | 120 | dated 2026-07-27; framed as "v1.1→v1.8 re-verification sweep"; predates v1.26 pilot (CAP-025 absent; blockchain capabilities absent) | `STATE.md` (this milestone) |
| `AUTONOMY_THESIS.md` | 65 | "Last refined: v1.21"; thesis fully folded into `NORTH_STAR.md` Vision (lines 1722) + Anti-Goals #2 | `NORTH_STAR.md` |
| `COST.md` | 106 | dated 2026-07-29; framed "v1.0 → v1.14"; predates v1.26 live pilot (ECS + ALB + DynamoDB + S3 costs not reflected) | A future cost milestone writes a fresh report; `STATE.md` Domain 7 notes cost tracking as a capability |
`schemas/contract.schema.json:7,24-48` — required fields: `id`,
`name`, `environment`, `infrastructure`. The `infrastructure` block is
`minProperties: 1` with `patternProperties` accepting any module name
key. Multi-module manifest is supported: one contract can declare
`infrastructure: { microservice: {...}, dynamodb: {...}, s3: {...} }`.
The constraint is the `modules/registry.json` (the module must be
registered), not the schema.
### 1.4 Consumer-side (nova-blockchain-exchange)
| File | Lines | Staleness evidence |
|---|---|---|
| `nova-blockchain-exchange/ROADMAP.md` | 57 | marks P3/P4/P5 as "planned" but v1.26 shipped (`v1.25.5`); phase narrative preserved in platform `ROADMAP.md` v1.26 section |
Per D-221: consumer archives land in
`.ciagent/nova-blockchain-exchange/archive/ROADMAP-v1.26.md`.
---
## 3. Platform Module Readiness (the critical finding)
## 2. Files to keep active (no-edit or fix-only)
### 3.1 The adapter is stateless (v1.11 rewrite)
### 2.1 No-edit (live code paths or durable)
`adapters/terraform/adapter.py:1-11` — the adapter is a "STATELESS
ASSEMBLER" that owns no module content. There is **no `TYPE_MAP`**,
`INPUT_MAP`, or `OUTPUT_MAP` (deleted in the v1.11 stateless rewrite;
`modules/STANDARDS.md:212-214` confirms). A new stack type requires a
new L1 module (`modules/l1/<name>/` with `interface.json` +
`terraform/main.tf` + `README.md` + `instance.json`) + a
`modules/registry.json` entry — not an adapter change.
| File | Why keep active |
|---|---|
| `CHECKPOINT.json` | Authoritative resume state — never archive |
| `config.json` | Operational config — never archive |
| `REGRESSION_REPORT.json` | Written by `core/regression_verify.py:705`; read by `core/metrics/collector.py:27` + `trust_snapshot.py:21` + metrics views (D-224: regenerates on next `run_regression.sh`) |
| `REGRESSION_REPORT.md` | Written by `core/regression_verify.py:704`; read by `scripts/run_regression.sh` (D-224) |
| `PERSONAS.md` | Regenerated at each milestone P0 by the lead-developer; not stale until then |
| `IAM_POLICY.md` | Live baseline, test-enforced (`tests/test_iam_policy_baseline.py`); D-207 future key-split pending (D-223) |
| `PLAN.md` | Active phase plan; reset to next milestone at next P0 |
| `ARCHITECTURE.md` | Durable target architecture (§1–§12 + §12.7 + §12.8 + §12.9) |
| `NORTH_STAR.md` | PO strategy; loaded every ci-run via `config.strategic_direction_file` |
| `nova-blockchain-exchange/PROJECT.md` | Consumer project charter; D-200..D-205 load-bearing |
| `nova-blockchain-exchange/REQUIREMENTS.md` | REQ-310..322 spec intent (shipped but spec stays for reference) |
| `nova-blockchain-exchange/README.md` | Consumer onboarding guide; still accurate (deploy workflow, secrets, contract shape, verification) |
### 3.2 ECS — ready
### 2.2 Fix-only (corrections to stale-but-kept files)
`modules/l1/ecs-service/terraform/main.tf:1,11`
`aws_ecs_task_definition` + `aws_ecs_service`. `interface.json:5-6`
`type: aws:ecs:task_definition`. `registry.json:29-37` — registered.
Tests: `test_adapter.py:164-185,257-360`, `test_contract_resolver.py:61-92`.
The `microservice` L2 (`modules/l2/microservice/composition.json`)
references 6 L1 children (ecs-cluster, ecr, iam-role, alb, ecs-service,
kms-key) — the ECS pattern is fully wired end-to-end.
### 3.3 S3 — ready
`modules/l1/s3/terraform/main.tf:1``aws_s3_bucket` (+ versioning +
SSE). `interface.json:5-6``type: aws:s3:bucket`. `registry.json:2-10`
— registered. Tests: `test_adapter.py:56-110,241-257`,
`test_contract_resolver.py:36-51,92-130`.
### 3.4 DynamoDB — GAP (REQ-322)
**No `modules/l1/dynamodb/` directory, no `registry.json` key, no
`interface.json`, no `terraform/`, no tests.** The blockchain exchange's
ledger table needs this primitive. REQ-322 authors it: `interface.json`
(stack type `aws:dynamodb:table`), `terraform/main.tf`
(`aws_dynamodb_table` with PK + optional SK, `PAY_PER_REQUEST` default,
encryption + PITR enabled per v1.8 NFR defaults), `README.md`,
`instance.json`, + `registry.json` entry. The adapter needs no change
(stateless); the contract's `infrastructure.dynamodb` block references
this primitive. This is the single platform-side module build-out for
the milestone.
### 3.5 Stale doc (not a blocker)
`adapters/README.md:49-54` references the deleted `TYPE_MAP`/
`INPUT_MAP`/`OUTPUT_MAP` — contradicts `adapter.py:1-11` +
`modules/STANDARDS.md:212-214`. REQ-321 (docs) should fix this.
| File | Fix |
|---|---|
| `PROJECT.md` | v1.26 phase-status block (lines 424431): P3/P4/P5 "pending" → "complete" with shipped tags `v1.25.3/4/5`; add STATE.md pointer (D-225: P2 phase) |
| `ROADMAP.md` | v1.26 P3/P4/P5 sections (lines 238, 261, 272) "planned" → "complete" with shipped tags; v1.26 Overview line "active" → "complete"; add STATE.md to P5 ship-update list (D-225: P2 phase) |
| `archive/README.md` | Add the 11 new archived files to the contents tables (P2 phase) |
---
## 4. Metric Pipeline Grounding (Post-Pilot targets)
## 3. STATE.md capability backfill sources
### 4.1 AI Decision Accuracy — outcome backfill (REQ-317)
The STATE.md backfill (36 capabilities across 10 domains) was sourced
from:
`core/metrics/decision_ledger.py:210-211` documents the event chain:
`confidence.computed → ai.decision.made → attestation.recorded →
run.completed/failed`. `collector.py:262` inserts `fact_decision.outcome`
as `"pending"`**there is no outcome-backfill step** wiring
`run.completed`/`run.failed` back into `fact_decision.outcome`. The AI
Decision Accuracy metric (`trust_snapshot.py:70-85`, `_get_ai_decision_accuracy`)
reads `decisions WHERE outcome='succeeded' ÷ total` — so it reads 0%
today (all pending). REQ-317 adds `core/metrics/outcome_backfill.py`
that reads run-manifest events and updates `fact_decision.outcome` +
`fact_decision.backfilled_at`. The PCR schema is unchanged (D-211).
### 4.2 Human Escalation Frequency — `reason='confidence'` tag (REQ-318)
`core/confidence_signal.py:184` — a `block` band sets
`human_override=True` in the `ai.decision.made` event.
`run_platform.sh:636` fails the pipeline on `block`. The Human
Escalation Frequency metric (`docs/metrics/human_escalation_frequency.md:11-12`)
is defined as `count(runs WHERE hitl_block=1 AND reason='confidence') ÷
total runs`. The `reason='confidence'` discriminator is **not currently
stored** — `hitl_block` is a boolean from the manifest. REQ-318 adds
`escalation_reason: 'confidence'` to the `ai.decision.made` event when
`band == 'block'` + persists it into `fact_run` via the collector.
### 4.3 Touchless Resolution Rate — denominator activates post-pilot
`docs/metrics/touchless_resolution_rate.md:12-15` — defined as a SQL
query over `fact_run` (`runs WHERE hitl_block=0 ÷ total runs`). The data
lands in `fact_run.hitl_block` via `collector.py:216-227`. No dedicated
emitter computes the ratio — it's a downstream query. The denominator
is 0 today (no consumer runs). The pilot run activates the denominator.
| Source | Used for |
|---|---|
| `core/regression_verify.py` (lines 129768) | CAP-001..CAP-025 IDs, names, tiers, evidence pointers |
| `.ciagent/CAPABILITY_INVENTORY.md` (pre-archive) | CAP-001..022 descriptions, defect notes, evidence |
| `modules/registry.json` | L1/L2 module catalog (13 L1 + 2 L2 entries) |
| `.ciagent/REQUIREMENTS.md` v1.25 traceability | REQ-291..309 → policy-engine capabilities |
| `.ciagent/nova-blockchain-exchange/REQUIREMENTS.md` + `.ciagent/REQUIREMENTS.md` v1.26 traceability | REQ-310..322 → pilot capabilities |
| `.ciagent/CHECKPOINT.json` | shipped tags `v1.25.0..v1.25.5` |
| `git log --all --oneline` | file paths for v1.26 shipped features |
| `.ciagent/PROJECT.md` load-bearing decisions | INV-1..INV-11 invariants |
| `docs/submission-readiness.md` + `schemas/contract.schema.json` | INV-1 contract surface |
---
## 5. kyverno-json Policy Extensibility
## 4. Persona assessment
`adapters/kyverno-json/kyverno_json_engine.py:74-80` — the engine is
**policy-dir agnostic**: it loads whatever subdir the caller passes.
Existing subdirs: `contract/`, `stack-ir/`, `plan-json/`, `meta/`,
`regression/`. Adding a new subdir (e.g. `pilot-readiness/`,
`settlement-finality/`) requires: (1) `mkdir
adapters/kyverno-json/policies/<name>/`, (2) drop `ValidatingPolicy`
YAML/JSON files, (3) wire a caller. No engine code change needed.
Test pattern: one test file per subdir (`tests/test_<name>_policies.py`).
v1.27 is a docs/chore milestone. The active roster:
The pilot adds two new policy subdirs: `pilot-readiness/`
(REQ-320, no-placeholder-account) + `settlement-finality/` (REQ-315,
all-matches-committed). Both follow the established pattern.
- **lead-developer** (active): owns the milestone narrative (STATE.md
authoring, PROJECT/ROADMAP fixes, archive README, PLAN/NORTH_STAR
wiring, this RESEARCH, CLARIFY, PLAN, final review + audit). Territory:
`.ciagent/`, `docs/`.
- **backend-engineer** (active, limited): no code changes in v1.27.
Consulted on the `core/confidence_signal.py` LSP diagnostic (pre-
existing, not touched by v1.27). No territory writes.
- **data-engineer** (inactive): no schema/migration/ORM changes.
- **policy-engineer** (inactive): no policy authoring.
- **frontend-engineer** (inactive): no UI.
- **blockchain-engineer** (inactive): no chain code.
Territory enforcement: warn. The milestone is `.ciagent/`-only; the
lead-developer owns all writes.
---
## 6. Env-JSON Wiring Reconciliation (REQ-319)
`core/environments/dev.json:4``account_id: "000000000000"` (placeholder).
`core/environment_check.py:48-53` warns (non-fatal) when account_id is
placeholder + env != dev. `adapters/terraform/adapter.py:116-117`
computes the state bucket as `nova-tfstate-<AWS_ACCOUNT_ID>-us-east-1`
from the `AWS_ACCOUNT_ID` env var, **not** from the env JSON's
`state_backend.bucket`. This is the wiring gap: the env JSON's
`state_backend` field is currently unused by the live apply path.
REQ-319 makes the adapter read `env.state_backend.bucket` when present
(falling back to the computed name for backwards compat) + updates
`dev.json` to the real account `581513795199` + real bucket
`nova-tfstate-581513795199-us-east-1`.
---
## 7. Risk Analysis
## 5. Risk analysis
| Risk | Likelihood | Impact | Mitigation |
|---|---|---|---|
| `NOVA_AWS_*` key lacks a needed IAM permission mid-pilot | Low (bootstrap succeeded → root-equivalent) | High (blocks apply) | D-207; the key has root-equivalent perms (empirically confirmed). |
| DynamoDB primitive takes longer than expected (new module) | Medium | Medium | REQ-322 is the single platform-side build-out; the `s3`/`rds` primitives are the template — straightforward. |
| Homegrown chain has a correctness bug (hash chain breaks) | Low | High | REQ-310 tests cover chain integrity, hash determinism, genesis, append/verify. |
| `deploy.yml@v1.25` ref doesn't resolve (floating tag) | Low | High | The platform's `release.yml` creates + force-moves the `v1.25` + `v1` floating tags on merge to main. The pilot contract uses `@v1.25`. |
| Settlement-finality policy false-negatives (blocks a valid promotion) | Medium | Medium | REQ-315 tests cover passing + failing fixtures; the policy is skip-when-kj-absent (graceful). |
| D-083 deferral challenged (audit ledger not tamper-evident) | Low | Low | D-204; the SQLite hash-chain + DynamoDB outbox is the pilot's audit record. Tamper-evidence is a future milestone. |
| Archive move breaks a relative path reference in an active file | Low | Medium | `grep` for the archived filenames across `.ciagent/` + `docs/` before commit; fix any dangling references in P2 |
| STATE.md capability row is inaccurate (wrong shipped tag / wrong file path) | Medium | Low | The backfill sources are the authoritative registries (regression_verify.py, registry.json, CHECKPOINT.json, git log); citations are direct |
| PROJECT.md phase-status fix conflicts with a future v1.26-era commit | Low | Low | v1.26 is shipped (main has the milestone merge); no v1.26-era commits will arrive |
| REGRESSION_REPORT stale state is mistaken for v1.27 scope | Low | Low | D-224 records the decision; STATE.md Domain 7 notes the current CAP range |
---
## 8. Persona Assessment
## 6. Verdict
See `PERSONAS.md` (next section, produced by the lead-developer at the
end of RESEARCH). The active roster: backend-engineer (blockchain core
+ settlement + outcome backfill), data-engineer (DynamoDB primitive +
metrics cold store), policy-engineer (kyverno-json policies), +
blockchain-engineer (custom, phase-specific — chain consensus, order
matching, settlement finality). frontend-engineer is deactivated (no
UI in the pilot).
v1.27 is feasible, scoped, and the sources are grounded. No new domain,
no new code, no schema breaks. The archive moves are lossless (git
history + archive directory both preserve bytes). The STATE.md
backfill is sourced from authoritative registries. Proceed to PLAN.