diff --git a/.ciagent/CHECKPOINT.json b/.ciagent/CHECKPOINT.json index 8ce8989..4915114 100644 --- a/.ciagent/CHECKPOINT.json +++ b/.ciagent/CHECKPOINT.json @@ -1,10 +1,10 @@ { "phase": 0, - "stage": "ideate", + "stage": "plan", "milestone": "v1.26", "phase_role": "pre_execution", "attempts": 0, - "updated_at": "2026-08-12T21:14:00Z", + "updated_at": "2026-08-12T21:15:00Z", "project": "acdl", "projects": ["acdl", "nova-blockchain-exchange"], "active_milestone": "v1.26", diff --git a/.ciagent/PLAN.md b/.ciagent/PLAN.md index 3d4d4ce..f6d6095 100644 --- a/.ciagent/PLAN.md +++ b/.ciagent/PLAN.md @@ -1,371 +1,389 @@ -# PLAN — v1.25 (kyverno-json Unified Policy Engine) +# PLAN — v1.26 (Live Pilot Estate Activation) -> Feature milestone. Tags on the **v1.24.x** line: v1.24.0 (P0) → -> v1.24.1 (P1) → v1.24.2 (P2) → v1.24.3 (P3) → v1.24.4 (P4) → v1.24.5 -> (P5 final = milestone release). 19 requirements (REQ-291..309), -> 4 execution phases + P0 pre-execution + P5 final review/ship. - -## Wave model - -Each phase is a **vertical slice** (end-to-end: policy files + Python -wiring + tests + docs). Phases are ordered by dependency: the engine -protocol (P1) must exist before policies (P2/P3) can be wired; the -pipeline wiring (P3) must exist before the meta-policies (P3) can -consume the merged PCR list; the regression-gate policies (P4) are -independent of the pipeline and can be authored in parallel with P3's -tests, but ship after P3 because they reference the engine registry -finalized in P1. Within each phase, the waves are the persona task -groups (parallelizable across personas when `parallelization.enabled: -true`, `max_concurrent_agents: 5`). - -## Phase breakdown - -### Phase P1 — engine-core (Wave 1, backend-engineer + policy-engineer + data-engineer) - -**Type:** `feat` (engine protocol + registry + kyverno-json engine adapter + install + tests) - -**Requirements:** REQ-291, REQ-292, REQ-293, REQ-294, REQ-308, REQ-309 - -**Must-haves:** -- `core/policy_engine.py` — `PolicyEngine` Protocol (PEP 544) + - `PolicyEngineRegistry` (selects from `config.json.policy.engine`) + - `NullEngine` fallback (emits `SKIPPED` when `policy` key absent) - (REQ-291) -- `.ciagent/config.json` gains `policy` object: `{"engine": - "kyverno-json", "policy_root": - "adapters/kyverno-json/policies"}` (REQ-292) -- `adapters/kyverno-json/kyverno_json_engine.py` — `KyvernoJsonEngine` - implementing the protocol: `is_configured()` guards on `which kj`; - `evaluate()` writes payload to temp JSON, invokes - `kj scan --policy --payload --output json`, translates - native output → `list[dict]` PCR records (`engine: "kyverno"`, - `ruleId` prefixed `KJ_`, severity from - `nova.cloudinit.dev/severity` annotation); defensive parsing - (malformed → `error` PCR, never exception); `is_configured()==false` - → single `SKIPPED` PCR (`KJ_ENGINE_NOT_CONFIGURED`) (REQ-293) -- `adapters/kyverno-json/__init__.py` exports `KyvernoJsonEngine`; - `adapters/kyverno-json/policies/_smoke.json` trivial - `require-contract-id` policy for round-trip validation; - `scripts/install-kyverno-json.sh` runs - `go install github.com/kyverno/kyverno-json/cmd/kj@latest`; - `.github/workflows/ci.yml` + `.gitea/workflows/ci.yml` install Go + kj - (cached) (REQ-294) -- `tests/test_policy_engine.py` — protocol conformance, registry - selection, unknown-engine `KeyError`, `NullEngine` fallback, - `is_configured()` false when `which kj` absent (mocked) (REQ-308) -- `tests/test_kyverno_json_engine.py` — `evaluate()` returns PCR dicts - validating against `schemas/policy_check_result.schema.json` (via - `jsonschema`); defensive parsing (malformed kyverno-json output → - `error` PCR); `is_configured()==false` → `SKIPPED` with - `KJ_ENGINE_NOT_CONFIGURED`; `pytest.skip("kj not installed")` when - `which kj` absent (REQ-309) - -**Vertical slice:** The `PolicyEngineRegistry.get_engine()` returns a -configured `KyvernoJsonEngine` that can `evaluate()` a trivial payload -against `_smoke.json` and produce a valid PCR list. The confidence -signal is unchanged — it already consumes `list[PolicyCheckResult]`. -The platform runs with or without the `kj` binary (`is_configured()` -guard). All existing tests pass (NullEngine fallback when `policy` key -absent in test config — but the v1.25 config.json *sets* the key, so -existing tests that use the real config get `KyvernoJsonEngine` with -`is_configured()==false` → `SKIPPED`). - -**Files touched:** -- `core/policy_engine.py` (NEW) -- `.ciagent/config.json` (MODIFIED — `policy` object) -- `adapters/kyverno-json/__init__.py` (NEW) -- `adapters/kyverno-json/kyverno_json_engine.py` (NEW) -- `adapters/kyverno-json/policies/_smoke.json` (NEW) -- `scripts/install-kyverno-json.sh` (NEW) -- `.github/workflows/ci.yml` (MODIFIED — Go + kj install step) -- `.gitea/workflows/ci.yml` (MODIFIED — Go + kj install step) -- `tests/test_policy_engine.py` (NEW) -- `tests/test_kyverno_json_engine.py` (NEW) - -**Verification:** `pytest tests/test_policy_engine.py -tests/test_kyverno_json_engine.py tests/test_confidence_signal.py -tests/test_adapter.py tests/test_checkov_adapter.py -tests/test_kyverno_adapter.py -v` (new tests pass or skip-without-kj; -existing adapter/confidence tests unchanged). `python3 -m py_compile -core/policy_engine.py adapters/kyverno-json/kyverno_json_engine.py`. +> Feature milestone. Tags on the **v1.25.x** line: v1.25.0 (P0) → +> v1.25.1 (P1) → v1.25.2 (P2) → v1.25.3 (P3) → v1.25.4 (P4) → v1.25.5 +> (P5 final = milestone release). 13 requirements (REQ-310..322), +> 5 phases (P0 pre-execution + 4 execution + 1 final). Multi-project: +> `acdl` (platform) + `nova-blockchain-exchange` (consumer). Tags run +> on the previous minor's patch line per `run.md` versioning logic +> (feature milestone — at least one feat phase; progressive patches per +> phase; the final phase's patch IS the milestone release; no separate +> minor tag). --- -### Phase P2 — contract + stack-IR policies (Wave 2, policy-engineer + backend-engineer) +## Phase 0 — Pre-Execution (complete, tag v1.25.0) -**Type:** `feat` (policies + resolver wiring + tests) +SPECIFY → CLARIFY → RESEARCH → IDEATE → PLAN → GRILL. All `.ciagent/` +MD, research, plans. Ships as `v1.25.0` on the v1.25.x line. -**Requirements:** REQ-295, REQ-296, REQ-297, REQ-298, REQ-299 +**Pre-run (Workstream A, on main before branch gate):** +- A1: flaky test fix (commit `8c68d68`, pushed). +- A2: ACDL_*→NOVA_* bootstrap migration (commit `f844fea`, pushed). +- A3: AWS bootstrap — S3 state bucket + DynamoDB outbox created. +- A4: `nova-blockchain-exchange` Gitea repo created + cloned. -**Must-haves:** -- `adapters/kyverno-json/policies/contract/` — 4 policies over consumer - contract JSON: `require-id-pattern.json`, - `require-env-in-enum.json`, `require-infrastructure-min-1.json`, - `forbid-unknown-fields.json` — each a `ValidatingPolicy` with one - `validate.assert` rule using JMESPath against the payload root; - severity via `nova.cloudinit.dev/severity` annotation (REQ-295) -- `core/contract_resolver.py` invokes - `PolicyEngineRegistry.get_engine().evaluate(contract_dict, - policies/contract/, contract_id)` **before** resolving; failures - feed the `policy` input as `fail` PCRs (no resolver exit — confidence - signal decides the gate, `--soft-fail` pattern); emits - `nova.policy.evaluated` metrics event (REQ-296) -- `adapters/kyverno-json/policies/stack-ir/` — 3 policies over - resolved Stack IR: `require-tagging-standard.json` (ports - `nova_tagging.py` — `nova:owner` + `nova:environment` tags on every - `resources[]` entry), `forbid-public-ingress.json` (v1.0 demo rule), - `require-encryption-by-default.json` (v1.8 D-encryption-default); - `~` modifier iterates `resources[]` (REQ-297) -- `core/contract_resolver.py` invokes the engine with the resolved - Stack IR and `policies/stack-ir/` **after** resolving; resulting PCRs - appended to the contract-policy PCRs; resolver return values and - exceptions unchanged (additive) (REQ-298) -- `tests/test_stack_ir_policies.py` + `tests/fixtures/stack_ir/` — - passing IR (all tags + encryption) + failing IR (missing tags, public - ingress, plaintext bucket); each policy in isolation + full dir as - bundle; `pytest.skip("kj not installed")` when `which kj` absent - (REQ-299) - -**Vertical slice:** A consumer contract passes through the resolver -and produces two PCR lists (contract policies pre-resolve, stack-IR -policies post-resolve) that feed the confidence signal. A contract -with a bad `id` or missing tags produces `fail` PCRs that lower the -confidence score. The resolver's existing tests pass unchanged (the -policy call is additive — it does not change resolver return values -or exceptions). - -**Files touched:** -- `adapters/kyverno-json/policies/contract/require-id-pattern.json` (NEW) -- `adapters/kyverno-json/policies/contract/require-env-in-enum.json` (NEW) -- `adapters/kyverno-json/policies/contract/require-infrastructure-min-1.json` (NEW) -- `adapters/kyverno-json/policies/contract/forbid-unknown-fields.json` (NEW) -- `adapters/kyverno-json/policies/stack-ir/require-tagging-standard.json` (NEW) -- `adapters/kyverno-json/policies/stack-ir/forbid-public-ingress.json` (NEW) -- `adapters/kyverno-json/policies/stack-ir/require-encryption-by-default.json` (NEW) -- `core/contract_resolver.py` (MODIFIED — pre/post resolve engine calls) -- `tests/test_stack_ir_policies.py` (NEW) -- `tests/fixtures/stack_ir/passing.json` (NEW) -- `tests/fixtures/stack_ir/failing.json` (NEW) - -**Verification:** `pytest tests/test_contract_resolver.py -tests/test_stack_ir_policies.py tests/test_policy_engine.py -v` -(existing resolver tests pass; new policy tests pass or skip-without- -kj). `python3 -m py_compile core/contract_resolver.py`. +**Phase 0 stages (on `phase/00-specify-clarify-research-plan`):** +- SPECIFY: v1.26 established in config.json + PROJECT.md + ROADMAP.md + + `.ciagent/nova-blockchain-exchange/{PROJECT,REQUIREMENTS,ROADMAP}.md`. +- CLARIFY: 10 ambiguities resolved (D-200..D-213). +- RESEARCH: PoA blockchain, deploy model, DynamoDB gap (REQ-322), + metric grounding, persona assessment (5 personas). +- IDEATE: 7 ideas accepted (I1..I7 → REQ-315..322), 3 deferred. +- PLAN: this file. +- GRILL: adversarial review (binding verdicts). --- -### Phase P3 — plan-JSON policies + meta-orchestration + pipeline wiring (Wave 3, policy-engineer + backend-engineer) +## Phase 1 — blockchain-core (tag v1.25.1) -**Type:** `feat` (plan-JSON policies + meta-policies + run_platform.sh wiring + tests) +**Goal:** The consumer repo has a working homegrown PoA blockchain + +order-matching engine + settlement service. All unit tests pass in the +consumer repo's own CI. -**Requirements:** REQ-300, REQ-301, REQ-302, REQ-303 +**Project:** `nova-blockchain-exchange` (consumer repo). +**Branch:** `nova-blockchain-exchange/phase/01-blockchain-core`. +**Persona:** blockchain-engineer (primary), lead-developer (coordination). -**Must-haves:** -- `adapters/kyverno-json/policies/plan-json/` — 3 policies over - `terraform show -json` output: `forbid-plaintext-secrets.json` (ports - CKV_AWS_41/45/46), `forbid-iam-wildcard.json` (ports CKV_AWS_1/40), - `require-kms-reference.json` (ports CKV_AWS_7/33); JMESPath over - `planned_values.root_module.resources[]` (REQ-300) -- `run_platform.sh` Step 5 gains a parallel kyverno-json pass: after - Checkov/Wiz produce raw PCRs, the script runs - `kj scan --policy adapters/kyverno-json/policies/plan-json/ - --payload -o json` and pipes through - `adapters/kyverno-json/kyverno_json_engine.py` to produce a second - PCR list; both lists concatenated and fed to the confidence signal; - `nova.policy.evaluated` event with both engine names; when - `which kj` is false, logs and proceeds with Checkov/Wiz list only - (no hard failure) (REQ-301) -- `tests/test_plan_json_policies.py` + `tests/fixtures/plan_json/` — - passing plan (no secrets, no wildcard, KMS alias) + failing plan - (plaintext password, `Action: "*"`, inline KMS key); policies in - isolation + bundle; `tests/test_run_platform_plan_json_policies.py` - asserts `run_platform.sh` has the kyverno-json Step 5 block + - concatenates PCR lists (script-substring assertion, pattern from - `tests/test_pipeline.py:79-95`) (REQ-302) -- `adapters/kyverno-json/policies/meta/` — `block-on-any-critical.json` - (asserts no PCR in merged list has `severity: critical` + `result: - fail`; if any does, emits `fail` PCR `KJ_META_BLOCK_CRITICAL` - severity `critical` — declarative source of truth; the - `confidence_signal.py` hard-override stays as defense-in-depth per - D-119) + `tagging-rules-agree.json` (cross-checks Checkov - `NOVA_TAG_NAMING` vs kj `KJ_REQUIRE_TAGGING_STANDARD` by - `resourceRef`; divergence emits `error` PCR per D-118); - `tests/test_meta_policies.py` (REQ-303) +### Wave 1 — chain core (REQ-310) +- **Task 1.1** (blockchain-engineer): `chain/block.py` — Block dataclass + (index, timestamp, prev_hash, transactions, nonce, hash). + `compute_hash()` deterministic (SHA-256). Unit test: `test_block.py`. +- **Task 1.2** (blockchain-engineer): `chain/ledger.py` — Ledger class: + `append_block()`, `verify_chain()`, `get_block(index)`, + `get_latest_block()`. Genesis block on init. Unit test: `test_ledger.py`. +- **Task 1.3** (blockchain-engineer): `chain/validator.py` — PoA + validator: single validator (config-driven), `propose_block(transactions)` + → Block, `commit_block(block)`. Unit test: `test_validator.py`. -**Vertical slice:** `run_platform.sh` Step 5 produces a merged PCR list -(Checkov/Wiz + kj plan-JSON policies + kj meta-policies over the -merged list) that feeds the confidence signal. A plan with a plaintext -secret produces two `fail` PCRs (one Checkov, one kj) for the same -resource — visible defense-in-depth. A critical finding anywhere -produces a `KJ_META_BLOCK_CRITICAL` meta-PCR that the confidence -signal's hard-override blocks. The pipeline runs with or without `kj` -(graceful skip). +### Wave 2 — order engine + settlement (REQ-311, REQ-312) — parallel with Wave 1 tail +- **Task 2.1** (blockchain-engineer): `engine/order.py` — Order + dataclass (id, side, symbol, price, size, timestamp). +- **Task 2.2** (blockchain-engineer): `engine/order_book.py` — + OrderBook: `add_order(order)`, `match_orders()` → list of Match + (price-time priority, partial fills). Unit test: `test_order_book.py`. +- **Task 2.3** (blockchain-engineer): `settlement/service.py` — + SettlementService: `settle(match)` → SettlementTransaction, + `submit(ledger)`. Idempotent (re-settling a match is a no-op once + final). Finality = block commit. Unit test: `test_settlement.py`. -**Files touched:** -- `adapters/kyverno-json/policies/plan-json/forbid-plaintext-secrets.json` (NEW) -- `adapters/kyverno-json/policies/plan-json/forbid-iam-wildcard.json` (NEW) -- `adapters/kyverno-json/policies/plan-json/require-kms-reference.json` (NEW) -- `adapters/kyverno-json/policies/meta/block-on-any-critical.json` (NEW) -- `adapters/kyverno-json/policies/meta/tagging-rules-agree.json` (NEW) -- `scripts/run_platform.sh` (MODIFIED — Step 5 kj parallel pass) -- `tests/test_plan_json_policies.py` (NEW) -- `tests/test_meta_policies.py` (NEW) -- `tests/test_run_platform_plan_json_policies.py` (NEW) -- `tests/fixtures/plan_json/passing.json` (NEW) -- `tests/fixtures/plan_json/failing.json` (NEW) +### Wave 3 — consumer CI (cross-cutting) +- **Task 3.1** (blockchain-engineer): `.github/workflows/ci.yml` + + `.gitea/workflows/ci.yml` — lint + pytest on chain/engine/settlement. +- **Task 3.2** (lead-developer): `nova-blockchain-exchange/README.md` — + repo overview + dev setup. -**Verification:** `pytest tests/test_plan_json_policies.py -tests/test_meta_policies.py tests/test_run_platform_plan_json_policies.py -tests/test_pipeline.py -v` (new tests pass or skip-without-kj; existing -pipeline tests pass). `python3 -m py_compile` on any modified Python. -Shellcheck on `run_platform.sh` if available. +**Must-haves (verify before ship):** +- `pytest tests/` in the consumer repo passes (chain integrity, hash + determinism, genesis, append/verify, match priority, partial fills, + settlement idempotency, finality check). +- The chain is deterministic (replay produces the same hash chain). +- The consumer CI workflow runs on push. + +**Ship:** tag `v1.25.1`, merge `phase/01` → `milestone/v1.26-pilot-activation`, +Gitea release (best-effort). Delete `phase/01`. --- -### Phase P4 — regression-gate policies + docs (Wave 4, policy-engineer + data-engineer + lead-developer) +## Phase 2 — consumer-contract-and-deploy (tag v1.25.2) -**Type:** `feat` (regression policies) + `docs` (adapter READMEs + ARCHITECTURE + STANDARDS + METRICS) +**Goal:** The consumer repo declares its infrastructure via +`contract.yaml` (validated against the platform's schema) + invokes the +platform's `deploy.yml@v1.25` workflow. The contract references the +`microservice` (ECS), `dynamodb`, + `s3` modules. -**Requirements:** REQ-304, REQ-305, REQ-306, REQ-307 +**Project:** `nova-blockchain-exchange` (consumer repo) + `acdl` +(platform repo — for the `deploy.yml@v1.25` ref + the `v1.25` floating +tag). +**Branch:** `nova-blockchain-exchange/phase/02-contract-and-deploy`. +**Persona:** blockchain-engineer (contract authoring), data-engineer +(registry/DynamoDB dependency check), lead-developer (deploy.yml ref). -**Must-haves:** -- `adapters/kyverno-json/policies/regression/` — 3 policies over - capability-inventory JSON frontmatter: `cap-013-adapter-dedup.json`, - `cap-023-metrics-collector.json`, `cap-024-deck-structure.json`; - emit `pass`/`fail` PCRs per capability; the existing - `core/regression_verify.py` is kept (drives the CI gate); the - policies are the declarative mirror (REQ-304) -- `tests/test_regression_policies.py` + - `tests/fixtures/capability_inventory/clean.json` + - `tests/fixtures/capability_inventory/drifted.json` — clean (all caps - pass) + drifted (duplicate adapter, missing metric status, broken - deck arc); regression gate still 287/287 baseline (new tests - additive, skip-without-kj) (REQ-305) -- `adapters/README.md` gains new kyverno-json adapter row + "Policy - Engine Protocol" section (Protocol, registry, swap boundary, - how-to-add-OpaEngine); `adapters/kyverno-json/README.md` documents - the engine, install path, policy directory layout, 4 policy - categories (REQ-306) -- `.ciagent/ARCHITECTURE.md` §12.7 (added in RESEARCH) is finalized; - `schemas/README.md` notes `engine: "kyverno"` shared by K8s adapter - + kj (distinguished by `ruleId` prefix); `modules/STANDARDS.md` - gains "Policy authoring standard" section for module owners; - `docs/METRICS.md` notes the policy engine is swappable (Strategic - Objective #2 — provable trust via a replaceable substrate) (REQ-307) +### Wave 1 — contract (REQ-313) +- **Task 1.1** (blockchain-engineer): `contract.yaml` — id + (`blkex`), name (`blockchain-exchange`), environment (dev), + infrastructure block (microservice + dynamodb + s3). +- **Task 1.2** (blockchain-engineer): `contracts/blockchain-exchange.dev.yml`, + `.qa.yml`, `.prod.yml` — per-env variants. +- **Task 1.3** (blockchain-engineer): `tests/test_contract_validates.py` + — schema validation against the platform's + `schemas/contract.schema.json`. -**Vertical slice:** The regression gate's capability checks are now -declarative policies auditable as artifacts. A new module owner can -read `modules/STANDARDS.md` "Policy authoring standard" and write a -per-module kyverno-json policy. A new engineer can read -`adapters/README.md` "Policy Engine Protocol" and implement an -`OpaEngine`. The 287/287 baseline is unchanged. +### Wave 2 — deploy invocation (REQ-314) +- **Task 2.1** (blockchain-engineer): `.github/workflows/deploy.yml` — + `uses: acdl/.github/workflows/deploy.yml@v1.25` with + `with: { contract: contract.yaml, mode: full, environment: dev }`. +- **Task 2.2** (blockchain-engineer): `.gitea/workflows/deploy.yml` — + byte-identical mirror. +- **Task 2.3** (blockchain-engineer): `tests/test_deploy_workflow_invocation.py` + — asserts the `uses:` ref + inputs. -**Files touched:** -- `adapters/kyverno-json/policies/regression/cap-013-adapter-dedup.json` (NEW) -- `adapters/kyverno-json/policies/regression/cap-023-metrics-collector.json` (NEW) -- `adapters/kyverno-json/policies/regression/cap-024-deck-structure.json` (NEW) -- `tests/test_regression_policies.py` (NEW) -- `tests/fixtures/capability_inventory/clean.json` (NEW) -- `tests/fixtures/capability_inventory/drifted.json` (NEW) -- `adapters/README.md` (MODIFIED — new row + PolicyEngine Protocol section) -- `adapters/kyverno-json/README.md` (NEW) -- `schemas/README.md` (MODIFIED — engine enum note) -- `modules/STANDARDS.md` (MODIFIED — Policy authoring standard section) -- `docs/METRICS.md` (MODIFIED — swappable engine narrative) +### Wave 3 — platform floating tag (cross-cutting) +- **Task 3.1** (lead-developer, on `acdl` repo): verify the `v1.25` + floating tag exists (created by `release.yml` on merge to main). If + not, create it pointing at the `v1.25.0` tag (Phase 0 ship). -**Verification:** `pytest tests/test_regression_policies.py -tests/test_kyverno_json_engine.py -v` (new tests pass or skip-without- -kj). Full regression gate `pytest tests/` still at 287/287 baseline + -new tests (skip without kj). Manual read of `adapters/README.md` + -`adapters/kyverno-json/README.md` + `modules/STANDARDS.md` policy -section for clarity. +**Must-haves (verify before ship):** +- `contract.yaml` validates against `schemas/contract.schema.json`. +- The deploy workflow invocation asserts the correct `uses:` ref + + inputs. +- The `v1.25` floating tag resolves. + +**Ship:** tag `v1.25.2`, merge `phase/02` → milestone, Gitea release. +Delete `phase/02`. --- -### Phase P5 — final review + audit + milestone ship (Final Phase) +## Phase 3 — pilot-metrics-and-policies (tag v1.25.3) -**Type:** `docs` (review + audit + milestone completion) +**Goal:** The platform repo gains the metric-grounding emitters, the +kyverno-json pilot policies, the DynamoDB L1 primitive, the env-JSON +wiring reconciliation, + the pilot regression CAP. The Post-Pilot +metrics are grounded (outcome backfill + escalation reason); the pilot- +readiness + settlement-finality policies are in place. -**Requirements:** All REQ-291..309 (mark complete) +**Project:** `acdl` (platform repo). +**Branch:** `acdl/phase/03-pilot-metrics-and-policies` (platform branch). +**Personas:** backend-engineer (emitters + adapter + regression), +data-engineer (DynamoDB primitive + env JSON + collector), +policy-engineer (kyverno-json policies). -**Must-haves:** -- `ciagent-review` multi-persona code review across P1..P4 - (lead-developer, backend-engineer, data-engineer, policy-engineer). - Auto-fix P0; flag P1+ for post-hoc review. If P1+ issues found, fix - them in this final phase (not loop back to EXECUTE). -- `ciagent-audit` — reconstruction test (git log ↔ `.ciagent/` files), - `.ciagent/` file discipline, branch hygiene, commit discipline. - Critical issues fixed in this phase. -- `ciagent-ship` (milestone) — merge `phase/05-final-review-ship` → - `milestone/v1.25-kyverno-json` → `main`; tag `v1.24.5` (= the v1.25 - release per the prev-minor tagging rule); create Gitea release with - full milestone summary (all phases, all requirements); delete all - milestone branches (local + remote). -- Update `REQUIREMENTS.md` (mark REQ-291..309 complete), - `ROADMAP.md` (mark v1.25 complete), `CHECKPOINT.json` - (milestone_complete: true), `NORTH_STAR.md` (note Strategic - Objective #2 — provable trust via a replaceable policy-engine - substrate). +### Wave 1 — DynamoDB primitive (REQ-322) — data-engineer +- **Task 1.1** (data-engineer): `modules/l1/dynamodb/interface.json` — + stack type `aws:dynamodb:table`, inputs (table_name, region, pk, sk, + billing_mode), outputs (table_arn, table_name). +- **Task 1.2** (data-engineer): `modules/l1/dynamodb/terraform/main.tf` + — `resource "aws_dynamodb_table" "this"` (PK + optional SK, + `PAY_PER_REQUEST` default, encryption + PITR enabled per v1.8 NFR). +- **Task 1.3** (data-engineer): `modules/l1/dynamodb/README.md` + + `instance.json`. +- **Task 1.4** (data-engineer): `modules/registry.json` — `dynamodb` + entry (kind `l1`, `terraform_dir`). +- **Task 1.5** (data-engineer): `modules/README.md` — catalog index. -**Vertical slice:** The v1.25 milestone is complete: kyverno-json is -the primary policy tool, behind a swappable adapter, with policies -over all 4 Nova artifacts. Tags v1.24.0..v1.24.5 on the v1.24.x line. -The milestone branch merges to main. +### Wave 2 — metric grounding (REQ-317, REQ-318) — backend-engineer + data-engineer — parallel +- **Task 2.1** (backend-engineer): `core/metrics/outcome_backfill.py` — + `backfill(decision_id, outcome)` updates `fact_decision.outcome` + + `backfilled_at`. Reads run-manifest events. +- **Task 2.2** (backend-engineer): `core/metrics/collector.py` — + invokes backfill after run completion. +- **Task 2.3** (backend-engineer): `tests/test_outcome_backfill.py`. +- **Task 2.4** (backend-engineer): `core/confidence_signal.py` — + `ai.decision.made` gains `escalation_reason: 'confidence'` when + `band == 'block'`. +- **Task 2.5** (backend-engineer): `core/metrics/collector.py` — + persists `escalation_reason` into `fact_run`. +- **Task 2.6** (backend-engineer): `tests/test_confidence_escalation_reason.py`. -**Verification:** `pytest tests/ -v` full suite passes (287 baseline + -new tests). `git log --oneline` shows the v1.25 phase commits. -`git tag` shows v1.24.0..v1.24.5. `git branch` shows no leftover -milestone/phase branches (all deleted post-ship). +### Wave 3 — env-JSON wiring + adapter (REQ-319) — backend-engineer + data-engineer — parallel +- **Task 3.1** (backend-engineer): `adapters/terraform/adapter.py` — + reads `env.state_backend.bucket` when present (fallback to computed + name for backwards compat). +- **Task 3.2** (data-engineer): `core/environments/dev.json` — + `account_id` → `581513795199`, `state_backend.bucket` → + `nova-tfstate-581513795199-us-east-1`. +- **Task 3.3** (data-engineer): `core/environments/{qa,prod,dr}.json` — + `state_backend.bucket` updated; `account_id` stays placeholder + (pilot-readiness policy blocks apply on placeholder, D-208). +- **Task 3.4** (backend-engineer): `tests/test_adapter_state_backend.py`. +- **Task 3.5** (backend-engineer): `tests/test_adapter.py` — add + `dynamodb` to `EXPECTED_L1_KEYS` + a resolution + emission test + (cross-territory: data-engineer authored the module, backend-engineer + owns the test). + +### Wave 4 — kyverno-json policies (REQ-315, REQ-320) — policy-engineer — parallel +- **Task 4.1** (policy-engineer): + `adapters/kyverno-json/policies/settlement-finality/all-matches-committed.json` + — kyverno-json policy over settlement-service status JSON (asserts + `all_committed: true`). +- **Task 4.2** (policy-engineer): + `adapters/kyverno-json/policies/pilot-readiness/no-placeholder-account.json` + — kyverno-json policy over env JSON (asserts + `account_id != "000000000000"`). +- **Task 4.3** (policy-engineer): `tests/test_settlement_finality_policy.py` + — passing + failing fixtures; skip when `kj` absent. +- **Task 4.4** (policy-engineer): `tests/test_pilot_readiness_policy.py` + — passing (real account) + failing (placeholder) fixtures; skip when + `kj` absent. + +### Wave 5 — regression CAP (REQ-316) — backend-engineer +- **Task 5.1** (backend-engineer): `core/regression_verify.py` — + CAP-025 (live-pilot-apply): the round-trip assertion. +- **Task 5.2** (backend-engineer): `tests/test_regression_pilot.py`. + +**Must-haves (verify before ship):** +- `pytest tests/` in the platform repo passes (170 existing + new tests). +- The DynamoDB primitive resolves + emits valid Terraform. +- The outcome backfill updates `fact_decision.outcome` (not `pending`). +- The `escalation_reason` field is emitted on `block` band. +- The adapter reads `env.state_backend.bucket` from the env JSON. +- The 2 new kyverno-json policies pass on valid fixtures + fail on + invalid fixtures (skip when `kj` absent). +- CAP-025 is in the regression gate. +- No existing tests regress (170 baseline holds). + +**Ship:** tag `v1.25.3`, merge `phase/03` → milestone, Gitea release. +Delete `phase/03`. --- -## Wave ordering (parallelization) +## Phase 4 — pilot-run-and-docs (tag v1.25.4) -With `parallelization.enabled: true`, `max_concurrent_agents: 5`, -`min_plans_for_parallel: 2`: +**Goal:** The pilot estate runs end-to-end against live AWS +`581513795199` (contract resolve → adapter compile → terraform plan → +policy scan → confidence signal → attestation → outbox record). Docs + +adapter README + onboarding guide are complete. -- **P1 Wave 1:** backend-engineer (protocol + registry + install) ‖ - data-engineer (config.json policy object) ‖ policy-engineer (engine - adapter + smoke policy). 3 concurrent personas. Merge in order: - data-engineer → backend-engineer → policy-engineer. -- **P2 Wave 2:** policy-engineer (contract + stack-IR policies) ‖ - backend-engineer (resolver wiring — depends on P1 registry). 2 - concurrent. Merge: policy-engineer → backend-engineer (wiring - references the policy dirs). -- **P3 Wave 3:** policy-engineer (plan-JSON + meta policies) ‖ - backend-engineer (run_platform.sh wiring — depends on P1 engine + - P2 resolver pattern). 2 concurrent. Merge: policy-engineer → - backend-engineer. -- **P4 Wave 4:** policy-engineer (regression policies) ‖ data-engineer - (capability-inventory fixtures) ‖ lead-developer (docs: READMEs, - STANDARDS, METRICS). 3 concurrent. Merge: data-engineer → - policy-engineer → lead-developer. +**Project:** `nova-blockchain-exchange` (consumer repo — the run) + +`acdl` (platform repo — docs). +**Branch:** `acdl/phase/04-pilot-run-and-docs` (platform branch for +docs); the run happens via the consumer's `deploy.yml` invocation. +**Personas:** blockchain-engineer (the run), lead-developer (docs), +backend-engineer (regression CAP-025 verification). -Territory enforcement: `warn` mode (per `config.json -personas.territory_enforcement: "warn"`). Cross-territory edits -(e.g., backend-engineer touching a policy file) emit a warning, not a -block. +### Wave 1 — the pilot run (REQ-316 verification, live) +- **Task 1.1** (blockchain-engineer): trigger the consumer's + `deploy.yml` with `mode: full, environment: dev` against + `581513795199`. The workflow checks out the consumer + platform + repos, runs `run_platform.sh`, applies the contract (ECS + + DynamoDB + S3), records the decision + attestation. +- **Task 1.2** (backend-engineer): verify CAP-025 (regression gate) + passes against the live run. +- **Task 1.3** (blockchain-engineer): capture the run's + `ai.decision.made` + `attestation.recorded` events from the Decision + Ledger → evidence for the milestone ship. -## Requirement → phase → persona matrix +### Wave 2 — docs (REQ-321) +- **Task 2.1** (lead-developer): `adapters/README.md` — new consumer + row + fix the stale `TYPE_MAP` references (IDEATE I8). +- **Task 2.2** (lead-developer): `docs/METRICS.md` — Post-Pilot metrics + grounded note (the 3 targets now have non-zero denominators post-run). +- **Task 2.3** (lead-developer): `.ciagent/ARCHITECTURE.md` §12.8 + (Pilot Estate). +- **Task 2.4** (lead-developer): + `.ciagent/nova-blockchain-exchange/README.md` — consumer onboarding + guide (how to invoke `deploy.yml@v1.25`, what secrets to set, what + the contract shape is). -| REQ | Phase | Primary persona | Type | -|-----|-------|-----------------|------| -| REQ-291 | P1 | backend-engineer | feat | -| REQ-292 | P1 | data-engineer | feat (config) | -| REQ-293 | P1 | policy-engineer | feat | -| REQ-294 | P1 | backend-engineer | feat (install) | -| REQ-295 | P2 | policy-engineer | feat | -| REQ-296 | P2 | backend-engineer | feat (wiring) | -| REQ-297 | P2 | policy-engineer | feat | -| REQ-298 | P2 | backend-engineer | feat (wiring) | -| REQ-299 | P2 | policy-engineer | test | -| REQ-300 | P3 | policy-engineer | feat | -| REQ-301 | P3 | backend-engineer | feat (pipeline) | -| REQ-302 | P3 | policy-engineer + backend-engineer | test | -| REQ-303 | P3 | policy-engineer | feat (meta) | -| REQ-304 | P4 | policy-engineer | feat | -| REQ-305 | P4 | policy-engineer + data-engineer | test | -| REQ-306 | P4 | policy-engineer + lead-developer | docs | -| REQ-307 | P4 | lead-developer | docs | -| REQ-308 | P1 | backend-engineer | test | -| REQ-309 | P1 | policy-engineer | test | \ No newline at end of file +**Must-haves (verify before ship):** +- The pilot run completes end-to-end (apply succeeds, decision recorded, + attestation recorded for dev — autonomous, no human approver). +- CAP-025 passes. +- The 3 Post-Pilot metrics have non-zero denominators (the run + contributed to `fact_run` + `fact_decision`). +- Docs are complete (adapter README, METRICS.md, ARCHITECTURE.md §12.8, + consumer onboarding guide). + +**Ship:** tag `v1.25.4`, merge `phase/04` → milestone, Gitea release. +Delete `phase/04`. + +--- + +## Phase 5 — final review + audit + milestone ship (tag v1.25.5) + +**Goal:** Multi-persona code review across P1..P4. Audit (reconstruction +test, branch hygiene, commit discipline). Milestone ship: merge to main, +tag `v1.25.5` (= the v1.26 release), Gitea release with full milestone +summary, delete all milestone branches. + +**Project:** both (`acdl` + `nova-blockchain-exchange`). +**Branch:** `phase/05-final-review-ship`. +**Personas:** lead-developer (review + audit + ship), backend-engineer +(review), data-engineer (review), policy-engineer (review), +blockchain-engineer (review — the chain core is reviewed). + +### Wave 1 — review +- **Task 1.1** (lead-developer): `ciagent-review` — multi-persona code + review across P1..P4. Auto-fix P0; flag P1+ for post-hoc review. +- **Task 1.2** (all personas): fix P0 issues in this phase. + +### Wave 2 — audit +- **Task 2.1** (lead-developer): `ciagent-audit` — reconstruction test + (git log ↔ `.ciagent/`), branch hygiene, commit discipline. +- **Task 2.2** (lead-developer): fix critical audit issues in this phase. + +### Wave 3 — milestone ship +- **Task 3.1** (lead-developer): merge `phase/05` → + `milestone/v1.26-pilot-activation` → `main`. +- **Task 3.2** (lead-developer): tag `v1.25.5` (= the v1.26 release per + prev-minor tagging rule). +- **Task 3.3** (lead-developer): create Gitea release with full milestone + summary (all phases, all 13 requirements). +- **Task 3.4** (lead-developer): delete all milestone branches (local + + remote). Tags preserve all history. +- **Task 3.5** (lead-developer): update `.ciagent/nova-blockchain-exchange/REQUIREMENTS.md` + (mark REQ-310..322 complete), `.ciagent/ROADMAP.md` (mark v1.26 + complete), `.ciagent/NORTH_STAR.md` (note Strategic Objectives #1 + + #3 — first real consumer estate; Post-Pilot denominators activated). +- **Task 3.6** (lead-developer): write checkpoint `stage: complete, + phase: 5, phase_role: final` + clear checkpoint (milestone complete). + +**Must-haves (verify before ship):** +- Review: 0 P0 issues unfixed; P1+ flagged for post-hoc. +- Audit: reconstruction test passes; branch hygiene clean; commit + discipline clean. +- Ship: `v1.25.5` tag exists; Gitea release created; milestone branches + deleted; main has the milestone merge. + +--- + +## Requirement → Phase Mapping + +| REQ | Phase | Wave | Persona | +|---|---|---|---| +| REQ-310 (blockchain core) | P1 | W1 | blockchain-engineer | +| REQ-311 (order engine) | P1 | W2 | blockchain-engineer | +| REQ-312 (settlement) | P1 | W2 | blockchain-engineer | +| REQ-313 (contract.yaml) | P2 | W1 | blockchain-engineer | +| REQ-314 (deploy invocation) | P2 | W2 | blockchain-engineer | +| REQ-315 (settlement-finality policy) | P3 | W4 | policy-engineer | +| REQ-316 (pilot regression CAP) | P3 | W5 + P4 W1 | backend-engineer | +| REQ-317 (outcome backfill) | P3 | W2 | backend-engineer | +| REQ-318 (escalation reason) | P3 | W2 | backend-engineer | +| REQ-319 (env-JSON wiring) | P3 | W3 | backend + data-engineer | +| REQ-320 (pilot-readiness policy) | P3 | W4 | policy-engineer | +| REQ-321 (docs) | P4 | W2 | lead-developer | +| REQ-322 (DynamoDB primitive) | P3 | W1 | data-engineer | + +--- + +## Wave Ordering Rationale + +- **P1 W1 → W2:** the chain core (block + ledger + validator) must land + before the order engine + settlement (they submit transactions to the + ledger). W3 (CI) is cross-cutting + can land any time after W1. +- **P2 W1 → W2:** the contract must land before the deploy invocation + (the invocation references the contract). W3 (floating tag) is cross- + cutting. +- **P3 W1 (DynamoDB) first:** the contract (P2) references `dynamodb` — + the primitive must exist before P2's contract can resolve. **Risk:** + P2's contract references a module that doesn't exist until P3. Resolution: P2's contract is authored but the `test_contract_validates.py` test only checks schema validity (not registry resolution) — the registry resolution test is in P3 (after the primitive lands). The contract's `dynamodb` block is schema-valid (the schema is open); the registry resolution happens at apply time (P4). + - **Alternative:** move REQ-322 to P2 W0 (before the contract). This + avoids the P2→P3 dependency. **Decision: move REQ-322 to P2 W0.** + See revised mapping below. + +### Revised: REQ-322 → P2 W0 + +REQ-322 (DynamoDB primitive) lands in P2 Wave 0 (before the contract) +so the contract's `dynamodb` block resolves at registry time, not just +schema time. This makes P2 self-contained: the primitive + the contract ++ the deploy invocation all land in P2. + +| REQ | Phase | Wave | Persona | +|---|---|---|---| +| REQ-310 (blockchain core) | P1 | W1 | blockchain-engineer | +| REQ-311 (order engine) | P1 | W2 | blockchain-engineer | +| REQ-312 (settlement) | P1 | W2 | blockchain-engineer | +| REQ-322 (DynamoDB primitive) | P2 | W0 | data-engineer | +| REQ-313 (contract.yaml) | P2 | W1 | blockchain-engineer | +| REQ-314 (deploy invocation) | P2 | W2 | blockchain-engineer | +| REQ-315 (settlement-finality policy) | P3 | W4 | policy-engineer | +| REQ-316 (pilot regression CAP) | P3 | W5 + P4 W1 | backend-engineer | +| REQ-317 (outcome backfill) | P3 | W2 | backend-engineer | +| REQ-318 (escalation reason) | P3 | W2 | backend-engineer | +| REQ-319 (env-JSON wiring) | P3 | W3 | backend + data-engineer | +| REQ-320 (pilot-readiness policy) | P3 | W4 | policy-engineer | +| REQ-321 (docs) | P4 | W2 | lead-developer | + +This revision is a binding plan decision (G-Q8 in the grill may +challenge it). \ No newline at end of file diff --git a/.ciagent/ROADMAP.md b/.ciagent/ROADMAP.md index d96d98e..83ed46f 100644 --- a/.ciagent/ROADMAP.md +++ b/.ciagent/ROADMAP.md @@ -2282,6 +2282,11 @@ Tags: `v1.25.0` (P0) → `v1.25.5` (P5 = milestone release). idempotent, finality = block commit. ### Phase P2 — consumer-contract-and-deploy (planned, tag v1.25.2) +- REQ-322: `modules/l1/dynamodb/` — new L1 primitive (interface.json + + terraform/main.tf + README.md + instance.json + registry.json entry). + The single platform-side module build-out (ECS + S3 already exist; + the adapter is stateless/registry-driven). Lands in P2 W0 (before the + contract) so the contract's `dynamodb` block resolves at registry time. - REQ-313: `nova-blockchain-exchange/contract.yaml` + per-env variants (dev/qa/prod) — validated against `schemas/contract.schema.json`. - REQ-314: `nova-blockchain-exchange/.github/workflows/deploy.yml` + @@ -2308,10 +2313,6 @@ Tags: `v1.25.0` (P0) → `v1.25.5` (P5 = milestone release). `nova-tfstate-581513795199-us-east-1`. - REQ-320: `adapters/kyverno-json/policies/pilot-readiness/no-placeholder-account.json` — declarative gate preventing apply against a placeholder account. -- REQ-322: `modules/l1/dynamodb/` — new L1 primitive (interface.json + - terraform/main.tf + README.md + instance.json + registry.json entry). - The single platform-side module build-out (ECS + S3 already exist; - the adapter is stateless/registry-driven). ### Phase P4 — pilot-run-and-docs (planned, tag v1.25.4) - REQ-321: `adapters/README.md` (new consumer row) +