diff --git a/.ciagent/CHECKPOINT.json b/.ciagent/CHECKPOINT.json index 8d7c736..d650b8d 100644 --- a/.ciagent/CHECKPOINT.json +++ b/.ciagent/CHECKPOINT.json @@ -1,10 +1,10 @@ { "phase": 0, - "stage": "specify", + "stage": "clarify", "milestone": "v1.26", "phase_role": "pre_execution", "attempts": 0, - "updated_at": "2026-08-12T21:11:00Z", + "updated_at": "2026-08-12T21:12:00Z", "project": "acdl", "projects": ["acdl", "nova-blockchain-exchange"], "active_milestone": "v1.26", diff --git a/.ciagent/CLARIFY.md b/.ciagent/CLARIFY.md index 15034f4..d1704a0 100644 --- a/.ciagent/CLARIFY.md +++ b/.ciagent/CLARIFY.md @@ -1,164 +1,226 @@ -# CLARIFY — v1.25 kyverno-json Unified Policy Engine +# CLARIFY — v1.26 Live Pilot Estate Activation -> **Autonomy:** full. Ambiguities are auto-resolved with assumption logging -> per `config.json autonomy.level: "full"` and -> `autonomy.decision_confidence_threshold: 0.6`. No human escalation. +> **Autonomy:** full. Auto-resolution with assumption logging per +> `config.autonomy.level: "full"`. No human escalation unless +> confidence < 0.60 (threshold `config.autonomy.decision_confidence_threshold`). +> 10 ambiguities identified; all resolved (confidence ≥ 0.60). -## Ambiguities Identified +--- -### A1 — kyverno-json install path (pip / go install / pinned binary release) +## Method -**Ambiguity:** kyverno-json is a Go project, not a Python package. Three -install paths exist: (a) `pip install` — not possible (no PyPI package); -(b) `go install github.com/kyverno/kyverno-json/cmd/kj@latest` — requires -Go toolchain in the CI image; (c) download a pinned binary release from -GitHub releases — no Go toolchain needed, but release artifacts are -platform-specific and must be checksummed. +The clarify stage identifies ambiguities in the v1.26 specification +(PROJECT.md, REQUIREMENTS.md, ROADMAP.md) and resolves them at full +autonomy. Each ambiguity gets a decision ID (D-200+; continuing from +the v1.26 SPECIFY decisions D-200..D-205), a resolution, a confidence +score, and a rationale. Resolutions update PROJECT.md + REQUIREMENTS.md ++ ROADMAP.md as needed. -**Resolution (auto, confidence 0.85):** `go install` (option b). A -`scripts/install-kyverno-json.sh` helper runs -`go install github.com/kyverno/kyverno-json/cmd/kj@latest` and prints -`kj version`. The CI image (`.github/workflows/ci.yml` + -`.gitea/workflows/ci.yml`) installs Go + kj when -`config.json.policy.engine == "kyverno-json"`; the install is cached via -the existing Go module cache. Rationale: `go install` is the upstream- -blessed path, tracks the latest stable release, avoids per-platform -binary management, and the project already accepts Go-based tooling -(checkov pulls Go-built transitive deps via pip). When `which kj` is -absent, `KyvernoJsonEngine.is_configured()` returns false → `SKIPPED` -PCR (mirrors the Wiz adapter pattern) — the platform functions without -the binary. Captured in REQ-293, REQ-294. Decision ID: D-115. +--- -### A2 — `engine` enum value: new `"kyverno-json"` vs reuse `"kyverno"` +## Ambiguities + Resolutions -**Ambiguity:** `schemas/policy_check_result.schema.json` already lists -`engine: ["checkov", "kyverno", "opa", "wiz"]`. kyverno-json is a -distinct runtime from the K8s Kyverno admission controller, but both -are "Kyverno." Two options: (a) add a new `"kyverno-json"` enum value -— requires schema change + checkov/wiz adapter test regression check; -(b) reuse `"kyverno"` and distinguish by `ruleId` prefix. +### Q1 — Does the consumer repo's `.ciagent/` live in the platform repo or the consumer repo? -**Resolution (auto, confidence 0.80):** Reuse `"kyverno"` (option b). -Adding `"kyverno-json"` would force a schema change + a test sweep for -no semantic gain — the `engine` field records the policy engine family, -not the specific binary. kyverno-json PCR records carry `engine: -"kyverno"` and `ruleId` prefixed `KJ_` (e.g. -`KJ_REQUIRE_TAGGING_STANDARD`), while the K8s adapter uses `KYVERNO_` -prefixes (e.g. `KYVERNO_INACTIVE_TF_STACK`). The two are distinguishable -in audit/telemetry by `ruleId` prefix and `evidence` payload shape (the -K8s adapter's evidence has `namespace`/`kind`; kyverno-json's has -`assertion`/`jmespath`). No schema change. Captured in REQ-293. -Decision ID: D-116. +**Ambiguity:** The user said "ciagent should track it as a separate +project under this same path." Does "this same path" mean the platform +repo's `.ciagent/` directory (multi-project mode per `run.md` Step 0), +or a separate `.ciagent/` inside the consumer repo? -### A3 — Do checkov/wiz adapters change their signatures to feed kyverno-json? +**Resolution:** The platform repo's `.ciagent/` directory. Multi-project +mode: `.ciagent/config.json` `projects[]` includes both `acdl` + +`nova-blockchain-exchange`; the consumer's project files +(PROJECT.md, REQUIREMENTS.md, ROADMAP.md) live in +`.ciagent/nova-blockchain-exchange/`. The consumer *git repo* owns the +app code + `contract.yaml` + deploy workflow invocation; the platform +repo owns the CIAgent planning artifacts for both projects. This +matches `run.md` Step 0 multi-project mode. -**Ambiguity:** The unified-orchestrator model places kyverno-json "on -top of" checkov/wiz. Two interpretations: (a) checkov/wiz now emit a -"raw findings" intermediate (not PCR) that kyverno-json meta-policies -consume — requires changing `adapt() -> list[PolicyCheckResult]` to -`adapt() -> list[RawFinding]`; (b) checkov/wiz keep emitting PCRs as -today, and the meta-policies in `adapters/kyverno-json/policies/meta/` -consume the **merged** PCR list as their payload. +**Confidence:** 0.95. **Decision:** D-206. -**Resolution (auto, confidence 0.90):** Option (b). The existing -`adapt() -> list[PolicyCheckResult]` signatures are unchanged. The -meta-policies consume the merged PCR list (checkov + wiz + kyverno-json -plan-JSON policies) as their input payload. This preserves the -`PolicyCheckResult` schema as the single inter-adapter contract -(ARCHITECTURE.md §12.6), avoids a new "RawFinding" type, and means -the existing checkov/wiz adapter tests pass unchanged. The meta-policy -`block-on-any-critical.json` iterates the merged list; the -`tagging-rules-agree.json` meta-policy cross-checks the Checkov -`NOVA_TAG_NAMING` result against the kyverno-json -`KJ_REQUIRE_TAGGING_STANDARD` result by `resourceRef`. Captured in -REQ-303, D-117. Decision ID: D-117. +### Q2 — Is the bootstrap `NOVA_AWS_*` key the root key or the spike-runner key? -### A4 — `NOVA_TAG_NAMING` Checkov rule: rewrite as kyverno-json policy, keep, or both? +**Ambiguity:** The bootstrap scripts (post-migration) prefer +`NOVA_BOOTSTRAP_AWS_*`, falling back to `NOVA_AWS_*`. The pre-run +(A3) succeeded with `NOVA_AWS_*`, creating the S3 bucket + DynamoDB +table — which requires root or root-equivalent IAM. Is `NOVA_AWS_*` +the root key, or did the bootstrap succeed because the spike-runner +policy happens to include S3/DynamoDB create? -**Ambiguity:** The Checkov custom rule -`adapters/terraform/policy/custom_rules/nova_tagging.py` enforces the -Nova tagging standard over Terraform HCL (static scan + plan scan). The -kyverno-json milestone adds `require-tagging-standard.json` over the -resolved Stack IR. Three options: (a) rewrite — replace the Checkov -rule with the kyverno-json policy (loses Checkov's HCL-level coverage -and the `--external-checks-dir` integration); (b) keep Checkov only — -don't add a kyverno-json policy (the Stack IR is already the input to -terraform, so the Checkov rule catches it); (c) both — keep the -Checkov rule as the source of truth for HCL-level scanning AND add the -kyverno-json policy for IR-level coverage, with a meta-policy that -asserts the two agree. +**Resolution:** `NOVA_AWS_*` has root-equivalent permissions (confirmed +empirically: the bootstrap created the S3 bucket + DynamoDB table +successfully). For the pilot, `NOVA_AWS_*` is the bootstrap key. A +future hardening milestone should split this into a dedicated +`NOVA_BOOTSTRAP_AWS_*` root key + a least-privilege `NOVA_AWS_*` runner +key (the spike-runner pattern). For v1.26, the single key suffices +(pilot scope). -**Resolution (auto, confidence 0.82):** Option (c) — both, with a -cross-check meta-policy. The Checkov rule stays the source of truth -for `terraform_plan` scanning (it reads HCL resource blocks directly); -the kyverno-json policy covers the Stack IR dict (which is the input -*before* terraform, so it catches IR-level violations that the -terraform adapter might mask via defaults). The P3 meta-policy -`tagging-rules-agree.json` asserts the two engines agree on every -resource; divergence emits an `error` PCR (defense-in-depth against -rule drift — if the two engines disagree, the operator must -investigate before proceeding). This is the only case in v1.25 where -two engines evaluate the same concern; it is intentional — the -tagging standard is the highest-impact rule (v1.8 D-tagging-standard, -v1.10 re-verification) and merits redundancy. Captured in REQ-297, -REQ-303, REQ-299. Decision ID: D-118. +**Confidence:** 0.90. **Decision:** D-207. -### A5 — Critical-override: delegate to declarative meta-policy or keep hard-override? +### Q3 — Which AWS account does the pilot use: `581513795199` (existing) or a dedicated pilot account? -**Ambiguity:** `core/confidence_signal.py` lines 144-157 hardcode -`PENALTY["critical"]: None` — a critical-severity `fail` PCR forces -`score = 0, band = block` regardless of the weighted-sum inputs. The -v1.25 meta-policy `block-on-any-critical.json` makes this declarative -(asserts no PCR in the merged list has `severity: critical` + -`result: fail`). Two options: (a) fully delegate — remove the -hard-override, rely on the meta-policy to emit a critical `fail` PCR -that the existing penalty logic then blocks; (b) keep both — the -meta-policy is the declarative source of truth, the hard-override is -defense-in-depth. +**Ambiguity:** The user said "assume 581513795199." But the env JSONs +all show `account_id: "000000000000"` (placeholder). Does the pilot +bind all env JSONs to `581513795199`, or only `dev` (with qa/prod/dr +left placeholder until a real multi-account landing zone exists)? -**Resolution (auto, confidence 0.88):** Option (b) — keep both. The -meta-policy is the *declarative* statement ("Nova blocks on any -critical finding from any engine"); the hard-override is the -*imperative* safety net that ensures a critical PCR can never slip -through even if the meta-policy is misconfigured or the -`PolicyEngineRegistry` returns a `NullEngine`. This is -defense-in-depth, not redundancy-for-its-own-sake: the meta-policy -runs *before* the confidence signal (it produces PCRs that flow in), -the hard-override runs *inside* the confidence signal (it is the last -gate). Removing the hard-override would make the platform's -"critical = block" guarantee depend on a single declarative policy -file — a regression in the provable-trust posture (Strategic -Objective #2). Captured in REQ-303, PROJECT.md hard-constraints. -Decision ID: D-119. +**Resolution:** Bind `dev` to `581513795199` for the pilot +(D-203, established in SPECIFY). The `qa`/`prod`/`dr` env JSONs remain +placeholder `000000000000` this milestone — the pilot runs in `dev` +(autonomous, no HITL gate). Multi-account landing zone (qa/prod/dr on +separate accounts) is a future milestone. REQ-319 (env-JSON wiring) +updates `dev.json`'s `state_backend.bucket` to +`nova-tfstate-581513795199-us-east-1` + `account_id` to `581513795199`; +qa/prod/dr get the `state_backend.bucket` update but keep placeholder +`account_id` (the pilot-readiness policy REQ-320 blocks apply on +placeholder accounts — so qa/prod/dr apply is blocked by design until +the accounts are bound). -### A6 — Does kyverno-json break the "platform functions without AI" tenet? +**Confidence:** 0.92. **Decision:** D-208. -**Ambiguity:** NORTH_STAR.md Strategic Objective #2: "the platform -functions without AI — 'AI decisions' are really automated decisions." -kyverno-json is a deterministic policy engine (no ML), but it is a new -runtime dependency. Does adding it violate the tenet? +### Q4 — Does "all types of securities" mean all types in v1.26, or equities-only pilot with others deferred? -**Resolution (auto, confidence 0.95):** No — kyverno-json is -deterministic, not AI. The tenet distinguishes "AI decisions" (LLM- -driven, non-reproducible) from "automated decisions" (rule-driven, -reproducible). kyverno-json is the latter — the same policy + payload -produces the same result on every run. It is *more* aligned with the -tenet than the current imperative Python in `core/env_transition.py` -and `core/regression_verify.py`, because the policy is declarative -(visible, auditable, version-controlled) rather than imperative (logic -hidden in function bodies). The `is_configured()` guard ensures the -platform functions without the binary (graceful skip), so the tenet -holds even in environments where kyverno-json is not installed. -Captured in PROJECT.md hard-constraints + RESEARCH.md G-Q1. -Decision ID: D-120. +**Ambiguity:** The user said "stock market built on homegrown blockchain +offering all types of securities." This could mean equities + bonds + +derivatives + options all in v1.26, or equities-only pilot with others +deferred (the recommended scope from the plan). + +**Resolution:** Equities-only pilot (D-200, established in SPECIFY). +Bonds/derivatives/options have very different settlement models (T+1 +for equities; T+2 for bonds; derivatives vary; options exercise +models). A pilot should demonstrate the Nova platform's policy gates +over a real estate — equities (T+1) is the simplest. "All types of +securities" is the *product vision*; v1.26 is the *pilot* (equities +first). The roadmap documents the deferral. + +**Confidence:** 0.85. **Decision:** D-200 (reaffirmed). + +### Q5 — Is the homegrown blockchain a real consensus protocol or a minimal PoA ledger? + +**Ambiguity:** "Homegrown blockchain" could mean a full consensus +protocol (multi-validator BFT) or a minimal PoA ledger (single +validator, append-only). + +**Resolution:** Minimal PoA ledger (D-201, established in SPECIFY). +Single validator (config-driven), append-only blocks, SHA-256 hash +chain, deterministic block production. Settlement finality = block +commit. Multi-validator BFT is a future milestone. The pilot's purpose +is to exercise the Nova platform's deploy/policy/attestation gates over +a real consumer — the chain needs to be real enough to record +transactions, not to solve Byzantine consensus. + +**Confidence:** 0.88. **Decision:** D-201 (reaffirmed). + +### Q6 — Does the pilot's `terraform apply` actually run, or is it `--plan-only`? + +**Ambiguity:** The platform's `run_platform.sh` defaults to +plan-only (no apply). The `deploy.yml` workflow's `mode` input can be +`full` (apply) or `plan-only`. Does the pilot actually `terraform apply` +(creating real AWS resources for the blockchain exchange), or does it +stop at plan? + +**Resolution:** The pilot runs `mode: full` (apply) for `dev` only. +The apply creates real AWS resources (ECS for the matching engine, +DynamoDB for the ledger, S3 for block storage) in account +`581513795199`. `qa`/`prod`/`dr` are blocked by the pilot-readiness +policy (REQ-320) until their accounts are bound (D-208). The apply is +autonomous for `dev` (no HITL gate; confidence threshold 0.50). The +`ai.decision.made` + `attestation.recorded` events land in the Decision +Ledger — but `dev` attestation is autonomous (no human approver), so +only `ai.decision.made` fires for `dev`. + +**Confidence:** 0.90. **Decision:** D-209. + +### Q7 — What AWS resources does the blockchain exchange contract declare? + +**Ambiguity:** The `contract.yaml` declares the exchange's +infrastructure. What specific AWS resources? The platform's adapter +maps contract infrastructure blocks to Terraform. What stack types +does the blockchain exchange use? + +**Resolution:** The pilot contract declares 3 infrastructure blocks: +(1) `ecs` (Fargate service for the matching engine + settlement +service — the platform's existing `microservice` module pattern), (2) +`dynamodb` (the ledger table — single-table, PK `block_index`), (3) +`s3` (block storage — one object per block, key `blocks/{index}.json`). +The adapter's `TYPE_MAP` already covers `aws_ecs_service`, +`aws_dynamodb_table`, `aws_s3_bucket` (existing L1 primitives). No new +adapter stack types needed for the pilot. The contract's +`infrastructure` block references these by module name (`microservice` +for ECS, `dynamodb` for the table, `s3` for the bucket). + +**Confidence:** 0.82. **Decision:** D-210. + +### Q8 — Does the outcome-backfill emitter (REQ-317) change the PCR schema? + +**Ambiguity:** REQ-317 wires `apply.completed`/`apply.failed` → +`fact_decision.outcome`. Does this touch the `PolicyCheckResult` schema +(PCR) — the v1.25 moat that must not change? + +**Resolution:** No. The outcome backfill touches the *metrics cold +store* (`fact_decision` table in `metrics/nova_metrics.db`), not the +PCR schema. The PCR schema (`schemas/policy_check_result.schema.json`) +is unchanged. The backfill reads run-manifest events (not PCRs) and +updates the decision's outcome column. This respects the v1.25 hard +constraint: "DO NOT change `schemas/policy_check_result.schema.json`." + +**Confidence:** 0.95. **Decision:** D-211. + +### Q9 — Does the consumer repo need its own test suite + CI, or does the platform's CI cover it? + +**Ambiguity:** The consumer repo (`nova-blockchain-exchange`) has app +code (blockchain, engine, settlement). Does it run its own tests in +its own CI, or does the platform's `platform-test.yml` cover it? + +**Resolution:** The consumer repo runs its own tests in its own CI +(`nova-blockchain-exchange/.github/workflows/ci.yml` — lint + pytest on +the blockchain/engine/settlement code). The platform's +`platform-test.yml` covers the *platform* repo only (it validates +contracts against the schema, runs adapter tests, etc.). The consumer +repo's `deploy.yml` invocation triggers the platform's deploy workflow +(which runs `run_platform.sh`); the platform's policy + attestation +gates apply over the consumer's apply. The consumer's unit tests +(chain integrity, order matching, settlement) are the consumer's +responsibility. REQ-310..312 include consumer-side tests +(`test_block.py`, `test_order_book.py`, `test_settlement.py`). + +**Confidence:** 0.88. **Decision:** D-212. + +### Q10 — Is the milestone a feature milestone (tags on v1.25.x) or a major milestone (breaking schema changes)? + +**Ambiguity:** v1.26 introduces a 2nd project (multi-project mode) + +new requirements. Does this break any schema (→ major milestone, tags +on v1.26.x), or is it a feature milestone (tags on v1.25.x)? + +**Resolution:** Feature milestone. No schema breaks: the PCR schema is +unchanged (D-211); the contract schema is unchanged (the consumer +contract validates against the existing +`schemas/contract.schema.json`); the env JSON gains a real +`account_id` (data, not schema). Multi-project mode is a config +change (not a schema break). Tags run on the **v1.25.x** patch line: +`v1.25.0` (P0) → `v1.25.5` (P5 = milestone release). Per `run.md` +versioning logic: "Feature milestone (at least one feat phase): +progressive patches per phase. The final phase's patch IS the milestone +release. No separate minor tag." + +**Confidence:** 0.92. **Decision:** D-213. + +--- ## Summary -6 ambiguities identified; 6 auto-resolved at full autonomy (no human -escalation). All resolutions are binding and recorded as D-115..D-120. -The resolutions are captured in PROJECT.md hard-constraints, -REQUIREMENTS.md v1.25 sections, and will be referenced in RESEARCH.md + -PLAN.md. No PROJECT.md or REQUIREMENTS.md structural changes beyond the -v1.25 sections added in SPECIFY — the resolutions are already embedded -in the requirement text (REQ-293, REQ-297, REQ-303, etc.) via the -"Decision" annotations. \ No newline at end of file +10 ambiguities identified; all auto-resolved at full autonomy +(confidence ≥ 0.60). 8 new decisions (D-206..D-213) + 3 reaffirmed +from SPECIFY (D-200, D-201, D-203). 0 escalations (all ≥ 0.60). The +resolutions are recorded in this file + reflected in PROJECT.md / +REQUIREMENTS.md / ROADMAP.md updates. + +**Key decisions:** +- D-206: `.ciagent/` for both projects in the platform repo (multi-project mode). +- D-207: `NOVA_AWS_*` has root-equivalent perms; single key for pilot. +- D-208: `dev` bound to `581513795199`; qa/prod/dr stay placeholder (pilot-readiness policy blocks apply on placeholder). +- D-209: Pilot runs `mode: full` (apply) for `dev` only; autonomous (no HITL gate). +- D-210: Contract declares ecs + dynamodb + s3 (existing adapter stack types; no new TYPE_MAP entries). +- D-211: Outcome backfill touches metrics cold store, NOT the PCR schema (v1.25 moat preserved). +- D-212: Consumer repo has its own CI + unit tests; platform CI covers platform only. +- D-213: Feature milestone; tags on v1.25.x (no schema breaks). \ No newline at end of file