From f020178c1579c5ec39739f6df5ff6cee91d63509 Mon Sep 17 00:00:00 2001 From: Jon Chery Date: Wed, 12 Aug 2026 18:06:05 +0000 Subject: [PATCH] =?UTF-8?q?docs(P00):=20clarify=20=E2=80=94=206=20ambiguit?= =?UTF-8?q?ies=20auto-resolved=20(full=20autonomy,=20D-115..D-120)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A1 install path → go install (D-115) A2 engine enum → reuse kyverno, distinguish by ruleId KJ_ prefix (D-116) A3 checkov/wiz signatures unchanged; meta-policies consume merged PCR list (D-117) A4 NOVA_TAG_NAMING kept + kyverno-json mirror + tagging-rules-agree meta-policy (D-118) A5 critical-override kept as defense-in-depth behind declarative meta-policy (D-119) A6 kyverno-json is deterministic not AI; is_configured guard ensures platform functions without it (D-120) ---ci--- project: acdl phase: 0 milestone: v1.25 status: clarify ---/ci--- --- .ciagent/CHECKPOINT.json | 9 +- .ciagent/CLARIFY.md | 225 ++++++++++++++++++++++++--------------- 2 files changed, 146 insertions(+), 88 deletions(-) diff --git a/.ciagent/CHECKPOINT.json b/.ciagent/CHECKPOINT.json index 90b5e4b..70e0e94 100644 --- a/.ciagent/CHECKPOINT.json +++ b/.ciagent/CHECKPOINT.json @@ -1,13 +1,16 @@ { "phase": 0, - "stage": "specify", + "stage": "clarify", "milestone": "v1.25", "phase_role": "pre_execution", "attempts": 0, - "updated_at": "2026-08-12T16:20:00Z", + "updated_at": "2026-08-12T16:25:00Z", "project": "acdl", "milestone_complete": false, "tag_line": "v1.24.x", "next_tag": "v1.24.0", - "notes": "v1.25 milestone started — kyverno-json Unified Policy Engine. 19 requirements (REQ-291..309), 6 phases. Tags on v1.24.x line. Branch gate passed: HEAD on phase/00-specify-clarify-research-plan-grill." + "decisions": ["D-115", "D-116", "D-117", "D-118", "D-119", "D-120"], + "ambiguities_resolved": 6, + "escalations": 0, + "notes": "v1.25 CLARIFY complete — 6 ambiguities auto-resolved at full autonomy (D-115..D-120). Install path = go install; engine enum reuse kyverno; checkov/wiz signatures unchanged; tagging-rule cross-check; critical-override defense-in-depth; kyverno-json is deterministic not AI." } \ No newline at end of file diff --git a/.ciagent/CLARIFY.md b/.ciagent/CLARIFY.md index 1b5ebe6..15034f4 100644 --- a/.ciagent/CLARIFY.md +++ b/.ciagent/CLARIFY.md @@ -1,109 +1,164 @@ -# CLARIFY — v1.24 Consumer Guide Accuracy & Env-Promotion Lifecycle Enforcement +# CLARIFY — v1.25 kyverno-json Unified Policy Engine > **Autonomy:** full. Ambiguities are auto-resolved with assumption logging -> per `config.json autonomy.level: "full"`. No human escalation. +> per `config.json autonomy.level: "full"` and +> `autonomy.decision_confidence_threshold: 0.6`. No human escalation. ## Ambiguities Identified -### A1 — Step 8 "change environment" vs Per-env section "no field editing" +### A1 — kyverno-json install path (pip / go install / pinned binary release) -**Ambiguity:** The consumer guide contains two mutually-exclusive promotion -models. Step 8 (line 290) says "Change `environment` in your contract." The -"Per-environment deployment" section (line 398) says "you do not edit the -`environment:` field… Promotion = running the matching job." The test -`test_consumer_guide_states_no_field_editing` asserts the no-editing model. +**Ambiguity:** kyverno-json is a Go project, not a Python package. Three +install paths exist: (a) `pip install` — not possible (no PyPI package); +(b) `go install github.com/kyverno/kyverno-json/cmd/kj@latest` — requires +Go toolchain in the CI image; (c) download a pinned binary release from +GitHub releases — no Go toolchain needed, but release artifacts are +platform-specific and must be checksummed. -**User directive (binding):** Both shapes are supported. Shape A (edit -environment in-place) is valid AND must trigger a destroy of the prior env. -Shape B (per-env caller workflows) is the alternative. The test must be -updated to assert both shapes. +**Resolution (auto, confidence 0.85):** `go install` (option b). A +`scripts/install-kyverno-json.sh` helper runs +`go install github.com/kyverno/kyverno-json/cmd/kj@latest` and prints +`kj version`. The CI image (`.github/workflows/ci.yml` + +`.gitea/workflows/ci.yml`) installs Go + kj when +`config.json.policy.engine == "kyverno-json"`; the install is cached via +the existing Go module cache. Rationale: `go install` is the upstream- +blessed path, tracks the latest stable release, avoids per-platform +binary management, and the project already accepts Go-based tooling +(checkov pulls Go-built transitive deps via pip). When `which kj` is +absent, `KyvernoJsonEngine.is_configured()` returns false → `SKIPPED` +PCR (mirrors the Wiz adapter pattern) — the platform functions without +the binary. Captured in REQ-293, REQ-294. Decision ID: D-115. -**Resolution (auto, confidence 0.95):** Adopt the user's directive. Step 8 -is rewritten to document Shape A with destroy-then-rebuild semantics. The -per-env section is preserved as Shape B with a lead sentence distinguishing -it. The test is renamed and a new test asserts the destroy semantics. This -is already captured in REQ-279, REQ-280, REQ-290. +### A2 — `engine` enum value: new `"kyverno-json"` vs reuse `"kyverno"` -### A2 — Prior-env source of truth: DynamoDB vs state-bucket scan vs SSM +**Ambiguity:** `schemas/policy_check_result.schema.json` already lists +`engine: ["checkov", "kyverno", "opa", "wiz"]`. kyverno-json is a +distinct runtime from the K8s Kyverno admission controller, but both +are "Kyverno." Two options: (a) add a new `"kyverno-json"` enum value +— requires schema change + checkov/wiz adapter test regression check; +(b) reuse `"kyverno"` and distinguish by `ruleId` prefix. -**Ambiguity:** Three options for detecting the prior environment: (a) query -the `nova-contracts` DynamoDB table, (b) scan the state bucket for other env -prefixes, (c) record last-applied env in an SSM parameter. +**Resolution (auto, confidence 0.80):** Reuse `"kyverno"` (option b). +Adding `"kyverno-json"` would force a schema change + a test sweep for +no semantic gain — the `engine` field records the policy engine family, +not the specific binary. kyverno-json PCR records carry `engine: +"kyverno"` and `ruleId` prefixed `KJ_` (e.g. +`KJ_REQUIRE_TAGGING_STANDARD`), while the K8s adapter uses `KYVERNO_` +prefixes (e.g. `KYVERNO_INACTIVE_TF_STACK`). The two are distinguishable +in audit/telemetry by `ruleId` prefix and `evidence` payload shape (the +K8s adapter's evidence has `namespace`/`kind`; kyverno-json's has +`assertion`/`jmespath`). No schema change. Captured in REQ-293. +Decision ID: D-116. -**Resolution (auto, confidence 0.85):** DynamoDB `nova-contracts` table -(user-selected). It already exists, is written by the contract ingestor -Lambda (`core/lambda/contract_ingestor.py:160-170`), and has the right shape -(PK `consumerRepo`, SK `contractId#submittedAt`, `environment` attribute). -A new `#LAST_APPLIED` SK suffix is added for the record-applied-env step -(REQ-283). This avoids coupling the platform to a specific state-bucket -layout (which differs across envs/accounts) and avoids a new SSM dependency. +### A3 — Do checkov/wiz adapters change their signatures to feed kyverno-json? -**Assumption:** The `nova-contracts` table is accessible from the deploy -role via the same ABAC scoping that the contract ingestor uses. If the -table is not accessible (e.g., local/CI mode without DynamoDB), the detect -step logs a warning and returns `None` (conservative — no prior env -assumed). This is documented in REQ-282. +**Ambiguity:** The unified-orchestrator model places kyverno-json "on +top of" checkov/wiz. Two interpretations: (a) checkov/wiz now emit a +"raw findings" intermediate (not PCR) that kyverno-json meta-policies +consume — requires changing `adapt() -> list[PolicyCheckResult]` to +`adapt() -> list[RawFinding]`; (b) checkov/wiz keep emitting PCRs as +today, and the meta-policies in `adapters/kyverno-json/policies/meta/` +consume the **merged** PCR list as their payload. -### A3 — Cross-account destroy +**Resolution (auto, confidence 0.90):** Option (b). The existing +`adapt() -> list[PolicyCheckResult]` signatures are unchanged. The +meta-policies consume the merged PCR list (checkov + wiz + kyverno-json +plan-JSON policies) as their input payload. This preserves the +`PolicyCheckResult` schema as the single inter-adapter contract +(ARCHITECTURE.md §12.6), avoids a new "RawFinding" type, and means +the existing checkov/wiz adapter tests pass unchanged. The meta-policy +`block-on-any-critical.json` iterates the merged list; the +`tagging-rules-agree.json` meta-policy cross-checks the Checkov +`NOVA_TAG_NAMING` result against the kyverno-json +`KJ_REQUIRE_TAGGING_STANDARD` result by `resourceRef`. Captured in +REQ-303, D-117. Decision ID: D-117. -**Ambiguity:** If the prior env (e.g., dev) and new env (e.g., qa) are in -different AWS accounts, the destroy step needs the prior env's role -credentials. The current scaffold uses one account. +### A4 — `NOVA_TAG_NAMING` Checkov rule: rewrite as kyverno-json policy, keep, or both? -**Resolution (auto, confidence 0.80):** v1.24 targets the same-account -case. Cross-account destroy is explicitly out of scope (documented in the -Out of Scope section). The `run_platform.sh` Step 0b notes this limitation. -A future milestone handles cross-account destroy via a pre-step that -assumes the prior env's role. This is the pragmatic path — the scaffold -(`core/environments/dev.json`) is single-account today. +**Ambiguity:** The Checkov custom rule +`adapters/terraform/policy/custom_rules/nova_tagging.py` enforces the +Nova tagging standard over Terraform HCL (static scan + plan scan). The +kyverno-json milestone adds `require-tagging-standard.json` over the +resolved Stack IR. Three options: (a) rewrite — replace the Checkov +rule with the kyverno-json policy (loses Checkov's HCL-level coverage +and the `--external-checks-dir` integration); (b) keep Checkov only — +don't add a kyverno-json policy (the Stack IR is already the input to +terraform, so the Checkov rule catches it); (c) both — keep the +Checkov rule as the source of truth for HCL-level scanning AND add the +kyverno-json policy for IR-level coverage, with a meta-policy that +asserts the two agree. -### A4 — Version tag in docs: `@v1.19` vs `ref: v1.9` +**Resolution (auto, confidence 0.82):** Option (c) — both, with a +cross-check meta-policy. The Checkov rule stays the source of truth +for `terraform_plan` scanning (it reads HCL resource blocks directly); +the kyverno-json policy covers the Stack IR dict (which is the input +*before* terraform, so it catches IR-level violations that the +terraform adapter might mask via defaults). The P3 meta-policy +`tagging-rules-agree.json` asserts the two engines agree on every +resource; divergence emits an `error` PCR (defense-in-depth against +rule drift — if the two engines disagree, the operator must +investigate before proceeding). This is the only case in v1.25 where +two engines evaluate the same concern; it is intentional — the +tagging standard is the highest-impact rule (v1.8 D-tagging-standard, +v1.10 re-verification) and merits redundancy. Captured in REQ-297, +REQ-303, REQ-299. Decision ID: D-118. -**Ambiguity:** The consumer guide says `uses: nova/.github/workflows/deploy.yml@v1.19` -but the actual `.github/workflows/deploy.yml` checks out the platform repo -at `ref: v1.9`. The reference table says sample contracts "use `@v1.19`" -but the sample contracts don't carry `uses:` (they're contracts, not -workflows). +### A5 — Critical-override: delegate to declarative meta-policy or keep hard-override? -**Resolution (auto, confidence 0.90):** REQ-281 corrects the reference -table wording to "used with caller workflow `@v1.19`" (the version pin -lives in the caller workflow, not the contract). The `@v1.19` tag in the -consumer-facing docs is the documented current version; the `ref: v1.9` in -deploy.yml is the platform-internal checkout ref. These are two different -references (consumer → platform workflow tag; platform workflow → platform -repo ref). The guide's `@v1.19` stays as the consumer-facing version. No -change to deploy.yml's `ref: v1.9` (that's an internal platform concern, -out of scope for this milestone). +**Ambiguity:** `core/confidence_signal.py` lines 144-157 hardcode +`PENALTY["critical"]: None` — a critical-severity `fail` PCR forces +`score = 0, band = block` regardless of the weighted-sum inputs. The +v1.25 meta-policy `block-on-any-critical.json` makes this declarative +(asserts no PCR in the merged list has `severity: critical` + +`result: fail`). Two options: (a) fully delegate — remove the +hard-override, rely on the meta-policy to emit a critical `fail` PCR +that the existing penalty logic then blocks; (b) keep both — the +meta-policy is the declarative source of truth, the hard-override is +defense-in-depth. -### A5 — Should Shape A destroy go through the HITL decommission pipeline? +**Resolution (auto, confidence 0.88):** Option (b) — keep both. The +meta-policy is the *declarative* statement ("Nova blocks on any +critical finding from any engine"); the hard-override is the +*imperative* safety net that ensures a critical PCR can never slip +through even if the meta-policy is misconfigured or the +`PolicyEngineRegistry` returns a `NullEngine`. This is +defense-in-depth, not redundancy-for-its-own-sake: the meta-policy +runs *before* the confidence signal (it produces PCRs that flow in), +the hard-override runs *inside* the confidence signal (it is the last +gate). Removing the hard-override would make the platform's +"critical = block" guarantee depend on a single declarative policy +file — a regression in the provable-trust posture (Strategic +Objective #2). Captured in REQ-303, PROJECT.md hard-constraints. +Decision ID: D-119. -**Ambiguity:** The decommission pipeline (2-step, HITL SRE gates) exists for -stack teardown. Should env-transition destroy use it? +### A6 — Does kyverno-json break the "platform functions without AI" tenet? -**Resolution (auto, confidence 0.85):** No. Env-transition is an automated -lifecycle step, not an explicit decommission. The destroy runs as a direct -`terraform destroy -auto-approve` against the prior env's state (REQ-284). -The decommission pipeline remains for explicit stack teardown with SRE -gates. This is documented in the Out of Scope section. Rationale: the -consumer already has HITL attestation on the *new* env (qa/prod/dr gates); -requiring a second SRE gate for the prior env's destroy would block -autonomous dev→qa promotion, contradicting the "lower environments are -autonomous" tenet. +**Ambiguity:** NORTH_STAR.md Strategic Objective #2: "the platform +functions without AI — 'AI decisions' are really automated decisions." +kyverno-json is a deterministic policy engine (no ML), but it is a new +runtime dependency. Does adding it violate the tenet? -### A6 — Phase count and ordering +**Resolution (auto, confidence 0.95):** No — kyverno-json is +deterministic, not AI. The tenet distinguishes "AI decisions" (LLM- +driven, non-reproducible) from "automated decisions" (rule-driven, +reproducible). kyverno-json is the latter — the same policy + payload +produces the same result on every run. It is *more* aligned with the +tenet than the current imperative Python in `core/env_transition.py` +and `core/regression_verify.py`, because the policy is declarative +(visible, auditable, version-controlled) rather than imperative (logic +hidden in function bodies). The `is_configured()` guard ensures the +platform functions without the binary (graceful skip), so the tenet +holds even in environments where kyverno-json is not installed. +Captured in PROJECT.md hard-constraints + RESEARCH.md G-Q1. +Decision ID: D-120. -**Ambiguity:** The requirements traceability table shows 3 phases (P1: -docs, P2: feat, P3: test) but the roadmap entry says "4 phases." +## Summary -**Resolution (auto, confidence 0.90):** 4 phases = P0 (pre-execution) + P1 -(docs fixes) + P2 (env-transition feat) + P3 (tests) + P4 (final -review/ship). The "4 phases" in the roadmap counts execution phases (P1-P3) -+ final (P4). This matches the run.md phase model (P0 pre-execution, P1..PN -execution, P N+1 final). The traceability table lists P1-P3 (execution); -P4 is the final phase (review + audit + ship, no new requirements). - -## Clarification Commit - -No changes to REQUIREMENTS.md or PROJECT.md from clarify — the ambiguities -are resolved and already captured in the requirements (REQ-276..290) and -the Out of Scope section. The resolutions above are logged for traceability. \ No newline at end of file +6 ambiguities identified; 6 auto-resolved at full autonomy (no human +escalation). All resolutions are binding and recorded as D-115..D-120. +The resolutions are captured in PROJECT.md hard-constraints, +REQUIREMENTS.md v1.25 sections, and will be referenced in RESEARCH.md + +PLAN.md. No PROJECT.md or REQUIREMENTS.md structural changes beyond the +v1.25 sections added in SPECIFY — the resolutions are already embedded +in the requirement text (REQ-293, REQ-297, REQ-303, etc.) via the +"Decision" annotations. \ No newline at end of file