docs(P00): clarify — 6 ambiguities auto-resolved (full autonomy, D-115..D-120)
A1 install path → go install (D-115) A2 engine enum → reuse kyverno, distinguish by ruleId KJ_ prefix (D-116) A3 checkov/wiz signatures unchanged; meta-policies consume merged PCR list (D-117) A4 NOVA_TAG_NAMING kept + kyverno-json mirror + tagging-rules-agree meta-policy (D-118) A5 critical-override kept as defense-in-depth behind declarative meta-policy (D-119) A6 kyverno-json is deterministic not AI; is_configured guard ensures platform functions without it (D-120) ---ci--- project: acdl phase: 0 milestone: v1.25 status: clarify ---/ci---
This commit is contained in:
Jon Chery
@@ -1,13 +1,16 @@
|
||||
{
|
||||
"phase": 0,
|
||||
"stage": "specify",
|
||||
"stage": "clarify",
|
||||
"milestone": "v1.25",
|
||||
"phase_role": "pre_execution",
|
||||
"attempts": 0,
|
||||
"updated_at": "2026-08-12T16:20:00Z",
|
||||
"updated_at": "2026-08-12T16:25:00Z",
|
||||
"project": "acdl",
|
||||
"milestone_complete": false,
|
||||
"tag_line": "v1.24.x",
|
||||
"next_tag": "v1.24.0",
|
||||
"notes": "v1.25 milestone started — kyverno-json Unified Policy Engine. 19 requirements (REQ-291..309), 6 phases. Tags on v1.24.x line. Branch gate passed: HEAD on phase/00-specify-clarify-research-plan-grill."
|
||||
"decisions": ["D-115", "D-116", "D-117", "D-118", "D-119", "D-120"],
|
||||
"ambiguities_resolved": 6,
|
||||
"escalations": 0,
|
||||
"notes": "v1.25 CLARIFY complete — 6 ambiguities auto-resolved at full autonomy (D-115..D-120). Install path = go install; engine enum reuse kyverno; checkov/wiz signatures unchanged; tagging-rule cross-check; critical-override defense-in-depth; kyverno-json is deterministic not AI."
|
||||
}
|
||||
+140
-85
@@ -1,109 +1,164 @@
|
||||
# CLARIFY — v1.24 Consumer Guide Accuracy & Env-Promotion Lifecycle Enforcement
|
||||
# CLARIFY — v1.25 kyverno-json Unified Policy Engine
|
||||
|
||||
> **Autonomy:** full. Ambiguities are auto-resolved with assumption logging
|
||||
> per `config.json autonomy.level: "full"`. No human escalation.
|
||||
> per `config.json autonomy.level: "full"` and
|
||||
> `autonomy.decision_confidence_threshold: 0.6`. No human escalation.
|
||||
|
||||
## Ambiguities Identified
|
||||
|
||||
### A1 — Step 8 "change environment" vs Per-env section "no field editing"
|
||||
### A1 — kyverno-json install path (pip / go install / pinned binary release)
|
||||
|
||||
**Ambiguity:** The consumer guide contains two mutually-exclusive promotion
|
||||
models. Step 8 (line 290) says "Change `environment` in your contract." The
|
||||
"Per-environment deployment" section (line 398) says "you do not edit the
|
||||
`environment:` field… Promotion = running the matching job." The test
|
||||
`test_consumer_guide_states_no_field_editing` asserts the no-editing model.
|
||||
**Ambiguity:** kyverno-json is a Go project, not a Python package. Three
|
||||
install paths exist: (a) `pip install` — not possible (no PyPI package);
|
||||
(b) `go install github.com/kyverno/kyverno-json/cmd/kj@latest` — requires
|
||||
Go toolchain in the CI image; (c) download a pinned binary release from
|
||||
GitHub releases — no Go toolchain needed, but release artifacts are
|
||||
platform-specific and must be checksummed.
|
||||
|
||||
**User directive (binding):** Both shapes are supported. Shape A (edit
|
||||
environment in-place) is valid AND must trigger a destroy of the prior env.
|
||||
Shape B (per-env caller workflows) is the alternative. The test must be
|
||||
updated to assert both shapes.
|
||||
**Resolution (auto, confidence 0.85):** `go install` (option b). A
|
||||
`scripts/install-kyverno-json.sh` helper runs
|
||||
`go install github.com/kyverno/kyverno-json/cmd/kj@latest` and prints
|
||||
`kj version`. The CI image (`.github/workflows/ci.yml` +
|
||||
`.gitea/workflows/ci.yml`) installs Go + kj when
|
||||
`config.json.policy.engine == "kyverno-json"`; the install is cached via
|
||||
the existing Go module cache. Rationale: `go install` is the upstream-
|
||||
blessed path, tracks the latest stable release, avoids per-platform
|
||||
binary management, and the project already accepts Go-based tooling
|
||||
(checkov pulls Go-built transitive deps via pip). When `which kj` is
|
||||
absent, `KyvernoJsonEngine.is_configured()` returns false → `SKIPPED`
|
||||
PCR (mirrors the Wiz adapter pattern) — the platform functions without
|
||||
the binary. Captured in REQ-293, REQ-294. Decision ID: D-115.
|
||||
|
||||
**Resolution (auto, confidence 0.95):** Adopt the user's directive. Step 8
|
||||
is rewritten to document Shape A with destroy-then-rebuild semantics. The
|
||||
per-env section is preserved as Shape B with a lead sentence distinguishing
|
||||
it. The test is renamed and a new test asserts the destroy semantics. This
|
||||
is already captured in REQ-279, REQ-280, REQ-290.
|
||||
### A2 — `engine` enum value: new `"kyverno-json"` vs reuse `"kyverno"`
|
||||
|
||||
### A2 — Prior-env source of truth: DynamoDB vs state-bucket scan vs SSM
|
||||
**Ambiguity:** `schemas/policy_check_result.schema.json` already lists
|
||||
`engine: ["checkov", "kyverno", "opa", "wiz"]`. kyverno-json is a
|
||||
distinct runtime from the K8s Kyverno admission controller, but both
|
||||
are "Kyverno." Two options: (a) add a new `"kyverno-json"` enum value
|
||||
— requires schema change + checkov/wiz adapter test regression check;
|
||||
(b) reuse `"kyverno"` and distinguish by `ruleId` prefix.
|
||||
|
||||
**Ambiguity:** Three options for detecting the prior environment: (a) query
|
||||
the `nova-contracts` DynamoDB table, (b) scan the state bucket for other env
|
||||
prefixes, (c) record last-applied env in an SSM parameter.
|
||||
**Resolution (auto, confidence 0.80):** Reuse `"kyverno"` (option b).
|
||||
Adding `"kyverno-json"` would force a schema change + a test sweep for
|
||||
no semantic gain — the `engine` field records the policy engine family,
|
||||
not the specific binary. kyverno-json PCR records carry `engine:
|
||||
"kyverno"` and `ruleId` prefixed `KJ_<policy_name>` (e.g.
|
||||
`KJ_REQUIRE_TAGGING_STANDARD`), while the K8s adapter uses `KYVERNO_`
|
||||
prefixes (e.g. `KYVERNO_INACTIVE_TF_STACK`). The two are distinguishable
|
||||
in audit/telemetry by `ruleId` prefix and `evidence` payload shape (the
|
||||
K8s adapter's evidence has `namespace`/`kind`; kyverno-json's has
|
||||
`assertion`/`jmespath`). No schema change. Captured in REQ-293.
|
||||
Decision ID: D-116.
|
||||
|
||||
**Resolution (auto, confidence 0.85):** DynamoDB `nova-contracts` table
|
||||
(user-selected). It already exists, is written by the contract ingestor
|
||||
Lambda (`core/lambda/contract_ingestor.py:160-170`), and has the right shape
|
||||
(PK `consumerRepo`, SK `contractId#submittedAt`, `environment` attribute).
|
||||
A new `#LAST_APPLIED` SK suffix is added for the record-applied-env step
|
||||
(REQ-283). This avoids coupling the platform to a specific state-bucket
|
||||
layout (which differs across envs/accounts) and avoids a new SSM dependency.
|
||||
### A3 — Do checkov/wiz adapters change their signatures to feed kyverno-json?
|
||||
|
||||
**Assumption:** The `nova-contracts` table is accessible from the deploy
|
||||
role via the same ABAC scoping that the contract ingestor uses. If the
|
||||
table is not accessible (e.g., local/CI mode without DynamoDB), the detect
|
||||
step logs a warning and returns `None` (conservative — no prior env
|
||||
assumed). This is documented in REQ-282.
|
||||
**Ambiguity:** The unified-orchestrator model places kyverno-json "on
|
||||
top of" checkov/wiz. Two interpretations: (a) checkov/wiz now emit a
|
||||
"raw findings" intermediate (not PCR) that kyverno-json meta-policies
|
||||
consume — requires changing `adapt() -> list[PolicyCheckResult]` to
|
||||
`adapt() -> list[RawFinding]`; (b) checkov/wiz keep emitting PCRs as
|
||||
today, and the meta-policies in `adapters/kyverno-json/policies/meta/`
|
||||
consume the **merged** PCR list as their payload.
|
||||
|
||||
### A3 — Cross-account destroy
|
||||
**Resolution (auto, confidence 0.90):** Option (b). The existing
|
||||
`adapt() -> list[PolicyCheckResult]` signatures are unchanged. The
|
||||
meta-policies consume the merged PCR list (checkov + wiz + kyverno-json
|
||||
plan-JSON policies) as their input payload. This preserves the
|
||||
`PolicyCheckResult` schema as the single inter-adapter contract
|
||||
(ARCHITECTURE.md §12.6), avoids a new "RawFinding" type, and means
|
||||
the existing checkov/wiz adapter tests pass unchanged. The meta-policy
|
||||
`block-on-any-critical.json` iterates the merged list; the
|
||||
`tagging-rules-agree.json` meta-policy cross-checks the Checkov
|
||||
`NOVA_TAG_NAMING` result against the kyverno-json
|
||||
`KJ_REQUIRE_TAGGING_STANDARD` result by `resourceRef`. Captured in
|
||||
REQ-303, D-117. Decision ID: D-117.
|
||||
|
||||
**Ambiguity:** If the prior env (e.g., dev) and new env (e.g., qa) are in
|
||||
different AWS accounts, the destroy step needs the prior env's role
|
||||
credentials. The current scaffold uses one account.
|
||||
### A4 — `NOVA_TAG_NAMING` Checkov rule: rewrite as kyverno-json policy, keep, or both?
|
||||
|
||||
**Resolution (auto, confidence 0.80):** v1.24 targets the same-account
|
||||
case. Cross-account destroy is explicitly out of scope (documented in the
|
||||
Out of Scope section). The `run_platform.sh` Step 0b notes this limitation.
|
||||
A future milestone handles cross-account destroy via a pre-step that
|
||||
assumes the prior env's role. This is the pragmatic path — the scaffold
|
||||
(`core/environments/dev.json`) is single-account today.
|
||||
**Ambiguity:** The Checkov custom rule
|
||||
`adapters/terraform/policy/custom_rules/nova_tagging.py` enforces the
|
||||
Nova tagging standard over Terraform HCL (static scan + plan scan). The
|
||||
kyverno-json milestone adds `require-tagging-standard.json` over the
|
||||
resolved Stack IR. Three options: (a) rewrite — replace the Checkov
|
||||
rule with the kyverno-json policy (loses Checkov's HCL-level coverage
|
||||
and the `--external-checks-dir` integration); (b) keep Checkov only —
|
||||
don't add a kyverno-json policy (the Stack IR is already the input to
|
||||
terraform, so the Checkov rule catches it); (c) both — keep the
|
||||
Checkov rule as the source of truth for HCL-level scanning AND add the
|
||||
kyverno-json policy for IR-level coverage, with a meta-policy that
|
||||
asserts the two agree.
|
||||
|
||||
### A4 — Version tag in docs: `@v1.19` vs `ref: v1.9`
|
||||
**Resolution (auto, confidence 0.82):** Option (c) — both, with a
|
||||
cross-check meta-policy. The Checkov rule stays the source of truth
|
||||
for `terraform_plan` scanning (it reads HCL resource blocks directly);
|
||||
the kyverno-json policy covers the Stack IR dict (which is the input
|
||||
*before* terraform, so it catches IR-level violations that the
|
||||
terraform adapter might mask via defaults). The P3 meta-policy
|
||||
`tagging-rules-agree.json` asserts the two engines agree on every
|
||||
resource; divergence emits an `error` PCR (defense-in-depth against
|
||||
rule drift — if the two engines disagree, the operator must
|
||||
investigate before proceeding). This is the only case in v1.25 where
|
||||
two engines evaluate the same concern; it is intentional — the
|
||||
tagging standard is the highest-impact rule (v1.8 D-tagging-standard,
|
||||
v1.10 re-verification) and merits redundancy. Captured in REQ-297,
|
||||
REQ-303, REQ-299. Decision ID: D-118.
|
||||
|
||||
**Ambiguity:** The consumer guide says `uses: nova/.github/workflows/deploy.yml@v1.19`
|
||||
but the actual `.github/workflows/deploy.yml` checks out the platform repo
|
||||
at `ref: v1.9`. The reference table says sample contracts "use `@v1.19`"
|
||||
but the sample contracts don't carry `uses:` (they're contracts, not
|
||||
workflows).
|
||||
### A5 — Critical-override: delegate to declarative meta-policy or keep hard-override?
|
||||
|
||||
**Resolution (auto, confidence 0.90):** REQ-281 corrects the reference
|
||||
table wording to "used with caller workflow `@v1.19`" (the version pin
|
||||
lives in the caller workflow, not the contract). The `@v1.19` tag in the
|
||||
consumer-facing docs is the documented current version; the `ref: v1.9` in
|
||||
deploy.yml is the platform-internal checkout ref. These are two different
|
||||
references (consumer → platform workflow tag; platform workflow → platform
|
||||
repo ref). The guide's `@v1.19` stays as the consumer-facing version. No
|
||||
change to deploy.yml's `ref: v1.9` (that's an internal platform concern,
|
||||
out of scope for this milestone).
|
||||
**Ambiguity:** `core/confidence_signal.py` lines 144-157 hardcode
|
||||
`PENALTY["critical"]: None` — a critical-severity `fail` PCR forces
|
||||
`score = 0, band = block` regardless of the weighted-sum inputs. The
|
||||
v1.25 meta-policy `block-on-any-critical.json` makes this declarative
|
||||
(asserts no PCR in the merged list has `severity: critical` +
|
||||
`result: fail`). Two options: (a) fully delegate — remove the
|
||||
hard-override, rely on the meta-policy to emit a critical `fail` PCR
|
||||
that the existing penalty logic then blocks; (b) keep both — the
|
||||
meta-policy is the declarative source of truth, the hard-override is
|
||||
defense-in-depth.
|
||||
|
||||
### A5 — Should Shape A destroy go through the HITL decommission pipeline?
|
||||
**Resolution (auto, confidence 0.88):** Option (b) — keep both. The
|
||||
meta-policy is the *declarative* statement ("Nova blocks on any
|
||||
critical finding from any engine"); the hard-override is the
|
||||
*imperative* safety net that ensures a critical PCR can never slip
|
||||
through even if the meta-policy is misconfigured or the
|
||||
`PolicyEngineRegistry` returns a `NullEngine`. This is
|
||||
defense-in-depth, not redundancy-for-its-own-sake: the meta-policy
|
||||
runs *before* the confidence signal (it produces PCRs that flow in),
|
||||
the hard-override runs *inside* the confidence signal (it is the last
|
||||
gate). Removing the hard-override would make the platform's
|
||||
"critical = block" guarantee depend on a single declarative policy
|
||||
file — a regression in the provable-trust posture (Strategic
|
||||
Objective #2). Captured in REQ-303, PROJECT.md hard-constraints.
|
||||
Decision ID: D-119.
|
||||
|
||||
**Ambiguity:** The decommission pipeline (2-step, HITL SRE gates) exists for
|
||||
stack teardown. Should env-transition destroy use it?
|
||||
### A6 — Does kyverno-json break the "platform functions without AI" tenet?
|
||||
|
||||
**Resolution (auto, confidence 0.85):** No. Env-transition is an automated
|
||||
lifecycle step, not an explicit decommission. The destroy runs as a direct
|
||||
`terraform destroy -auto-approve` against the prior env's state (REQ-284).
|
||||
The decommission pipeline remains for explicit stack teardown with SRE
|
||||
gates. This is documented in the Out of Scope section. Rationale: the
|
||||
consumer already has HITL attestation on the *new* env (qa/prod/dr gates);
|
||||
requiring a second SRE gate for the prior env's destroy would block
|
||||
autonomous dev→qa promotion, contradicting the "lower environments are
|
||||
autonomous" tenet.
|
||||
**Ambiguity:** NORTH_STAR.md Strategic Objective #2: "the platform
|
||||
functions without AI — 'AI decisions' are really automated decisions."
|
||||
kyverno-json is a deterministic policy engine (no ML), but it is a new
|
||||
runtime dependency. Does adding it violate the tenet?
|
||||
|
||||
### A6 — Phase count and ordering
|
||||
**Resolution (auto, confidence 0.95):** No — kyverno-json is
|
||||
deterministic, not AI. The tenet distinguishes "AI decisions" (LLM-
|
||||
driven, non-reproducible) from "automated decisions" (rule-driven,
|
||||
reproducible). kyverno-json is the latter — the same policy + payload
|
||||
produces the same result on every run. It is *more* aligned with the
|
||||
tenet than the current imperative Python in `core/env_transition.py`
|
||||
and `core/regression_verify.py`, because the policy is declarative
|
||||
(visible, auditable, version-controlled) rather than imperative (logic
|
||||
hidden in function bodies). The `is_configured()` guard ensures the
|
||||
platform functions without the binary (graceful skip), so the tenet
|
||||
holds even in environments where kyverno-json is not installed.
|
||||
Captured in PROJECT.md hard-constraints + RESEARCH.md G-Q1.
|
||||
Decision ID: D-120.
|
||||
|
||||
**Ambiguity:** The requirements traceability table shows 3 phases (P1:
|
||||
docs, P2: feat, P3: test) but the roadmap entry says "4 phases."
|
||||
## Summary
|
||||
|
||||
**Resolution (auto, confidence 0.90):** 4 phases = P0 (pre-execution) + P1
|
||||
(docs fixes) + P2 (env-transition feat) + P3 (tests) + P4 (final
|
||||
review/ship). The "4 phases" in the roadmap counts execution phases (P1-P3)
|
||||
+ final (P4). This matches the run.md phase model (P0 pre-execution, P1..PN
|
||||
execution, P N+1 final). The traceability table lists P1-P3 (execution);
|
||||
P4 is the final phase (review + audit + ship, no new requirements).
|
||||
|
||||
## Clarification Commit
|
||||
|
||||
No changes to REQUIREMENTS.md or PROJECT.md from clarify — the ambiguities
|
||||
are resolved and already captured in the requirements (REQ-276..290) and
|
||||
the Out of Scope section. The resolutions above are logged for traceability.
|
||||
6 ambiguities identified; 6 auto-resolved at full autonomy (no human
|
||||
escalation). All resolutions are binding and recorded as D-115..D-120.
|
||||
The resolutions are captured in PROJECT.md hard-constraints,
|
||||
REQUIREMENTS.md v1.25 sections, and will be referenced in RESEARCH.md +
|
||||
PLAN.md. No PROJECT.md or REQUIREMENTS.md structural changes beyond the
|
||||
v1.25 sections added in SPECIFY — the resolutions are already embedded
|
||||
in the requirement text (REQ-293, REQ-297, REQ-303, etc.) via the
|
||||
"Decision" annotations.
|
||||
Reference in New Issue
Block a user