diff --git a/.ciagent/ARCHITECTURE.md b/.ciagent/ARCHITECTURE.md index 86368c3..b7cd604 100644 --- a/.ciagent/ARCHITECTURE.md +++ b/.ciagent/ARCHITECTURE.md @@ -879,3 +879,67 @@ config entry in `config.json` (`strategic_direction_file: ".ciagent/NORTH_STAR.md"`) that the run workflow reads at SPECIFY. This ensures the strategic direction survives across milestones without being overwritten by status updates. + +### §12.7 — Policy Engine Registry (v1.25, REQ-291) + +The policy-engine abstraction is first-class: a swappable `PolicyEngine` +protocol so the engine may change without touching the confidence +signal, the pipeline, or the `PolicyCheckResult` schema. This is the +**swap boundary** that keeps the platform's compliance posture +replaceable (Strategic Objective #2 — provable trust via a replaceable +substrate, not a vendor lock-in). + +``` + contract.yml ─┐ ┌─→ list[PolicyCheckResult] ─┐ + stack IR ─────┼─→ PolicyEngine.evaluate ├─→ list[PolicyCheckResult] ─┼─→ confidence_signal + plan JSON ────┤ (protocol) └─→ list[PolicyCheckResult] ─┘ (engine-agnostic, + PCR list ─────┘ unchanged) + │ + ▼ + ┌─ KyvernoJsonEngine (shells to `kj scan`; engine: "kyverno") + └─ OpaEngine (future — same protocol; engine: "opa") + + checkov/wiz ──→ raw findings ──→ (merged PCR list is the meta-policy payload) +``` + +**The protocol (`core/policy_engine.py`):** +```python +class PolicyEngine(Protocol): + @property + def name(self) -> str: ... + def is_configured(self) -> bool: ... + def evaluate(self, payload, policy_dir: Path, contract_id: str) -> list[dict]: ... +``` + +**The registry** reads `config.json.policy.engine` (default +`"kyverno-json"`) and returns the active engine. A `NullEngine` is the +fallback when the `policy` key is absent (emits `SKIPPED` PCRs — +backward compatibility for tests that don't set the key). The +confidence signal is **untouched** — it already consumes +`list[PolicyCheckResult]` engine-agnostically (§12.6). v1.25 only +changes *who produces* the PCR list, not *what* the list is. + +**Engine enum reuse (D-116):** kyverno-json PCR records carry +`engine: "kyverno"` (no new enum value). The `engine` field records the +policy-engine *family*, not the specific binary. The K8s Kyverno adapter +and the kyverno-json engine are distinguished by `ruleId` prefix +(`KYVERNO_` vs `KJ_`) and `evidence` payload shape (`namespace`/`kind` +vs `assertion`/`jmespath`). + +**Defense-in-depth (D-119):** the declarative meta-policy +`block-on-any-critical` (asserts no PCR has `severity: critical` + +`result: fail`) is the *source of truth* for "critical = block". The +`confidence_signal.py` `PENALTY["critical"]: None` hard-override stays +as the *imperative* safety net — the meta-policy runs *before* the +confidence signal (produces PCRs that flow in), the hard-override runs +*inside* it (the last gate). Removing the hard-override would make the +"critical = block" guarantee depend on a single policy file — a +regression in provable trust. + +**Graceful degradation (D-120):** `KyvernoJsonEngine.is_configured()` +returns false when `which kj` is absent → `evaluate()` returns a single +`SKIPPED` PCR (`ruleId: "KJ_ENGINE_NOT_CONFIGURED"`). The platform +functions without the binary (the "platform functions without AI / +deterministic scripts" tenet holds — kyverno-json is deterministic, not +AI; the `is_configured()` guard ensures the platform runs even when the +binary is not installed). diff --git a/.ciagent/CHECKPOINT.json b/.ciagent/CHECKPOINT.json index 70e0e94..1faebbe 100644 --- a/.ciagent/CHECKPOINT.json +++ b/.ciagent/CHECKPOINT.json @@ -1,16 +1,17 @@ { "phase": 0, - "stage": "clarify", + "stage": "research", "milestone": "v1.25", "phase_role": "pre_execution", "attempts": 0, - "updated_at": "2026-08-12T16:25:00Z", + "updated_at": "2026-08-12T16:35:00Z", "project": "acdl", "milestone_complete": false, "tag_line": "v1.24.x", "next_tag": "v1.24.0", - "decisions": ["D-115", "D-116", "D-117", "D-118", "D-119", "D-120"], - "ambiguities_resolved": 6, - "escalations": 0, - "notes": "v1.25 CLARIFY complete — 6 ambiguities auto-resolved at full autonomy (D-115..D-120). Install path = go install; engine enum reuse kyverno; checkov/wiz signatures unchanged; tagging-rule cross-check; critical-override defense-in-depth; kyverno-json is deterministic not AI." + "personas_active": ["lead-developer", "backend-engineer", "policy-engineer", "data-engineer"], + "personas_deactivated": ["frontend-engineer"], + "new_personas": ["policy-engineer"], + "assumptions": ["A1", "A2", "A3", "A4", "A5"], + "notes": "v1.25 RESEARCH complete — kyverno-json CLI/policy/assertion-tree surface documented; 4 policy targets analyzed; PolicyEngine protocol + OPA-equivalent swap surface; new policy-engineer persona; ARCHITECTURE.md §12.7 added. Latency impact <1s (parallel with checkov)." } \ No newline at end of file diff --git a/.ciagent/PERSONAS.md b/.ciagent/PERSONAS.md index 1725415..3a064ff 100644 --- a/.ciagent/PERSONAS.md +++ b/.ciagent/PERSONAS.md @@ -1,83 +1,132 @@ --- project: acdl -milestone: v1.24 +milestone: v1.25 generated_at: 2026-08-12 generator: lead-developer verification_toolchain: - typecheck: "python3 -m py_compile core/env_transition.py tests/test_env_transition.py tests/test_run_platform_env_transition.py" - test: "pytest tests/test_env_transition.py tests/test_run_platform_env_transition.py tests/test_consumer_guide_per_env_section.py tests/test_adapter.py tests/test_contract_resolver.py tests/test_deploy_workflow_env_input.py tests/test_pipeline.py -v" - lint: "ruff check core/env_transition.py tests/test_env_transition.py tests/test_run_platform_env_transition.py 2>/dev/null || python3 -m py_compile core/env_transition.py" + typecheck: "python3 -m py_compile core/policy_engine.py adapters/kyverno-json/kyverno_json_engine.py tests/test_policy_engine.py tests/test_kyverno_json_engine.py" + test: "pytest tests/test_policy_engine.py tests/test_kyverno_json_engine.py tests/test_adapter.py tests/test_contract_resolver.py tests/test_confidence_signal.py tests/test_checkov_adapter.py tests/test_kyverno_adapter.py tests/test_pipeline.py -v" + lint: "ruff check core/policy_engine.py adapters/kyverno-json/ 2>/dev/null || python3 -m py_compile core/policy_engine.py" note: | - v1.24 is the Consumer Guide Accuracy & Env-Promotion Lifecycle - Enforcement milestone — a mixed docs+feat+test milestone. Two active - personas: lead-developer (consumer-guide.md edits + guide test - updates), backend-engineer (core/env_transition.py + run_platform.sh - Step 0b + deploy.yml + adapter doc comment + env_transition tests + - pipeline tests). frontend-engineer stays deactivated (no UI). No - data-engineer (no schema changes — the nova-contracts table already - exists). No new personas. + v1.25 is the kyverno-json Unified Policy Engine milestone — a feat + milestone. Four active personas: lead-developer (coordination + + docs + ARCHITECTURE.md §12.7), backend-engineer (core/policy_engine.py + protocol + registry + contract_resolver.py wiring + run_platform.sh + Step 5 + pipeline tests), policy-engineer (adapters/kyverno-json/ + engine + policies across all 4 target dirs + meta-policies + policy + tests + adapter README + STANDARDS.md policy-authoring section), + data-engineer (config.json policy object + schemas/README.md note + + capability-inventory JSON fixture for regression policies). + frontend-engineer stays deactivated (no UI). The policy-engineer is a + new custom persona created for this milestone's policy domain (see + RESEARCH.md §4 — kyverno-json + JMESPath is a distinct framework from + backend-engineer's fastify/hono). --- -# ACDL — Persona Roster (v1.24 Consumer Guide Accuracy & Env-Promotion Lifecycle Enforcement) +# ACDL — Persona Roster (v1.25 kyverno-json Unified Policy Engine) -> v1.24 roster. Two active personas + two deactivated. This is a mixed -> docs+feat+test milestone: the work is consumer-guide accuracy fixes -> (lead-developer), a new env-transition detect-and-destroy platform -> feature (backend-engineer), and test coverage for both (split). +> v1.25 roster. Four active personas + one deactivated. This is a feat +> milestone: the work is a swappable policy-engine protocol + a new +> adapter + policies across 4 Nova artifacts + pipeline wiring + docs. +> The policy-engineer is a new custom persona — kyverno-json + JMESPath +> is a specialized domain that doesn't fit backend-engineer's +> fastify/hono frameworks or data-engineer's drizzle/postgresql. ## Active personas ### lead-developer - **Domain:** coordination + docs - **Frameworks:** [] -- **Constraints:** ["pragmatic", "battle-tested defaults", "docs match code"] +- **Constraints:** ["pragmatic", "battle-tested defaults", "docs match code", "swap boundary is the moat"] - **Territory:** - - `docs/consumer-guide.md` - - `tests/test_consumer_guide_per_env_section.py` - - `.ciagent/*.md` (PLAN.md, RESEARCH.md, etc.) -- **Reason:** Owns the consumer guide narrative — the 5 accuracy fixes - (stale fields table, inconsistent caller, "dev only" phrasing, Step 8 - rewrite with destroy semantics, reference table wording) and the - consumer-guide test updates (rename + new destroy-on-env-change test). - No UI work (frontend-engineer deactivated). No Python/bash platform - code (backend-engineer territory). + - `.ciagent/ARCHITECTURE.md` (§12.7 Policy Engine Registry — NEW) + - `.ciagent/PROJECT.md` (v1.25 section) + - `.ciagent/REQUIREMENTS.md` (v1.25 section) + - `.ciagent/ROADMAP.md` (v1.25 section) + - `.ciagent/PLAN.md`, `.ciagent/RESEARCH.md`, `.ciagent/CLARIFY.md`, + `.ciagent/GRILL.md`, `.ciagent/PERSONAS.md` + - `docs/METRICS.md` (swappable engine narrative — REQ-307) +- **Reason:** Owns the milestone coordination + the architecture + narrative. The swap boundary (PolicyEngine protocol) is the moat per + Strategic Objective #2 — the lead-developer owns the boundary + description in ARCHITECTURE.md §12.7 and the docs/METRICS.md note. + No Python policy code (backend-engineer + policy-engineer territory). + No UI (frontend-engineer deactivated). ### backend-engineer -- **Domain:** backend (Python + bash + YAML) +- **Domain:** backend (Python + bash + pipeline wiring) - **Frameworks:** ["boto3", "terraform"] -- **Constraints:** ["api-first", "fail-closed", "no orphan paths", "state-key determinism"] +- **Constraints:** ["api-first", "strict-typing", "engine-agnostic confidence signal", "fail-soft when kj absent"] - **Territory:** - - `core/env_transition.py` (NEW) - - `scripts/run_platform.sh` (Step 0b insert + record-applied-env) - - `.github/workflows/deploy.yml` (NOVA_CONSUMER_REPO env) - - `adapters/terraform/adapter.py` (doc comment only) - - `tests/test_env_transition.py` (NEW) - - `tests/test_run_platform_env_transition.py` (NEW) -- **Reason:** Owns the env-transition detect-and-destroy feature: the new - `core/env_transition.py` module (DynamoDB query + record), the - `run_platform.sh` Step 0b orchestrator (re-resolve + terraform destroy + - evidence event + fail-closed), the deploy.yml env var passthrough, and - the two new test files. Uses boto3 (DynamoDB) + terraform (destroy) + - bash (pipeline orchestration). + - `core/policy_engine.py` (NEW — PolicyEngine Protocol + PolicyEngineRegistry + NullEngine) + - `core/contract_resolver.py` (MODIFIED — invoke registry pre/post resolve) + - `scripts/run_platform.sh` (MODIFIED — Step 5 kyverno-json parallel pass) + - `scripts/install-kyverno-json.sh` (NEW) + - `tests/test_policy_engine.py` (NEW — protocol conformance, registry, NullEngine) + - `tests/test_run_platform_plan_json_policies.py` (NEW — script-substring assertion) + - `.github/workflows/ci.yml` + `.gitea/workflows/ci.yml` (MODIFIED — Go + kj install) +- **Reason:** Owns the Python protocol layer + the pipeline wiring. The + `PolicyEngine` Protocol + `PolicyEngineRegistry` are Python structural- + typing constructs (PEP 544) — backend-engineer's strict-typing + constraint. The `contract_resolver.py` wiring + `run_platform.sh` + Step 5 are backend territory. Does NOT write kyverno-json policy + files (policy-engineer territory) — only the Python that *invokes* the + engine. Does NOT modify the confidence signal (it already consumes + `list[PolicyCheckResult]` engine-agnostically — PROJECT.md hard- + constraint). + +### policy-engineer +- **Domain:** policy (declarative compliance rules) +- **Frameworks:** ["kyverno-json", "jmespath", "kyverno ValidatingPolicy"] +- **Constraints:** ["declarative-policies", "no-imperative-rules", "schema-validated", "severity-via-annotation", "assertion-trees-not-foreach"] +- **Territory:** + - `adapters/kyverno-json/` (NEW — engine impl + __init__.py + README) + - `adapters/kyverno-json/kyverno_json_engine.py` (NEW — KyvernoJsonEngine) + - `adapters/kyverno-json/policies/` (NEW — all 4 target dirs: contract/, stack-ir/, plan-json/, meta/, regression/) + - `adapters/kyverno-json/policies/_smoke.json` (NEW) + - `adapters/README.md` (MODIFIED — new adapter row + PolicyEngine Protocol section) + - `tests/test_kyverno_json_engine.py` (NEW — PCR schema validity, defensive parsing) + - `tests/test_stack_ir_policies.py` (NEW) + - `tests/test_plan_json_policies.py` (NEW) + - `tests/test_meta_policies.py` (NEW) + - `tests/test_regression_policies.py` (NEW) + - `tests/fixtures/stack_ir/`, `tests/fixtures/plan_json/`, `tests/fixtures/capability_inventory.json` (NEW) + - `modules/STANDARDS.md` (MODIFIED — Policy authoring standard section — REQ-307) +- **Reason:** The policy-engineer owns the declarative policy artifacts. + kyverno-json's `ValidatingPolicy` + assertion trees + JMESPath is a + distinct framework from backend-engineer's fastify/hono and requires + its own constraints: no imperative rules (everything is an assertion + tree), severity via the `nova.cloudinit.dev/severity` annotation (not + in the engine adapter), no `forEach` (use the `~` modifier). The + adapter pattern (engine ↔ protocol ↔ registry) is backend-engineer + territory, but the policy *content* and the engine *translation* + (`_to_pcr()`) are policy-engineer territory because they require + kyverno-json output-shape knowledge. Created per RESEARCH.md §4 — this + is a phase-spanning persona (active for P1..P4), not phase-specific. + +### data-engineer +- **Domain:** data (config schema + structured fixtures) +- **Frameworks:** ["jsonschema", "yaml"] +- **Constraints:** ["schema-first", "type-safe config", "backward-compatible additions"] +- **Territory:** + - `.ciagent/config.json` (MODIFIED — new `policy` object: engine + policy_root) + - `schemas/policy_check_result.schema.json` (READ-ONLY — no change per D-116) + - `schemas/README.md` (MODIFIED — note engine: "kyverno" shared by K8s adapter + kj) + - `tests/fixtures/capability_inventory.json` (NEW — clean + drifted inventory fixtures for regression policies) +- **Reason:** The `config.json.policy` object is a schema-first addition + (new top-level key with `engine` + `policy_root` fields). The + capability-inventory JSON fixtures for the regression-gate policies + (REQ-304) are structured data — the data-engineer owns the fixture + shape. The `policy_check_result.schema.json` is read-only (D-116 — no + enum change); the data-engineer documents the `engine: "kyverno"` + sharing in `schemas/README.md`. No migrations (no database). No Python + (backend-engineer + policy-engineer territory). ## Deactivated personas ### frontend-engineer -- **Active:** false -- **Reason:** No UI work in v1.24. The consumer guide is markdown docs - (lead-developer territory). Deactivated per v1.17/v1.18/v1.22/v1.23 - precedent. - -### data-engineer -- **Active:** false -- **Reason:** No schema changes in v1.24. The `nova-contracts` DynamoDB - table already exists with the right shape (PK `consumerRepo`, SK - `contractId#submittedAt`). The env-transition module only adds a new - `#LAST_APPLIED` SK suffix — no schema migration, no ORM, no new tables. - The backend-engineer handles the boto3 queries. - -## Phase-specific notes - -- No phase-specific personas. Both active personas span P1-P3. -- P4 (final review + audit + ship) is lead-developer territory - (orchestration + docs completion). \ No newline at end of file +- **active:** false +- **Reason:** ACDL has no frontend (no package.json — confirmed in + config.json personas.personas[frontend-engineer].reason). v1.25 adds + no UI work — the policy engine is backend + policy artifacts only. + Deactivated per the v1.15+ convention. \ No newline at end of file diff --git a/.ciagent/RESEARCH.md b/.ciagent/RESEARCH.md index 4eccb46..fcaf2b9 100644 --- a/.ciagent/RESEARCH.md +++ b/.ciagent/RESEARCH.md @@ -1,187 +1,438 @@ -# Nova — v1.24 Research Findings +# Nova — v1.25 Research Findings -> Phase: research (pre-execution). Milestone: v1.24 (Consumer Guide Accuracy -> & Env-Promotion Lifecycle Enforcement). Status: research. -> Researcher: ci-researcher. Autonomy: full. +> Phase: research (pre-execution). Milestone: v1.25 (kyverno-json Unified +> Policy Engine). Status: research. Researcher: ci-researcher. +> Autonomy: full. ## 1. Problem domain -Two distinct problem spaces in one milestone: +Nova's compliance/policy posture is fragmented across three engines with +three rule languages and three adapter shapes (see PROJECT.md v1.25 +"Why" for the full diagnosis). The `PolicyCheckResult` schema +(`schemas/policy_check_result.schema.json`) is already the engine-agnostic +contract that `core/confidence_signal.py` consumes — the *contract* is +right; the *orchestration* is fragmented. There is no single declarative +place where "what Nova considers compliant" lives. The K8s-only Kyverno +adapter (`adapters/kyverno/`) can't help because it only speaks to K8s +manifests and the platform emits Terraform (D-053). -### 1a. Consumer guide accuracy (docs) +`kyverno-json` is the correction: a Kyverno-ecosystem runtime that applies +Kyverno policies to **any** JSON/YAML payload. It becomes the **unified +orchestrator** of compliance checks, behind a swappable `PolicyEngine` +protocol so OPA can replace it one day. Checkov and Wiz remain as +raw-finding adapters feeding *into* kyverno-json meta-policies. -The consumer guide (`docs/consumer-guide.md`, 477 lines) has 5 accuracy -defects identified in review: +## 2. kyverno-json — the engine surface -1. **Step 3 contract fields table (lines 141-147)** lists `uses`, `module`, - `environment`, `inputs`. The actual schema - (`schemas/contract.schema.json:7`) requires `id`, `name`, `environment`, - `infrastructure`. The `uses` field was dropped in v1.10.2 (REQ-50 - superseded) and `module` was replaced by the `infrastructure` map key. - The worked examples (lines 111-137) use the correct fields. +### 2.1 What it is -2. **Step 4 caller (lines 173-183)** omits `environment:` in `with:`, while - Step 2 (lines 94-101) shows `environment: dev`. The two canonical caller - snippets disagree. +[kyverno-json](https://github.com/kyverno/kyverno-json) is a standalone Go +binary from the Kyverno project. It is a **separate runtime** from the +Kyverno K8s admission controller — same policy lineage, different +application target. Where Kyverno (K8s) evaluates `ClusterPolicy` +resources against Kubernetes manifests at admission time, kyverno-json +evaluates `ValidatingPolicy` resources against **any** JSON or YAML +payload file via the CLI (`kj scan`) or a Go library. It is **not** a +Python package (no PyPI release); it is installed via +`go install github.com/kyverno/kyverno-json/cmd/kj@latest` (D-115) or by +downloading a pinned binary from GitHub releases. -3. **Step 5 stage 8 (lines 232, 253)** says "(dev only)" for the apply - stage. Higher environments *do* apply — they apply after HITL - attestation per `docs/environments/index.md:44-54`. +### 2.2 CLI surface (the v1.25 invocation path) -4. **Step 8 (lines 290-306)** says "Change `environment` in your contract" - to promote, which contradicts the same doc's "Per-environment - deployment" section (lines 398-402): "you do not edit the `environment:` - field… Promotion = running the matching job." The test - `test_consumer_guide_states_no_field_editing` asserts the no-editing - model. +The v1.25 engine uses the `kj scan` subcommand: -5. **Reference table (lines 329-330)** says sample contracts "use `@v1.19`" - but the sample contracts (`contracts/static-assets.yml`, - `contracts/microservice.yml`) don't carry `uses:` — the version pin - lives in the caller workflow. +``` +kyverno-json scan [flags] -### 1b. Environment-promotion lifecycle enforcement (feat) +Flags: + --labels strings Labels selectors for policies + --output string Output format (text or json) (default "text") + --payload string Path to payload (json or yaml file) + --policy strings Path to kyverno-json policies + --pre-process strings JMESPath expression used to pre process payload +``` -**Root cause confirmed by code inspection:** +The `KyvernoJsonEngine.evaluate()` implementation (REQ-293) invokes: +``` +kj scan --policy --payload --output json +``` +and parses the JSON `results[]` array. The `--pre-process` flag is +available for JMESPath pre-projection (noted for the meta-policy use case +where the payload is the merged PCR list and a pre-process expression +can index by `ruleId` — recorded as a future optimization, not used in +v1.25's initial implementation). -- `adapters/terraform/adapter.py:129` sets the Terraform state key to: - `spike/{stack_name}/{environment}/terraform.tfstate` -- `stack_name` = `contract["id"]` (stable across env changes, per - `core/contract_resolver.py:584`). -- When a consumer edits `environment:` from `dev` → `qa` on the same - contract `id`, the state key changes from `spike/assets/dev/` to - `spike/assets/qa/`. Terraform initializes a **fresh state file** in the - new env's state path. The prior env's resources remain live in AWS with - their state file untouched. **No destroy ever runs.** This orphans - resources. +Other subcommands (`kj jp`, `kj serve`, `kj playground`, `kj docs`) are +out of scope for v1.25. `kj serve` is the long-running web-app mode +(noted as a future consideration for lower-latency evaluation in the +Out of Scope section of REQUIREMENTS.md). `kj jp` is the JMESPath REPL — +useful for policy authoring/debugging, not invoked by the engine. -**The user's binding directive:** Editing `environment:` on a stable -`contract.id` is a valid promotion path (Shape A). The platform **must** -destroy the prior env's resources before building the new env. There must -be **no path that orphans resources** — fail closed if the destroy fails. +### 2.3 Policy structure (the `ValidatingPolicy` resource) -## 2. Existing codebase structure (integration points) +kyverno-json policies are Kubernetes-style resources (cluster-scoped) +belonging to the `json.kyverno.io` API group, kind `ValidatingPolicy`, +version `v1alpha1`: -### DynamoDB `nova-contracts` table (the prior-env source of truth) +```yaml +apiVersion: json.kyverno.io/v1alpha1 +kind: ValidatingPolicy +metadata: + name: # becomes the KJ_ ruleId prefix +spec: + rules: + - name: + identifier: # optional — path to the unique entry id + match: # assertion tree — which payload entries + any: # the rule applies to + - + exclude: # optional — exclude matching entries + any: + - + context: # optional — named bindings available to + - name: # the rule's assertions ($) + variable: + validate: + message: "" # optional per-rule message + assert: + all: # all assertions must hold + - check: + message: "" + # OR + any: # at least one assertion must hold + - check: +``` -- **Table:** `nova-contracts` (env var `CONTRACTS_TABLE`, default - `nova-contracts`). Defined in `core/lambda/contract_ingestor.py:25`. -- **Schema:** PK `consumerRepo` (S), SK `contractId#submittedAt` (S). - Attributes: `contractId`, `contract`, `environment`, `status`, - `submittedAt`. -- **Written by:** `_submit_contract()` at - `core/lambda/contract_ingestor.py:135-176`. The consumer's deploy - workflow submits the contract via the Lambda Function URL. -- **Read pattern for env-transition:** Query by PK `consumerRepo` + SK - begins_with `contractId#` + FilterExpression `status = "submitted"` → - sort by `submittedAt` desc → take the latest → read its `environment`. - This is the last-submitted env. For the last-*applied* env, a new - `#LAST_APPLIED` SK suffix is added (REQ-283). -- **Test pattern:** `tests/test_contract_ingestor.py:74-110` - (`moto_contracts_table` fixture) uses moto `mock_aws` to create the - table. The env-transition tests will mirror this pattern. +Key differences from K8s Kyverno policies: +- **Always cluster-scoped** — no `namespace` field. +- **No `forEach`, pattern operators, anchors, or wildcards.** Iteration + is done via the `~` projection modifier in assertion trees (see §2.4). +- **Assertion trees** with JMESPath expressions replace Kyverno's + pattern-matching syntax (see §2.4). -### Outbox writer (evidence events) +### 2.4 Assertion trees (the rule language) -- `core/outbox_writer.py` writes hash-chained evidence events to - `nova-outbox` table. PK `contractId`, SK `eventType#eventTs`. -- The env-transition destroy step emits a `nova.env.destroyed` event via - this writer (REQ-284c). Pattern: build an event dict with `contractId`, - `eventType: "ENV_DESTROYED"`, `environment: `, `ts`, then - call `write_event()`. +An `assert` declaration contains an `all` or `any` list. Each entry has a +`check` (the assertion tree — a nested JMESPath projection) and an +optional `message`. **All comparisons happen in the leaves of the tree.** -### Contract resolver (environment override) +A simple example (assert a pod doesn't use the default service account): +```yaml +validate: + assert: + all: + - message: "serviceAccountName 'default' is not allowed" + check: + spec: + (serviceAccountName == 'default'): false +``` -- `core/contract_resolver.py:460-483` `resolve()` accepts - `environment_override` — when set, it overrides the contract's - `environment` field **before** schema validation and interpolation - (D-088). This is the mechanism the destroy step uses to re-resolve the - contract against the prior env: `resolve(contract, env_override=prior_env)`. -- Already tested in `tests/test_deploy_workflow_env_input.py:35-52`. +The `(expression)` syntax evaluates a JMESPath expression; the result +becomes the current object for descendants; the leaf value is compared +to the expected value. -### run_platform.sh (where Step 0b goes) +**Iteration via the `~` modifier.** The `~` prefix on a key applies +descendant assertions to **each element** of an array/map individually +(rather than comparing the whole array). Given `foo.bar: [1,2,3]`: +```yaml +check: + foo: + ~.bar: # iterate each element + (@ < `5`): true # assert each element < 5 +``` +The `~index_name.bar` form binds the index (array) or key (map) to +`$index_name` for use in descendants. This is how v1.25 iterates +`resources[]` in the Stack IR policies (REQ-297) and +`planned_values.root_module.resources[]` in the plan-JSON policies +(REQ-300). -- `scripts/run_platform.sh` is the platform pipeline. Step 0 (lines 222-239) - is the environment onboarding check. Step 1 (lines 241-249) is contract - validation. **Step 0b goes between them** (after onboarding, before - validation). -- The destroy step mirrors the existing `--destroy` mode (lines 376-394): - `terraform init -reconfigure` + `terraform destroy -auto-approve`. The - difference: it runs against the *prior* env's state key, not the current - one. -- `CONTRACT_ID` is already set at line 217 (`NOVA_CONTRACT_ID` with a - default). `CONSUMER_REPO` needs to be derived from `GITHUB_REPOSITORY` - or a new `NOVA_CONSUMER_REPO` env var (REQ-286). +**Explicit bindings** via `->binding_name` allow descendants to refer +to a parent node via `$binding_name`. Built-in bindings: `$payload` +(the whole input), `$policy`, `$rule`. -### deploy.yml (consumer repo → platform) +**Escaping** via `\key\` prevents projection when a payload key collides +with the projection syntax. Not needed for Nova payloads (no `(key)` +fields), noted for completeness. -- `.github/workflows/deploy.yml:132` runs - `bash platform/scripts/run_platform.sh $MODE_FLAG $ENV_FLAG "${{ inputs.contract }}"`. -- To pass `NOVA_CONSUMER_REPO`, add `NOVA_CONSUMER_REPO=${{ github.repository }}` - as an env var on the "Run the platform pipeline" step (REQ-286). +### 2.5 Output shape (what `kj scan --output json` produces) -### Test patterns +The JSON output is a `results[]` array. Each result entry has (at +minimum): +- `policy`: the policy metadata.name +- `rule`: the rule name +- `result`: `"pass"` | `"fail"` | `"error"` | `"skip"` (lowercase) +- `message`: the assertion message (or engine error message) +- `resource`: the matched payload entry (the `identifier` value, or the + whole payload when no identifier/match) +- `namespace`/`kind`/`name`: K8s-style fields (present but empty for + non-K8s payloads — the K8s Kyverno adapter's evidence uses these; the + kyverno-json engine's evidence uses `assertion`/`jmespath` instead) +- `severity`: not present by default (kyverno-json does not assign + severities — the Nova policy author assigns severity via a Nova- + specific annotation; see §2.6) -- **Shell script assertions:** `tests/test_pipeline.py:79-95` reads the - script text and asserts substrings. The env-transition tests follow this - pattern for `run_platform.sh`. -- **DynamoDB mocking:** `tests/test_contract_ingestor.py:74-110` uses moto - `mock_aws` + `boto3.client` + `create_table`. The env-transition tests - follow this pattern. -- **Consumer guide assertions:** `tests/test_consumer_guide_per_env_section.py` - reads `docs/consumer-guide.md` text and asserts substrings. The updated - tests follow this pattern. +The `KyvernoJsonEngine._to_pcr()` translator (REQ-293) maps: +- `policy` → `ruleId` (prefixed `KJ_` per D-116) +- `result` → `result` (`pass`/`fail`/`error` → pass/fail/error; + `skip`/`skipped` → skipped) +- `message` → `message` +- `resource` → `resourceRef` + `evidence.resource` +- severity from the policy's `metadata.annotations` (see §2.6) +- `engine: "kyverno"` (per D-116 — no new enum value) -## 3. Persona assessment (v1.24) +### 2.6 Severity assignment (Nova convention) -This milestone has two distinct work territories: +kyverno-json does not natively assign severities to results. Nova's +confidence signal requires a `severity` per PCR (critical/high/medium/ +low/info). The v1.25 convention: each Nova policy file declares its +severity via a `metadata.annotations` field: -1. **Docs (P1):** `docs/consumer-guide.md` edits + test updates. This is - lead-developer territory (narrative/docs + test assertions). -2. **Platform code (P2):** `core/env_transition.py` (new), `scripts/run_platform.sh` - edits, `.github/workflows/deploy.yml` edit, `adapters/terraform/adapter.py` - doc comment. This is backend-engineer territory (Python + bash + YAML). -3. **Tests (P3):** `tests/test_env_transition.py` (new), `tests/test_run_platform_env_transition.py` - (new), `tests/test_consumer_guide_per_env_section.py` updates. Split: - backend-engineer for the env_transition + pipeline tests; lead-developer - for the consumer guide test updates. +```yaml +metadata: + name: forbid-public-ingress + annotations: + nova.cloudinit.dev/severity: high +``` -**Roster:** lead-developer (docs + guide tests) + backend-engineer (Python + -bash + YAML + pipeline tests). frontend-engineer stays deactivated (no UI). -data-engineer not needed (no schema changes — the `nova-contracts` table -already exists with the right shape; we only add a new SK suffix). No new -personas. +The `KyvernoJsonEngine._to_pcr()` reads this annotation from the loaded +policy YAML (not from the scan result — the result doesn't carry it) and +applies it to every result that policy produces. Default when absent: +`info`. This keeps severity in the policy (declarative, version- +controlled) rather than in the engine adapter (imperative). The +annotation key is `nova.cloudinit.dev/severity` (matches the existing +`nova.cloudinit.dev` namespace used in `schemas/tagging-standard.json`). -## 4. Key decisions logged +## 3. The four policy targets (v1.25 scope) -| ID | Decision | Confidence | Source | -|----|----------|------------|--------| -| D-201 | Both promotion shapes supported (A: edit+destroy, B: per-env callers) | 0.95 | User directive | -| D-202 | Prior-env source of truth = `nova-contracts` DynamoDB table | 0.85 | User directive + code inspection | -| D-203 | Detect-and-destroy at pipeline start (Step 0b) | 0.85 | User directive | -| D-204 | Fail closed on destroy failure (no orphan path) | 0.95 | User directive | -| D-205 | Cross-account destroy out of scope (same-account only) | 0.80 | CLARIFY A3 | -| D-206 | Env-transition destroy is NOT the HITL decommission pipeline | 0.85 | CLARIFY A5 | -| D-207 | Last-applied env recorded via `#LAST_APPLIED` SK in `nova-contracts` | 0.85 | RESEARCH §2 | -| D-208 | State key `spike/{id}/{env}/` stays as-is (correct for both shapes) | 0.90 | RESEARCH §1b | +### 3.1 Consumer contract JSON (REQ-295) -## 5. Pitfalls +The payload is the parsed contract dict (the raw YAML loaded as JSON). +Policies assert the `contract.schema.json` constraints declaratively: +`require-id-pattern` (JMESPath regex `^[a-z][a-z0-9-]{2,5}$` over +`id`), `require-env-in-enum` (`environment` in `["dev","qa","prod","dr"]`), +`require-infrastructure-min-1` (`length(infrastructure) > 0`), +`forbid-unknown-fields` (keys subset of the 4 allowed). These are the +declarative equivalent of the jsonschema constraints — they let Nova +apply its own compliance posture (e.g. forbid a specific env for a +specific consumer) on top of schema validity without editing the +jsonschema. -1. **Destroy needs the prior env's Terraform config, not the new env's.** - The destroy step must re-resolve the contract with - `environment_override=prior_env` so the emitted TF matches the prior - env's resources. If we resolve with the new env, the destroy plan won't - match the prior state → terraform tries to create, not destroy. -2. **`terraform init -reconfigure` is required** when switching state - backends between envs (if envs use different state buckets). The - `-reconfigure` flag tells Terraform to forget the previous backend config. -3. **The `deletion_protection` NFR (REQ-86) blocks destroy.** The destroy - step must resolve with `deletion_protection: false` injected (same as - decommission Step 2 in `scripts/run_decommission.sh:34-37`). Without - this, `terraform destroy` fails on `prevent_destroy` lifecycle blocks. -4. **DynamoDB may not be reachable in local/CI mode.** The detect step - must handle `ClientError` / `EndpointNotFound` gracefully → log warning - + return `None` (conservative). This is documented in REQ-282. -5. **The consumer guide test `test_consumer_guide_states_no_field_editing` - will fail after the Step 8 rewrite.** It must be updated in the same - phase as the guide edit (P1) or the test suite breaks. \ No newline at end of file +**Invocation point:** `core/contract_resolver.py` pre-resolve (REQ-296). +Early-fail: if a contract policy fails, the resolver still proceeds +(the confidence signal decides the gate, consistent with the existing +`--soft-fail` Checkov pattern) — but the failing PCRs are in the +`policy` input, which lowers the score. + +### 3.2 Resolved Target Stack IR JSON (REQ-297) + +The payload is the resolved Stack IR dict produced by +`core/contract_resolver.py` (the merged module outputs). Policies +assert over `resources[]` (the array of resolved resources): +`require-tagging-standard` (every resource's `tags` has `nova:owner` + +`nova:environment` — ports +`adapters/terraform/policy/custom_rules/nova_tagging.py`), +`forbid-public-ingress` (no resource has `public_ingress: true` — the +v1.0 demo rule, now declarative), `require-encryption-by-default` (every +S3/EBS/KMS-aliased resource carries encryption config — ports the v1.8 +D-encryption-default rule). The `~` modifier iterates `resources[]`. + +**Invocation point:** `core/contract_resolver.py` post-resolve (REQ-298). +Additive — the resolver's return values and exceptions are unchanged; +the PCRs are appended to the contract-policy PCRs. + +### 3.3 Terraform plan JSON (REQ-300) + +The payload is `terraform show -json ` output. Policies assert +over `planned_values.root_module.resources[]`: +`forbid-plaintext-secrets` (no `aws_db_instance.password` / +`aws_iam_user.login_profile.password` in plaintext — ports +`CKV_AWS_41/45/46`), `forbid-iam-wildcard` (no `Action: "*"` or +`Resource: "*"` in `aws_iam_policy.PolicyDocument` — ports +`CKV_AWS_1/40`), `require-kms-reference` (KMS keys referenced by alias, +not inline key material — ports `CKV_AWS_7/33`). These are declarative +**mirrors** of `checkov_adapter.py:RULE_MAP` — the Checkov rule stays +the source of truth for `terraform_plan` scanning; the kyverno-json +policy covers the same plan JSON with a different rule language +(defense-in-depth against engine drift). + +**Invocation point:** `run_platform.sh` Step 5 (REQ-301). After +Checkov/Wiz produce raw PCRs, the script runs `kj scan` over the plan +JSON; both PCR lists concatenate into the confidence signal's `policy` +input. When `which kj` is false, the script logs and proceeds with the +Checkov/Wiz list only. + +### 3.4 PolicyCheckResult records (meta-policies, REQ-303) + +The payload is the **merged** `list[PolicyCheckResult]` produced by +checkov + wiz + the plan-JSON policies. This is the most novel target — +kyverno-json policies over the policy results themselves. +`block-on-any-critical` asserts no PCR has `severity: "critical"` + +`result: "fail"`; if any does, the meta-policy emits a `fail` PCR with +`ruleId: "KJ_META_BLOCK_CRITICAL"` and severity `critical`. This is the +declarative source of truth for "critical = block" (D-119 — the +`confidence_signal.py` `PENALTY["critical"]: None` hard-override stays +as defense-in-depth). `tagging-rules-agree` cross-checks the Checkov +`NOVA_TAG_NAMING` result against the kyverno-json +`KJ_REQUIRE_TAGGING_STANDARD` result by `resourceRef`; divergence emits +an `error` PCR (D-118). + +**Invocation point:** after the three target policies (contract/stack- +IR/plan-JSON) produce their PCR lists, the merged list is the payload +for the meta-policies. The meta-policy PCRs are appended to the merged +list, which is what the confidence signal consumes. + +## 4. The `PolicyEngine` swap boundary + +### 4.1 Protocol shape (REQ-291) + +A Python `Protocol` (PEP 544 — structural subtyping, no inheritance): +```python +class PolicyEngine(Protocol): + @property + def name(self) -> str: ... + def is_configured(self) -> bool: ... + def evaluate(self, payload: dict | str, policy_dir: Path, + contract_id: str) -> list[dict]: ... +``` +`list[dict]` (not `list[PolicyCheckResult]` — there's no dataclass; the +schema is enforced via `jsonschema` validation in tests, matching the +existing adapter pattern). The registry selects the active engine from +`config.json.policy.engine`. A `NullEngine` is the fallback when the +`policy` key is absent (emits `SKIPPED` — backward compatibility for +tests that don't set the key). + +### 4.2 The OPA-equivalent surface (future swap) + +OPA (Open Policy Agent) is the most likely future replacement. The +mapping: +| Nova `PolicyEngine` member | kyverno-json impl | OPA equivalent | +|---|---|---| +| `name` | `"kyverno-json"` | `"opa"` | +| `is_configured()` | `which kj` | `which opa` | +| `evaluate(payload, policy_dir, contract_id)` | `kj scan --policy --payload -o json` | `opa eval -d -i 'data.nova.<...>'` | +| Policy file format | `ValidatingPolicy` (YAML) | Rego (`.rego`) | +| Result shape | `results[]` (pass/fail/error/skip) | `result` (set of violations) | +| Severity | Nova annotation `nova.cloudinit.dev/severity` | Nova convention (Rego `metadata` or a wrapper) | + +The protocol is minimal (3 members) specifically so the OPA +implementation is a known quantity: an `OpaEngine` class that shells to +`opa eval`, translates the Rego violation set to PCR dicts, and +implements `is_configured()` via `which opa`. The policy *files* would +need rewriting (Rego, not ValidatingPolicy) — but the protocol, the +registry, the confidence signal, and the PCR schema are all untouched. +This is the swap boundary the user asked for ("Implemented as an +adapter since we might one day decide to replace it with something else +like OPA"). + +### 4.3 Why not a full plugin registry? + +A `setuptools` entry-point plugin registry (like checkov's +`--external-checks-dir`) was considered and rejected: Nova has 1 active +engine today (kyverno-json) and at most 2 in the foreseeable future +(kyverno-json + OPA). A `Protocol` + `dict` registry in +`core/policy_engine.py` is the right weight — discoverable, typed, +testable, and ~40 lines. An entry-point registry adds packaging +complexity (entry-point metadata, version resolution) for no gain at +this scale. The `register(name, factory)` method on the registry is +the extension point if a future milestone needs runtime plugin +discovery. + +## 5. Latency / MTTR impact (G-Q3 anticipation) + +NORTH_STAR.md MTTR target: < 60s p95. `run_platform.sh` Step 5 today +runs Checkov over the terraform plan (typically 5-15s for a small +stack). Adding `kj scan` over the same plan JSON adds: +- Process spawn: ~50ms (Go binary startup) +- Policy load: ~20ms (a handful of YAML files) +- Assertion evaluation: ~100-500ms (JMESPath over a small plan) +- Total: < 1s for a typical Nova stack + +The kyverno-json pass runs **in parallel** with Checkov (REQ-301 — the +script launches both and waits on both), so the wall-clock impact is +`max(checkov_time, kj_time)` ≈ checkov_time (kj is faster). The +contract + stack-IR policies run during resolve (already a fast step). +Meta-policies run over the merged list (in-memory, < 10ms). **No +measurable MTTR impact** is expected. This will be verified in P3 +VERIFY with a timing assertion. + +## 6. "Platform functions without AI" tenet (G-Q1 / D-120) + +kyverno-json is deterministic (same policy + payload → same result, +every run). It is not an LLM, not a probabilistic model, not a +"judgement" engine. The NORTH_STAR.md tenet ("the platform functions +without AI — 'AI decisions' are really automated decisions") +distinguishes AI (non-reproducible) from automation (reproducible). +kyverno-json is the latter. Adding it is **more** aligned with the +tenet than the current imperative Python in `core/env_transition.py` +and `core/regression_verify.py`, because the policy is declarative +(visible, auditable, version-controlled) rather than imperative (logic +hidden in function bodies). The `is_configured()` guard ensures the +platform functions without the binary (graceful skip → `SKIPPED` PCR +→ confidence signal proceeds). + +## 7. ECS policy catalog overlap (prior art) + +The kyverno-json catalog ships ECS policies that overlap with Nova's +L1 modules: `ecs-cluster-enable-logging`, `ecs-cluster-required- +container-insights`, `ecs-service-public-ip`, `ecs-service-required- +latest-platform-fargate`, `ecs-task-definition-fs-read-only`. These are +**reference policies**, not drop-in Nova policies — they target the +AWS ECS API shape (`type: aws_ecs_service` etc.), not Nova's Stack IR +shape. v1.25 policies target the Nova IR (REQ-297) and the terraform +plan JSON (REQ-300), not the raw AWS API. The catalog is useful as +prior art for JMESPath patterns over ECS resources — the +`ecs-service-public-ip` policy's `contains('$allowed-values', +@.assign_public_ip)` pattern informs the Nova `forbid-public-ingress` +policy shape. No catalog policies are imported directly in v1.25. + +## 8. Risks & mitigations + +| Risk | Likelihood | Impact | Mitigation | +|---|---|---|---| +| `kj` binary not in CI image | medium | blocks P3+ tests | `is_configured()` guard + `pytest.skip` + `scripts/install-kyverno-json.sh` | +| kyverno-json output shape changes across versions | low | breaks `_to_pcr()` | pin `@latest` to a known-good commit in `install-kyverno-json.sh` after P1 smoke; defensive parsing (malformed → `error` PCR, not exception) | +| Policy explosion (4 targets × N rules) | medium | maintenance load | wave ordering (PLAN); policies co-located per target dir; meta-policy cross-check keeps the set auditable | +| Checkov + kj tagging-rule drift | medium | false `error` PCRs | `tagging-rules-agree` meta-policy emits `error` on divergence (visible, not silent); the Checkov rule stays source of truth for HCL, kj for IR | +| OPA swap turns out harder than the protocol implies | low | future milestone rework | RESEARCH §4.2 documents the OPA-equivalent surface; the protocol is the contract, not the implementation | +| `--pre-process` needed for meta-policies but undocumented behavior | low | meta-policy bugs | v1.25 meta-policies use plain assertion trees over the PCR list (no pre-process); `--pre-process` noted as a future optimization only | + +## 9. Assumptions (logged, full autonomy) + +- A1: `kj scan --output json` produces a stable `results[]` array shape. + Will be verified in P1 smoke test (`_smoke.json` policy + a trivial + payload); if the shape differs, `_to_pcr()` is adjusted defensively + (malformed → `error` PCR). Confidence: 0.85. +- A2: The `nova.cloudinit.dev/severity` annotation convention is + read by the engine from the policy YAML (loaded once per evaluate() + call). kyverno-json does not validate unknown annotations — they pass + through. Confidence: 0.90. +- A3: The `~` projection modifier iterates `resources[]` in the Stack + IR and `planned_values.root_module.resources[]` in the plan JSON + correctly. Verified in P2/P3 tests. Confidence: 0.85. +- A4: `go install` works in the CI image (Go toolchain available or + installable). If not, the binary-release download path is the + documented fallback in `install-kyverno-json.sh`. Confidence: 0.80. +- A5: The `NullEngine` fallback (when `policy` key absent in + config.json) keeps all existing tests passing — they don't set the + key, so they get `NullEngine` → `SKIPPED` PCRs → confidence signal + proceeds with `policy` input `[SKIPPED]` → per-input score 1.0 + (skipped counts as pass in `_per_input_score`). Confidence: 0.95 + (verified against `confidence_signal.py:84-89`). + +## 10. Decisions referenced + +D-115 (install path), D-116 (engine enum reuse), D-117 (adapter +signatures unchanged), D-118 (tagging cross-check), D-119 (critical- +override defense-in-depth), D-120 (deterministic not AI). See +CLARIFY.md for the full resolution text. + +## 11. Architecture updates (deferred to RESEARCH-stage file edits) + +- `.ciagent/ARCHITECTURE.md` gains §12.7 "Policy Engine Registry" with + the registry diagram. Deferred to the RESEARCH commit (this file's + commit) — the section is authored as part of this research. +- `schemas/README.md` notes `engine: "kyverno"` is shared by the K8s + adapter and kyverno-json (distinguished by `ruleId` prefix). +- `modules/STANDARDS.md` gains a "Policy authoring standard" section + (P4, REQ-307). +- `docs/METRICS.md` notes the policy engine is swappable (P4, REQ-307). \ No newline at end of file