The v1.25 kyverno-json engine adapter and policies were authored but never
validated against the real `kj` binary — the test suite
`pytest.skip("kj not installed")` when `kj` was absent, masking the bug.
With `kj` v0.0.3 now installed, the 3 failing-fixture tests
(stack-ir/plan-json/regression) showed 0 fails (all passed falsely). Root
causes (3 substrate bugs) and fixes:
1. ENGINE — bare-list output format. `kj scan --output json` emits a bare
JSON LIST at the top level (NOT `{"results": [...]}`); each entry has
`resource` + `results[].rules[]` with `violations[]` (fail) / `error`
string (eval error) / neither (pass). The v1.25 `_translate` did
`out.get("results", [])` on a dict → `out` is a list → returned `[]` →
emitted a single KJ_NO_RESULTS pass PCR. Rewrote `_translate` to parse
the real v0.0.3 nested shape (policy.metadata.name, rule.name,
violations[].errors[].field/detail/value). Future-proofs to also accept
the legacy dict shape. Preserves RESULT_MAP, severity-from-annotation,
is_configured(), _skipped_not_configured, _error_pcr, the temp-file
payload write, and the subprocess invocation.
2. ENGINE — `.json` policies not loaded by `kj`. The upstream loader
(pkg/policy/load.go) uses fileinfo.IsYaml() which only matches
`.yaml`/`.yml` — `.json` files are silently skipped (0 policies).
Nova policies are authored as `.json` (TestPolicyFilesExist asserts the
filenames). Added `_materialize_yaml_policy_dir`: mirrors the source
tree to a temp dir, copying every `.json` policy to a `.yaml` twin
(JSON is a valid YAML subset, verified against kj v0.0.3). Source
`.json` files remain untouched.
3. POLICIES — `validate` wrapper + check syntax. Removed the `validate`
wrapper from all 16 policies (kj v0.0.3 ignores `validate`-wrapped
rules — `assert` goes directly under the rule). Fixed the check syntax:
a check entry is `expression: expected_value` (e.g.
`(regex_match(..., @)): true`), not `field: (expression)` (which
compared a bool to nothing → "types not comparable"). For per-resource
checks over stack-IR/plan-JSON, `~.resources` (descendant anchor) is
required for per-element iteration; a plain path applies to the whole
array. For type-scoped rules (s3/ebs encryption, iam/db/kms), the type
guard is folded into the expression (`type == '...' && !<has-prop>`)
so non-matching resources short-circuit to false. cap-013 dedup uses
`max(map(&length(@), values(group_by(adapters, &@)))) == `1`` (no
`duplicates` JMESPath fn exists). Preserved all policy metadata
(apiVersion, kind, metadata.name, severity + title annotations) —
TestPolicyValidity/TestPolicyFilesExist still pass.
INSTALL SCRIPT — the v1.25 `go install .../cmd/kj@latest` failed: the
`cmd/kj` path does not exist in v0.0.3 (upstream produces a binary named
`kyverno-json`). Fixed to `go install github.com/kyverno/kyverno-json@latest`
+ symlink `kyverno-json` → `kj` (GOBIN and /usr/local/bin fallbacks).
Idempotent: short-circuits when `kj` is already on PATH and working.
Verification: `which kj` → /usr/local/bin/kj; `kj version` → v0.0.3.
test_kyverno_json_engine + test_stack_ir_policies + test_plan_json_policies
+ test_meta_policies + test_regression_policies: 36 passed, 0 skips
(_require_kj no longer skips). Full suite (excluding pre-existing hang in
test_verify_regression_mode.py): 776 passed, 6 failed — all 6 failures are
pre-existing (confirmed by stashing this commit's diff and re-running);
the only in-scope-acceptable failure is
test_module_standards.py::test_all_l1_have_required_files (dynamodb
extension drift, data-engineer's later wave).
---ci---
project: acdl
phase: 3
milestone: v1.26
status: execute
wave: W0.5
---
P1-1 (correctness): run_platform.sh Step 5c now invokes the meta-policies
(block-on-any-critical, tagging-rules-agree) over the merged PCR list after
Step 5b, appending the meta-PCRs to pcr.json before the confidence signal
runs. Closes the D-118/D-119 declarative-critical-block gap (the
confidence_signal.py hard-override stays as defense-in-depth).
P1-2 (testing): test_meta_policies.py behavioral assertions strengthened —
test_no_critical_passes asserts no fails, test_critical_fail_present asserts
a non-pass result, test_pcrs_validate_against_schema validates output.
P1-3 (correctness): _smoke.json assertion rewritten from malformed
'{{ to_string(@) }}' to valid JMESPath '(regex_match(...))'.
---ci---
project: acdl
phase: 5
milestone: v1.25
status: execute
phase_role: final
---/ci---
Jon Chery