Jon Chery
a63c85bc51
Relocate completed-milestone history to .ciagent/archive/ (byte-identical
snapshots of PROJECT/REQUIREMENTS/ROADMAP/ARCHITECTURE pre-compression +
verbatim moves of REVIEW/AUDIT/VERIFY/PRE_MORTEM). Slim the in-place files
to retain only active-milestone (v1.26) + immediate-predecessor (v1.25)
context + durable vision/tenets/scope/RACI/capability-status/load-bearing
decisions. REGRESSION_REPORT.{json,md} stay in place (live read/write
targets of core/metrics/collector.py + core/regression_verify.py).
Working context: 11,164 → 4,152 lines (~63% reduction). Archive preserves
8,615 lines. Lossless via relocation + git history. No test regressions
(761 passed; same 3 pre-existing failures as baseline).
---ci---
project: acdl
phase: 2
milestone: v1.26
status: execute
lessons:
- REGRESSION_REPORT.{json,md} are live operational files (read by
core/metrics/collector.py + core/regression_verify.py) — must NOT be
archived. Pre-flight grep for code references to candidate archive
paths before any move.
- test_no_purged_loaded_term scans .ciagent/PROJECT.md + CLARIFY.md +
docs/ for 'penetrat' — slimmed files must not reintroduce it. Historical
description of the purge ('removed the term ...') is safe in ROADMAP.
- Git rename detection (R) works for pure file moves; snapshot-then-slim
shows as A + M. Both preserve history.
---/ci---
87 lines
4.0 KiB
Markdown
87 lines
4.0 KiB
Markdown
# VERIFY — P1 engine-core (v1.25)
|
|
|
|
> 4-layer verify gate: structural, behavioral, security, quality.
|
|
> Phase: P1. Requirements: REQ-291..294, 308, 309. Result: PASS.
|
|
|
|
## Structural
|
|
|
|
- `core/policy_engine.py` exists, implements `PolicyEngine` Protocol
|
|
(PEP 544, `@runtime_checkable`), `PolicyEngineRegistry` with
|
|
`register()` + `get_engine()`, `NullEngine` fallback.
|
|
- `adapters/kyverno-json/kyverno_json_engine.py` exists, exports
|
|
`KyvernoJsonEngine` with `name`, `is_configured()`, `evaluate()`.
|
|
- `adapters/kyverno-json/__init__.py` loads the engine by file path
|
|
(the dir name has a hyphen — not a valid Python package name).
|
|
- `adapters/kyverno-json/policies/_smoke.json` exists (trivial policy
|
|
for round-trip validation).
|
|
- `scripts/install-kyverno-json.sh` exists (go install kj@latest).
|
|
- `.ciagent/config.json` has the `policy` object
|
|
(`engine: kyverno-json`, `policy_root`).
|
|
- `.gitea/workflows/ci.yml` + `.github/workflows/ci.yml` have the
|
|
Go + kj install step (best-effort, tests skip when kj absent).
|
|
- `tests/test_policy_engine.py` (10 tests) +
|
|
`tests/test_kyverno_json_engine.py` (16 tests) exist.
|
|
|
|
## Behavioral
|
|
|
|
- `pytest tests/test_policy_engine.py tests/test_kyverno_json_engine.py`:
|
|
**24 passed, 2 skipped** (kj not installed — expected;
|
|
`pytest.skip("kj not installed")`).
|
|
- `NullEngine` satisfies the `PolicyEngine` Protocol (G-Q8a —
|
|
`isinstance(NullEngine(), PolicyEngine)` is True). Proves the swap
|
|
boundary is real without implementing OPA.
|
|
- `KyvernoJsonEngine.is_configured()` returns `False` when
|
|
`which kj` is absent → `evaluate()` returns a single
|
|
`KJ_ENGINE_NOT_CONFIGURED` SKIPPED PCR (distinct `ruleId` from
|
|
NullEngine's `NULL_ENGINE_INACTIVE` — G-Q4).
|
|
- PCR records validate against `schemas/policy_check_result.schema.json`
|
|
(via `jsonschema.validate` in tests).
|
|
- Defensive parsing: malformed kyverno-json output → `error` PCR
|
|
(`KJ_ENGINE_ERROR`), never an exception.
|
|
- Severity annotation reading (G-Q10a): policies with
|
|
`nova.cloudinit.dev/severity: high` produce PCRs with `severity: high`;
|
|
policies without the annotation default to `info`.
|
|
- Registry: `get_engine()` returns the configured engine; unknown
|
|
engine name raises `KeyError`; `policy` key absent → `NullEngine`.
|
|
- No regression: `pytest tests/test_confidence_signal.py
|
|
tests/test_adapter.py tests/test_checkov_adapter.py
|
|
tests/test_kyverno_adapter.py tests/test_contract_resolver.py` —
|
|
**132 passed** (unchanged).
|
|
|
|
## Security
|
|
|
|
- No new secrets, no new network calls in the engine core (the engine
|
|
shells to a local binary; the binary makes no network calls for
|
|
`scan`).
|
|
- `is_configured()` guard ensures the platform runs without the binary
|
|
(no hard dependency that could be exploited as a DoS vector).
|
|
- The engine writes the payload to a temp file (`tempfile.NamedTemporaryFile`)
|
|
and unlinks it in a `finally` block (no leftover payload on disk).
|
|
- No `shell=True` in the `subprocess.run` call (command is a list —
|
|
no shell injection surface).
|
|
|
|
## Quality
|
|
|
|
- `python3 -m py_compile` passes on all new Python files.
|
|
- The `PolicyEngine` Protocol is minimal (3 members) — the swap
|
|
boundary is the moat (NORTH_STAR Strategic Objective #2).
|
|
- The `NullEngine` proves a second implementation exists (structural
|
|
conformance) — the OPA swap is a known quantity (RESEARCH §4.2).
|
|
- Tests use `pytest.skip` when `which kj` is absent, so the CI matrix
|
|
passes with or without the binary (the suite is green in both cases).
|
|
|
|
## Must-have checklist
|
|
|
|
- [x] `PolicyEngine` Protocol + `PolicyEngineRegistry` + `NullEngine`
|
|
(REQ-291)
|
|
- [x] `config.json.policy` object (REQ-292)
|
|
- [x] `KyvernoJsonEngine` adapter (REQ-293)
|
|
- [x] `__init__.py` + `_smoke.json` + `install-kyverno-json.sh` + CI
|
|
install (REQ-294)
|
|
- [x] `test_policy_engine.py` — protocol conformance, registry,
|
|
NullEngine fallback (REQ-308)
|
|
- [x] `test_kyverno_json_engine.py` — PCR schema validity, defensive
|
|
parsing, skip-without-kj (REQ-309)
|
|
|
|
**Verdict: PASS** — all P1 must-haves met, no regressions, 24 new
|
|
tests pass (2 skip-without-kj), 132 existing tests unchanged. |