continuous-intelligence/acdl
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 128 Wiki Activity
Files
c19cc68d156e40b5effccf8de5453e915014fb18
acdl/.ciagent/GRILL.md
T
CIAgent Orchestrator 0789c27ca2
Nova Slides Render / render (push) Failing after 14m27s
Details
docs(P00): complete v1.29 pre-execution — SPECIFY+CLARIFY+RESEARCH+PLAN+GRILL+MVP/UX 
---ci---
project: acdl
phase: 0
milestone: v1.29
status: complete
---/ci---
2026-08-20 05:00:35 +00:00

19 KiB
Raw Blame History

GRILL — v1.28 CLI Canonicalization + Identity Layer

Adversarial review of the v1.28 SPECIFY + CLARIFY + RESEARCH + PLAN. Griller: ci-griller subagent. Autonomy: full. All 9 axes reviewed; every claim verified against the live codebase.

Overall verdict: PROCEED-WITH-CONDITIONS · Confidence 0.76

The plan is fundamentally sound — architecture correct, re-mapping clean (no ID collisions), technical depth accurate (DER→raw, strong- read revocation, stdin TTY), highest-risk item (kj binary) has a Fargate fallback. Not unfeasible, not over-scoped beyond an agent-driven repo's capacity, not security-broken by design.

3 critical conditions (must-fix before P1) + 16 tracked conditions. No escalations (all axes ≥ 0.70 confidence).

Axis verdicts

Axis Verdict Confidence Critical condition
§1 Feasibility PROCEED-WITH-CONDITIONS 0.82 C-1.1 KMS asym verify; C-1.2 Argon2 fail-closed test
§2 Scope PROCEED-WITH-CONDITIONS 0.74 C-2.1 fold P5 into P4; C-2.2 P4 overload
§3 Cost PROCEED-WITH-CONDITIONS 0.70 C-3.1 cost estimate; C-3.2 CodeArtifact P1 task
§4 Schedule PROCEED-WITH-CONDITIONS 0.76 C-4.1 P4 critical path; C-4.2 per-phase exit
§5 Technical Depth PROCEED-WITH-CONDITIONS 0.80 C-5.1 ABAC shape; C-5.2 JWS KDF
§6 Operational Readiness PROCEED-WITH-CONDITIONS 0.72 C-6.1 ABAC fail-closed; C-6.2 threat model; C-6.3 ops guide
§7 Security Posture PROCEED-WITH-CONDITIONS 0.73 C-7.1 ABAC fail-closed; C-7.2 Argon2 params; C-7.3 cred file
§8 Dependency Risk PROCEED-WITH-CONDITIONS 0.83 C-8.1 CodeArtifact P1; C-8.2 pin kj version
§9 Re-mapping Integrity PROCEED-WITH-CONDITIONS 0.84 C-9.1 traceability fix; C-9.2 INV audit

Critical conditions (the 3 must-fix-before-P1)

🔴 C-6.1 / C-7.1 — ABAC fail-closed

The token-vend Lambda's behavior on kj absence/error is unspecified. Without fail-closed, INV-17 is documentation, not a runtime guarantee — a kj load failure would bypass the ABAC gate (every PAT gets a token). Fix applied to PLAN.md P4 Wave 4 Task 4.1: "If KyvernoJsonEngine.is_configured() returns false or evaluate() raises, return 403 + audit token.vend.denied (reason: abac_eval_failed). Never fail open. Test: tests/test_abac_fail_closed.py."

🔴 C-5.2 — JWS-from-PAT key derivation

REQ-332's AC ("public key derivable from the PAT") is unimplementable without a specified KDF. A PAT is a JWT, not a keypair. Fix applied to PLAN.md P2 Wave 2 Task 2.3 + REQ-332 AC: the JWS uses HMAC-SHA256 with a key derived via HKDF-SHA256(PAT_bytes, salt='nova-local-attestation', info='jws-signing-key') → 32-byte symmetric key. The "public key derivable" AC is re-interpreted: the verification key is derived from the PAT via the same KDF (the PAT is the shared secret). This is a symmetric scheme, not asymmetric.

🔴 C-9.1 — Traceability drift

REQUIREMENTS.md §v1.28 traceability table mapped 16 REQs to P2; PLAN.md splits them across P2/P3/P4/P5/P6. Fix applied to REQUIREMENTS.md — traceability table updated to match PLAN.md phase structure.

Tracked conditions (16 — applied to PLAN.md as amendments)

  • C-1.1 KMS asymmetric key verification before P4 Wave 3 (one aws kms create-key --key-spec ECC_NIST_P256 call).
  • C-1.2 Argon2 fail-closed test in P3 Wave 2 (Lambda returns 503 on ImportError, not a crash or pure-Python hash).
  • C-2.1 Fold P5 (idp-setup) into P4 as P4 Wave 8 → reduces to 6 execution phases (P1..P6, P7 = final). Applied.
  • C-2.2 P4 is a double-length phase; acknowledged in P4 header.
  • C-3.1 Cost envelope subsection added to PLAN.md.
  • C-3.2 / C-8.1 CodeArtifact provisioning = P1 Wave 0 task with binary go/no-go gate; Gitea wheel index fallback documented.
  • C-4.1 P4 flagged as critical-path phase (kj spike = highest- probability schedule slip; Fargate = +1 week).
  • C-4.2 Per-phase exit criteria added to PLAN.md.
  • C-5.1 requested_claims = list of claim names (the policy asserts the subject is allowed to request those claims).
  • C-6.2 Threat model (REQ-347) adds: JWKS DDoS surface, PAT theft
    • max TTL (≤24h dev, ≤1h service-account), ABAC fail-closed, INV-18..21 compression audit.
  • C-6.3 Operator guide (REQ-345) adds: KMS rotation, layer update, PITR restore, emergency PAT revocation.
  • C-7.2 Argon2id parameters: t=3, m=65536 KiB, p=1 (OWASP min).
  • C-7.3 ~/.nova/credentials.json stores OIDC token + PAT metadata (jti, exp, type), NOT the raw PAT.
  • C-8.2 kj pinned to a specific release + SHA256 recorded.
  • C-9.2 Threat model includes INV-18..21 compression audit (verify spec's attestation invariant semantics are captured by INV-15/16/17 + REQ-332).

Escalations

None. All 9 axes resolved at confidence ≥ 0.70. No human escalation required (full autonomy).

Grill complete

The plan proceeds with the 3 critical fixes and 16 tracked conditions applied to PLAN.md + REQUIREMENTS.md. The binding decisions above are the authoritative grill record. Next: MVP/UX CHECK → SHIP phase 0.

GRILL — v1.29 Reposplit + Identity Layer Bring-Live

Adversarial red-team review of the v1.29 SPECIFY + CLARIFY + RESEARCH + PLAN. Griller: CIAgent griller (red-team persona). Autonomy: full. All 9 review axes grilled; every claim verified against the live codebase (publish.yml, kj-version.txt, nova/idp/setup.py, existing v1.28 test files). Date: 2026-08-20.

Overall verdict: PROCEED-WITH-CONDITIONS · Confidence 0.72

The plan is architecturally sound and the in-acdl scope is well-bounded. The scope split (Terraform out-of-band in nova-platform-ops, acdl authors publish/scrub/archive/guide/consumer-bump) is the correct boundary per Vision §4. The technical depth is accurate (D-239 ECR tag correction, D-240 Terraform precondition floor, CloudFront OAC pitfall, ECR tag mutability → pin-by-digest). The cost envelope is realistic.

However, the covered-reference pattern — as currently structured — is a deferred-trust assertion for 14 of 17 requirements. The plan ships REQ-355..366 + 371 as "complete" on the strength of a markdown pointer (the operator guide's cutover-gate section) to CI in a repo that does not yet exist and has no CIAgent presence. The M1.5 verification gate, the one surface acdl genuinely owns, can be authored-but-never-run-green and the milestone still ships. Four critical fixes convert "documented" into "evidenced-by-operator- attestation-in-the-guide-which-acdl-audits-at-P6."

4 critical fixes (must apply before EXECUTE) + 6 tracked conditions. No escalations (all axes resolved at confidence ≥ 0.60; the user confirmed the binding verdict on the covered-reference pattern).

Axis verdicts

Axis Verdict Confidence Forcing finding
§1 Feasibility PROCEED-WITH-CONDITIONS 0.70 KJ-SOURCE: kj v0.0.3 source repo unverified by RESEARCH (CF-1)
§2 Scope PROCEED-WITH-CONDITIONS 0.74 Covered-reference = deferred-trust for 14/17 REQs (G-1 + CF-2)
§3 Cost PROCEED 0.82 $30-40/month realistic at pilot volume; no hidden budget shock
§4 Requirements coverage PROCEED-WITH-CONDITIONS 0.76 All REQs mapped; covered-reference verification surface weak (CF-2)
§5 Technical risks PROCEED-WITH-CONDITIONS 0.72 KJ-STATIC mitigation sound; KJ-LOCKSTEP by-construction good; M1.5 gate not enforced (CF-1)
§6 Testability REJECT-AS-WRITTEN → PROCEED-WITH-CONDITIONS 0.66 "Verified via cutover gates in operator guide" is a punt absent CF-1/CF-2/CF-3/CF-4
§7 Security PROCEED-WITH-CONDITIONS 0.68 INV-18 (AuthType=AWS_IAM), TFM-HITL, IAM-NARROW unverifiable from acdl (CF-2)
§8 Timeline/sequencing PROCEED 0.80 P1→P2 ordering safe (acdl-local scrub); P5 smoke hedges (CF-3)
§9 Adversarial PROCEED-WITH-CONDITIONS 0.70 Dominant silent-failure = M1.5 never runs green (CF-1 addresses)

Critical fixes (must apply before EXECUTE)

🔴 CF-1 — M1.5 green is a HARD P6 milestone-ship gate; spike extended

Finding: P1 authors the M1.5 gate tests (Wave 3) but P1's exit criterion explicitly marks the live KMS round-trip as "covered- reference, runs in nova-platform-ops CI." P6 ships the milestone with no requirement that M1.5 ever ran green. The dominant silent-failure path (user-confirmed): M1.5 never runs green → 14 REQs ship "complete" on paper while Nova-idp is not live.

Fix (binding):

  1. P6 Wave 2 (ciagent-ship) MUST NOT ship v1.28.6 until the operator guide (docs/operator-guide-platform-ops.md) contains an operator-attested "M1.5 Verification Gate Result" row recording: (a) the 8-item spike all-green on 3 consecutive rebuilds in nova-platform-ops CI; (b) the rebuild run IDs / commit SHAs; (c) the operator attestor identity. The P6 audit step (Wave 1) verifies this row exists + is non-empty. Absent the row → P6 blocks → escalate.
  2. The M1.5 8-item spike (PLAN Happy Path §3.3 Edge 5) is EXTENDED from 8 to 12 items by adding:
    • Item 9 (JWKS-EDGE-ONLY): direct JWKS Function URL GET (bypassing CloudFront) returns 403; via-CloudFront GET returns 200. Proves AuthType: AWS_IAM + OAC pinning (INV-18). Without this, the AuthType: NONE pitfall (RESEARCH §4) is undetected.
    • Item 10 (IAM-NARROW): aws iam get-role-policy on the OIDC role asserts no Action: "*" and no Resource: "*" (REQ-360).
    • Item 11 (TFM-HITL): a terraform apply workflow_dispatch triggered by the PR author is rejected (exit non-zero, gitea.triggering_actor == PR author); a dispatch by a distinct user proceeds (REQ-357, RESEARCH §10).
    • Item 12 (rollback drill): revert nova_platform_version pin → terraform apply → assert the prior ECR digest runs (proves D-236 rollback; guards against ECR tag mutability, RESEARCH §2).

Binding decision G-2.1: the covered-reference pattern is accepted as a verification surface only with CF-1 applied. M1.5 green (evidenced by operator attestation in the guide) is the ship gate.

🔴 CF-2 — Covered-reference REQs gated by operator-attested evidence rows

Finding: 14 of 17 REQs (355..366, 371) are "verified via cutover gates in the operator guide" (CLARIFY G4). This is a deferred-trust assertion: if nova-platform-ops is never built, or builds the wrong thing, or its CI silently passes, the REQs ship "complete" on the strength of a markdown pointer. The user confirmed this is a deferred-trust assertion, not a verification.

Fix (binding): The operator guide (P4 Wave 1 Task 1.1) "Cutover Gates" section MUST list each covered-reference REQ with: (a) the gate entry (M1/M1.5/M2); (b) the verification command; (c) a placeholder "Result" column. The P6 audit step (Wave 1) verifies that every covered-reference REQ has a non-empty, green "Result" entry (operator-attested). A REQ with an empty or red Result → P6 blocks. This converts "documented" to "evidenced-by-operator-attestation- audited-by-acdl-at-P6."

Binding decision G-1: the covered-reference pattern is accepted as a verification surface with CF-1 + CF-2 applied. Without them, it is a punt and the grill would REJECT.

🔴 CF-3 — P5 smoke test must run against a real v1.29.x tag (no hedge)

Finding: P5 bumps the consumer deploy.yml @v1.25 → @v1.29 and runs a smoke test "against the v1.29 publish artifacts." But publish.yml triggers on v1.29.* tags (P1 Wave 0), and the milestone release tag is v1.28.6. P5 Wave 1 Task 1.2 hedges: "If the v1.29 publish artifacts are not yet available... mark as covered-reference: requires v1.29.0 tag." This hedge lets P5 ship green without the smoke test ever running against real artifacts — a second silent- failure path.

Fix (binding):

  1. P1 Wave 4 (regression + ship) MUST push a v1.29.0 tag (or the first v1.29.x tag) as part of P1 ship, triggering publish.yml and producing the v1.29 artifacts. Document this in PLAN P1.
  2. P5 Wave 1 Task 1.2's hedge clause is REMOVED. The P5 smoke test MUST run against the published v1.29.x artifacts. If the artifacts are absent (P1 failed to publish), P5 fails closed — no hedge to "covered-reference."
  3. The milestone release tag remains v1.28.6 (the v1.28.x line per the tagging convention); the v1.29.0 artifact tag is a P1 intermediate tag, not the release. This resolves the tag-semantics ambiguity the grill surfaced.

🔴 CF-4 — kj v0.0.3 source-fetch path confirmed before P1 Wave 1

Finding: P1 Wave 1 Task 1.1b says "fetches the kj Go source at the pinned SHA" citing "RESEARCH §7 — source repo confirmed in P1 RESEARCH." RESEARCH §7 confirms the build command (CGO_ENABLED=0) but is silent on the source repository. Assumption ledger item #1 says "RESEARCH will confirm the source repository + build commands" — RESEARCH did NOT confirm the source repo. kj-version.txt pins v0.0.3 + SHA 4ebb9a19... but the grill cannot determine whether this is a source commit SHA or a binary digest, or what repo it lives in. P1 Wave 1 is built on an open assumption.

Fix (binding): Before P1 Wave 1 starts (P1 Wave 0 or a new Wave 0.5), the backend-engineer MUST confirm: (a) the kj source repo URL

  • the commit at SHA 4ebb9a19...; (b) go build reproduces a binary whose SHA-256 matches the recorded one (or the SHA is a source commit, in which case the build is the verification); (c) the fetched source compiles CGO_ENABLED=0 to a statically-linked binary (KJ-STATIC). If the source is not fetchable at the pinned SHA → P1 fails closed → escalate (this is a spec dependency, not a CIAgent ambiguity per assumption #1). Document the confirmed repo URL + commit in platform/abac/kj-version.txt (add a third line: the source repo URL).

Tracked conditions (apply during execution)

  • TC-1 (KJ-STATIC audit, P1 Wave 1 Task 1.2): file(1) asserts statically linked + readelf -d asserts no NEEDED entries, as a CI gate. Already in PLAN; tracked for enforcement.
  • TC-2 (KJ-LOCKSTEP by construction, covered-reference): both image-bearing resources reference a single data.aws_ecr_image.kj_image; image_uri = repo@digest. Verified via CF-1 item 12 (rollback drill)
    • CF-2 (operator-attested result row for REQ-371).
  • TC-3 (CloudFront OAC pitfall, P4 operator guide): the guide MUST document the AuthType: NONE → OAC-ignored pitfall (RESEARCH §4) as a callout. CF-1 item 9 mechanically verifies it. Already in PLAN P4 Wave 0 Task 0.3b; tracked.
  • TC-4 (ECR tag format, P1 Wave 1 Task 1.1f): assert tag matches ^[a-zA-Z0-9._-]+$ before push (D-239). Already in PLAN; tracked.
  • TC-5 (import idempotency, covered-reference REQ-361): CI import treats "Resource already managed by Terraform" as idempotent success (grep the message, not just exit code). Documented in RESEARCH §1; tracked for the ops repo (operator-attested via CF-2).
  • TC-6 (Fargate sunset discipline, P4 operator guide): D-237 — ≥30 consecutive days green + architecture review before deletion. Already in PLAN P4 Wave 0 Task 0.3f; tracked.

Binding decisions (this grill session)

ID Decision Rationale Confidence
G-1 The covered-reference pattern is accepted as a verification surface, but ONLY with CF-1 (M1.5 green = hard P6 gate + spike extended to 12 items) + CF-2 (operator-attested result rows for every covered-reference REQ, audited at P6). Without these, it is a deferred-trust assertion (punt) and the grill would REJECT. User-confirmed: covered-reference is a deferred-trust assertion; M1.5 must be a hard gate; TFM-HITL/IAM-NARROW/JWKS-EDGE-ONLY are unverifiable from acdl absent the extended spike. 0.78
G-2.1 M1.5 green (3 consecutive rebuilds of the 12-item spike) is a binding P6 milestone-ship gate, evidenced by an operator-attested row in the operator guide. The P6 audit verifies the row exists + is green. Dominant silent-failure path = M1.5 never runs green → 14 REQs false-"complete." User-confirmed. 0.85
G-2.2 The M1.5 spike is extended 8 → 12 items, adding: JWKS-EDGE-ONLY direct-URL-403 check, IAM-NARROW no-wildcard assertion, TFM-HITL self-approval-rejection check, rollback drill. INV-18, REQ-360, REQ-357 are otherwise unverifiable from acdl. Rollback is untested (D-236). 0.80
G-3 P1 MUST push a v1.29.0 (or first v1.29.x) intermediate tag at P1 ship to produce publish artifacts; P5's "covered-reference: requires v1.29.0 tag" hedge is REMOVED; the smoke test must run against real artifacts or P5 fails closed. P5's hedge is a second silent-failure path. User-confirmed. 0.82
G-4 The kj v0.0.3 source-fetch path (repo URL + commit at SHA 4ebb9a19...) must be confirmed before P1 Wave 1; the confirmed repo URL is recorded as a third line in platform/abac/kj-version.txt. If unfetchable → P1 fails closed → escalate. RESEARCH §7 is silent on the source repo; P1 Wave 1 is built on an open assumption. User-confirmed. 0.80
G-5 The covered-reference REQs (355..366, 371) are NOT marked "complete" at P6 unless their operator-guide cutover-gate row is non-empty + green (CF-2). An empty/red row blocks the milestone ship. Converts "documented" → "evidenced-by-operator-attestation-audited-by-acdl." 0.78

Escalations

None. All 9 axes resolved at confidence ≥ 0.66. The user confirmed the binding verdict (G-1: accepted with 4 conditions). No human escalation required (full autonomy). The kj source-fetch (CF-4) has a fail-closed path: if RESEARCH's open assumption is wrong, P1 fails closed and escalates at that point — but the grill does not pre-escalate a spec dependency the plan already flags.

Evidence verified against the live codebase

  • .github/workflows/publish.yml line 47-55: trigger is push: branches: [main] (P1 Wave 0 changes to tags: ['v1.29.*'] — matches PLAN).
  • .gitea/workflows/publish.yml exists (P2 removes it — matches PLAN).
  • platform/abac/kj-version.txt: 2 lines (v0.0.3 + SHA 4ebb9a19...) — matches PLAN; RESEARCH §7 silent on source repo (CF-4).
  • nova/idp/setup.py: 50 lines, --check/--apply/--verify/--dry-run (P3 adds terraform delegation — matches PLAN).
  • core/lambda/nova_idp_setup.py exists (P3 archives its CFN — matches).
  • tests/test_idp_auth.py + tests/test_kms_roundtrip.py EXIST (from v1.28); tests/test_abac_e2e.py does NOT exist (P1 Wave 3 authors it — matches PLAN).
  • pyproject.toml version = 1.14.0 (P2 bumps to 1.29.0 — matches PLAN; note: v1.28 did not bump it, a v1.28 carry-over the grill flags as minor but does not block on).

Grill complete

The v1.29 plan proceeds with 4 critical fixes (CF-1 M1.5 hard gate

  • spike extension; CF-2 operator-attested result rows; CF-3 P5 live smoke no-hedge; CF-4 kj source confirmation) and 6 tracked conditions. The covered-reference pattern is accepted as a verification surface only because CF-1 + CF-2 convert "documented" into "evidenced-by-operator-attestation-audited-by-acdl- at-P6." Without those fixes, the grill would REJECT: 14 of 17 REQs would ship "complete" on the strength of a markdown pointer to a nonexistent repo's CI.

Next: apply the 4 critical fixes to PLAN.md + REQUIREMENTS.md, then MVP/UX CHECK → SHIP phase 0.

Reference in New Issue View Git Blame Copy Permalink