Adversarial review of PLAN_v0.14.md across 9 axes. Verdict: RETHINK (confidence 0.45). The research foundation is strong but the plan diverges from it and from the codebase in load-bearing ways. 4 binding decisions (G-001..G-004): - G-001: migration number 0009 (not 0007 — already taken by certs) - G-002: omit :Z flag (contradicts CLARIFY D-258 + REQ-172) - G-003: wire real mTLS now (scope expansion — plan has no such phase) - G-004: P2 T6 must remove legacy systemd unit + binary on upgrade 12 binding conditions (C-50..C-61) + 14 phase challenges (PC-01..14). 1 escalation (E-001): G-003 mTLS direction undetermined in plan. Key findings: - F1.1: migration 0007 collision (BLOCKER) - F1.2: certpaths.CAPath() does not exist (compile BLOCKER) - F2.2: REQ-100 traefik-on-public-ip opt-out regressed by baked image - F5.1: no v0.13 -> v0.14 upgrade path (BLOCKER) - F7.2: podman pull violates R-001 offline-first ---ci--- status: grill milestone: v0.14 binding-decisions: - G-001: migration 0009_ingress_mode.sql (not 0007) - G-002: omit :Z, use :ro on both mounts - G-003: wire real mTLS now (scope expansion, plan must add phase) - G-004: P2 T6 must remove legacy systemd unit + binary on upgrade escalations: - E-001: G-003 mTLS direction chosen but plan has no phase for it (conf 0.55) verdict: rethink confidence: 0.45
30 KiB
GRILL v0.14: Ingress Bootstrap Completeness — Red-Team Review
Date: 2026-08-10 Reviewer: ci-griller (adversarial) Subject: PLAN_v0.14.md (9 phases, P0–P8) Confidence in plan as-written: 0.45 — RETHINK (was Proceed-eligible until the mTLS scope expansion was chosen) Verdict: RETHINK — the plan is technically grounded in strong research but contains (a) one outright correctness defect that will break execution, (b) two requirements/plan contradictions that ship broken config, (c) one scope expansion chosen during this grill that adds a 10th phase the plan does not contain, and (d) one regression of a Completed requirement. The research is the strongest artifact; the plan diverges from it in load-bearing ways.
How this grill was conducted
Every plan claim was checked against the actual codebase (internal/traefik/install.go, internal/emitter/nft.go, internal/emitter/traefik.go, internal/cli/init.go, internal/cli/upgrade.go, internal/cli/doctor_nft.go, internal/proxmox/bootstrap.go, internal/linux/bootstrap.go, internal/store/migrations/, internal/certpaths/, internal/sshpush/, scripts/release.sh, .coreci.yml, Dockerfile, git tags, git log). Findings cite file:line. Four binding questions were asked interactively; answers recorded as G-001..G-004 below.
Per-Axis Findings
Axis 1 — Feasibility
Finding F1.1 (BLOCKER — migration number collision). PLAN P5 T3 (line 194) specifies "Schema migration 0007: ALTER TABLE nodes ADD COLUMN ingress_mode TEXT DEFAULT ''". Migration 0007 already exists — internal/store/migrations/0007_certs_serial_unique.sql (added v0.7, P1-001). The current head is 0008_audit_tamper_evidence.sql. The migrator runs files in lexical order and records applied versions in schema_migrations. Reusing 0007 will either (a) silently no-op on DBs that already recorded 0007, leaving ingress_mode un-added, or (b) break the migration ledger. This is a guaranteed execution-time defect.
Evidence: internal/store/migrations/0007_certs_serial_unique.sql, internal/store/migrations/0008_audit_tamper_evidence.sql, internal/cli/init_test.go:84 (test asserts head = 0008).
Resolved by: G-001 (use 0009_ingress_mode.sql).
Finding F1.2 (BLOCKER — certpaths.CAPath() does not exist). PLAN P3 T5 step 2 (line 142) says "Push cluster root CA … from certpaths.CAPath() if exists". The function certpaths.CAPath() does not exist. The real API is certpaths.CACertPath() (internal/certpaths/certpaths.go:35) and certpaths.CAKeyPath() (:39). P4 T1 step 2 repeats the same phantom reference. This will not compile.
Evidence: internal/certpaths/certpaths.go:35 (func CACertPath() string); grep for CAPath returns zero matches in internal/certpaths/.
Finding F1.3 (no install_test.go to rewrite). PLAN P2 T10 (line 112) says "rewrite internal/traefik/install_test.go". That file does not exist — internal/traefik/ contains only install.go (glob confirms). The task is "create", not "rewrite". Minor, but signals the plan was written against an imagined codebase shape, not the real one.
Evidence: glob internal/traefik/*.go → only install.go.
Finding F1.4 (good — fake-SSH harness exists and is adequate). PLAN P7 T8 claims a "hermetic fake-SSH harness" can assert pct create + podman run inside LXC. Verified: internal/proxmox/ssh_session_test.go:23 (fakeSSHServer) and internal/sshpush/transport_test.go:27 implement an in-process SSH server with runCommand(cmd) pattern-matching. It can assert the right commands are sent (e.g. pct create … --features nesting=1,keyctl=1,fuse=1). It cannot actually create an LXC or run podman — but the plan only claims command assertion, which is achievable. The harness is real and reusable.
Finding F1.5 (good — callsites verified). PLAN P2 T3/T4/T5 reference init.go:254-266, linux/bootstrap.go:160-172, proxmox/bootstrap.go:250-255. Verified: init.go:256 calls installTraefikLocal(); linux/bootstrap.go:170 calls traefik.InstallRemote("", sshExecFn); proxmox/bootstrap.go:253 calls traefik.InstallRemote("", runRemote). The line numbers are accurate within a few lines. The plan was written against the real callsites.
Axis 2 — Scope
Finding F2.1 (BLOCKER — mTLS scope expansion chosen, no phase exists). During this grill (G-003) the operator chose "Wire real mTLS now" over the plan's tls: {} default-cert approach. This requires step-ca to mint server certs into /etc/traefik/dynamic + dynamic tls.certificates + tls.options.default.clientAuth.caFiles. Step-ca server-cert minting was deferred since v0.11 and is not implemented. The 9-phase plan contains no such phase. The operator then chose (G-004-adjacent) to add a step-ca cert minting phase, growing v0.14 from 9 to 10 phases. The plan as written does not reflect this. Until a P-step for cert minting is added, the plan is incomplete relative to the chosen direction.
Impact: v0.14 cannot ship real mTLS on its current 9 phases. Either add the phase (10 phases, more risk) or revert to tls: {} and defer mTLS to v0.15.
Finding F2.2 (regression — traefik-on-public-ip opt-out lost). REQ-100 (Complete, v0.11) established the --public-binding=traefik-on-public-ip opt-out: traefik binds :443/:80 directly instead of 127.0.0.1:8443/8080 + nft DNAT. The existing RenderTraefikStaticConfig (internal/emitter/traefik.go:273) implements this via TraefikStaticOpts.PublicBinding. The v0.14 plan's baked image (docker/orca-traefik/traefik.yml, PLAN lines 33-37) hard-codes 127.0.0.1:8443/8080 — there is no opt-out path in the image. Baking the static config into the image freezes out the opt-out mode that v0.11 shipped. This is a regression of a Completed requirement.
Evidence: internal/emitter/traefik.go:255-266 (publicWebSecure/publicWeb switch on PublicBinding); internal/emitter/traefik.go:282-300 (rendered static config); REQUIREMENTS REQ-100 line 218.
Finding F2.3 (9 phases is borderline; 10 is too many). Even before the mTLS expansion, 9 phases for an "ingress bootstrap completeness" milestone is heavy. P5 (proxmox native) and P6 (floating-IP) are the two most complex (LXC creation + podman-in-LXC + apt-get + nft-inside-LXC). They could potentially be merged into one phase with two code paths, since they share ProvisionIngressLXC plumbing. With the mTLS expansion, 10 phases is too many for a single milestone — split v0.14 into v0.14a (linux ingress + mTLS) and v0.14b (proxmox ingress).
Finding F2.4 (good — research-validated decisions are sound). The 7 research topics (nft postrouting, pve-firewall priority, LXC features, traefik Dockerfile, pct create syntax, podman restart, SELinux) are well-sourced and the recommended approaches are technically correct. This is the strongest artifact in the v0.14 dossier.
Axis 3 — Dependencies
Finding F3.1 (good — ordering is correct). P1 (image) → P2 (reconciler) → P3 (nft+init) → P4 (remote linux) → P5 (proxmox native) → P6 (floating-IP) → P7 (doctor+tests) → P8 (ship). Each phase references the prior phase's output (P3 T6 calls EnsureTraefikContainerLocal from P2; P5 T4 step 10 calls EnsureTraefikContainerRemote from P2). No phase can ship before its prerequisite.
Finding F3.2 (hidden dependency — P2 T8 depends on P1 T1). P2 T8 (drop certResolver: orca) edits internal/emitter/traefik.go. P1 T1 bakes the static config. The dynamic config (traefik.go) and static config (docker/orca-traefik/traefik.yml) must be consistent: if the static config has no certificatesResolvers.orca (P1 T1 correctly omits it) but the dynamic config still references certResolver: orca (until P2 T8), traefik logs a warning on every reload. The plan orders P1 before P2, so there is a window (P1 shipped, P2 not yet) where the published image + the live dynamic config are inconsistent. This is acceptable only if P1 and P2 ship in quick succession; if P1 stalls, the image is published with a known TLS-config mismatch.
Axis 4 — Security
Finding F4.1 (good — nft injection guard already present). internal/emitter/nft.go:104-130 (partitionTrustedProbes) validates every TrustedProbes entry as IP/CIDR before rendering (F9 guard). The v0.14 plan adds DNATTarget (P3 T1) — this is a NEW string field rendered directly into dnat to <DNATTarget>:8443. The plan does not specify validation of DNATTarget. If DNATTarget is user-controllable (via --floating-ip or cluster config), an unvalidated value is an nft-syntax injection vector. The existing F9 guard covers TrustedProbes; DNATTarget needs the same treatment.
Binding: C-51 (see below).
Finding F4.2 (container escape surface — LXC nesting). --features nesting=1,keyctl=1,fuse=1 (P5/P6) is the documented requirement for podman-in-LXC, but nesting exposes host procfs/sysfs to the guest (Proxmox docs, RESEARCH Topic 3). This is an accepted tradeoff for container-in-container, but the plan does not document the threat-model acceptance. For a "production hardening" lineage, the ingress LXC is now a privileged-ish surface (nesting+keyctl) running a podman container pulling an image from a registry. The supply chain is: git.cloudinit.dev/coreci/orca-traefik:<version> (P1). If the registry is compromised or the tag is re-pushed, the ingress LXC runs attacker code at the host-LXC boundary. The plan has no image-signing/verification step (podman pull --cert-dir or cosign).
Binding: C-52 (see below).
Finding F4.3 (nft rules injection via re-apply). P3 T4 changes flush table to delete table. RESEARCH Topic 1 establishes that delete table on a missing table errors, and the fix is to pre-create the table (nft add table inet orca-ingress 2>/dev/null || true) before nft -f. P3 T5 step 4 does this. Good. But the re-apply path (P3 T5 step 5, P4 T1 step 5) runs nft -f which replaces the whole table — if a concurrent process (pve-firewall, operator) adds rules to orca-ingress between the pre-create and the nft -f, they are wiped. This is by design (orca owns the table) but should be documented as "orca is the sole owner of table inet orca-ingress".
Axis 5 — Operational
Finding F5.1 (BLOCKER — no upgrade path from v0.13 binary+systemd to v0.14 podman). P2 T6 (line 104) says "update legacy cutover to pull new image + recreate container instead of sed-ing traefik.yml". But internal/cli/upgrade.go:244-407 implements a Traefik :443 → 127.0.0.1:8443 cutover that does systemctl restart traefik (line 382) — it restarts the systemd service, not a container. P2 removes systemd unit generation. There is no phase that: (a) detects the legacy orca-traefik.service, (b) stops+disables it, (c) removes /usr/local/bin/traefik + /etc/systemd/system/orca-traefik.service, (d) then runs EnsureTraefikContainerLocal. Without this, upgrading a live v0.13 cluster leaves a dead systemd unit AND a new podman container both trying to bind 127.0.0.1:8443 → port conflict, traefik down.
Resolved by: G-004 (P2 T6 must remove legacy unit + binary, idempotent, tested).
Finding F5.2 (podman/podman-restart.service unavailable). P2 T2 enables podman-restart.service. RESEARCH Topic 6 establishes this is not enabled by default on Ubuntu 24.04 and must be enabled. But the plan does not specify what happens if podman is not installed on the target host (a fresh linux node join, P4). EnsureTraefikContainerLocal/Remote calls podman pull/podman run — if podman is absent, this fails. The plan has no "install podman first" step for the linux topology (P3/P4). For proxmox (P5/P6) the plan installs podman via apt-get install -y podman conmon crun fuse-overlayfs inside the LXC (T4 step 4 / T1 step 6). But for linux nodes (P3/P4), there is no podman-install step. This is an under-specification.
Binding: C-50 (see below).
Finding F5.3 (good — --network host is correct). RESEARCH Topic 3 confirms --network host inside an LXC binds the LXC's netns, so traefik binds 127.0.0.1:8080/8443 on the LXC loopback and nft on the PVE host DNATs to the LXC IP. This is sound. The DNATTarget parameterization (D-262) correctly distinguishes 127.0.0.1 (linux/localhost) from <lxc-ip> (proxmox native).
Finding F5.4 (reboot persistence chain has a gap). RESEARCH Topic 6 establishes the chain: Proxmox boot → --onboot 1 starts LXC → LXC systemd starts → podman-restart.service restarts container. P5 T4 step 6 and P6 T1 step 8 enable podman-restart.service inside the LXC. But --onboot 1 starts the LXC after the Proxmox host's network is up — if the floating IP is on a bridge that depends on a physical link that's slow to come up, the LXC may start before the bridge is ready, and the floating-IP eth0 config may fail. The plan does not address LXC-start ordering relative to bridge readiness.
Axis 6 — Testing
Finding F6.1 (good — harness is real). fakeSSHServer (Axis 1 F1.4) can assert pct create with the right --features and net0 args, apt-get install podman, podman run with --network host. P7 T8's claims are achievable.
Finding F6.2 (gap — no test for the v0.13→v0.14 upgrade). G-004 requires the upgrade path to be tested. The plan's P2 Wave 4 (T10) tests the reconciler but not the legacy-removal path. There is no test that: (a) simulates a host with orca-traefik.service present, (b) runs the upgrade, (c) asserts the unit is stopped+disabled+removed, (d) asserts the podman container is running. This must be added to P2 T10.
Finding F6.3 (gap — no test for DNATTarget validation). F4.1 identifies DNATTarget as an injection vector. P3 T8 (nft_test) asserts DNATTarget substitution but does not specify a test for invalid DNATTarget values (e.g. 1.2.3.4:8443; flush ruleset). The existing F9 guard test pattern should be extended.
Finding F6.4 (good — doctor_nft extension is incremental). P3 T7 extends doctor_nft.go. The existing file (internal/cli/doctor_nft.go) is well-structured with nftCheckResult lines; adding postrouting/masquerade/DNATTarget assertions is straightforward.
Axis 7 — Performance
Finding F7.1 (apt-get install podman on every proxmox join — 30-60s). CLARIFY D-263 acknowledges "~30-60s to the join time". P5 T4 step 4 and P6 T1 step 6 run apt-get update && apt-get install -y podman conmon crun fuse-overlayfs nftables inside the LXC on every join. This is acceptable for a one-time bootstrap but painful if re-run. The reconciler must be idempotent (skip if podman already installed). The plan does not specify an idempotency check for the apt-get step.
Binding: C-53 (see below).
Finding F7.2 (image pull latency). podman pull orca-traefik:<tag> (P2 T1 step 3, P5/P6) pulls from git.cloudinit.dev/coreci/. On a proxmox host behind a slow link, this can take 10-30s for a ~150MB traefik image. The plan has no pull-timeout. If the registry is unreachable (offline-first is R-001!), the pull fails and traefik never starts. R-001 (offline-first) is violated: the plan depends on a registry pull at bootstrap time. There is no "pre-pull" or "bundle image into the LXC template" fallback.
Binding: C-54 (see below) — this is a tension with R-001 that the plan does not acknowledge.
Finding F7.3 (nft re-apply disruption). nft -f replaces the table atomically (single transaction). Existing connections are NOT disrupted (conntrack holds them). New connections during the apply window (<1ms) may be dropped. This is acceptable. No finding.
Axis 8 — Cost
Finding F8.1 (2 images per release — sustainable). P1 adds orca-traefik image alongside orca. .coreci.yml gets a container-publish-traefik step (P1 T5). scripts/release.sh gets a second docker block (P1 T4). The traefik image is small (~150MB, Alpine-based). Registry storage: 2 images × N releases. At v0.13.x cadence (8 tags), that's 16 image-tags per milestone. Sustainable for a private Gitea registry. No finding.
Finding F8.2 (good — release.sh extension is minimal). P1 T4 adds ~8 lines to scripts/release.sh after line 212. Verified the insertion point (line 213 is end of existing docker block). Clean.
Axis 9 — Completeness (3 topologies)
Finding F9.1 (linux topology — covered by P3+P4). orca init (P3) bootstraps nft+podman on the lead; orca node join --type linux (P4) does it remotely. Complete.
Finding F9.2 (proxmox-native — covered by P5, but LXC IP discovery is hand-wavy). P5 T4 step 9 says "Discover LXC IP via pct config <vmid> (parse net0 line) or pct exec <vmid> -- hostname -I". But in native mode the LXC is created without a static IP (P5 T4 step 2 has no ip= in the pct create — unlike P6 which has ip=<floating-ip>/<prefix>). So the LXC gets a DHCP/bridge IP that is not known at create time. P5 step 9 discovers it after pct start. But step 10 then re-applies nft with the discovered IP. This is a two-phase apply: first apply with default 127.0.0.1 (wrong for native), then re-apply with LXC IP. The plan does not specify what happens to traffic between the first and second apply (it DNATs to 127.0.0.1:8443 on the PVE host where nothing listens → connections refused). There is a window of ingress downtime during native-mode bootstrap.
Binding: C-55 (see below).
Finding F9.3 (floating-IP — covered by P6, but MAC uniqueness is untested). P6 T4 generates a random 02:XX:XX:XX:XX:XX MAC in interactive mode. The plan does not check for MAC collision on the bridge. RESEARCH Topic 5 pitfall 2: "hwaddr must be unique on the bridge". A random 02: prefix has 46 bits of entropy — collision is unlikely on a single bridge but not impossible across a multi-node cluster.
Binding: C-56 (see below).
Finding F9.4 (no localhost/lead topology with podman). P3 bootstraps the lead via EnsureTraefikContainerLocal. But orca init runs on the lead — if the lead has no podman installed (fresh host), EnsureTraefikContainerLocal fails. Same as F5.2 but for the lead. The plan assumes podman is present on the lead. No install step.
Binding Conditions (gates that MUST be met before a phase ships)
| ID | Gate | Phase | Severity |
|---|---|---|---|
| C-50 | P2 must not break existing orca init on a host without podman installed — either install podman as part of BootstrapLocalIngress (P3) or emit a clear error with install instructions. Same for orca node join --type linux (P4). |
P2/P3/P4 | BLOCKER |
| C-51 | NftClusterConfig.DNATTarget must be validated as net.ParseIP or ip:port before rendering. Unvalidated values are an nft-syntax injection vector (same F9 guard as TrustedProbes). Test required. |
P3 | BLOCKER |
| C-52 | The orca-traefik image supply chain must be documented: registry is public (anonymous pull per REQ-045), no image signing in v0.14. Document the threat-model acceptance: a compromised registry = attacker code in the ingress LXC. Add podman image trust or cosign verification as a v0.15 hardening item. |
P1/P8 | High |
| C-53 | The apt-get install podman step inside the LXC (P5 T4 step 4, P6 T1 step 6) must be idempotent: check command -v podman first, skip if present. Re-running join on an existing LXC must not re-run apt-get. |
P5/P6 | High |
| C-54 | R-001 (offline-first) tension: podman pull at bootstrap requires registry reachability. Either (a) document that ingress bootstrap requires online access (exception to R-001), or (b) pre-bundle the orca-traefik image into the LXC template / load from a local archive. The plan must acknowledge this tension explicitly. |
P2/P5/P6 | BLOCKER |
| C-55 | P5 native-mode bootstrap must not create a window of ingress downtime. The first nft apply must use the LXC IP (discovered after pct start but before the first nft apply), OR the plan must accept and document the downtime window. Two-phase apply (default → LXC IP) is a transient outage. |
P5 | High |
| C-56 | P6 MAC generation must check for collision against existing nodes' MACs in the cluster registry. Reject or regenerate on collision. | P6 | Medium |
| C-57 | P2 T6 must implement the v0.13→v0.14 upgrade: detect orca-traefik.service, stop+disable, remove /usr/local/bin/traefik + unit file, then EnsureTraefikContainerLocal. Idempotent. Tested with a simulated-legacy-host test (F6.2). |
P2 | BLOCKER |
| C-58 | The baked docker/orca-traefik/traefik.yml must not regress REQ-100's traefik-on-public-ip opt-out. Either (a) bake both configs and select via env/flag, or (b) document that the opt-out is dropped in v0.14 and update REQ-100, or (c) mount the static config from host (not baked) so RenderTraefikStaticConfig still works. |
P1 | High |
| C-59 | Migration for ingress_mode must be 0009_ingress_mode.sql, NOT 0007 (already taken by certs_serial_unique). |
P5 | BLOCKER |
| C-60 | certpaths.CAPath() references in P3 T5 and P4 T1 must be corrected to certpaths.CACertPath(). |
P3/P4 | BLOCKER (compile) |
| C-61 | REQ-172 must be amended: --restart=always → --restart=unless-stopped (per RESEARCH Topic 6 + PLAN), and :Z → omitted (per G-002). The requirement text contradicts the plan and research. |
P2 | High |
| C-62 | If mTLS is in-scope for v0.14 (per G-003), a new phase must be added implementing step-ca server-cert minting into /etc/traefik/dynamic + dynamic tls.certificates + tls.options.default.clientAuth.caFiles. The plan currently has no such phase (P2 T8 emits tls: {}). |
NEW PHASE | BLOCKER |
Phase Challenges (specific challenges a phase must overcome)
| ID | Challenge | Phase |
|---|---|---|
| PC-01 | P1 must bake a static config that doesn't regress the traefik-on-public-ip opt-out (REQ-100). Baking freezes the config; the opt-out needs a runtime switch. |
P1 |
| PC-02 | P2 must handle the 3-way TLS contradiction (CLARIFY D-257 vs RESEARCH Topic 4 vs PLAN T8). Per G-003, real mTLS is chosen — P2 alone cannot deliver it; a new phase is needed. | P2 + new |
| PC-03 | P2 T6 must remove the legacy systemd unit + binary without breaking a running v0.13 cluster. The existing upgrade.go cutover logic (lines 244-407) must be rewritten to stop+disable+remove the unit, not systemctl restart traefik. |
P2 |
| PC-04 | P3 must install podman on the lead if absent (C-50). The plan assumes podman is present. BootstrapLocalIngress must either install it or fail with a clear message. |
P3 |
| PC-05 | P3 T5 must use certpaths.CACertPath() not the phantom certpaths.CAPath(). |
P3 |
| PC-06 | P3 T1 (DNATTarget) must validate input (C-51). New string field rendered into nft ruleset — injection risk. |
P3 |
| PC-07 | P3 T4 (first-apply flush-table fix) must work across nft versions. RESEARCH establishes delete table on missing table is version-dependent. The pre-create approach (nft add table … 2>/dev/null || true before nft -f) is robust; the plan uses it (T5 step 4). Verify on Proxmox kernel. |
P3 |
| PC-08 | P5 native-mode has a two-phase nft apply (default 127.0.0.1 → LXC IP) that creates a transient outage window. Must be eliminated or documented (C-55). | P5 |
| PC-09 | P5/P6 apt-get install podman inside LXC takes 30-60s and must be idempotent (C-53). Re-join must not re-install. |
P5/P6 |
| PC-10 | P5/P6 podman pull requires registry reachability, violating R-001 (offline-first). Must be acknowledged or mitigated (C-54). |
P5/P6 |
| PC-11 | P5/P6 LXC reboot chain (Proxmox boot → --onboot 1 → LXC systemd → podman-restart.service) has a gap: LXC may start before bridge is ready. Floating-IP eth0 config may fail. |
P5/P6 |
| PC-12 | P6 MAC generation must check for collision (C-56). | P6 |
| PC-13 | P7 T8 integration test must include the v0.13→v0.14 upgrade path (F6.2): simulated legacy host → upgrade → assert unit removed + podman running. | P7 |
| PC-14 | If the new mTLS phase is added (G-003), it must mint server certs into /etc/traefik/dynamic atomically (C-10 protocol) and traefik must reload them via the file provider watch. Step-ca minting was deferred since v0.11 — this is net-new work, not a completion. |
new phase |
Binding Decisions (from interactive grill)
| ID | Decision | Rationale | Confidence | Alternatives rejected |
|---|---|---|---|---|
| G-001 | Migration for ingress_mode is 0009_ingress_mode.sql |
0007 is already certs_serial_unique; 0008 is audit_tamper_evidence. Reusing 0007 breaks the migrator. |
0.95 | Renumber existing (breaks deployed DBs); accept collision (guaranteed defect) |
| G-002 | Omit :Z flag on volume mounts; use :ro on both |
RESEARCH Topic 7: :Z relabels host dirs to private container label, blocks host-side orca writes on SELinux. No-op on Ubuntu/Proxmox but a latent footgun. PLAN P2 T1 + RESEARCH agree; CLARIFY D-258 + REQ-172 are wrong. |
0.90 | Keep :Z (wrong on future SELinux); :z shared (unnecessary) |
| G-003 | Wire real mTLS now (dynamic tls.certificates + clientAuth.caFiles) |
Operator chose this over tls: {} default. Requires step-ca server-cert minting which is not implemented (deferred since v0.11). |
0.55 | tls: {} now (plan's approach, defers mTLS); keep certResolver: orca (broken — key doesn't exist in traefik v3.3) |
| G-004 | P2 T6 must remove legacy systemd unit + binary on upgrade from v0.13 | Without it, upgrading a live v0.13 cluster leaves a dead systemd unit + a new podman container both binding 127.0.0.1:8443 → port conflict. Idempotent + tested. | 0.90 | Leave legacy unit (port conflict); fresh-installs only (unacceptable) |
Escalations (unresolved, confidence < 0.60)
| ID | Escalation | Confidence | Reason |
|---|---|---|---|
| E-001 | G-003 (real mTLS now) creates a scope expansion that the 9-phase plan does not contain. The operator chose to add a step-ca cert minting phase (growing v0.14 to 10 phases) but the plan has not been updated to reflect this. Until the new phase is specified (scope, tasks, tests), v0.14's mTLS direction is undetermined. The plan as written ships tls: {} (no mTLS), which contradicts G-003. |
0.55 | The choice is made but the plan does not reflect it. This is a plan-spec gap, not a technical unknown. |
Meta — Closing Review
What the auditor would flag:
- The plan ships a known-broken TLS config. RESEARCH Topic 4 calls
certResolver: orca"the biggest v0.14 finding" and says it does not exist in traefik v3.3. CLARIFY D-257 and REQ-171 still specify it. The plan (P2 T8) drops it but emitstls: {}(no real TLS). G-003 chose real mTLS, which the plan doesn't contain. Three artifacts disagree on TLS. - The plan regresses REQ-100 (
traefik-on-public-ipopt-out) by baking the static config. - The plan has no upgrade path from v0.13 (binary+systemd) to v0.14 (podman). G-004 binds the fix but the plan must be updated.
- The plan references a phantom function (
certpaths.CAPath()) and a phantom migration number (0007). Both will fail at compile/execution time. - The plan's offline-first claim (R-001) is violated by
podman pullat bootstrap (C-54).
What the project is NOT doing that it should:
- Image supply-chain verification (no cosign, no
podman image trust). - LXC-start ordering relative to bridge readiness (reboot persistence gap).
- Idempotency check for
apt-get install podmaninside LXC. - MAC collision check on the bridge.
- A test for the v0.13→v0.14 upgrade path.
Simplest 80%-value version: Ship P1 (image) + P2 (reconciler, with legacy-removal) + P3 (nft+init) + P4 (linux remote) + P7 (doctor+tests). Defer P5 (proxmox native) and P6 (floating-IP) to v0.15. This delivers the linux topology (the most common) + the podman migration + nft completeness, and avoids the two most complex phases (LXC+podman-in-LXC). If G-003 (real mTLS) holds, add the mTLS phase to v0.14a. Proxmox ingress becomes v0.14b/v0.15.
What must be true for v0.14 to succeed in 90 days:
- The 3-way TLS contradiction is resolved in the plan (not just in this grill). Today: unresolved.
- The migration number is 0009. Today: plan says 0007 (wrong).
- The upgrade path from v0.13 is specified and tested. Today: not specified.
- The
certpaths.CAPath()phantom is fixed. Today: not fixed. - The offline-first tension (podman pull) is acknowledged. Today: not acknowledged.
- If mTLS is in-scope, the new phase is written. Today: no such phase.
Confidence: 0.45 that the plan as-written can ship v0.14 without rework. The research is strong; the plan diverges from it and from the codebase in load-bearing ways. The mTLS scope expansion (G-003) makes it worse unless the plan is updated.
Verdict
RETHINK (confidence 0.45).
The plan must be revised to:
- Fix the migration number → 0009 (C-59, G-001).
- Fix
certpaths.CAPath()→certpaths.CACertPath()(C-60). - Add the v0.13→v0.14 upgrade path to P2 T6 (C-57, G-004).
- Resolve the TLS model: either add a new mTLS phase (G-003) or revert to
tls: {}and defer mTLS to v0.15. The plan cannot shipcertResolver: orca(broken) and cannot shiptls: {}if G-003 holds. - Amend REQ-172:
--restart=unless-stopped(notalways), omit:Z(C-61, G-002). - Address the offline-first tension (C-54) or document the exception.
- Address the REQ-100 regression (C-58): bake-vs-mount the static config.
- Add
DNATTargetvalidation (C-51). - Add podman-install step for linux topology (C-50).
- If mTLS is in-scope, write the new phase (C-62, E-001).
Once these are addressed, the plan is feasible. The research foundation is solid; the plan just needs to actually follow it.
This grill is recorded in .ciagent/GRILL_v0.14.md. Escalations are visible via ciagent audit. Binding decisions (G-001..G-004) should be promoted to PROJECT.md via a follow-up clarify or explicitly by the operator. The grill surfaces; it does not rewrite.