orca/.ciagent/ROADMAP.md

Roadmap: Orca

Milestone v0.1: Foundation — COMPLETE

• Phase 0: Project Initialization & Specification
• Phase 1: Core CLI Skeleton & Command Parsing
• Phase 2: Basic Node Management (Join/Leave)
• Phase 3: Simple Task Execution Engine
• Phase 4: Local State Persistence
• Phase 5: Basic Health Checking
• Phase 6: CoreCI Full Release Flow

Tagged v0.2.0 (next-minor per feature-milestone promotion rule).

Deferred to v0.2 (out of scope for v0.1)

• Multi-node scheduling (D-004 decision: single-node only in v0.1)
• mTLS for inter-node communication (REQ-011, REQ-023)
• gosec + govulncheck in CI pipeline (REQ-014)
• iter.Seq streaming job lists (REQ-022)
• Frontend / devops personas (no web UI; CoreCI handles release)

Milestone v0.2: Networking, Observability, Security Hardening — COMPLETE (merged to main via v0.3)

Scope: extend v0.1 with secure cross-node transport, multi-node scheduling, richer CI security scanning, and streaming I/O.

• Phase 8: mTLS handshake + internal CA with CSR join (Wave 1) — shipped v0.2.1
• Phase 9: Multi-node scheduling & job dispatch (Wave 1) — shipped v0.2.2
• Phase 10: gosec + govulncheck + gitleaks in CI (Wave 2) — shipped v0.2.3
• Phase 11: iter.Seq streaming job/node lists (Wave 2) — completed in v0.3 P01 (shipped v0.3.1)

Milestone tag: v0.4.0 (shipped — v0.2 work merged to main via v0.3 milestone).

Per-phase tags: v0.2.1 (P01), v0.2.2 (P02), v0.2.3 (P03) — all shipped.

Milestone v0.3: Scheduling & Streaming Completion — COMPLETE

Scope: complete the two work items deferred from v0.2 that were not already shipped in P08-P10. A re-init SPECIFY codebase audit confirmed that REQ-014/027/028/029/031/037/039/040 all shipped in P08-P10 despite stale REQUIREMENTS.md marking them Pending. The remaining work is lean:

• Phase 0: Pre-execution (specify → clarify → research → plan → grill) — shipped v0.3.0
• Phase 1: iter.Seq streaming for --watch flags (REQ-022, REQ-030) — shipped v0.3.1
• Phase 2: orca doctor network + db full implementation (REQ-032 completion) — shipped v0.3.2
• Phase 3: Final review + ship + audit (milestone release) — shipped v0.3.3

Milestone tag: v0.4.0 (next-minor per feature-milestone promotion rule).

Per-phase tags: v0.3.0 (P0), v0.3.1 (P01), v0.3.2 (P02), v0.3.3 (P03 final = milestone release). Per .ciagent/RELEASE_POLICY.md, every phase tag produces a Gitea release.

Per-phase REQ coverage

• P01 — iter.Seq streaming for --watch flags

  • REQ-022 (iter.Seq for streaming job lists, Go 1.25+)
  • REQ-030 (--watch output format mode: table default vs streaming JSON per event)
  • Applies to both orca job list --watch and orca node list --watch (D-024, per ARCHITECTURE.md CLI layer + D-017)

• P02 — orca doctor network + db full implementation

  • REQ-032 (completion: network reachability via mTLS /healthz probe, db integrity via PRAGMA integrity_check + migration version)
  • Replaces NetworkStub and DBStub from v0.2 P01

v0.3 is a completion milestone, not a direction change

The vision ("minimalist, offline-first, CLI-first orchestration engine") is unchanged. v0.3 closes out the v0.2 deferrals and merges the accumulated v0.2 work to main.

Milestone v0.5: Distribution — COMPLETE

Scope: make Orca installable, distributable, and containerized. The engine functionality from v0.1–v0.3 is unchanged; this milestone is purely about delivery surface.

• Phase 0: Pre-execution (specify → clarify → research → plan) — shipped v0.4.1 (+ repo public)
• Phase 1: Namespace unification (ORCA_HOME + --system) (REQ-041, REQ-042) — shipped v0.4.2
• Phase 2: install.sh + in-place update + README quickstart (REQ-043, REQ-044) — shipped v0.4.3
• Phase 3: Docker release (Dockerfile + Gitea container registry) (REQ-046) — shipped v0.4.4
• Phase 4: Final review + ship + audit (milestone release) — shipped v0.4.5

Operational prerequisite (P0 ship): repo + org visibility flipped to public (REQ-045) — unauth releases API + asset download + docker pull all verified HTTP 200.

Milestone tag: v0.4.5 (final phase patch = milestone release per feature-milestone promotion rule). Per-phase tags: v0.4.1…v0.4.5.

Milestone v0.6: Node Bootstrap & Proxmox

Milestone v0.6: Node Bootstrap & Proxmox — COMPLETE

Scope: make orca init produce a fully working single-node cluster (CA + server cert + DB + localhost node registered with auto-detected OS), and add Proxmox 8 & 9 as a first-class remote node type joined over SSH with least-privilege role delegation.

• Phase 0: Pre-execution (specify → clarify → research → plan → grill) — tag v0.5.0
• Phase 1: orca init full bootstrap + localhost node + schema 0006 (REQ-047, REQ-048, REQ-049) — tag v0.5.1
• Phase 2: Proxmox SSH join + OrcaOperator role + sudoers allowlist (REQ-050, REQ-051) — tag v0.5.2
• Phase 3: doctor os + doctor proxmox SSH probe + audit logging (REQ-052) — tag v0.5.3
• Phase 4: Final review + ship + audit (milestone release) — tag v0.5.4

Milestone type: feature (P1/P2/P3 ship feat phases). Milestone tag: v0.5.4 (final phase patch = milestone release per feature-milestone promotion rule). Per-phase tags: v0.5.0…v0.5.4.

Tags run on the previous minor's patch line (v0.5.x) per branch-strategy.md. The milestone branch label uses the milestone number (milestone/v0.6-node-bootstrap-proxmox); no separate minor tag is created.

Milestone v0.7: Hardening & Completion — COMPLETE

Scope: NFR milestone closing gaps surfaced by the v0.7 IDEATE stage — an unreachable command tree, a missing config file layer, low test coverage in core packages, and the long-deferred pprof endpoint.

• Phase 0: Pre-execution (specify → clarify → research → ideate → plan) — tag v0.6.0 (shipped)
• Phase 1: Register orca cert command tree + cert_repo tests (REQ-053) — tag v0.6.1 (shipped)
• Phase 2: HCL config file parsing — internal/config package (REQ-054) — tag v0.6.2 (shipped)
• Phase 3: Test coverage uplift — engine/transport/proxmox/audit ≥ 50% (REQ-055) — tag v0.6.3 (shipped)
• Phase 4: --pprof opt-in on orca daemon (REQ-056) — tag v0.6.4 (shipped)
• Phase 5: Final review + ship + audit (milestone release) — tag v0.6.5 (shipped)

Milestone type: NFR (all phases are fix/test/chore; no feat phases). Milestone tag: v0.6.5 (final phase patch = milestone release per NFR-milestone progressive-patch rule). Per-phase tags: v0.6.0…v0.6.5. Tags run on the previous minor's patch line (v0.6.x) per branch-strategy.md. The milestone branch label uses the milestone number (milestone/v0.7-hardening-completion); no separate minor tag.

Milestone v0.8: Coverage & Trust Hardening — COMPLETE

Scope: continue the v0.7 hardening theme. v0.7 P03's ≥ 50% floor left six packages still under 50% (engine 8.3%, proxmox 5.1%, cli 27.6%, transport 26.3%, store 46.7%, jobspec 47.6%) and three packages with no tests at all (internal/audit, internal/certpaths, cmd/orca). v0.8 also closes the two SSH-trust "future enhancement" hooks deferred in v0.6 (D-035 --host-key-fingerprint pre-pin, RESEARCH_v0.6 §80 orca node key-reset) and adds a requirements-hygiene gate to prevent the stale-REQ-status drift seen after v0.7 ship.

• Phase 0: Pre-execution (specify → clarify → research → plan → grill) — tag v0.7.0 (shipped)
• Phase 1: Test coverage uplift round 2 — 6 packages to ≥ 70%, 3 zero-test packages to first tests (REQ-057) — tag v0.7.1 (shipped)
• Phase 2: SSH trust hardening — --host-key-fingerprint pre-pin + orca node key-reset + TOFU bugfix + HostKeyFingerprint population (REQ-058, REQ-059) — tag v0.7.2 (shipped)
• Phase 3: Requirements-hygiene gate — make verify-reqs + verify assertion (REQ-060) — tag v0.7.3 (shipped)
• Phase 4: Final review + ship + audit (milestone release) — tag v0.7.4 (shipped)

Milestone type: NFR (P01 test, P02 chore on trust surface per D-043, P03 chore, P04 docs/review). Final phase patch IS the milestone release per NFR-milestone progressive-patch rule. Per-phase tags: v0.7.0…v0.7.4. Tags run on the previous minor's patch line (v0.7.x) per branch-strategy.md. The milestone branch label uses the milestone number (milestone/v0.8-coverage-trust-hardening); no separate minor tag.

Per-phase REQ coverage

• P01 — Coverage uplift round 2

  • REQ-057 (raise internal/engine, internal/proxmox, internal/cli, internal/transport, internal/store, internal/jobspec to ≥ 70%; add first tests for internal/audit, internal/certpaths, cmd/orca)

• P02 — SSH trust hardening

  • REQ-058 (--host-key-fingerprint <sha256> pre-pin flag on orca node join --type proxmox; fail fast on mismatch; supersedes TOFU for pre-pinned deployments)
  • REQ-059 (orca node key-reset <node> clears persisted SSH host key so next doctor proxmox/dispatch re-pins via TOFU or --host-key-fingerprint)

• P03 — Requirements-hygiene gate

  • REQ-060 (make verify-reqs target + verify-stage assertion: every REQ Complete in ROADMAP.md has matching Complete row in REQUIREMENTS.md; enforced in CI validate pipeline)

v0.8 is a continuation milestone, not a direction change

The vision ("minimalist, offline-first, CLI-first orchestration engine") is unchanged. v0.8 closes the coverage debt left by v0.7's 50% floor and the trust-surface gaps explicitly deferred in v0.6.

Milestone v0.9: Re-architecture Foundation & Workloads — COMPLETE

Scope: This milestone SUPERSPEDES the shipped v0.1–v0.8 architecture per the adopted PRD (.ciagent/PRD_v0.9.md). The re-architecture is justified on six grounds recorded in the PROJECT.md Supersession Table: (1) the v0.8 daemon model is operationally failing, (2) step-ca is externally mandated, (3) multi-tenancy is a hard product requirement, (4) WASM is a hard workload requirement, (5) SSH-push is the only viable deployment target, (6) vision correction. The 16 load-bearing rules (R-001…R-016) are invariants. The ci-griller reviewed the re-architecture adversarially; the user overrode the Re-architecture Justification REPLAN with the six-part evidence basis; the 19 binding conditions (C-01..C-19) and 10 phase challenges (PC-01..PC-10) from GRILL_v0.9.md are adopted as execution gates. 30 net-new requirements (REQ-061..REQ-090) derive from IDEATION_v0.9.md.

Milestone type: feature (P01..P10 ship feat phases; P00/P0X are chore/docs).

• Phase 0: Pre-execution (specify → clarify → research → ideate → plan → grill) — tag v0.8.0 (shipped; this is the phase you are reading)
• Phase P00: Deprecation sweep + bash tooling gate + render contract + doc banners (REQ-068,072,088,089,090; gates C-03,C-05,C-06,C-15..C-18) — tag v0.8.1 ✓
• Phase P0a1: Multi-namespace path resolver + config demotion + known_hosts flock (REQ-063,069,070,071; gate C-07) — tag v0.8.2 ✓
• Phase P0a2: Namespace CRUD + inheritance engine (REQ-082) — tag v0.8.3 ✓
• Phase P0b: Markdown jobspec parser + dispatcher + fuzz (REQ-064,067) — tag v0.8.4 ✓
• Phase P0c: Job/Service/DaemonSet schemas + emitter interface (REQ-074) — tag v0.8.5 ✓
• Phase P01: SSH-push transport (REQ-073) — tag v0.8.6 ✓
• Phase P02: Service block + Traefik emitter (REQ-077; gate C-10) — tag v0.8.7 ✓
• Phase P03/P04/P08: Update stanza + lifecycle hooks + socket plumbing (combined) — tag v0.8.8 ✓
• Phase P05: CLI-side scheduler + CEL constraints (REQ-083) — tag v0.8.9 ✓
• Phase P06: Task groups (multi-process services) — tag v0.8.10 ✓
• Phase P07a/b/c: Runtime abstraction — 5 backends (REQ-078; gate C-01) — tag v0.8.11 ✓
• Phase P09: Syncthing storage replication (REQ-081; gates C-02,C-14) — tag v0.8.12 ✓
• Phase P10: Lead rules + step-ca (REQ-076) — tag v0.8.13 ✓
• Phase P0X: Ship + audit (REQ-062,068) — tag v0.8.14 ✓

Milestone tag: v0.8.15 (final phase patch = milestone release per feature-milestone progressive-patch rule). Per-phase tags: v0.8.1…v0.8.14. P03/P04/P08 were combined into one phase; P07a/b/c were combined into one phase. Actual execution: 14 tagged phases. Tags run on the previous minor's patch line (v0.8.x) per branch-strategy.md. The milestone branch label uses the milestone number (milestone/v0.9-rearchitecture); no separate minor tag.

Per-phase REQ coverage (v0.9)

• P00 — Deprecation/migration/test-infra/persona/docs foundation (REQ-072, REQ-085, REQ-088, REQ-089, REQ-090)
• P0a1 — Path resolver + config demotion + known_hosts flock (REQ-063, REQ-069, REQ-070, REQ-071)
• P0a2 — Namespace inheritance resolver (REQ-082)
• P0b — Markdown parser + adapter + fuzz (REQ-064, REQ-067)
• P0c — Schemas + emitter interface (REQ-074)
• P01 — SSH-push transport (REQ-073)
• P02 — Service + Traefik emitter (REQ-077)
• P05 — CLI-side scheduler (REQ-083)
• P07a/b/c — Runtime abstraction (REQ-078) + step-ca integration (REQ-076)
• P09 — Syncthing replication (REQ-081)
• P0X — Coverage gate (REQ-062) + deprecation warnings (REQ-068)

v0.9 is a DIRECTION CHANGE — first in the project's history

Every prior milestone (v0.1–v0.8) explicitly said "the vision is unchanged; this milestone is not a direction change." v0.9 is the first milestone that reverses the vision's anti-patterns (daemon-on-every-node, internal CA, HCL-canonical, single-namespace, no-container-runtime, no-SPIFFE). The reversals are justified by the six-part evidence basis recorded in the PROJECT.md Supersession Table.

Milestone v0.10: Docs & Install Hardening — COMPLETE

Scope: close the documentation gap left by the v0.9 re-architecture and fix the release/install pipeline bug that caused install.sh to resolve to v0.4.5 instead of the latest release. The v0.9 re-architecture shipped a complete CLI surface (markdown jobspec, orca ns, orca node capacity, CLI-side scheduler, emitters, Traefik ingress) but no operator-facing reference documentation. This milestone ships that documentation plus a worked full-stack example with ingress configured, and hardens the release pipeline so every Gitea release carries a Linux binary asset.

Milestone type: feature (P1 ships fix phases; P2/P3/P4 ship docs phases; at least one non-docs phase makes this a feature milestone per the versioning logic).

• Phase 0: Pre-execution (specify → clarify → research → ideate → plan → grill) — tag v0.9.0
• Phase P1: release.sh + install.sh fix (REQ-097, REQ-098) — tag v0.9.1
• Phase P2: docs/cli.md + docs/jobspec.md + docs/ingress.md (REQ-091, REQ-092, REQ-093) — tag v0.9.2
• Phase P3: examples/full-stack/ (REQ-094) — tag v0.9.3
• Phase P4: README.md + docs/namespace.md refresh (REQ-095, REQ-096) — tag v0.9.4
• Phase P5: Final review + ship + audit (milestone release) — tag v0.9.5 = v0.10.0 milestone release

Milestone tag: v0.9.5 (final phase patch = milestone release per feature-milestone progressive-patch rule). Per-phase tags: v0.9.0…v0.9.5. Tags run on the previous minor's patch line (v0.9.x) per branch-strategy.md. The milestone branch label uses the milestone number (milestone/v0.10-docs-cli-examples); no separate minor tag.

Per-phase REQ coverage (v0.10 docs milestone)

• P1 — release.sh cross-build + asset verification (REQ-097); install.sh fallback walk (REQ-098)
• P2 — CLI reference (REQ-091); jobspec reference (REQ-092); ingress guide (REQ-093)
• P3 — full-stack examples (REQ-094)
• P4 — README refresh (REQ-095); namespace.md v0.9 layout (REQ-096)

Root cause of the v0.4.5 install (documented in RESEARCH_v0.10.md)

The v0.8.x releases (v0.8.0–v0.8.15) shipped with zero binary assets attached to their Gitea releases. install.sh resolves "latest" → v0.8.15, looks for orca-v0.8.15-linux-amd64.tar.gz, finds nothing, and errors out. The v0.4.5 install came from an earlier run or a pinned --version. The fix is forward: release.sh cross-builds amd64 and verifies the asset post-create; install.sh walks backward through releases if the latest lacks the asset.

Milestone v0.11: Production Hardening — COMPLETE

Scope: ship a cluster that operators can run. Builds on the v0.9 re-architecture foundation with the production-grade subsystems: secrets, transactions, ACL/SPIFFE, backup/restore, drain, recovery, and the v0.8→v1.0 migration. Phase 0 adopts 4 new load-bearing rules (R-017…R-020) and 23 new decisions (D-215…D-237) from 5 research docs covering ingress hardening, drift detection, platform-engineer positioning, strategic framing, and the systemd Path unit implementation. No new phases added; scope is folded into existing phases per operator decisions Q2=C (add 5 CLI commands), Q3=A (fold ingress into P15.5).

Milestone type: feature (multiple feat phases).

• Phase 0: Pre-execution (specify → clarify → research → plan → grill) — tag v0.10.0
• Phase P00: CLI cache layer (REQ-062 cache floor; R-008) — tag v0.10.1
• Phase P01: Metrics endpoint (hand-rolled text exposition) — tag v0.10.2
• Phase P01.5: SPIFFE SVID minting spike (REQ-076; gate C-08 — if spike fails, fall back to mTLS identity) — tag v0.10.3
• Phase P02: ACL (SPIFFE + token identities) — tag v0.10.4
• Phase P03: Secrets subsystem (REQ-080; gate C-19 threat model) — tag v0.10.5
• Phase P04: Backup/restore (tar + signed) — tag v0.10.6
• Phase P05: Drain + daemon drain-and-stop (REQ-061) + orca job migrate (REQ-116) — tag v0.10.7
• Phase P06: Alloc history (CLI-side SQLite retention; REQ-071 cache DB) + orca logs --all-nodes --since (REQ-117) — tag v0.10.8
• Phase P07: Recovery (orca restore) — tag v0.10.9
• Phase P08: Integration tests — expand hermetic harness (REQ-087) + drift-detection integration tests (auto-remediation, NFS fallback, cooldown, secret exclusion) — tag v0.10.10
• Phase P09: Collector + aggregator (opt-in; gates C-11, C-12, C-14) + drift-event aggregation extension (REQ-107, D-237) — tag v0.10.11
• Phase P10a: Transactional plane (REQ-075, REQ-079; gate C-09; gate C-23 cluster-wide vs ns-scoped txn distinction) — tag v0.10.12
• Phase P10b: Drift detection (R-018/R-019/R-020; REQ-103..REQ-113; orca drift CLI, systemd Path unit emitter, orca-drift-notify.sh, orca-remediate.sh, cadence config, --force+per-ns gate, orca system user, NFS detection) — depends on P10a — tag v0.10.13
• Phase P11: orca job lint (REQ-084) — tag v0.10.14
• Phase P12: orca job verify (dry-run txn through lead) — tag v0.10.15
• Phase P13: orca ns subcommands (full surface) + deprecation warnings (REQ-068) — tag v0.10.16
• Phase P14a: v0.8→v1.0 data migration (REQ-066; gate C-07; gate C-25 post-cutover verification + rollback; gate C-27 orca user creation) + orca upgrade --to-vX (REQ-115, thin wrapper, handles R-017 binding cutover) — tag v0.10.17
• Phase P14b: Daemon cutover + running-allocation adoption + orca cluster rotate-lead (REQ-114) — tag v0.10.18
• Phase P14c: Mixed-version tolerance + no-orca-on-server enforcement (REQ-065, REQ-086; implements C-13) — tag v0.10.19
• Phase P15: README quickstart (REQ-089; Nomad-inspired framing per Q5=A, honest-trade-offs table from research doc 3) — tag v0.10.20
• Phase P15.5: Threat model + security review (gate C-19; gate C-28 two sub-waves) + ingress hybrid (R-017; nft emitter REQ-099, Traefik binding REQ-100, orca doctor nft REQ-101, orca nft CLI REQ-102) + orca doctor mTLS (REQ-118) — tag v0.10.21
• Phase P16: Final review + ship + audit — v0.11.0 milestone release — tag v0.10.22 (v1.0.0 cut separately after UAT sign-off)

Milestone tag: v0.11.0 (the v0.11 milestone release tag; v1.0.0 is UAT-gated and cut separately after v0.11 completion per operator decision — the v1.0.0 tag marks production-ready sign-off, not a separate milestone). Per-phase patches run on the v0.10.x line per branch-strategy.md. Per-phase tags: v0.10.0…v0.10.21.

Per-phase REQ coverage (v0.11)

• P00 — CLI cache (R-008)
• P01.5 — SPIFFE spike (REQ-076; C-08)
• P03 — Secrets (REQ-080; C-19)
• P05 — Drain + daemon stop (REQ-061) + orca job migrate (REQ-116)
• P06 — Alloc history (REQ-071 cache DB) + orca logs --all-nodes --since (REQ-117)
• P08 — Integration tests (REQ-087) + drift-detection integration tests
• P09 — Collector + aggregator (C-11, C-12, C-14) + drift-event aggregation (REQ-107, D-237)
• P10a — Transactional plane (REQ-075, REQ-079; C-09; C-23)
• P10b — Drift detection (R-018/R-019/R-020; REQ-103..REQ-113)
• P11 — Job lint (REQ-084)
• P13 — ns subcommands + deprecation warnings (REQ-068)
• P14a/b/c — Migration (REQ-066, REQ-065, REQ-086; C-07, C-13) + orca upgrade (REQ-115) + orca cluster rotate-lead (REQ-114)
• P15 — README (REQ-089; Q5=A framing)
• P15.5 — Threat model (C-19) + ingress hybrid (R-017; REQ-099..REQ-102) + orca doctor mTLS (REQ-118)

New load-bearing rules adopted in Phase 0

• R-017 — Ingress hybrid: nft DNAT → Traefik on 127.0.0.1:8443; opt-out via --public-binding; service { ingress: native } per-workload opt-in
• R-018 — Drift cadence: default 60s; critical 5s + systemd Path units; standard 30s
• R-019 — Drift detector is a BACKSTOP; primary = systemd/Traefik/step-ca/Syncthing
• R-020 — Hard gate: applier refuses txns on pre-flight drift; --force + per-ns scoping override

Risk register (from grill + research, for ongoing monitoring)

• step-ca single-instance SPOF (mitigation: C-12 doc; v1.x HA via systemd failover)
• master.key passphrase-less 0600 (mitigation: C-19 threat model; consider OS keyring in v1.x)
• wasmtime CGO breaks cross-compile (mitigation: C-01 spike; fallback to podman/process primary)
• bash control plane drift (mitigation: C-15..C-18 render-format contract + bats gate)
• daemon cutover orphans running allocs (mitigation: P14b split; test adoption)
• 27→35+ phase scope (mitigation: C-04 resolved — operator accepted 40 phases; v0.11 grows to 24 phases per grill C-24 split of P10→P10a/P10b; scope folded in, no other new phases)
• R-020 deadlock (mitigation: --force flag + per-namespace scoping per Q4=A; drifted peer in ns-A doesn't block ns-B)
• P10 sizing (mitigation: P10 is the largest phase — drift detection + txn plane; grill may split into P10a/P10b if vertical slice is too large)
• Ingress default migration (mitigation: orca upgrade [REQ-115] handles Traefik binding cutover from :443 to 127.0.0.1:8443 for existing v0.9/v0.10 clusters)
• orca system user on peers (mitigation: net-new operational requirement; peer-setup emits useradd -r orca idempotently; documented in P10)

Deferred to v1.x (out of scope for v0.11)

• sqlite-wal-shared state backend (R-009 abstractions ship in v1.0; backend in v1.x)
• git state backend
• file+flock state backend
• orca cluster setup-shared UX
• HA step-ca (active/passive via systemd)
• Journald log shipping (optional centralized audit)
• Network policy (nftables snippets)
• GPU / TPU constraints

Deferred to v2.x (out of scope for v1.x)

• Full Nomad-HCL parser with no conversion round-trip
• Nomad-API subset for migrating existing Nomad fleets
• Nomad driver bridge
• Helm-equivalent templating (probably never)
• Service mesh beyond Traefik
• CRDs / Operators / Plugin model
• Leader-elected Raft coordinator
• External CA / Let's Encrypt / cert transparency
• Online-only features (HSTS, OCSP stapling, telemetry)

Milestone v0.12: Security Hardening (Zero-Trust Identity) — COMPLETE

Scope: comprehensive security hardening across the entire attack surface, including the operating system itself, plus adoption of a zero-trust identity model. The v0.12 threat-model review (Phase 0 RESEARCH) surfaced 25 distinct findings (F1..F25) spanning injection, traversal, ACL, audit, crypto, OS scripts, emitters, sudoers, system users, file modes, daemon auth, backup, SQLite, install.sh, and migration. v0.12 closes all of them and adopts R-021 (no Orca credentials) as the load-bearing architectural change: human identity is exclusively external (OIDC), machine identity is exclusively mTLS/SPIFFE, and no passwords/Orca-issued-tokens/CA-key-passphrases exist anywhere in the system.

The operator locked two architectural decisions: (1) bundled Dex by default + BYO external IdP override (D-239), and (2) master key seal-to-OIDC + Shamir 3-of-5 recovery (D-241). A third decision added WebAuthn (passkeys) as the bundled password-free authenticator for Dex (D-240) -- passkeys are public-key credentials (private key never leaves the authenticator), directly satisfying R-021.

Milestone type: feature (P04 OIDC+Dex and P05 WebAuthn ship feat phases; the rest are fix/chore/test/docs/refactor).

• Phase 0: Pre-execution (specify -> clarify -> research -> ideate -> plan -> grill) -- tag v0.11.0
• Phase P0[0-9]: Command injection fix (podman/wasm shellQuote) (REQ-119, F3) -- tag v0.11.1
• Phase P0[0-9]: Namespace path traversal fix (REQ-120, F4) -- tag v0.11.2
• Phase P0[0-9]: Txn apply path allowlist (REQ-121, F5) -- tag v0.11.3
• Phase P0[0-9]: OIDC client + bundled Dex (REQ-144; BYO-IdP override) -- tag v0.11.4
• Phase P0[0-9]: WebAuthn connector for Dex (REQ-148; passkeys, browser auth+register) -- tag v0.11.5
• Phase P0[0-9]: ACL rewrite to OIDC claims + enforcement (REQ-145, REQ-122, F1) -- tag v0.11.6
• Phase P0[0-9]: Remove all password/token paths (breaking; REQ-146, R-021, C-34) -- tag v0.11.7
• Phase P0[0-9]: Master key seal-to-OIDC + Shamir 3-of-5 (REQ-147, C-35) -- tag v0.11.8
• Phase P0[0-9]: Daemon auth hardening (REQ-123, REQ-124, F6, F24) -- tag v0.11.9
• Phase P0+: Audit log tamper-evidence (REQ-125, F2) -- tag v0.11.10
• Phase P0+: SVID chain validation (REQ-126, F9) -- tag v0.11.11
• Phase P0+: Backup symlink validation (REQ-127, F7) -- tag v0.11.12
• Phase P0+: step-ca /tmp hardening (REQ-128, F10) -- tag v0.11.13
• Phase P0+: Master key rotation (re-seal to OIDC; REQ-129, F12, C-30) -- tag v0.11.14
• Phase P0+: File-mode audit expansion (REQ-130, F13) -- tag v0.11.15
• Phase P0+: aggregate.sh JSON injection + drift-gate parse fix (REQ-131, F11, F18) -- tag v0.11.16
• Phase P0+: install.sh checksum+GPG verification (REQ-132, F14) -- tag v0.11.17
• Phase P0+: nftables ruleset hardening (REQ-133, F21) -- tag v0.11.18
• Phase P0+: sudoers hardening (REQ-134, F22) -- tag v0.11.19
• Phase P0+: System user consistency (REQ-135, F23) -- tag v0.11.20
• Phase P0+: SQLite file-mode + at-rest encryption (REQ-136, F8, C-31) -- tag v0.11.21
• Phase P0+: Migration safety + identity migration (REQ-137, F19, C-34) -- tag v0.11.22
• Phase P0+: Legacy CA/mTLS/daemon + step-ca password-provisioner deletion (REQ-138, F16; gate C-29: P06/P08/P09/P11) -- tag v0.11.23
• Phase P0+: known_hosts tightening + transport hardening (REQ-139, F15, F25) -- tag v0.11.24
• Phase P0+: Drift event authentication (REQ-140, F18) -- tag v0.11.25
• Phase P0+: Security integration test suite (REQ-141, C-33) -- tag v0.11.26
• Phase P0+: Zero-trust + OIDC + WebAuthn + threat-model docs (REQ-142) -- tag v0.11.27
• Phase P0+: Final review + ship + audit (milestone release) -- tag v0.11.28 = v0.12 milestone release

Milestone tag: v0.11.28 (final phase patch = milestone release per feature-milestone progressive-patch rule; no separate v0.12.0 tag). Per-phase tags: v0.11.0..v0.11.28 (29 tags). Tags run on the previous minor's patch line (v0.11.x) per branch-strategy.md. The milestone branch label uses the milestone number (milestone/v0.12-security-hardening); no separate minor tag.

The v1.0.0 production-ready tag stays deferred for post-v0.12 UAT (per v0.11 PRD; v0.12 is a minor feature milestone, not the v1.0 cut).

Per-phase REQ coverage (v0.12)

• P01 -- Command injection (REQ-119, F3)
• P02 -- Namespace path traversal (REQ-120, F4)
• P03 -- Txn apply path allowlist (REQ-121, F5)
• P04 -- OIDC client + bundled Dex (REQ-144; D-239, D-242, D-246)
• P05 -- WebAuthn connector (REQ-148; D-240, D-243, D-244, C-38)
• P06 -- ACL rewrite + enforcement (REQ-145, REQ-122, F1)
• P07 -- Remove password/token paths (REQ-146, R-021, C-34)
• P08 -- Master key seal-to-OIDC + Shamir (REQ-147, D-241, C-35)
• P09 -- Daemon auth (REQ-123, REQ-124, F6, F24)
• P10 -- Audit tamper-evidence (REQ-125, F2)
• P11 -- SVID chain validation (REQ-126, F9)
• P12 -- Backup symlink validation (REQ-127, F7)
• P13 -- step-ca /tmp hardening (REQ-128, F10)
• P14 -- Master key rotation (REQ-129, F12, C-30)
• P15 -- File-mode audit expansion (REQ-130, F13)
• P16 -- aggregate.sh JSON injection + drift-gate (REQ-131, F11, F18)
• P17 -- install.sh checksum+GPG (REQ-132, F14)
• P18 -- nftables ruleset hardening (REQ-133, F21)
• P19 -- sudoers hardening (REQ-134, F22)
• P20 -- System user consistency (REQ-135, F23)
• P21 -- SQLite file-mode + encryption (REQ-136, F8, C-31)
• P22 -- Migration safety + identity migration (REQ-137, F19, C-34)
• P23 -- Dual-write closure (REQ-138, F16; gate C-29)
• P24 -- known_hosts + transport hardening (REQ-139, F15, F25)
• P25 -- Drift event authentication (REQ-140, F18)
• P26 -- Security integration test suite (REQ-141, C-33)
• P27 -- Zero-trust + OIDC + WebAuthn + threat-model docs (REQ-142)
• P28 -- Final review + ship + audit (REQ-143)

New load-bearing rule adopted in Phase 0

• R-021 -- Orca never issues, stores, or accepts human-identity credentials. Human identity is exclusively external (OIDC). Machine identity is exclusively mTLS/SPIFFE. No passwords, no Orca-issued tokens, no CA-key passphrases.

Binding conditions (for GRILL ratification; C-29..C-38)

• C-29: P23 (dual-write closure) gated on P06/P08/P09/P11 all shipped.
• C-30: P14 (master key rotation) reversible; --dry-run mandatory; auto-rollback to old sealed key on any ns failure.
• C-31: P21 (SQLite encryption): CGO-free fallback to file-mode 0600 + documented threat if SQLCipher needs CGO. No CGO.
• C-32: Human-gate: leaked GITEA_TOKEN (F17) rotated + .env re-seeded before P28 ships. History-scrub best-effort, non-blocking. Escalation hook in ---ci---.
• C-33: P26 (security integration tests) in .coreci.yml validate, gates merges -- not opt-in.
• C-34: P07 (password/token removal) breaking. orca upgrade (P22) refuses v0.11 clusters using --password/bare-tokens without --accept-identity-migration. No silent breakage.
• C-35: P08 (Shamir recovery): 3-of-5 shards printed at seal time, operator stores offline. If IdP lost AND quorum unavailable -> cluster unrecoverable by design (documented residual risk). No backdoor.
• C-36: OIDC client secret (confidential clients) at ClusterDir()/oidc-client-secret (0600), rotatable via orca auth rotate-client-secret, never committed. Public PKCE clients avoid even this.
• C-37: P04 (bundled Dex): if WebAuthn proves infeasible, bundled Dex ships mTLS-client-cert-only; password-based upstreams require BYO external IdP. The "no Orca credentials" invariant holds regardless. (Largely moot -- WebAuthn solves it.)
• C-38: P05 (WebAuthn): RP ID must match the cluster's Traefik-served domain; orca auth init-idp configures it. HTTPS secure context via Traefik (step-ca cert). P26 integration tests use the WebAuthn virtual-authenticator API -- no hardware key required in CI.

Risk register (from grill + research, for ongoing monitoring)

• P07 breaking change (mitigation: C-34 migration gate)
• P08 master key seal is riskiest (mitigation: --dry-run, atomic, auto-rollback, C-35 Shamir recovery)
• P21 SQLite encryption may need CGO (mitigation: C-31 fallback to file-mode 0600)
• P23 dual-write closure high-impact (mitigation: gate C-29; full test coverage before deletion)
• P05 WebAuthn connector is new ground (mitigation: C-37 mTLS-client-cert fallback; virtual-authenticator tests in P26)
• Bundled Dex is a new systemd unit + Traefik route (mitigation: orca doctor oidc health check)
• C-32 human gate could stall final ship (mitigation: ship as v0.11.28-rc1 if rotation pending)
• 29 phases is large (mitigation: grill may split/merge; operator accepted "more than 20 if warranted")

Deferred to v1.x (out of scope for v0.12)

• HA step-ca (active/passive via systemd)
• sqlite-wal-shared / git / file+flock state backends
• OS keyring integration for master key (v0.12 uses OIDC seal instead)
• Full cluster-rolling-upgrade orchestrator (v0.12 ships the thin orca upgrade wrapper only)
• Live-migrate with storage replication (v0.12 ships drain+reschedule only)
• Journald log shipping (optional centralized audit)
• Network policy (nftables snippets beyond the ingress ruleset)
• GPU / TPU constraints

Deferred to v2.x (out of scope for v1.x)

• Full Nomad-HCL parser with no conversion round-trip
• Nomad-API subset for migrating existing Nomad fleets
• Nomad driver bridge
• Helm-equivalent templating (probably never)
• Service mesh beyond Traefik
• CRDs / Operators / Plugin model
• Leader-elected Raft coordinator
• External CA / Let's Encrypt / cert transparency
• Online-only features (HSTS, OCSP stapling, telemetry)

Milestone v0.13: Production Hardening Round 2 + UAT Plan — COMPLETE

Scope: final production hardening round before the v1.0.0 production-ready tag. Three deep codebase sweeps (security, reliability, feature/doc claims) surfaced ~60 gaps beyond v0.12 — the most critical being that orca job run runs locally via exec.CommandContext and never invokes the scheduler/emitter/SSH-push path (the documented deployment model is non-functional), jobspec schedule:/timeout: are silently dropped by the markdown parser (DaemonSet is fundamentally broken), acl.Check is called zero times in the codebase (v0.12's headline zero-trust feature is library-complete but not wired), and several command-injection vectors remain (orca logs --job backtick RCE via %q, tar-slip in restore, sudoers injection, etc.). v0.13 closes all critical/high/medium findings and delivers the UAT plan + signoff script that gates the v1.0.0 cut.

Load-bearing architectural changes:

• R-022 — orca job run deploys to remote nodes via the scheduler → emitter → SSH-push pipeline. The local exec.CommandContext path is removed. Constraints/capacity/affinity are enforced. This makes the documented deployment model functional and is the prerequisite for the UAT plan.
• R-023 — Zero-trust enforcement is operationally wired: acl.Check is invoked on every daemon handler + sshpush + txn apply path; acl.json is 0600; audit actor carries OIDC sub/SVID; WebAuthn registration requires auth; cluster seal/unseal + doctor audit/doctor modes CLI commands exist.

Phases (14 total: P0 + P01..P12 + P13 final)

• Phase P0: Pre-execution (SPECIFY→CLARIFY→RESEARCH→IDEATE→PLAN→GRILL) — tag v0.12.0
• Phase P01: Toolchain & dependency vulns (REQ-149) — tag v0.12.1
• Phase P02: Input validation & injection hardening (REQ-150) — tag v0.12.2
• Phase P03: Scheduler/deployment wiring + jobspec parser (REQ-151, REQ-152) — tag v0.12.3
• Phase P04: ACL enforcement + WebAuthn registration auth (REQ-153) — tag v0.12.4
• Phase P05: Seal/audit CLI + chain race + key zeroing (REQ-154) — tag v0.12.5
• Phase P06: auth init-idp real + auth register (REQ-155) — tag v0.12.6
• Phase P07: Concurrency safety (REQ-156) — tag v0.12.7
• Phase P08: Transport & SSH safety (REQ-157) — tag v0.12.8
• Phase P09: Migration & operational safety (REQ-158) — tag v0.12.9
• Phase P10: Observability & metrics (REQ-159) — tag v0.12.10
• Phase P11: Doc drift round 2 (REQ-160) — tag v0.12.11
• Phase P12: --type linux + UAT plan + signoff script (REQ-161, REQ-162, REQ-163) — tag v0.12.12
• Phase P13: Final review + ship + audit (milestone release) — tag v0.12.13 = v0.13 milestone release

Milestone tag: v0.12.13 (final phase patch = milestone release per feature-milestone rule; no separate v0.13.0 tag). Per-phase tags: v0.12.0..v0.12.13 (14 tags). Tags run on the previous minor's patch line (v0.12.x). The milestone branch label uses the milestone number (milestone/v0.13-production-hardening-2); no separate minor tag.

The v1.0.0 production-ready tag stays deferred for post-v0.13 UAT signoff (operator runs scripts/uat-signoff.sh, pastes output back; CI agent verifies and cuts v1.0.0).

Per-phase REQ coverage (v0.13)

• P01 — Toolchain bump (REQ-149)
• P02 — Injection hardening (REQ-150)
• P03 — Scheduler wiring + jobspec parser (REQ-151, REQ-152)
• P04 — ACL enforcement + WebAuthn reg auth (REQ-153)
• P05 — Seal/audit CLI + chain race + key zeroing (REQ-154)
• P06 — auth init-idp real + auth register (REQ-155)
• P07 — Concurrency safety (REQ-156)
• P08 — Transport & SSH safety (REQ-157)
• P09 — Migration & operational safety (REQ-158)
• P10 — Observability & metrics (REQ-159)
• P11 — Doc drift round 2 (REQ-160)
• P12 — --type linux + UAT plan + signoff (REQ-161, REQ-162, REQ-163)
• P13 — Final review + ship + audit

New load-bearing rules adopted in Phase 0

• R-022 — orca job run deploys to remote nodes via scheduler → emitter → SSH-push. Local exec path removed. Constraints/capacity/ affinity enforced.
• R-023 — Zero-trust enforcement is operationally wired: acl.Check on every request path; acl.json 0600; audit actor = OIDC sub/SVID; WebAuthn registration requires auth.

Binding conditions (for GRILL ratification — C-39..C-49)

• C-39: P03 (scheduler wiring) is the riskiest phase — changes the core job run path. Must not break existing job run (local fallback if no remote nodes registered). Full test coverage before P04 ships.
• C-40: P04 (ACL enforcement) is deny-by-default — must not lock out the operator. Bootstrap ACL grants cluster-admin to the init cert's SPIFFE SVID. Staged rollout: log-only mode for first run, enforce after bootstrap ACL verified.
• C-41: P05 (seal) — C-35 residual risk still applies (IdP lost + Shamir quorum unavailable → cluster unrecoverable). No backdoor.
• C-42: P12 (UAT plan + signoff) is the v1.0 gate artifact. If P01..P11 slip, P12 still ships (honest signal via failing assertions). The signoff script is idempotent and read-only.
• C-43: verify-reqs bold-format regex must be fixed in P11 so
• C-44: P03 MUST fail-closed when scheduler selects a node but SSH-push fails. Local fallback only when len(registeredNodes)==0. Test case mandatory.
• C-45: P04 MUST implement log-only/dry-run mode as default for first invocation after ACL wiring. Enforce mode after bootstrap ACL verified.
• C-46: P12 dependency table MUST include P05 (seal) and P06 (auth init-idp) in addition to P03 and P04.
• C-47: P12 uat-signoff.sh MUST include explicit assertions for: (a) job deployed to remote node, (b) ACL deny-by-default, (c) seal/unseal round-trip, (d) OIDC health check.
• C-48: P12 docs/uat.md MUST document hardware prerequisites (Proxmox VE 8/9 host required). Alternative UAT path (3x Ubuntu, Proxmox claims skipped) MUST be documented.
• C-49: Plan narrative MUST soften "last hardening round" to "last hardening round before UAT validation." UAT will likely surface 3-7 issues requiring patch release. the consistency gate works for v0.12 AND v0.13.

Risk register (for grill + research, for ongoing monitoring)

• P03 scheduler wiring is riskiest (mitigation: C-39 local fallback)
• P04 ACL deny-by-default could lock out operator (mitigation: C-40 bootstrap ACL + staged rollout)
• P05 seal residual risk (mitigation: C-41 documented, no backdoor)
• P02 injection hardening is high-count (11 sub-fixes; mitigation: each is small and independently testable)
• 14 phases is large (mitigation: operator accepted "no limit on phases"; many phases are small fix bundles)
• UAT plan depends on P03 (scheduler) being functional (mitigation: P12 ships regardless; failing assertions are honest signal)

Deferred to v1.x (out of scope for v0.13) — unchanged from v0.12

• HA step-ca (active/passive via systemd)
• sqlite-wal-shared / git / file+flock state backends
• OS keyring integration for master key
• Full cluster-rolling-upgrade orchestrator (v0.13 ships the thin orca upgrade wrapper only)
• Live-migrate with storage replication (v0.13 ships drain+reschedule only)
• Journald log shipping (optional centralized audit)
• Network policy (nftables snippets beyond the ingress ruleset)
• GPU / TPU constraints
• jobspec health prober (v0.13 adds lint warning; enforcement deferred)
• jobspec update rolling/canary controller (v0.13 adds lint warning; enforcement deferred)
• jobspec schedule.cron scheduler loop (v0.13 adds lint warning; enforcement deferred)

Milestone v0.14: Ingress Bootstrap Completeness — IN PROGRESS

Scope: ensure that linux & proxmox types are properly bootstrapped with traefik during cluster init or node join. All cluster endpoints are provisioned as sockets (R-007); routing between jobs and services depends on traefik being present and properly configured. v0.13 shipped traefik binary + systemd unit + empty dynamic dir but never wrote the static config nor applied nft rules. v0.14 replaces the binary+systemd model with a podman container running a custom orca-traefik image, and completes the nft SNAT+DNAT ingress stack on every node type.

New load-bearing rule: R-024 — Traefik runs exclusively as a podman container from the orca-traefik image published per release. Every orca-managed ingress surface bootstraps nft DNAT + SNAT/MASQUERADE + podman run --restart=always --network host with dynamic-config + step-ca root CA volume mounts.

Three topologies:

1. Linux: host → nft → podman run orca-traefik (host network)
2. Proxmox Native: host → nft → LXC (nesting=1) → podman run orca-traefik
3. Proxmox Floating-IP: LXC (owns floating IP) → nft (inside LXC) → podman run orca-traefik

Milestone type: feature (multiple feat phases). Tags on v0.13.x patch line: v0.13.0 (P0) ... v0.13.8 (P8 final = v0.14 milestone release).

• Phase 0: Pre-execution (SPECIFY→CLARIFY→RESEARCH→PLAN→GRILL) — tag v0.13.0
• Phase 1: orca-traefik container image + release pipeline (REQ-171) — tag v0.13.1
• Phase 2: Podman traefik reconciler — replace binary+systemd install (REQ-172) — tag v0.13.2
• Phase 3: nft SNAT+DNAT + orca init ingress bootstrap (REQ-173) — tag v0.13.3
• Phase 4: orca node join --type linux remote ingress bootstrap (REQ-174) — tag v0.13.4
• Phase 5: Proxmox native ingress mode — LXC + podman traefik (REQ-175) — tag v0.13.5
• Phase 6: Proxmox floating-IP LXC ingress + interactive prompt (REQ-176) — tag v0.13.6
• Phase 7: doctor ingress + docs + integration tests (REQ-177,178,179) — tag v0.13.7
• Phase 8: Final review + ship + audit (milestone release) — tag v0.13.8 = v0.14 milestone release

Per-phase REQ coverage (v0.14)

• P1 — Dockerfile.traefik + release pipeline (REQ-171)
• P2 — Podman traefik reconciler (REQ-172)
• P3 — nft SNAT+DNAT + init bootstrap (REQ-173)
• P4 — Linux node join remote ingress (REQ-174)
• P5 — Proxmox native ingress — LXC + podman (REQ-175)
• P6 — Proxmox floating-IP LXC ingress (REQ-176)
• P7 — doctor ingress + docs + tests (REQ-177,178,179)
• P8 — Final review + ship + audit

v0.14 is a continuation milestone, not a direction change

The vision ("minimalist, offline-first, CLI-first orchestration engine") is unchanged. v0.14 completes the ingress bootstrap that v0.13 left non-functional (binary installed but no config, no nft applied). The podman-container model is the operator's constraint; the architecture's socket+traefik routing design (R-007, R-017) is unchanged.