orca/.ciagent/ROADMAP.md

# Roadmap: Orca

## Milestone v0.1: Foundation — **COMPLETE**

- [x] Phase 0: Project Initialization & Specification
- [x] Phase 1: Core CLI Skeleton & Command Parsing
- [x] Phase 2: Basic Node Management (Join/Leave)
- [x] Phase 3: Simple Task Execution Engine
- [x] Phase 4: Local State Persistence
- [x] Phase 5: Basic Health Checking
- [x] Phase 6: CoreCI Full Release Flow

**Tagged `v0.2.0`** (next-minor per feature-milestone promotion rule).

## Deferred to v0.2 (out of scope for v0.1)

- Multi-node scheduling (D-004 decision: single-node only in v0.1)
- mTLS for inter-node communication (REQ-011, REQ-023)
- `gosec` + `govulncheck` in CI pipeline (REQ-014)
- `iter.Seq` streaming job lists (REQ-022)
- Frontend / devops personas (no web UI; CoreCI handles release)

## Milestone v0.2: Networking, Observability, Security Hardening — **COMPLETE (merged to main via v0.3)**

Scope: extend v0.1 with secure cross-node transport, multi-node scheduling,
richer CI security scanning, and streaming I/O.

- [x] Phase 8: mTLS handshake + internal CA with CSR join (Wave 1) — shipped v0.2.1
- [x] Phase 9: Multi-node scheduling & job dispatch (Wave 1) — shipped v0.2.2
- [x] Phase 10: `gosec` + `govulncheck` + gitleaks in CI (Wave 2) — shipped v0.2.3
- [x] Phase 11: `iter.Seq` streaming job/node lists (Wave 2) — **completed in v0.3 P01** (shipped v0.3.1)

**Milestone tag**: `v0.4.0` (shipped — v0.2 work merged to main via v0.3 milestone).

Per-phase tags: `v0.2.1` (P01), `v0.2.2` (P02), `v0.2.3` (P03) — all shipped.

## Milestone v0.3: Scheduling & Streaming Completion — **COMPLETE**

Scope: complete the two work items deferred from v0.2 that were not
already shipped in P08-P10. A re-init SPECIFY codebase audit confirmed
that REQ-014/027/028/029/031/037/039/040 all shipped in P08-P10 despite
stale REQUIREMENTS.md marking them Pending. The remaining work is lean:

- [x] Phase 0: Pre-execution (specify → clarify → research → plan → grill) — shipped v0.3.0
- [x] Phase 1: `iter.Seq` streaming for `--watch` flags (REQ-022, REQ-030) — shipped v0.3.1
- [x] Phase 2: `orca doctor` network + db full implementation (REQ-032 completion) — shipped v0.3.2
- [x] Phase 3: Final review + ship + audit (milestone release) — shipped v0.3.3

**Milestone tag**: `v0.4.0` (next-minor per feature-milestone promotion rule).

Per-phase tags: `v0.3.0` (P0), `v0.3.1` (P01), `v0.3.2` (P02), `v0.3.3` (P03 final = milestone release).
Per `.ciagent/RELEASE_POLICY.md`, every phase tag produces a Gitea release.

### Per-phase REQ coverage

- **P01 — `iter.Seq` streaming for `--watch` flags**
  - REQ-022 (`iter.Seq` for streaming job lists, Go 1.25+)
  - REQ-030 (`--watch` output format mode: table default vs streaming JSON per event)
  - Applies to both `orca job list --watch` and `orca node list --watch`
    (D-024, per ARCHITECTURE.md CLI layer + D-017)

- **P02 — `orca doctor` network + db full implementation**
  - REQ-032 (completion: network reachability via mTLS `/healthz` probe,
    db integrity via `PRAGMA integrity_check` + migration version)
  - Replaces `NetworkStub` and `DBStub` from v0.2 P01

### v0.3 is a completion milestone, not a direction change

The vision ("minimalist, offline-first, CLI-first orchestration
engine") is unchanged. v0.3 closes out the v0.2 deferrals and merges
the accumulated v0.2 work to main.

## Milestone v0.5: Distribution — **COMPLETE**

Scope: make Orca installable, distributable, and containerized. The
engine functionality from v0.1–v0.3 is unchanged; this milestone is
purely about delivery surface.

- [x] Phase 0: Pre-execution (specify → clarify → research → plan) — shipped `v0.4.1` (+ repo public)
- [x] Phase 1: Namespace unification (`ORCA_HOME` + `--system`) (REQ-041, REQ-042) — shipped `v0.4.2`
- [x] Phase 2: `install.sh` + in-place update + README quickstart (REQ-043, REQ-044) — shipped `v0.4.3`
- [x] Phase 3: Docker release (Dockerfile + Gitea container registry) (REQ-046) — shipped `v0.4.4`
- [x] Phase 4: Final review + ship + audit (milestone release) — shipped `v0.4.5`

**Operational prerequisite (P0 ship)**: repo + org visibility flipped to
public (REQ-045) — unauth releases API + asset download + docker pull all
verified HTTP 200.

**Milestone tag**: `v0.4.5` (final phase patch = milestone release per
feature-milestone promotion rule). Per-phase tags: `v0.4.1`…`v0.4.5`.

## Milestone v0.6: Node Bootstrap & Proxmox

## Milestone v0.6: Node Bootstrap & Proxmox — **COMPLETE**

Scope: make `orca init` produce a fully working single-node cluster
(CA + server cert + DB + localhost node registered with auto-detected
OS), and add Proxmox 8 & 9 as a first-class remote node type joined
over SSH with least-privilege role delegation.

- [x] Phase 0: Pre-execution (specify → clarify → research → plan → grill) — tag `v0.5.0`
- [x] Phase 1: `orca init` full bootstrap + localhost node + schema 0006 (REQ-047, REQ-048, REQ-049) — tag `v0.5.1`
- [x] Phase 2: Proxmox SSH join + OrcaOperator role + sudoers allowlist (REQ-050, REQ-051) — tag `v0.5.2`
- [x] Phase 3: `doctor os` + `doctor proxmox` SSH probe + audit logging (REQ-052) — tag `v0.5.3`
- [x] Phase 4: Final review + ship + audit (milestone release) — tag `v0.5.4`

**Milestone type**: feature (P1/P2/P3 ship `feat` phases).
**Milestone tag**: `v0.5.4` (final phase patch = milestone release per
feature-milestone promotion rule). Per-phase tags: `v0.5.0`…`v0.5.4`.

Tags run on the previous minor's patch line (v0.5.x) per
branch-strategy.md. The milestone branch label uses the milestone
number (`milestone/v0.6-node-bootstrap-proxmox`); no separate minor
tag is created.

## Milestone v0.7: Hardening & Completion — **COMPLETE**

Scope: NFR milestone closing gaps surfaced by the v0.7 IDEATE stage —
an unreachable command tree, a missing config file layer, low test
coverage in core packages, and the long-deferred pprof endpoint.

- [x] Phase 0: Pre-execution (specify → clarify → research → ideate → plan) — tag `v0.6.0` (shipped)
- [x] Phase 1: Register `orca cert` command tree + cert_repo tests (REQ-053) — tag `v0.6.1` (shipped)
- [x] Phase 2: HCL config file parsing — `internal/config` package (REQ-054) — tag `v0.6.2` (shipped)
- [x] Phase 3: Test coverage uplift — engine/transport/proxmox/audit ≥ 50% (REQ-055) — tag `v0.6.3` (shipped)
- [x] Phase 4: `--pprof` opt-in on `orca daemon` (REQ-056) — tag `v0.6.4` (shipped)
- [x] Phase 5: Final review + ship + audit (milestone release) — tag `v0.6.5` (shipped)

**Milestone type**: NFR (all phases are fix/test/chore; no `feat` phases).
**Milestone tag**: `v0.6.5` (final phase patch = milestone release per
NFR-milestone progressive-patch rule). Per-phase tags: `v0.6.0`…`v0.6.5`.
Tags run on the previous minor's patch line (v0.6.x) per
branch-strategy.md. The milestone branch label uses the milestone
number (`milestone/v0.7-hardening-completion`); no separate minor tag.

## Milestone v0.8: Coverage & Trust Hardening — **COMPLETE**

Scope: continue the v0.7 hardening theme. v0.7 P03's ≥ 50% floor left
six packages still under 50% (engine 8.3%, proxmox 5.1%, cli 27.6%,
transport 26.3%, store 46.7%, jobspec 47.6%) and three packages with
no tests at all (`internal/audit`, `internal/certpaths`, `cmd/orca`).
v0.8 also closes the two SSH-trust "future enhancement" hooks deferred
in v0.6 (D-035 `--host-key-fingerprint` pre-pin, RESEARCH_v0.6 §80
`orca node key-reset`) and adds a requirements-hygiene gate to prevent
the stale-REQ-status drift seen after v0.7 ship.

- [x] Phase 0: Pre-execution (specify → clarify → research → plan → grill) — tag `v0.7.0` (shipped)
- [x] Phase 1: Test coverage uplift round 2 — 6 packages to ≥ 70%, 3 zero-test packages to first tests (REQ-057) — tag `v0.7.1` (shipped)
- [x] Phase 2: SSH trust hardening — `--host-key-fingerprint` pre-pin + `orca node key-reset` + TOFU bugfix + `HostKeyFingerprint` population (REQ-058, REQ-059) — tag `v0.7.2` (shipped)
- [x] Phase 3: Requirements-hygiene gate — `make verify-reqs` + verify assertion (REQ-060) — tag `v0.7.3` (shipped)
- [x] Phase 4: Final review + ship + audit (milestone release) — tag `v0.7.4` (shipped)

**Milestone type**: NFR (P01 test, P02 chore on trust surface per
D-043, P03 chore, P04 docs/review). Final phase patch IS the milestone
release per NFR-milestone progressive-patch rule. Per-phase tags:
`v0.7.0`…`v0.7.4`. Tags run on the previous minor's patch line (v0.7.x)
per branch-strategy.md. The milestone branch label uses the milestone
number (`milestone/v0.8-coverage-trust-hardening`); no separate minor
tag.

### Per-phase REQ coverage

- **P01 — Coverage uplift round 2**
  - REQ-057 (raise `internal/engine`, `internal/proxmox`,
    `internal/cli`, `internal/transport`, `internal/store`,
    `internal/jobspec` to ≥ 70%; add first tests for `internal/audit`,
    `internal/certpaths`, `cmd/orca`)

- **P02 — SSH trust hardening**
  - REQ-058 (`--host-key-fingerprint <sha256>` pre-pin flag on
    `orca node join --type proxmox`; fail fast on mismatch; supersedes
    TOFU for pre-pinned deployments)
  - REQ-059 (`orca node key-reset <node>` clears persisted SSH host
    key so next `doctor proxmox`/dispatch re-pins via TOFU or
    `--host-key-fingerprint`)

- **P03 — Requirements-hygiene gate**
  - REQ-060 (`make verify-reqs` target + verify-stage assertion:
    every REQ `Complete` in ROADMAP.md has matching `Complete` row in
    REQUIREMENTS.md; enforced in CI `validate` pipeline)

### v0.8 is a continuation milestone, not a direction change

The vision ("minimalist, offline-first, CLI-first orchestration
engine") is unchanged. v0.8 closes the coverage debt left by v0.7's
50% floor and the trust-surface gaps explicitly deferred in v0.6.

## Milestone v0.9: Re-architecture Foundation & Workloads

**Scope**: This milestone SUPERSPEDES the shipped v0.1–v0.8 architecture per
the adopted PRD (`.ciagent/PRD_v0.9.md`). The re-architecture is justified on
six grounds recorded in the PROJECT.md Supersession Table: (1) the v0.8 daemon
model is operationally failing, (2) step-ca is externally mandated, (3)
multi-tenancy is a hard product requirement, (4) WASM is a hard workload
requirement, (5) SSH-push is the only viable deployment target, (6) vision
correction. The 16 load-bearing rules (R-001…R-016) are invariants. The
ci-griller reviewed the re-architecture adversarially; the user overrode the
Re-architecture Justification REPLAN with the six-part evidence basis; the
19 binding conditions (C-01..C-19) and 10 phase challenges (PC-01..PC-10)
from `GRILL_v0.9.md` are adopted as execution gates. 30 net-new requirements
(REQ-061..REQ-090) derive from `IDEATION_v0.9.md`.

**Milestone type**: feature (P01..P10 ship `feat` phases; P00/P0X are
chore/docs).

- [ ] Phase 0: Pre-execution (specify → clarify → research → ideate → plan → grill) — tag `v0.8.0` (shipped; this is the phase you are reading)
- [ ] Phase P00: Deprecation sweep + migration-ordering decision + txn-design spike + hermetic test-infra bootstrap + persona reactivation + doc banners (REQ-072, REQ-085, REQ-088, REQ-089, REQ-090; gates C-03 ✅, C-05, C-06, C-15..C-18) — tag `v0.8.1`
- [ ] Phase P0a1: Multi-namespace path resolver + config HCL demotion + known_hosts flock (REQ-063, REQ-069, REQ-070, REQ-071; gate C-07) — tag `v0.8.2`
- [ ] Phase P0a2: Namespace CRUD + inheritance engine (REQ-082) — tag `v0.8.3`
- [ ] Phase P0b: Markdown jobspec parser + dispatcher + fuzz (REQ-064, REQ-067) — tag `v0.8.4`
- [ ] Phase P0c: Job/Service/DaemonSet schemas + emitter interface (REQ-074) — tag `v0.8.5`
- [ ] Phase P01: SSH-push transport + host-path volumes (REQ-073) — tag `v0.8.6`
- [ ] Phase P02: Service block + checks + restart + Traefik emitter (REQ-077; gate C-10) — tag `v0.8.7`
- [ ] Phase P03: Update stanza (rolling/canary) — tag `v0.8.8`
- [ ] Phase P04: Lifecycle hooks (systemd ExecStop) — tag `v0.8.9`
- [ ] Phase P05: Constraints & affinity (CEL) + CLI-side scheduler (REQ-083) — tag `v0.8.10`
- [ ] Phase P06: Task groups (multi-process services) — tag `v0.8.11`
- [ ] Phase P07a: Process + podman runtimes (REQ-078) — tag `v0.8.12`
- [ ] Phase P07b: wasmtime runtime (REQ-078; **gate C-01** — CGO eval) — tag `v0.8.13`
- [ ] Phase P07c: pve-vm + pve-ct runtimes (REQ-078; extends REQ-076) — tag `v0.8.14`
- [ ] Phase P08: Socket plumbing (R-007) — tag `v0.8.15`
- [ ] Phase P09: Storage replication via Syncthing (REQ-081; **gates C-02, C-14**) — tag `v0.8.16`
- [ ] Phase P10: Lead rules + migration (REQ-076 step-ca integration) — tag `v0.8.17`
- [ ] Phase P0X: Ship + audit (REQ-062 coverage gate; REQ-068 deprecation warnings) — tag `v0.8.18`

**Milestone tag**: `v0.8.18` (final phase patch = milestone release per
feature-milestone progressive-patch rule). Per-phase tags: `v0.8.1`…`v0.8.18`.
Tags run on the previous minor's patch line (v0.8.x) per branch-strategy.md.
The milestone branch label uses the milestone number
(`milestone/v0.9-rearchitecture`); no separate minor tag.

### Per-phase REQ coverage (v0.9)

- **P00** — Deprecation/migration/test-infra/persona/docs foundation (REQ-072, REQ-085, REQ-088, REQ-089, REQ-090)
- **P0a1** — Path resolver + config demotion + known_hosts flock (REQ-063, REQ-069, REQ-070, REQ-071)
- **P0a2** — Namespace inheritance resolver (REQ-082)
- **P0b** — Markdown parser + adapter + fuzz (REQ-064, REQ-067)
- **P0c** — Schemas + emitter interface (REQ-074)
- **P01** — SSH-push transport (REQ-073)
- **P02** — Service + Traefik emitter (REQ-077)
- **P05** — CLI-side scheduler (REQ-083)
- **P07a/b/c** — Runtime abstraction (REQ-078) + step-ca integration (REQ-076)
- **P09** — Syncthing replication (REQ-081)
- **P0X** — Coverage gate (REQ-062) + deprecation warnings (REQ-068)

### v0.9 is a DIRECTION CHANGE — first in the project's history

Every prior milestone (v0.1–v0.8) explicitly said "the vision is unchanged;
this milestone is not a direction change." v0.9 is the first milestone that
reverses the vision's anti-patterns (daemon-on-every-node, internal CA,
HCL-canonical, single-namespace, no-container-runtime, no-SPIFFE). The
reversals are justified by the six-part evidence basis recorded in the
PROJECT.md Supersession Table.

## Milestone v0.10: Production Hardening

**Scope**: ship a cluster that operators can run. Builds on the v0.9
re-architecture foundation with the production-grade subsystems:
secrets, transactions, ACL/SPIFFE, backup/restore, drain, recovery, and
the v0.8→v1.0 migration.

**Milestone type**: feature (multiple `feat` phases).

- [ ] Phase 0: Pre-execution (specify → clarify → research → plan → grill) — tag `v0.9.0`
- [ ] Phase P00: CLI cache layer (REQ-062 cache floor; R-008) — tag `v0.9.1`
- [ ] Phase P01: Metrics endpoint (hand-rolled text exposition) — tag `v0.9.2`
- [ ] Phase P01.5: SPIFFE SVID minting spike (REQ-076; **gate C-08** — if spike fails, fall back to mTLS identity) — tag `v0.9.3`
- [ ] Phase P02: ACL (SPIFFE + token identities) — tag `v0.9.4`
- [ ] Phase P03: Secrets subsystem (REQ-080; **gate C-19** threat model) — tag `v0.9.5`
- [ ] Phase P04: Backup/restore (tar + signed) — tag `v0.9.6`
- [ ] Phase P05: Drain + daemon drain-and-stop (REQ-061) — tag `v0.9.7`
- [ ] Phase P06: Alloc history (CLI-side SQLite retention; REQ-071 cache DB) — tag `v0.9.8`
- [ ] Phase P07: Recovery (`orca restore`) — tag `v0.9.9`
- [ ] Phase P08: Integration tests — expand hermetic harness (REQ-087) — tag `v0.9.10`
- [ ] Phase P09: Collector + aggregator (opt-in; **gates C-11, C-12, C-14**) — tag `v0.9.11`
- [ ] Phase P10: Transactional plane (REQ-075, REQ-079; **gate C-09** orca-pull.sh failure contract) — tag `v0.9.12`
- [ ] Phase P11: `orca job lint` (REQ-084) — tag `v0.9.13`
- [ ] Phase P12: `orca job verify` (dry-run txn through lead) — tag `v0.9.14`
- [ ] Phase P13: `orca ns` subcommands (full surface) + deprecation warnings (REQ-068) — tag `v0.9.15`
- [ ] Phase P14a: v0.8→v1.0 data migration (REQ-066; **gate C-07** CA migration spec) — tag `v0.9.16`
- [ ] Phase P14b: Daemon cutover + running-allocation adoption — tag `v0.9.17`
- [ ] Phase P14c: Mixed-version tolerance + no-orca-on-server enforcement (REQ-065, REQ-086; implements C-13) — tag `v0.9.18`
- [ ] Phase P15: README quickstart (REQ-089) — tag `v0.9.19`
- [ ] Phase P15.5: Threat model + security review (**gate C-19**) — tag `v0.9.20`
- [ ] Phase P16: Final review + ship + audit — **v0.10.0 milestone release** — tag `v0.9.21` (v1.0.0 cut separately after UAT sign-off)

**Milestone tag**: `v0.10.0` (the v0.10 milestone release tag; v1.0.0 is
UAT-gated and cut separately after v0.10 completion per operator decision —
the v1.0.0 tag marks production-ready sign-off, not a separate milestone).
Per-phase patches run on the v0.9.x line per branch-strategy.md. Per-phase
tags: `v0.9.0`…`v0.9.21`.

### Per-phase REQ coverage (v0.10)

- **P00** — CLI cache (R-008)
- **P01.5** — SPIFFE spike (REQ-076; C-08)
- **P03** — Secrets (REQ-080; C-19)
- **P05** — Drain + daemon stop (REQ-061)
- **P06** — Alloc history (REQ-071 cache DB)
- **P08** — Integration tests (REQ-087)
- **P10** — Transactional plane (REQ-075, REQ-079; C-09)
- **P11** — Job lint (REQ-084)
- **P13** — ns subcommands + deprecation warnings (REQ-068)
- **P14a/b/c** — Migration (REQ-066, REQ-065, REQ-086; C-07, C-13)
- **P15** — README (REQ-089)
- **P15.5** — Threat model (C-19)

### Risk register (from grill, for ongoing monitoring)

- **step-ca single-instance SPOF** (mitigation: C-12 doc; v1.x HA via systemd failover)
- **master.key passphrase-less 0600** (mitigation: C-19 threat model; consider OS keyring in v1.x)
- **wasmtime CGO breaks cross-compile** (mitigation: C-01 spike; fallback to podman/process primary)
- **bash control plane drift** (mitigation: C-15..C-18 render-format contract + bats gate)
- **daemon cutover orphans running allocs** (mitigation: P14b split; test adoption)
- **27→35+ phase scope** (mitigation: C-04 resolved — operator decision: keep 2 milestones v0.9 + v0.10, keep all phases, v1.0 is UAT-gated after v0.10; current count v0.9=18 + v0.10=22 = 40 phases, exceeds 35 soft limit but operator accepted)

## Deferred to v1.x (out of scope for v0.10)

- `sqlite-wal-shared` state backend (R-009 abstractions ship in v1.0; backend in v1.x)
- `git` state backend
- `file+flock` state backend
- `orca cluster setup-shared` UX
- HA `step-ca` (active/passive via systemd)
- Journald log shipping (optional centralized audit)
- Network policy (`nftables` snippets)
- GPU / TPU constraints

## Deferred to v2.x (out of scope for v1.x)

- Full Nomad-HCL parser with no conversion round-trip
- Nomad-API subset for migrating existing Nomad fleets
- Nomad driver bridge
- Helm-equivalent templating (probably never)
- Service mesh beyond Traefik
- CRDs / Operators / Plugin model
- Leader-elected Raft coordinator
- External CA / Let's Encrypt / cert transparency
- Online-only features (HSTS, OCSP stapling, telemetry)