diff --git a/.ciagent/PLAN_v0.13.md b/.ciagent/PLAN_v0.13.md new file mode 100644 index 0000000..c98928a --- /dev/null +++ b/.ciagent/PLAN_v0.13.md @@ -0,0 +1,426 @@ +# PLAN v0.13: Production Hardening Round 2 + UAT Plan + +**Status**: complete (2026-08-07). 14 phases (P0 + P01..P12 + P13 +final). Each phase ships a patch tag on the v0.12.x line. This plan +references requirement IDs from REQUIREMENTS.md and follows the +vertical-slice integrity rule (each phase is independently shippable). + +## Phase 0: Pre-execution (this phase) + +**Status**: complete. SPECIFY → CLARIFY → RESEARCH → IDEATE → PLAN → +GRILL → SHIP. Ships as `v0.12.0`. + +## Phase 1: Toolchain & dependency vulns (REQ-149) + +**Tag**: `v0.12.1` | **Type**: fix | **Persona**: security-engineer + +### Wave 1 (single task) +- **T1**: Bump `go.mod` from `go 1.25.0` to `go 1.25.12` (or latest + 1.25.x). Run `go mod tidy`. Run `govulncheck -show verbose ./...` and + triage the 6 imported third-party vulns. Bump any dep with a + reachable trace (webauthn, cobra, modernc/sqlite, go-jose, coreos/ + go-oidc, x/crypto, oauth2). Verify `make build && make test && make + lint` all pass. + +### Must-haves +- [ ] `go.mod` declares `go 1.25.12`+ +- [ ] `govulncheck ./...` reports zero stdlib vulns with call traces +- [ ] `make build && make test && make lint` pass + +## Phase 2: Input validation & injection hardening (REQ-150) + +**Tag**: `v0.12.2` | **Type**: fix | **Persona**: backend-engineer + +### Wave 1 (11 sub-fixes, all in `internal/`) +- **T1**: `orca logs --job` — validate against `^[A-Za-z0-9_-]+$`; + replace `fmt.Sprintf("journalctl -u %q", ...)` with `shellQuote` + (critical: backtick RCE via SSH fanout) +- **T2**: pprof `isLoopback(":6060")` — treat empty host as non- + loopback/bind-all; reject unless explicit public-allow flag wired; + remove phantom `--pprof-allow-public` references; make loopback-only + a hard invariant +- **T3**: backup restore tar-slip — replace `HasPrefix(name, "..")` + with `filepath.Rel(target, dest)` containment check +- **T4**: `orca txn rollback` — validate txn ID against `^T-[0-9a-f]{16}$` +- **T5**: `orca nft diff --against` — validate txn ID before + `filepath.Join` +- **T6**: `drain stopAlloc` — validate `allocID` against + `^[A-Za-z0-9_-]+$` before `systemctl stop` +- **T7**: `cluster_compat` — `shellQuote(first)` for peer dir name +- **T8**: `runtime/podman.go` — use `shellQuote(image)` not `%q` +- **T9**: nft `TrustedProbes` — validate each entry with + `net.ParseIP`/`net.ParseCIDR`; fix ipv4/ipv6 mismatch +- **T10**: sudoers — validate `--proxmox-user`/`--proxmox-role` against + `^[a-z_][a-z0-9_-]{0,31}$`; write to fixed `/etc/sudoers.d/orca`; + `shellQuote` all pveum/useradd; `validateSudoers` check actual file +- **T11**: `nft country block add` — validate `^[A-Z]{2}$` + +### Wave 2 (tests) +- **T12**: Add injection/traversal regression tests for each sub-fix; + extend `tests/security_integration_test.go` with negative tests + +### Must-haves +- [ ] All 11 injection/traversal vectors fixed with validation +- [ ] Regression tests for each vector +- [ ] `tests/security_integration_test.go` passes + +## Phase 3: Scheduler/deployment wiring + jobspec parser (REQ-151, REQ-152) + +**Tag**: `v0.12.3` | **Type**: feat | **Persona**: lead-developer + +### Wave 1 (jobspec parser fixes — REQ-152) +- **T1**: Add `case "schedule":` and `case "timeout":` to top-level + switch in `internal/jobspec/markdown.go` +- **T2**: Fix DaemonSet — parser must not default `Count` to 1 for + DaemonSet (validator rejects `Count != 0`) +- **T3**: `restart:` policy → systemd `Restart=`/`StartLimitBurst` in + `internal/emitter/systemd.go` +- **T4**: Add `job lint` warnings for advisory-only fields (cron, + health, update, affinity) — honest "not enforced in this version" + +### Wave 2 (scheduler wiring — REQ-151) +- **T5**: Wire `internal/scheduler.Schedule()` into `orca job run` — + replace local `exec.CommandContext` path with: evaluate constraints/ + capacity/affinity → render systemd units → SSH-push to target +- **T6**: `--target` overrides scheduler selection (manual pinning) +- **T7**: Without `--target`, scheduler bin-packs across `ready` nodes +- **T8**: Local fallback when no remote nodes registered (single-node + dev mode — preserves backward compatibility) +- **T9**: `systemd-analyze verify` on rendered unit before deploy + +### Wave 3 (tests) +- **T10**: Scheduler constraint/capacity/affinity enforcement tests +- **T11**: DaemonSet spec passes lint and runs +- **T12**: `timeout:` on Jobs enforced (kill after duration) +- **T13**: Local fallback test (no remote nodes) + +### Must-haves +- [ ] `orca job run --target ` deploys via SSH-push to remote +- [ ] Scheduler evaluates constraints/capacity/affinity +- [ ] DaemonSet works (schedule parsed, Count correct) +- [ ] `timeout:` enforced on Jobs +- [ ] `restart:` translated to systemd unit +- [ ] Local fallback when no remote nodes +- [ ] `systemd-analyze verify` before deploy + +## Phase 4: ACL enforcement + WebAuthn registration auth (REQ-153) + +**Tag**: `v0.12.4` | **Type**: fix | **Persona**: backend-engineer + +### Wave 1 (ACL wiring) +- **T1**: Wire `acl.Check` into `dispatch_handler.go` — extract OIDC + sub/SPIFFE SVID from mTLS peer cert, check against ACL +- **T2**: Wire `acl.Check` into `jobs_handler.go`, `nodes_handler.go`, + `tasks_handler.go`, `health_handler.go` +- **T3**: Wire `acl.Check` into `internal/sshpush/` — validate + `ORCA_OIDC_TOKEN` bearer against JWKS +- **T4**: Wire `acl.Check` into `internal/txn/txn.go` apply path +- **T5**: Thread OIDC sub/SVID into audit `actor` field +- **T6**: Fix `acl.json` mode 0644→0600 +- **T7**: Add flock on `acl.json` for concurrent grant/revoke +- **T8**: Bootstrap ACL: grant `cluster-admin` to init cert's SVID + +### Wave 2 (WebAuthn registration auth) +- **T9**: Fix WebAuthn unauthenticated registration — require existing + session or admin bootstrap token; no overwriting existing creds + without re-auth + +### Wave 3 (tests) +- **T10**: Extend `tests/security_integration_test.go` with deny-by- + default enforcement test per handler +- **T11**: WebAuthn registration auth test (unauthenticated rejected) + +### Must-haves +- [ ] `acl.Check` called in all 5 daemon handlers + sshpush + txn +- [ ] `acl.json` mode 0600 +- [ ] Audit actor = OIDC sub/SVID +- [ ] WebAuthn registration requires auth +- [ ] Bootstrap ACL grants cluster-admin to init SVID +- [ ] Deny-by-default enforcement tests pass + +## Phase 5: Seal/audit CLI + chain race + key zeroing (REQ-154) + +**Tag**: `v0.12.5` | **Type**: feat+fix | **Persona**: security-engineer + +### Wave 1 (CLI commands) +- **T1**: Implement `orca cluster seal`/`unseal` (wraps `internal/seal/`; + OIDC token exchange; Shamir 3-of-5 shards; sealed blob 0600) +- **T2**: Implement `orca doctor audit` (wraps `AuditRepo.VerifyChain`) +- **T3**: Implement `orca doctor modes` (wraps `EnforceFileModes`) + +### Wave 2 (fixes) +- **T4**: Fix audit hash-chain race — `Append` uses `BEGIN IMMEDIATE` + transaction +- **T5**: Fix `secrets rotate-master` to actually re-seal to OIDC +- **T6**: Zero master key / namespace keys / SVID private keys after + use (defense-in-depth) + +### Wave 3 (tests) +- **T7**: Seal→unseal→secrets get round-trip test +- **T8**: `doctor audit` tamper-detection test +- **T9**: `doctor modes` 0644-rejection test +- **T10**: Audit chain concurrent-write integrity test +- **T11**: Key zeroing verification test + +### Must-haves +- [ ] `orca cluster seal`/`unseal` work (round-trip) +- [ ] `orca doctor audit` verifies chain +- [ ] `orca doctor modes` checks file modes +- [ ] Audit chain survives concurrent appends +- [ ] `secrets rotate-master` re-seals to OIDC +- [ ] Keys zeroed after use + +## Phase 6: auth init-idp real + auth register (REQ-155) + +**Tag**: `v0.12.6` | **Type**: feat | **Persona**: security-engineer + +### Wave 1 +- **T1**: Implement `orca auth init-idp` — render Dex systemd unit + + config template + Traefik dynamic route from `internal/webauthn/` + connector; RP ID = cluster Traefik domain; HTTPS via step-ca cert; + atomic deploy with rollback +- **T2**: Implement `orca auth register` (browser flow to WebAuthn + registration endpoint) +- **T3**: `loadOIDCConfig` config-file loading (`oidc.issuer` in config) +- **T4**: `orca doctor oidc` health check + +### Wave 2 (tests) +- **T5**: Hermetic Dex+Traefik config render test +- **T6**: `doctor oidc` health check test +- **T7**: Virtual-authenticator WebAuthn flow test (C-38) + +### Must-haves +- [ ] `auth init-idp` deploys Dex+Traefik+systemd +- [ ] `auth register` opens browser flow +- [ ] `oidc.issuer` loadable from config file +- [ ] `doctor oidc` health check works + +## Phase 7: Concurrency safety (REQ-156) + +**Tag**: `v0.12.7` | **Type**: fix | **Persona**: data-engineer + backend-engineer + +### Wave 1 (SQLite) +- **T1**: Add `busy_timeout(5000)` + `SetMaxOpenConns(1)` to all 4 DSNs + (store, cache, recovery, webauthn) + +### Wave 2 (flocks + locks) +- **T2**: Secrets file flock (concurrent `secrets set` on same ns) +- **T3**: Upgrade lock file (refuse concurrent `orca upgrade`) +- **T4**: Backup lock file +- **T5**: Cache invalidation by write commands (node join/leave, ns + create/delete, job run/stop) +- **T6**: `Executor.Run` mutex scope fix (hold only for DB inserts) +- **T7**: `ns create` atomic dir+ns.md write +- **T8**: `writeCurrentLead` atomic write +- **T9**: Consolidate 3 divergent `writeAtomic` impls onto + `security.WriteAtomic` +- **T10**: WebAuthn session stores guarded with `sync.Mutex` + +### Wave 3 (tests) +- **T11**: Concurrent secrets set test (no data loss) +- **T12**: Concurrent upgrade rejection test +- **T13**: Cache invalidation read-after-write test +- **T14**: SQLite concurrent writer test (no "database is locked") + +### Must-haves +- [ ] All SQLite DSNs have busy_timeout +- [ ] Concurrent secrets set preserves all writes +- [ ] Concurrent upgrade rejected +- [ ] Cache invalidated by writes (read-after-write consistency) +- [ ] WebAuthn session stores thread-safe + +## Phase 8: Transport & SSH safety (REQ-157) + +**Tag**: `v0.12.8` | **Type**: fix | **Persona**: backend-engineer + +### Wave 1 +- **T1**: Replace substring matching in `transport.IsTransient` AND + `sshpush.isTransient` with typed sentinels (`errors.Is`) +- **T2**: `rotateSSHKeys` 2-phase atomic swap +- **T3**: `known_hosts` flock field read by `dial()` +- **T4**: IPv6 `net.JoinHostPort` in proxmox SSH dial + drain + `splitHostPort` +- **T5**: Explicit timeouts for peer-setup, drift remediate/ack, txn + rollback, job restart +- **T6**: `verifyCutover` use `security.ClientTLSConfig` with orca CA +- **T7**: OIDC callback server `ReadHeaderTimeout: 5s` +- **T8**: Root SIGINT/SIGTERM handler for non-watch commands + +### Wave 2 (tests) +- **T9**: Typed-error classification test +- **T10**: rotate-lead 2-phase with partial-peer failure test +- **T11**: IPv6 SSH dial test +- **T12**: Signal handling clean-exit test + +### Must-haves +- [ ] No substring matching in transport retry logic +- [ ] rotateSSHKeys atomic 2-phase +- [ ] IPv6 addresses work in SSH dial +- [ ] All SSH commands have explicit timeouts +- [ ] SIGINT/SIGTERM triggers clean exit + +## Phase 9: Migration & operational safety (REQ-158) + +**Tag**: `v0.12.9` | **Type**: fix | **Persona**: data-engineer + +### Wave 1 +- **T1**: Migration transaction + torn-write fix +- **T2**: `job stop` real `systemctl stop` via SSH +- **T3**: DB retention/compaction for jobs/tasks/audit_log +- **T4**: `orca logs --lines` cap + `--since` upper bound +- **T5**: Cache DB mode 0600 +- **T6**: `upgrade.go` cutover backup-file + atomic-rename + +### Wave 2 (tests) +- **T7**: Migration transaction-rollback test +- **T8**: `job stop` actually-stops test +- **T9**: DB retention compaction test +- **T10**: Logs `--lines` cap test + +### Must-haves +- [ ] Migration is transactional + recoverable from torn write +- [ ] `job stop` sends `systemctl stop` via SSH +- [ ] DB retention prevents unbounded growth +- [ ] Logs output is bounded + +## Phase 10: Observability & metrics (REQ-159) + +**Tag**: `v0.12.10` | **Type**: feat | **Persona**: backend-engineer + +### Wave 1 +- **T1**: Add metrics: `orca_jobs_by_state`, `orca_drift_events_total`, + `orca_ssh_errors_total`, `orca_txn_apply_total`, + `orca_txn_rollback_total`, `orca_acl_denials_total`, + `orca_audit_chain_head` +- **T2**: New `docs/metrics.md` with Prometheus scrape config +- **T3**: Security headers middleware on daemon + +### Wave 2 (tests) +- **T4**: Metric exposition format + counter increment tests + +### Must-haves +- [ ] 7 new metrics exposed at /metrics +- [ ] `docs/metrics.md` exists +- [ ] Security headers set on daemon responses + +## Phase 11: Doc drift round 2 (REQ-160) + +**Tag**: `v0.12.11` | **Type**: docs | **Persona**: lead-developer + +### Wave 1 (README + CHANGELOG) +- **T1**: README — update status banner, latest tag, subcommand table + (add auth/nft/peer-setup/secrets rotate-master), correct "mTLS by + default" claim, add missing docs to table +- **T2**: CHANGELOG regen + +### Wave 2 (docs/*) +- **T3**: `docs/cli.md` — complete rewrite covering all ~40 subcommands +- **T4**: `docs/webauthn.md` — add `auth register` +- **T5**: `docs/namespace.md` — add inherit/set-constraint +- **T6**: `docs/install.md`+`docker.md` — update version refs +- **T7**: `docs/security-runbook.md` — match P05 reality +- **T8**: `docs/security-scanning.md` — gosec.json + +### Wave 3 (code-level doc fixes) +- **T9**: Fix `verify-reqs` bold-format regex (bypasses v0.12) +- **T10**: Fix ROADMAP/REQUIREMENTS v0.12 status hygiene +- **T11**: `internal/proxmox/bootstrap.go` comments (password→key auth) +- **T12**: Deprecate `orca status` stub +- **T13**: Help text fixes (`job run` HCL→markdown, `job stop` + daemon→SSH-push) +- **T14**: `make verify-docs` target (cli.md ↔ `orca --help`) + +### Must-haves +- [ ] README accurate (status, tag, subcommands, claims) +- [ ] `docs/cli.md` covers all subcommands +- [ ] `verify-reqs` works for v0.12 and v0.13 +- [ ] `make verify-docs` passes + +## Phase 12: --type linux + UAT plan + signoff (REQ-161, REQ-162, REQ-163) + +**Tag**: `v0.12.12` | **Type**: feat | **Persona**: lead-developer + uat-engineer + +### Wave 1 (--type linux — REQ-161) +- **T1**: Implement `internal/linux/bootstrap.go` (mirrors Proxmox + pattern without PVE role/sudoers) +- **T2**: Wire `orca node join --type linux --host --ssh-user root + --ssh-key ` + +### Wave 2 (UAT plan — REQ-162) +- **T3**: Write `docs/uat.md` — 3-host topology, step-by-step, claim + matrix (~35 claims), signoff procedure + +### Wave 3 (UAT signoff — REQ-163) +- **T4**: Write `scripts/uat-signoff.sh` — ~35 named assertions, + idempotent, read-only, exit 0 iff all pass +- **T5**: Write `scripts/uat-smoke.sh` — pure-CLI subset for CI validate + +### Wave 4 (tests) +- **T6**: `--type linux` bootstrap round-trip test (mock SSH) +- **T7**: `uat-signoff.sh` syntax + assertion-count test +- **T8**: `uat-smoke.sh` in `.coreci.yml` validate + +### Must-haves +- [ ] `orca node join --type linux` works (SSH bootstrap) +- [ ] `docs/uat.md` covers 3-host topology + all claims +- [ ] `scripts/uat-signoff.sh` has ~35 assertions, idempotent +- [ ] `scripts/uat-smoke.sh` runs in CI + +## Phase 13: Final review + ship + audit + +**Tag**: `v0.12.13` = v0.13 milestone release | **Type**: chore + +### Wave 1 +- **T1**: `ciagent-review` — multi-persona code review across P01..P12 +- **T2**: `ciagent-audit` — reconstruction test, branch hygiene, commit + discipline; fix any remaining verify-reqs discrepancies +- **T3**: Update REQUIREMENTS.md — mark all v0.13 REQs as complete +- **T4**: Update ROADMAP.md — mark v0.13 as **COMPLETE** +- **T5**: Merge `phase/13` → `milestone/v0.13` → `main` +- **T6**: Tag `v0.12.13` (milestone release) +- **T7**: Create release with full milestone summary + +### Must-haves +- [ ] All v0.13 REQs marked complete in REQUIREMENTS.md +- [ ] ROADMAP.md marks v0.13 COMPLETE (with bold) +- [ ] `verify-reqs` passes for v0.12 and v0.13 +- [ ] Milestone merged to main +- [ ] `v0.12.13` tag created +- [ ] v1.0.0 NOT cut (deferred for UAT signoff) + +## Wave ordering summary + +| Phase | Waves | Tasks | Depends on | +|-------|-------|-------|------------| +| P01 | 1 | 1 | P0 | +| P02 | 2 | 12 | P0 | +| P03 | 3 | 13 | P0 | +| P04 | 3 | 11 | P0 (P03 for scheduler context) | +| P05 | 3 | 11 | P0 | +| P06 | 2 | 7 | P05 (seal) | +| P07 | 3 | 14 | P0 | +| P08 | 2 | 12 | P0 | +| P09 | 2 | 10 | P0 | +| P10 | 2 | 4 | P04 (acl denials metric), P05 (audit chain head) | +| P11 | 3 | 14 | P01..P10 (docs reflect reality) | +| P12 | 4 | 8 | P03 (scheduler for UAT), P04 (ACL for UAT) | +| P13 | 1 | 7 | P01..P12 | + +## Vertical slice integrity + +Each phase is independently shippable: +- P01 (toolchain) — bumps go version, no API change +- P02 (injection) — validates inputs, no API change +- P03 (scheduler) — changes `job run` behavior (local→remote), local + fallback preserves backward compat +- P04 (ACL) — adds enforcement, bootstrap ACL prevents lockout +- P05 (seal) — adds new CLI commands, no breaking change +- P06 (init-idp) — replaces stub, no breaking change +- P07 (concurrency) — adds locks/timeouts, no API change +- P08 (transport) — replaces substring with typed errors, no API change +- P09 (migration) — fixes migration safety + job stop, job stop is + behavioral change (soft→hard stop) — documented +- P10 (metrics) — adds metrics, no API change +- P11 (docs) — docs only, no code behavior change +- P12 (UAT) — adds new command + docs + scripts, no breaking change +- P13 (final) — review + ship, no new features