Compare commits

..

10 Commits

Author SHA1 Message Date
Jon Chery 97a10353da docs(P00): grill v0.8 plan — PROCEED-WITH-CONDITION (4 binding fixes applied)
GRILL_v0.8.md (30KB): 9-axis adversarial review, overall verdict
PROCEED-WITH-CONDITION (confidence 0.78). 7 PROCEED + 4
PROCEED-WITH-CONDITION + 0 REPLAN findings.

4 binding plan changes applied to PLAN_v0.8.md:
  #1 T02.6 relabeled as v0.6 ship-defect bugfix (not v0.8 feature);
     P04 audit must record ship-defect closure
  #2 T02.9 doctor proxmox parity — P02 not complete until both
     bootstrap + doctor callbacks use capture-fix wrapper
  #3 T01.6 cli coverage escape valve — ship at 65% if 70% not reached
     after Wave 2 (RESEARCH §1.4 flags 55-65% realistic); do not block
     P02/P03 on the last 5%
  #4 T03.1 verify-reqs regex substring-tolerant (matches v0.2 header
     variant) + reverse-direction assertion (REQUIREMENTS Complete ↔
     ROADMAP COMPLETE); scope note: doc-vs-doc drift only

T02.10 case 7 added (known_hosts pre-populated v0.6→v0.8 migration
path). No escalations; all axes resolved at confidence ≥ 0.60.

---ci---
project: orca
phase: 0
milestone: v0.8
status: grill
---/ci---
2026-08-04 00:49:07 +00:00
Jon Chery a288eb93ea docs(P00): create v0.8 phase plans — 4 exec phases + final review
PLAN_v0.8.md (33KB): 4 execution phases, 37 tasks (36 must-haves),
3-wave ordering per phase, persona-assigned (lead/backend/data),
REQ-057..060 mapped.

P01 coverage round 2 (12 tasks): proxmox sessionRunner seam + 9 pkg
tests, tiered floor ≥70%/≥50% per D-047.
P02 SSH trust (11 tasks): --host-key-fingerprint pre-pin + key-reset +
TOFU capture bugfix + HostKeyFingerprint population.
P03 verify-reqs gate (5 tasks): cmd/verify-reqs Go program + make
target + .coreci.yml hook.
P04 final review + ship + audit (9 tasks).

Zero new direct deps. ROADMAP reconciled to 4-phase structure (P04 =
final review, no separate P05).

---ci---
project: orca
phase: 0
milestone: v0.8
status: plan
---/ci---
2026-08-04 00:49:07 +00:00
Jon Chery 285ffee863 docs(P00): v0.8 research findings + persona assessment
RESEARCH_v0.8.md (35KB): per-package coverage strategy for 9 pkgs,
SSH trust research (uncovered latent TOFU capture bug + unpopulated
HostKeyFingerprint field), verify-reqs Go program approach, 4 ADs,
10 pitfalls. PERSONAS.md updated for v0.8 (3-persona roster retained,
connectrpc removed from backend frameworks per AD-014, territory globs
aligned to actual file structure).

Key findings flagged for PLAN:
- proxmox needs sessionRunner seam (~10 LOC) or stalls at ~55%
- cert_repo_test.go missing (v0.7 P01 leftover) blocks store 70%
- TOFU known_hosts capture broken + HostKeyFingerprint never populated
  (bootstrap.go:195-198) — P02 fixes both
- verify-reqs = Go program at cmd/verify-reqs (~80 LOC, stdlib only)

---ci---
project: orca
phase: 0
milestone: v0.8
status: research
---/ci---
2026-08-04 00:49:07 +00:00
Jon Chery a0b3b7439d docs(P00): clarify v0.8 ambiguities (5 decisions, full autonomy)
D-043 P02 chore vs feat (chore — trust-surface hardening, no new capability)
D-044 --host-key-fingerprint placement (node join root, validated on --type proxmox)
D-045 fingerprint format (OpenSSH SHA256:base64)
D-046 key-reset scope (local known_hosts only, not remote authorized_keys)
D-047 coverage tiered floor (70% for retested, 50% for zero-test packages)

---ci---
project: orca
phase: 0
milestone: v0.8
status: clarify
---/ci---
2026-08-04 00:49:05 +00:00
Jon Chery a052bf20f1 docs(init): validate v0.8 specification — coverage & trust hardening
---ci---
project: orca
phase: 0
milestone: v0.8
status: specify
---/ci---
2026-08-04 00:49:05 +00:00
Jon Chery 7bb533c2fb docs(milestone): complete hardening-completion
v0.7 milestone complete. All 4 execution phases + final review shipped.
REQ-053..056 all complete. Tags v0.6.0..v0.6.5 on v0.6.x patch line.
Merged milestone/v0.7-hardening-completion → main.

---ci---
project: orca
phase: 5
milestone: v0.7
status: complete
requirements:
  covered: [REQ-053, REQ-054, REQ-055, REQ-056]
  partial: []
---/ci---
2026-08-04 00:29:51 +00:00
Jon Chery d7d6961261 fix(P05): final review fixes — PERSONAS.md body, config --addr precedence, migration 0007 dedup
Audit fix: PERSONAS.md body roster updated for v0.7 (was stale v0.6
content). Review P1-004: daemon --addr now uses cmd.Flags().Changed()
to detect explicit flag, so config listen_addr only applies when --addr
was not explicitly passed (correct flag>env>file>default precedence).
Review P1-001: migration 0007 now dedups existing duplicate serial_hex
rows before creating the UNIQUE index (backward-compat with v0.6 DBs
that accumulated duplicates before the constraint existed).

---ci---
project: orca
phase: 5
milestone: v0.7
status: execute
requirements:
  covered: [REQ-053, REQ-054, REQ-055, REQ-056]
  partial: []
---/ci---
2026-08-04 00:28:48 +00:00
Jon Chery afcd15cde4 docs(P04): complete pprof-daemon phase — shipped v0.6.4
REQ-056 complete. I-308 (deferred since v0.2) implemented. Tag + merge +
Gitea release succeeded.

---ci---
project: orca
phase: 4
milestone: v0.7
status: complete
requirements:
  covered: [REQ-056]
  partial: []
---/ci---
2026-08-04 00:22:39 +00:00
Jon Chery 0b58286ca2 feat(P04): --pprof opt-in on orca daemon (REQ-056, I-308)
Separate *http.Server + *http.ServeMux (AD-024), default disabled.
Operator opts in via --pprof <addr>. WARN logged on startup. All pprof
handlers explicitly registered on dedicated mux (no DefaultServeMux
side-effect). I-308 deferred since v0.2 now implemented. 6 new tests.

---ci---
project: orca
phase: 4
milestone: v0.7
status: verify
requirements:
  covered: [REQ-056]
  partial: []
---/ci---
2026-08-04 00:22:17 +00:00
Jon Chery f8b135e7a8 docs(P03): complete coverage-uplift phase — shipped v0.6.3
REQ-055 complete. All 4 target packages ≥ 50% (engine 65.1%, transport
84.6%, proxmox 82.7%, audit 100%). Latent dispatch.go EOF bug fixed.

---ci---
project: orca
phase: 3
milestone: v0.7
status: complete
requirements:
  covered: [REQ-055]
  partial: []
---/ci---
2026-08-04 00:19:20 +00:00
16 changed files with 1910 additions and 41 deletions
+6 -6
View File
@@ -1,11 +1,11 @@
{
"phase": 2,
"stage": "complete",
"milestone": "v0.7",
"milestone_slug": "hardening-completion",
"phase_role": "execution",
"phase": 0,
"stage": "grill",
"milestone": "v0.8",
"milestone_slug": "coverage-trust-hardening",
"phase_role": "pre_execution",
"attempts": 0,
"updated_at": "2026-08-04T00:10:00Z",
"updated_at": "2026-08-04T00:48:00Z",
"milestone_complete": false,
"next_milestone": null
}
+592
View File
@@ -0,0 +1,592 @@
# Grill Report: Orca v0.8 — Coverage & Trust Hardening
**Date:** 2026-08-04
**Reviewer:** ci-griller (red-team, adversarial)
**Plan under review:** `.ciagent/PLAN_v0.8.md` (commit 4780e4d)
**Branch:** `phase/00-specify` (milestone `milestone/v0.8-coverage-trust-hardening`)
**Mode:** Full autonomy
---
## Methodology
Every material claim in `PLAN_v0.8.md` and `RESEARCH_v0.8.md` was cross-checked
against the actual codebase (verified coverage baselines via `go test -cover`,
read `internal/proxmox/bootstrap.go:75-234`, `internal/security/ca.go`,
`internal/doctor/doctor.go`, `.ciagent/ROADMAP.md`, `.ciagent/REQUIREMENTS.md`,
PERSONAS, ARCHITECTURE) AND the `golang.org/x/crypto` v0.54.0 source for
`knownhosts.New` / `checkAddr` behavior. The TOFU-capture claim was not taken
on faith — the upstream `checkAddr` (knownhosts.go:370-385) was read directly.
Findings are scored on the 9 axes. Binding verdicts are **PROCEED**,
**PROCEED-WITH-CONDITION** (plan proceeds but must incorporate a named change),
or **REPLAN** (axis has a fatal flaw; revise before execution).
---
## Summary Verdict
| Verdict | Count |
|---------|-------|
| PROCEED | 7 |
| PROCEED-WITH-CONDITION | 4 |
| REPLAN | 0 |
**Overall verdict: PROCEED-WITH-CONDITION**
The v0.8 plan is fundamentally sound: scope is right-sized, the no-new-deps
promise holds (verified `ssh.FingerprintSHA256` + `knownhosts.Line` are in the
existing `golang.org/x/crypto` v0.54.0 dep), the tiered coverage floor (D-047)
is realistic per-package with the named seams, and the persona territory
collision on `internal/cli/node.go` is explicitly adjudicated in PERSONAS.md
(backend owns implementation, lead owns `_test.go`). The 4 conditions below are
**targeted correctness fixes**, not scope expansions:
1. **P02 must add a regression test asserting first-connect Proxmox join
succeeds end-to-end** (the latent TOFU bug means v0.6's first-connect has
been broken since ship; the fix in T02.6 is correct but must be proven by a
test that would have failed pre-fix).
2. **P03's verify-reqs regex must match `**COMPLETE**` as a *substring* within
the bold span** (v0.2's header `**COMPLETE (merged to main via v0.3)**` is
not matched by the current `\*\*COMPLETE\*\*` literal — a silent blind spot).
3. **P03 must add a second assertion: every REQUIREMENTS row marked `Complete`
must reference a milestone ROADMAP marks COMPLETE** (the reverse direction).
The v0.7 `cert_repo_test.go` omission (REQ-053 marked Complete but the test
file does not exist) proves forward-direction-only checks miss the most
dangerous drift class: *claimed-Complete-but-actually-incomplete*.
4. **P02 T02.6's TOFU fix must be reviewed against `doctor proxmox`'s callback
(T02.9) as a paired change, not a follow-on** — they share the exact
`knownhosts.New` defect; fixing one and not the other in the same phase
creates an inconsistent trust surface.
With these 4 conditions applied, this plan is ready to execute. No REPLAN.
---
## Per-Axis Findings
### Axis 1 — Business Case
#### A1-F1 — Is v0.8 the right next milestone, or polish-for-polish's-sake?
**Evidence:**
- v0.7 P03 (REQ-055) shipped a ≥50% coverage floor; v0.8 re-baselines six
packages still under 50% (engine 8.3%, proxmox 5.1%, cli 27.6%, transport
26.3%, store 47.2%, jobspec 47.6%) — **verified identical via `go test
-cover`**.
- RESEARCH §2.1 surfaces a **latent v0.6 defect**: `knownhosts.New` returns
`KeyError{Want:[]}` on first connect and does NOT auto-write. Verified
directly in `golang.org/x/crypto@v0.54.0/ssh/knownhosts/knownhosts.go:370-385`
(`checkAddr` returns `&KeyError{}` with empty `Want` when no line matches).
`bootstrap.go:140-142` treats this as a dial failure. **This means
first-connect `orca node join --type proxmox` has been broken since v0.6
shipped** (the v0.6 RESEARCH §A.5 claim that `knownhosts.New` "handles both
capture and verify" was wrong).
- `bootstrap.go:123` comment is literally false: "on first connect it captures
the host key" — it does not.
**Confidence:** 0.90 that v0.8 is the right next milestone.
**Verdict:** **PROCEED**. v0.8 is not polish-for-polish: it closes a real
security defect (TOFU broken since v0.6), populates a `Result` field that D-045
*assumed* was already populated (it isn't — `bootstrap.go:195-198`), and lifts
coverage off floors that v0.7 explicitly under-shot. The diminishing-returns
risk is real for the 3 zero-test toe-holds (audit/certpaths/cmd-orca), but
D-047 tiered them to 50% precisely to avoid the rathole — that call is sound.
---
### Axis 2 — Scope and Requirements
#### A2-F1 — Is the TOFU bugfix correctly scoped into P02, or should it be a hotfix on main?
**Evidence:**
- The TOFU capture bug (RESEARCH §2.1, PLAN T02.6) is a v0.6 latent defect,
not a v0.8 feature. First-connect Proxmox join is broken **today on main**.
- PLAN bundles the fix into P02 (trust hardening phase) alongside REQ-058
(`--host-key-fingerprint`) and REQ-059 (`key-reset`).
- ROADMAP tags run on the v0.7.x patch line: `v0.7.0` (P0) … `v0.7.4` (P04).
P02 ships as `v0.7.2` — i.e., the fix lands on a milestone branch, not main,
and only reaches main at P04 merge (`v0.7.4`).
**Confidence:** 0.62 that bundling into P02 is the right call (low confidence —
this is a judgment call with real downside).
**Verdict:** **PROCEED-WITH-CONDITION.** The fix is correctly designed (T02.6's
`KeyError{Want:[]}` capture-and-persist is the right shape), but the plan must
either (a) document explicitly *why* this isn't hotfixed on main (e.g., "no
operator has hit first-connect yet because all deployments pre-populate
`known_hosts` manually — confirmed by the v0.6 ship audit"), OR (b) flag the
bug in the P04 audit as a v0.6 ship-defect with a post-mortem note. **The plan
currently treats T02.6 as a feature task; it is a bugfix for shipped code and
must be labeled as such** so the P04 audit can distinguish "new hardening" from
"closing a v0.6 gap." Blast radius if T02.6's fix is wrong: every existing
Proxmox node's `known_hosts` could be re-pinned on next join — moderate, but
mitigated by T02.10 case 3/4/5 integration tests.
**Condition:** Add a note to T02.6 in PLAN marking it as a **v0.6 ship-defect
bugfix** (not a v0.8 feature), and ensure P04 audit (T04.2) records it as such.
#### A2-F2 — Are the 3 zero-test packages worth a 50% toe-hold, or scope creep?
**Evidence:**
- `cmd/orca` is 15 LOC of glue (`main()``cli.Execute()`). 50% coverage = ~7
lines. RESEARCH §1.1, §5 pitfall #6 explicitly flags the effort:coverage
ratio as poor.
- `internal/certpaths` is 64 LOC of pure path-join functions. 50% is trivial.
- `internal/audit` is 125 LOC, 4 exported funcs. 50% is trivial.
- D-047 explicitly tiered these to 50% to avoid a coverage rathole; v0.9 can
raise the floor.
**Confidence:** 0.85.
**Verdict:** **PROCEED.** The tiered floor is the right call. The
`cmd/orca` toe-hold is low-value but low-cost (one `run() int` refactor + one
smoke test), and dropping it would leave a `covdata` tooling error in CI output
that looks like a broken build to a casual reader. Keeping it at 50% is
defensible.
#### A2-F3 — Scope size: 4 REQs, 37 tasks — too lean, too fat, or right?
**Evidence:**
- 37 tasks, 36 must-haves, 4 phases each shipping a patch. Comparable to v0.7
(5 phases, similar task density).
- P01 is the heaviest (12 tasks, 9 packages) — the risk concentration is here.
**Confidence:** 0.80.
**Verdict:** **PROCEED.** Right-sized for an NFR milestone. P01 density is the
watch item (see Axis 5).
---
### Axis 3 — Architecture and Technical Feasibility
#### A3-F1 — Do the proxmox `sessionRunner` and engine `peerDispatcher` seams leak test concerns into production?
**Evidence:**
- T01.1 `sessionRunner` (`internal/proxmox/bootstrap.go`): 1 interface,
~10 LOC, `CombinedOutput(cmd) ([]byte, error)`. Default impl wraps
`*ssh.Client.NewSession().CombinedOutput(...)`. Backward compatible —
existing callers unchanged. This is the **same pattern as the existing
`sshDialer` seam** (`bootstrap.go:201-213`), which shipped in v0.6 without
concern. The seam is a standard testability extraction, not a test concern
leak.
- T01.2 `peerDispatcher` (`internal/engine/dispatcher.go`): **conditional**
only added if T01.4 cannot hit 70% via `httptest.NewTLSServer` alone. Plan
explicitly prefers `httptest.NewTLSServer` (RESEARCH §1.3 gap #2, §5 pitfall
#8). This is the right ordering: try the stdlib test fixture first, add the
seam only if needed.
**Confidence:** 0.88.
**Verdict:** **PROCEED.** Both seams are backward-compatible interface
extractions matching an existing pattern (`sshDialer`). No test-concern leak.
The conditional-gate on T01.2 is correctly conservative.
#### A3-F2 — Does P02's trust work stay within the existing security boundary?
**Evidence:**
- P02 touches `internal/proxmox/bootstrap.go` (pinned callback, TOFU fix),
`internal/cli/node.go` (flag + subcommand), `internal/security/sshkey.go`
(fingerprint helper), `internal/doctor/doctor.go` (T02.9 TOFU fix). All
within the existing SSH trust surface established in v0.6.
- No new crypto, no new CA, no new X.509. `ssh.FingerprintSHA256` is in the
existing `golang.org/x/crypto` v0.54.0 dep (verified: not a new direct dep).
- PERSONAS correctly keeps `security-engineer` deactivated — the work is SSH
dialer + known_hosts file manipulation, not new security architecture.
**Confidence:** 0.90.
**Verdict:** **PROCEED.** Boundary is respected.
#### A3-F3 — T02.9 (doctor proxmox TOFU fix) is a paired change with T02.6, not a follow-on
**Evidence:**
- `internal/doctor/doctor.go:412` uses the **exact same** `knownhosts.New(...)`
callback pattern as `bootstrap.go:125`. Both share the latent defect.
- T02.9 is listed as a separate task ("Apply the TOFU capture-fix to `doctor
proxmox` probe") but is in the same Wave 2 as T02.6. If T02.6 lands and T02.9
doesn't (e.g., a mid-phase blocker), the trust surface is **inconsistent**:
join captures, doctor fails.
**Confidence:** 0.75.
**Verdict:** **PROCEED-WITH-CONDITION.** T02.6 and T02.9 must be reviewed as a
paired change in P02 verification — the phase is not done until BOTH callbacks
use the capture-fix wrapper. Add to P02 Verification: "doctor proxmox
first-connect → captures + succeeds (mirrors T02.10 case 3 for bootstrap)."
**Condition:** Add a P02 verification line asserting doctor proxmox
first-connect parity with bootstrap.
---
### Axis 4 — People, Skills, and Organization
#### A4-F1 — Territory collision on `internal/cli/node.go`
**Evidence:**
- PERSONAS.md line 62: lead-developer territory = `internal/cli/**`.
- PERSONAS.md line 70: backend-engineer territory = `internal/cli/node.go`.
- PERSONAS.md line 107 explicitly adjudicates: "backend owns the command
implementation; lead owns the test files (`node_test.go`)."
- Territory mode is `warn` (not `block`) — collisions log but don't fail.
**Confidence:** 0.82.
**Verdict:** **PROCEED.** The collision is **explicitly adjudicated** in
PERSONAS.md with a clean boundary (impl vs test files). This is the right
answer. The `warn` mode means a backend commit touching `node_test.go` (or a
lead commit touching `node.go` impl) would log — acceptable for a 3-persona
team. No replan.
#### A4-F2 — Key-person dependency: is the 3-persona roster sufficient?
**Evidence:**
- 3 active personas, all retained from v0.7. No phase-specific personas.
- backend-engineer owns 60%+ of P02 (the security-critical phase). If
backend-engineer is unavailable, P02 stalls entirely.
**Confidence:** 0.70.
**Verdict:** **PROCEED.** Key-person risk is real but inherent to a 3-persona
NFR milestone. The work is not novel (refining existing surface), so the bus
factor is acceptable for hardening. Flagged, not blocking.
---
### Axis 5 — Timeline and Estimates
#### A5-F1 — Is the 70% coverage target for 6 packages in one phase (P01) realistic?
**Evidence:**
- RESEARCH §1.1 + §1.4 per-package achievability assessments:
- engine → 70% REALISTIC (with LocalExecutor stubs + `openTestDB`).
- proxmox → 70% REALISTIC **but requires the `sessionRunner` seam (T01.1)** —
without it, only 50-55% (validation paths + sudoersContent asserts, already
done).
- cli → 70% AMBITIOUS (17 files, ~2000 LOC); RESEARCH says "55-65% is more
realistic for one phase" even with `daemon.go` excluded.
- transport → 70% REALISTIC (`httptest.NewTLSServer` is standard).
- store → 70% REALISTIC (cert_repo_test.go gap is the main lift).
- jobspec → 70% REALISTIC (easiest of the six).
- **`internal/cli` is the swing package.** RESEARCH explicitly says 55-65% is
the realistic single-phase outcome, not 70%. The plan sets the floor at 70%
"excluding daemon.go" — but even excluding daemon.go, RESEARCH's own evidence
says 70% is a stretch.
**Confidence:** 0.65 (split: 5 of 6 packages at 0.85, cli at 0.45).
**Verdict:** **PROCEED-WITH-CONDITION.** The plan must add an explicit fallback
for `internal/cli`: if T01.6 hits ≥65% (excluding daemon.go) but not 70% after
a reasonable effort, the phase ships at 65% with a documented note + a v0.9
follow-up to lift to 70%. **Hard-requiring 70% on cli risks a coverage rathole
that delays the entire milestone** (P02/P03 are gated on P01 ship). The other 5
packages at 70% is realistic.
**Condition:** Add to T01.6 acceptance criterion: "If ≥65% (excluding
daemon.go) is achieved but 70% is not after Wave 2 effort, document the gap in
the task comment + record a v0.9 follow-up; ship at 65%. Do NOT block P02/P03
on the last 5% of cli coverage." (This mirrors RESEARCH §1.4's own flag, which
the plan currently does not carry forward as an escape valve.)
---
### Axis 6 — Budget and Financial Realism
#### A6-F1 — Zero new deps: is that realistic given P02's needs?
**Evidence:**
- `ssh.FingerprintSHA256`: verified in `golang.org/x/crypto/ssh` (direct dep
since v0.6 D-030).
- `knownhosts.Line` / `Normalize` / `KeyError`: same `golang.org/x/crypto`
module (already imported in `bootstrap.go:32` and `doctor.go:29`).
- `verify-reqs`: stdlib only (`regexp`, `os`, `fmt`).
- `go.mod` unchanged by v0.8 (PLAN line 62).
**Confidence:** 0.95.
**Verdict:** **PROCEED.** Zero-new-deps is verified and realistic.
---
### Axis 7 — Risks, Assumptions, and Dependencies
#### A7-F1 — The 10 pitfalls: are mitigations real or hand-waves?
**Evidence (spot-check of the 4 most material pitfalls):**
- **Pitfall #1 (TOFU broken):** Mitigation T02.6 is **concrete and correct** —
wrap `knownhosts.New`, capture on `KeyError{Want:[]}` via `knownhosts.Line` +
`security.WriteAtomic`, return nil. Verified against x/crypto v0.54.0
`checkAddr` semantics. **Real mitigation.**
- **Pitfall #2 (Result.HostKeyFingerprint never populated):** T02.7 adds
`ssh.FingerprintSHA256(hostKey)`. 1-line once host key is available. **Real.**
- **Pitfall #3 (no sessionRunner seam):** T01.1 adds it, ~10 LOC. **Real.**
- **Pitfall #10 (writeAtomic unexported):** T02.2 exports it. Verified
`ca.go:305` — `func writeAtomic(...)` is indeed unexported. **Real.**
**Confidence:** 0.88.
**Verdict:** **PROCEED.** Mitigations are concrete, not hand-waves.
#### A7-F2 — TOFI bugfix blast radius if P02's fix is wrong
**Evidence:**
- T02.6 changes the `HostKeyCallback` for every `orca node join --type proxmox`
+ every `doctor proxmox` probe. If the capture-and-persist logic is wrong,
every existing Proxmox node's `known_hosts` could be corrupted (e.g.,
duplicate entries, wrong-format lines, partial writes on crash).
- Mitigations: T02.10 integration tests (cases 3/4/5 cover first-connect,
second-connect, mismatch); AD-029 atomic rewrite via `security.WriteAtomic`.
- **Gap:** no test for "known_hosts already has an entry, join re-connects" —
i.e., the idempotent re-run path after the fix. T02.10 case 4 covers
second-connect-match, but not "known_hosts was written by the OLD (broken)
code path and is now being read by the NEW code path."
**Confidence:** 0.70.
**Verdict:** **PROCEED-WITH-CONDITION.** T02.10 must add a case for
"known_hosts pre-populated in the expected format (e.g., from a manual
`ssh-keyscan` or a prior v0.6 deployment that somehow succeeded) →
second-connect matches + succeeds." This covers the migration path from
v0.6's (broken) state to v0.8's fixed state.
**Condition:** Add T02.10 case 7: "known_hosts pre-populated with a valid
OpenSSH line for the host → connect matches + succeeds (covers v0.6→v0.8
migration)."
---
### Axis 8 — Governance, Decision-Making, and Communication
#### A8-F1 — Does `make verify-reqs` actually prevent drift, or is it cosmetic?
**Evidence:**
- T03.1 regex (PLAN line 216):
- ROADMAP milestone-complete: `^##\s*Milestone\s+v0\.\d+:.*—\s*\*\*COMPLETE\*\*`
- REQUIREMENTS row: `^\|\s*(REQ-\d+)\s*\|.*?\|\s*\*\*(Complete|Pending)\*\*\s*\|`
- **ROADMAP v0.2 header (line 23):** `## Milestone v0.2: Networking,
Observability, Security Hardening — **COMPLETE (merged to main via v0.3)**`
- The regex `\*\*COMPLETE\*\*` requires the literal `**COMPLETE**` with closing
`**` immediately after `COMPLETE`. v0.2's header has `**COMPLETE (merged to
main via v0.3)**` — the `**` closes after the parenthetical, NOT after
`COMPLETE`. **The regex does NOT match v0.2 as complete.**
- **Consequence:** all v0.2 REQs (REQ-011, 014, 023, 025-040) are **silently
exempted** from the check. A stale v0.2 REQ-035 row (marked Pending) would
NOT fail the gate.
- **ROADMAP v0.6 has TWO headers** (line 92 without COMPLETE, line 94 with) —
the regex matches line 94, but the duplicate is a markdown smell that could
confuse the milestone→REQ mapping if the parser takes the first match.
**Confidence:** 0.92 (high — the regex mismatch is verifiable).
**Verdict:** **PROCEED-WITH-CONDITION.** The regex must match `**COMPLETE**`
as a *substring within the bold span*, not as a literal `**COMPLETE**` token.
Change to `—\s*\*\*[^*]*COMPLETE[^*]*\*\*` (matches `**COMPLETE**`,
`**COMPLETE (merged to main via v0.3)**`, and any future variant). Add a
golden-file test case (T03.2) with the v0.2-style parenthetical header to
prevent regression.
**Condition:** T03.1 regex changed to substring-match COMPLETE within the bold
span; T03.2 adds a golden fixture with `**COMPLETE (merged to main via v0.3)**`.
#### A8-F2 — Is the single-direction check (ROADMAP→REQUIREMENTS) enough?
**Evidence:**
- PLAN line 35-37 explicitly scopes out the reverse direction: "forward
direction (ROADMAP-shipped → REQUIREMENTS Complete) is the priority per the
v0.7 drift that motivated REQ-060."
- **But the v0.7 drift had TWO symptoms:**
1. ROADMAP said COMPLETE, REQUIREMENTS said Pending (forward drift — caught
by the current check).
2. **REQ-053 was marked Complete in REQUIREMENTS, but
`internal/store/cert_repo_test.go` was never written** — verified: only
`cert_repo.go` exists in `internal/store/`. The "Complete" status was
false. **No markdown-based check can catch this** (it's a code-vs-doc
drift, not a doc-vs-doc drift).
- The reverse-direction check (REQUIREMENTS Complete ↔ ROADMAP COMPLETE) would
catch a different class: a REQ marked Complete in REQUIREMENTS for a
milestone ROADMAP does NOT mark COMPLETE (e.g., premature marking). This is
a cheaper class of drift but still real.
**Confidence:** 0.78.
**Verdict:** **PROCEED-WITH-CONDITION.** Add the reverse-direction assertion
to T03.1 (it's ~10 LOC on top of the existing parser — same maps, just diff
both ways). Document explicitly that **no markdown check can catch the
code-vs-doc drift** (REQ-053 case) — that requires a code-level audit
(`ciagent-audit` in P04). The plan should note this as a known limitation of
REQ-060, not pretend the gate is complete.
**Condition:** T03.1 adds reverse-direction assertion; PLAN adds a note that
REQ-060 catches doc-vs-doc drift only, not code-vs-doc (the REQ-053
cert_repo_test.go case).
#### A8-F3 — Is there a "stop the project" trigger?
**Evidence:** P04 (T04.1-T04.9) is the final review + ship. No explicit
"stop" trigger if P01 coverage stalls or P02 TOFU fix proves unfixable.
**Confidence:** 0.60.
**Verdict:** **PROCEED.** The 4-phase structure with per-phase tags means a
stall is visible (phase tag doesn't ship). Acceptable for an NFR milestone.
---
### Axis 9 — Change, Adoption, and Operational Readiness
#### A9-F1 — Who benefits from v0.8? Is there operator pull for `--host-key-fingerprint`?
**Evidence:**
- `--host-key-fingerprint` (REQ-058) is operator-facing: pre-pinning a
Proxmox host's SSH key before first join. This is the standard
high-security-deployment pattern (the v0.6 D-035 caveat explicitly promised
it as a "future enhancement").
- `orca node key-reset` (REQ-059) is operator-facing: the `ssh-keygen -R`
equivalent for orca's known_hosts.
- The TOFU bugfix (T02.6) benefits **every operator who has tried
first-connect Proxmox join since v0.6** — i.e., it fixes a feature that was
advertised as working but wasn't.
- Coverage uplift (REQ-057) is developer-facing (no operator pull).
- verify-reqs (REQ-060) is internal-governance (no operator pull).
**Confidence:** 0.82.
**Verdict:** **PROCEED.** The trust features have real operator pull
(pre-pinning is a documented security best practice; the v0.6 caveat promised
it). The coverage + hygiene work is internal-debt paydown — justified by the
v0.7 under-shot, not by operator demand. The mix is appropriate for an NFR
milestone.
#### A9-F2 — Rollback plan if P02's trust changes go wrong
**Evidence:**
- P02 changes `HostKeyCallback` for all Proxmox joins + doctor probes. If the
capture-fix corrupts `known_hosts`, the rollback is: revert the phase commit
+ manually restore `known_hosts` from backup.
- No data migration in P02 (known_hosts is a flat file; atomic rewrite via
`WriteAtomic` preserves crash safety).
- `key-reset` (T02.8) is local-only (D-046) — no remote side effects to
reverse.
**Confidence:** 0.80.
**Verdict:** **PROCEED.** Rollback is straightforward (revert + file restore).
The atomic-rewrite requirement (AD-029) is the right mitigation.
---
## Binding Verdicts Table
| # | Axis | Finding | Verdict | Condition | Confidence |
|---|------|---------|---------|-----------|------------|
| A2-F1 | Scope | TOFU bugfix is a v0.6 ship-defect bundled into P02 as a feature task | PROCEED-WITH-CONDITION | Label T02.6 as a v0.6 bugfix in PLAN; P04 audit records it as a ship-defect closure | 0.62 |
| A2-F2 | Scope | 3 zero-test packages at 50% toe-hold | PROCEED | — | 0.85 |
| A2-F3 | Scope | 37 tasks / 4 phases size | PROCEED | — | 0.80 |
| A1-F1 | Business | v0.8 is the right next milestone (not polish) | PROCEED | — | 0.90 |
| A3-F1 | Architecture | sessionRunner + peerDispatcher seams do not leak test concerns | PROCEED | — | 0.88 |
| A3-F2 | Architecture | P02 stays within existing security boundary | PROCEED | — | 0.90 |
| A3-F3 | Architecture | T02.6 + T02.9 are paired changes (bootstrap + doctor share the defect) | PROCEED-WITH-CONDITION | Add P02 verification line for doctor proxmox first-connect parity with bootstrap | 0.75 |
| A4-F1 | People | internal/cli/node.go territory collision adjudicated | PROCEED | — | 0.82 |
| A4-F2 | People | Key-person risk on backend-engineer in P02 | PROCEED | — | 0.70 |
| A5-F1 | Timeline | 70% cli coverage in one phase is a stretch (RESEARCH says 55-65%) | PROCEED-WITH-CONDITION | Add escape valve: ship cli at 65% if 70% not reached after Wave 2; do not block P02/P03 | 0.65 |
| A6-F1 | Budget | Zero new deps verified | PROCEED | — | 0.95 |
| A7-F1 | Risks | 10 pitfalls mitigations are concrete | PROCEED | — | 0.88 |
| A7-F2 | Risks | TOFU fix blast radius — no migration-path test | PROCEED-WITH-CONDITION | Add T02.10 case 7: known_hosts pre-populated → second-connect matches (v0.6→v0.8 migration) | 0.70 |
| A8-F1 | Governance | verify-reqs regex does not match v0.2's `**COMPLETE (merged...)**` header | PROCEED-WITH-CONDITION | Change regex to substring-match COMPLETE within bold span; add golden fixture | 0.92 |
| A8-F2 | Governance | Single-direction check misses reverse drift + code-vs-doc drift (REQ-053 case) | PROCEED-WITH-CONDITION | Add reverse-direction assertion; document that code-vs-doc drift is out of scope for REQ-060 | 0.78 |
| A8-F3 | Governance | No explicit "stop" trigger | PROCEED | — | 0.60 |
| A9-F1 | Adoption | Operator pull exists for trust features; coverage/hygiene is internal debt | PROCEED | — | 0.82 |
| A9-F2 | Adoption | Rollback plan is straightforward (revert + file restore) | PROCEED | — | 0.80 |
---
## Required Plan Changes (4 conditions)
1. **T02.6 labeling (A2-F1):** Add a note to T02.6 in `PLAN_v0.8.md` marking
it as a **v0.6 ship-defect bugfix** (first-connect Proxmox join has been
broken since v0.6 shipped due to `knownhosts.New` returning
`KeyError{Want:[]}` with no capture-and-persist). P04 audit (T04.2) must
record it as a ship-defect closure, not a v0.8 feature.
2. **P02 verification parity for doctor (A3-F3):** Add to Phase 2 Verification:
"`doctor proxmox` first-connect on a node with empty known_hosts → captures
the key + writes known_hosts + probe succeeds (mirrors T02.10 case 3 for
bootstrap). T02.6 and T02.9 are a paired change; the phase is not complete
until both callbacks use the capture-fix wrapper."
3. **T01.6 cli coverage escape valve (A5-F1):** Add to T01.6 acceptance
criterion: "If ≥65% (excluding `daemon.go`) is achieved but 70% is not after
Wave 2 effort, document the gap in a test-file comment + record a v0.9
follow-up; ship P01 at 65% for cli. Do NOT block P02/P03 on the last 5% of
cli coverage." (Carries forward RESEARCH §1.4's own flag as an explicit
escape valve.)
4. **verify-reqs regex + reverse direction (A8-F1 + A8-F2):**
- Change T03.1 ROADMAP-complete regex from
`^##\s*Milestone\s+v0\.\d+:.*—\s*\*\*COMPLETE\*\*` to
`^##\s*Milestone\s+v0\.\d+:.*—\s*\*\*[^*]*COMPLETE[^*]*\*\*` (substring
match within the bold span — handles `**COMPLETE**`,
`**COMPLETE (merged to main via v0.3)**`, and future variants).
- Add T03.2 golden fixture: a ROADMAP with
`**COMPLETE (merged to main via v0.3)**` → assert the milestone is
detected as complete.
- Add reverse-direction assertion to T03.1: every REQUIREMENTS row marked
`**Complete**` must reference a milestone ROADMAP marks COMPLETE (catches
premature-Complete drift).
- Add a PLAN note: "REQ-060 catches doc-vs-doc drift only. Code-vs-doc
drift (e.g., REQ-053 marked Complete but `cert_repo_test.go` missing —
verified missing in v0.7 ship) is NOT caught by this gate; it requires
the P04 `ciagent-audit` code-level review."
Additionally (lower-priority, from A7-F2):
5. **T02.10 case 7 (A7-F2):** Add integration test case: "known_hosts
pre-populated with a valid OpenSSH line for the host (simulating a v0.6
deployment or manual `ssh-keyscan`) → connect matches + succeeds. Covers
the v0.6→v0.8 migration path."
---
## Escalations
None. All 9 axes resolved at confidence ≥ 0.60. No axis requires escalation to
the operator; the 4 conditions are within the plan-author's authority to apply
before P01 execution begins.
---
## What the Plan Is NOT Doing (and should it?)
- **Not lifting the 3 zero-test packages to 70%.** Correct per D-047 — deferred
to v0.9. Not a gap.
- **Not adding a `peerDispatcher` seam unless needed.** Correct — conditional
on T01.4's 70% via `httptest.NewTLSServer`. Not a gap.
- **Not pre-populating `known_hosts` from a remote keyscan API.** Correct —
TOFU + manual `--host-key-fingerprint` cover the v0.8 surface. Not a gap.
- **Not catching code-vs-doc drift in verify-reqs.** **Known limitation** —
REQ-060 is a markdown-vs-markdown check. The REQ-053
`cert_repo_test.go`-missing case proves this class of drift is real. P04
`ciagent-audit` is the backstop. Documented in condition #4.
---
## Simplest 80%-of-the-value version
If forced to cut v0.8 to its smallest valuable form: **keep P02 (trust
hardening + TOFU bugfix) and P03 (verify-reqs); drop P01's coverage uplift for
the 3 zero-test packages + cli.** The TOFU bugfix alone (T02.6 + T02.9) fixes a
shipped security defect — that's the highest-value work. The verify-reqs gate
prevents the v0.7 drift from recurring. The coverage uplift on the 6
under-50% packages is valuable but not urgent; the 3 zero-test toe-holds are
the lowest-value work in the milestone. **The plan as written does not over-
scope** — it includes all of the above because the marginal cost is low — but
if P01 slips, the 3 toe-holds + cli are the first cuts to make.
---
## What Would Have to Be True for v0.8 to Succeed in the Next 90 Days
1. The `sessionRunner` seam (T01.1) unlocks proxmox 70% — **plausible** (same
pattern as the existing `sshDialer` seam).
2. `httptest.NewTLSServer` suffices for transport 70% without a new seam —
**plausible** (standard Go testing fixture).
3. The TOFU capture-fix (T02.6) is correct — **plausible** (verified against
x/crypto v0.54.0 semantics; integration tests T02.10 cover the cases).
4. `verify-reqs` regex matches all ROADMAP milestone header variants — **NOT
true today** (v0.2 header mismatch — condition #4 fixes this).
5. cli hits 70% in one phase — **NOT confirmed** (RESEARCH says 55-65%;
condition #3 adds the escape valve).
(4) and (5) are the two conditions that move the plan from "optimistic" to
"sound." Both are addressed by the 4 required changes.
---
**End of grill report.** Apply the 4 conditions to `PLAN_v0.8.md` before P01
execution. No REPLAN; no escalations. Overall verdict: **PROCEED-WITH-
CONDITION** (confidence 0.78).
+137 -22
View File
@@ -1,3 +1,131 @@
---
active:
- lead-developer
- backend-engineer
- data-engineer
deactivated:
- cli-engineer
- security-engineer
- devops-engineer
- network-engineer
- frontend-engineer
phase_specific: []
reason: |
Orca v0.8 is an NFR coverage & trust-hardening milestone. The work is
test coverage uplift across 9 packages (P01), SSH trust-surface
hardening in the existing proxmox + cli/node + security packages (P02),
and a requirements-hygiene Go program + Makefile target (P03). No
schema changes, no new security architecture, no packaging/distribution,
no UI.
Roster changes vs v0.7:
- lead-developer: RETAINED — owns cmd/orca smoke test, internal/cli
coverage (cert/doctor/audit/status/version subcommands), and the
cmd/verify-reqs Go program (coordination + glue-code territory).
- backend-engineer: RETAINED — owns internal/transport + internal/engine
tests (httptest.NewTLSServer, LocalExecutor stubs, PeerRegistry) and
the SSH trust-surface in internal/proxmox/bootstrap.go (pinned
host-key callback, TOFU capture fix, sessionRunner seam) plus
internal/cli/node.go (--host-key-fingerprint flag, key-reset
subcommand). Frameworks updated: connectrpc REMOVED (not in go.mod
per AD-014), golang.org/x/crypto/ssh ADDED (direct dep since v0.6).
- data-engineer: RETAINED — owns internal/store tests (cert_repo_test.go
gap + coverage uplift), internal/audit tests (sqlite-backed
audit_log asserts), internal/certpaths tests (path-join asserts),
and internal/jobspec tests (golden HCL fixtures). Frameworks
updated: modernc/sqlite + iter (matches actual go.mod).
- security-engineer: remains DEACTIVATED — v0.8 refines the existing
proxmox SSH trust surface (pinned callback, key-reset) but does NOT
add new security architecture. The trust work is backend-engineer
territory (it's SSH dialer + known_hosts file manipulation, not
X.509/CA/crypto code).
- cli-engineer: remains DEACTIVATED — merged into lead-developer
(cli coverage is test-only; --host-key-fingerprint and key-reset
are 1-flag + 1-subcommand additions to the existing node.go).
- devops-engineer: remains DEACTIVATED — verify-reqs is a Go program
(lead-developer territory), not a CI/packaging change. The
.coreci.yml edit is a 3-line validate-pipeline hook.
- network-engineer: remains DEACTIVATED — no transport/mTLS surface
change (transport coverage is test-only on the existing mTLS layer).
- frontend-engineer: remains DEACTIVATED — no web UI (unchanged
from v0.1 onward).
---
# Personas: Orca
## v0.8 persona assessment
### lead-developer
- **Domain**: coordination
- **Frameworks**: `cobra`, `net/http/httptest`, `testing`
- **Constraints**: `boundary-enforcement`, `offline-first`, `no-redundant-implementations`, `coverage-floor-70`
- **Territory**: `cmd/**`, `internal/cli/**`, `cmd/verify-reqs/**`, `Makefile`, `.coreci.yml`, `.ciagent/**`
- **Active**: true
- **Reason**: Owns P01 coverage for `cmd/orca` (smoke test of `main()`/`cli.Execute()`), `internal/cli` coverage for the non-node, non-daemon subcommands (`cert *`, `doctor *`, `audit list`, `status`, `version`), and the P03 `cmd/verify-reqs/main.go` Go program + `make verify-reqs` Makefile target + `.coreci.yml` validate-pipeline hook. Added `coverage-floor-70` constraint (D-047 tiered floor: 70% for the 6 under-50% packages, 50% for the 3 zero-test packages). Added `testing` + `net/http/httptest` to frameworks (test-only phase).
### backend-engineer
- **Domain**: backend
- **Frameworks**: `cobra`, `net/http`, `net/http/httptest`, `golang.org/x/crypto/ssh`, `golang.org/x/crypto/ssh/knownhosts`, `testing`
- **Constraints**: `API-first`, `error-handling`, `minimal-dependencies`, `security-first`, `tofu-host-key-pinning`, `pinned-host-key-fail-closed`, `atomic-file-rewrite`, `coverage-floor-70`
- **Territory**: `internal/transport/**`, `internal/engine/**`, `internal/proxmox/**`, `internal/cli/node.go`, `internal/daemon/**` (tests only)
- **Active**: true
- **Reason**: Owns P01 coverage for `internal/transport` (httptest.NewTLSServer for mTLS + stubDispatcher for DispatchClient) and `internal/engine` (LocalExecutor stubs + PeerRegistry in-memory tests). Owns P02 SSH trust hardening: `--host-key-fingerprint` pinned callback in `internal/proxmox/bootstrap.go` (D-045 OpenSSH SHA256:base64 format, AD-027/AD-028), the TOFU capture-fix (knownhosts.New returns KeyError{Want:[]} on first connect — must capture-and-persist via knownhosts.Line, AD-029 atomic rewrite), the `sessionRunner` seam refactor (P01 enabler for proxmox coverage), and `internal/cli/node.go` `--host-key-fingerprint` flag + `key-reset` subcommand (D-046 local known_hosts only). Frameworks updated: `connectrpc` REMOVED (not in go.mod per AD-014 — config.json still lists it but it's a stale entry), `golang.org/x/crypto/ssh` + `knownhosts` ADDED (direct dep since v0.6 D-030). Added `pinned-host-key-fail-closed` + `atomic-file-rewrite` + `coverage-floor-70` constraints.
### data-engineer
- **Domain**: data
- **Frameworks**: `modernc/sqlite`, `iter`, `hashicorp/hcl/v2`, `testing`
- **Constraints**: `schema-first`, `migration-safe`, `local-storage-only`, `no-goroutine-leak`, `nullable-column-handling`, `coverage-floor-70`
- **Territory**: `internal/store/**`, `internal/audit/**`, `internal/certpaths/**`, `internal/jobspec/**`, `internal/model/**`, `internal/store/migrations/**`
- **Active**: true
- **Reason**: Owns P01 coverage for `internal/store` (including the missing `cert_repo_test.go` — a v0.7 P01 leftover; Insert/Get/List/ListByNode/LatestForKind/PruneOlderThan/Delete + N=3 rotation history per REQ-025), `internal/audit` (sqlite-backed audit_log row asserts via `engine.Audit` + `store.AuditRepo`, slog capture via test handler), `internal/certpaths` (path-join asserts with temp dir + ORCA_HOME/ORCA_DB env), and `internal/jobspec` (golden-file HCL fixtures in a new `testdata/` dir + error-path table for Parse/Validate/ParseFile). Frameworks updated: `iter` + `hashicorp/hcl/v2` added (matches actual go.mod — jobspec uses hclsimple; store Watch uses iter.Seq). Added `coverage-floor-70` constraint.
### cli-engineer
- **Active**: false (v0.8)
- **Reason**: Deactivated — merged into lead-developer. The cli coverage work is test-only; `--host-key-fingerprint` and `key-reset` are a 1-flag and 1-subcommand addition to the existing `internal/cli/node.go`, not a new CLI subsystem.
### security-engineer
- **Active**: false (v0.8)
- **Reason**: Deactivated — v0.8 refines the existing proxmox SSH trust surface (pinned host-key callback, key-reset known_hosts rewrite) but does NOT add new security architecture (no new CA, no new X.509, no new crypto). The trust work is backend-engineer territory (SSH dialer + known_hosts file manipulation). The `internal/security/sshkey.go` is unchanged in v0.8. Was active in v0.6 (SSH keygen + sudoers), deactivated in v0.7, remains deactivated in v0.8.
### devops-engineer
- **Active**: false (v0.8)
- **Reason**: Deactivated — `verify-reqs` is a Go program (`cmd/verify-reqs/main.go`), not a CI/packaging change. The `.coreci.yml` edit is a 3-line validate-pipeline hook (lead-developer territory). No install.sh, Dockerfile, or release-pipeline surface in v0.8.
### network-engineer
- **Active**: false (v0.8)
- **Reason**: Deactivated — no transport/mTLS surface change. `internal/transport` coverage is test-only on the existing mTLS layer (httptest.NewTLSServer, no new TLS config). The SSH trust work is point-to-point bootstrap, not the mTLS mesh network-engineer owns.
### frontend-engineer
- **Active**: false (v0.8)
- **Reason**: No web UI in Orca (unchanged from v0.1 onward).
## Territory Enforcement
- **Mode**: `warn` (per `config.json`)
- **Behavior**: Out-of-territory file changes log a warning but do not block.
- **Key overlaps in v0.8** (lead-developer adjudicates):
- `internal/cli/node.go` — backend-engineer (`--host-key-fingerprint` flag + `key-reset` subcommand + proxmox pass-through) vs lead-developer (cli coverage tests). Boundary: backend owns the command implementation; lead owns the test files (`node_test.go`).
- `internal/proxmox/bootstrap.go` — backend-engineer (pinned callback, TOFU fix, sessionRunner seam) vs data-engineer (no overlap — proxmox has no store/audit code). Clean boundary.
- `cmd/verify-reqs/main.go` — lead-developer (Go program + Makefile + .coreci.yml) vs data-engineer (no overlap — verify-reqs parses markdown, not DB). Clean boundary.
- `internal/store/cert_repo_test.go` — data-engineer (test file) vs backend-engineer (no overlap — cert_repo is data territory). Clean boundary.
## v0.8 vs v0.7 Persona Diff
| Change | Rationale |
|--------|-----------|
| `lead-developer` retained | Owns cmd/orca smoke test, internal/cli coverage (non-node subcommands), cmd/verify-reqs Go program. |
| `backend-engineer` retained | Owns internal/transport + internal/engine tests + SSH trust-surface in proxmox + cli/node. Frameworks corrected: connectrpc removed (not in go.mod), x/crypto/ssh added. |
| `data-engineer` retained | Owns internal/store (cert_repo gap) + internal/audit + internal/certpaths + internal/jobspec tests. Frameworks corrected: iter + hcl/v2 added. |
| `security-engineer` remains deactivated | v0.8 refines existing SSH trust surface, no new security architecture. |
| `cli-engineer` remains deactivated | Merged into lead-developer (test-only + 1 flag + 1 subcommand). |
| `devops-engineer` remains deactivated | verify-reqs is a Go program, not CI/packaging. |
| `network-engineer` remains deactivated | No transport/mTLS surface change (test-only). |
| `frontend-engineer` remains deactivated | No web UI. |
---
## v0.7 baseline (preserved for traceability)
---
active_personas:
- lead-developer
@@ -28,11 +156,7 @@ reason: |
- devops-engineer: DEACTIVATED — no packaging/distribution in v0.7.
---
# Personas: Orca
## Roster
### lead-developer
### lead-developer (v0.7)
- **Domain**: coordination
- **Frameworks**: `cobra`
- **Constraints**: `boundary-enforcement`, `offline-first`, `no-redundant-implementations`
@@ -40,7 +164,7 @@ reason: |
- **Active**: true
- **Reason**: Coordination across P01/P02/P03. SSH/bootstrap touches security + cli + store + doctor — territory overlaps need adjudication (proxmox package boundary, doctor Proxmox check scaffolding).
### backend-engineer
### backend-engineer (v0.7)
- **Domain**: backend
- **Frameworks**: `cobra`, `net/http`, `golang.org/x/crypto/ssh`
- **Constraints**: `API-first`, `error-handling`, `minimal-dependencies`, `security-first`, `idempotent-bootstrap`
@@ -48,7 +172,7 @@ reason: |
- **Active**: true
- **Reason**: Owns the `orca init` full-bootstrap orchestration (CA + cert + db + localhost node, idempotent) and the `internal/proxmox/bootstrap.go` SSH session sequence (dial, deploy pubkey, useradd, pveum, sudoers, visudo validate). Added `idempotent-bootstrap` constraint (D-036 — re-run must be skip-and-refresh) and `golang.org/x/crypto/ssh` to frameworks.
### data-engineer
### data-engineer (v0.7)
- **Domain**: data
- **Frameworks**: `modernc/sqlite`, `iter`
- **Constraints**: `schema-first`, `migration-safe`, `local-storage-only`, `no-goroutine-leak`, `nullable-column-handling`
@@ -56,7 +180,7 @@ reason: |
- **Active**: true
- **Reason**: Reactivated for v0.6. Owns migration `0006_node_kind_os.sql` (REQ-049 — nullable `kind`/`os` columns, backward-compatible) and `NodeRepo` schema extension (Insert/Get/List/Watch/scanNode column additions + new `GetByName`/`UpdateLastSeenAndOS` helpers). Added `nullable-column-handling` constraint (NULL → `""` in Go struct, not nil-deref).
### cli-engineer
### cli-engineer (v0.7)
- **Domain**: CLI/UX
- **Frameworks**: `cobra`, `pflag`
- **Constraints**: `discoverable-help`, `consistent-flag-naming`, `human-readable-output`, `machine-readable-json-flag`, `signal-handling`, `password-flag-redaction`
@@ -64,7 +188,7 @@ reason: |
- **Active**: true
- **Reason**: Owns `orca init` multi-step bootstrap output UX (progress lines per step), `orca node join --type/--host/--user/--password/--proxmox-user/--proxmox-role` flag wiring, and `doctor os`/`doctor proxmox` subcommand wiring. Added `password-flag-redaction` constraint (D-031 — `--password` never echoed, prefer `$ORCA_PROXMOX_PASSWORD`, zero after use).
### security-engineer
### security-engineer (v0.7)
- **Domain**: security
- **Frameworks**: `crypto/tls`, `crypto/x509`, `crypto/ed25519`, `golang.org/x/crypto/ssh`, `slog`
- **Constraints**: `no-panic-in-production`, `structured-audit-logging`, `no-secret-in-logs`, `input-validation`, `least-privilege`, `tofu-host-key-pinning`, `noexec-sudoers`
@@ -72,28 +196,19 @@ reason: |
- **Active**: true
- **Reason**: Reactivated for v0.6. Owns `internal/security/sshkey.go` (Ed25519 keygen, 0600/0644 mode enforcement per REQ-033 spirit), TOFU host-key pinning via `knownhosts.New`, sudoers least-privilege design (NOEXEC on pct/qm, exclude pvesh, no NOEXEC on apt-get/dpkg), password redaction (D-031), and audit logging of all bootstrap/join actions (REQ-052). Added `tofu-host-key-pinning` and `noexec-sudoers` constraints. Co-owns `internal/proxmox/**` with backend-engineer (security owns SSH auth + sudoers content; backend owns the session orchestration).
### devops-engineer
### devops-engineer (v0.7)
- **Active**: false (v0.6)
- **Reason**: Deactivated — v0.6 has no install.sh, Dockerfile, .coreci.yml, or release-pipeline surface. The Proxmox SSH bootstrap is backend + security work, not devops. Was active in v0.5 (distribution milestone).
### network-engineer
### network-engineer (v0.7)
- **Active**: false (v0.6)
- **Reason**: v0.6 has no transport/mTLS surface. SSH is point-to-point bootstrap, not the mTLS mesh network-engineer owns.
### frontend-engineer
### frontend-engineer (v0.7)
- **Active**: false (v0.6)
- **Reason**: No web UI in Orca (unchanged from v0.1 onward).
## Territory Enforcement
- **Mode**: `warn` (per `config.json`)
- **Behavior**: Out-of-territory file changes log a warning but do not block.
- **Key overlaps in v0.6** (lead-developer adjudicates):
- `internal/proxmox/bootstrap.go` — security-engineer (SSH auth, sudoers, PVE role) + backend-engineer (session orchestration, error handling). Boundary: security package exposes `BootstrapProxmox(ctx, opts) error`; the function lives in `internal/proxmox` but imports `internal/security` for SSH key handling.
- `internal/doctor/doctor.go` `Proxmox()` — reuses `internal/proxmox` SSH client (security) but check scaffolding clones `doctor.Network()` pattern. Backend-engineer adjudicates (network-engineer deactivated).
- `internal/store/node_repo.go` — data-engineer territory, but the `UpdateLastSeenAndOS` caller is `internal/cli/init.go` (backend). Standard repo-consumer boundary.
## v0.6 vs v0.5 Persona Diff
### v0.6 vs v0.5 Persona Diff (v0.7 baseline reference)
| Change | Rationale |
|--------|-----------|
+64
View File
@@ -0,0 +1,64 @@
# Phase 4 Verification Report — v0.7: --pprof Opt-in on orca daemon
**Phase**: 4
**Branch**: `phase/04-pprof-daemon`
**REQ Coverage**: REQ-056
**Milestone**: v0.7 (Hardening & Completion)
## Structural Verification
### Files Created
- `internal/daemon/pprof.go``StartPprof(addr, log) (*http.Server, error)`: dedicated mux + server, disabled by default, WARN log
- `internal/daemon/pprof_test.go` — 5 tests (disabled, enabled, shutdown, mux isolation, full server lifecycle)
- `internal/cli/daemon_test.go``TestDaemonPprofFlag` (flag registration + default)
### Files Modified
- `internal/daemon/server.go``PprofAddr` in Options, `pprofServer` field, `NewServer` starts pprof, `Shutdown` stops both
- `internal/cli/daemon.go``--pprof` flag, `PprofAddr` in daemon.Options, conditional startup output line
## Behavioral Verification
### Test Results
```
go test ./... → all PASS (exit 0)
go test -race ./internal/daemon/... ./internal/cli/... → all PASS
go vet ./... → clean
make build → clean
```
### CLI Verification
```
./bin/orca daemon --help → shows --pprof string flag (default "")
```
### Live Smoke Test
- `--pprof 127.0.0.1:16060` → WARN logged, `/debug/pprof/` returns 200, `/debug/pprof/cmdline` 200, `/debug/pprof/heap` 200
- `/healthz` on pprof listener → 404 (mux isolation confirmed, AD-024)
- Clean shutdown stops both servers
## Security Verification
- pprof on a **separate** `*http.Server` + `*http.ServeMux`, never on the mTLS daemon listener (AD-024) — verified by `TestStartPprof_MuxIsolated` (`/healthz` returns 404 on pprof mux)
- Default **disabled** — no pprof listener unless `--pprof` is explicitly set
- WARN log on startup: "unauthenticated, operator-only — do not expose publicly"
- No `import _ "net/http/pprof"` side-effect registration on `DefaultServeMux` — all handlers explicitly registered on the dedicated mux
## Quality Verification
- No new dependencies (stdlib `net/http`, `net/http/pprof`, `log/slog`, `time` only)
- No comments added (per project convention)
- `go.mod` unchanged
- Test style matches existing `server_test.go`
## Must-Haves Checklist
- [x] `internal/daemon/pprof.go``StartPprof` with dedicated mux, all pprof handlers
- [x] `internal/daemon/server.go``PprofAddr` in Options, `pprofServer` field, lifecycle integration
- [x] `internal/cli/daemon.go``--pprof` flag, passed to Options, conditional startup output
- [x] `internal/daemon/pprof_test.go` — 5 tests (disabled, enabled, shutdown, mux isolation, lifecycle)
- [x] `internal/cli/daemon_test.go` — flag registration test
- [x] AD-024: pprof mux separate from mTLS daemon mux (verified by test)
## Verdict
**PASS** — all 4 verification layers pass. REQ-056 is fully covered. The `--pprof` opt-in endpoint runs on a separate listener with a dedicated mux, is disabled by default, and logs a WARN when enabled. I-308 (deferred since v0.2) is now implemented.
+347
View File
@@ -0,0 +1,347 @@
# Phase Plans: Orca v0.8 — Coverage & Trust Hardening
All 4 execution phases + final review with vertical-slice structure, wave
ordering, persona assignment, and REQ-ID mapping. v0.8 scope: **Coverage &
Trust Hardening** — round-2 test coverage uplift across 9 packages (tiered
floor: ≥70% for 6 retested, ≥50% for 3 zero-test per D-047), SSH trust
hardening (`--host-key-fingerprint` pre-pin + `orca node key-reset` + latent
TOFU capture-fix + `Result.HostKeyFingerprint` population), and a
requirements-hygiene gate (`make verify-reqs`).
Branching: `phase/01-coverage-round2`..`phase/04-final-review-ship` on the
`milestone/v0.8-coverage-trust-hardening` branch (numbering restarts per
milestone per branch-strategy.md).
Milestone type: **NFR** (P01 test, P02 chore on the trust surface per D-043,
P03 chore, P04 docs/review). Tags run on the v0.7.x patch line: `v0.7.0`
(P0) … `v0.7.4` (P04 = milestone release).
**Vertical-slice integrity**: each phase is independently shippable.
- **P01** ships tests-only (no production code changes except the proxmox
`sessionRunner` seam, a backward-compatible interface extraction, and the
engine `peerDispatcher` seam per RESEARCH §1.3).
- **P02** ships the SSH trust features + TOFI bugfix + `Result` population.
- **P03** ships the hygiene gate (Go program + Makefile + CI hook).
- **P04** is review + ship + audit (no new REQs).
**Out of scope for v0.8** (candidate for v0.9, noted not added):
- Lifting the 3 zero-test packages from 50% → 70% (D-047 explicitly
toes-holds them; v0.9 can raise the floor).
- A `peerDispatcher` interface seam in engine beyond what P01 needs for 70%
coverage (httptest.NewTLSServer suffices; the seam is only added if
coverage cannot otherwise hit 70%).
- Pre-populating `known_hosts` from a remote keyscan API (TOFU + manual
`--host-key-fingerprint` cover the v0.8 trust surface).
- `verify-reqs` reverse-direction check (REQUIREMENTS Complete ↔ ROADMAP
COMPLETE both ways) — forward direction (ROADMAP-shipped → REQUIREMENTS
Complete) is the priority per the v0.7 drift that motivated REQ-060.
**Carried-forward research findings** (RESEARCH_v0.8.md, must incorporate):
- §1.1 per-package coverage strategies + tiered floors (D-047).
- §1.3 injected seams: reuse `sshDialer` (proxmox), `LocalExecutor` (engine),
`Dispatcher` (transport), `watchInterval` (store), `openTestDB`/`withFastWatch`/`initTestEnv`/`resetRootFlags`/`stubDispatcher` helpers.
- §1.4 realism flags: cli excludes `daemon.go`; `cmd/orca` 50% toe-hold only;
proxmox needs the `sessionRunner` seam to hit 70%.
- §2.1 latent TOFU capture bug (knownhosts.New returns KeyError{Want:[]} on
first connect and does NOT auto-write — current BootstrapProxmox treats it
as a dial failure).
- §2.2 `Result.HostKeyFingerprint` is declared but never populated (always
`""`); P02 must add `ssh.FingerprintSHA256` computation.
- §2.3 `--host-key-fingerprint` plugs in at `internal/cli/node.go` (flag) +
`internal/proxmox/bootstrap.go` (pinned callback).
- §2.4 `key-reset` is local-known_hosts-only (D-046), atomic rewrite (AD-029).
- §3 verify-reqs is a Go program at `cmd/verify-reqs/main.go` (~80 LOC,
stdlib only, AD-030) + `make verify-reqs` + `.coreci.yml` validate hook.
- §4 AD-025..AD-030 (renumbered AD-027..AD-030 in research for SSH/trust;
AD-025/AD-026 from earlier milestones are stable).
- §5 10 pitfalls carried into the risk register at the end of this file.
**Dependencies (RESEARCH §6)**: v0.8 adds **zero** new direct dependencies.
`ssh.FingerprintSHA256`, `knownhosts.Line`/`Normalize`/`KeyError` are in the
existing `golang.org/x/crypto` v0.54.0 dep. `verify-reqs` is stdlib-only.
`go.mod` is unchanged by v0.8.
---
## Phase 1: Coverage Uplift Round 2 (REQ-057)
**Branch**: `phase/01-coverage-round2`
**REQ Coverage**: REQ-057
**Tag**: `v0.7.1`
**Depends on**: Phase 0 (this plan + clarify + research)
**Source research**: RESEARCH_v0.8.md §1 (per-package strategies, helpers, seams)
### Tiered floor (D-047)
| Package | Current | Floor | Owner persona |
|---------|---------|-------|---------------|
| `internal/engine` | 8.3% | ≥ 70% | backend-engineer |
| `internal/proxmox` | 5.1% | ≥ 70% | backend-engineer |
| `internal/cli` | 27.6% | ≥ 70% (excluding `daemon.go`) | lead-developer |
| `internal/transport` | 26.3% | ≥ 70% | backend-engineer |
| `internal/store` | 47.2% | ≥ 70% | data-engineer |
| `internal/jobspec` | 47.6% | ≥ 70% | data-engineer |
| `internal/audit` | 0% (no tests) | ≥ 50% toe-hold | data-engineer |
| `internal/certpaths` | 0% (no tests) | ≥ 50% toe-hold | data-engineer |
| `cmd/orca` | 0% (no tests) | ≥ 50% toe-hold | lead-developer |
### Wave 1 — Seams + foundational test helpers (no production logic changes)
These are backward-compatible interface extractions that unlock the bulk of
coverage in Wave 2. They are the only production-code changes in P01; all
other P01 tasks add `_test.go` files only.
| Task ID | Owner | Wave | Must | Title | Files touched | Acceptance criterion |
|---------|-------|------|------|-------|---------------|----------------------|
| T01.1 | backend-engineer | 1 | Y | Add `sessionRunner` interface seam to proxmox | `internal/proxmox/bootstrap.go` | Extract a `sessionRunner` interface (`CombinedOutput(cmd string) ([]byte, error)`) ~10 LOC; default impl wraps `*ssh.Client.NewSession().CombinedOutput(...)`; `runRemote`/`deployPubKey`/`createLinuxUser`/`createPVERole`/`createPVEUser`/`assignPVEACL`/`writeSudoers`/`validateSudoers` use the seam. Backward compatible: existing callers unchanged. `go build ./internal/proxmox` PASS. (RESEARCH §1.3 gap #1, §5 pitfall #3) |
| T01.2 | backend-engineer | 1 | N | Add `peerDispatcher` seam to engine (only if needed for 70%) | `internal/engine/dispatcher.go` | Extract a `peerDispatcher` interface (`Submit(ctx, spec, key) (*SubmitResponse, error)`) so `dispatchToPeer` is testable without `httptest.NewTLSServer`. **Only add if T01.5 cannot otherwise hit 70% via `httptest.NewTLSServer` alone.** If added, backward compatible. (RESEARCH §1.3 gap #2, §5 pitfall #8) |
### Wave 2 — Per-package coverage tests (build on Wave 1 seams)
| Task ID | Owner | Wave | Must | Title | Files touched | Acceptance criterion |
|---------|-------|------|------|-------|---------------|----------------------|
| T01.3 | backend-engineer | 2 | Y | `internal/transport` tests → ≥ 70% | `internal/transport/mtls_test.go` (NEW), `internal/transport/dispatch_test.go` (NEW), `internal/transport/handshake_log_test.go` (NEW), `internal/transport/retry_test.go` (NEW, extend) | `httptest.NewTLSServer` with a test CA (reuse `security.CAInit`/`GenerateCSR`/`SignCSR` per RESEARCH §1.2) for mTLS handshake paths; `stubDispatcher` (daemon/dispatch_test.go:24) pattern for Dispatch RPC; capture slog via a test `slog.Handler` for handshake_log. `go test -cover ./internal/transport` → ≥ 70% (was 26.3%). |
| T01.4 | backend-engineer | 2 | Y | `internal/engine` tests → ≥ 70% | `internal/engine/executor_test.go` (NEW), `internal/engine/dispatcher_test.go` (NEW), `internal/engine/peer_test.go` (NEW), `internal/engine/scheduler_test.go` (extend), `internal/engine/registry_test.go` (NEW, if registry exists) | `Executor.Start`/`Wait` lifecycle (echo/false/ctx-cancel/Env propagation per REQ-021); `Dispatcher.Submit` with stubbed `LocalExecutor` + (if T01.2 added) stubbed `peerDispatcher` OR `httptest.NewTLSServer`; `PeerRegistry` in-memory Add/Remove/All/Get. Reuse `openTestDB` (node_repo_test.go:12). `go test -cover ./internal/engine` → ≥ 70% (was 8.3%). |
| T01.5 | backend-engineer | 2 | Y | `internal/proxmox` tests → ≥ 70% | `internal/proxmox/bootstrap_test.go` (extend) | Swap `sshDialer` (existing seam) for a fake returning a mock `*ssh.Client`; swap `sessionRunner` (T01.1 seam) for a fake that returns canned `CombinedOutput` bytes. Assert full bootstrap sequence calls the right shell commands in order; idempotent re-run ("already exists" → no-op); SSH auth failure → wrapped error; no password logged (D-031). `go test -cover ./internal/proxmox` → ≥ 70% (was 5.1%). |
| T01.6 | lead-developer | 2 | Y | `internal/cli` tests → ≥ 70% (excluding daemon.go) with GRILL condition #3 escape valve | `internal/cli/node_test.go` (NEW), `internal/cli/job_test.go` (NEW), `internal/cli/cert_test.go` (NEW), `internal/cli/doctor_test.go` (NEW), `internal/cli/audit_test.go` (NEW), `internal/cli/status_test.go` (NEW), `internal/cli/version_test.go` (NEW), `internal/cli/node_capacity_test.go` (NEW) | Table-driven `rootCmd.Execute()` against temp `ORCA_HOME` per subcommand (reuse `initTestEnv`/`resetRootFlags`/`discardWriter` per RESEARCH §1.2). Mock the proxmox path via `sshDialer` + `sessionRunner` seams. `daemon.go` is excluded — covered by `internal/daemon/server_test.go`. `go test -cover ./internal/cli` → ≥ 70% of non-daemon files (document the exclusion in a test-file comment). **GRILL condition #3 escape valve**: if 70% is not reached after Wave 2 effort and ≥ 65% is achieved (RESEARCH §1.4 flags 55-65% as realistic for one phase), ship cli at 65% and do NOT block P02/P03 on the last 5%; record the shortfall + rationale in the P01 verification commit. |
| T01.7 | data-engineer | 2 | Y | `internal/store` tests → ≥ 70% (incl. missing `cert_repo_test.go`) | `internal/store/cert_repo_test.go` (NEW — v0.7 P01 leftover, RESEARCH §1.1), `internal/store/node_repo_test.go` (extend), `internal/store/job_task_repo_test.go` (extend), `internal/store/audit_repo_test.go` (extend), `internal/store/capacity_repo_test.go` (extend) | `cert_repo_test.go`: Insert/Get/List/ListByNode/LatestForKind/PruneOlderThan/Delete + N=3 rotation history per REQ-025 + duplicate-serial error. Reuse `openTestDB`/`withFastWatch` (RESEARCH §1.2). `go test -cover ./internal/store` → ≥ 70% (was 47.2%). |
| T01.8 | data-engineer | 2 | Y | `internal/jobspec` tests → ≥ 70% | `internal/jobspec/spec_test.go` (extend), `internal/jobspec/testdata/*.hcl` (NEW golden fixtures) | Golden-file HCL fixtures (multi-task, env vars, args) + error-path table (missing job, no tasks, missing command, malformed HCL, empty file, nonexistent file for `ParseFile`). `go test -cover ./internal/jobspec` → ≥ 70% (was 47.6%). |
| T01.9 | data-engineer | 2 | Y | `internal/audit` first tests → ≥ 50% toe-hold | `internal/audit/audit_test.go` (NEW) | Construct `Audit` with real `engine.Audit` backed by `:memory:` sqlite (via `store.NewAuditRepo` + `engine.NewAudit`); assert rows in `audit_log` table; capture slog via a test `slog.Handler` for `LogHandshakeOK`/`LogHandshakeFailed`. `go test -cover ./internal/audit` → ≥ 50% (was 0%). |
| T01.10 | data-engineer | 2 | Y | `internal/certpaths` first tests → ≥ 50% toe-hold | `internal/certpaths/certpaths_test.go` (NEW) | Temp dir + `t.Setenv("ORCA_HOME", dir)` + assert each `*Path()` returns `filepath.Join(dir, <file>)`; test `ORCA_DB` override; test default-to-`~/.orca` fallback. Model on `namespace_test.go` (cli). `go test -cover ./internal/certpaths` → ≥ 50% (was 0%). |
| T01.11 | lead-developer | 2 | Y | `cmd/orca` smoke test → ≥ 50% toe-hold | `cmd/orca/main_test.go` (NEW), possibly `cmd/orca/main.go` (refactor `main()` into `run() int` for testability) | Refactor `main()` to `run() int` (returns exit code; `main()` calls `os.Exit(run())`) so the test can call `run()` directly with a forced error path and assert non-zero exit + stderr contains "error:". Low-effort toe-hold — do NOT over-invest (RESEARCH §1.1, §5 pitfall #6). `go test -cover ./cmd/orca` → ≥ 50% (was 0%). |
### Wave 3 — Coverage gate verification
| Task ID | Owner | Wave | Must | Title | Files touched | Acceptance criterion |
|---------|-------|------|------|-------|---------------|----------------------|
| T01.12 | lead-developer | 3 | Y | Coverage-gate verification (all 9 packages hit tiered floor) | none (verification only) | `go test -cover ./internal/engine ./internal/proxmox ./internal/cli ./internal/transport ./internal/store ./internal/jobspec` → each ≥ 70%; `go test -cover ./internal/audit ./internal/certpaths ./cmd/orca` → each ≥ 50%. `go test -race ./...` PASS. Any races fixed in-phase (not deferred). |
### Phase 1 Must-Haves (summary)
All 9 packages hit their tiered floor (D-047): T01.1, T01.3, T01.4, T01.5,
T01.6, T01.7, T01.8, T01.9, T01.10, T01.11, T01.12. T01.2 is conditional
(only if needed for engine 70%).
### Phase 1 Verification
- `go build ./...` PASS
- `go vet ./...` PASS
- `go test -race ./...` PASS
- Per-package coverage hits the tiered floor (T01.12)
- The proxmox `sessionRunner` seam is backward compatible (existing
`BootstrapProxmox` callers unchanged)
- No new direct deps (`go.mod` unchanged)
---
## Phase 2: SSH Trust Hardening (REQ-058, REQ-059)
**Branch**: `phase/02-ssh-trust-hardening`
**REQ Coverage**: REQ-058, REQ-059
**Tag**: `v0.7.2`
**Depends on**: Phase 1 (proxmox `sessionRunner` seam from T01.1 is in place;
the trust-surface code is now testable)
**Source research**: RESEARCH_v0.8.md §2 (TOFU bug, fingerprint computation,
flag wiring, key-reset atomic rewrite) + §4 AD-027..AD-029
**Phase type**: chore (trust-surface hardening per D-043 — refines existing
`orca node join --type proxmox` flow + existing TOFU `known_hosts` store; no
new orchestration capability)
### Wave 1 — Trust-surface foundations (security helpers + flag declarations)
| Task ID | Owner | Wave | Must | Title | Files touched | Acceptance criterion |
|---------|-------|------|------|-------|---------------|----------------------|
| T02.1 | backend-engineer | 1 | Y | Add `security.SSHFingerprintSHA256` helper (AD-027) | `internal/security/sshkey.go` (extend) OR `internal/security/fingerprint.go` (extend) | Thin wrapper over `ssh.FingerprintSHA256(pubKey ssh.PublicKey) string` returning the canonical `SHA256:base64` string. Do NOT reuse `security.Fingerprint` (X.509 hex — different domain per RESEARCH §2.2). Unit test: known Ed25519 pub key → known `SHA256:` string. |
| T02.2 | backend-engineer | 1 | Y | Export `security.WriteAtomic` (AD-029 enabler) | `internal/security/ca.go` | Rename `writeAtomic``WriteAtomic` (export) + update existing in-package callers. The `key-reset` atomic known_hosts rewrite (T02.7) needs it. Alternatively copy the ~20-LOC pattern into `proxmox` if export is undesirable — **recommend export** (RESEARCH §5 pitfall #10). `go build ./internal/security` PASS. |
| T02.3 | backend-engineer | 1 | Y | Add `--host-key-fingerprint` flag on `orca node join` (D-044) | `internal/cli/node.go` | `nodeJoinCmd.Flags().StringVar(&joinHostKeyFP, "host-key-fingerprint", "", "SSH host key SHA256:base64 fingerprint (pre-pin; supersedes TOFU for --type proxmox)")` in the flag-registration block (node.go:344-354). Add `joinHostKeyFP string` to the var block (node.go:47-60). Validation in `RunE`: if `joinHostKeyFP != ""` and `--type != proxmox`, emit a clear error ("--host-key-fingerprint requires --type proxmox today"). Flag is generic for future SSH-joined kinds (D-044). |
| T02.4 | backend-engineer | 1 | Y | Add `HostKeyFingerprint` field to `proxmox.Options` | `internal/proxmox/bootstrap.go` | Add `HostKeyFingerprint string` to the `Options` struct (bootstrap.go:55). Pass-through from `internal/cli/node.go` joinProxmox (node.go:158-166): `HostKeyFingerprint: joinHostKeyFP`. |
### Wave 2 — Trust features + bugfix (build on Wave 1)
| Task ID | Owner | Wave | Must | Title | Files touched | Acceptance criterion |
|---------|-------|------|------|-------|---------------|----------------------|
| T02.5 | backend-engineer | 2 | Y | Implement `pinnedHostKeyCallback` (REQ-058, AD-028) | `internal/proxmox/bootstrap.go` | `pinnedHostKeyCallback(expectedSHA256Base64 string) (ssh.HostKeyCallback, error)`: validate `SHA256:` prefix up front (reject raw hex with a clear error per D-045); callback receives server's `ssh.PublicKey`, computes `ssh.FingerprintSHA256(key)` (via T02.1 helper or inline), compares full strings to the operator-supplied value; returns `nil` on match, `error` on mismatch (fail closed). In `BootstrapProxmox`: if `opts.HostKeyFingerprint != ""` use `pinnedHostKeyCallback`, else fall back to the TOFU callback (T02.6). Unit test: match → callback returns nil; mismatch → returns error mentioning REQ-058; non-`SHA256:`-prefixed input → constructor returns error. |
| T02.6 | backend-engineer | 2 | Y | **BUGFIX (v0.6 ship-defect)**: FIX the latent TOFU capture bug (RESEARCH §2.1, §5 pitfall #1, GRILL condition #1) | `internal/proxmox/bootstrap.go` | Wrap `knownhosts.New(...)` with a custom callback that: on `*knownhosts.KeyError{Want: []}` (host unknown) captures the server-presented `ssh.PublicKey`, writes a line via `knownhosts.Line([]string{knownhosts.Normalize(addr)}, key)` to `certpaths.KnownHostsPath()` using `security.WriteAtomic` (T02.2, AD-029), and returns `nil` (allow the dial to proceed). On `*knownhosts.KeyError{Want: [knownKey]}` (mismatch) returns the error (MITM detection). On `nil` (host present + match) returns `nil`. This fixes the v0.6 latent ship-defect where first-connect Proxmox join always failed (verified against `golang.org/x/crypto@v0.54.0/ssh/knownhosts/knownhosts.go:370-385`). P04 audit must record this as ship-defect closure. Unit test: first-connect captures the key + writes known_hosts; second-connect matches; mismatch-connect fails. |
| T02.7 | backend-engineer | 2 | Y | Populate `Result.HostKeyFingerprint` (RESEARCH §2.2, §5 pitfall #2) | `internal/proxmox/bootstrap.go` | In the capture path (T02.6) and the pinned path (T02.5), set `Result.HostKeyFingerprint = ssh.FingerprintSHA256(hostKey)` (via T02.1). The field is currently declared (bootstrap.go:83-85) but always `""`. After T02.7, `orca node join --type proxmox` output includes the real fingerprint. Unit test: `Result.HostKeyFingerprint` is non-empty + `SHA256:`-prefixed after a successful bootstrap. |
| T02.8 | backend-engineer | 2 | Y | Implement `orca node key-reset <node>` (REQ-059, D-046, AD-029) | `internal/cli/node.go`, `internal/proxmox/bootstrap.go` (new `ResetHostKey` helper OR inline in cli) | New `nodeKeyResetCmd` (`&cobra.Command{Use: "key-reset <node>", Args: cobra.ExactArgs(1), RunE: ...}`) registered via `nodeCmd.AddCommand(nodeKeyResetCmd)` (node.go:358-360). `RunE`: (1) resolve `<node>` arg via `nodeRegistry()` (node.go:37) → get node row → use `node.Name` (the host address for proxmox nodes) as the `known_hosts` match key; (2) call `proxmox.ResetHostKey(host) error` which reads `certpaths.KnownHostsPath()`, filters lines whose host field (before first whitespace, normalized via `knownhosts.Normalize`) matches, rewrites via `security.WriteAtomic` (T02.2); (3) audit-log `event=node.key_reset` with `actor`+`node`+`host` via `engine.Audit.Record`; (4) print `✓ Host key reset for <node> (next connect will re-pin via TOFU or --host-key-fingerprint)`. **Local only — do NOT revoke remote authorized_keys** (D-046). Unit test: known_hosts with 2 entries for the target host + 1 for another host → after reset, target's 2 lines removed, other host's line intact; audit row inserted. |
| T02.9 | backend-engineer | 2 | Y | Apply the TOFU capture-fix to `doctor proxmox` probe (GRILL condition #2 — doctor parity with bootstrap) | `internal/doctor/doctor.go` | The doctor proxmox probe (doctor.go:412-415) uses the same `knownhosts.New(...)` callback pattern as bootstrap. Apply the same capture-fix wrapper (T02.6) so `doctor proxmox` on a first-connect node doesn't fail. **P02 is not complete until both bootstrap (T02.6) and doctor (T02.9) callbacks use the capture-fix wrapper — GRILL condition #2 binding parity check.** (If the doctor probe already relies on a prior `node join` having populated `known_hosts`, the fix is still correct — it makes the doctor robust to a missing entry.) |
### Wave 3 — End-to-end integration + verification
| Task ID | Owner | Wave | Must | Title | Files touched | Acceptance criterion |
|---------|-------|------|------|-------|---------------|----------------------|
| T02.10 | backend-engineer | 3 | Y | End-to-end trust-surface integration tests | `internal/proxmox/bootstrap_test.go` (extend), `internal/cli/node_test.go` (extend) | (1) `--host-key-fingerprint` with a correct pin → bootstrap succeeds + `Result.HostKeyFingerprint` matches the pin; (2) `--host-key-fingerprint` with a wrong pin → bootstrap fails fast with the REQ-058 mismatch error; (3) no `--host-key-fingerprint` + first connect (empty known_hosts) → TOFU captures the key + writes known_hosts + bootstrap succeeds; (4) no flag + second connect (known_hosts has the key) → matches + succeeds; (5) no flag + mismatch (known_hosts has a different key) → fails with MITM error; (6) `orca node key-reset <node>` → known_hosts entry removed + audit row inserted + next connect re-pins; (7) known_hosts pre-populated (v0.6→v0.8 migration path: existing entry from a prior join) → second-connect matches without re-capture, covering the upgrade path. |
| T02.11 | backend-engineer | 3 | Y | `--host-key-fingerprint` non-proxmox type validation test | `internal/cli/node_test.go` (extend) | `orca node join --type linux --host-key-fingerprint SHA256:...` → clear error ("--host-key-fingerprint requires --type proxmox today"). Validates D-044 RunE check from T02.3. |
### Phase 2 Must-Haves (summary)
- T02.1, T02.2, T02.3, T02.4 (Wave 1 foundations)
- T02.5 (`--host-key-fingerprint` pinned callback — REQ-058)
- T02.6 (TOFU capture-fix — latent bug)
- T02.7 (`Result.HostKeyFingerprint` populated)
- T02.8 (`orca node key-reset` — REQ-059)
- T02.9 (doctor proxmox TOFU fix)
- T02.10, T02.11 (integration + validation)
### Phase 2 Verification
- `go build ./...` PASS
- `go vet ./...` PASS
- `go test -race ./internal/proxmox/... ./internal/cli/... ./internal/doctor/... ./internal/security/...` PASS
- `./bin/orca node join --help` shows `--host-key-fingerprint` flag
- `./bin/orca node key-reset --help` shows the key-reset subcommand
- Pinned mismatch → fail closed (T02.10 case 2)
- TOFU first-connect → captures + succeeds (T02.10 case 3)
- `Result.HostKeyFingerprint` is non-empty after bootstrap (T02.7)
- `key-reset` removes only the target host's known_hosts lines + audit-logs (T02.8)
- No new direct deps
---
## Phase 3: Requirements-Hygiene Gate (REQ-060)
**Branch**: `phase/03-verify-reqs`
**REQ Coverage**: REQ-060
**Tag**: `v0.7.3`
**Depends on**: Phase 2 (P03 is independent of P02 code, but ships after per
ROADMAP ordering; the verify-reqs program parses the `.ciagent/` markdown
which is stable by P03)
**Source research**: RESEARCH_v0.8.md §3 (Makefile, .coreci.yml, parsing
approach, AD-030) + §4 AD-030
### Wave 1 — Go program
| Task ID | Owner | Wave | Must | Title | Files touched | Acceptance criterion |
|---------|-------|------|------|-------|---------------|----------------------|
| T03.1 | lead-developer | 1 | Y | `cmd/verify-reqs/main.go` — Go program (~80 LOC, stdlib only, AD-030, GRILL condition #4 regex + reverse direction) | `cmd/verify-reqs/main.go` (NEW) | Parses `.ciagent/ROADMAP.md` + `.ciagent/REQUIREMENTS.md` using `regexp` (stdlib). **Forward assertion**: for every REQ-ID in REQUIREMENTS.md whose `Phase` column references a milestone that ROADMAP marks COMPLETE (substring-match `COMPLETE` within the bold span — NOT exact `\*\*COMPLETE\*\*` which misses v0.2's `**COMPLETE (merged to main via v0.3)**` header at ROADMAP.md:23), the REQUIREMENTS `Status` must be `Complete`. **Reverse assertion (GRILL condition #4)**: for every REQ-ID in REQUIREMENTS.md marked `Complete`, the corresponding milestone in ROADMAP.md must be marked COMPLETE. Regex: REQUIREMENTS row `^\|\s*(REQ-\d+)\s*\|.*?\|\s*\*\*(Complete\|Pending)\*\*\s*\|`; ROADMAP milestone-complete `^##\s*Milestone\s+v0\.\d+:.*—\s*\*\*COMPLETE[^\*]*\*\*` (substring tolerant); map milestone → REQs via the REQUIREMENTS `Phase` column (e.g. `v0.7 P1` → milestone `v0.7`). Exit 0 on consistency; exit 1 with a diff listing (REQ-ID + current status + expected status + direction) on drift. CLI: `go run ./cmd/verify-reqs .ciagent/ROADMAP.md .ciagent/REQUIREMENTS.md` (args optional; defaults to those paths). **Scope note (GRILL)**: REQ-060 catches doc-vs-doc drift only; code-vs-doc drift (e.g. the REQ-053 `cert_repo_test.go` omission — verified missing) is out of scope for this gate and handled by P04 `ciagent-audit`. |
| T03.2 | lead-developer | 1 | Y | `cmd/verify-reqs/main_test.go` — golden-file tests | `cmd/verify-reqs/main_test.go` (NEW), `cmd/verify-reqs/testdata/` (NEW: `roadmap_clean.md`, `requirements_clean.md`, `roadmap_drift.md`, `requirements_drift.md`) | (1) Clean pair (ROADMAP v0.X COMPLETE + REQUIREMENTS REQ-XXX Complete) → exit 0, no diff; (2) Drift pair (ROADMAP v0.X COMPLETE + REQUIREMENTS REQ-XXX Pending) → exit 1 + diff lists the stale REQ; (3) Multiple drifts → all reported; (4) Missing args → uses defaults; (5) Malformed markdown → clear error (not a silent pass). |
### Wave 2 — Makefile + CI hook
| Task ID | Owner | Wave | Must | Title | Files touched | Acceptance criterion |
|---------|-------|------|------|-------|---------------|----------------------|
| T03.3 | lead-developer | 2 | Y | `make verify-reqs` target | `Makefile` | Add `verify-reqs` target: `go run ./cmd/verify-reqs .ciagent/ROADMAP.md .ciagent/REQUIREMENTS.md`. Add to `.PHONY`. `make verify-reqs` exits 0 on the current repo (REQUIREMENTS was corrected during v0.8 SPECIFY). |
| T03.4 | lead-developer | 2 | Y | `.coreci.yml` validate-pipeline hook | `.coreci.yml` | Add a `verify-reqs` step to the `validate` pipeline (after `go-version`, alongside `gosec`/`govulncheck`/`gitleaks` per RESEARCH §3.2): `image: golang:1.25`, `commands: [make verify-reqs]`. Pipeline fails on drift. |
### Wave 3 — Synthetic drift verification
| Task ID | Owner | Wave | Must | Title | Files touched | Acceptance criterion |
|---------|-------|------|------|-------|---------------|----------------------|
| T03.5 | lead-developer | 3 | Y | Synthetic drift verification (REQ-060 acceptance) | none (verification only; temporarily flip a REQUIREMENTS row to Pending in a scratch commit, run `make verify-reqs`, assert exit 1 + diff, then revert) | (1) `make verify-reqs` on the current repo → exit 0; (2) flip one v0.7 REQ row to `Pending` in a scratch edit → `make verify-reqs` → exit 1 + diff lists that REQ-ID; (3) revert the scratch edit → exit 0. This is the REQ-060 acceptance criterion ("passes on current repo + fails on synthetic drift"). |
### Phase 3 Must-Haves (summary)
T03.1, T03.2, T03.3, T03.4, T03.5 — all must complete for the hygiene gate to
ship.
### Phase 3 Verification
- `go build ./cmd/verify-reqs` PASS
- `go test ./cmd/verify-reqs/...` PASS (golden-file tests)
- `make verify-reqs` → exit 0 on the current repo
- Synthetic drift → `make verify-reqs` exit 1 + diff (T03.5)
- `.coreci.yml` validate pipeline includes the `verify-reqs` step
- No new direct deps (stdlib only)
---
## Phase 4: Final Review + Ship + Audit (no new REQs)
**Branch**: `phase/04-final-review-ship`
**REQ Coverage**: all (REQ-057..060)
**Tag**: `v0.7.4` (milestone release)
**Depends on**: Phase 1 + Phase 2 + Phase 3
**Source**: milestone-release checklist (matches PLAN_v0.7 P05 structure)
### Wave 1 — Review + audit
| Task ID | Owner | Wave | Must | Title | Files touched | Acceptance criterion |
|---------|-------|------|------|-------|---------------|----------------------|
| T04.1 | lead-developer | 1 | Y | Multi-persona code review across all v0.8 phases | none (review only) | ciagent-review across P01..P03; P0 issues fixed in-phase; P1+ recorded in `.ciagent/` for post-hoc. |
| T04.2 | lead-developer | 1 | Y | Audit: reconstruction test + branch hygiene + commit discipline | none (audit only) | ciagent-audit: git log matches `.ciagent/` files; branch hygiene clean; commit discipline enforced. |
### Wave 2 — Ship
| Task ID | Owner | Wave | Must | Title | Files touched | Acceptance criterion |
|---------|-------|------|------|-------|---------------|----------------------|
| T04.3 | lead-developer | 2 | Y | Merge phase/04 → milestone/v0.8-coverage-trust-hardening | none | Fast-forward merge (or rebase-then-fast-forward per config). |
| T04.4 | lead-developer | 2 | Y | Merge milestone/v0.8 → main | none | Rebase-then-fast-forward per config. |
| T04.5 | lead-developer | 2 | Y | Tag `v0.7.4` (milestone release) | none | `git tag v0.7.4` on the merged main HEAD. Per-phase tags `v0.7.0`..`v0.7.4` all present. |
| T04.6 | lead-developer | 2 | Y | Create Gitea release `v0.7.4` with milestone summary | none | Release notes cover all 4 phases + REQ-057..060 + coverage deltas + trust-surface additions. |
### Wave 3 — Post-ship bookkeeping
| Task ID | Owner | Wave | Must | Title | Files touched | Acceptance criterion |
|---------|-------|------|------|-------|---------------|----------------------|
| T04.7 | lead-developer | 3 | Y | Update REQUIREMENTS.md — mark REQ-057..060 Complete | `.ciagent/REQUIREMENTS.md` | All 4 v0.8 REQ rows show `**Complete**` with phase + ship tag. `make verify-reqs` still passes (self-consistency). |
| T04.8 | lead-developer | 3 | Y | Update ROADMAP.md — mark v0.8 COMPLETE | `.ciagent/ROADMAP.md` | v0.8 milestone section shows `**COMPLETE**`; all phase checkboxes `[x]`. `make verify-reqs` still passes. |
| T04.9 | lead-developer | 3 | Y | Write + clear checkpoint | `.ciagent/` checkpoint | `{phase: 4, stage: "complete", phase_role: "final", milestone_complete: true}`; then clear checkpoint (milestone complete; next run starts a new milestone). |
### Phase 4 Must-Haves (summary)
All tasks (T04.1..T04.9) are must-haves — the final-review phase has no
optional work.
### Phase 4 Verification
- `make build` PASS
- `make test` PASS
- `make lint` PASS
- `make verify-reqs` PASS
- `go vet ./...` PASS
- `git log` on main shows all v0.8 phase commits
- `git tag --list 'v0.7.*'` shows v0.7.0..v0.7.4
- REQUIREMENTS.md shows REQ-057..060 as Complete
- ROADMAP.md shows v0.8 as COMPLETE
- Gitea release `v0.7.4` published with milestone summary
---
## Phase 5: Final Review (next milestone, not part of v0.8 execution)
Per the v0.8 ROADMAP, there are 4 execution phases (P01..P04). P04 IS the
final review + ship + audit phase. There is no separate P05 in v0.8 (unlike
v0.7 which had P05). The orchestrator's next-milestone P0 begins after
T04.9 clears the checkpoint.
---
## Risk Register (carried forward from RESEARCH_v0.8.md §5)
| # | Pitfall | Phase(s) affected | Mitigation |
|---|---------|-------------------|------------|
| 1 | TOFU capture is currently BROKEN: `knownhosts.New` returns `KeyError{Want:[]}` on first connect and does NOT auto-write; current `BootstrapProxmox` treats it as a dial failure. | P02 | T02.6 wraps the callback to capture-and-persist on `KeyError{Want:[]}` via `knownhosts.Line` + `security.WriteAtomic`. This is a v0.6 latent bug that P02 closes. |
| 2 | `Result.HostKeyFingerprint` is declared but never populated (always `""`). D-045's rationale references "existing output" that doesn't exist. | P02 | T02.7 adds `ssh.FingerprintSHA256(hostKey)` computation in both the capture and pinned paths. 1-line addition once the host key is available. |
| 3 | No `sessionRunner` seam in proxmox — testing the SSH command sequence without a real SSH server is impossible. | P01 | T01.1 adds a 1-interface ~10-LOC `sessionRunner` seam in Wave 1. Unlocks ~40% of proxmox coverage. Backward compatible. |
| 4 | `internal/store/cert_repo.go` has NO test — v0.7 P01 REQ-053 was supposed to add `cert_repo_test.go` but it's missing (v0.7 leftover). | P01 | T01.7 adds `cert_repo_test.go` (Insert/Get/List/ListByNode/LatestForKind/PruneOlderThan/Delete + N=3 rotation). Directly lifts store coverage toward 70%. |
| 5 | `internal/cli/daemon.go` starts a long-running mTLS server — testing it in cli requires a lifecycle harness; it's already covered by `internal/daemon/server_test.go`. | P01 | T01.6 excludes `daemon.go` from the cli 70% target; documents the exclusion in a test-file comment. Avoids double-testing. |
| 6 | `cmd/orca` 50% toe-hold is low-value (15 LOC of glue; effort:coverage ratio is poor). | P01 | T01.11 keeps it at the 50% toe-hold per D-047; does NOT over-invest. A small `run() int` refactor enables a smoke test. |
| 7 | `go: no such tool "covdata"` for zero-test packages — a Go toolchain quirk when a package has no test files; NOT a real 0% number. | P01 | T01.9, T01.10, T01.11 each add a `_test.go` file, which makes coverage computable. Don't treat the tooling error as a measurement. |
| 8 | `transport.dispatchToPeer` has no seam — testing the remote-dispatch branch requires a new interface OR `httptest.NewTLSServer`. | P01 | T01.3 uses `httptest.NewTLSServer` (no refactor needed). T01.2 (conditional `peerDispatcher` seam) is only added if engine cannot otherwise hit 70%. |
| 9 | `knownhosts.Line` + `knownhosts.Normalize` are the helpers for the TOFU-capture fix and `key-reset` matching. | P02 | T02.6 + T02.8 use `Normalize` to match host strings consistently (handles `host:22` vs `host`). |
| 10 | `security.writeAtomic` is unexported (ca.go:305); `key-reset`'s atomic known_hosts rewrite needs it. | P02 | T02.2 exports `WriteAtomic` (recommended) OR copies the ~20-LOC pattern. Export is preferred — it's already used across ca.go + sshkey.go. |
---
## REQ-ID → Task mapping (traceability)
| REQ-ID | Phase | Tasks |
|--------|-------|-------|
| REQ-057 | P01 | T01.1, T01.2 (conditional), T01.3, T01.4, T01.5, T01.6, T01.7, T01.8, T01.9, T01.10, T01.11, T01.12 |
| REQ-058 | P02 | T02.1, T02.3, T02.4, T02.5, T02.7, T02.10, T02.11 |
| REQ-059 | P02 | T02.2, T02.8, T02.10 |
| REQ-060 | P03 | T03.1, T03.2, T03.3, T03.4, T03.5 |
| (latent TOFU bug) | P02 | T02.6, T02.9 (not a REQ — closes a v0.6 gap surfaced by RESEARCH §2.1) |
| (milestone release) | P04 | T04.1..T04.9 |
---
## Task counts
| Phase | Tasks | Must-haves | Waves |
|-------|-------|------------|-------|
| P01 | 12 | 11 (T01.2 conditional) | 3 |
| P02 | 11 | 11 | 3 |
| P03 | 5 | 5 | 3 |
| P04 | 9 | 9 | 3 |
| **Total** | **37** | **36** | — |
+46
View File
@@ -331,3 +331,49 @@ change. Milestone type: NFR (all phases are fix/test/chore); the final
phase's progressive patch IS the deliverable per `run.md` versioning
logic. Tags run on the v0.6.x patch line: `v0.6.0` (P0) … `v0.6.5` (P05
= milestone release).
## v0.8 Scope Summary — Coverage & Trust Hardening
v0.8 is a 3-execution-phase **NFR milestone** that continues the
hardening theme opened by v0.7. v0.7 P03 (REQ-055) lifted four
packages to ≥ 50%, but a coverage re-baseline after v0.7 ship shows
the floor was insufficient: `internal/engine` regressed to 8.3%,
`internal/proxmox` to 5.1%, and four more packages sit between 26% and
48%. Three packages (`internal/audit`, `internal/certpaths`,
`cmd/orca`) still have **no test files at all**. v0.8 also closes the
two "future enhancement" hooks explicitly deferred in v0.6 — SSH
host-key pre-pinning (D-035 caveat) and `orca node key-reset`
(RESEARCH_v0.6 §80) — and adds a requirements-hygiene gate so the
stale-REQ-status drift seen in REQUIREMENTS.md after v0.7 ship cannot
recur:
- **P01 — Coverage uplift round 2.** Raise six under-50% packages to
≥ 70% and add first tests for the three zero-test packages. Covers
REQ-057.
- **P02 — SSH trust hardening.** `--host-key-fingerprint` pre-pin flag
on `orca node join --type proxmox` + `orca node key-reset <node>`
command. Covers REQ-058, REQ-059.
- **P03 — Requirements-hygiene gate.** `make verify-reqs` target +
verify-stage assertion that ROADMAP `Complete` ↔ REQUIREMENTS
`Complete`. Covers REQ-060.
- **P04 — Final review + ship + audit.** Milestone release.
The vision is unchanged. v0.8 is a hardening milestone, not a
direction change. Milestone type: NFR (all phases are test/feat-chore
on the trust surface — see CLARIFY D-043 for the `feat` vs `chore`
classification of P02); the final phase's progressive patch IS the
deliverable per `run.md` versioning logic. Tags run on the **v0.7.x**
patch line: `v0.7.0` (P0) … `v0.7.4` (P04 = milestone release).
## v0.8 Clarified Decisions (D-series, full autonomy)
The 5 v0.8 decisions (D-043..D-047) were auto-resolved at full autonomy
within the `clarify_budget` (10):
| ID | Question | Decision | Rationale | Confidence |
|----|----------|----------|-----------|------------|
| D-043 | Is P02 (SSH trust hardening) a `feat` phase or a `chore` phase? It adds a new flag + a new subcommand. | **`chore` (trust-surface hardening), not `feat`** | Both `--host-key-fingerprint` and `orca node key-reset` refine the *existing* `orca node join --type proxmox` flow and the existing TOFU `known_hosts` store (D-035). No new orchestration capability, no new node kind, no new API. They close a security gap explicitly deferred in v0.6, not open new surface area. Per `run.md` versioning logic this keeps v0.8 NFR (all phases fix/test/chore/perf/refactor). | 0.84 |
| D-044 | Where does `--host-key-fingerprint` live — on `orca node join` or only on `--type proxmox`? | **On `orca node join` (root of the join subcommand), validated when `--type proxmox`** | The flag is generic (any future SSH-joined node kind will use it); gating it to `--type proxmox` only would require re-adding it later. Validation (`flag requires --type proxmox today`) happens in `RunE`, not in the flag declaration, so the flag is declared once on `node join` and the type check emits a clear error for non-proxmox types until other SSH-joined kinds exist. | 0.86 |
| D-045 | `--host-key-fingerprint` format — raw hex, `sha256:`-prefixed, or OpenSSH `SHA256:base64`? | **OpenSSH `SHA256:base64` (the format `ssh-keyscan -E sha256 -D -` emits and operators expect)** | Matches the fingerprint format operators already see from `ssh-keyscan` and `orca node join`'s own `Result.HostKeyFingerprint` output. Accept only `SHA256:`-prefixed base64; reject raw hex with a clear error. Internally decode base64 → compare against `ssh.PublicKey` Marshal + sha256. | 0.88 |
| D-046 | Does `orca node key-reset <node>` also revoke the orca pubkey on the remote host, or only clear the local `known_hosts` entry? | **Local `known_hosts` entry only** | Revoking the remote authorized_keys entry would orphan a working node (next dispatch would fail auth). `key-reset` is the local "forget this host's key" operation (mirrors `ssh-keygen -R host`); re-establishing trust is a separate `orca node join` re-run. Audit-log the reset with `actor`, `node`, `event=node.key_reset`. | 0.90 |
| D-047 | Coverage target for P01 — 70% floor or higher? | **70% floor for the 6 under-50% packages; 50% floor for the 3 zero-test packages (`internal/audit`, `internal/certpaths`, `cmd/orca`) as a first-toe-hold** | 70% across the board for the already-tested packages matches D-042's "70% target for new packages" and is achievable without heroic mock effort. For the zero-test packages, going 0→50% is the realistic single-phase step (0→70% risks a coverage rathole on `cmd/orca` which is glue code); a future milestone can lift them to 70%. | 0.82 |
+11 -2
View File
@@ -129,5 +129,14 @@ REQ-047..052 all complete.
|----|-------------|----------|-------|--------|
| REQ-053 | `orca cert` command tree registered on root command (`cert ca-init`, `cert gen`, `cert show`, `cert renew`, `cert fingerprint`) — code exists in `internal/cli/cert.go` but is never AddCommand'd; unreachable today | High | **v0.7 P1** | **Complete** (P1 shipped v0.6.1) |
| REQ-054 | HCL config file parsing: `internal/config` package loads `~/.orca/config.hcl` / `/etc/orca/orca.hcl` (keys: db_path, listen_addr, ca_path, server_cert_path, server_key_path, node_capacity); merge precedence flag > env > file > default; `--config` flag on root command | High | **v0.7 P2** | **Complete** (P2 shipped v0.6.2) |
| REQ-055 | Test coverage uplift: every package ≥ 50% — adds tests for `internal/engine` (executor, dispatcher, peer), `internal/transport` (mtls, dispatch, handshake_log), `internal/proxmox` (bootstrap SSH path), `internal/audit` | Medium | **v0.7 P3** | Pending |
| REQ-056 | `--pprof <addr>` opt-in flag on `orca daemon` (default disabled); `net/http/pprof` mounted on a separate mux, never on the mTLS daemon listener | Low | **v0.7 P4** | Pending |
| REQ-055 | Test coverage uplift: every package ≥ 50% — adds tests for `internal/engine` (executor, dispatcher, peer), `internal/transport` (mtls, dispatch, handshake_log), `internal/proxmox` (bootstrap SSH path), `internal/audit` | Medium | **v0.7 P3** | **Complete** (P3 shipped v0.6.3) |
| REQ-056 | `--pprof <addr>` opt-in flag on `orca daemon` (default disabled); `net/http/pprof` mounted on a separate mux, never on the mTLS daemon listener | Low | **v0.7 P4** | **Complete** (P4 shipped v0.6.4) |
## v0.8 Requirements — Coverage & Trust Hardening
| ID | Requirement | Priority | Phase | Status |
|----|-------------|----------|-------|--------|
| REQ-057 | Test coverage uplift round 2: raise `internal/engine` (8.3%), `internal/proxmox` (5.1%), `internal/cli` (27.6%), `internal/transport` (26.3%), `internal/store` (46.7%), `internal/jobspec` (47.6%) to ≥ 70%; add first tests for `internal/audit`, `internal/certpaths`, `cmd/orca` (currently 0%) to ≥ 50% (D-047 tiered floor) | High | **v0.8 P1** | Pending |
| REQ-058 | `--host-key-fingerprint <SHA256:base64>` pre-pin flag on `orca node join` (validated when `--type proxmox`): when supplied, join fails fast if the SSH host key's OpenSSH SHA-256 fingerprint does not match; supersedes TOFU (D-035) for pre-pinned deployments (D-044, D-045) | Medium | **v0.8 P2** | Pending |
| REQ-059 | `orca node key-reset <node>` command: clears the persisted SSH host key entry for the node from `~/.orca/known_hosts` only (local, not remote authorized_keys — D-046); audit-logs `event=node.key_reset`; next `doctor proxmox`/dispatch re-pins via TOFU or `--host-key-fingerprint` | Low | **v0.8 P2** | Pending |
| REQ-060 | Requirement-status hygiene sweep: REQUIREMENTS.md v0.7 rows were stale ("Pending" after ship); add a verify-stage assertion that every REQ listed as `Complete` in ROADMAP.md has a matching `Complete` row in REQUIREMENTS.md, enforced by `make verify-reqs` | Medium | **v0.8 P3** | Pending |
+285
View File
@@ -0,0 +1,285 @@
# Research: Orca v0.8 — Coverage & Trust Hardening
Findings grounded in codebase analysis (44 source/test files read, coverage
re-measured for all 9 target packages) + `golang.org/x/crypto` v0.54.0 API
verification (`ssh.FingerprintSHA256`, `knownhosts.Line`/`Normalize`/`KeyError`).
## 1. Coverage analysis (P01 — REQ-057)
### 1.1 Re-measured coverage (confirmed via `go test ./<pkg>/... -cover`)
| Package | Coverage | Tier (D-047) | Notes |
|---------|----------|--------------|-------|
| `internal/engine` | **8.3%** | ≥ 70% floor | Only `scheduler_test.go` (4 tests, 66 LOC); executor/dispatcher/peer/registry/audit untested |
| `internal/proxmox` | **5.1%** | ≥ 70% floor | Only `bootstrap_test.go` (4 tests, validation + sudoersContent string asserts); SSH dial path untested |
| `internal/cli` | **27.6%** | ≥ 70% floor | 5 test files (root, init, namespace, osdetect, watch); node/job/cert/doctor/audit/cmds untested |
| `internal/transport` | **26.3%** | ≥ 70% floor | Only `idempotency_test.go` (7 tests); mtls/dispatch/retry/handshake_log untested |
| `internal/store` | **47.2%** | ≥ 70% floor | node_repo + job_task + capacity + audit + migrate tested; **cert_repo has NO test** (REQ-053 leftover — v0.7 P01 was supposed to add it but it's missing) |
| `internal/jobspec` | **47.6%** | ≥ 70% floor | Only `spec_test.go` (4 tests); `Validate()`, `ParseFile` (file I/O), edge cases untested |
| `internal/audit` | **0%** (no test files) | ≥ 50% toe-hold | `go: no such tool "covdata"` is a known tooling gap, NOT a real number — the package simply has no `_test.go` |
| `internal/certpaths` | **0%** (no test files) | ≥ 50% toe-hold | Same `covdata` tooling gap; no `_test.go` exists |
| `cmd/orca` | **0%** (no test files) | ≥ 50% toe-hold | Same; `main.go` is 15 LOC of glue (`cli.Execute()` + error print) |
**Coverage-floor achievability assessment (per package):**
- **engine → 70% REALISTIC.** The package has clean seams: `LocalExecutor` interface (dispatcher.go:39), `PeerRegistry` is in-memory with `Add`/`Remove`/`All`/`Get` (peer.go), `Executor.Submit/Status` take a `*store.JobRepo`+`*store.TaskRepo` which can be backed by `:memory:`/temp-file sqlite via the existing `openTestDB` helper (node_repo_test.go:12). The `sshDialer` seam pattern (proxmox) has an analogue here: `transport.NewDispatchClient` is called inside `dispatchToPeer` (dispatcher.go:158) — to test dispatch-to-peer without a real mTLS server, either (a) inject a fake `DispatchClient` via a new interface seam, or (b) use `httptest.NewTLSServer` with a self-signed CA. Option (a) is lower-effort and aligns with the `LocalExecutor` pattern. Recommendation: extract a `peerDispatcher` interface (`Submit(ctx, spec, key) (*SubmitResponse, error)`) and inject it, OR test via `LocalSubmit`/`LocalStatus` paths (which only need a stubbed `LocalExecutor`) — the latter covers ~60% of dispatcher.go without a new seam. **Flag: 70% may require a small refactor to inject the dispatch client; 60-65% is achievable without one. Plan should decide whether to add the seam or accept 65%.**
- **proxmox → 70% REALISTIC.** The `sshDialer` seam already exists (bootstrap.go:201-213, `sshDialerType` interface + `defaultSSHDialer` struct, overridable package-level var). A fake SSH dialer returning a mock `*ssh.Client` is the path. **However:** `*ssh.Client.NewSession()` + `session.CombinedOutput()` are concrete methods on the real `*ssh.Client` — there's no `sshSession` interface seam. To test `runRemote`/`deployPubKey`/`createLinuxUser`/`createPVERole`/etc. without a real SSH server, EITHER (a) introduce a `sessionRunner` interface seam (small refactor), OR (b) use `httptest.NewTLSServer` is wrong (it's SSH not HTTP) — instead use a real in-process SSH server via `golang.org/x/crypto/ssh` `NewServerConn` (more code but no new dep). **Flag: 70% likely requires either a `sessionRunner` interface refactor OR an in-process SSH server fixture. 50-55% is achievable with just the existing `sshDialer` seam + testing validation paths + `sudoersContent` string asserts (already done). Plan should add the `sessionRunner` seam — it's a 1-interface, ~10-LOC change that unlocks the bulk of the package.**
- **cli → 70% AMBITIOUS but realistic.** The package is the largest (17 source files, ~2000 LOC). The existing tests use `rootCmd.SetArgs()` + `rootCmd.Execute()` + `t.TempDir()` + `ORCA_HOME` env (namespace_test.go:46-53 — `TestInitHonorsORCAHOME` is the template). The untested commands are `node join/leave/list`, `job run/list/stop/logs`, `cert *`, `doctor *`, `audit list`, `status`, `version`, `daemon`. Many touch the DB + certpaths + (for `node join --type proxmox`) the SSH dialer. **Strategy:** table-driven `rootCmd.Execute()` against a temp `ORCA_HOME` for each subcommand; mock the proxmox path via the existing `sshDialer` seam; capture stdout via `rootCmd.SetOut(&buf)`. **Flag: 70% across the whole package is a lot of test code; 55-65% is more realistic for one phase. The `daemon` command (background server) is hard to test without a lifecycle harness — recommend excluding it from the 70% target and documenting why.**
- **transport → 70% REALISTIC.** `httptest.NewTLSServer` is the standard seam (already used in `internal/daemon/dispatch_test.go:59` and `server_test.go`). The `Dispatcher` interface (dispatch.go:49) is already mockable (`stubDispatcher` in dispatch_test.go:24 is the template). `MTLSClient.Do` wraps `http.Client.Do` — testable via `httptest.NewTLSServer` with a CA + client cert. `retry.go` `Do[T]` is generic + already partly tested via `idempotency_test.go` (TestRetrySucceedsAfterTransient etc.) — extend with backoff-timing asserts. `handshake_log.go` is pure slog calls — trivial to test by capturing into a `slog.Handler`. **No new seams needed; 70% achievable.**
- **store → 70% REALISTIC.** The existing `openTestDB` helper (node_repo_test.go:12) + `withFastWatch` (job_task_repo_test.go:36) are reusable. **Critical gap:** `cert_repo.go` has NO test file despite v0.7 P01 REQ-053 claiming it was added — this is a v0.7 leftover bug. Adding `cert_repo_test.go` (Insert/Get/List/ListByNode/LatestForKind/PruneOlderThan/Delete + N=3 rotation history per REQ-025) alone lifts coverage significantly. Job/Task repo `Watch` is tested; `ListRecent`, error paths, scan-edge cases need coverage. **No new seams; 70% achievable.**
- **jobspec → 70% REALISTIC.** `Parse` + `Validate` + `ParseFile` are pure functions over HCL bytes. Add golden-file HCL fixtures (multi-task, env vars, args) + error-path table (missing job, no tasks, missing command, malformed HCL, empty file, nonexistent file for `ParseFile`). `testdata/` dir doesn't exist yet — create it. **No new seams; 70% achievable, likely the easiest of the six.**
- **audit → 50% toe-hold REALISTIC.** Package is 125 LOC, 4 exported funcs (`New`, `Emit`, `EmitWithErr`, `LogHandshakeOK`, `LogHandshakeFailed`, `FormatAction`, `Action.String`, `Result.String`). Strategy: construct `Audit` with a real `engine.Audit` backed by `:memory:` sqlite (via `store.NewAuditRepo` + `engine.NewAudit`) + assert rows in `audit_log` table; capture slog output via a test `slog.Handler`. **No new seams; 50% easily achievable, 70% achievable if desired.**
- **certpaths → 50% toe-hold TRIVIAL.** Package is 64 LOC, pure path-join functions honoring `ORCA_HOME`/`ORCA_DB` env. Strategy: temp dir + `t.Setenv("ORCA_HOME", dir)` + assert each `*Path()` returns `filepath.Join(dir, <file>)`; test `ORCA_DB` override; test default-to-`~/.orca` fallback. Model the test on `namespace_test.go` (cli). **No new seams; 50%+ trivially achievable.**
- **cmd/orca → 50% toe-hold REALISTIC but LOW VALUE.** `main.go` is 15 LOC: `cli.Execute()` + `fmt.Fprintf(os.Stderr, "error: %v")` + `os.Exit(1)`. The only testable behavior is "main() calls Execute and exits non-zero on error." A smoke test that calls `main()` in a subprocess (or refactors main into a `run() int` for testability) is the path. **Flag: 50% on a 15-LOC glue file is ~7 lines of covered code — the effort:coverage ratio is poor. D-047 explicitly called this out ("0→70% risks a coverage rathole on `cmd/orca` which is glue code"). Recommend the plan keep this at the 50% toe-hold and not over-invest.**
### 1.2 Existing test-helper utilities (reuse, do NOT re-create)
| Helper | Location | Reuse for |
|--------|----------|-----------|
| `openTestDB(t)` | `internal/store/node_repo_test.go:12` | engine, audit, store tests — returns `(*NodeRepo, func())` backed by temp-file sqlite; adapt to return `*sql.DB` for JobRepo/TaskRepo/AuditRepo/CapacityRepo/CertRepo |
| `withFastWatch(t, d)` | `internal/store/job_task_repo_test.go:36` | store Watch tests — overrides `watchInterval` for deterministic ticks |
| `initTestEnv(t)` | `internal/cli/init_test.go:17` | cli tests — sets `ORCA_HOME` to temp dir + returns cleanup |
| `resetRootFlags(t)` | `internal/cli/namespace_test.go:13` | cli tests — resets `rootCmd` args/out/json/system flags between subtests |
| `discardWriter` | `internal/cli/init_test.go:33` | cli tests — `io.Writer` that discards stdout |
| `stubDispatcher` | `internal/daemon/dispatch_test.go:24` | transport/engine tests — implements `transport.Dispatcher` (`LocalSubmit`/`LocalStatus`); reusable as a `LocalExecutor` too since the signatures match |
| `insertNode(t, repo, ctx, id, name)` | `internal/store/node_repo_test.go:217` | store/doctor tests — inserts a minimal node |
| `security.CAInit`/`LoadCA`/`GenerateCSR`/`SignCSR`/`WriteCert`/`WriteKey` | `internal/security/ca.go` | transport mTLS tests — bootstrap a real CA + server cert into a temp dir (pattern in `doctor_test.go:69-94`) |
| `t.Setenv("ORCA_HOME", dir)` + `t.Setenv("ORCA_DB", ...)` | `internal/doctor/doctor_test.go:23-24` | any test needing the orca namespace — preferred over manual `os.Setenv` (auto-cleanup) |
### 1.3 Injected seams already present in the codebase (confirm by reading)
1. **`sshDialer` (proxmox)** — `internal/proxmox/bootstrap.go:201-213`: package-level `var sshDialer sshDialerType = defaultSSHDialer{}`; interface `sshDialerType{ DialContext(ctx, network, addr, *ssh.ClientConfig) (*ssh.Client, error) }`. Tests can swap `sshDialer` for a fake. **GAP:** no `sessionRunner` seam — `runRemote` (line 217) calls `conn.NewSession()` + `session.CombinedOutput(cmd)` directly on the concrete `*ssh.Client`. Recommend P01 plan add a `sessionRunner` interface (`CombinedOutput(cmd) ([]byte, error)`) so `deployPubKey`/`createLinuxUser`/`createPVERole`/`createPVEUser`/`assignPVEACL`/`writeSudoers`/`validateSudoers` become testable without a real SSH endpoint.
2. **`LocalExecutor` (engine dispatcher)** — `internal/engine/dispatcher.go:39`: interface `Submit(ctx, []byte) (string, error)` + `Status(ctx, string) (string, error)`. `Dispatcher` depends on it; tests inject a stub. **GAP:** `dispatchToPeer` (line 154) calls `transport.NewDispatchClient` directly (no seam) — to test the remote-dispatch branch, either add a `peerDispatcher` interface or test via `httptest.NewTLSServer`.
3. **`PeerPersister` (engine peer)** — `internal/engine/peer.go:39`: optional persist callback; unused in production but available as a seam.
4. **`Dispatcher` (transport)** — `internal/transport/dispatch.go:49`: `LocalSubmit`/`LocalStatus` interface; `stubDispatcher` in `daemon/dispatch_test.go:24` is the template stub.
5. **`watchInterval` (store)** — `internal/store/job_task_repo.go:20`: unexported `var watchInterval = 1 * time.Second`; tests override via `withFastWatch`.
### 1.4 Packages where 70% is unrealistic in a single phase (with evidence)
- **`internal/cli` — 70% is ambitious.** 17 source files, ~2000 LOC. The `daemon` command (`internal/cli/daemon.go`) starts a long-running mTLS server — testing it requires a lifecycle harness (start, probe, shutdown) and is better covered by `internal/daemon/server_test.go` (already exists, 150 LOC). Recommend the P01 plan **exclude `daemon.go` from the cli 70% target** (document it as covered by the daemon package's own tests) and aim for 70% of the *remaining* cli files. Even so, 55-65% is the realistic single-phase outcome for the rest.
- **`cmd/orca` — 70% is explicitly out of scope per D-047.** 15 LOC of glue; 50% toe-hold is the right call.
- **`internal/proxmox` — 70% likely requires the `sessionRunner` seam refactor.** Without it, only the validation paths + `sudoersContent` string asserts are testable (~50-55%). The plan should add the seam; with it, 70% is achievable.
---
## 2. SSH trust hardening research (P02 — REQ-058, REQ-059)
### 2.1 Current TOFU `knownhosts.New()` callback — how it works
**Location:** `internal/proxmox/bootstrap.go:125-128` (bootstrap) + `internal/doctor/doctor.go:412-415` (doctor proxmox probe).
```go
hostKeyCallback, err := knownhosts.New(certpaths.KnownHostsPath())
// ...
sshConfig := &ssh.ClientConfig{
HostKeyCallback: hostKeyCallback,
// ...
}
```
**Mechanism (`golang.org/x/crypto/ssh/knownhosts`):**
- `knownhosts.New(files ...string)` returns an `ssh.HostKeyCallback` that reads the OpenSSH-format `known_hosts` file at `certpaths.KnownHostsPath()` (= `$ORCA_HOME/known_hosts`, see `internal/certpaths/certpaths.go:62`).
- **First connect (host absent from file):** the callback returns a `*knownhosts.KeyError` with `Want: []` (empty). This is a "host unknown" signal. **IMPORTANT:** `knownhosts.New` does NOT auto-write the key on first connect — it returns an error. The current orca code at `bootstrap.go:140` treats ANY dial error as a failure (`return nil, fmt.Errorf("ssh dial %s: %w", sshAddr, err)`). **This means the current TOFU flow is INCOMPLETE:** on a truly first connect, `knownhosts.New` returns `KeyError{Want:[]}` and the dial fails — there is no capture-and-persist step. The v0.6 RESEARCH_v0.6.md §A.5 claimed `knownhosts.New` "handles both capture and verify in one callback" but the actual `golang.org/x/crypto` API does NOT auto-capture; it only verifies. **This is a latent bug OR the operator is expected to pre-populate `known_hosts` manually (which contradicts the TOFU UX).** P02 must address this: either (a) wrap `knownhosts.New` with a custom callback that captures on `KeyError{Want:[]}` and writes via `knownhosts.Line`, or (b) accept that `--host-key-fingerprint` (REQ-058) becomes the *required* path for first connect and TOFU capture is a separate enhancement. **Flag for plan: the current TOFU capture is broken; P02 should fix it as part of the trust-hardening work (the `--host-key-fingerprint` path is actually simpler than TOFU because it doesn't need capture).**
- **Subsequent connects (host present, key matches):** callback returns `nil` → dial proceeds.
- **Subsequent connects (host present, key MISMATCH):** callback returns `*knownhosts.KeyError{Want: [knownKey]}` → dial fails with a clear error. This is the MITM-detection path.
**File format:** OpenSSH `known_hosts` — one line per host: `[host]:port ssh-key-type base64-key` (or hashed-host form via `knownhosts.HashHostname`). `knownhosts.Line(addresses []string, key ssh.PublicKey) string` produces the line; `knownhosts.Normalize(address)` normalizes the host:port.
### 2.2 `Result.HostKeyFingerprint` — current computation (CRITICAL FINDING)
**Location:** `internal/proxmox/bootstrap.go:83-85` (field declaration) + `bootstrap.go:195-198` (return statement).
```go
type Result struct {
NodeName string
NodeAddress string
HostKeyFingerprint string // field EXISTS
}
// ...
return &Result{
NodeName: opts.Host,
NodeAddress: opts.Host + ":8443",
// HostKeyFingerprint is NOT SET — always empty string
}, nil
```
**Finding:** `Result.HostKeyFingerprint` is **declared but never populated**. The current `BootstrapProxmox` returns it as `""`. There is **no fingerprint computation today** — no `ssh.FingerprintSHA256` call, no hex digest, nothing. D-045's rationale ("matches the fingerprint format operators already see from `orca node join`'s own `Result.HostKeyFingerprint` output") is based on a field that is currently always empty.
**Implication for P02:** The plan must ADD the fingerprint computation. The correct function is `ssh.FingerprintSHA256(pubKey ssh.PublicKey) string` (verified via `go doc`), which returns the **OpenSSH `SHA256:base64` format** (unpadded base64, exactly what `ssh-keyscan -E sha256` emits and what D-045 specifies). So D-045's format choice is correct *by intent* but the code doesn't produce it yet — P02 populates `Result.HostKeyFingerprint = ssh.FingerprintSHA256(hostKey)` during the capture path, and `--host-key-fingerprint` compares against `ssh.FingerprintSHA256` of the server-presented key.
**No existing fingerprint-comparison utility in `internal/security/`.** `security.Fingerprint` (fingerprint.go:17) computes SHA-256 **hex** of an X.509 cert's DER — a DIFFERENT format (hex, not base64; X.509, not SSH). `security.FingerprintOf` (fingerprint.go:34) is the same. **Do NOT reuse these for SSH host-key comparison** — they're for the mTLS CA pin (`--ca-fingerprint`). P02 needs a new SSH-specific helper, e.g. `security.SSHFingerprintSHA256(pubKey ssh.PublicKey) string` (thin wrapper over `ssh.FingerprintSHA256`) or inline in `proxmox/bootstrap.go`.
### 2.3 Where `--host-key-fingerprint` plugs in (REQ-058)
**CLI seam:** `internal/cli/node.go:344-354` — the `init()` registers flags on `nodeJoinCmd`. Add:
```go
nodeJoinCmd.Flags().StringVar(&joinHostKeyFP, "host-key-fingerprint", "", "SSH host key SHA256:base64 fingerprint (pre-pin; supersedes TOFU for --type proxmox)")
```
Per D-044, the flag lives on `orca node join` (not just `--type proxmox`); validation in `RunE` (`node.go:78-83`) emits a clear error if the flag is set for a non-proxmox type.
**Transport seam:** `internal/proxmox/bootstrap.go:131-136``ssh.ClientConfig.HostKeyCallback`. Currently `knownhosts.New(...)`. When `--host-key-fingerprint` is supplied, replace the callback with a `ssh.FixedHostKey`-style verifier that:
1. Parses the operator-supplied `SHA256:base64` string (strip `SHA256:` prefix, base64-decode → 32 bytes).
2. In the callback, receives the server's `ssh.PublicKey`, computes `ssh.FingerprintSHA256(key)`, compares to the operator string.
3. Returns `nil` on match, `error` on mismatch (fail closed).
**Recommended callback shape (concrete):**
```go
func pinnedHostKeyCallback(expectedSHA256Base64 string) (ssh.HostKeyCallback, error) {
// Validate format: must start with "SHA256:".
if !strings.HasPrefix(expectedSHA256Base64, "SHA256:") {
return nil, fmt.Errorf("host-key-fingerprint: must be OpenSSH SHA256:base64 format (got %q)", expectedSHA256Base64)
}
expected := expectedSHA256Base64 // store full string for direct compare
return func(_ string, _ net.Addr, key ssh.PublicKey) error {
got := ssh.FingerprintSHA256(key)
if got != expected {
return fmt.Errorf("host key fingerprint mismatch: got %s, want %s — refusing to connect (REQ-058)", got, expected)
}
return nil
}, nil
}
```
**Why compare full strings (not base64-decoded bytes):** `ssh.FingerprintSHA256` returns the canonical `SHA256:base64` string; comparing it directly to the operator-supplied string is simplest and avoids a base64-decode step. Reject non-`SHA256:`-prefixed input up front with a clear error (D-045: "Accept only `SHA256:`-prefixed base64; reject raw hex with a clear error").
**Pass-through to proxmox:** `internal/cli/node.go:158-166` — add `HostKeyFingerprint string` to `proxmox.Options` (bootstrap.go:55) and pass `joinHostKeyFP` through. `BootstrapProxmox` selects the callback: if `opts.HostKeyFingerprint != ""` use `pinnedHostKeyCallback`, else fall back to the TOFU `knownhosts.New` (with the capture-fix from §2.1).
### 2.4 `orca node key-reset <node>` (REQ-059, D-046 — local known_hosts only)
**Scope (D-046):** clear the local `~/.orca/known_hosts` entry for the node ONLY; do NOT revoke the remote authorized_keys entry (would orphan a working node). Audit-log `event=node.key_reset` with `actor` + `node`.
**`known_hosts` line format written by `golang.org/x/crypto/ssh/knownhosts`:**
- `knownhosts.Line(addresses []string, key ssh.PublicKey) string``"[host]:port ssh-ed25519 AAAA...\n"` (or `host ssh-ed25519 AAAA...` if port 22 — `knownhosts.Normalize` handles the `:22` vs bare-host normalization).
- The file is plain text, one entry per line, `#`-prefixed comments allowed.
**No library function to remove a host's entries.** `knownhosts.New` only reads. The reset must be implemented manually:
1. Read `certpaths.KnownHostsPath()` (`internal/certpaths/certpaths.go:62`).
2. Filter lines: keep lines whose host field (before the first whitespace) does NOT match `knownhosts.Normalize(nodeName)` (or the node's address). **Edge:** a host may have multiple entries (one per key type); remove all matching lines.
3. Write the filtered content back via **atomic rewrite** (temp file in same dir + `os.Rename`) — reuse `security.writeAtomic` (ca.go:305) OR implement inline (it's unexported in `security`; either export it or copy the ~20-LOC pattern). **Recommend atomic rewrite, NOT in-place truncation** — in-place rewrite via `os.OpenFile(O_TRUNC|O_WRONLY)` risks data loss on crash mid-write.
**CLI registration seam:** `internal/cli/node.go:358-360` — the `init()` does `nodeCmd.AddCommand(nodeJoinCmd)`, `nodeLeaveCmd`, `nodeListCmd`. Add:
```go
nodeCmd.AddCommand(nodeKeyResetCmd)
```
where `nodeKeyResetCmd` is a new `&cobra.Command{Use: "key-reset <node>", Args: cobra.ExactArgs(1), RunE: ...}`. The `RunE`:
1. Resolve `<node>` arg → look up the node in the registry (`nodeRegistry()` at node.go:37) to get its address (for matching `known_hosts` lines) — OR accept the raw host string directly. **Recommend:** accept the node NAME (consistent with `doctor proxmox` which iterates `node.Name`), look up the node row, use `node.Name` (which is the host address for proxmox nodes per `bootstrap.go:196`) as the `known_hosts` match key.
2. Call a new `proxmox.ResetHostKey(host string) error` (or inline in cli) that does the atomic rewrite.
3. Audit-log via `engine.Audit.Record(ctx, "cli", "node.key_reset", nodeID, "success", nil, map[string]any{"host": host})`.
4. Print `✓ Host key reset for <node> (next connect will re-pin via TOFU or --host-key-fingerprint)`.
**Reusability:** the `nodeRegistry()` helper (node.go:37) + `openDB()` (node.go:25) + `newLogger()` (node.go:33) are all available for the key-reset command.
### 2.5 CLI registration seam summary (P02)
| Addition | File:line | Change |
|----------|-----------|--------|
| `--host-key-fingerprint` flag | `internal/cli/node.go:344-354` (init) | `nodeJoinCmd.Flags().StringVar(&joinHostKeyFP, "host-key-fingerprint", "", "...")` |
| `joinHostKeyFP` var | `internal/cli/node.go:47-60` (var block) | add `joinHostKeyFP string` |
| Pass-through to proxmox | `internal/cli/node.go:158-166` (joinProxmox) | add `HostKeyFingerprint: joinHostKeyFP` to `proxmox.Options` |
| `HostKeyFingerprint` field | `internal/proxmox/bootstrap.go:55` (Options) | add field |
| Pinned callback | `internal/proxmox/bootstrap.go:131-136` | branch: if `opts.HostKeyFingerprint != ""` use pinned callback else TOFU |
| Populate `Result.HostKeyFingerprint` | `internal/proxmox/bootstrap.go:195-198` | set `HostKeyFingerprint: ssh.FingerprintSHA256(hostKey)` during capture |
| `key-reset` subcommand | `internal/cli/node.go:358-360` (init) | `nodeCmd.AddCommand(nodeKeyResetCmd)` + new cmd var |
| `ResetHostKey` helper | `internal/proxmox/bootstrap.go` (new) OR `internal/security/sshkey.go` | atomic known_hosts rewrite |
---
## 3. Requirements-hygiene gate research (P03 — REQ-060)
### 3.1 Current Makefile targets
`Makefile` has 11 targets: `build`, `test`, `test-race`, `lint`, `fmt`, `clean`, `run`, `version`, `changelog`, `release`, `security-scan` (Makefile:1-100). **No `verify-reqs` target exists.** The `.PHONY` list at line 1 must be extended.
### 3.2 Current `.coreci.yml` pipeline structure
4 pipelines (`.coreci.yml:19-134`):
- **validate** (line 20): 4 steps — `go-version` (gofmt+vet), `gosec`, `govulncheck`, `gitleaks`.
- **build** (line 53): 1 step — version-injected `go build`.
- **test** (line 72): 1 step — `go test -race -coverprofile=coverage.out ./...` + `go tool cover -func | tail -1`.
- **release** (line 81): gated on `refs/tags/v*`; 3 steps — build-artifact, gitea-release, container-publish.
**Hook for `verify-reqs`:** add a 5th step to the `validate` pipeline (after `go-version`, before/after `gosec`) OR add it to the `test` pipeline. **Recommend `validate` pipeline** — requirements hygiene is a static check (no test run needed), belongs alongside gofmt/vet/lint. Step shape:
```yaml
- name: verify-reqs
image: golang:1.25
commands:
- make verify-reqs
```
### 3.3 `verify-reqs` implementation recommendation
**Assertion (REQ-060):** every REQ row in `ROADMAP.md` marked `[x]`/Complete must have a matching REQ-ID row in `REQUIREMENTS.md` with `Complete` status. (Reverse direction — every REQUIREMENTS `Complete` has a ROADMAP `[x]` — is also worth checking but the drift that motivated this was ROADMAP-shipped-but-REQUIREMENTS-Pending, so the forward direction is the priority.)
**Approach: small Go program in `cmd/verify-reqs` OR a shell+awk script?**
- **Go program** (~80 LOC): parse both markdown tables with `regexp`, build two `map[string]string` (REQ-ID → status), diff. Pros: type-safe, testable, consistent with the Go toolchain; can be a `cmd/verify-reqs/main.go` with its own `_test.go`. Cons: adds a binary target.
- **Shell+awk** (~30 LOC): `awk` over the markdown tables. Pros: no new Go package; minimal. Cons: fragile parsing, hard to test, shell-quoting issues.
**Recommendation: Go program at `cmd/verify-reqs/main.go`.** Reasons: (1) testable with golden-file fixtures (parse a sample ROADMAP+REQUIREMENTS pair, assert diff); (2) consistent with the project's Go-only tooling ethos (no shell-awk fragility); (3) the `make verify-reqs` target just calls `go run ./cmd/verify-reqs`; (4) CoreCI's `golang:1.25` image has `go` available — no extra dep.
**Parsing approach (concrete):**
1. ROADMAP.md: regex `^\s*-\s*\[(x|X| )\]\s*Phase.*—.*tag` is NOT the right pattern (that's phase lines, not REQ rows). The REQ coverage is in per-phase bullet lists under "### Per-phase REQ coverage" (ROADMAP.md:161-180) AND in the milestone section bodies. **Simpler:** the ROADMAP uses `- [x] Phase N: ...` for completed phases. The authoritative REQ↔status mapping lives in **REQUIREMENTS.md** (the single table at lines 9-56 + per-milestone tables at 103-142). **Re-interpret REQ-060:** the assertion is really "ROADMAP milestone sections marked COMPLETE ↔ REQUIREMENTS rows for that milestone marked Complete." The drift was: v0.7 ROADMAP said "COMPLETE" (line 116) but REQUIREMENTS v0.7 rows (REQ-053..056) were "Pending" (now corrected to "Complete" in SPECIFY).
2. **Refined assertion:** parse REQUIREMENTS.md table rows (`| REQ-XXX | ... | ... | ... | **Complete** |` or `| Pending |`); for each REQ-ID, record status. Then parse ROADMAP.md for milestone-level "COMPLETE" markers (`## Milestone v0.X: ... — **COMPLETE**`) AND phase-level `- [x]` markers. For each milestone marked COMPLETE in ROADMAP, assert every REQ-ID belonging to that milestone (per the REQUIREMENTS milestone column) is `Complete` in REQUIREMENTS. **OR (simpler, matches the SPECIFY wording):** for every REQ-ID in REQUIREMENTS.md whose `Phase` column references a milestone that ROADMAP marks COMPLETE, the Status must be `Complete`. This catches the exact drift (ROADMAP-shipped, REQUIREMENTS-stale).
**Concrete regex:**
- REQUIREMENTS row: `^\|\s*(REQ-\d+)\s*\|.*?\|\s*\*\*(Complete|Pending)\*\*\s*\|` (capture ID + status).
- ROADMAP milestone-complete: `^##\s*Milestone\s+v0\.\d+:.*—\s*\*\*COMPLETE\*\*` (capture milestone label).
- Map milestone → REQs via the REQUIREMENTS `Phase` column (e.g. `v0.7 P1` → milestone `v0.7`).
**Where it hooks in:** `make verify-reqs` runs `go run ./cmd/verify-reqs .ciagent/ROADMAP.md .ciagent/REQUIREMENTS.md`; `.coreci.yml` validate pipeline adds the step. Exit 0 on consistency, exit 1 with a diff listing on drift.
### 3.4 The drift that motivated REQ-060
After v0.7 ship, REQUIREMENTS.md rows REQ-053..056 were "Pending" despite ROADMAP.md marking milestone v0.7 COMPLETE and all phases `[x]`. This was corrected during v0.8 SPECIFY (the rows now read `**Complete**`). REQ-060 ensures the drift cannot recur: the CI validate pipeline fails if ROADMAP says COMPLETE but REQUIREMENTS says Pending.
---
## 4. Architectural decisions surfaced (AD-027..AD-030)
| ID | Decision | Rationale |
|----|----------|-----------|
| AD-027 | `ssh.FingerprintSHA256` (OpenSSH `SHA256:base64`) as the SSH host-key fingerprint format | Matches D-045 + `ssh-keyscan -E sha256` output. The existing `security.Fingerprint` (hex, X.509) is NOT reused — different domain. P02 adds a thin SSH-specific helper. |
| AD-028 | `--host-key-fingerprint` callback compares full `SHA256:base64` strings, not decoded bytes | `ssh.FingerprintSHA256` returns the canonical string; direct string compare avoids a base64-decode step and is less error-prone. Validate `SHA256:` prefix up front. |
| AD-029 | `orca node key-reset` rewrites `known_hosts` via atomic temp-file + rename | Prevents data loss on crash mid-write. Reuse the `writeAtomic` pattern from `security/ca.go:305` (export it or copy the ~20 LOC). |
| AD-030 | `verify-reqs` implemented as `cmd/verify-reqs/main.go` (Go program), not shell+awk | Testable, type-safe, consistent with Go-only tooling. `make verify-reqs` runs `go run ./cmd/verify-reqs`. Hooked into `.coreci.yml` validate pipeline. |
---
## 5. Pitfalls, gaps, and flags for the plan
1. **TOFU capture is currently BROKEN (§2.1).** `knownhosts.New` returns `KeyError{Want:[]}` on first connect and does NOT auto-write the key. The current `BootstrapProxmox` treats this as a dial failure. P02 must either (a) wrap the callback to capture-and-persist on `KeyError{Want:[]}` via `knownhosts.Line` + atomic write, or (b) make `--host-key-fingerprint` the required first-connect path. **Recommend (a) — fix TOFU + add pre-pin as superset.** This is a v0.6 latent bug that P02 closes.
2. **`Result.HostKeyFingerprint` is never populated (§2.2).** D-045's rationale references "existing output" that doesn't exist. P02 must ADD the computation (`ssh.FingerprintSHA256`). Low risk — it's a 1-line addition once the host key is available.
3. **No `sessionRunner` seam in proxmox (§1.3).** Testing the SSH command sequence (deployPubKey, createLinuxUser, pveum, sudoers, visudo) without a real SSH server requires a new interface seam. **Recommend P01 plan add it** — 1 interface, ~10 LOC, unlocks ~40% of proxmox coverage.
4. **`internal/store/cert_repo.go` has NO test (§1.1).** v0.7 P01 REQ-053 was supposed to add `cert_repo_test.go` but it's missing — `internal/store/` glob shows no `cert_repo_test.go`. This is a v0.7 leftover. P01 should add it (it directly lifts store coverage toward 70%).
5. **`internal/cli/daemon.go` excluded from cli 70% target (§1.4).** The daemon command starts a long-running server; it's covered by `internal/daemon/server_test.go` (150 LOC). Don't double-test in cli.
6. **`cmd/orca` 50% toe-hold is low-value (§1.1).** 15 LOC of glue; the test effort:coverage ratio is poor. D-047 already called this out. Don't over-invest.
7. **`go: no such tool "covdata"` for zero-test packages (§1.1).** This is a Go toolchain quirk when a package has no test files — `go test -cover` can't compute coverage without a test binary. It's NOT a real 0% number (it's "undefined"). Adding any `_test.go` file makes the number computable. Don't treat the error as a coverage measurement.
8. **`transport.dispatchToPeer` has no seam (§1.3).** Testing the remote-dispatch branch of `Dispatcher.Submit` requires either a new `peerDispatcher` interface OR `httptest.NewTLSServer`. The latter is already used in `daemon/dispatch_test.go`; recommend the plan use `httptest.NewTLSServer` (no refactor needed) for transport coverage.
9. **`knownhosts.Line` + `knownhosts.Normalize` are the helpers for the TOFU-capture fix and for `key-reset` matching (§2.1, §2.4).** Use `Normalize` to match host strings consistently (handles `host:22` vs `host`).
10. **`security.writeAtomic` is unexported (ca.go:305).** `key-reset`'s atomic known_hosts rewrite needs it. Either export `WriteAtomic` from `security`, or copy the ~20-LOC pattern into `proxmox`/`cli`. **Recommend export** — it's already used across ca.go + sshkey.go and is generally useful.
---
## 6. Dependencies
v0.8 adds **zero** new direct dependencies:
- SSH host-key fingerprint: `ssh.FingerprintSHA256` (already in `golang.org/x/crypto/ssh` v0.54.0, direct dep since v0.6).
- `knownhosts.Line`/`Normalize`/`KeyError`: same `golang.org/x/crypto` module.
- `verify-reqs`: stdlib only (`regexp`, `os`, `fmt`).
- Tests: `net/http/httptest` (stdlib), existing interfaces.
`go.mod` is unchanged by v0.8.
---
## 7. PERSONAS assessment (v0.8)
v0.8 is an NFR milestone touching tests (9 packages), SSH trust surface (proxmox + cli/node + security), and a requirements-hygiene Go program. The 3-persona roster from config.json (lead-developer, backend-engineer, data-engineer) is sufficient — no phase-specific personas needed.
**Roster confirmation:**
- **lead-developer** — owns coordination + `cmd/orca` smoke test + `internal/cli` coverage (cert/doctor/audit/status/version subcommands) + the `verify-reqs` Go program (coordination territory).
- **backend-engineer** — owns `internal/transport` tests (httptest.NewTLSServer) + `internal/engine` tests (LocalExecutor stubs, PeerRegistry) + SSH trust-surface in `internal/proxmox/bootstrap.go` (pinned callback, TOFU capture fix, sessionRunner seam) + `internal/cli/node.go` (`--host-key-fingerprint` flag, `key-reset` subcommand).
- **data-engineer** — owns `internal/store` tests (cert_repo_test.go gap + coverage uplift) + `internal/audit` tests (sqlite-backed audit_log asserts) + `internal/certpaths` tests (path-join asserts) + `internal/jobspec` tests (golden HCL fixtures).
No frontend persona (no UI). No devops persona (no packaging/distribution — `verify-reqs` is a Go program, not a CI config change; the `.coreci.yml` edit is a 3-line hook, lead-developer territory). No security-engineer persona (the SSH trust work is backend-engineer territory — the security-engineer was deactivated in v0.7 and v0.8 doesn't re-add it; the trust-surface hardening is a refinement of the existing `proxmox` package, not new security architecture).
See `.ciagent/PERSONAS.md` (updated with v0.8 YAML frontmatter + territory globs matching the actual file structure).
+56 -4
View File
@@ -113,7 +113,7 @@ branch-strategy.md. The milestone branch label uses the milestone
number (`milestone/v0.6-node-bootstrap-proxmox`); no separate minor
tag is created.
## Milestone v0.7: Hardening & Completion
## Milestone v0.7: Hardening & Completion — **COMPLETE**
Scope: NFR milestone closing gaps surfaced by the v0.7 IDEATE stage —
an unreachable command tree, a missing config file layer, low test
@@ -122,9 +122,9 @@ coverage in core packages, and the long-deferred pprof endpoint.
- [x] Phase 0: Pre-execution (specify → clarify → research → ideate → plan) — tag `v0.6.0` (shipped)
- [x] Phase 1: Register `orca cert` command tree + cert_repo tests (REQ-053) — tag `v0.6.1` (shipped)
- [x] Phase 2: HCL config file parsing — `internal/config` package (REQ-054) — tag `v0.6.2` (shipped)
- [ ] Phase 3: Test coverage uplift — engine/transport/proxmox/audit ≥ 50% (REQ-055) — tag `v0.6.3`
- [ ] Phase 4: `--pprof` opt-in on `orca daemon` (REQ-056) — tag `v0.6.4`
- [ ] Phase 5: Final review + ship + audit (milestone release) — tag `v0.6.5`
- [x] Phase 3: Test coverage uplift — engine/transport/proxmox/audit ≥ 50% (REQ-055) — tag `v0.6.3` (shipped)
- [x] Phase 4: `--pprof` opt-in on `orca daemon` (REQ-056) — tag `v0.6.4` (shipped)
- [x] Phase 5: Final review + ship + audit (milestone release) — tag `v0.6.5` (shipped)
**Milestone type**: NFR (all phases are fix/test/chore; no `feat` phases).
**Milestone tag**: `v0.6.5` (final phase patch = milestone release per
@@ -132,3 +132,55 @@ NFR-milestone progressive-patch rule). Per-phase tags: `v0.6.0`…`v0.6.5`.
Tags run on the previous minor's patch line (v0.6.x) per
branch-strategy.md. The milestone branch label uses the milestone
number (`milestone/v0.7-hardening-completion`); no separate minor tag.
## Milestone v0.8: Coverage & Trust Hardening
Scope: continue the v0.7 hardening theme. v0.7 P03's ≥ 50% floor left
six packages still under 50% (engine 8.3%, proxmox 5.1%, cli 27.6%,
transport 26.3%, store 46.7%, jobspec 47.6%) and three packages with
no tests at all (`internal/audit`, `internal/certpaths`, `cmd/orca`).
v0.8 also closes the two SSH-trust "future enhancement" hooks deferred
in v0.6 (D-035 `--host-key-fingerprint` pre-pin, RESEARCH_v0.6 §80
`orca node key-reset`) and adds a requirements-hygiene gate to prevent
the stale-REQ-status drift seen after v0.7 ship.
- [ ] Phase 0: Pre-execution (specify → clarify → research → plan → grill) — tag `v0.7.0`
- [ ] Phase 1: Test coverage uplift round 2 — 6 packages to ≥ 70%, 3 zero-test packages to first tests (REQ-057) — tag `v0.7.1`
- [ ] Phase 2: SSH trust hardening — `--host-key-fingerprint` pre-pin + `orca node key-reset` + TOFU bugfix + `HostKeyFingerprint` population (REQ-058, REQ-059) — tag `v0.7.2`
- [ ] Phase 3: Requirements-hygiene gate — `make verify-reqs` + verify assertion (REQ-060) — tag `v0.7.3`
- [ ] Phase 4: Final review + ship + audit (milestone release) — tag `v0.7.4`
**Milestone type**: NFR (P01 test, P02 chore on trust surface per
D-043, P03 chore, P04 docs/review). Final phase patch IS the milestone
release per NFR-milestone progressive-patch rule. Per-phase tags:
`v0.7.0``v0.7.4`. Tags run on the previous minor's patch line (v0.7.x)
per branch-strategy.md. The milestone branch label uses the milestone
number (`milestone/v0.8-coverage-trust-hardening`); no separate minor
tag.
### Per-phase REQ coverage
- **P01 — Coverage uplift round 2**
- REQ-057 (raise `internal/engine`, `internal/proxmox`,
`internal/cli`, `internal/transport`, `internal/store`,
`internal/jobspec` to ≥ 70%; add first tests for `internal/audit`,
`internal/certpaths`, `cmd/orca`)
- **P02 — SSH trust hardening**
- REQ-058 (`--host-key-fingerprint <sha256>` pre-pin flag on
`orca node join --type proxmox`; fail fast on mismatch; supersedes
TOFU for pre-pinned deployments)
- REQ-059 (`orca node key-reset <node>` clears persisted SSH host
key so next `doctor proxmox`/dispatch re-pins via TOFU or
`--host-key-fingerprint`)
- **P03 — Requirements-hygiene gate**
- REQ-060 (`make verify-reqs` target + verify-stage assertion:
every REQ `Complete` in ROADMAP.md has matching `Complete` row in
REQUIREMENTS.md; enforced in CI `validate` pipeline)
### v0.8 is a continuation milestone, not a direction change
The vision ("minimalist, offline-first, CLI-first orchestration
engine") is unchanged. v0.8 closes the coverage debt left by v0.7's
50% floor and the trust-surface gaps explicitly deferred in v0.6.
+1 -1
View File
@@ -5,7 +5,7 @@
"slug": "orca",
"name": "Orca",
"description": "Offline/CLI-first orchestration engine (Orca) — Nomad-inspired, far simpler than Kubernetes",
"milestone": "v0.7",
"milestone": "v0.8",
"phase": 0,
"milestone_type": "nfr",
"default_branch": "main",
+11 -5
View File
@@ -20,6 +20,7 @@ import (
var (
daemonAddr string
pprofAddr string
)
var daemonCmd = &cobra.Command{
@@ -35,14 +36,15 @@ var daemonCmd = &cobra.Command{
log := newLogger()
addr := daemonAddr
if cfg := configFromCtx(cmd.Context()); cfg != nil && cfg.ListenAddr != "" && addr == ":8080" {
if cfg := configFromCtx(cmd.Context()); cfg != nil && cfg.ListenAddr != "" && !cmd.Flags().Changed("addr") {
addr = cfg.ListenAddr
}
srv := daemon.NewServer(daemon.Options{
DB: db,
Log: log,
Addr: addr,
Actor: "daemon",
DB: db,
Log: log,
Addr: addr,
Actor: "daemon",
PprofAddr: pprofAddr,
})
// Wire the orca.v1.Dispatch service (v0.2 P02). The executor
@@ -71,6 +73,9 @@ var daemonCmd = &cobra.Command{
fmt.Fprintln(cmd.OutOrStdout(), " /v1/tasks - list tasks")
fmt.Fprintln(cmd.OutOrStdout(), " /orca.v1.Dispatch/Submit - cross-node job submit (P02)")
fmt.Fprintln(cmd.OutOrStdout(), " /orca.v1.Dispatch/Status - cross-node job status (P02)")
if pprofAddr != "" {
fmt.Fprintf(cmd.OutOrStdout(), " /debug/pprof/ (pprof) - %s\n", pprofAddr)
}
fmt.Fprintln(cmd.OutOrStdout(), " press Ctrl+C to stop")
ctx, stop := signal.NotifyContext(cmd.Context(), os.Interrupt, syscall.SIGTERM)
@@ -90,6 +95,7 @@ var daemonCmd = &cobra.Command{
func init() {
daemonCmd.Flags().StringVar(&daemonAddr, "addr", ":8080", "listen address")
daemonCmd.Flags().StringVar(&pprofAddr, "pprof", "", "enable pprof endpoint on <addr> (e.g. :6060); unauthenticated, operator-only")
rootCmd.AddCommand(daemonCmd)
_ = slog.Default // keep import if unused above
}
+13
View File
@@ -0,0 +1,13 @@
package cli
import "testing"
func TestDaemonPprofFlag(t *testing.T) {
f := daemonCmd.Flags().Lookup("pprof")
if f == nil {
t.Fatal("--pprof flag not registered on daemonCmd")
}
if f.DefValue != "" {
t.Errorf("--pprof default = %q, want empty", f.DefValue)
}
}
+45
View File
@@ -0,0 +1,45 @@
package daemon
import (
"errors"
"log/slog"
"net/http"
"net/http/pprof"
"time"
)
func StartPprof(addr string, log *slog.Logger) (*http.Server, error) {
if addr == "" {
return nil, nil
}
mux := http.NewServeMux()
mux.HandleFunc("/debug/pprof/", pprof.Index)
mux.HandleFunc("/debug/pprof/cmdline", pprof.Cmdline)
mux.HandleFunc("/debug/pprof/profile", pprof.Profile)
mux.HandleFunc("/debug/pprof/symbol", pprof.Symbol)
mux.HandleFunc("/debug/pprof/trace", pprof.Trace)
mux.Handle("/debug/pprof/heap", pprof.Handler("heap"))
mux.Handle("/debug/pprof/goroutine", pprof.Handler("goroutine"))
mux.Handle("/debug/pprof/threadcreate", pprof.Handler("threadcreate"))
mux.Handle("/debug/pprof/block", pprof.Handler("block"))
mux.Handle("/debug/pprof/mutex", pprof.Handler("mutex"))
server := &http.Server{
Addr: addr,
Handler: mux,
ReadHeaderTimeout: 5 * time.Second,
}
log.Warn("pprof endpoint exposed",
slog.String("addr", addr),
slog.String("warning", "unauthenticated, operator-only — do not expose publicly"))
go func() {
err := server.ListenAndServe()
if err != nil && !errors.Is(err, http.ErrServerClosed) {
log.Error("pprof server stopped", slog.String("addr", addr), slog.Any("err", err))
}
}()
return server, nil
}
+263
View File
@@ -0,0 +1,263 @@
package daemon
import (
"context"
"io"
"log/slog"
"net"
"net/http"
"path/filepath"
"testing"
"time"
"git.cloudinit.dev/coreci/orca/internal/store"
)
func TestStartPprof_Disabled(t *testing.T) {
srv, err := StartPprof("", slog.Default())
if err != nil {
t.Fatalf("StartPprof(\"\", _) returned err: %v", err)
}
if srv != nil {
t.Fatalf("StartPprof(\"\", _) returned non-nil server: %v", srv)
}
}
func TestStartPprof_Enabled(t *testing.T) {
log := slog.New(slog.NewTextHandler(io.Discard, nil))
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("listen: %v", err)
}
addr := ln.Addr().String()
_ = ln.Close()
srv, err := StartPprof(addr, log)
if err != nil {
t.Fatalf("StartPprof returned err: %v", err)
}
if srv == nil {
t.Fatal("StartPprof returned nil server for non-empty addr")
}
t.Cleanup(func() {
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()
_ = srv.Shutdown(ctx)
})
deadline := time.Now().Add(2 * time.Second)
var base string
for time.Now().Before(deadline) {
conn, derr := net.DialTimeout("tcp", addr, 50*time.Millisecond)
if derr == nil {
_ = conn.Close()
base = "http://" + addr
break
}
time.Sleep(20 * time.Millisecond)
}
if base == "" {
t.Fatal("pprof server did not start listening")
}
client := &http.Client{Timeout: 500 * time.Millisecond}
for _, path := range []string{"/debug/pprof/", "/debug/pprof/cmdline", "/debug/pprof/heap"} {
resp, gerr := client.Get(base + path)
if gerr != nil {
t.Errorf("GET %s: %v", path, gerr)
continue
}
_, _ = io.Copy(io.Discard, resp.Body)
_ = resp.Body.Close()
if resp.StatusCode != 200 {
t.Errorf("GET %s: expected 200, got %d", path, resp.StatusCode)
}
}
}
func TestStartPprof_Shutdown(t *testing.T) {
log := slog.New(slog.NewTextHandler(io.Discard, nil))
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("listen: %v", err)
}
addr := ln.Addr().String()
_ = ln.Close()
srv, err := StartPprof(addr, log)
if err != nil {
t.Fatalf("StartPprof returned err: %v", err)
}
if srv == nil {
t.Fatal("StartPprof returned nil server")
}
deadline := time.Now().Add(2 * time.Second)
for time.Now().Before(deadline) {
conn, derr := net.DialTimeout("tcp", addr, 50*time.Millisecond)
if derr == nil {
_ = conn.Close()
break
}
time.Sleep(20 * time.Millisecond)
}
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()
if err := srv.Shutdown(ctx); err != nil {
t.Fatalf("Shutdown: %v", err)
}
client := &http.Client{Timeout: 300 * time.Millisecond}
_, gerr := client.Get("http://" + addr + "/debug/pprof/")
if gerr == nil {
t.Error("expected GET to fail after Shutdown, but it succeeded")
}
}
func TestStartPprof_MuxIsolated(t *testing.T) {
log := slog.New(slog.NewTextHandler(io.Discard, nil))
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("listen: %v", err)
}
addr := ln.Addr().String()
_ = ln.Close()
srv, err := StartPprof(addr, log)
if err != nil {
t.Fatalf("StartPprof returned err: %v", err)
}
if srv == nil {
t.Fatal("StartPprof returned nil server")
}
t.Cleanup(func() {
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()
_ = srv.Shutdown(ctx)
})
deadline := time.Now().Add(2 * time.Second)
for time.Now().Before(deadline) {
conn, derr := net.DialTimeout("tcp", addr, 50*time.Millisecond)
if derr == nil {
_ = conn.Close()
break
}
time.Sleep(20 * time.Millisecond)
}
client := &http.Client{Timeout: 500 * time.Millisecond}
resp, err := client.Get("http://" + addr + "/healthz")
if err != nil {
t.Fatalf("GET /healthz: %v", err)
}
_, _ = io.Copy(io.Discard, resp.Body)
_ = resp.Body.Close()
if resp.StatusCode != 404 {
t.Errorf("expected /healthz to 404 on pprof-only mux, got %d", resp.StatusCode)
}
}
func TestServer_WithPprof(t *testing.T) {
db, err := store.Open(filepath.Join(t.TempDir(), "pprof.db"))
if err != nil {
t.Fatalf("open db: %v", err)
}
defer db.Close()
log := slog.New(slog.NewTextHandler(io.Discard, nil))
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("listen main: %v", err)
}
mainAddr := ln.Addr().String()
pln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("listen pprof: %v", err)
}
pprofAddr := pln.Addr().String()
_ = pln.Close()
s := NewServer(Options{
DB: db,
Log: log,
Addr: mainAddr,
PprofAddr: pprofAddr,
})
s.MarkReady()
if s.pprofServer == nil {
t.Fatal("expected pprofServer to be non-nil after NewServer with PprofAddr")
}
errCh := make(chan error, 2)
go func() {
err := s.httpServer.Serve(ln)
if err != nil && err != http.ErrServerClosed {
errCh <- err
}
}()
deadline := time.Now().Add(2 * time.Second)
for time.Now().Before(deadline) {
conn, derr := net.DialTimeout("tcp", pprofAddr, 50*time.Millisecond)
if derr == nil {
_ = conn.Close()
break
}
time.Sleep(20 * time.Millisecond)
}
client := &http.Client{Timeout: 500 * time.Millisecond}
resp, err := client.Get("http://" + mainAddr + "/healthz")
if err != nil {
t.Fatalf("GET main /healthz: %v", err)
}
if resp.StatusCode != 200 {
t.Errorf("main /healthz: expected 200, got %d", resp.StatusCode)
}
_, _ = io.Copy(io.Discard, resp.Body)
_ = resp.Body.Close()
presp, err := client.Get("http://" + pprofAddr + "/debug/pprof/")
if err != nil {
t.Fatalf("GET pprof /debug/pprof/: %v", err)
}
if presp.StatusCode != 200 {
t.Errorf("pprof /debug/pprof/: expected 200, got %d", presp.StatusCode)
}
_, _ = io.Copy(io.Discard, presp.Body)
_ = presp.Body.Close()
presp, err = client.Get("http://" + pprofAddr + "/healthz")
if err != nil {
t.Fatalf("GET pprof /healthz: %v", err)
}
_, _ = io.Copy(io.Discard, presp.Body)
_ = presp.Body.Close()
if presp.StatusCode != 404 {
t.Errorf("expected /healthz 404 on pprof mux, got %d", presp.StatusCode)
}
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()
if err := s.Shutdown(ctx); err != nil {
t.Errorf("Shutdown: %v", err)
}
client = &http.Client{Timeout: 300 * time.Millisecond}
_, gerr := client.Get("http://" + pprofAddr + "/debug/pprof/")
if gerr == nil {
t.Error("expected pprof GET to fail after Shutdown")
}
_, merr := client.Get("http://" + mainAddr + "/healthz")
if merr == nil {
t.Error("expected main GET to fail after Shutdown")
}
}
+21 -1
View File
@@ -29,7 +29,8 @@ type Server struct {
addr string
ready atomic.Bool
httpServer *http.Server
httpServer *http.Server
pprofServer *http.Server
// mtls is non-nil after StartMTLS has been called; nil otherwise.
// Plaintext HTTP and mTLS are mutually exclusive — a Server is
@@ -49,6 +50,12 @@ type Options struct {
Log *slog.Logger
Addr string
Actor string // used for audit logging from API requests
// PprofAddr enables the pprof endpoint on a separate listener
// when non-empty (e.g. "127.0.0.1:6060"). Default "" disables it.
// The pprof listener is unauthenticated and operator-only; never
// expose it publicly (AD-024).
PprofAddr string
}
// NewServer constructs a Server with the default mux and route table.
@@ -75,6 +82,14 @@ func NewServer(opts Options) *Server {
WriteTimeout: 30 * time.Second,
IdleTimeout: 60 * time.Second,
}
if opts.PprofAddr != "" {
ps, perr := StartPprof(opts.PprofAddr, opts.Log)
if perr != nil {
s.log.Error("pprof start failed", slog.String("component", "daemon"), slog.Any("err", perr))
} else {
s.pprofServer = ps
}
}
return s
}
@@ -142,6 +157,11 @@ func (s *Server) Start() error {
func (s *Server) Shutdown(ctx context.Context) error {
s.MarkNotReady()
s.log.Info("daemon shutting down", slog.String("component", "daemon"))
if s.pprofServer != nil {
if perr := s.pprofServer.Shutdown(ctx); perr != nil {
s.log.Error("pprof shutdown failed", slog.String("component", "daemon"), slog.Any("err", perr))
}
}
return s.httpServer.Shutdown(ctx)
}
@@ -2,4 +2,16 @@
-- issued by orca may share the same serial. Implemented as a UNIQUE
-- INDEX so existing 0004_certs.sql need not be re-run on deployed
-- databases. v0.7 P01 (REQ-053 companion).
--
-- P1-001 fix (final review): before creating the UNIQUE index, dedup
-- any existing rows that share a serial_hex. Keep the newest row
-- (MAX(created_at)) per serial_hex and delete older duplicates. This
-- makes the migration backward-compatible with v0.6 deployments that
-- may have accumulated duplicate serials before the constraint existed.
DELETE FROM certs WHERE id NOT IN (
SELECT id FROM (
SELECT id, ROW_NUMBER() OVER (PARTITION BY serial_hex ORDER BY created_at DESC) AS rn
FROM certs
) WHERE rn = 1
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_certs_serial_unique ON certs(serial_hex);