RESEARCH_v0.8.md (35KB): per-package coverage strategy for 9 pkgs, SSH trust research (uncovered latent TOFU capture bug + unpopulated HostKeyFingerprint field), verify-reqs Go program approach, 4 ADs, 10 pitfalls. PERSONAS.md updated for v0.8 (3-persona roster retained, connectrpc removed from backend frameworks per AD-014, territory globs aligned to actual file structure). Key findings flagged for PLAN: - proxmox needs sessionRunner seam (~10 LOC) or stalls at ~55% - cert_repo_test.go missing (v0.7 P01 leftover) blocks store 70% - TOFU known_hosts capture broken + HostKeyFingerprint never populated (bootstrap.go:195-198) — P02 fixes both - verify-reqs = Go program at cmd/verify-reqs (~80 LOC, stdlib only) ---ci--- project: orca phase: 0 milestone: v0.8 status: research ---/ci---
34 KiB
Research: Orca v0.8 — Coverage & Trust Hardening
Findings grounded in codebase analysis (44 source/test files read, coverage
re-measured for all 9 target packages) + golang.org/x/crypto v0.54.0 API
verification (ssh.FingerprintSHA256, knownhosts.Line/Normalize/KeyError).
1. Coverage analysis (P01 — REQ-057)
1.1 Re-measured coverage (confirmed via go test ./<pkg>/... -cover)
| Package | Coverage | Tier (D-047) | Notes |
|---|---|---|---|
internal/engine |
8.3% | ≥ 70% floor | Only scheduler_test.go (4 tests, 66 LOC); executor/dispatcher/peer/registry/audit untested |
internal/proxmox |
5.1% | ≥ 70% floor | Only bootstrap_test.go (4 tests, validation + sudoersContent string asserts); SSH dial path untested |
internal/cli |
27.6% | ≥ 70% floor | 5 test files (root, init, namespace, osdetect, watch); node/job/cert/doctor/audit/cmds untested |
internal/transport |
26.3% | ≥ 70% floor | Only idempotency_test.go (7 tests); mtls/dispatch/retry/handshake_log untested |
internal/store |
47.2% | ≥ 70% floor | node_repo + job_task + capacity + audit + migrate tested; cert_repo has NO test (REQ-053 leftover — v0.7 P01 was supposed to add it but it's missing) |
internal/jobspec |
47.6% | ≥ 70% floor | Only spec_test.go (4 tests); Validate(), ParseFile (file I/O), edge cases untested |
internal/audit |
0% (no test files) | ≥ 50% toe-hold | go: no such tool "covdata" is a known tooling gap, NOT a real number — the package simply has no _test.go |
internal/certpaths |
0% (no test files) | ≥ 50% toe-hold | Same covdata tooling gap; no _test.go exists |
cmd/orca |
0% (no test files) | ≥ 50% toe-hold | Same; main.go is 15 LOC of glue (cli.Execute() + error print) |
Coverage-floor achievability assessment (per package):
- engine → 70% REALISTIC. The package has clean seams:
LocalExecutorinterface (dispatcher.go:39),PeerRegistryis in-memory withAdd/Remove/All/Get(peer.go),Executor.Submit/Statustake a*store.JobRepo+*store.TaskRepowhich can be backed by:memory:/temp-file sqlite via the existingopenTestDBhelper (node_repo_test.go:12). ThesshDialerseam pattern (proxmox) has an analogue here:transport.NewDispatchClientis called insidedispatchToPeer(dispatcher.go:158) — to test dispatch-to-peer without a real mTLS server, either (a) inject a fakeDispatchClientvia a new interface seam, or (b) usehttptest.NewTLSServerwith a self-signed CA. Option (a) is lower-effort and aligns with theLocalExecutorpattern. Recommendation: extract apeerDispatcherinterface (Submit(ctx, spec, key) (*SubmitResponse, error)) and inject it, OR test viaLocalSubmit/LocalStatuspaths (which only need a stubbedLocalExecutor) — the latter covers ~60% of dispatcher.go without a new seam. Flag: 70% may require a small refactor to inject the dispatch client; 60-65% is achievable without one. Plan should decide whether to add the seam or accept 65%. - proxmox → 70% REALISTIC. The
sshDialerseam already exists (bootstrap.go:201-213,sshDialerTypeinterface +defaultSSHDialerstruct, overridable package-level var). A fake SSH dialer returning a mock*ssh.Clientis the path. However:*ssh.Client.NewSession()+session.CombinedOutput()are concrete methods on the real*ssh.Client— there's nosshSessioninterface seam. To testrunRemote/deployPubKey/createLinuxUser/createPVERole/etc. without a real SSH server, EITHER (a) introduce asessionRunnerinterface seam (small refactor), OR (b) usehttptest.NewTLSServeris wrong (it's SSH not HTTP) — instead use a real in-process SSH server viagolang.org/x/crypto/sshNewServerConn(more code but no new dep). Flag: 70% likely requires either asessionRunnerinterface refactor OR an in-process SSH server fixture. 50-55% is achievable with just the existingsshDialerseam + testing validation paths +sudoersContentstring asserts (already done). Plan should add thesessionRunnerseam — it's a 1-interface, ~10-LOC change that unlocks the bulk of the package. - cli → 70% AMBITIOUS but realistic. The package is the largest (17 source files, ~2000 LOC). The existing tests use
rootCmd.SetArgs()+rootCmd.Execute()+t.TempDir()+ORCA_HOMEenv (namespace_test.go:46-53 —TestInitHonorsORCAHOMEis the template). The untested commands arenode join/leave/list,job run/list/stop/logs,cert *,doctor *,audit list,status,version,daemon. Many touch the DB + certpaths + (fornode join --type proxmox) the SSH dialer. Strategy: table-drivenrootCmd.Execute()against a tempORCA_HOMEfor each subcommand; mock the proxmox path via the existingsshDialerseam; capture stdout viarootCmd.SetOut(&buf). Flag: 70% across the whole package is a lot of test code; 55-65% is more realistic for one phase. Thedaemoncommand (background server) is hard to test without a lifecycle harness — recommend excluding it from the 70% target and documenting why. - transport → 70% REALISTIC.
httptest.NewTLSServeris the standard seam (already used ininternal/daemon/dispatch_test.go:59andserver_test.go). TheDispatcherinterface (dispatch.go:49) is already mockable (stubDispatcherin dispatch_test.go:24 is the template).MTLSClient.Dowrapshttp.Client.Do— testable viahttptest.NewTLSServerwith a CA + client cert.retry.goDo[T]is generic + already partly tested viaidempotency_test.go(TestRetrySucceedsAfterTransient etc.) — extend with backoff-timing asserts.handshake_log.gois pure slog calls — trivial to test by capturing into aslog.Handler. No new seams needed; 70% achievable. - store → 70% REALISTIC. The existing
openTestDBhelper (node_repo_test.go:12) +withFastWatch(job_task_repo_test.go:36) are reusable. Critical gap:cert_repo.gohas NO test file despite v0.7 P01 REQ-053 claiming it was added — this is a v0.7 leftover bug. Addingcert_repo_test.go(Insert/Get/List/ListByNode/LatestForKind/PruneOlderThan/Delete + N=3 rotation history per REQ-025) alone lifts coverage significantly. Job/Task repoWatchis tested;ListRecent, error paths, scan-edge cases need coverage. No new seams; 70% achievable. - jobspec → 70% REALISTIC.
Parse+Validate+ParseFileare pure functions over HCL bytes. Add golden-file HCL fixtures (multi-task, env vars, args) + error-path table (missing job, no tasks, missing command, malformed HCL, empty file, nonexistent file forParseFile).testdata/dir doesn't exist yet — create it. No new seams; 70% achievable, likely the easiest of the six. - audit → 50% toe-hold REALISTIC. Package is 125 LOC, 4 exported funcs (
New,Emit,EmitWithErr,LogHandshakeOK,LogHandshakeFailed,FormatAction,Action.String,Result.String). Strategy: constructAuditwith a realengine.Auditbacked by:memory:sqlite (viastore.NewAuditRepo+engine.NewAudit) + assert rows inaudit_logtable; capture slog output via a testslog.Handler. No new seams; 50% easily achievable, 70% achievable if desired. - certpaths → 50% toe-hold TRIVIAL. Package is 64 LOC, pure path-join functions honoring
ORCA_HOME/ORCA_DBenv. Strategy: temp dir +t.Setenv("ORCA_HOME", dir)+ assert each*Path()returnsfilepath.Join(dir, <file>); testORCA_DBoverride; test default-to-~/.orcafallback. Model the test onnamespace_test.go(cli). No new seams; 50%+ trivially achievable. - cmd/orca → 50% toe-hold REALISTIC but LOW VALUE.
main.gois 15 LOC:cli.Execute()+fmt.Fprintf(os.Stderr, "error: %v")+os.Exit(1). The only testable behavior is "main() calls Execute and exits non-zero on error." A smoke test that callsmain()in a subprocess (or refactors main into arun() intfor testability) is the path. Flag: 50% on a 15-LOC glue file is ~7 lines of covered code — the effort:coverage ratio is poor. D-047 explicitly called this out ("0→70% risks a coverage rathole oncmd/orcawhich is glue code"). Recommend the plan keep this at the 50% toe-hold and not over-invest.
1.2 Existing test-helper utilities (reuse, do NOT re-create)
| Helper | Location | Reuse for |
|---|---|---|
openTestDB(t) |
internal/store/node_repo_test.go:12 |
engine, audit, store tests — returns (*NodeRepo, func()) backed by temp-file sqlite; adapt to return *sql.DB for JobRepo/TaskRepo/AuditRepo/CapacityRepo/CertRepo |
withFastWatch(t, d) |
internal/store/job_task_repo_test.go:36 |
store Watch tests — overrides watchInterval for deterministic ticks |
initTestEnv(t) |
internal/cli/init_test.go:17 |
cli tests — sets ORCA_HOME to temp dir + returns cleanup |
resetRootFlags(t) |
internal/cli/namespace_test.go:13 |
cli tests — resets rootCmd args/out/json/system flags between subtests |
discardWriter |
internal/cli/init_test.go:33 |
cli tests — io.Writer that discards stdout |
stubDispatcher |
internal/daemon/dispatch_test.go:24 |
transport/engine tests — implements transport.Dispatcher (LocalSubmit/LocalStatus); reusable as a LocalExecutor too since the signatures match |
insertNode(t, repo, ctx, id, name) |
internal/store/node_repo_test.go:217 |
store/doctor tests — inserts a minimal node |
security.CAInit/LoadCA/GenerateCSR/SignCSR/WriteCert/WriteKey |
internal/security/ca.go |
transport mTLS tests — bootstrap a real CA + server cert into a temp dir (pattern in doctor_test.go:69-94) |
t.Setenv("ORCA_HOME", dir) + t.Setenv("ORCA_DB", ...) |
internal/doctor/doctor_test.go:23-24 |
any test needing the orca namespace — preferred over manual os.Setenv (auto-cleanup) |
1.3 Injected seams already present in the codebase (confirm by reading)
sshDialer(proxmox) —internal/proxmox/bootstrap.go:201-213: package-levelvar sshDialer sshDialerType = defaultSSHDialer{}; interfacesshDialerType{ DialContext(ctx, network, addr, *ssh.ClientConfig) (*ssh.Client, error) }. Tests can swapsshDialerfor a fake. GAP: nosessionRunnerseam —runRemote(line 217) callsconn.NewSession()+session.CombinedOutput(cmd)directly on the concrete*ssh.Client. Recommend P01 plan add asessionRunnerinterface (CombinedOutput(cmd) ([]byte, error)) sodeployPubKey/createLinuxUser/createPVERole/createPVEUser/assignPVEACL/writeSudoers/validateSudoersbecome testable without a real SSH endpoint.LocalExecutor(engine dispatcher) —internal/engine/dispatcher.go:39: interfaceSubmit(ctx, []byte) (string, error)+Status(ctx, string) (string, error).Dispatcherdepends on it; tests inject a stub. GAP:dispatchToPeer(line 154) callstransport.NewDispatchClientdirectly (no seam) — to test the remote-dispatch branch, either add apeerDispatcherinterface or test viahttptest.NewTLSServer.PeerPersister(engine peer) —internal/engine/peer.go:39: optional persist callback; unused in production but available as a seam.Dispatcher(transport) —internal/transport/dispatch.go:49:LocalSubmit/LocalStatusinterface;stubDispatcherindaemon/dispatch_test.go:24is the template stub.watchInterval(store) —internal/store/job_task_repo.go:20: unexportedvar watchInterval = 1 * time.Second; tests override viawithFastWatch.
1.4 Packages where 70% is unrealistic in a single phase (with evidence)
internal/cli— 70% is ambitious. 17 source files, ~2000 LOC. Thedaemoncommand (internal/cli/daemon.go) starts a long-running mTLS server — testing it requires a lifecycle harness (start, probe, shutdown) and is better covered byinternal/daemon/server_test.go(already exists, 150 LOC). Recommend the P01 plan excludedaemon.gofrom the cli 70% target (document it as covered by the daemon package's own tests) and aim for 70% of the remaining cli files. Even so, 55-65% is the realistic single-phase outcome for the rest.cmd/orca— 70% is explicitly out of scope per D-047. 15 LOC of glue; 50% toe-hold is the right call.internal/proxmox— 70% likely requires thesessionRunnerseam refactor. Without it, only the validation paths +sudoersContentstring asserts are testable (~50-55%). The plan should add the seam; with it, 70% is achievable.
2. SSH trust hardening research (P02 — REQ-058, REQ-059)
2.1 Current TOFU knownhosts.New() callback — how it works
Location: internal/proxmox/bootstrap.go:125-128 (bootstrap) + internal/doctor/doctor.go:412-415 (doctor proxmox probe).
hostKeyCallback, err := knownhosts.New(certpaths.KnownHostsPath())
// ...
sshConfig := &ssh.ClientConfig{
HostKeyCallback: hostKeyCallback,
// ...
}
Mechanism (golang.org/x/crypto/ssh/knownhosts):
knownhosts.New(files ...string)returns anssh.HostKeyCallbackthat reads the OpenSSH-formatknown_hostsfile atcertpaths.KnownHostsPath()(=$ORCA_HOME/known_hosts, seeinternal/certpaths/certpaths.go:62).- First connect (host absent from file): the callback returns a
*knownhosts.KeyErrorwithWant: [](empty). This is a "host unknown" signal. IMPORTANT:knownhosts.Newdoes NOT auto-write the key on first connect — it returns an error. The current orca code atbootstrap.go:140treats ANY dial error as a failure (return nil, fmt.Errorf("ssh dial %s: %w", sshAddr, err)). This means the current TOFU flow is INCOMPLETE: on a truly first connect,knownhosts.NewreturnsKeyError{Want:[]}and the dial fails — there is no capture-and-persist step. The v0.6 RESEARCH_v0.6.md §A.5 claimedknownhosts.New"handles both capture and verify in one callback" but the actualgolang.org/x/cryptoAPI does NOT auto-capture; it only verifies. This is a latent bug OR the operator is expected to pre-populateknown_hostsmanually (which contradicts the TOFU UX). P02 must address this: either (a) wrapknownhosts.Newwith a custom callback that captures onKeyError{Want:[]}and writes viaknownhosts.Line, or (b) accept that--host-key-fingerprint(REQ-058) becomes the required path for first connect and TOFU capture is a separate enhancement. Flag for plan: the current TOFU capture is broken; P02 should fix it as part of the trust-hardening work (the--host-key-fingerprintpath is actually simpler than TOFU because it doesn't need capture). - Subsequent connects (host present, key matches): callback returns
nil→ dial proceeds. - Subsequent connects (host present, key MISMATCH): callback returns
*knownhosts.KeyError{Want: [knownKey]}→ dial fails with a clear error. This is the MITM-detection path.
File format: OpenSSH known_hosts — one line per host: [host]:port ssh-key-type base64-key (or hashed-host form via knownhosts.HashHostname). knownhosts.Line(addresses []string, key ssh.PublicKey) string produces the line; knownhosts.Normalize(address) normalizes the host:port.
2.2 Result.HostKeyFingerprint — current computation (CRITICAL FINDING)
Location: internal/proxmox/bootstrap.go:83-85 (field declaration) + bootstrap.go:195-198 (return statement).
type Result struct {
NodeName string
NodeAddress string
HostKeyFingerprint string // field EXISTS
}
// ...
return &Result{
NodeName: opts.Host,
NodeAddress: opts.Host + ":8443",
// HostKeyFingerprint is NOT SET — always empty string
}, nil
Finding: Result.HostKeyFingerprint is declared but never populated. The current BootstrapProxmox returns it as "". There is no fingerprint computation today — no ssh.FingerprintSHA256 call, no hex digest, nothing. D-045's rationale ("matches the fingerprint format operators already see from orca node join's own Result.HostKeyFingerprint output") is based on a field that is currently always empty.
Implication for P02: The plan must ADD the fingerprint computation. The correct function is ssh.FingerprintSHA256(pubKey ssh.PublicKey) string (verified via go doc), which returns the OpenSSH SHA256:base64 format (unpadded base64, exactly what ssh-keyscan -E sha256 emits and what D-045 specifies). So D-045's format choice is correct by intent but the code doesn't produce it yet — P02 populates Result.HostKeyFingerprint = ssh.FingerprintSHA256(hostKey) during the capture path, and --host-key-fingerprint compares against ssh.FingerprintSHA256 of the server-presented key.
No existing fingerprint-comparison utility in internal/security/. security.Fingerprint (fingerprint.go:17) computes SHA-256 hex of an X.509 cert's DER — a DIFFERENT format (hex, not base64; X.509, not SSH). security.FingerprintOf (fingerprint.go:34) is the same. Do NOT reuse these for SSH host-key comparison — they're for the mTLS CA pin (--ca-fingerprint). P02 needs a new SSH-specific helper, e.g. security.SSHFingerprintSHA256(pubKey ssh.PublicKey) string (thin wrapper over ssh.FingerprintSHA256) or inline in proxmox/bootstrap.go.
2.3 Where --host-key-fingerprint plugs in (REQ-058)
CLI seam: internal/cli/node.go:344-354 — the init() registers flags on nodeJoinCmd. Add:
nodeJoinCmd.Flags().StringVar(&joinHostKeyFP, "host-key-fingerprint", "", "SSH host key SHA256:base64 fingerprint (pre-pin; supersedes TOFU for --type proxmox)")
Per D-044, the flag lives on orca node join (not just --type proxmox); validation in RunE (node.go:78-83) emits a clear error if the flag is set for a non-proxmox type.
Transport seam: internal/proxmox/bootstrap.go:131-136 — ssh.ClientConfig.HostKeyCallback. Currently knownhosts.New(...). When --host-key-fingerprint is supplied, replace the callback with a ssh.FixedHostKey-style verifier that:
- Parses the operator-supplied
SHA256:base64string (stripSHA256:prefix, base64-decode → 32 bytes). - In the callback, receives the server's
ssh.PublicKey, computesssh.FingerprintSHA256(key), compares to the operator string. - Returns
nilon match,erroron mismatch (fail closed).
Recommended callback shape (concrete):
func pinnedHostKeyCallback(expectedSHA256Base64 string) (ssh.HostKeyCallback, error) {
// Validate format: must start with "SHA256:".
if !strings.HasPrefix(expectedSHA256Base64, "SHA256:") {
return nil, fmt.Errorf("host-key-fingerprint: must be OpenSSH SHA256:base64 format (got %q)", expectedSHA256Base64)
}
expected := expectedSHA256Base64 // store full string for direct compare
return func(_ string, _ net.Addr, key ssh.PublicKey) error {
got := ssh.FingerprintSHA256(key)
if got != expected {
return fmt.Errorf("host key fingerprint mismatch: got %s, want %s — refusing to connect (REQ-058)", got, expected)
}
return nil
}, nil
}
Why compare full strings (not base64-decoded bytes): ssh.FingerprintSHA256 returns the canonical SHA256:base64 string; comparing it directly to the operator-supplied string is simplest and avoids a base64-decode step. Reject non-SHA256:-prefixed input up front with a clear error (D-045: "Accept only SHA256:-prefixed base64; reject raw hex with a clear error").
Pass-through to proxmox: internal/cli/node.go:158-166 — add HostKeyFingerprint string to proxmox.Options (bootstrap.go:55) and pass joinHostKeyFP through. BootstrapProxmox selects the callback: if opts.HostKeyFingerprint != "" use pinnedHostKeyCallback, else fall back to the TOFU knownhosts.New (with the capture-fix from §2.1).
2.4 orca node key-reset <node> (REQ-059, D-046 — local known_hosts only)
Scope (D-046): clear the local ~/.orca/known_hosts entry for the node ONLY; do NOT revoke the remote authorized_keys entry (would orphan a working node). Audit-log event=node.key_reset with actor + node.
known_hosts line format written by golang.org/x/crypto/ssh/knownhosts:
knownhosts.Line(addresses []string, key ssh.PublicKey) string→"[host]:port ssh-ed25519 AAAA...\n"(orhost ssh-ed25519 AAAA...if port 22 —knownhosts.Normalizehandles the:22vs bare-host normalization).- The file is plain text, one entry per line,
#-prefixed comments allowed.
No library function to remove a host's entries. knownhosts.New only reads. The reset must be implemented manually:
- Read
certpaths.KnownHostsPath()(internal/certpaths/certpaths.go:62). - Filter lines: keep lines whose host field (before the first whitespace) does NOT match
knownhosts.Normalize(nodeName)(or the node's address). Edge: a host may have multiple entries (one per key type); remove all matching lines. - Write the filtered content back via atomic rewrite (temp file in same dir +
os.Rename) — reusesecurity.writeAtomic(ca.go:305) OR implement inline (it's unexported insecurity; either export it or copy the ~20-LOC pattern). Recommend atomic rewrite, NOT in-place truncation — in-place rewrite viaos.OpenFile(O_TRUNC|O_WRONLY)risks data loss on crash mid-write.
CLI registration seam: internal/cli/node.go:358-360 — the init() does nodeCmd.AddCommand(nodeJoinCmd), nodeLeaveCmd, nodeListCmd. Add:
nodeCmd.AddCommand(nodeKeyResetCmd)
where nodeKeyResetCmd is a new &cobra.Command{Use: "key-reset <node>", Args: cobra.ExactArgs(1), RunE: ...}. The RunE:
- Resolve
<node>arg → look up the node in the registry (nodeRegistry()at node.go:37) to get its address (for matchingknown_hostslines) — OR accept the raw host string directly. Recommend: accept the node NAME (consistent withdoctor proxmoxwhich iteratesnode.Name), look up the node row, usenode.Name(which is the host address for proxmox nodes perbootstrap.go:196) as theknown_hostsmatch key. - Call a new
proxmox.ResetHostKey(host string) error(or inline in cli) that does the atomic rewrite. - Audit-log via
engine.Audit.Record(ctx, "cli", "node.key_reset", nodeID, "success", nil, map[string]any{"host": host}). - Print
✓ Host key reset for <node> (next connect will re-pin via TOFU or --host-key-fingerprint).
Reusability: the nodeRegistry() helper (node.go:37) + openDB() (node.go:25) + newLogger() (node.go:33) are all available for the key-reset command.
2.5 CLI registration seam summary (P02)
| Addition | File:line | Change |
|---|---|---|
--host-key-fingerprint flag |
internal/cli/node.go:344-354 (init) |
nodeJoinCmd.Flags().StringVar(&joinHostKeyFP, "host-key-fingerprint", "", "...") |
joinHostKeyFP var |
internal/cli/node.go:47-60 (var block) |
add joinHostKeyFP string |
| Pass-through to proxmox | internal/cli/node.go:158-166 (joinProxmox) |
add HostKeyFingerprint: joinHostKeyFP to proxmox.Options |
HostKeyFingerprint field |
internal/proxmox/bootstrap.go:55 (Options) |
add field |
| Pinned callback | internal/proxmox/bootstrap.go:131-136 |
branch: if opts.HostKeyFingerprint != "" use pinned callback else TOFU |
Populate Result.HostKeyFingerprint |
internal/proxmox/bootstrap.go:195-198 |
set HostKeyFingerprint: ssh.FingerprintSHA256(hostKey) during capture |
key-reset subcommand |
internal/cli/node.go:358-360 (init) |
nodeCmd.AddCommand(nodeKeyResetCmd) + new cmd var |
ResetHostKey helper |
internal/proxmox/bootstrap.go (new) OR internal/security/sshkey.go |
atomic known_hosts rewrite |
3. Requirements-hygiene gate research (P03 — REQ-060)
3.1 Current Makefile targets
Makefile has 11 targets: build, test, test-race, lint, fmt, clean, run, version, changelog, release, security-scan (Makefile:1-100). No verify-reqs target exists. The .PHONY list at line 1 must be extended.
3.2 Current .coreci.yml pipeline structure
4 pipelines (.coreci.yml:19-134):
- validate (line 20): 4 steps —
go-version(gofmt+vet),gosec,govulncheck,gitleaks. - build (line 53): 1 step — version-injected
go build. - test (line 72): 1 step —
go test -race -coverprofile=coverage.out ./...+go tool cover -func | tail -1. - release (line 81): gated on
refs/tags/v*; 3 steps — build-artifact, gitea-release, container-publish.
Hook for verify-reqs: add a 5th step to the validate pipeline (after go-version, before/after gosec) OR add it to the test pipeline. Recommend validate pipeline — requirements hygiene is a static check (no test run needed), belongs alongside gofmt/vet/lint. Step shape:
- name: verify-reqs
image: golang:1.25
commands:
- make verify-reqs
3.3 verify-reqs implementation recommendation
Assertion (REQ-060): every REQ row in ROADMAP.md marked [x]/Complete must have a matching REQ-ID row in REQUIREMENTS.md with Complete status. (Reverse direction — every REQUIREMENTS Complete has a ROADMAP [x] — is also worth checking but the drift that motivated this was ROADMAP-shipped-but-REQUIREMENTS-Pending, so the forward direction is the priority.)
Approach: small Go program in cmd/verify-reqs OR a shell+awk script?
- Go program (~80 LOC): parse both markdown tables with
regexp, build twomap[string]string(REQ-ID → status), diff. Pros: type-safe, testable, consistent with the Go toolchain; can be acmd/verify-reqs/main.gowith its own_test.go. Cons: adds a binary target. - Shell+awk (~30 LOC):
awkover the markdown tables. Pros: no new Go package; minimal. Cons: fragile parsing, hard to test, shell-quoting issues.
Recommendation: Go program at cmd/verify-reqs/main.go. Reasons: (1) testable with golden-file fixtures (parse a sample ROADMAP+REQUIREMENTS pair, assert diff); (2) consistent with the project's Go-only tooling ethos (no shell-awk fragility); (3) the make verify-reqs target just calls go run ./cmd/verify-reqs; (4) CoreCI's golang:1.25 image has go available — no extra dep.
Parsing approach (concrete):
- ROADMAP.md: regex
^\s*-\s*\[(x|X| )\]\s*Phase.*—.*tagis NOT the right pattern (that's phase lines, not REQ rows). The REQ coverage is in per-phase bullet lists under "### Per-phase REQ coverage" (ROADMAP.md:161-180) AND in the milestone section bodies. Simpler: the ROADMAP uses- [x] Phase N: ...for completed phases. The authoritative REQ↔status mapping lives in REQUIREMENTS.md (the single table at lines 9-56 + per-milestone tables at 103-142). Re-interpret REQ-060: the assertion is really "ROADMAP milestone sections marked COMPLETE ↔ REQUIREMENTS rows for that milestone marked Complete." The drift was: v0.7 ROADMAP said "COMPLETE" (line 116) but REQUIREMENTS v0.7 rows (REQ-053..056) were "Pending" (now corrected to "Complete" in SPECIFY). - Refined assertion: parse REQUIREMENTS.md table rows (
| REQ-XXX | ... | ... | ... | **Complete** |or| Pending |); for each REQ-ID, record status. Then parse ROADMAP.md for milestone-level "COMPLETE" markers (## Milestone v0.X: ... — **COMPLETE**) AND phase-level- [x]markers. For each milestone marked COMPLETE in ROADMAP, assert every REQ-ID belonging to that milestone (per the REQUIREMENTS milestone column) isCompletein REQUIREMENTS. OR (simpler, matches the SPECIFY wording): for every REQ-ID in REQUIREMENTS.md whosePhasecolumn references a milestone that ROADMAP marks COMPLETE, the Status must beComplete. This catches the exact drift (ROADMAP-shipped, REQUIREMENTS-stale).
Concrete regex:
- REQUIREMENTS row:
^\|\s*(REQ-\d+)\s*\|.*?\|\s*\*\*(Complete|Pending)\*\*\s*\|(capture ID + status). - ROADMAP milestone-complete:
^##\s*Milestone\s+v0\.\d+:.*—\s*\*\*COMPLETE\*\*(capture milestone label). - Map milestone → REQs via the REQUIREMENTS
Phasecolumn (e.g.v0.7 P1→ milestonev0.7).
Where it hooks in: make verify-reqs runs go run ./cmd/verify-reqs .ciagent/ROADMAP.md .ciagent/REQUIREMENTS.md; .coreci.yml validate pipeline adds the step. Exit 0 on consistency, exit 1 with a diff listing on drift.
3.4 The drift that motivated REQ-060
After v0.7 ship, REQUIREMENTS.md rows REQ-053..056 were "Pending" despite ROADMAP.md marking milestone v0.7 COMPLETE and all phases [x]. This was corrected during v0.8 SPECIFY (the rows now read **Complete**). REQ-060 ensures the drift cannot recur: the CI validate pipeline fails if ROADMAP says COMPLETE but REQUIREMENTS says Pending.
4. Architectural decisions surfaced (AD-027..AD-030)
| ID | Decision | Rationale |
|---|---|---|
| AD-027 | ssh.FingerprintSHA256 (OpenSSH SHA256:base64) as the SSH host-key fingerprint format |
Matches D-045 + ssh-keyscan -E sha256 output. The existing security.Fingerprint (hex, X.509) is NOT reused — different domain. P02 adds a thin SSH-specific helper. |
| AD-028 | --host-key-fingerprint callback compares full SHA256:base64 strings, not decoded bytes |
ssh.FingerprintSHA256 returns the canonical string; direct string compare avoids a base64-decode step and is less error-prone. Validate SHA256: prefix up front. |
| AD-029 | orca node key-reset rewrites known_hosts via atomic temp-file + rename |
Prevents data loss on crash mid-write. Reuse the writeAtomic pattern from security/ca.go:305 (export it or copy the ~20 LOC). |
| AD-030 | verify-reqs implemented as cmd/verify-reqs/main.go (Go program), not shell+awk |
Testable, type-safe, consistent with Go-only tooling. make verify-reqs runs go run ./cmd/verify-reqs. Hooked into .coreci.yml validate pipeline. |
5. Pitfalls, gaps, and flags for the plan
- TOFU capture is currently BROKEN (§2.1).
knownhosts.NewreturnsKeyError{Want:[]}on first connect and does NOT auto-write the key. The currentBootstrapProxmoxtreats this as a dial failure. P02 must either (a) wrap the callback to capture-and-persist onKeyError{Want:[]}viaknownhosts.Line+ atomic write, or (b) make--host-key-fingerprintthe required first-connect path. Recommend (a) — fix TOFU + add pre-pin as superset. This is a v0.6 latent bug that P02 closes. Result.HostKeyFingerprintis never populated (§2.2). D-045's rationale references "existing output" that doesn't exist. P02 must ADD the computation (ssh.FingerprintSHA256). Low risk — it's a 1-line addition once the host key is available.- No
sessionRunnerseam in proxmox (§1.3). Testing the SSH command sequence (deployPubKey, createLinuxUser, pveum, sudoers, visudo) without a real SSH server requires a new interface seam. Recommend P01 plan add it — 1 interface, ~10 LOC, unlocks ~40% of proxmox coverage. internal/store/cert_repo.gohas NO test (§1.1). v0.7 P01 REQ-053 was supposed to addcert_repo_test.gobut it's missing —internal/store/glob shows nocert_repo_test.go. This is a v0.7 leftover. P01 should add it (it directly lifts store coverage toward 70%).internal/cli/daemon.goexcluded from cli 70% target (§1.4). The daemon command starts a long-running server; it's covered byinternal/daemon/server_test.go(150 LOC). Don't double-test in cli.cmd/orca50% toe-hold is low-value (§1.1). 15 LOC of glue; the test effort:coverage ratio is poor. D-047 already called this out. Don't over-invest.go: no such tool "covdata"for zero-test packages (§1.1). This is a Go toolchain quirk when a package has no test files —go test -covercan't compute coverage without a test binary. It's NOT a real 0% number (it's "undefined"). Adding any_test.gofile makes the number computable. Don't treat the error as a coverage measurement.transport.dispatchToPeerhas no seam (§1.3). Testing the remote-dispatch branch ofDispatcher.Submitrequires either a newpeerDispatcherinterface ORhttptest.NewTLSServer. The latter is already used indaemon/dispatch_test.go; recommend the plan usehttptest.NewTLSServer(no refactor needed) for transport coverage.knownhosts.Line+knownhosts.Normalizeare the helpers for the TOFU-capture fix and forkey-resetmatching (§2.1, §2.4). UseNormalizeto match host strings consistently (handleshost:22vshost).security.writeAtomicis unexported (ca.go:305).key-reset's atomic known_hosts rewrite needs it. Either exportWriteAtomicfromsecurity, or copy the ~20-LOC pattern intoproxmox/cli. Recommend export — it's already used across ca.go + sshkey.go and is generally useful.
6. Dependencies
v0.8 adds zero new direct dependencies:
- SSH host-key fingerprint:
ssh.FingerprintSHA256(already ingolang.org/x/crypto/sshv0.54.0, direct dep since v0.6). knownhosts.Line/Normalize/KeyError: samegolang.org/x/cryptomodule.verify-reqs: stdlib only (regexp,os,fmt).- Tests:
net/http/httptest(stdlib), existing interfaces.
go.mod is unchanged by v0.8.
7. PERSONAS assessment (v0.8)
v0.8 is an NFR milestone touching tests (9 packages), SSH trust surface (proxmox + cli/node + security), and a requirements-hygiene Go program. The 3-persona roster from config.json (lead-developer, backend-engineer, data-engineer) is sufficient — no phase-specific personas needed.
Roster confirmation:
- lead-developer — owns coordination +
cmd/orcasmoke test +internal/clicoverage (cert/doctor/audit/status/version subcommands) + theverify-reqsGo program (coordination territory). - backend-engineer — owns
internal/transporttests (httptest.NewTLSServer) +internal/enginetests (LocalExecutor stubs, PeerRegistry) + SSH trust-surface ininternal/proxmox/bootstrap.go(pinned callback, TOFU capture fix, sessionRunner seam) +internal/cli/node.go(--host-key-fingerprintflag,key-resetsubcommand). - data-engineer — owns
internal/storetests (cert_repo_test.go gap + coverage uplift) +internal/audittests (sqlite-backed audit_log asserts) +internal/certpathstests (path-join asserts) +internal/jobspectests (golden HCL fixtures).
No frontend persona (no UI). No devops persona (no packaging/distribution — verify-reqs is a Go program, not a CI config change; the .coreci.yml edit is a 3-line hook, lead-developer territory). No security-engineer persona (the SSH trust work is backend-engineer territory — the security-engineer was deactivated in v0.7 and v0.8 doesn't re-add it; the trust-surface hardening is a refinement of the existing proxmox package, not new security architecture).
See .ciagent/PERSONAS.md (updated with v0.8 YAML frontmatter + territory globs matching the actual file structure).