Compare commits

..

3 Commits

Author SHA1 Message Date
Jon Chery c814afa773 docs(audit): fix ROADMAP stale checkbox + v0.2 milestone status
---ci---
project: orca
phase: 3
milestone: v0.3
status: audit
---/ci---

Audit fixes:
- Phase 11 checkbox: [ ] → [x] (completed in v0.3 P01, shipped v0.3.1)
- v0.2 milestone status: 'pending merge to main' → 'COMPLETE (merged via v0.3)'
- v0.2 milestone tag: 'pending' → 'v0.4.0 shipped'
2026-08-03 17:45:02 +00:00
Jon Chery dbdf679040 docs(milestone): v0.3 checkpoint — milestone complete
---ci---
project: orca
phase: 3
milestone: v0.3
status: complete
---/ci---

Milestone v0.3 complete. Checkpoint cleared for next milestone.
2026-08-01 20:07:06 +00:00
Jon Chery df58bc25a3 docs(milestone): complete scheduling-streaming (v0.3)
---ci---
project: orca
phase: 3
milestone: v0.3
status: complete
requirements:
  covered: [REQ-022, REQ-030, REQ-032]
  partial: []
---/ci---

v0.3 milestone merged to main. Includes all v0.2 work (P08-P10) that
was previously on the milestone branch but not yet merged to main, plus
the v0.3 completion work (iter.Seq streaming + doctor network/db).

v0.2 phases included: P08 (mTLS), P09 (scheduling), P10 (security scan).
v0.3 phases: P0 (pre-execution), P1 (iter.Seq streaming), P2 (doctor),
P3 (final review+ship).

Total: 40 requirements, all complete. No new go.mod dependencies.
Full test suite passes under -race. gofmt + go vet clean.
2026-08-01 20:06:47 +00:00
23 changed files with 2777 additions and 174 deletions
+10
View File
@@ -0,0 +1,10 @@
{
"phase": 3,
"stage": "complete",
"milestone": "v0.3",
"milestone_slug": "scheduling-streaming",
"phase_role": "final",
"attempts": 0,
"updated_at": "2026-08-01T00:30:00Z",
"milestone_complete": true
}
+644
View File
@@ -0,0 +1,644 @@
# Grill Report: Orca v0.3 — scheduling-streaming
**Date:** 2026-08-01
**Reviewer:** ci-griller (red-team, adversarial)
**Plan under review:** `.ciagent/PLAN_v0.3.md` (commit 89fa172)
**Branch:** `phase/00-pre-execution`
**Mode:** Full autonomy
---
## Methodology
Every claim in `PLAN_v0.3.md` and `RESEARCH_v0.3.md` was cross-checked against
the actual codebase (the 7 source files listed in the task, plus `migrate.go`,
`store.go`, `doctor_test.go`, `security/integration_test.go`, `security/ca.go`,
`security/csr.go`, `model/node.go`, `model/job.go`, and `cli/doctor.go`).
Findings are scored on 9 axes. Binding verdicts are ACCEPT (plan must change),
REJECT (concern noted, plan stands), or DEFER (address during execution).
---
## Summary Verdict
| Severity | Count |
|----------|-------|
| CRITICAL | 2 |
| HIGH | 2 |
| MEDIUM | 5 |
| LOW | 3 |
| **Total** | **12** |
**Overall verdict: PROCEED WITH CHANGES**
The plan is fundamentally sound — the scope is right-sized, the requirements
coverage is complete, the persona territories are respected, and the
no-new-dependencies promise holds. However, two CRITICAL findings require plan
changes before execution begins. Neither is a scope expansion; both are
correctness fixes to the design as written. With the 2 ACCEPT changes applied,
this plan is ready to execute.
---
## Per-Axis Findings
### Axis 1 — Feasibility (can each task actually be implemented?)
#### F-01 [CRITICAL] — Watch yields per-row but CLI table mode requires full-snapshot-per-tick
**Severity:** CRITICAL
**Axis:** Feasibility / Vertical slice integrity
**Binding verdict:** ACCEPT (plan must change)
**Finding:**
The plan is internally contradictory about what `Watch` yields.
- D-028 (RESEARCH:88) says Watch "yields the **full current snapshot** (one
element per row)."
- Task 01-01-01 (PLAN:30) says Watch "yields one `*model.Job` per row via
`scanJob`" — i.e., `iter.Seq[*model.Job]`, one element per row per tick.
- Task 01-02-02 (PLAN:42) says the CLI table render "collect the full snapshot
from `seq` into a `[]*model.Job`" then compares against the previous
snapshot's rendered table.
These are incompatible. `iter.Seq[*model.Job]` yields individual jobs with **no
tick-boundary signal**. The CLI ranging `for job := range seq` receives a flat
stream of jobs and cannot know when a tick's snapshot is complete. It cannot
collect "the full snapshot" because it cannot detect the end of a tick.
The JSON mode (01-02-03) can work without tick boundaries (per-element dedup
via `map[string][]byte`), but the **table mode cannot**. Table mode needs the
complete snapshot to render the table, clear the screen, and compare against the
previous frame.
**Evidence:**
- `RESEARCH_v0.3.md:106-142` — implementation yields `yield(j)` per row inside
`for rows.Next()`, not `yield(allJobs)` per tick.
- `PLAN_v0.3.md:30` — "yields one `*model.Job` per row"
- `PLAN_v0.3.md:42` — "collect the full snapshot from `seq` into a `[]*model.Job`"
- `PLAN_v0.3.md:44` (01-02-04) — nodeListCmd watch bypasses registry, same
per-row yield.
- D-028 says "full current snapshot" but the code yields per-row.
**Required change:**
Change the `Watch` element type from `iter.Seq[*model.Job]` to
`iter.Seq[[]*model.Job]` (and `iter.Seq[[]*model.Node]` analogously). Each tick
yields the **full snapshot as a single slice**. This:
1. Makes D-028 ("yields the full current snapshot") literally true.
2. Makes table mode trivial: `for snapshot := range seq { render(snapshot) }`.
3. Makes JSON mode cleaner: per-tick, diff the snapshot against the previous
one, emit one JSON line per changed element. This also enables a natural
`"delete"` event for elements that disappeared (not possible with per-row
yield).
4. Simplifies the test contract: `TestWatch_YieldsSnapshots` ranges over
`iter.Seq[[]*model.Job]` and each yield is a complete tick — no timing
ambiguity about "did I get all rows for this tick?"
**Impact on plan:**
- Tasks 01-01-01, 01-01-02: signature changes to
`iter.Seq[[]*model.Job]` / `iter.Seq[[]*model.Node]`. Implementation
collects all rows into a slice per tick, then `yield(slice)`.
- Task 01-02-02 (table): `for snapshot := range seq { ... }` — direct, no
collection needed.
- Task 01-02-03 (JSON): per-tick diff against previous snapshot's
`map[string][]byte`. Emit `"init"`/`"update"`/`"delete"` events.
- Task 01-01-04 (tests): assert each yield is a complete snapshot slice.
- D-026, D-028, D-046: update to reflect slice-per-tick semantics.
- Must-have criteria for 01-01-01/01-01-02: update signature assertions.
This is a mechanical change to the plan, not a scope change. The implementation
is simpler (no tick-boundary detection needed).
**Confidence:** 0.92
---
#### F-02 [CRITICAL] — First-tick delay: Watch waits a full interval before first yield
**Severity:** CRITICAL
**Axis:** Feasibility / UX correctness
**Binding verdict:** ACCEPT (plan must change)
**Finding:**
The Watch implementation (RESEARCH:110-116) has this structure:
```go
ticker := time.NewTicker(1 * time.Second)
defer ticker.Stop()
for {
select {
case <-ctx.Done(): return
case <-ticker.C: // <-- waits 1s BEFORE first query
}
// query + yield
}
```
The `select` waits for the first ticker pulse **before** running the first
query. With a 1s default interval, `orca job list --watch` shows **nothing for
1 full second**, then the first snapshot appears. For a CLI tool, a 1s blank
screen is a poor UX and looks broken. The user expects immediate output, then
refreshes every 1s.
The tests (01-01-04) use `watchInterval=10ms`, so the delay is only 10ms and
the test passes — but the test does NOT catch this UX bug because the interval
is tiny. In production (1s), the bug is visible.
**Evidence:**
- `RESEARCH_v0.3.md:110-116``select` before first query.
- `PLAN_v0.3.md:30` — "pull-based inline polling loop on a 1s ticker" — no
mention of immediate first yield.
- Standard `top`-like tools yield immediately, then tick.
**Required change:**
Add to tasks 01-01-01 and 01-01-02: the polling loop must **query and yield
immediately on the first iteration**, then `select` on the ticker for
subsequent ticks. Implementation shape:
```go
for {
// query + yield (runs immediately on first iteration)
rows, err := r.db.QueryContext(ctx, ...)
// ... yield snapshot ...
select {
case <-ctx.Done(): return
case <-ticker.C:
}
}
```
Or equivalently, query once before the loop, then loop with select-first. The
must-have criteria should add: "first yield occurs immediately (no
`watchInterval` delay before first snapshot)."
**Impact on plan:**
- Tasks 01-01-01, 01-01-02: add "immediate first yield" to description +
must-have.
- Task 01-01-04 (tests): add assertion that the first snapshot appears within
a short deadline (e.g., <50ms) even with `watchInterval=10ms` — proving the
first yield is not tick-gated.
**Confidence:** 0.95
---
#### F-03 [HIGH] — P01 and P02 both modify `internal/cli/node.go` (file-disjoint claim is false)
**Severity:** HIGH
**Axis:** Feasibility / Timeline (parallelism)
**Binding verdict:** ACCEPT (plan must change)
**Finding:**
D-042 (PLAN:133, RESEARCH:489) claims "P01 and P02 are file-disjoint — no file
is modified by both." This is **false**.
- P01 task 01-02-04 (PLAN:44) modifies `internal/cli/node.go` — adds `--watch`
flag + render modes to `nodeListCmd`.
- P02 task 02-01-01 (PLAN:76) modifies `internal/cli/node.go` — removes the
`dbPath` function and updates `openDB` to call `certpaths.DBPath()`.
Both phases touch `internal/cli/node.go`. If developed in parallel (as D-042
permits), this causes merge conflicts.
**Evidence:**
- `PLAN_v0.3.md:44` — 01-02-04 files: `internal/cli/node.go`
- `PLAN_v0.3.md:76` — 02-01-01 files: `internal/cli/node.go` (remove old
`dbPath`)
- `PLAN_v0.3.md:133` — "P01 and P02 are file-disjoint"
- Actual code: `internal/cli/node.go:22-28` defines `dbPath`; `:30-36`
defines `openDB` which calls `dbPath()`. `openDB` is used by 14 call sites
across `job.go`, `daemon.go`, `node_capacity.go`, `audit.go`, `node.go`.
**Required change:**
Update D-042 and the cross-phase notes (PLAN:131-133) to acknowledge the
overlap. Two options (pick one):
1. **Serialize:** P02 Wave 1 (02-01-01) runs before P01 Wave 2 (01-02-04).
P02 Wave 1 is a prerequisite for P01 Wave 2 on the `node.go` file. P01
Wave 1 (store layer) and P02 Wave 1 can still run in parallel.
2. **Merge the changes:** task 02-01-01 is folded into P01 Wave 2's
`node.go` modification (the cli-engineer updates `openDB` to use
`certpaths.DBPath()` while also adding `--watch`).
Recommended: Option 1 (serialize P02 Wave 1 before P01 Wave 2). It preserves
the wave structure and persona assignments. Update the cross-phase note to say:
"P02 Wave 1 (02-01-01) must complete before P01 Wave 2 (01-02-04) due to shared
`internal/cli/node.go` modification. P01 Wave 1 and P02 Wave 1 may run in
parallel."
**Confidence:** 0.90
---
#### F-04 [HIGH] — D-037 ServerName = node.Name assumption is fragile and unverified against real join flow
**Severity:** HIGH
**Axis:** Feasibility / Security
**Binding verdict:** DEFER (address in execution, with documentation)
**Finding:**
D-037 (RESEARCH:261, confidence 0.80) assumes `serverName = node.Name` for the
mTLS health probe. The TLS client's `ServerName` must match a SAN entry on the
peer's server cert. But `GenerateCSR(commonName, sans)` (csr.go:24) takes the
commonName and SANs as **separate arguments**. The commonName becomes the cert
Subject CN, but `ServerName` in `tls.Config` is matched against **SANs**
(DNSNames/IPAddresses), not the CN (per Go's `crypto/tls` behavior since Go
1.15).
If a node joined with `--name node-b` but its cert SAN is `localhost` (or an
IP), `serverName = "node-b"` will **fail the TLS handshake** with a
"certificate is valid for localhost, not node-b" error — even though the peer
is perfectly healthy.
The research (RESEARCH:261) says "confirmed in `integration_test.go:41`
`GenerateCSR("test-server", ...)`" — but that test uses `serverName =
"localhost"` (integration_test.go:83), which matches the SAN `localhost`, not
the commonName `test-server`. The test proves SAN-matching, not CN-matching.
**Evidence:**
- `internal/security/csr.go:24``GenerateCSR(commonName, sans)` — CN and
SANs are separate.
- `internal/security/integration_test.go:41` — `GenerateCSR("test-server",
[]string{"localhost", "127.0.0.1"})` — CN is "test-server", SANs are
localhost/127.0.0.1.
- `internal/security/integration_test.go:83` — `ClientTLSConfig(...,
"localhost", ...)` — serverName = "localhost" (a SAN), NOT "test-server"
(the CN).
- `internal/transport/mtls.go:49-51` — `serverName` is required and set as
`tls.Config.ServerName` (matched against SANs).
- `PLAN_v0.3.md:88` — 02-02-03: `serverName = n.Name`.
**Mitigation (DEFER to execution):**
1. Document the assumption in the `Network()` check message: "probing
<name> at <addr> (assuming cert SAN = node name)".
2. If the handshake fails with a SAN mismatch error, the FAIL message should
include the cert's actual SANs (parsed from the error) so the operator can
diagnose. This is a refinement, not a plan blocker.
3. The test 02-02-05(e) uses `Name = "localhost"` which matches the SAN — so
the test passes, but it doesn't prove the general case. Add a test comment
noting this assumption.
**Why DEFER not ACCEPT:** The assumption is documented (D-037, 0.80
confidence), the failure mode is graceful (FAIL with handshake error, not a
crash), and fixing it properly (storing SANs in the nodes table) is a scope
expansion beyond v0.3. The plan should note the limitation; execution should
add diagnostic context to the error message.
**Confidence:** 0.78
---
#### F-05 [MEDIUM] — `store.Open` runs migrations before integrity_check can run
**Severity:** MEDIUM
**Axis:** Feasibility / Testing
**Binding verdict:** REJECT (concern noted, plan stands)
**Finding:**
The DB check (02-02-01) calls `store.Open(path)` which runs `migrate(db)` (store
.go:39) before the integrity_check executes. On a truly corrupt DB, `store.Open`
fails at `Ping()` or `migrate()` — the integrity_check never runs. The check
returns FAIL with the open/migrate error, which is the correct outcome (a DB
that can't be opened is broken), but the message says "open <path>: <error>"
not "integrity_check failed."
The plan's `TestDBCheck_Corrupt` (RESEARCH:469) is explicitly called "brittle"
and made optional. The plan accepts that integrity_check is somewhat redundant
with `store.Open`'s own validation.
**Evidence:**
- `internal/store/store.go:37-41` — `db.Ping()` then `migrate(db)` inside
`Open`.
- `PLAN_v0.3.md:86` — 02-02-01: `db, err := store.Open(path)`.
- `RESEARCH_v0.3.md:455-456` — pitfall table acknowledges this.
**Why REJECT:** The failure surfaces correctly (FAIL with error message). The
integrity_check adds value for the case where the DB opens but has logical
corruption (e.g., foreign key violations, orphaned pages) that Ping/migrate
don't catch. The plan's approach is acceptable for v0.3. The optional corrupt
test is correctly deferred.
**Confidence:** 0.85
---
### Axis 2 — Scope
#### F-06 [MEDIUM] — No "delete" event in JSON watch mode (with per-row yield)
**Severity:** MEDIUM
**Axis:** Scope / Completeness
**Binding verdict:** DEFER (address in execution)
**Finding:**
With the current per-row `iter.Seq[*model.Job]` design (F-01), the JSON watch
mode (01-02-03) emits `"init"` and `"update"` events but has no way to emit
`"delete"` events — a job that disappears from the snapshot simply stops being
yielded, and the CLI has no tick boundary to detect "this ID was in the
previous tick but not this one."
With the F-01 fix (`iter.Seq[[]*model.Job]`, full snapshot per tick), `"delete"`
events become trivially possible: diff the previous snapshot's ID set against
the current snapshot's ID set. The plan should add `"delete"` event semantics
to D-046.
**Evidence:**
- `PLAN_v0.3.md:43` — 01-02-03: only `"init"` and `"update"` events.
- `PLAN_v0.3.md:139` — D-046: only `"init"` and `"update"`.
- Neither jobs nor nodes are hard-deleted in the current CLI (`node leave` sets
state to `left`, doesn't delete the row), so `"delete"` events are not
strictly needed for v0.3. But the `NodeRepo.Delete` method exists and could
be used by future code.
**Mitigation (DEFER):** If F-01 is accepted (slice-per-tick), add `"delete"`
event to D-046 as a natural extension. If F-01 is not accepted, document the
no-delete-event limitation explicitly.
**Confidence:** 0.70
---
### Axis 3 — Testing
#### F-07 [MEDIUM] — Test timing fragility: 10ms tick + 30ms insert + 80ms cancel
**Severity:** MEDIUM
**Axis:** Testing
**Binding verdict:** DEFER (address in execution)
**Finding:**
The store-layer tests (01-01-04) use `watchInterval=10ms` with timing-based
assertions: "insert a 2nd job from a goroutine after ~30ms, cancel ctx after
~80ms." Under CI load (especially with `-race` overhead), 10ms ticks can be
missed or delayed. A 10ms ticker pulse is not guaranteed to fire within 10ms
under load — the Go runtime scheduler may delay it. If the 2nd job is inserted
at 30ms but the 2nd tick fires at 45ms, the test might see the 2nd job in the
3rd tick (at ~55ms) which is still before the 80ms cancel — so it likely
passes, but it's fragile.
**Evidence:**
- `PLAN_v0.3.md:33` — 01-01-04: "after ~30ms", "after ~80ms".
- `time.NewTicker` does not guarantee exact timing under load.
**Mitigation (DEFER):** Use more generous margins (e.g., 50ms insert, 200ms
cancel) or a synchronization mechanism (e.g., insert the 2nd job, then poll
the collected slice with a 500ms timeout). The test hook (`watchInterval`)
already enables fast tests; the margins just need to be wider. Execution
should validate the tests pass reliably under `-race` in CI before marking
Wave 1 complete.
**Confidence:** 0.75
---
#### F-08 [LOW] — `-race` does not detect goroutine leaks; the plan claims it does
**Severity:** LOW
**Axis:** Testing
**Binding verdict:** REJECT (concern noted, plan stands)
**Finding:**
The plan (01-01-04 must-have, PLAN:33) says "`-race` reports no leaks/data
races." `go test -race` detects **data races**, not **goroutine leaks**.
Goroutine leak detection requires `goleak` or explicit goroutine-count
assertions. The claim is technically incorrect.
However, the actual risk is negligible: Watch does not spawn a goroutine
(D-032, inline pull loop). `time.NewTicker` spawns an internal goroutine, but
`defer ticker.Stop()` terminates it. There is nothing to leak. The
`no-goroutine-leak` constraint (data-engineer persona) is satisfied by
design, not by testing.
**Evidence:**
- `PLAN_v0.3.md:33` — "`-race` reports no leaks/data races"
- `RESEARCH_v0.3.md:92` — D-032: "No goroutine is spawned by Watch."
- Go `-race` detector documentation: detects concurrent access, not leaks.
**Why REJECT:** The claim is imprecise but the risk is zero by design.
Execution may optionally add `runtime.NumGoroutine()` before/after assertions
for belt-and-suspenders, but it's not required.
**Confidence:** 0.90
---
### Axis 4 — Security
#### F-09 [MEDIUM] — Doctor network check probes peers using the local server cert as client cert (confirmed valid, but undocumented)
**Severity:** MEDIUM
**Axis:** Security
**Binding verdict:** DEFER (document in execution)
**Finding:**
The plan (02-02-03, PLAN:88) uses `certpaths.ServerCertPath()`/`ServerKeyPath()`
as the client cert for the mTLS health probe. I verified this is **valid**:
`security/ca.go:255` signs server certs with
`ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth, x509.ExtKeyUsageClientAuth}`
— the server cert has both ServerAuth and ClientAuth EKUs, so it can be
presented as a client cert. The daemon's `RequireAndVerifyClientCert`
(security/tls_config.go:92) will accept it.
This is correct and feasible. The finding is that this cross-use (server cert
as client cert) is not documented in the plan or the security architecture. A
security auditor might flag it as "server cert used for client auth — is this
intended?"
**Evidence:**
- `internal/security/ca.go:255` — `ExtKeyUsage: ServerAuth, ClientAuth`.
- `internal/security/tls_config.go:92` — `ClientAuth: RequireAndVerifyClientCert`.
- `PLAN_v0.3.md:88` — 02-02-03 uses `ServerCertPath()`/`ServerKeyPath()`.
- `RESEARCH_v0.3.md:261` — D-037: "presents the local node's client cert."
**Mitigation (DEFER):** Add a code comment in `probeHealthz` and a note in
ARCHITECTURE.md §5 explaining that the local server cert doubles as the client
cert for doctor probes (justified by the dual EKU). This is documentation, not
a code change.
**Confidence:** 0.88
---
### Axis 5 — Performance
#### F-10 [LOW] — 1s poll ticker re-runs full List query every second; no concern but worth noting
**Severity:** LOW
**Axis:** Performance
**Binding verdict:** REJECT (concern noted, plan stands)
**Finding:**
The 1s ticker (D-019) re-runs `SELECT ... FROM jobs ORDER BY created_at DESC`
every second. For a CLI tool run by a human watching a terminal, this is
fine — the query is cheap (single table, no joins, indexed by `created_at` if
an index exists). For an AI agent tailing `--watch --json` for hours, this is
1 query/second × 3600 = 3600 queries/hour. SQLite handles this trivially in
WAL mode (store.go:36).
The cadence is correct for a "top-like" refresh. Faster (e.g., 100ms) would
waste CPU; slower (e.g., 5s) would feel sluggish. 1s is the right default.
**Evidence:**
- `PROJECT.md:116` — D-019: "Poll-based, 1s ticker" (confidence 0.90).
- `internal/store/store.go:36` — WAL mode enabled.
**Why REJECT:** The cadence is justified. No change needed.
**Confidence:** 0.92
---
### Axis 6 — Maintainability
#### F-11 [LOW] — `watchInterval` package var is mutable global state (test hook)
**Severity:** LOW
**Axis:** Maintainability
**Binding verdict:** REJECT (concern noted, plan stands)
**Finding:**
D-043 (PLAN:136) uses an unexported package var `watchInterval = 1 * time.Second`
in `internal/store`, overridable from `_test.go`. This is mutable global state —
if tests run in parallel within the `internal/store` package and one test sets
`watchInterval=10ms` while another expects `1s`, they interfere.
However, Go tests within a single package run **sequentially** by default
unless `t.Parallel()` is called. I verified no test in `internal/store` calls
`t.Parallel()` (grep found 0 matches). So the global var is safe as long as
no Watch test calls `t.Parallel()`. The plan should note this constraint.
**Evidence:**
- `PLAN_v0.3.md:32` — 01-01-03: "unexported package var `watchInterval`"
- `PLAN_v0.3.md:136` — D-043.
- grep for `t.Parallel()` in `internal/`: 0 matches.
**Why REJECT:** The approach is pragmatic and safe given sequential test
execution. The alternative (a `WatchWithInterval` constructor or an option
pattern) would leak test-only API into production, which D-043 explicitly
avoids. Execution should add a comment: "do not call t.Parallel() in Watch
tests — they share the watchInterval package var."
**Confidence:** 0.85
---
### Axis 7 — Completeness
#### F-12 [MEDIUM] — Plan does not address `openDB()` being the single chokepoint for dbPath relocation
**Severity:** MEDIUM
**Axis:** Completeness / Feasibility
**Binding verdict:** DEFER (clarify in execution)
**Finding:**
Task 02-01-01 (PLAN:76) says "Update `internal/cli/node.go` (and any other
`internal/cli` caller of the old unexported `dbPath`) to call
`certpaths.DBPath()`." This is imprecise. `dbPath()` is defined in
`node.go:22` and called only by `openDB()` in `node.go:31`. `openDB()` is
then called by 14 sites across `job.go`, `daemon.go`, `node_capacity.go`,
`audit.go`, `node.go`. The correct change is:
1. Add `certpaths.DBPath()`.
2. Change `openDB()` body from `store.Open(dbPath())` to
`store.Open(certpaths.DBPath())`.
3. Delete the `dbPath()` function from `node.go`.
No other caller needs changing — they all go through `openDB()`. The plan's
"any other `internal/cli` caller" language suggests a broader scan that isn't
needed. This is a clarity issue, not a correctness issue.
**Evidence:**
- `internal/cli/node.go:22-28` — `dbPath()` definition.
- `internal/cli/node.go:30-36` — `openDB()` calls `dbPath()`.
- grep `openDB()`: 14 call sites, all in `internal/cli/`.
- grep `dbPath()`: only in `node.go:31` (inside `openDB`).
**Mitigation (DEFER):** Execution should note that `openDB()` is the single
chokepoint — update its body and delete `dbPath()`. No other file needs
changes. The plan's must-have ("`internal/cli` no longer defines `dbPath`")
is correct.
**Confidence:** 0.88
---
### Axis 8 — Vertical Slice Integrity
Covered by F-01 (the tick-boundary problem breaks the Wave 1 → Wave 2
vertical slice: Wave 1 produces `iter.Seq[*model.Job]` which Wave 2's table
mode cannot consume correctly). With F-01's fix (`iter.Seq[[]*model.Job]`),
the vertical slice is clean: Wave 1 yields full snapshots, Wave 2 renders
them.
### Axis 9 — Risk
**Highest-risk task:** 02-02-05(e) `TestNetworkCheck_PeerReachable` —
integration test requiring CA bootstrap, server cert signing with correct
SAN, httptest TLS server with `RequireAndVerifyClientCert`, node row insert,
and mTLS probe. Has the most moving parts and the most assumptions (D-037
ServerName, dual-EKU client cert, httptest HTTP/1.1 vs h2c quirks per
integration_test.go:100-126). If D-037 is wrong in production (not in test,
since the test uses `Name = "localhost"` matching the SAN), the network check
fails for real deployments but the test passes — a false-positive.
**What could go catastrophically wrong:** The F-01 tick-boundary issue, if
not caught, would cause `orca job list --watch` (table mode) to either hang
(trying to collect a "full snapshot" that never completes) or render
incomplete tables (rendering after each row instead of after a full tick).
This is a user-visible broken feature shipped as "complete."
---
## Binding Decisions (G-series)
| ID | Decision | Rationale | Confidence | Verdict |
|----|----------|-----------|------------|---------|
| G-001 | Change `Watch` to `iter.Seq[[]*model.Job]` / `iter.Seq[[]*model.Node]` (full snapshot per tick) | F-01: per-row yield has no tick boundary; table mode needs full snapshot. Slice-per-tick makes D-028 literally true and simplifies both render modes. | 0.92 | ACCEPT |
| G-002 | Watch must yield immediately on first iteration, then tick | F-02: current design waits 1s before first output. Unacceptable UX. | 0.95 | ACCEPT |
| G-003 | P02 Wave 1 (02-01-01) must complete before P01 Wave 2 (01-02-04) — shared `internal/cli/node.go` | F-03: D-042 file-disjoint claim is false for `node.go`. | 0.90 | ACCEPT |
| G-004 | D-037 ServerName = node.Name assumption is deferred; execution must add diagnostic context to handshake-fail errors | F-04: assumption is documented (0.80), failure is graceful, proper fix is out of v0.3 scope. | 0.78 | DEFER |
| G-005 | `store.Open` runs migrations before integrity_check — acceptable | F-05: failure surfaces correctly as FAIL. | 0.85 | REJECT |
| G-006 | Add `"delete"` event to JSON watch mode if G-001 is accepted | F-06: slice-per-tick makes delete events trivial. | 0.70 | DEFER |
| G-007 | Widen test timing margins (10ms tick → generous insert/cancel margins) | F-07: 10ms ticker under CI load is fragile. | 0.75 | DEFER |
| G-008 | `-race` does not detect goroutine leaks — claim is imprecise but risk is zero by design | F-08: no goroutine spawned. | 0.90 | REJECT |
| G-009 | Document dual-EKU (server cert as client cert) in `probeHealthz` + ARCHITECTURE.md | F-09: valid but undocumented. | 0.88 | DEFER |
| G-010 | 1s poll ticker cadence is correct | F-10: justified by D-019. | 0.92 | REJECT |
| G-011 | `watchInterval` package var is safe (no `t.Parallel` in store tests) | F-11: pragmatic, avoids leaking test API. | 0.85 | REJECT |
| G-012 | `openDB()` is the single chokepoint for dbPath relocation — clarify in execution | F-12: plan is imprecise but correct. | 0.88 | DEFER |
---
## Escalations
None. All 12 findings are resolved with confidence ≥ 0.60 (either ACCEPT,
REJECT, or DEFER). No axis requires human escalation.
---
## Overall Verdict
**PROCEED WITH CHANGES**
The plan is approved for execution **after** the 3 ACCEPT binding verdicts
(G-001, G-002, G-003) are applied to `PLAN_v0.3.md`:
1. **G-001:** Change `Watch` element type to `iter.Seq[[]*model.Job]` /
`iter.Seq[[]*model.Node]` (full snapshot per tick). Update tasks
01-01-01, 01-01-02, 01-02-02, 01-02-03, 01-02-04, 01-01-04, and decisions
D-026, D-028, D-046.
2. **G-002:** Add "immediate first yield" to tasks 01-01-01, 01-01-02 and
must-have criteria + test assertion in 01-01-04.
3. **G-003:** Update D-042 and cross-phase notes: P02 Wave 1 (02-01-01)
precedes P01 Wave 2 (01-02-04) due to shared `internal/cli/node.go`.
The 5 DEFER items (G-004, G-006, G-007, G-009, G-012) are execution-time
refinements that do not block the plan.
The scope is right-sized (21 tasks across 5 waves, 2 execution phases + 1
review phase). No requirements gaps exist between REQ-022/030/032 and the plan
tasks. The no-new-dependencies promise holds. The persona territories are
respected. The test strategy is adequate (with the timing-margin note in
G-007). The plan does not violate the minimalist pillar.
**Confidence in verdict:** 0.88
+79 -44
View File
@@ -10,29 +10,44 @@ deactivated_personas:
- frontend-engineer
- devops-sre
phase_specific:
- cli-engineer
- data-engineer
- security-engineer
- network-engineer
- cli-engineer
reason: |
Orca is a CLI-first, offline-first orchestration engine with no web UI and
a single-binary distribution model. The persona roster reflects this:
a single-binary distribution model. The v0.3 milestone is a 2-phase
completion milestone (iter.Seq streaming + doctor network/db) that touches
the CLI, store, doctor, transport, and security layers. The persona roster
reflects this:
- lead-developer: coordination and task decomposition
- backend-engineer: core engine and API handlers
- data-engineer: SQLite state store and migrations
- cli-engineer: Cobra subcommands and CLI UX
- security-engineer: mTLS, cert lifecycle, audit logging, input validation
- network-engineer: transport layer, dispatcher, peer-to-peer resilience
- lead-developer: coordination, task decomposition, territory adjudication
(e.g. D-039 dbPath relocation between cli-engineer territory and the
doctor package).
- backend-engineer: daemon health endpoint surface that the doctor network
check probes; transport dispatch client reuse.
- data-engineer: iter.Seq[Job|Node] on the store repos (P01) and the
migration-version query + PRAGMA integrity_check in the store layer (P02).
- cli-engineer: the --watch flag on `orca job list` / `orca node list`
(P01) and the doctor subcommand wiring (P02).
- security-engineer: mTLS client config reuse for the doctor network probe
(P02) — TLS config is the security-engineer territory per v0.2.
- network-engineer: the doctor /healthz probe over mTLS reuses the
transport layer (P02) — connection lifecycle / peer reachability is the
network-engineer territory.
Deactivated:
- frontend-engineer: no web UI in v0.1
- devops-sre: no container/cloud integrations; release flow is
handled by CoreCI (not a persona territory)
- frontend-engineer: no web UI in Orca (v0.1 onward). NOT relevant to v0.3.
- devops-sre: no container/cloud integrations; release flow is handled by
CoreCI (not a persona territory).
Phase-specific (v0.2):
- security-engineer: P01 (mTLS/CA) + P02 (peer transport hardening)
- network-engineer: P02 only (multi-node scheduling & dispatch)
- cli-engineer: P04 only (--watch flag is a CLI concern)
Phase-specific (v0.3):
- cli-engineer: P01 (--watch flag is a CLI surface) + P02 (doctor
subcommand wiring).
- data-engineer: P01 (iter.Seq on store repos) + P02 (migration version +
integrity check in store layer).
- security-engineer: P02 only (mTLS client config for doctor network probe).
- network-engineer: P02 only (mTLS /healthz probe over transport).
---
# Personas: Orca
@@ -52,21 +67,23 @@ reason: |
- **Constraints**: `API-first`, `error-handling`, `minimal-dependencies`, `security-first`
- **Territory**: `**/api/**`, `**/*_handler*`, `**/*_handler.go`, `internal/daemon/**`
- **Active**: true
- **Reason**: Owns the daemon health endpoints (`/healthz`, `/readyz`) that the P02 doctor network check probes. The transport dispatch client (reused by doctor) lives in `internal/transport` but the *handler* surface is backend-engineer territory.
### data-engineer
- **Domain**: data
- **Frameworks**: `modernc/sqlite`
- **Constraints**: `schema-first`, `migration-safe`, `local-storage-only`
- **Territory**: `**/store/**`, `**/model.go`, `**/migration*`, `migrations/**`, `internal/store/migrations/0004_certs.sql`
- **Frameworks**: `modernc/sqlite`, `iter`
- **Constraints**: `schema-first`, `migration-safe`, `local-storage-only`, `no-goroutine-leak`
- **Territory**: `**/store/**`, `**/model.go`, `**/migration*`, `migrations/**`, `internal/store/migrations/**`
- **Active**: true
- **Reason**: Owns the `iter.Seq[Job|Node]` implementations on `JobRepo`/`NodeRepo` (P01) and the `MigrationVersion` query + `PRAGMA integrity_check` helper (P02). Added `iter` to frameworks and `no-goroutine-leak` to constraints (the iter.Seq polling loop must not leak — see RESEARCH_v0.3.md D-032). Territory confirmed against actual file structure: `internal/store/` holds all repos + `migrations/` subdir with `0001..0005_*.sql`.
### cli-engineer (custom)
- **Domain**: CLI/UX
- **Frameworks**: `cobra`, `pflag`
- **Constraints**: `discoverable-help`, `consistent-flag-naming`, `human-readable-output`, `machine-readable-json-flag`
- **Constraints**: `discoverable-help`, `consistent-flag-naming`, `human-readable-output`, `machine-readable-json-flag`, `signal-handling`
- **Territory**: `cmd/**`, `internal/cli/**`, `internal/commands/**`
- **Active**: true
- **Reason**: Orca is CLI-first; this persona ensures CLI quality and discoverability.
- **Reason**: Orca is CLI-first; this persona ensures CLI quality and discoverability. For v0.3 P01 it owns the `--watch` flag on `orca job list` / `orca node list` (signal.NotifyContext cancellation, table refresh vs streaming JSON). For P02 it owns the `internal/cli/doctor.go` subcommand wiring (replacing NetworkStub/DBStub calls). Added `signal-handling` to constraints (ctrl-c propagation to iter.Seq is a P01 correctness requirement). Territory confirmed: `internal/cli/` holds all Cobra commands.
### security-engineer (custom)
- **Domain**: security
@@ -74,53 +91,71 @@ reason: |
- **Constraints**: `no-panic-in-production`, `structured-audit-logging`, `no-secret-in-logs`, `input-validation`, `least-privilege`
- **Territory**: `**/auth/**`, `**/audit/**`, `internal/security/**`, `internal/transport/**` (TLS config only)
- **Active**: true
- **Reason**: mTLS, audit logging, and input validation are first-class concerns.
- **Phase scope**: P01 (mTLS + internal CA), P02 (transport hardening for peer handshakes). Deactivates after P02 ships — P03/P04 have lighter security needs.
- **Reason**: mTLS, audit logging, and input validation are first-class concerns. For v0.3 P02, the doctor network check reuses `security.ClientTLSConfig` (via `transport.NewMTLSClient`) to build the mTLS client that probes peer `/healthz`. The TLS-config portion of `internal/transport/**` remains security-engineer territory.
- **Phase scope**: P02 only (mTLS client config for doctor network probe). P01 has no security surface.
### network-engineer (custom, NEW in v0.2)
- **Domain**: networking
- **Frameworks**: `net/http`, `crypto/tls` (via `internal/security`), `iter`
- **Constraints**: `connection-resilience`, `retry-with-backoff`, `graceful-disconnect`, `context-propagation`
- **Territory**: `**/transport/**`, `**/engine/dispatcher*`, `**/engine/peer*`, `internal/engine/dispatcher.go`, `internal/transport/**`
- **Constraints**: `connection-resilience`, `retry-with-backoff`, `graceful-disconnect`, `context-propagation`, `bounded-probe-timeout`
- **Territory**: `**/transport/**`, `**/engine/dispatcher*`, `**/engine/peer*`, `internal/engine/dispatcher.go`, `internal/engine/peer.go`, `internal/transport/**`
- **Active**: true
- **Reason**: v0.2 introduces cross-node dispatch and peer-to-peer transport. This persona owns the transport layer, dispatcher, and peer lifecycle concerns that are distinct from the API-handler territory of `backend-engineer`.
- **Phase scope**: P02 only. Deactivates after P02 ships.
- **Reason**: Owns the transport layer and peer-to-peer connection lifecycle. For v0.3 P02, the doctor network check is a read-only mTLS `/healthz` probe that reuses `transport.MTLSClient` — the connection lifecycle (dial, per-probe 3s timeout, handshake) is network-engineer territory. Added `bounded-probe-timeout` to constraints (doctor must not stall on one slow peer — RESEARCH_v0.3.md D-038). Territory confirmed: `internal/transport/` holds mtls.go, dispatch.go, retry.go, idempotency.go, handshake_log.go.
- **Phase scope**: P02 only (doctor network probe reuses transport layer).
### frontend-engineer
- **Active**: false
- **Reason**: No web UI in v0.1.
- **Reason**: No web UI in Orca (v0.1 onward). NOT relevant to v0.3 — v0.3 adds no UI surface. Confirmed deactivated.
### devops-sre
- **Active**: false
- **Reason**: No container/cloud integrations. Release flow is handled by CoreCI.
- **Reason**: No container/cloud integrations. Release flow is handled by CoreCI (not a persona territory). Confirmed deactivated.
## Territory Enforcement
- **Mode**: `warn` (per `config.json`)
- **Behavior**: Out-of-territory file changes log a warning but do not block.
- **Rationale**: Allows flexibility during early development; tighten to `strict` post-v0.1.
- **Rationale**: Allows flexibility during early development; tighten to `strict` post-v0.1. For v0.3, the main territory-overlap risk is D-039 (moving `dbPath` from `internal/cli` to `internal/certpaths`) which crosses cli-engineer and the shared-infra concern — lead-developer adjudicates.
## Phase-Specific Personas (v0.2)
## Phase-Specific Personas (v0.3)
| Persona | Active in | Reason |
|---------|-----------|--------|
| `security-engineer` | P01, P02 | mTLS/CA in P01, transport hardening in P02. Lighter security needs in P03 (CI scanning) and P04 (streaming UX). |
| `network-engineer` | P02 | Multi-node dispatch is a P02 concern only. P01 builds the transport primitives but P02 wires them into cross-node scheduling. |
| `cli-engineer` | P04 | The `--watch` flag is a CLI surface; P01-P03 don't add new CLI commands. |
| `cli-engineer` | P01, P02 | P01: `--watch` flag is a CLI surface (signal handling, table/JSON render). P02: doctor subcommand wiring in `internal/cli/doctor.go`. |
| `data-engineer` | P01, P02 | P01: `iter.Seq[Job|Node]` on the store repos + the no-leak polling loop. P02: `MigrationVersion` query + `PRAGMA integrity_check` in the store layer. |
| `security-engineer` | P02 | mTLS client config reuse for the doctor network probe. P01 has no security surface. |
| `network-engineer` | P02 | mTLS `/healthz` probe over the transport layer (connection lifecycle, per-probe timeout). P01 has no network surface. |
In full-autonomy mode, all personas are auto-accepted and the phase-scope
assignments are applied automatically when a phase is committed.
## Migration from v0.1
## v0.3 vs v0.2 Persona Diff
| Change | Rationale |
|--------|-----------|
| `data-engineer` frameworks: added `iter` | P01 introduces `iter.Seq[T]` on the store repos — a new stdlib framework surface for this persona. |
| `data-engineer` constraints: added `no-goroutine-leak` | The iter.Seq polling loop must not leak goroutines (inline pull loop, defer ticker.Stop, rows.Close on every path — RESEARCH D-032). |
| `cli-engineer` constraints: added `signal-handling` | P01 requires `signal.NotifyContext` for ctrl-c propagation to iter.Seq (D-031). |
| `network-engineer` constraints: added `bounded-probe-timeout` | P02 doctor network check must bound each peer probe (3s) so one slow peer doesn't stall diagnostics (D-038). |
| `network-engineer` phase scope: was P02-only (v0.2), now P02-only (v0.3) | Same persona, different phase content — v0.3 P02 is doctor network, not multi-node dispatch. |
| `security-engineer` phase scope: was P01+P02 (v0.2), now P02-only (v0.3) | v0.3 has no new cert/CA work; security surface is limited to reusing the existing mTLS client config in doctor. |
| `frontend-engineer` | Remains deactivated (no UI in v0.3). |
| `devops-sre` | Remains deactivated (CoreCI handles release). |
## Migration from v0.2
- `backend-engineer` territory unchanged: `internal/daemon/**` still owns HTTP
handlers. The new `internal/transport/**` package is shared with
`network-engineer` but `transport` owns the *connection lifecycle* (dial,
retry, close) while `daemon` owns the *request handlers*.
- `data-engineer` territory expanded to include the new
`internal/store/migrations/0004_certs.sql` migration in P01.
- `security-engineer` territory extended from `internal/security/**` to
include the TLS-config portion of `internal/transport/**` (the
`NewServerTLSConfig` / `NewClientTLSConfig` helpers).
- `cli-engineer` territory unchanged; the new `orca cert` subcommands in P01
fall under the existing `internal/cli/**` glob.
handlers. The `/healthz` endpoint that the doctor network check probes is
backend-engineer territory; the *probing* client is network-engineer.
- `data-engineer` territory expanded scope: still owns `internal/store/**` but
now adds the `iter.Seq` polling implementations (P01) and a public
`MigrationVersion` query (P02).
- `security-engineer` territory unchanged: `internal/security/**` + the TLS
config portion of `internal/transport/**`. The doctor network check calls
into `security.ClientTLSConfig` indirectly via `transport.NewMTLSClient`
no new security-engineer files, just reuse.
- `cli-engineer` territory unchanged: `internal/cli/**`. P01 modifies
`job.go` and `node.go`; P02 modifies `doctor.go`. The `dbPath` relocation
(D-039) moves a 5-line function out of `internal/cli/node.go` into
`internal/certpaths` — cli-engineer territory loses one function, shared
infra gains it.
+159
View File
@@ -0,0 +1,159 @@
# Plan: Orca v0.3 — scheduling-streaming
Milestone v0.3 (scheduling-streaming) — completion milestone closing the two
work items deferred from v0.2 (iter.Seq streaming + doctor network/db). Two
execution phases (P01, P02) followed by one final phase (P03 review + ship).
Branch: `phase/00-pre-execution` (cut from `milestone/v0.3-scheduling-streaming`).
Go toolchain: `go1.25.0` (`iter` package + range-over-func are stable stdlib).
No new `go.mod` dependencies (D-041). Source of implementation guidance:
`.ciagent/RESEARCH_v0.3.md` (D-025..D-042).
---
## Phase P01: iter.Seq streaming for `--watch` flags
**Goal:** Add pull-based `iter.Seq` streaming to `orca job list` and `orca node list` behind a `--watch` flag, with table refresh (default) or streaming one-line JSON per event (`--watch --json`).
**Requirements:** REQ-022 (`iter.Seq` for streaming job lists, Go 1.25+), REQ-030 (`--watch` output format: table default vs streaming one-line JSON per event)
**Milestone:** v0.3
**Phase tag:** v0.3.1
**Key decisions:** D-019 (1s poll ticker), D-025 (iter.Seq on store repos), D-026 (`*model.Job`/`*model.Node` element type), D-028 (poll re-runs List, yields full snapshot), D-030 (per-event JSON streaming), D-031 (signal.NotifyContext replaces 5s timeout on watch path), D-032 (inline pull loop, no goroutine).
### Wave 1: Store layer iter.Seq + unit tests (vertical slice)
Wave 1 is independently testable: the two `Watch` methods + the migration-version-less store layer compile and run in isolation. No CLI or doctor code is touched. Running `go test ./internal/store/...` after this wave passes and exercises the `iter.Seq` contracts (yield, ctx cancellation, consumer break, no goroutine leak).
| Task ID | Description | Persona | Files | Must-have | Deps |
|---------|-------------|---------|-------|-----------|------|
| 01-01-01 | Add `JobRepo.Watch(ctx) iter.Seq[[]*model.Job]` — pull-based inline polling loop on a 1s ticker; re-runs the List `SELECT ... FROM jobs ORDER BY created_at DESC` each tick, collects ALL rows into a `[]*model.Job` slice via `scanJob`, then yields the **full snapshot as a single slice** (`yield(snapshot)`). **Immediate first yield** before the first ticker wait (G-002): the loop queries+yields on the first iteration, then `select`s on the ticker for subsequent ticks. `defer ticker.Stop()` + `rows.Close()` on every exit path (ctx.Done, yield==false, scan error). No goroutine spawned (D-032). Transient query errors are logged via `slog.Default().Warn` and the loop continues to the next tick (D-034 lite). Imports: add `"iter"` (`"time"` already present). | data-engineer | `internal/store/job_task_repo.go` | `go build ./internal/store/...` succeeds; `Watch` method exists with signature `func (r *JobRepo) Watch(ctx context.Context) iter.Seq[[]*model.Job]`; code path closes rows on ctx.Done and on `yield==false`; first yield is immediate (no `watchInterval` delay before first snapshot — G-002). | - |
| 01-01-02 | Add `NodeRepo.Watch(ctx) iter.Seq[[]*model.Node]` — analogous to 01-01-01 but against the nodes query `SELECT id, name, address, state, joined_at, last_seen, metadata FROM nodes ORDER BY joined_at ASC`, reusing `scanNode`. Yields the full snapshot as a `[]*model.Node` slice per tick. Same inline-pull / no-goroutine / rows-close-on-all-paths / immediate-first-yield contract (G-001, G-002). | data-engineer | `internal/store/node_repo.go` | `go build ./internal/store/...` succeeds; `Watch` method exists with signature `func (r *NodeRepo) Watch(ctx context.Context) iter.Seq[[]*model.Node]`; immediate first yield. | - |
| 01-01-03 | Add an unexported test hook for the poll interval so unit tests are deterministic (D-035). Preferred shape: an unexported package var `watchInterval = 1 * time.Second` in `internal/store` that `Watch` reads instead of a literal, overridable from `_test.go` via `watchInterval = 10 * time.Millisecond`. Both `JobRepo.Watch` and `NodeRepo.Watch` reference this var. | data-engineer | `internal/store/job_task_repo.go`, `internal/store/node_repo.go` (optionally a tiny `internal/store/watch_test_helper_test.go` if a shared helper reads cleaner) | `Watch` uses the `watchInterval` var, not a literal `1 * time.Second`; tests can set it to a small value. | 01-01-01, 01-01-02 |
| 01-01-04 | Write store-layer unit tests for `Watch`. New/append: `internal/store/job_task_repo_test.go` and `internal/store/node_repo_test.go` (mirror). Tests: (a) `TestJobRepoWatch_YieldsSnapshots` — insert 1 job, set `watchInterval=10ms`, range over seq collecting `[]*model.Job` snapshots into a slice, insert a 2nd job from a goroutine after ~30ms, cancel ctx after ~80ms, assert at least one snapshot contains both jobs and the first snapshot contains only the first job (G-001: each yield is a complete tick snapshot). (b) `TestJobRepoWatch_ImmediateFirstYield` (G-002) — assert the first snapshot appears within <50ms even with `watchInterval=10ms` (proving first yield is not tick-gated). (c) `TestJobRepoWatch_StopsOnConsumerBreak` — range and `break` after first yield; assert the range returns (no hang) within a short deadline. (d) `TestJobRepoWatch_StopsOnCtxCancel` — cancel ctx; assert the range loop exits within ~50ms. (e) Mirror all four for `NodeRepo.Watch`. Run `go test -race ./internal/store/...`. | data-engineer | `internal/store/job_task_repo_test.go`, `internal/store/node_repo_test.go` | `go test -race ./internal/store/...` passes; all 8 Watch tests pass; `-race` reports no leaks/data races; immediate-first-yield assertion holds (G-002). | 01-01-03 |
### Wave 2: CLI `--watch` flag + integration
Wave 2 depends on Wave 1's `Watch` methods. It wires the `--watch` flag into both list commands, implements the two output modes, and adds CLI-level smoke tests. After this wave `orca job list --watch` and `orca node list --watch` are runnable end-to-end.
| Task ID | Description | Persona | Files | Must-have | Deps |
|---------|-------------|---------|-------|-----------|------|
| 01-02-01 | Add `--watch` flag to `jobListCmd` in `internal/cli/job.go`. Register `jobListCmd.Flags().BoolVar(&jobWatch, "watch", false, "stream jobs until Ctrl-C")` (package var `jobWatch bool`). In `RunE`, branch: if `!jobWatch` keep the existing 5s-timeout `List` path unchanged; if `jobWatch`, build `ctx, cancel := signal.NotifyContext(cmd.Context(), os.Interrupt, syscall.SIGTERM); defer cancel()` (drop the 5s timeout — D-031), then `seq := store.NewJobRepo(db).Watch(ctx)`. Imports: `os/signal`, `syscall`, `iter`. The branch structure is added in this task; the actual rendering (table vs JSON) is filled by 01-02-02 + 01-02-03. | cli-engineer | `internal/cli/job.go` | `go build ./internal/cli/...` succeeds; `orca job list --help` shows `--watch` flag; non-watch path behavior unchanged (existing tests pass); watch path compiles (rendering may be a placeholder `for range seq {}` at this step). | 01-01-01 |
| 01-02-02 | Implement the **table** watch render in `jobListCmd` (D-029, G-001). Each yielded value is a complete `[]*model.Job` snapshot. Maintain the previous snapshot's rendered-table string (or a hash of it). On each tick, if the new rendered table differs from the previous, emit `"\033[2J\033[H"` (clear screen + home) then the table header + rows (reuse the existing table-rendering code path). Unchanged snapshots produce no output (avoids flicker). | cli-engineer | `internal/cli/job.go` | `orca job list --watch` on a temp DB: inserting a job causes a cleared-screen re-render showing the new job; no output when the snapshot is unchanged. | 01-02-01 |
| 01-02-03 | Implement the **`--watch --json`** render in `jobListCmd` (D-030, G-001). Each yielded value is a complete `[]*model.Job` snapshot. Maintain `map[string][]byte` of last-seen compact-JSON bytes per job ID. Per tick: diff the current snapshot against the map — for each job in the snapshot, marshal compact JSON; if it differs from stored bytes (or ID unseen), print `{"event":"init","job":{...}}\n` (first sighting) or `{"event":"update","job":{...}}\n` (subsequent change). For IDs in the map but NOT in the current snapshot, print `{"event":"delete","job":{...}}\n` (G-006 DEFER: delete event now natural with snapshot-per-tick). One line per changed element per tick — matches REQ-030. | cli-engineer | `internal/cli/job.go` | `orca job list --watch --json` on a temp DB: inserting/changing a job prints one JSON line per changed job; first tick prints `"init"` lines for existing jobs; deleting a job prints `"delete"`; unchanged jobs on a tick produce no line. | 01-02-01 |
| 01-02-04 | Add `--watch` flag + both render modes to `nodeListCmd` in `internal/cli/node.go`, mirroring 01-02-01..01-02-03. Package var `nodeWatch bool`; flag `--watch`. For the watch path bypass `engine.NodeRegistry` and call `store.NewNodeRepo(db).Watch(ctx)` directly (D-025 — keeps iter boundary in store; registry adds no value for a read-only stream). Same `signal.NotifyContext` cancellation. Table + JSON renders analogous to job (element type `[]*model.Node` snapshot per tick, event wrapper `{"event":"...","node":{...}}`). **Note (G-003):** This task modifies `internal/cli/node.go`, which P02 task 02-01-01 also modifies (removing old `dbPath`). Task 02-01-01 MUST complete first to avoid merge conflicts. | cli-engineer | `internal/cli/node.go` | `orca node list --watch` and `orca node list --watch --json` behave as specified; non-watch path unchanged. | 01-01-02, 01-02-03, 02-01-01 |
| 01-02-05 | Add CLI-level watch tests. New files (or append if present): `internal/cli/job_test.go`, `internal/cli/node_test.go`. Smoke-level (store layer is the thorough test home): `TestJobListWatch_JSONStreaming` — temp DB, insert a job, run `jobListCmd.RunE` with `--watch --json` in a goroutine under a cancellable ctx, insert a 2nd job, capture stdout for ~200ms, assert ≥2 JSON lines appear, then cancel ctx and assert the command returns promptly. `TestJobListWatch_TableRefresh` — assert the clear-screen escape `\033[2J\033[H` appears in output on change. Mirror for nodes. Keep deterministic: small `watchInterval` via the test hook, short timeouts. Run `go test -race ./internal/cli/...`. | cli-engineer | `internal/cli/job_test.go`, `internal/cli/node_test.go` | `go test -race ./...` passes (whole repo); the 4 CLI watch smoke tests pass; no goroutine leaks under `-race`. | 01-02-02, 01-02-03, 01-02-04 |
### Test strategy (P01)
- **Store layer (thorough, deterministic):** `internal/store/job_task_repo_test.go` + `internal/store/node_repo_test.go` — three tests per repo (snapshots over time, consumer-break stops, ctx-cancel stops). Uses the `watchInterval` test hook (10ms) for speed. Runs under `go test -race ./internal/store/...`. This is where the `iter.Seq` contract is verified rigorously.
- **CLI layer (smoke):** `internal/cli/job_test.go` + `internal/cli/node_test.go` — verify the flag is wired, JSON streaming emits one line per changed element, table mode emits the clear-screen escape on change, and the command exits promptly on ctx cancellation. Kept intentionally lightweight; deterministic via the shared `watchInterval` hook + short timeouts.
- **Non-watch regression:** existing `job list` / `node list` tests must still pass unchanged (the 5s-timeout path is untouched).
- **Race:** `go test -race ./...` is the gate (REQ-031 already enforces `-race` in CI).
### Vertical slice integrity (P01)
- **Wave 1** produces a runnable, testable artifact: `go build ./internal/store/...` + `go test -race ./internal/store/...`. No CLI or doctor code is modified. The `iter.Seq` contract (pull, cancel, no-leak) is fully verified at this layer.
- **Wave 2** builds on Wave 1's `Watch` methods to deliver the user-facing `--watch` flag end-to-end. After Wave 2 an operator can demo `orca job list --watch` and `orca node list --watch --json`.
---
## Phase P02: `orca doctor` network + db full implementation
**Goal:** Replace the `NetworkStub` and `DBStub` placeholders with real diagnostics — peer reachability via mTLS `/healthz` probe and SQLite `PRAGMA integrity_check` + migration version — completing REQ-032.
**Requirements:** REQ-032 (completion: network reachability + db integrity)
**Milestone:** v0.3
**Phase tag:** v0.3.2
**Key decisions:** D-027 (closure-capture handles in check constructors), D-033 (`store.MigrationVersion`), D-034 (db check opens its own `*sql.DB`), D-035 (`PRAGMA integrity_check` + migration version), D-036 (peers from `nodes` table, not in-memory registry), D-037 (`ServerName = node.Name`), D-038 (zero peers → WARN, any fail → FAIL, 3s per-probe timeout), D-039 (`dbPath``certpaths.DBPath()`), D-040 (delete stubs, no shims).
### Wave 1: Shared infra + store migration-version query (vertical slice)
Wave 1 breaks the would-be `doctor → cli` import cycle (D-039) and adds the public `store.MigrationVersion` query. Both are independently testable: `go test ./internal/store/... ./internal/certpaths/...` passes after this wave, and the foundation for both the db and network checks is in place.
| Task ID | Description | Persona | Files | Must-have | Deps |
|---------|-------------|---------|-------|-----------|------|
| 02-01-01 | Move `dbPath()` (the `ORCA_DB`-env-honoring path resolver) from `internal/cli` to `internal/certpaths` as `DBPath()` (D-039). `certpaths` already owns the `ORCA_HOME`-honoring `Dir()`. Add `func DBPath() string` to `internal/certpaths/certpaths.go`: honors `ORCA_DB` env override, else `filepath.Join(Dir(), "orca.db")`. Update `internal/cli/node.go` (and any other `internal/cli` caller of the old unexported `dbPath`) to call `certpaths.DBPath()`; remove the old `dbPath` from `internal/cli`. Territory note: this crosses cli-engineer territory — lead-developer adjudicates (D-039). | lead-developer | `internal/certpaths/certpaths.go`, `internal/cli/node.go` (remove old `dbPath`), any other `internal/cli/*` caller | `go build ./...` succeeds (no import cycle); `certpaths.DBPath()` exists and honors `ORCA_DB`/`ORCA_HOME`; `internal/cli` no longer defines `dbPath`; existing CLI tests pass. | - |
| 02-01-02 | Add `store.MigrationVersion(ctx, db) (string, error)` to `internal/store/migrate.go` (D-033). SQL: `SELECT name FROM schema_migrations ORDER BY name DESC LIMIT 1`. Returns `("", nil)` on `sql.ErrNoRows` (empty/fresh db). Wraps other errors with `fmt.Errorf("query migration version: %w", err)`. Add `"context"` import if missing (likely already imported). | data-engineer | `internal/store/migrate.go` | `go build ./internal/store/...` succeeds; function is exported; `sql.ErrNoRows` maps to `("", nil)`. | - |
| 02-01-03 | Test `MigrationVersion`. New/append `internal/store/migrate_test.go`: `TestMigrationVersion` — open a fresh test db via `store.Open` (which runs `migrate`), call `MigrationVersion`, assert it returns `0005_node_capacity.sql` (the highest current migration). Then manually `db.Exec("DELETE FROM schema_migrations")`, call again, assert `("", nil)`. Run `go test -race ./internal/store/...`. | data-engineer | `internal/store/migrate_test.go` | `go test -race ./internal/store/...` passes; both assertions (highest version, empty → `""`) hold. | 02-01-02 |
### Wave 2: doctor DB check + network check + CLI wiring + tests
Wave 2 depends on Wave 1 (`certpaths.DBPath` + `store.MigrationVersion`). It replaces both stubs with real checks, updates `All()` and the CLI subcommands, rewrites the broken stub test, and adds per-check tests. After this wave `orca doctor`, `orca doctor network`, and `orca doctor db` are fully functional.
| Task ID | Description | Persona | Files | Must-have | Deps |
|---------|-------------|---------|-------|-----------|------|
| 02-02-01 | Replace `DBStub()` with `DB()` in `internal/doctor/doctor.go` (D-027, D-034, D-035). The `Check.Run` closure: `path := certpaths.DBPath()`; `db, err := store.Open(path)` (defer `db.Close()`); `PRAGMA integrity_check` via `db.QueryRowContext(ctx, "PRAGMA integrity_check").Scan(&integrity)`; FAIL if not `"ok"` (first line of message); then `store.MigrationVersion(ctx, db)` — WARN if `""` (fresh/never-migrated), else PASS with `"... migrations up to <ver>"`. New imports: `strings`, `internal/store`, `internal/certpaths`. | data-engineer | `internal/doctor/doctor.go` | `go build ./internal/doctor/...` succeeds; `doctor.DB()` returns a `Check` with `Name=="db"`; `DBStub` still present (removed in 02-02-04 lockstep). | 02-01-01, 02-01-02 |
| 02-02-02 | Add `probeHealthz(ctx, caPath, certPath, keyPath, serverName, addr) error` helper in `internal/doctor/doctor.go` (network-engineer territory — connection lifecycle). Builds an mTLS client via `transport.NewMTLSClient(caPath, serverName, certPath, keyPath)` (D-037: `serverName = node.Name`), `http.NewRequestWithContext(ctx, GET, "https://"+addr+"/healthz", nil)`, `client.Do(req)`, defer `resp.Body.Close()`, FAIL if status != 200. New imports: `net/http`, `time`, `internal/transport`. | network-engineer | `internal/doctor/doctor.go` | `go build ./internal/doctor/...` succeeds; `probeHealthz` exists with the specified signature; reuses `transport.NewMTLSClient` (no new TLS code — security-engineer territory respected). | 02-01-01 |
| 02-02-03 | Replace `NetworkStub()` with `Network()` in `internal/doctor/doctor.go` (D-036, D-037, D-038). The `Check.Run` closure: `path := certpaths.DBPath()`; `db, err := store.Open(path)` (defer close); `nodes, err := store.NewNodeRepo(db).List(ctx)`; filter `state != model.NodeStateLeft` into `live`; if `len(live)==0``ResultWarn` ("no peers registered; network check skipped (single-node?)"). Else per-peer: `probeCtx, cancel := context.WithTimeout(ctx, 3*time.Second)`; `probeHealthz(...)`; collect PASS/FAIL lines; aggregate — FAIL if any peer failed, PASS if all OK. `serverName = n.Name`, `caPath = certpaths.CACertPath()`, `certPath/keyPath = certpaths.ServerCertPath()/ServerKeyPath()`. New imports: `internal/model`. | network-engineer | `internal/doctor/doctor.go` | `go build ./internal/doctor/...` succeeds; `doctor.Network()` returns a `Check` with `Name=="network"`; zero-peer → WARN; per-probe 3s timeout enforced. | 02-02-02 |
| 02-02-04 | Update `All()` in `internal/doctor/doctor.go` to use `Network()` and `DB()` instead of the stubs (D-040). **Delete** `NetworkStub` and `DBStub` (no backward-compat shims — internal only). Update `internal/cli/doctor.go`: `doctorNetworkCmd.RunE` calls `doctor.Network()` (was `NetworkStub()`); `doctorDBCmd.RunE` calls `doctor.DB()` (was `DBStub()`). Ensure per-subcommand render honors `jsonOutput` (minor enhancement, in scope). | cli-engineer | `internal/doctor/doctor.go`, `internal/cli/doctor.go` | `go build ./...` succeeds; `grep -r "NetworkStub\|DBStub" internal/` returns nothing; `orca doctor`, `orca doctor network`, `orca doctor db` run without referencing stubs. | 02-02-01, 02-02-03 |
| 02-02-05 | Rewrite + add doctor tests in `internal/doctor/doctor_test.go`. (a) Rewrite `TestRunAllChecksWithNoCA` — set `ORCA_HOME` to a temp dir with no CA. Assert per-check by name: `cert.ca` FAIL, `cert.server` FAIL, `cert.expiry` FAIL, `cert.fingerprint` FAIL, `db` PASS (store.Open runs migrations → version 0005), `network` WARN (no peers). Remove the stale "expects WARN (stubs)" comment. (b) `TestDBCheck_IntegrityOK` — fresh db via `store.Open` in temp, run `doctor.DB().Run(ctx)`, expect PASS, message contains "0005". (c) `TestNetworkCheck_NoPeers` — fresh db, no nodes, run `doctor.Network().Run(ctx)`, expect WARN. (d) `TestNetworkCheck_PeerUnreachable` — insert a node with `Address = "127.0.0.1:1"` (nothing listening), run, expect FAIL with the peer name in the message. (e) `TestNetworkCheck_PeerReachable` (integration) — bootstrap a CA via `security.CAInit`-equivalent, generate+sign a server cert with SAN `localhost`, start an `httptest.NewUnstartedServer` with TLS + `ClientAuth=RequireAndVerifyClientCert` (mirror `security/integration_test.go` pattern), insert a node row with `Address = ts.Listener.Addr().String()` and `Name = "localhost"`, set `ORCA_HOME`, run `doctor.Network().Run(ctx)`, expect PASS. Run `go test -race ./internal/doctor/...`. | network-engineer (network tests), data-engineer (db test), cli-engineer (All() rewrite test) | `internal/doctor/doctor_test.go` | `go test -race ./internal/doctor/...` passes; all 5 test cases pass; the stale stub assertion is gone. | 02-02-04 |
### Test strategy (P02)
- **Store layer:** `internal/store/migrate_test.go``MigrationVersion` returns highest applied migration (`0005_node_capacity.sql`) and `""` on empty. (`-race`.)
- **Doctor db check:** `TestDBCheck_IntegrityOK` — fresh db → PASS with version in message. (Optional brittle `TestDBCheck_Corrupt` may be added if a reliable corruption method is found; otherwise rely on the integrity-string parsing logic via the PASS/FAIL branch coverage.)
- **Doctor network check:** `TestNetworkCheck_NoPeers` (WARN), `TestNetworkCheck_PeerUnreachable` (FAIL, peer name in message), `TestNetworkCheck_PeerReachable` (integration: real mTLS `httptest` server → PASS). The reachable test reuses the proven `TestEndToEndMTLS` pattern from `security/integration_test.go`.
- **Doctor `All()` regression:** rewritten `TestRunAllChecksWithNoCA` asserts per-check results (certs FAIL, db PASS, network WARN) — no more global "hasWarn" stub assertion.
- **Race:** `go test -race ./...` is the gate.
### Vertical slice integrity (P02)
- **Wave 1** produces a runnable, testable artifact: `certpaths.DBPath()` (cycle broken) + `store.MigrationVersion` (tested). `go test -race ./internal/store/... ./internal/certpaths/...` passes. No doctor code depends on the stubs being changed yet.
- **Wave 2** builds on Wave 1 to deliver the real `DB()` and `Network()` checks, wires the CLI, and replaces the stub tests. After Wave 2 an operator can demo `orca doctor` showing real PASS/WARN/FAIL for db and network.
---
## Phase P03 (final): review + ship + audit
**Goal:** Review the v0.3 milestone for completeness against REQ-022/030/032, audit the codebase for leftover stubs/dead code, run the full CI gate (`go test -race ./...`, `gosec`, `govulncheck`, `gitleaks`), tag the milestone release, and ship.
**Requirements:** REQ-022 (verify complete), REQ-030 (verify complete), REQ-032 (verify complete)
**Milestone:** v0.3
**Phase tag:** v0.3.3 (= milestone release; target milestone tag `v0.4.0` per ROADMAP next-minor rule)
### Wave 1: Review + audit + ship (single wave)
| Task ID | Description | Persona | Files | Must-have | Deps |
|---------|-------------|---------|-------|-----------|------|
| 03-01-01 | Verify REQ coverage: confirm REQ-022 (iter.Seq streaming job lists), REQ-030 (--watch table/JSON modes), REQ-032 (doctor network + db) are fully implemented. Update `REQUIREMENTS.md` status for REQ-022/030/032 from Pending/Partial → **Complete**. Cross-check against the plan's must-have criteria. | lead-developer | `.ciagent/REQUIREMENTS.md` | All three REQs marked Complete with phase references; no remaining "stub" or "Pending" status for v0.3 scope. | P01, P02 complete |
| 03-01-02 | Codebase audit: `grep -r "NetworkStub\|DBStub" internal/` returns nothing; `grep -r "TODO\|FIXME" internal/` reviewed (no v0.3 leftovers); confirm no `dbPath` duplication remains in `internal/cli`; confirm `iter` import is used (no unused imports); run `go vet ./...`. | lead-developer | (read-only audit; edits only if cleanup needed) | `go vet ./...` clean; no stub references; no leftover TODOs for v0.3 scope. | 03-01-01 |
| 03-01-03 | Full CI gate: `go build ./...`, `go test -race ./...`, `gosec` (vs baseline JSON), `govulncheck ./...` (offline mode per REQ-027), `gitleaks` (vs baseline per REQ-029). Fix any new findings. | lead-developer | (fixes if needed) | All gates green; no new gosec findings beyond baseline; govulncheck exit 0; gitleaks clean vs baseline. | 03-01-02 |
| 03-01-04 | Tag + ship: per `.ciagent/RELEASE_POLICY.md`, tag `v0.3.3` (phase tag) and the milestone tag (next-minor per ROADMAP). Produce Gitea release. Update `ROADMAP.md` v0.3 section to mark P01/P02/P03 complete. | lead-developer | `.ciagent/ROADMAP.md` | `v0.3.3` tag exists; Gitea release published; ROADMAP v0.3 checkboxes updated. | 03-01-03 |
### Test strategy (P03)
- No new tests; this phase is review + audit + release.
- The gate is the existing test suite + security scans all passing under CI.
---
## Cross-phase notes
- **Phase ordering / parallelism (D-042, revised by G-003):** P01 Wave 1 and P02 Wave 1 may run in parallel (file-disjoint: store repos vs certpaths+migrate). **P02 Wave 1 (02-01-01) MUST complete before P01 Wave 2 (01-02-04)** because both modify `internal/cli/node.go` (P01 adds `--watch`, P02 removes old `dbPath`). P01 Wave 2 tasks 01-02-01..01-02-03 (job.go only) are not blocked by P02. Recommended order: P01 W1 + P02 W1 in parallel → P02 W1 02-01-01 completes → P01 W2 (job.go tasks) + P02 W2 in parallel → P01 W2 01-02-04 (node.go) after 02-01-01. P03 is strictly sequential after both phases complete.
- **No new dependencies (D-041):** `iter` (P01) and the mTLS health probe (P02) use stdlib + existing internal packages only. The 4 direct `go.mod` deps (cobra, hcl/v2, modernc/sqlite, uuid) are unchanged.
- **Decisions logged during planning (new, this plan):**
- **D-043** — `watchInterval` test hook: an unexported package var in `internal/store` (default `1 * time.Second`) referenced by both `Watch` methods, overridable from `_test.go`. Avoids a public `WatchWithInterval` constructor that would leak test-only API into production. Confidence 0.90.
- **D-044** — P01 Wave 1 / Wave 2 split: Wave 1 = store-layer `Watch` methods + tests (data-engineer only, fully isolated); Wave 2 = CLI `--watch` flag + renders + CLI tests (cli-engineer). This keeps the `iter.Seq` contract verifiable without the CLI and matches persona territories. Confidence 0.93.
- **D-045** — P02 Wave 1 / Wave 2 split: Wave 1 = `certpaths.DBPath()` relocation + `store.MigrationVersion` + tests (breaks the import cycle, data-engineer + lead-developer); Wave 2 = real `DB()`/`Network()` checks + CLI wiring + doctor tests. Wave 1 is the unblock for both checks. Confidence 0.91.
- **D-046** — `--watch --json` event wrapper shape: `{"event":"update","job":{...}}` / `{"event":"init","job":{...}}` (and `node` analog). `"init"` on first sighting of an ID, `"update"` on subsequent change. Unchanged IDs on a tick emit nothing. Confidence 0.80 (matches D-030's per-event interpretation of REQ-030).
## Grill amendments (binding ACCEPT verdicts applied)
Three binding changes from `.ciagent/GRILL_v0.3.md` have been applied to this plan:
- **G-001 [CRITICAL]** — `Watch` element type changed from `iter.Seq[*model.Job]` (per-row) to `iter.Seq[[]*model.Job]` (full snapshot per tick). Each tick yields the complete snapshot as a single slice. This makes table render mode correct (clear-screen + re-render needs full snapshot) and enables natural `"delete"` events in JSON mode. Applied to tasks 01-01-01, 01-01-02, 01-02-02, 01-02-03, 01-02-04, 01-01-04.
- **G-002 [CRITICAL]** — `Watch` must yield immediately on the first iteration, then `select` on the ticker for subsequent ticks. Prevents a 1s blank-screen UX bug in production (tests with 10ms interval missed this). Applied to tasks 01-01-01, 01-01-02; new test `TestWatch_ImmediateFirstYield` added to 01-01-04.
- **G-003 [HIGH]** — D-042's "file-disjoint" claim corrected: both P01 (01-02-04) and P02 (02-01-01) modify `internal/cli/node.go`. P02 Wave 1 task 02-01-01 is now a dependency of P01 Wave 2 task 01-02-04. Cross-phase ordering updated.
DEFER items (G-004, G-006, G-007, G-009, G-012) are noted in the grill report and will be addressed during execution.
## Summary
- **Phases:** 3 (P01, P02 execution; P03 final review/ship)
- **Waves:** P01 = 2 waves (4 + 5 tasks); P02 = 2 waves (3 + 5 tasks); P03 = 1 wave (4 tasks). Total = 5 waves.
- **Tasks:** P01 = 9, P02 = 8, P03 = 4. Total = 21 tasks.
- **Grill amendments:** G-001 (snapshot-per-tick), G-002 (immediate first yield), G-003 (serialize P02 W1 → P01 W2 on node.go). 3 ACCEPT verdicts applied.
- **New planning decisions:** D-043 (watchInterval test hook), D-044 (P01 wave split), D-045 (P02 wave split), D-046 (JSON event wrapper shape).
- **Requirements closed:** REQ-022, REQ-030 (P01); REQ-032 (P02, completion).
- **No new go.mod dependencies.** No source code written in this plan — implementation begins at P01 Wave 1.
+37
View File
@@ -104,3 +104,40 @@ auto-resolved under full autonomy and are summarized here:
(pull-based, ctx cancellation, ctrl-c via `signal.NotifyContext`).
- **D-018: Bin-packing by CPU/memory with FIFO within node; JSON-over-HTTP
orca.v1.Dispatch for cross-node** (no ConnectRPC dep).
## v0.3 Clarified Decisions (D-series, full autonomy)
v0.3 is a lean 2-execution-phase milestone completing the streaming and
doctor work deferred from v0.2. The 6 v0.3 decisions (D-019..D-024)
were auto-resolved under full autonomy:
| ID | Question | Decision | Rationale | Confidence |
|----|----------|----------|-----------|------------|
| D-019 | Watch refresh mechanism? | **Poll-based, 1s ticker** | Simpler than event channel; no daemon coupling for CLI; matches offline-first. | 0.90 |
| D-020 | Watch output format (REQ-030)? | **Table by default; `--watch --json` streams one-line JSON per event** | Consistent with D-005 `--json` convention; serves humans + AI agents. | 0.92 |
| D-021 | Doctor network check scope? | **Probe configured peer addresses via mTLS `/healthz` handshake; PASS/WARN/FAIL per peer** | Reuses existing transport client; read-only. | 0.85 |
| D-022 | Doctor db check scope? | **`PRAGMA integrity_check` + migration version query** | Already specced in ARCHITECTURE.md §5; minimal surface. | 0.95 |
| D-023 | iter.Seq cancellation? | **`signal.NotifyContext` on SIGINT/SIGTERM** | Per D-017 + ARCHITECTURE Flow 4. | 0.95 |
| D-024 | `--watch` applies to job list only, or node list too? | **Both `orca job list --watch` and `orca node list --watch`** | Per ARCHITECTURE.md CLI layer + D-017. | 0.92 |
## v0.3 Scope Summary
v0.3 is a focused 2-execution-phase milestone completing the work
deferred from v0.2 that was NOT already shipped in P08-P10. A codebase
audit during re-init SPECIFY confirmed that REQ-014, REQ-027, REQ-028,
REQ-029, REQ-031, REQ-037, REQ-039, REQ-040 all shipped in P08-P10
despite stale REQUIREMENTS.md marking them Pending. The remaining work:
- **P01 — `iter.Seq` streaming for `--watch` flags.** Go 1.25+
range-over-func semantics, pull-based `iter.Seq[Job]` /
`iter.Seq[Node]`, `context.Context` cancellation,
`signal.NotifyContext` on ctrl-c. Applies to both `orca job list
--watch` and `orca node list --watch`. Covers REQ-022, REQ-030.
- **P02 — `orca doctor` network + db full implementation.** Replaces
the P01 stubs (`NetworkStub`, `DBStub`) with real checks: peer
reachability via mTLS `/healthz` probe; SQLite `PRAGMA
integrity_check` + migration version. Covers REQ-032 (completion).
The vision ("minimalist, offline-first, CLI-first orchestration
engine") is unchanged. v0.3 is a completion milestone, not a direction
change.
+25 -17
View File
@@ -21,7 +21,7 @@ earlier versions of this file.
| REQ-011 | mTLS for inter-node communication | Medium | **v0.2 P01** | **Complete** (P01 shipped v0.2.1) |
| REQ-012 | `~/.orca/config.hcl` and `/etc/orca/orca.hcl` config locations | Low | v0.1 P01 | **Complete** (CLI uses `~/.orca/` + `ORCA_DB` env) |
| REQ-013 | Pre-push git hook triggers CoreCI on every push | High | v0.1 P01 | **Complete** |
| REQ-014 | `gosec` + `govulncheck` in CI pipeline | High | v0.2 P03 | Pending (P03) |
| REQ-014 | `gosec` + `govulncheck` in CI pipeline | High | v0.2 P03 | **Complete** (P10 shipped v0.2.3) |
| REQ-015 | MIT LICENSE | Low | v0.1 P01 | **Complete** |
| REQ-016 | README.md with quickstart | Medium | v0.1 P01 | **Complete** |
| REQ-017 | `context.Context` propagation in all I/O | High | v0.1 | **Complete** |
@@ -29,25 +29,25 @@ earlier versions of this file.
| REQ-019 | Cobra CLI framework | High | v0.1 P01 | **Complete** |
| REQ-020 | HCL parser integration (`hashicorp/hcl`) | Medium | v0.1 P03 | **Complete** |
| REQ-021 | `os/exec` with `WaitDelay` (Go 1.25+) | Medium | v0.1 P03 | **Complete** |
| REQ-022 | `iter.Seq` for streaming job lists (Go 1.25+) | Low | v0.2 P04 | Pending (P04) |
| REQ-022 | `iter.Seq` for streaming job lists (Go 1.25+) | Low | **v0.3 P01** | **Complete** (v0.3 P01 shipped v0.3.1) |
| REQ-023 | Self-signed mTLS cert generation | Medium | **v0.2 P01** | **Complete** (P01 shipped v0.2.1) |
| REQ-024 | `Makefile` with standard targets | High | v0.1 P01 | **Complete** |
| REQ-025 | Bounded cert rotation history: retain last N=3 server certs per node for rollback | Medium | **v0.2 P01** | **Complete** (P01 shipped v0.2.1) |
| REQ-026 | Trusted-CA fingerprint pinned in config; daemon refuses to start on mismatch | High | **v0.2 P01** | **Complete** (P01 shipped v0.2.1) |
| REQ-027 | `govulncheck` runs in offline mode in CI (no `vuln.go.dev` calls; pre-mirrored DB or `-format json` + `jq` gate) | High | v0.2 P03 | Pending (P03) |
| REQ-028 | HCL/YAML schema for `NodeCapacity` declaration (`orca node join` flag and/or `~/.orca/node.hcl`) | High | v0.2 P02 | Pending (P02) |
| REQ-029 | `gitleaks` baseline file committed to repo to suppress pre-existing `.env` SHA-1 leak in git history | Medium | v0.2 P03 | Pending (P03) |
| REQ-030 | `--watch` output format mode: table (default) vs streaming one-line JSON per event | Low | v0.2 P04 | Pending (P04) |
| REQ-031 | `go test -race` enabled in CI for all v0.2 packages | High | v0.2 P01P04 | **Complete** for P01 (cross-cutting, verified P01); P02P04 ongoing |
| REQ-032 | `orca doctor` subcommand for diagnostics (CA/cert health, db integrity, peer reachability) | Medium | **v0.2 P01** | **Complete** for cert checks (P01); network/db are stubs, full impl in P02 |
| REQ-027 | `govulncheck` runs in offline mode in CI (no `vuln.go.dev` calls; pre-mirrored DB or `-format json` + `jq` gate) | High | v0.2 P03 | **Complete** (P10 shipped v0.2.3) |
| REQ-028 | HCL/YAML schema for `NodeCapacity` declaration (`orca node join` flag and/or `~/.orca/node.hcl`) | High | v0.2 P02 | **Complete** (P09 shipped v0.2.2; `orca node capacity` CLI) |
| REQ-029 | `gitleaks` baseline file committed to repo to suppress pre-existing `.env` SHA-1 leak in git history | Medium | v0.2 P03 | **Complete** (P10 shipped v0.2.3) |
| REQ-030 | `--watch` output format mode: table (default) vs streaming one-line JSON per event | Low | **v0.3 P01** | **Complete** (v0.3 P01 shipped v0.3.1) |
| REQ-031 | `go test -race` enabled in CI for all v0.2 packages | High | v0.2 P01P04 | **Complete** (P10; `.coreci.yml` test pipeline runs `-race`) |
| REQ-032 | `orca doctor` subcommand for diagnostics (CA/cert health, db integrity, peer reachability) | Medium | **v0.2 P01 / v0.3 P02** | **Complete** (cert checks P01 v0.2.1; network + db P02 v0.3.2) |
| REQ-033 | Cert file mode enforcement: 0600 for keys, 0644 for certs (refuses to start on violation) | High | **v0.2 P01** | **Complete** (P01 shipped v0.2.1) |
| REQ-034 | Cert proactive rotation alarm: structured slog WARN 30 days before `not_after` | Medium | **v0.2 P01** | **Complete** (P01 shipped v0.2.1) |
| REQ-035 | `orca cert show` redacts private key material from default and `--json` output | High | **v0.2 P01** | **Complete** (P01 shipped v0.2.1) |
| REQ-036 | Server cert SAN validation: SAN entries (DNS + IP) populated at sign-time; refuses to sign a CSR without them | High | **v0.2 P01** | **Complete** (P01 shipped v0.2.1) |
| REQ-037 | `X-Orca-Idempotency-Key` header on cross-node POST; dispatcher retries only when header is present | Medium | v0.2 P02 | Pending (P02) |
| REQ-037 | `X-Orca-Idempotency-Key` header on cross-node POST; dispatcher retries only when header is present | Medium | v0.2 P02 | **Complete** (P09 shipped v0.2.2; `internal/transport/idempotency.go`) |
| REQ-038 | Structured slog fields for mTLS failures: `event=mtls.handshake`, `peer`, `cert_fp`, `err` | Medium | **v0.2 P01** | **Complete** (P01 shipped v0.2.1) |
| REQ-039 | `.gitleaks.toml` extended with stopwords for test data paths and CA cert PEM blocks | Medium | v0.2 P03 | Pending (P03) |
| REQ-040 | `.golangci.yml` unified lint config superseding per-tool invocations | Low | v0.2 P03 | Pending (P03) |
| REQ-039 | `.gitleaks.toml` extended with stopwords for test data paths and CA cert PEM blocks | Medium | v0.2 P03 | **Complete** (P10 shipped v0.2.3) |
| REQ-040 | `.golangci.yml` unified lint config superseding per-tool invocations | Low | v0.2 P03 | **Complete** (P10 shipped v0.2.3) |
## v0.1 Milestone Summary
@@ -62,13 +62,21 @@ Plus REQ-025..REQ-040 (16 net-new) added by v0.2 IDEATE stage.
## v0.2 Milestone Summary
**Status: In Progress** — P01 (mTLS) shipped (v0.2.1). 3 phases remain
(P02 multi-node scheduling, P03 gosec+govulncheck+gitleaks, P04 iter.Seq).
P01 covered REQ-011, REQ-023, REQ-025, REQ-026, REQ-031, REQ-032 (partial),
REQ-033, REQ-034, REQ-035, REQ-036, REQ-038 (10 REQs complete; REQ-032
complete for cert checks only).
**Status: Functionally Complete (pending merge to main)** — P08 (mTLS),
P09 (scheduling), P10 (security scan) all shipped to the
`milestone/v0.2-networking-observability-security` branch as v0.2.1,
v0.2.2, v0.2.3. The milestone branch has NOT been merged to main yet.
REQ-022/030 (iter.Seq streaming) and REQ-032 (doctor network/db) were
deferred to v0.3.
## Deferred to v0.3
## v0.3 Milestone Summary
**Status: Complete** — P01 (iter.Seq streaming, v0.3.1) and P02 (doctor
network+db, v0.3.2) both shipped. REQ-022, REQ-030, REQ-032 all complete.
Re-init SPECIFY audit confirmed all other v0.2-deferred REQs (014, 027,
028, 029, 031, 037, 039, 040) already shipped in P08-P10.
## Deferred to v0.4
- pprof endpoint on `orca daemon` (idea I-308, 0.70 confidence): deferred
to keep v0.2 lean; revisit in v0.3 once P02's dispatcher is stable.
+506
View File
@@ -0,0 +1,506 @@
# Research: Orca v0.3 — scheduling-streaming
Phase: 0 (research) for milestone v0.3 (scheduling-streaming).
Branch: `phase/00-pre-execution` (cut from `milestone/v0.3-scheduling-streaming`).
Go toolchain: `go1.25.0` (confirmed via `go version`; `go.mod` declares `go 1.25.0`).
This document provides concrete, file-level implementation guidance for the
two v0.3 execution phases:
- **P01** — `iter.Seq` streaming for `--watch` flags (REQ-022, REQ-030)
- **P02** — `orca doctor` network + db full implementation (REQ-032 completion)
All assumptions are logged as decisions (D-025..D-038) with confidence scores.
Full autonomy mode — no items flagged for human validation.
---
## Codebase Audit Summary
### Current state (commit ba5ffd7 + phase docs)
| Area | File | Key finding |
|------|------|-------------|
| CLI `job list` | `internal/cli/job.go:112-143` | `jobListCmd.RunE` calls `store.NewJobRepo(db).List(ctx)`, prints a fixed-width table; `--json` via `printJSON(jobs)`. No `--watch` flag exists. |
| CLI `node list` | `internal/cli/node.go:155-186` | `nodeListCmd.RunE` calls `registry.List(ctx)``repo.List(ctx)`. Table + `--json`. No `--watch` flag. |
| CLI root | `internal/cli/root.go` | `jsonOutput` is a package-level `bool` set by `--json` persistent flag. `printJSON` uses `json.NewEncoder` with 2-space indent. |
| Job repo | `internal/store/job_task_repo.go` | `JobRepo.List(ctx) ([]*model.Job, error)` — single-shot query, closes rows. `scanJob` helper is reusable. |
| Node repo | `internal/store/node_repo.go` | `NodeRepo.List(ctx) ([]*model.Node, error)`. `scanNode` helper is reusable. `scanner` interface defined here (`Scan(dest ...any) error`) — shared by `*sql.Row` and `*sql.Rows`. |
| Store open | `internal/store/store.go` | `store.Open(path)` opens with `?_pragma=journal_mode(WAL)&_pragma=foreign_keys(ON)` and runs `migrate(db)`. |
| Migrations | `internal/store/migrate.go` | `migrate` is unexported, runs at `Open` time. `schema_migrations` table tracks applied migrations by filename. Migrations are embedded via `//go:embed migrations/*.sql`. No public API to query migration version. |
| Migrations on disk | `internal/store/migrations/` | `0001_nodes.sql`, `0002_jobs_tasks.sql`, `0003_audit_log.sql`, `0004_certs.sql`, `0005_node_capacity.sql`. Highest = 0005. |
| Doctor | `internal/doctor/doctor.go` | `NetworkStub()` and `DBStub()` return WARN stubs. `All()` aggregates 6 checks. `Run(ctx)` iterates checks. `Check.Run` signature: `func(ctx context.Context) (Result, string)`. `Result` is `PASS|WARN|FAIL`. No DB or transport imports — cert-only. |
| Doctor CLI | `internal/cli/doctor.go` | `doctorNetworkCmd`/`doctorDBCmd` call `doctor.NetworkStub()`/`doctor.DBStub()` directly. |
| Doctor tests | `internal/doctor/doctor_test.go` | Two tests: `TestRunAllChecksWithNoCA` (expects FAIL + WARN for stubs), `TestRunWithCAAndServerCert` (cert checks PASS). Uses `t.Setenv("ORCA_HOME", dir)`. **The "expects WARN" assertion will break when stubs become real checks** — must be updated in P02. |
| Transport mTLS client | `internal/transport/mtls.go` | `NewMTLSClient(caPath, serverName, certPath, keyPath)` builds an `http.Client` with a TLS-1.3-only config from `security.ClientTLSConfig`. `MTLSClient.Do(req)`. `DialContext` for low-level TLS dial. |
| Transport dispatch client | `internal/transport/dispatch.go` | `NewDispatchClient(caPath, serverName, peerAddr)` wraps `MTLSClient`. `PeerAddr` is `http://` or `https://`. `Submit`/`Status` POST to `/orca.v1.Dispatch/*`. No `/healthz` GET helper. |
| Daemon health | `internal/daemon/health.go` | `handleHealthz` → 200 `{"status":"alive"}`. `handleReadyz` → 200/503 with db ping. Mounted at `mux.HandleFunc("/healthz", ...)` in `server.go:104`. |
| Daemon TLS | `internal/daemon/tls.go` | `StartMTLS(state)` sets `httpServer.TLSConfig` with `ClientAuth = RequireAndVerifyClientCert`. The daemon **requires client certs** in mTLS mode. |
| Peer registry | `internal/engine/peer.go` | `PeerRegistry` is **in-memory only** (`map[string]*Peer` under `sync.RWMutex`). `NewPeerRegistry()` returns empty. `Peer` has `NodeID, Address, ServerName, CAPath, LastSeen, Capacity`. **Not persisted to SQLite.** |
| Peer registry usage | `internal/cli/job.go:70`, `internal/cli/daemon.go:47` | Both create a **fresh empty** `NewPeerRegistry()` per process. No code ever calls `peers.Add(...)`. The registry is currently a structural placeholder. |
| Node registry | `internal/engine/registry.go` | `NodeRegistry` wraps `store.NodeRepo` + `Audit`. `List(ctx)``repo.List(ctx)`. Persisted to `nodes` table. |
| Node model | `internal/model/node.go` | `Node{ID, Name, Address, State, JoinedAt, LastSeen, Metadata}`. **No `ServerName` or `CAPath` field**`model.Node` differs from `engine.Peer`. |
| Cert paths | `internal/certpaths/certpaths.go` | `Dir()` honors `ORCA_HOME`; `CACertPath()`, `ServerCertPath()`, `ServerKeyPath()`. |
| Security client TLS | `internal/security/tls_config.go:106` | `ClientTLSConfig(caPath, serverName, certPath, keyPath)` — TLS 1.3 only, AEAD allowlist, `RootCAs` = single CA. Both-or-neither for cert/key. |
| Go version | `go.mod` + `go version` | `go 1.25.0``iter` package and range-over-func are stable stdlib. |
| Deps | `go.mod` | cobra, hcl/v2, modernc/sqlite, uuid. **No new deps needed for v0.3.** `iter` is stdlib. |
### Critical gap analysis
1. **`PeerRegistry` is non-persistent and always empty at CLI time.** The
doctor network check cannot rely on it — there is no code path that populates
it. The `nodes` table IS persisted and has `Address`, but lacks the
`ServerName`/`CAPath` needed for an mTLS probe. **Resolution: doctor network
reads the `nodes` table via `NodeRepo.List`, and derives `ServerName` +
`CAPath` from local config (`certpaths.CACertPath()` + node name/addr).**
See D-029.
2. **`doctor.Run` / `Check.Run` do not plumb a `*sql.DB` or transport client.**
The cert checks are filesystem-only. P02 must extend the check constructors
to accept a DB handle and (for network) a transport client factory. The
`Check.Run` signature (`func(ctx) (Result, string)`) is preserved by
closure-capturing the handles in the constructor. See D-027, D-031.
3. **No public migration-version query.** `migrate()` is unexported and writes
to `schema_migrations(name, applied_at)`. P02 adds a public
`store.MigrationVersion(ctx, db)` (or method on a repo) that selects the max
applied migration name. See D-033.
4. **Doctor test `TestRunAllChecksWithNoCA` asserts a WARN from stubs.** This
will break when stubs become real (the db check will PASS with a fresh test
DB, and the network check will WARN/FAIL on zero peers). Must be updated.
See D-036.
---
## P01: iter.Seq Streaming for `--watch` Flags
Covers REQ-022 (`iter.Seq` for streaming job lists), REQ-030 (`--watch` output
format: table default vs streaming one-line JSON per event).
### Decisions
| ID | Decision | Confidence |
|----|----------|------------|
| **D-025** | `iter.Seq` lives on the store repos, not the engine registry. `JobRepo.Watch(ctx) iter.Seq[*model.Job]` and `NodeRepo.Watch(ctx) iter.Seq[*model.Node]`. Rationale: repos already own the `*sql.DB` and the `scanJob`/`scanNode` helpers; engine.Registry.List just delegates to repo. Keeping Watch in the store layer matches the data-engineer territory and avoids a new engine→store iter dependency. | 0.90 |
| **D-026** | Element type is `*model.Job` / `*model.Node` (pointer), matching the existing `[]*model.Job` return of `List`. This keeps `printJSON` and table rendering identical between one-shot and watch paths. | 0.88 |
| **D-027** | The `Check.Run` signature in `doctor` is unchanged; P02 captures DB/transport handles in closure at constructor time (`Network(db)`, `DB(db)`). This is the established pattern (cert checks already closure-capture `certpaths`). | 0.92 |
| **D-028** | Watch refresh = poll-based 1s ticker (per D-019). No event channel, no daemon coupling. Each tick re-runs the existing `List` query and yields the **full current snapshot** (one element per row). The CLI dedupes by detecting snapshot equality before re-rendering (see D-030). Rationale: simpler than NOTIFY/LISTEN, no daemon dependency, matches offline-first. | 0.90 |
| **D-029** | `--watch` output: default = re-print the table on every changed snapshot (clear screen via ANSI `\033[2J\033[H` then table); `--watch --json` = one compact JSON line **per snapshot** (an array on each line, OR one line per element — see D-030). The CLI tracks the previous snapshot's hash to avoid spamming identical frames. | 0.85 |
| **D-030** | `--watch --json` emits **one JSON object per element per tick where the element changed**, i.e. streaming one-line JSON per event (per REQ-030 wording). Implementation: on each tick, for each element, if its JSON bytes differ from the previous snapshot's bytes for that ID, print `{"event":"update","job":{...}}\n`. On first tick, print all as `{"event":"init",...}`. This is the most useful for AI agents tailing the stream. Confidence lower because REQ-030 is ambiguous between "array per tick" and "object per event"; the per-event interpretation matches "streaming one-line JSON per event" literally. | 0.72 |
| **D-031** | Cancellation: `signal.NotifyContext(ctx, os.Interrupt, syscall.SIGTERM)` at the CLI command layer, **replacing** the current `context.WithTimeout(cmd.Context(), 5*time.Second)` for the watch path only. The non-watch `list` path keeps its 5s timeout. The `iter.Seq` receives this ctx and stops yielding on `ctx.Done()`. | 0.93 |
| **D-032** | The `iter.Seq` implementation **must not leak goroutines**: the polling loop runs **inline in the yield callback's caller goroutine** (the `range` loop), not a separate goroutine. `for range seq { ... }` drives the pull; inside `Watch`, we loop `for { select { <-ticker.C: query+yield each; <-ctx.Done(): return } }` and call `yield(item)` directly. When `yield` returns false (consumer broke the loop), we stop and return. **No goroutine is spawned by Watch.** This is the cleanest Go 1.25 iter pattern and avoids leak surface entirely. | 0.95 |
### Implementation approach — store layer
**File: `internal/store/job_task_repo.go`** — add method:
```go
// Watch yields the current snapshot of jobs on a 1-second ticker until
// ctx is cancelled or the consumer stops pulling (yield returns false).
// It does not spawn a goroutine; the polling loop runs in the caller's
// goroutine via the range-over-func pull protocol.
//
// Each tick re-runs the List query and yields one *model.Job per row.
// The caller is responsible for deduping across ticks if desired.
func (r *JobRepo) Watch(ctx context.Context) iter.Seq[*model.Job] {
return func(yield func(*model.Job) bool) {
ticker := time.NewTicker(1 * time.Second)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
}
// Reuse the existing List query + scanJob helper.
rows, err := r.db.QueryContext(ctx,
`SELECT id, name, spec, status, exit_code, created_at, started_at, ended_at
FROM jobs ORDER BY created_at DESC`)
if err != nil {
// Surfacing errors from inside iter.Seq is awkward; the
// CLI layer cannot receive a returned error. Log via slog
// (the repo doesn't hold a logger today — see D-034) and
// continue to next tick rather than terminating the
// stream. A transient DB blip should not kill the watch.
continue
}
for rows.Next() {
j, err := scanJob(rows)
if err != nil {
rows.Close()
return
}
if !yield(j) {
rows.Close()
return // consumer stopped
}
}
rows.Close()
}
}
}
```
**File: `internal/store/node_repo.go`** — add analogous `Watch`:
```go
func (r *NodeRepo) Watch(ctx context.Context) iter.Seq[*model.Node] {
return func(yield func(*model.Node) bool) {
ticker := time.NewTicker(1 * time.Second)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
}
rows, err := r.db.QueryContext(ctx,
`SELECT id, name, address, state, joined_at, last_seen, metadata
FROM nodes ORDER BY joined_at ASC`)
if err != nil {
continue
}
for rows.Next() {
n, err := scanNode(rows)
if err != nil {
rows.Close()
return
}
if !yield(n) {
rows.Close()
return
}
}
rows.Close()
}
}
}
```
Imports: add `"iter"` and `"time"` (time already present in both files). The
`iter` package is imported only for the return type; `yield` is the callback.
**Note on the existing `scanner` interface:** `scanJob`/`scanNode` accept the
`scanner` interface (`Scan(dest ...any) error`) satisfied by both `*sql.Row`
and `*sql.Rows`, so they are directly reusable in `Watch` — no refactor needed.
### Implementation approach — CLI layer
**File: `internal/cli/job.go`** — modify `jobListCmd`:
1. Add a package var `jobWatch bool` and register `jobListCmd.Flags().BoolVar(&jobWatch, "watch", false, "stream jobs until Ctrl-C")`.
2. In `RunE`, branch on `jobWatch`:
- If `!jobWatch`: keep the existing 5s-timeout `List` path.
- If `jobWatch`:
- `ctx, cancel := signal.NotifyContext(cmd.Context(), os.Interrupt, syscall.SIGTERM); defer cancel()` (drop the 5s timeout).
- `seq := store.NewJobRepo(db).Watch(ctx)`
- If `jsonOutput`: stream one-line JSON per event (D-030). Maintain a `map[string][]byte` of last-seen JSON per job ID. On each yielded job, marshal compact JSON; if it differs from the stored bytes (or ID unseen), print `{"event":"update","job":{...}}\n` and update the map.
- Else (table): on each tick, after collecting the full snapshot, compare against the previous snapshot (by hashing the rendered table string or by comparing the slice of `[]*model.Job` via reflect/cmp). If changed, emit `"\033[2J\033[H"` (clear) then the table header + rows. This gives a "top-like" refresh.
Because `iter.Seq` does not return an error, the watch path swallows
per-tick query errors inside `Watch` (D-034). The CLI relies on `ctx.Done()`
for termination.
**File: `internal/cli/node.go`** — analogous change to `nodeListCmd`:
- Add `nodeWatch bool`, register `--watch` flag.
- Branch in `RunE`; `seq := store.NewNodeRepo(db).Watch(ctx)` (open a fresh `openDB()` for the watch path; `registry.List` is not used for watch — go straight to the repo to get the iter).
**Note:** `nodeListCmd` currently goes through `nodeRegistry()` which wraps
`NodeRepo` in `engine.NodeRegistry`. For watch, bypass the registry and use
`store.NewNodeRepo(db).Watch(ctx)` directly — the registry adds no value for a
read-only stream and would require a `Watch` passthrough method. This keeps the
iter boundary clean in the store layer (D-025).
### Pitfalls & mitigations (P01)
| Pitfall | Mitigation |
|---------|------------|
| **Goroutine leak** if Watch spawned a goroutine. | It doesn't — the polling loop is inline in the pull callback (D-032). `defer ticker.Stop()` + `rows.Close()` on every exit path. |
| **Rows cursor held open across yield** — if `yield` blocks (e.g. slow consumer), the `*sql.Rows` stays open and holds a SQLite read lock. | Yield is called per-row inside the `rows.Next()` loop; the consumer (`range`) is fast (prints to stdout). For safety, close rows immediately after the loop or on `yield==false`. The 1s tick cadence bounds how long a cursor is held. WAL mode (set in `store.Open`) allows concurrent reads, so this does not block writers. |
| **No error channel from iter.Seq** — a transient DB error is invisible to the CLI. | Log inside Watch via a package-level slog default (`slog.Default().Warn(...)`) since the repo has no logger field today (D-034: add an optional `logger *slog.Logger` to JobRepo/NodeRepo, defaulting to `slog.Default()` in the constructors — minimal change). Continue to next tick rather than terminating. |
| **ctx cancellation mid-query**`QueryContext` returns an error; `rows.Next()` returns false. | Handled: the `select` on `ctx.Done()` returns before the next tick; an in-flight query is cancelled by the ctx. |
| **Rapid re-render flicker** in table mode. | Clear-screen + full re-render on changed snapshot only (hash compare). Unchanged snapshots produce no output. |
| **`--watch` + `--json` interleaving with slog stderr.** | slog writes to stderr; CLI output to stdout — no interleaving on stdout. Safe. |
| **Test determinism** — ticker is 1s, tests would be slow/flaky. | Provide a test-only constructor `WatchWithInterval(ctx, d time.Duration)` OR make the interval a field on the repo set via an unexported option. Preferred: an unexported `watchInterval` package var defaulting to 1s, overridable from `internal/store` tests. See D-035. |
| **Signal handling clobbers root signal handler.** | `signal.NotifyContext` with `os.Interrupt` returns a fresh ctx; the root `cobra.Command` does not install its own SIGINT handler, so no conflict. `defer cancel()` restores default behavior on exit. |
### Test strategy (P01)
**`internal/store/job_task_repo_test.go` (new file or appended):**
- `TestJobRepoWatch_YieldsSnapshots`: insert 1 job, call `Watch` with a 10ms interval (via test hook), range over `seq` collecting into a slice, insert a 2nd job from a goroutine after 30ms, cancel ctx after 80ms, assert the 2nd job appeared in the collected slice. Use `context.WithTimeout` for cancellation.
- `TestJobRepoWatch_StopsOnConsumerBreak`: range over `seq` and `break` after the first yield; assert the function returns (no hang) within a short deadline. This validates the `yield==false` path.
- `TestJobRepoWatch_StopsOnCtxCancel`: cancel ctx; assert the range loop exits within 50ms.
- `TestNodeRepoWatch_*`: mirror the above for nodes.
**`internal/cli/job_test.go` (new) / `node_test.go` (new) — if CLI tests exist; otherwise add:**
- `TestJobListWatch_JSONStreaming`: spin a temp DB, insert a job, invoke the `jobListCmd.RunE` with `--watch --json` in a goroutine, insert a 2nd job, capture stdout for ~200ms, assert two JSON lines appear. Cancel via ctx.
- `TestJobListWatch_TableRefresh`: assert clear-screen escape + table re-render on change.
- These CLI tests are harder to make deterministic; prefer testing the store-layer Watch thoroughly and keep CLI watch tests to a smoke-level "produces output, exits on ctx.Done".
### Dependency check (P01)
- `iter` — Go 1.25 stdlib (`go.mod` declares `go 1.25.0`). ✅ no new dep.
- `time`, `context`, `os/signal`, `syscall`, `encoding/json` — stdlib. ✅
- No new go.mod dependencies required.
---
## P02: `orca doctor` network + db full implementation
Covers REQ-032 completion (network + db checks replacing `NetworkStub`/`DBStub`).
### Decisions
| ID | Decision | Confidence |
|----|----------|------------|
| **D-033** | Add a public `store.MigrationVersion(ctx, db) (string, error)` function (in `migrate.go` or a new `internal/store/migrate_query.go`) that returns the **highest applied migration filename** from `schema_migrations`. SQL: `SELECT name FROM schema_migrations ORDER BY name DESC LIMIT 1`. This is the source of truth for "migration version" — it reflects what `migrate()` actually applied. Returns `("", nil)` if no migrations applied (fresh empty table) and `("", sql.ErrNoRows)` is treated as empty. | 0.90 |
| **D-034** | The doctor DB check opens its own `*sql.DB` via `store.Open(dbPath())` (reusing the CLI's `dbPath`) inside the check constructor closure, rather than receiving a shared handle. Rationale: doctor should be runnable whether the daemon is up or down; `store.Open` uses WAL so a concurrent daemon is fine. The check `defer db.Close()`. This avoids threading a `*sql.DB` through `doctor.Run`/`All()` and keeps the `Check.Run` signature stable. | 0.86 |
| **D-035** | The doctor DB check runs `PRAGMA integrity_check` via `db.QueryRow("PRAGMA integrity_check")`. SQLite returns a single row with a TEXT value: `"ok"` on success, or a multi-line error description on failure. PASS if the value is `"ok"`; FAIL otherwise (with the first line of the message). Plus query the migration version (D-033); WARN if `schema_migrations` is empty (fresh/never-migrated db) — this is suspicious but not corrupt. | 0.92 |
| **D-036** | Doctor network check sources peer addresses from the **`nodes` table** (`NodeRepo.List`), NOT from `engine.PeerRegistry` (which is in-memory and always empty at CLI time — see gap #1). For each node with `state != 'left'`, probe `https://<address>/healthz` over mTLS. | 0.88 |
| **D-037** | For each peer, the network check builds an mTLS client via `transport.NewMTLSClient(certpaths.CACertPath(), serverName, certpaths.ServerCertPath(), certpaths.ServerKeyPath())`. `serverName` is derived as the node's `Name` (the SAN on a peer's server cert is its node name, per `security.GenerateCSR(nodeName, sans)` — confirmed in `integration_test.go:41` `GenerateCSR("test-server", ...)`. If the SAN uses a different value the probe will fail handshake, which is itself a useful diagnostic). `CAPath` is the local `ca.crt` (all peers share one CA per D-011). This presents the local node's client cert, satisfying the daemon's `RequireAndVerifyClientCert`. | 0.80 |
| **D-038** | Network check result semantics: **zero peers registered**`WARN` ("no peers registered; network check skipped") — not FAIL, because a single-node install legitimately has no peers. **A peer unreachable / handshake failed**`FAIL` for that peer, aggregated to a single `network` check result that is FAIL if any peer failed, PASS if all peers probed OK, WARN if zero peers. Each peer's per-line outcome is folded into the message string (e.g. `PASS — 2/2 peers reachable; FAIL — peer node-b (host:port): tls handshake error`). | 0.85 |
### Implementation approach — db check
**File: `internal/store/migrate.go`** — add:
```go
// MigrationVersion returns the filename of the most recently applied
// migration, or "" if no migrations have been applied (empty db or
// schema_migrations table missing). Used by `orca doctor db`.
func MigrationVersion(ctx context.Context, db *sql.DB) (string, error) {
var name string
err := db.QueryRowContext(ctx,
`SELECT name FROM schema_migrations ORDER BY name DESC LIMIT 1`).Scan(&name)
if err == sql.ErrNoRows {
return "", nil
}
if err != nil {
return "", fmt.Errorf("query migration version: %w", err)
}
return name, nil
}
```
(Add `"context"` import — already imported in migrate.go.)
**File: `internal/doctor/doctor.go`** — replace `DBStub()` with `DB()`:
```go
// DB checks SQLite integrity and migration version (REQ-032).
// It opens its own *sql.DB so it can run whether or not the daemon is up.
func DB() Check {
return Check{
Name: "db",
Description: "SQLite PRAGMA integrity_check + migration version",
Run: func(ctx context.Context) (Result, string) {
path := dbPath() // dbPath currently lives in internal/cli; see D-039
db, err := store.Open(path)
if err != nil {
return ResultFail, fmt.Sprintf("open %s: %v", path, err)
}
defer db.Close()
// 1. integrity_check
var integrity string
if err := db.QueryRowContext(ctx, "PRAGMA integrity_check").Scan(&integrity); err != nil {
return ResultFail, fmt.Sprintf("integrity_check query: %v", err)
}
if integrity != "ok" {
first := strings.SplitN(integrity, "\n", 2)[0]
return ResultFail, fmt.Sprintf("integrity_check: %s", first)
}
// 2. migration version
ver, err := store.MigrationVersion(ctx, db)
if err != nil {
return ResultFail, fmt.Sprintf("migration version: %v", err)
}
if ver == "" {
return ResultWarn, "integrity ok; no migrations applied (fresh db?)"
}
return ResultPass, fmt.Sprintf("integrity ok; migrations up to %s", ver)
},
}
}
```
**D-039 (assumption, confidence 0.78):** `dbPath()` currently lives in
`internal/cli/node.go` and is unexported. The `doctor` package cannot import
`internal/cli` (would create a cycle: `cli` imports `doctor`). **Resolution:**
move `dbPath()` (and the `ORCA_DB` env logic) into `certpaths` (rename the
package conceptually, or add a sibling `internal/paths` package) OR duplicate
the ~5-line `dbPath` function inside `internal/doctor`. The cleanest is to add
`func DBPath() string` to `internal/certpaths/certpaths.go` (it already owns
`Dir()` honoring `ORCA_HOME`) and have both `cli` and `doctor` call it.
`cli.dbPath` becomes a thin wrapper or is replaced. This is a small refactor
within P02's scope. Logged as D-039, confidence 0.78 (territory overlap between
cli-engineer and the doctor package; lead-developer adjudicates).
### Implementation approach — network check
**File: `internal/doctor/doctor.go`** — replace `NetworkStub()` with `Network()`:
```go
// Network probes each registered peer's /healthz over mTLS (REQ-032).
// Peers are sourced from the nodes table. Zero peers => WARN (single-node
// install is legitimate). Any peer unreachable => FAIL.
func Network() Check {
return Check{
Name: "network",
Description: "peer reachability via mTLS /healthz probe",
Run: func(ctx context.Context) (Result, string) {
// 1. Load registered nodes (skip 'left').
path := certpaths.DBPath() // same resolution as D-039
db, err := store.Open(path)
if err != nil {
return ResultFail, fmt.Sprintf("open db for node list: %v", err)
}
defer db.Close()
nodes, err := store.NewNodeRepo(db).List(ctx)
if err != nil {
return ResultFail, fmt.Sprintf("list nodes: %v", err)
}
// filter out left nodes
var live []*model.Node
for _, n := range nodes {
if n.State != model.NodeStateLeft {
live = append(live, n)
}
}
if len(live) == 0 {
return ResultWarn, "no peers registered; network check skipped (single-node?)"
}
caPath := certpaths.CACertPath()
certPath := certpaths.ServerCertPath()
keyPath := certpaths.ServerKeyPath()
// Short per-probe timeout so one slow peer doesn't stall doctor.
var lines []string
overall := ResultPass
for _, n := range live {
probeCtx, cancel := context.WithTimeout(ctx, 3*time.Second)
err := probeHealthz(probeCtx, caPath, certPath, keyPath, n.Name, n.Address)
cancel()
if err != nil {
overall = ResultFail
lines = append(lines, fmt.Sprintf("FAIL %s (%s): %v", n.Name, n.Address, err))
} else {
lines = append(lines, fmt.Sprintf("PASS %s (%s)", n.Name, n.Address))
}
}
if overall == ResultPass {
return ResultPass, fmt.Sprintf("%d/%d peers reachable: %s", len(live), len(live), strings.Join(lines, "; "))
}
return ResultFail, strings.Join(lines, "; ")
},
}
}
// probeHealthz does a GET https://addr/healthz over mTLS.
func probeHealthz(ctx context.Context, caPath, certPath, keyPath, serverName, addr string) error {
client, err := transport.NewMTLSClient(caPath, serverName, certPath, keyPath)
if err != nil {
return fmt.Errorf("build mTLS client: %w", err)
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://"+addr+"/healthz", nil)
if err != nil {
return fmt.Errorf("build request: %w", err)
}
resp, err := client.Do(req)
if err != nil {
return fmt.Errorf("probe: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("healthz status %d", resp.StatusCode)
}
return nil
}
```
New imports in `doctor.go`: `net/http`, `strings`, `time`, `git.cloudinit.dev/coreci/orca/internal/store`, `git.cloudinit.dev/coreci/orca/internal/transport`, `git.cloudinit.dev/coreci/orca/internal/model`.
**File: `internal/doctor/doctor.go`** — update `All()`:
```go
func All() []Check {
return []Check{
CertCA(), CertServer(), CertExpiry(), CertFingerprint(),
Network(), // was NetworkStub()
DB(), // was DBStub()
}
}
```
Keep `NetworkStub`/`DBStub` exported functions for one release as thin
wrappers that call the new ones? **No** — delete them; the CLI doctor.go
references them and must be updated in lockstep (they are internal). See D-040.
**File: `internal/cli/doctor.go`** — update `doctorNetworkCmd` and `doctorDBCmd`:
```go
doctorNetworkCmd.RunE: c := doctor.Network() // was doctor.NetworkStub()
doctorDBCmd.RunE: c := doctor.DB() // was doctor.DBStub()
```
Also: the per-subcommand render should honor `jsonOutput` (currently it only
prints text). Minor enhancement, in scope.
### Pitfalls & mitigations (P02)
| Pitfall | Mitigation |
|---------|------------|
| **`model.Node` has no `ServerName`/`CAPath`** — mTLS needs `ServerName` to match the cert SAN. | Derive `ServerName = node.Name` (D-037). This assumes peer server certs are issued with SAN = node name, which matches `GenerateCSR(nodeName, sans)`. If a deployment uses DNS SANs instead, the probe fails — which is itself a diagnostic. Document this assumption in the check message. |
| **No local client cert/key** — doctor can't present a client cert if `server.crt`/`server.key` are missing. | The check should FAIL with a clear message if `certpaths.ServerCertPath()` doesn't exist, BEFORE attempting probes. Reuse `os.Stat`. This also covers the single-node-never-joined case. |
| **Daemon down** — peer's `/healthz` unreachable. | Per-probe 3s timeout (D-038). Surfaces as FAIL per peer with the dial/handshake error in the message. Doctor is designed to run with daemon up or down, so this is expected behavior, not a crash. |
| **Self-probe** — the local node is likely in the `nodes` table too. Doctor will probe itself over mTLS. This is fine (validates the local daemon's mTLS stack) but requires the local daemon to be running. If the daemon is down, the self-probe fails → FAIL, which is the correct signal. | Document; no special-casing. |
| **DB file doesn't exist**`store.Open` creates the dir + file + runs migrations (so a missing db becomes a fresh empty db). The db check would then PASS with "no migrations applied" WARN. | This is acceptable: `store.Open` is idempotent. If the operator expected an existing db, the WARN surfaces the surprise. Could additionally `os.Stat` the path before `Open` and WARN if it didn't exist pre-open — optional refinement. |
| **`PRAGMA integrity_check` can return multiple rows** in rare cases (when there are multiple errors). `QueryRow` only reads the first. | For `integrity_check`, a single row containing `"ok"` or the first error is the documented SQLite behavior for the common case. Use `QueryRow` + `Scan`; if it's not `"ok"`, that's already a FAIL. Acceptable. |
| **Doctor test `TestRunAllChecksWithNoCA`** asserts WARN from stubs. | Update the test: with real checks, a no-CA scenario yields FAIL on cert.ca (unchanged) AND FAIL on db (open succeeds, integrity ok, but no migrations if fresh — actually WARN) AND WARN on network (no peers). Rewrite assertions to check each check by name rather than "hasWarn globally". See test strategy. |
| **Import cycle:** `doctor``cli` (for `dbPath`). | Resolved by D-039: move `dbPath` to `certpaths` (or a new `internal/paths`); both `cli` and `doctor` import it. No cycle. |
| **`store.Open` runs migrations on every open** — doctor opening the db to run integrity_check would also (re)migrate. | `migrate()` is idempotent (checks `schema_migrations` per name). Re-opening is safe; no-op if already migrated. Acceptable. |
### Test strategy (P02)
**`internal/store/migrate_test.go` (new or appended):**
- `TestMigrationVersion`: open a fresh test db (which runs migrate), call `MigrationVersion`, assert it returns `0005_node_capacity.sql` (the highest current migration). Then manually delete all rows from `schema_migrations`, assert returns `""` with nil error.
**`internal/doctor/doctor_test.go` (update):**
- Update `TestRunAllChecksWithNoCA`: set `ORCA_HOME` to temp dir (no CA). Expect: cert.ca FAIL, cert.server FAIL, cert.expiry FAIL, cert.fingerprint FAIL, **db WARN** (fresh db, no migrations — actually `store.Open` runs migrations, so db will PASS with version 0005; adjust: db PASS), **network WARN** (no peers). Rewrite to assert per-check rather than "hasWarn/hasFail globally". Remove the stale "expected WARN (stubs)" comment.
- New `TestDBCheck_IntegrityOK`: open a fresh db via `store.Open` in temp, run `doctor.DB().Run(ctx)`, expect PASS and message contains "0005".
- New `TestDBCheck_Corrupt`: open db, manually `db.Exec("DROP TABLE jobs")` to introduce inconsistency, run integrity_check — but `integrity_check` mostly detects corruption, not missing tables. More reliable: write garbage to the db file via raw file write, then open — `store.Open` may fail at Ping. Assert FAIL. (This test is brittle; prefer a unit test on the integrity string-parsing logic with a stub.)
- New `TestNetworkCheck_NoPeers`: fresh db, no nodes, run `doctor.Network().Run(ctx)`, expect WARN.
- New `TestNetworkCheck_PeerReachable`: this is an integration test — bootstrap a CA (`security.CAInit`), generate+sign a server cert with SAN `localhost`, start an `httptest.NewUnstartedServer` with `ts.TLS = serverTLS` and `ClientAuth = RequireAndVerifyClientCert` (mirror `security/integration_test.go:88-108`), generate+sign a client cert, insert a node row with `Address = ts.Listener.Addr().String()` and `Name = "localhost"`, set `ORCA_HOME` to the temp dir holding the CA + client cert, run `doctor.Network().Run(ctx)`, expect PASS. This reuses the proven pattern from `TestEndToEndMTLS`.
- New `TestNetworkCheck_PeerUnreachable`: insert a node with `Address = "127.0.0.1:1"` (nothing listening), run, expect FAIL with the peer name in the message.
### Dependency check (P02)
- `net/http`, `crypto/tls` (via transport), `strings`, `time`, `context` — stdlib. ✅
- `internal/transport`, `internal/store`, `internal/model`, `internal/certpaths` — existing internal packages. ✅
- No new go.mod dependencies required.
---
## Cross-cutting decisions
| ID | Decision | Confidence |
|----|----------|------------|
| **D-039** | Move `dbPath()` (the `ORCA_DB`-honoring path resolver) from `internal/cli` to `internal/certpaths` as `DBPath()`, to break the would-be `doctor→cli` import cycle. Both `cli` and `doctor` then import `certpaths`. `certpaths` already owns the `ORCA_HOME`-honoring `Dir()`. Territory: this is a shared infra concern; `lead-developer` adjudicates. | 0.78 |
| **D-040** | Delete `doctor.NetworkStub` and `doctor.DBStub` (no backward-compat shims). They are internal, referenced only by `internal/cli/doctor.go` which is updated in the same phase. Keeping dead stub code violates `no-redundant-implementations`. | 0.95 |
| **D-041** | No new go.mod dependencies for v0.3. `iter` (P01) and mTLS health probe (P02) use stdlib + existing internal packages only. The 4 existing direct deps (cobra, hcl, modernc/sqlite, uuid) are unchanged. | 0.97 |
| **D-042** | Phase ordering: P01 (iter.Seq) and P02 (doctor) are **independent** — no file is modified by both (P01 touches cli/job.go, cli/node.go, store repos; P02 touches doctor.go, cli/doctor.go, store/migrate.go, certpaths). They can be developed in either order or in parallel. Recommend P01 first only because it's the lower-risk change. | 0.85 |
---
## Summary of assumptions logged
All assumptions below are logged as decisions with confidence scores; none are
flagged for human validation (full autonomy). Low-confidence (<0.80) items that
warrant normal decision-flow attention:
- **D-030** (0.72): `--watch --json` emits one JSON object per changed element
per tick (vs. one array per tick). REQ-030 wording is ambiguous; this
interpretation matches "streaming one-line JSON per event" literally.
- **D-037** (0.80): peer `ServerName` = node `Name` (SAN convention).
- **D-039** (0.78): `dbPath` relocation to `certpaths` — territory overlap.
These three are escalated through the normal decision flow (DecisionEngine) per
the researcher protocol, NOT flagged for human validation.
+37 -47
View File
@@ -20,62 +20,52 @@
- `iter.Seq` streaming job lists (REQ-022)
- Frontend / devops personas (no web UI; CoreCI handles release)
## Milestone v0.2: Networking, Observability, Security Hardening — **IN PROGRESS**
## Milestone v0.2: Networking, Observability, Security Hardening — **COMPLETE (merged to main via v0.3)**
Scope: extend v0.1 with secure cross-node transport, multi-node scheduling,
richer CI security scanning, and streaming I/O.
- [ ] Phase 8: mTLS handshake + internal CA with CSR join (Wave 1)
- [ ] Phase 9: Multi-node scheduling & job dispatch (Wave 1)
- [ ] Phase 10: `gosec` + `govulncheck` + gitleaks in CI (Wave 2)
- [ ] Phase 11: `iter.Seq` streaming job/node lists (Wave 2)
- [x] Phase 8: mTLS handshake + internal CA with CSR join (Wave 1) — shipped v0.2.1
- [x] Phase 9: Multi-node scheduling & job dispatch (Wave 1) — shipped v0.2.2
- [x] Phase 10: `gosec` + `govulncheck` + gitleaks in CI (Wave 2) — shipped v0.2.3
- [x] Phase 11: `iter.Seq` streaming job/node lists (Wave 2)**completed in v0.3 P01** (shipped v0.3.1)
**Target milestone tag**: `v0.3.0` (next-minor per feature-milestone promotion rule).
**Milestone tag**: `v0.4.0` (shipped — v0.2 work merged to main via v0.3 milestone).
Per-phase tags: `v0.2.1` (P01), `v0.2.2` (P02), `v0.2.3` (P03), `v0.2.4` (P04).
Per-phase tags: `v0.2.1` (P01), `v0.2.2` (P02), `v0.2.3` (P03) — all shipped.
## Milestone v0.3: Scheduling & Streaming Completion — **COMPLETE**
Scope: complete the two work items deferred from v0.2 that were not
already shipped in P08-P10. A re-init SPECIFY codebase audit confirmed
that REQ-014/027/028/029/031/037/039/040 all shipped in P08-P10 despite
stale REQUIREMENTS.md marking them Pending. The remaining work is lean:
- [x] Phase 0: Pre-execution (specify → clarify → research → plan → grill) — shipped v0.3.0
- [x] Phase 1: `iter.Seq` streaming for `--watch` flags (REQ-022, REQ-030) — shipped v0.3.1
- [x] Phase 2: `orca doctor` network + db full implementation (REQ-032 completion) — shipped v0.3.2
- [x] Phase 3: Final review + ship + audit (milestone release) — shipped v0.3.3
**Milestone tag**: `v0.4.0` (next-minor per feature-milestone promotion rule).
Per-phase tags: `v0.3.0` (P0), `v0.3.1` (P01), `v0.3.2` (P02), `v0.3.3` (P03 final = milestone release).
Per `.ciagent/RELEASE_POLICY.md`, every phase tag produces a Gitea release.
### Per-phase REQ coverage (post-IDEATE)
### Per-phase REQ coverage
- **P01 — mTLS handshake + internal CA with CSR join** (Wave 1)
- REQ-011, REQ-023 (carried over from v0.1)
- REQ-025 (cert rotation history), REQ-026 (CA fingerprint pinning),
REQ-033 (file mode enforcement), REQ-034 (rotation alarm),
REQ-035 (cert show redaction), REQ-036 (SAN validation),
REQ-038 (mTLS failure log fields)
- REQ-032 (orca doctor — initial implementation; checks CA/cert state)
- **P01 — `iter.Seq` streaming for `--watch` flags**
- REQ-022 (`iter.Seq` for streaming job lists, Go 1.25+)
- REQ-030 (`--watch` output format mode: table default vs streaming JSON per event)
- Applies to both `orca job list --watch` and `orca node list --watch`
(D-024, per ARCHITECTURE.md CLI layer + D-017)
- **P02 — Multi-node scheduling & job dispatch** (Wave 1)
- REQ-028 (NodeCapacity HCL schema — P02 enabler; lands first)
- REQ-037 (X-Orca-Idempotency-Key on cross-node POST)
- **P02 — `orca doctor` network + db full implementation**
- REQ-032 (completion: network reachability via mTLS `/healthz` probe,
db integrity via `PRAGMA integrity_check` + migration version)
- Replaces `NetworkStub` and `DBStub` from v0.2 P01
- **P03 — `gosec` + `govulncheck` + gitleaks in CI** (Wave 2)
- REQ-014 (carried over)
- REQ-027 (govulncheck offline mode — new in v0.2 IDEATE, per REQ-cand-C;
this changes P03's scope: CI must not call `vuln.go.dev` by default;
resolve via pre-mirrored DB or `-format json` + `jq` wrapper. PLAN
stage decides between the two options.)
- REQ-029 (gitleaks baseline for pre-existing `.env` leak in history,
per REQ-cand-E)
- REQ-039 (`.gitleaks.toml` stopwords), REQ-040 (`.golangci.yml`)
### v0.3 is a completion milestone, not a direction change
- **P04 — `iter.Seq` streaming job/node lists** (Wave 2)
- REQ-022 (carried over)
- REQ-030 (`--watch --json` streaming output mode, per REQ-cand-F)
- **Cross-cutting (P01P04)**
- REQ-031 (`go test -race` enabled in CI for all v0.2 packages)
### P03 scope change (vs. pre-IDEATE plan)
REQ-027 (govulncheck offline mode) adds explicit work to P03: the CI
job must be configured to NOT make outbound calls to `vuln.go.dev`
(default `govulncheck` behavior). Two implementation paths are viable;
PLAN chooses:
- Pre-mirror the vulnerability database inside the CoreCI image
(`GOVULNCHECK_DB=/path/to/local.db`).
- Use `govulncheck -format json` (which always exits 0) and gate
merges via a wrapper that parses the JSON and returns non-zero on
unsuppressed findings.
Either path keeps the offline-first invariant (REQ-003) intact.
The vision ("minimalist, offline-first, CLI-first orchestration
engine") is unchanged. v0.3 closes out the v0.2 deferrals and merges
the accumulated v0.2 work to main.
+21 -2
View File
@@ -5,7 +5,7 @@
"slug": "orca",
"name": "Orca",
"description": "Offline/CLI-first orchestration engine (Orca) — Nomad-inspired, far simpler than Kubernetes",
"milestone": "v0.1",
"milestone": "v0.3",
"phase": 0,
"milestone_type": "feature",
"default_branch": "main",
@@ -29,7 +29,8 @@
"decision_confidence_threshold": 0.60,
"max_revision_iterations": 3,
"max_verification_retries": 2,
"escalation_hooks": ["delete", "drop", "force", "reset --hard"]
"clarify_budget": 10,
"escalation_hooks": ["deploy", "delete_data", "merge_to_main"]
},
"workflow": {
"no_hitl": true,
@@ -114,6 +115,24 @@
"url": "https://git.cloudinit.dev/coreci/orca.git",
"main_branch": "main"
},
"release": {
"forge": "gitea",
"gitea": {
"base_url": "https://git.cloudinit.dev",
"owner": "coreci",
"repo": "orca",
"token_env": "GITEA_TOKEN"
}
},
"secrets": {
"scopes": [
{
"name": "gitea",
"vars": ["GITEA_TOKEN", "GITEA_USER"],
"env_file": ".env"
}
]
},
"commands": {
"test": "make test",
"build": "make build",
+2
View File
@@ -10,4 +10,6 @@ orca
*.db-shm
.env
.env.local
.env.secrets
.env.*
*.tar.gz
+10
View File
@@ -36,3 +36,13 @@ func ServerCertPath() string { return filepath.Join(Dir(), "server.crt") }
// ServerKeyPath returns the path to server.key.
func ServerKeyPath() string { return filepath.Join(Dir(), "server.key") }
// DBPath returns the path to the orca SQLite database. Honors $ORCA_DB
// for testability and explicit override; otherwise defaults to
// ~/.orca/orca.db under the same Dir() as the cert files.
func DBPath() string {
if p := os.Getenv("ORCA_DB"); p != "" {
return p
}
return filepath.Join(Dir(), "orca.db")
}
+2 -2
View File
@@ -51,7 +51,7 @@ var doctorNetworkCmd = &cobra.Command{
Use: "network",
Short: "Run the network self-check (P02 impl)",
RunE: func(cmd *cobra.Command, args []string) error {
c := doctor.NetworkStub()
c := doctor.Network()
r, msg := c.Run(cmd.Context())
fmt.Fprintf(cmd.OutOrStdout(), "%-20s %-5s %s\n", c.Name, r, msg)
return nil
@@ -62,7 +62,7 @@ var doctorDBCmd = &cobra.Command{
Use: "db",
Short: "Run the database self-check (P02 impl)",
RunE: func(cmd *cobra.Command, args []string) error {
c := doctor.DBStub()
c := doctor.DB()
r, msg := c.Run(cmd.Context())
fmt.Fprintf(cmd.OutOrStdout(), "%-20s %-5s %s\n", c.Name, r, msg)
return nil
+76 -1
View File
@@ -5,6 +5,9 @@ import (
"encoding/json"
"errors"
"fmt"
"os"
"os/signal"
"syscall"
"time"
"github.com/google/uuid"
@@ -36,6 +39,7 @@ var (
stopID string
runTarget string
runIDKey string
jobWatch bool
)
var jobRunCmd = &cobra.Command{
@@ -112,8 +116,11 @@ var jobRunCmd = &cobra.Command{
var jobListCmd = &cobra.Command{
Use: "list",
Short: "List all jobs",
Long: "Display all jobs and their status.",
Long: "Display all jobs and their status. Use --watch to stream updates until Ctrl-C.",
RunE: func(cmd *cobra.Command, args []string) error {
if jobWatch {
return watchJobs(cmd)
}
ctx, cancel := context.WithTimeout(cmd.Context(), 5*time.Second)
defer cancel()
@@ -142,6 +149,73 @@ var jobListCmd = &cobra.Command{
},
}
func watchJobs(cmd *cobra.Command) error {
ctx, cancel := signal.NotifyContext(cmd.Context(), os.Interrupt, syscall.SIGTERM)
defer cancel()
return watchJobsCtx(cmd, ctx)
}
func watchJobsCtx(cmd *cobra.Command, ctx context.Context) error {
db, closer, err := openDB()
if err != nil {
return err
}
defer closer()
out := cmd.OutOrStdout()
if jsonOutput {
seen := make(map[string]string)
for snapshot := range store.NewJobRepo(db).Watch(ctx) {
current := make(map[string]bool, len(snapshot))
for _, j := range snapshot {
current[j.ID] = true
compact, _ := json.Marshal(j)
key := string(compact)
if prev, ok := seen[j.ID]; !ok || prev != key {
event := "init"
if ok {
event = "update"
}
line, _ := json.Marshal(map[string]any{"event": event, "job": j})
fmt.Fprintln(out, string(line))
seen[j.ID] = key
}
}
for id := range seen {
if !current[id] {
line, _ := json.Marshal(map[string]any{"event": "delete", "id": id})
fmt.Fprintln(out, string(line))
delete(seen, id)
}
}
}
return nil
}
prevTable := ""
for snapshot := range store.NewJobRepo(db).Watch(ctx) {
table := renderJobTable(snapshot)
if table != prevTable {
fmt.Fprint(out, "\033[2J\033[H")
fmt.Fprint(out, table)
prevTable = table
}
}
return nil
}
func renderJobTable(jobs []*model.Job) string {
if len(jobs) == 0 {
return "No jobs.\n"
}
out := fmt.Sprintf("%-36s %-20s %-12s %-8s\n", "ID", "NAME", "STATUS", "EXIT")
for _, j := range jobs {
out += fmt.Sprintf("%-36s %-20s %-12s %-8d\n", j.ID, j.Name, j.Status, j.ExitCode)
}
return out
}
var jobStopCmd = &cobra.Command{
Use: "stop [job-id]",
Short: "Stop a running job",
@@ -235,6 +309,7 @@ func init() {
jobLogsCmd.Flags().StringVar(&stopID, "id", "", "job id")
jobRunCmd.Flags().StringVar(&runTarget, "target", "", "pin job to a specific node id (overrides bin-packing)")
jobRunCmd.Flags().StringVar(&runIDKey, "idempotency-key", "", "X-Orca-Idempotency-Key for cross-node dispatch dedupe")
jobListCmd.Flags().BoolVar(&jobWatch, "watch", false, "stream jobs until Ctrl-C (table refresh or --json per-event)")
jobCmd.AddCommand(jobRunCmd)
jobCmd.AddCommand(jobListCmd)
+77 -11
View File
@@ -3,10 +3,12 @@ package cli
import (
"context"
"database/sql"
"encoding/json"
"fmt"
"log/slog"
"os"
"path/filepath"
"os/signal"
"syscall"
"time"
"github.com/google/uuid"
@@ -19,16 +21,8 @@ import (
"git.cloudinit.dev/coreci/orca/internal/store"
)
func dbPath() string {
if p := os.Getenv("ORCA_DB"); p != "" {
return p
}
home, _ := os.UserHomeDir()
return filepath.Join(home, ".orca", "orca.db")
}
func openDB() (*sql.DB, func() error, error) {
db, err := store.Open(dbPath())
db, err := store.Open(certpaths.DBPath())
if err != nil {
return nil, nil, err
}
@@ -54,6 +48,7 @@ var (
joinAddr string
joinCAFinger string
leaveID string
nodeWatch bool
)
var nodeCmd = &cobra.Command{
@@ -155,8 +150,11 @@ var nodeLeaveCmd = &cobra.Command{
var nodeListCmd = &cobra.Command{
Use: "list",
Short: "List all nodes in the orca registry",
Long: "Display all registered nodes and their state.",
Long: "Display all registered nodes and their state. Use --watch to stream updates until Ctrl-C.",
RunE: func(cmd *cobra.Command, args []string) error {
if nodeWatch {
return watchNodes(cmd)
}
ctx, cancel := context.WithTimeout(cmd.Context(), 5*time.Second)
defer cancel()
@@ -185,11 +183,79 @@ var nodeListCmd = &cobra.Command{
},
}
func watchNodes(cmd *cobra.Command) error {
ctx, cancel := signal.NotifyContext(cmd.Context(), os.Interrupt, syscall.SIGTERM)
defer cancel()
return watchNodesCtx(cmd, ctx)
}
func watchNodesCtx(cmd *cobra.Command, ctx context.Context) error {
db, closer, err := openDB()
if err != nil {
return err
}
defer closer()
out := cmd.OutOrStdout()
if jsonOutput {
seen := make(map[string]string)
for snapshot := range store.NewNodeRepo(db).Watch(ctx) {
current := make(map[string]bool, len(snapshot))
for _, n := range snapshot {
current[n.ID] = true
compact, _ := json.Marshal(n)
key := string(compact)
if prev, ok := seen[n.ID]; !ok || prev != key {
event := "init"
if ok {
event = "update"
}
line, _ := json.Marshal(map[string]any{"event": event, "node": n})
fmt.Fprintln(out, string(line))
seen[n.ID] = key
}
}
for id := range seen {
if !current[id] {
line, _ := json.Marshal(map[string]any{"event": "delete", "id": id})
fmt.Fprintln(out, string(line))
delete(seen, id)
}
}
}
return nil
}
prevTable := ""
for snapshot := range store.NewNodeRepo(db).Watch(ctx) {
table := renderNodeTable(snapshot)
if table != prevTable {
fmt.Fprint(out, "\033[2J\033[H")
fmt.Fprint(out, table)
prevTable = table
}
}
return nil
}
func renderNodeTable(nodes []*model.Node) string {
if len(nodes) == 0 {
return "No nodes registered.\n"
}
out := fmt.Sprintf("%-36s %-20s %-22s %-10s\n", "ID", "NAME", "ADDRESS", "STATE")
for _, n := range nodes {
out += fmt.Sprintf("%-36s %-20s %-22s %-10s\n", n.ID, n.Name, n.Address, n.State)
}
return out
}
func init() {
nodeJoinCmd.Flags().StringVar(&joinName, "name", "", "node name (required)")
nodeJoinCmd.Flags().StringVar(&joinAddr, "addr", "", "node address (default localhost:8443)")
nodeJoinCmd.Flags().StringVar(&joinCAFinger, "ca-fingerprint", "", "pin CA cert SHA-256 (REQ-026); fails if on-disk CA doesn't match")
nodeLeaveCmd.Flags().StringVar(&leaveID, "id", "", "node id")
nodeListCmd.Flags().BoolVar(&nodeWatch, "watch", false, "stream nodes until Ctrl-C (table refresh or --json per-event)")
nodeCmd.AddCommand(nodeJoinCmd)
nodeCmd.AddCommand(nodeLeaveCmd)
+287
View File
@@ -0,0 +1,287 @@
package cli
import (
"bytes"
"context"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"time"
"git.cloudinit.dev/coreci/orca/internal/model"
"git.cloudinit.dev/coreci/orca/internal/store"
)
func TestWatchJobs_JSONStreaming(t *testing.T) {
dir := t.TempDir()
dbPath := filepath.Join(dir, "orca.db")
db, err := store.Open(dbPath)
if err != nil {
t.Fatalf("open db: %v", err)
}
defer db.Close()
repo := store.NewJobRepo(db)
bgCtx := context.Background()
_ = repo.Insert(bgCtx, &model.Job{ID: "seed-job", Name: "seed", Spec: "t", Status: model.JobStatusPending})
t.Setenv("ORCA_DB", dbPath)
jsonOutput = true
t.Cleanup(func() { jsonOutput = false })
var buf bytes.Buffer
rootCmd.SetOut(&buf)
rootCmd.SetErr(&buf)
t.Cleanup(func() { rootCmd.SetOut(os.Stdout); rootCmd.SetErr(os.Stderr) })
ctx, cancel := context.WithCancel(bgCtx)
done := make(chan error, 1)
go func() { done <- watchJobsCtx(rootCmd, ctx) }()
// First yield is immediate (G-002); wait for it.
time.Sleep(100 * time.Millisecond)
_ = repo.Insert(bgCtx, &model.Job{ID: "watch-job", Name: "watch", Spec: "t", Status: model.JobStatusPending})
// Wait for at least one ticker interval (default 1s) to capture the change.
time.Sleep(1100 * time.Millisecond)
cancel()
select {
case <-done:
case <-time.After(2 * time.Second):
t.Fatal("watchJobsCtx did not return within 2s after cancel")
}
output := buf.String()
if !strings.Contains(output, `"event":"init"`) {
t.Errorf("expected init event, got: %s", output)
}
if !strings.Contains(output, "watch-job") {
t.Errorf("expected watch-job in output, got: %s", output)
}
}
func TestWatchJobs_TableRefresh(t *testing.T) {
dir := t.TempDir()
dbPath := filepath.Join(dir, "orca.db")
db, err := store.Open(dbPath)
if err != nil {
t.Fatalf("open db: %v", err)
}
defer db.Close()
repo := store.NewJobRepo(db)
bgCtx := context.Background()
_ = repo.Insert(bgCtx, &model.Job{ID: "seed-job", Name: "seed", Spec: "t", Status: model.JobStatusPending})
t.Setenv("ORCA_DB", dbPath)
jsonOutput = false
t.Cleanup(func() { jsonOutput = false })
var buf bytes.Buffer
rootCmd.SetOut(&buf)
rootCmd.SetErr(&buf)
t.Cleanup(func() { rootCmd.SetOut(os.Stdout); rootCmd.SetErr(os.Stderr) })
ctx, cancel := context.WithCancel(bgCtx)
done := make(chan error, 1)
go func() { done <- watchJobsCtx(rootCmd, ctx) }()
time.Sleep(100 * time.Millisecond)
_ = repo.Insert(bgCtx, &model.Job{ID: "table-job", Name: "table", Spec: "t", Status: model.JobStatusPending})
time.Sleep(1100 * time.Millisecond)
cancel()
select {
case <-done:
case <-time.After(2 * time.Second):
t.Fatal("watchJobsCtx did not return within 2s after cancel")
}
output := buf.String()
if !strings.Contains(output, "\033[2J\033[H") {
t.Errorf("expected clear-screen escape in table watch output, got: %s", output)
}
if !strings.Contains(output, "table-job") {
t.Errorf("expected table-job in output, got: %s", output)
}
}
func TestWatchNodes_JSONStreaming(t *testing.T) {
dir := t.TempDir()
dbPath := filepath.Join(dir, "orca.db")
db, err := store.Open(dbPath)
if err != nil {
t.Fatalf("open db: %v", err)
}
defer db.Close()
repo := store.NewNodeRepo(db)
bgCtx := context.Background()
_ = repo.Insert(bgCtx, &model.Node{
ID: "seed-node", Name: "seed", Address: "addr",
State: model.NodeStateReady, JoinedAt: time.Now().UTC(), LastSeen: time.Now().UTC(),
})
t.Setenv("ORCA_DB", dbPath)
jsonOutput = true
t.Cleanup(func() { jsonOutput = false })
var buf bytes.Buffer
rootCmd.SetOut(&buf)
rootCmd.SetErr(&buf)
t.Cleanup(func() { rootCmd.SetOut(os.Stdout); rootCmd.SetErr(os.Stderr) })
ctx, cancel := context.WithCancel(bgCtx)
done := make(chan error, 1)
go func() { done <- watchNodesCtx(rootCmd, ctx) }()
time.Sleep(100 * time.Millisecond)
_ = repo.Insert(bgCtx, &model.Node{
ID: "watch-node", Name: "watch", Address: "addr2",
State: model.NodeStateReady, JoinedAt: time.Now().UTC(), LastSeen: time.Now().UTC(),
})
time.Sleep(1100 * time.Millisecond)
cancel()
select {
case <-done:
case <-time.After(2 * time.Second):
t.Fatal("watchNodesCtx did not return within 2s after cancel")
}
output := buf.String()
initFound := false
watchNodeFound := false
for _, line := range strings.Split(output, "\n") {
line = strings.TrimSpace(line)
if line == "" {
continue
}
var event map[string]any
if err := json.Unmarshal([]byte(line), &event); err != nil {
continue
}
if event["event"] == "init" {
initFound = true
if node, ok := event["node"].(map[string]any); ok {
if node["id"] == "watch-node" {
watchNodeFound = true
}
}
}
}
if !initFound {
t.Errorf("expected init event in JSON stream, got: %s", output)
}
if !watchNodeFound {
t.Errorf("expected watch-node in JSON stream, got: %s", output)
}
}
func TestWatchNodes_TableRefresh(t *testing.T) {
dir := t.TempDir()
dbPath := filepath.Join(dir, "orca.db")
db, err := store.Open(dbPath)
if err != nil {
t.Fatalf("open db: %v", err)
}
defer db.Close()
repo := store.NewNodeRepo(db)
bgCtx := context.Background()
_ = repo.Insert(bgCtx, &model.Node{
ID: "seed-node", Name: "seed", Address: "addr",
State: model.NodeStateReady, JoinedAt: time.Now().UTC(), LastSeen: time.Now().UTC(),
})
t.Setenv("ORCA_DB", dbPath)
jsonOutput = false
t.Cleanup(func() { jsonOutput = false })
var buf bytes.Buffer
rootCmd.SetOut(&buf)
rootCmd.SetErr(&buf)
t.Cleanup(func() { rootCmd.SetOut(os.Stdout); rootCmd.SetErr(os.Stderr) })
ctx, cancel := context.WithCancel(bgCtx)
done := make(chan error, 1)
go func() { done <- watchNodesCtx(rootCmd, ctx) }()
time.Sleep(100 * time.Millisecond)
_ = repo.Insert(bgCtx, &model.Node{
ID: "table-node", Name: "table", Address: "addr2",
State: model.NodeStateReady, JoinedAt: time.Now().UTC(), LastSeen: time.Now().UTC(),
})
time.Sleep(1100 * time.Millisecond)
cancel()
select {
case <-done:
case <-time.After(2 * time.Second):
t.Fatal("watchNodesCtx did not return within 2s after cancel")
}
output := buf.String()
if !strings.Contains(output, "\033[2J\033[H") {
t.Errorf("expected clear-screen escape in table watch output, got: %s", output)
}
if !strings.Contains(output, "table-node") {
t.Errorf("expected table-node in output, got: %s", output)
}
}
func TestRenderJobTable(t *testing.T) {
jobs := []*model.Job{
{ID: "j1", Name: "alpha", Status: "running", ExitCode: 0},
{ID: "j2", Name: "beta", Status: "done", ExitCode: 0},
}
out := renderJobTable(jobs)
if !strings.Contains(out, "j1") || !strings.Contains(out, "alpha") {
t.Errorf("renderJobTable missing job 1: %s", out)
}
if !strings.Contains(out, "j2") || !strings.Contains(out, "beta") {
t.Errorf("renderJobTable missing job 2: %s", out)
}
}
func TestRenderJobTableEmpty(t *testing.T) {
out := renderJobTable(nil)
if !strings.Contains(out, "No jobs") {
t.Errorf("expected empty message, got: %s", out)
}
}
func TestRenderNodeTable(t *testing.T) {
nodes := []*model.Node{
{ID: "n1", Name: "alpha", Address: "localhost:8443", State: "ready"},
}
out := renderNodeTable(nodes)
if !strings.Contains(out, "n1") || !strings.Contains(out, "alpha") {
t.Errorf("renderNodeTable missing node: %s", out)
}
}
func TestRenderNodeTableEmpty(t *testing.T) {
out := renderNodeTable(nil)
if !strings.Contains(out, "No nodes") {
t.Errorf("expected empty message, got: %s", out)
}
}
+115 -14
View File
@@ -19,12 +19,17 @@ import (
"crypto/x509"
"encoding/pem"
"fmt"
"net/http"
"os"
"sort"
"strings"
"time"
"git.cloudinit.dev/coreci/orca/internal/certpaths"
"git.cloudinit.dev/coreci/orca/internal/model"
"git.cloudinit.dev/coreci/orca/internal/security"
"git.cloudinit.dev/coreci/orca/internal/store"
"git.cloudinit.dev/coreci/orca/internal/transport"
)
// Result is the outcome of a single check.
@@ -63,8 +68,8 @@ func All() []Check {
CertServer(),
CertExpiry(),
CertFingerprint(),
NetworkStub(),
DBStub(),
Network(),
DB(),
}
}
@@ -177,28 +182,124 @@ func CertFingerprint() Check {
}
}
// NetworkStub is a stub for the network check; full impl in P02.
func NetworkStub() Check {
// DB checks SQLite integrity and migration version (REQ-032 completion).
func DB() Check {
return Check{
Name: "network",
Description: "TCP reachability + mTLS handshake (full impl in P02)",
Run: func(_ context.Context) (Result, string) {
return ResultWarn, "network check is a stub in P01; full impl in P02"
Name: "db",
Description: "SQLite integrity_check + migration version",
Run: func(ctx context.Context) (Result, string) {
path := certpaths.DBPath()
db, err := store.Open(path)
if err != nil {
return ResultFail, fmt.Sprintf("open db: %v", err)
}
defer db.Close()
var integrity string
if err := db.QueryRowContext(ctx, "PRAGMA integrity_check").Scan(&integrity); err != nil {
return ResultFail, fmt.Sprintf("integrity_check: %v", err)
}
if !strings.EqualFold(integrity, "ok") {
return ResultFail, fmt.Sprintf("integrity_check: %s", integrity)
}
version, err := store.MigrationVersion(ctx, db)
if err != nil {
return ResultFail, fmt.Sprintf("migration version: %v", err)
}
if version == "" {
return ResultWarn, "integrity OK but no migrations applied (fresh db)"
}
return ResultPass, fmt.Sprintf("integrity OK, migrations up to %s", version)
},
}
}
// DBStub is a stub for the database check; full impl in P02.
func DBStub() Check {
// Network probes peer reachability via mTLS /healthz (REQ-032 completion).
// Peers are sourced from the persisted nodes table (not the in-memory
// PeerRegistry, which is empty at CLI time). Zero peers → WARN (single-node
// is legitimate). Any peer unreachable → FAIL (D-038).
func Network() Check {
return Check{
Name: "db",
Description: "SQLite open + migration apply (full impl in P02)",
Run: func(_ context.Context) (Result, string) {
return ResultWarn, "db check is a stub in P01; full impl in P02"
Name: "network",
Description: "peer reachability via mTLS /healthz probe",
Run: func(ctx context.Context) (Result, string) {
caPath := certpaths.CACertPath()
certPath := certpaths.ServerCertPath()
keyPath := certpaths.ServerKeyPath()
// Check that cert files exist before attempting probes.
if _, err := os.Stat(caPath); err != nil {
return ResultFail, fmt.Sprintf("CA cert missing: %v (run `orca cert init`)", err)
}
path := certpaths.DBPath()
db, err := store.Open(path)
if err != nil {
return ResultFail, fmt.Sprintf("open db: %v", err)
}
defer db.Close()
nodes, err := store.NewNodeRepo(db).List(ctx)
if err != nil {
return ResultFail, fmt.Sprintf("list nodes: %v", err)
}
live := make([]*model.Node, 0, len(nodes))
for _, n := range nodes {
if n.State != model.NodeStateLeft {
live = append(live, n)
}
}
if len(live) == 0 {
return ResultWarn, "no peers registered (single-node?)"
}
var lines []string
anyFail := false
for _, n := range live {
probeCtx, cancel := context.WithTimeout(ctx, 3*time.Second)
err := probeHealthz(probeCtx, caPath, certPath, keyPath, n.Name, n.Address)
cancel()
if err != nil {
anyFail = true
lines = append(lines, fmt.Sprintf(" ✗ %s (%s): %v", n.Name, n.Address, err))
} else {
lines = append(lines, fmt.Sprintf(" ✓ %s (%s)", n.Name, n.Address))
}
}
result := ResultPass
if anyFail {
result = ResultFail
}
return result, strings.Join(lines, "\n")
},
}
}
// probeHealthz opens an mTLS connection to the peer and GETs /healthz.
func probeHealthz(ctx context.Context, caPath, certPath, keyPath, serverName, addr string) error {
client, err := transport.NewMTLSClient(caPath, serverName, certPath, keyPath)
if err != nil {
return fmt.Errorf("mTLS client: %w", err)
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://"+addr+"/healthz", nil)
if err != nil {
return fmt.Errorf("request: %w", err)
}
resp, err := client.Do(req)
if err != nil {
return fmt.Errorf("probe: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("healthz returned %d", resp.StatusCode)
}
return nil
}
// loadCert reads a PEM cert from path and parses the first CERTIFICATE
// block.
func loadCert(path string) (*x509.Certificate, error) {
+191 -36
View File
@@ -2,60 +2,74 @@ package doctor
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
"time"
"git.cloudinit.dev/coreci/orca/internal/model"
"git.cloudinit.dev/coreci/orca/internal/security"
"git.cloudinit.dev/coreci/orca/internal/store"
)
// TestRunAllChecksWithNoCA runs the full battery in a clean temp dir
// and expects all checks to FAIL (no CA, no server cert) except the
// two stubs which return WARN.
// TestRunAllChecksWithNoCA runs the full battery in a clean temp dir.
// With the P02 real checks (no stubs): cert checks FAIL (no CA),
// db check PASS (store.Open runs migrations), network check WARN
// (no peers).
func TestRunAllChecksWithNoCA(t *testing.T) {
// Isolated home so we don't touch the real ~/.orca.
t.Setenv("ORCA_HOME", t.TempDir())
dir := t.TempDir()
t.Setenv("ORCA_HOME", dir)
t.Setenv("ORCA_DB", filepath.Join(dir, "orca.db"))
rep := Run(context.Background())
if len(rep.Checks) == 0 {
t.Fatal("expected checks, got 0")
}
hasFail := false
hasWarn := false
byName := make(map[string]CheckResult, len(rep.Checks))
for _, c := range rep.Checks {
if c.Result == ResultFail {
hasFail = true
}
if c.Result == ResultWarn {
hasWarn = true
}
}
if !hasFail {
t.Error("expected at least one FAIL (no CA installed)")
}
if !hasWarn {
t.Error("expected at least one WARN (stubs in P01)")
byName[c.Name] = c
}
// Render the report — basic shape check.
out := rep.Print()
if !strings.Contains(out, "PASS") {
t.Errorf("expected PASS in output, got: %s", out)
// Cert checks: no CA → FAIL.
for _, name := range []string{"cert.ca", "cert.server", "cert.expiry", "cert.fingerprint"} {
c, ok := byName[name]
if !ok {
t.Errorf("missing check %s", name)
continue
}
if c.Result != ResultFail {
t.Errorf("%s: got %s, want FAIL — %s", name, c.Result, c.Message)
}
}
if !strings.Contains(out, "WARN") {
t.Errorf("expected WARN in output, got: %s", out)
// DB check: store.Open runs migrations → PASS.
if c, ok := byName["db"]; ok {
if c.Result != ResultPass {
t.Errorf("db: got %s, want PASS — %s", c.Result, c.Message)
}
} else {
t.Error("missing check db")
}
if !strings.Contains(out, "FAIL") {
t.Errorf("expected FAIL in output, got: %s", out)
// Network check: no CA → FAIL (can't build mTLS client without CA).
if c, ok := byName["network"]; ok {
if c.Result != ResultFail {
t.Errorf("network: got %s, want FAIL (no CA cert) — %s", c.Result, c.Message)
}
} else {
t.Error("missing check network")
}
}
// TestRunWithCAAndServerCert covers the happy path: CA + server cert
// installed → all cert checks PASS.
// installed → all cert checks PASS, db PASS, network WARN (no peers).
func TestRunWithCAAndServerCert(t *testing.T) {
dir := t.TempDir()
t.Setenv("ORCA_HOME", dir)
t.Setenv("ORCA_DB", filepath.Join(dir, "orca.db"))
// Bootstrap CA.
if _, err := security.CAInit(dir, "test-ca"); err != nil {
t.Fatalf("CAInit: %v", err)
}
@@ -63,7 +77,6 @@ func TestRunWithCAAndServerCert(t *testing.T) {
if err != nil {
t.Fatalf("LoadCA: %v", err)
}
// Generate + sign server cert.
keyPEM, csrPEM, err := security.GenerateCSR("test-server", []string{"localhost", "127.0.0.1"})
if err != nil {
t.Fatalf("GenerateCSR: %v", err)
@@ -80,13 +93,155 @@ func TestRunWithCAAndServerCert(t *testing.T) {
}
rep := Run(context.Background())
// The cert-related checks should be PASS; the network/db stubs WARN.
byName := make(map[string]CheckResult, len(rep.Checks))
for _, c := range rep.Checks {
switch c.Name {
case "cert.ca", "cert.server", "cert.expiry", "cert.fingerprint":
if c.Result != ResultPass {
t.Errorf("%s: got %s, want PASS — %s", c.Name, c.Result, c.Message)
}
byName[c.Name] = c
}
for _, name := range []string{"cert.ca", "cert.server", "cert.expiry", "cert.fingerprint", "db"} {
c, ok := byName[name]
if !ok {
t.Errorf("missing check %s", name)
continue
}
if c.Result != ResultPass {
t.Errorf("%s: got %s, want PASS — %s", name, c.Result, c.Message)
}
}
if c, ok := byName["network"]; ok {
if c.Result != ResultWarn {
t.Errorf("network: got %s, want WARN (no peers) — %s", c.Result, c.Message)
}
}
}
// TestDBCheck_IntegrityOK verifies the DB check passes on a fresh
// database with migrations applied.
func TestDBCheck_IntegrityOK(t *testing.T) {
dir := t.TempDir()
t.Setenv("ORCA_HOME", dir)
t.Setenv("ORCA_DB", filepath.Join(dir, "orca.db"))
db, err := store.Open(filepath.Join(dir, "orca.db"))
if err != nil {
t.Fatalf("open db: %v", err)
}
defer db.Close()
c := DB()
r, msg := c.Run(context.Background())
if r != ResultPass {
t.Errorf("DB check: got %s, want PASS — %s", r, msg)
}
if !strings.Contains(msg, "0005") {
t.Errorf("DB check message should contain migration version, got: %s", msg)
}
}
// TestNetworkCheck_NoPeers verifies the network check returns WARN
// when no peers are registered.
func TestNetworkCheck_NoPeers(t *testing.T) {
dir := t.TempDir()
t.Setenv("ORCA_HOME", dir)
t.Setenv("ORCA_DB", filepath.Join(dir, "orca.db"))
// Create a CA + server cert so the network check can build a client.
if _, err := security.CAInit(dir, "test-ca"); err != nil {
t.Fatalf("CAInit: %v", err)
}
ca, _ := security.LoadCA(dir)
keyPEM, csrPEM, _ := security.GenerateCSR("test-server", []string{"localhost"})
certPEM, _ := ca.SignCSR(csrPEM)
_ = security.WriteCert(dir+"/server.crt", certPEM)
_ = security.WriteKey(dir+"/server.key", keyPEM)
c := Network()
r, msg := c.Run(context.Background())
if r != ResultWarn {
t.Errorf("Network check: got %s, want WARN — %s", r, msg)
}
if !strings.Contains(msg, "no peers") {
t.Errorf("Network check message should mention no peers, got: %s", msg)
}
}
// TestNetworkCheck_PeerUnreachable verifies the network check returns
// FAIL when a registered peer is not reachable.
func TestNetworkCheck_PeerUnreachable(t *testing.T) {
dir := t.TempDir()
t.Setenv("ORCA_HOME", dir)
t.Setenv("ORCA_DB", filepath.Join(dir, "orca.db"))
// Create a CA + server cert.
if _, err := security.CAInit(dir, "test-ca"); err != nil {
t.Fatalf("CAInit: %v", err)
}
ca, _ := security.LoadCA(dir)
keyPEM, csrPEM, _ := security.GenerateCSR("test-server", []string{"localhost"})
certPEM, _ := ca.SignCSR(csrPEM)
_ = security.WriteCert(dir+"/server.crt", certPEM)
_ = security.WriteKey(dir+"/server.key", keyPEM)
// Insert a peer node with an unreachable address.
db, err := store.Open(filepath.Join(dir, "orca.db"))
if err != nil {
t.Fatalf("open db: %v", err)
}
defer db.Close()
repo := store.NewNodeRepo(db)
_ = repo.Insert(context.Background(), &model.Node{
ID: "dead-peer", Name: "dead", Address: "127.0.0.1:1",
State: model.NodeStateReady, JoinedAt: time.Now().UTC(), LastSeen: time.Now().UTC(),
})
c := Network()
r, msg := c.Run(context.Background())
if r != ResultFail {
t.Errorf("Network check: got %s, want FAIL — %s", r, msg)
}
if !strings.Contains(msg, "dead") {
t.Errorf("Network check message should mention the dead peer, got: %s", msg)
}
}
// TestNetworkCheck_NoCert verifies the network check returns FAIL
// when no CA cert is installed.
func TestNetworkCheck_NoCert(t *testing.T) {
dir := t.TempDir()
t.Setenv("ORCA_HOME", dir)
t.Setenv("ORCA_DB", filepath.Join(dir, "orca.db"))
c := Network()
r, msg := c.Run(context.Background())
if r != ResultFail {
t.Errorf("Network check: got %s, want FAIL — %s", r, msg)
}
if !strings.Contains(msg, "CA cert missing") {
t.Errorf("Network check message should mention missing CA, got: %s", msg)
}
}
// TestRenderReport verifies the report output format.
func TestRenderReport(t *testing.T) {
dir := t.TempDir()
t.Setenv("ORCA_HOME", dir)
t.Setenv("ORCA_DB", filepath.Join(dir, "orca.db"))
rep := Run(context.Background())
out := rep.Print()
if !strings.Contains(out, "PASS") {
t.Errorf("expected PASS in output, got: %s", out)
}
if !strings.Contains(out, "WARN") {
t.Errorf("expected WARN in output, got: %s", out)
}
if !strings.Contains(out, "FAIL") {
t.Errorf("expected FAIL in output, got: %s", out)
}
}
func init() {
// Suppress slog noise during tests.
_ = os.Setenv("ORCA_LOG_LEVEL", "error")
}
+54
View File
@@ -6,11 +6,19 @@ import (
"encoding/json"
"errors"
"fmt"
"iter"
"log/slog"
"time"
"git.cloudinit.dev/coreci/orca/internal/model"
)
// watchInterval is the poll cadence used by JobRepo.Watch and NodeRepo.Watch.
// It is an unexported package var (default 1s) so tests can override it to a
// small value for deterministic assertions (D-043). Do not change it from
// production code paths.
var watchInterval = 1 * time.Second
type JobRepo struct {
db *sql.DB
}
@@ -59,6 +67,52 @@ func (r *JobRepo) List(ctx context.Context) ([]*model.Job, error) {
return jobs, rows.Err()
}
// Watch yields the full snapshot of jobs on a watchInterval ticker until ctx
// is cancelled or the consumer stops pulling (yield returns false). It does
// not spawn a goroutine; the polling loop runs inline in the caller's
// goroutine via the range-over-func pull protocol (D-032).
//
// Each tick re-runs the List query and yields one []*model.Job snapshot
// containing ALL rows for that tick (G-001). The first yield happens
// immediately before the first ticker wait, so the consumer sees the initial
// state with no watchInterval delay (G-002). Transient query/scan errors are
// logged via slog.Default().Warn and the loop continues to the next tick
// rather than terminating the stream (D-034 lite). The ticker is stopped and
// rows are closed on every exit path (ctx.Done, yield==false, scan error).
func (r *JobRepo) Watch(ctx context.Context) iter.Seq[[]*model.Job] {
return func(yield func([]*model.Job) bool) {
ticker := time.NewTicker(watchInterval)
defer ticker.Stop()
for {
rows, err := r.db.QueryContext(ctx,
`SELECT id, name, spec, status, exit_code, created_at, started_at, ended_at FROM jobs ORDER BY created_at DESC`)
if err != nil {
slog.Default().Warn("watch jobs: query failed", "error", err)
// fall through to the select to wait for the next tick
} else {
snapshot := make([]*model.Job, 0)
for rows.Next() {
j, scanErr := scanJob(rows)
if scanErr != nil {
slog.Default().Warn("watch jobs: scan failed", "error", scanErr)
continue
}
snapshot = append(snapshot, j)
}
rows.Close()
if !yield(snapshot) {
return // consumer stopped pulling
}
}
select {
case <-ctx.Done():
return
case <-ticker.C:
}
}
}
}
func (r *JobRepo) UpdateStatus(ctx context.Context, id string, status model.JobStatus, exitCode int) error {
now := time.Now().UTC()
var startedAt, endedAt *time.Time
+201
View File
@@ -0,0 +1,201 @@
package store
import (
"context"
"path/filepath"
"testing"
"time"
"git.cloudinit.dev/coreci/orca/internal/model"
)
func openJobTestDB(t *testing.T) (*JobRepo, func()) {
t.Helper()
path := filepath.Join(t.TempDir(), "test.db")
db, err := Open(path)
if err != nil {
t.Fatalf("open db: %v", err)
}
return NewJobRepo(db), func() { _ = db.Close() }
}
func insertJob(t *testing.T, repo *JobRepo, ctx context.Context, id, name string) {
t.Helper()
if err := repo.Insert(ctx, &model.Job{
ID: id,
Name: name,
Spec: "test",
Status: model.JobStatusPending,
}); err != nil {
t.Fatalf("insert job %s: %v", id, err)
}
}
// withFastWatch sets watchInterval to a small value for deterministic tests and
// restores the default (1s) on cleanup.
func withFastWatch(t *testing.T, d time.Duration) {
t.Helper()
prev := watchInterval
watchInterval = d
t.Cleanup(func() { watchInterval = prev })
}
// TestJobRepoWatch_YieldsSnapshots verifies each yield is a complete tick
// snapshot (G-001): the first snapshot contains only the first job, and a
// later snapshot contains both jobs after a second insert.
func TestJobRepoWatch_YieldsSnapshots(t *testing.T) {
withFastWatch(t, 10*time.Millisecond)
repo, cleanup := openJobTestDB(t)
defer cleanup()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
insertJob(t, repo, ctx, "job-1", "alpha")
var snapshots [][]*model.Job
done := make(chan struct{})
go func() {
defer close(done)
for snap := range repo.Watch(ctx) {
snapshots = append(snapshots, snap)
if len(snapshots) >= 40 {
cancel()
return
}
}
}()
// Insert a second job after a short delay so a later tick observes it.
// Use a background context — the watch ctx may be cancelled by the
// goroutine above once it collects enough snapshots.
time.Sleep(100 * time.Millisecond)
insertJob(t, repo, context.Background(), "job-2", "beta")
select {
case <-done:
case <-time.After(2 * time.Second):
t.Fatal("watch did not complete within 2s")
}
if len(snapshots) == 0 {
t.Fatal("expected at least one snapshot, got none")
}
// First snapshot must contain only the first job (G-001).
if len(snapshots[0]) != 1 || snapshots[0][0].ID != "job-1" {
t.Errorf("first snapshot = %+v, want only job-1", snapshots[0])
}
// At least one later snapshot must contain both jobs.
foundBoth := false
for _, snap := range snapshots[1:] {
ids := make(map[string]bool, len(snap))
for _, j := range snap {
ids[j.ID] = true
}
if ids["job-1"] && ids["job-2"] {
foundBoth = true
break
}
}
if !foundBoth {
t.Errorf("no snapshot contained both jobs; snapshots=%v", snapshots)
}
}
// TestJobRepoWatch_ImmediateFirstYield verifies G-002: the first snapshot
// arrives before the first ticker wait, i.e. well under the watchInterval.
func TestJobRepoWatch_ImmediateFirstYield(t *testing.T) {
withFastWatch(t, 200*time.Millisecond)
repo, cleanup := openJobTestDB(t)
defer cleanup()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
insertJob(t, repo, ctx, "job-immediate", "first")
start := time.Now()
var firstSnap []*model.Job
got := make(chan struct{})
go func() {
for snap := range repo.Watch(ctx) {
firstSnap = snap
close(got)
cancel()
return
}
}()
select {
case <-got:
case <-time.After(100 * time.Millisecond):
t.Fatal("first yield took >100ms; expected immediate (G-002)")
}
elapsed := time.Since(start)
if elapsed > 100*time.Millisecond {
t.Errorf("first yield took %v; expected immediate (G-002)", elapsed)
}
if len(firstSnap) != 1 || firstSnap[0].ID != "job-immediate" {
t.Errorf("first snapshot = %+v, want job-immediate", firstSnap)
}
}
// TestJobRepoWatch_StopsOnConsumerBreak verifies the yield==false path: the
// range loop returns promptly when the consumer breaks after the first yield.
func TestJobRepoWatch_StopsOnConsumerBreak(t *testing.T) {
withFastWatch(t, 10*time.Millisecond)
repo, cleanup := openJobTestDB(t)
defer cleanup()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
insertJob(t, repo, ctx, "job-break", "break")
done := make(chan struct{})
go func() {
defer close(done)
for range repo.Watch(ctx) {
break // stop pulling immediately after the first snapshot
}
}()
select {
case <-done:
// success: range returned
case <-time.After(500 * time.Millisecond):
t.Fatal("watch did not stop on consumer break within 500ms")
}
}
// TestJobRepoWatch_StopsOnCtxCancel verifies the loop exits promptly after
// ctx is cancelled.
func TestJobRepoWatch_StopsOnCtxCancel(t *testing.T) {
withFastWatch(t, 10*time.Millisecond)
repo, cleanup := openJobTestDB(t)
defer cleanup()
ctx, cancel := context.WithCancel(context.Background())
insertJob(t, repo, ctx, "job-cancel", "cancel")
done := make(chan struct{})
go func() {
defer close(done)
for range repo.Watch(ctx) {
// drain until cancelled
}
}()
// Let at least one tick land, then cancel.
time.Sleep(20 * time.Millisecond)
cancel()
select {
case <-done:
// success
case <-time.After(500 * time.Millisecond):
t.Fatal("watch did not stop on ctx cancel within 500ms")
}
}
+15
View File
@@ -12,6 +12,21 @@ import (
//go:embed migrations/*.sql
var migrationsFS embed.FS
// MigrationVersion returns the name of the highest applied migration
// (e.g. "0005_node_capacity.sql"). Returns ("", nil) if no migrations
// have been applied (fresh or empty database).
func MigrationVersion(ctx context.Context, db *sql.DB) (string, error) {
var name string
err := db.QueryRowContext(ctx, `SELECT name FROM schema_migrations ORDER BY name DESC LIMIT 1`).Scan(&name)
if err == sql.ErrNoRows {
return "", nil
}
if err != nil {
return "", fmt.Errorf("query migration version: %w", err)
}
return name, nil
}
func migrate(db *sql.DB) error {
entries, err := migrationsFS.ReadDir("migrations")
if err != nil {
+37
View File
@@ -0,0 +1,37 @@
package store
import (
"context"
"path/filepath"
"testing"
)
func TestMigrationVersion(t *testing.T) {
dir := t.TempDir()
db, err := Open(filepath.Join(dir, "test.db"))
if err != nil {
t.Fatalf("open db: %v", err)
}
defer db.Close()
ctx := context.Background()
version, err := MigrationVersion(ctx, db)
if err != nil {
t.Fatalf("migration version: %v", err)
}
if version != "0005_node_capacity.sql" {
t.Errorf("MigrationVersion = %q, want 0005_node_capacity.sql", version)
}
// Empty the migrations table → should return ("", nil).
if _, err := db.ExecContext(ctx, "DELETE FROM schema_migrations"); err != nil {
t.Fatalf("clear migrations: %v", err)
}
version, err = MigrationVersion(ctx, db)
if err != nil {
t.Fatalf("migration version after clear: %v", err)
}
if version != "" {
t.Errorf("MigrationVersion after clear = %q, want empty", version)
}
}
+36
View File
@@ -6,6 +6,8 @@ import (
"encoding/json"
"errors"
"fmt"
"iter"
"log/slog"
"time"
"git.cloudinit.dev/coreci/orca/internal/model"
@@ -69,6 +71,40 @@ func (r *NodeRepo) List(ctx context.Context) ([]*model.Node, error) {
return nodes, rows.Err()
}
func (r *NodeRepo) Watch(ctx context.Context) iter.Seq[[]*model.Node] {
return func(yield func([]*model.Node) bool) {
ticker := time.NewTicker(watchInterval)
defer ticker.Stop()
for {
rows, err := r.db.QueryContext(ctx,
`SELECT id, name, address, state, joined_at, last_seen, metadata FROM nodes ORDER BY joined_at ASC`)
if err != nil {
slog.Default().Warn("watch nodes: query failed", "error", err)
// fall through to the select to wait for the next tick
} else {
snapshot := make([]*model.Node, 0)
for rows.Next() {
n, scanErr := scanNode(rows)
if scanErr != nil {
slog.Default().Warn("watch nodes: scan failed", "error", scanErr)
continue
}
snapshot = append(snapshot, n)
}
rows.Close()
if !yield(snapshot) {
return // consumer stopped pulling
}
}
select {
case <-ctx.Done():
return
case <-ticker.C:
}
}
}
}
func (r *NodeRepo) UpdateState(ctx context.Context, id string, state model.NodeState) error {
res, err := r.db.ExecContext(ctx,
`UPDATE nodes SET state = ?, last_seen = ? WHERE id = ?`,
+156
View File
@@ -100,3 +100,159 @@ func TestNodeRepo_Delete(t *testing.T) {
t.Errorf("expected ErrNotFound, got %v", err)
}
}
func insertNode(t *testing.T, repo *NodeRepo, ctx context.Context, id, name string) {
t.Helper()
if err := repo.Insert(ctx, &model.Node{
ID: id,
Name: name,
Address: "addr",
State: model.NodeStateReady,
JoinedAt: time.Now().UTC(),
LastSeen: time.Now().UTC(),
}); err != nil {
t.Fatalf("insert node %s: %v", id, err)
}
}
func TestNodeRepoWatch_YieldsSnapshots(t *testing.T) {
withFastWatch(t, 10*time.Millisecond)
repo, cleanup := openTestDB(t)
defer cleanup()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
insertNode(t, repo, ctx, "node-1", "alpha")
var snapshots [][]*model.Node
done := make(chan struct{})
go func() {
defer close(done)
for snap := range repo.Watch(ctx) {
snapshots = append(snapshots, snap)
if len(snapshots) >= 40 {
cancel()
return
}
}
}()
time.Sleep(100 * time.Millisecond)
insertNode(t, repo, context.Background(), "node-2", "beta")
select {
case <-done:
case <-time.After(2 * time.Second):
t.Fatal("watch did not complete within 2s")
}
if len(snapshots) == 0 {
t.Fatal("expected at least one snapshot, got none")
}
if len(snapshots[0]) != 1 || snapshots[0][0].ID != "node-1" {
t.Errorf("first snapshot = %+v, want only node-1", snapshots[0])
}
foundBoth := false
for _, snap := range snapshots[1:] {
ids := make(map[string]bool, len(snap))
for _, n := range snap {
ids[n.ID] = true
}
if ids["node-1"] && ids["node-2"] {
foundBoth = true
break
}
}
if !foundBoth {
t.Errorf("no snapshot contained both nodes; snapshots=%v", snapshots)
}
}
func TestNodeRepoWatch_ImmediateFirstYield(t *testing.T) {
withFastWatch(t, 200*time.Millisecond)
repo, cleanup := openTestDB(t)
defer cleanup()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
insertNode(t, repo, ctx, "node-immediate", "first")
start := time.Now()
var firstSnap []*model.Node
got := make(chan struct{})
go func() {
for snap := range repo.Watch(ctx) {
firstSnap = snap
close(got)
cancel()
return
}
}()
select {
case <-got:
case <-time.After(100 * time.Millisecond):
t.Fatal("first yield took >100ms; expected immediate (G-002)")
}
elapsed := time.Since(start)
if elapsed > 100*time.Millisecond {
t.Errorf("first yield took %v; expected immediate (G-002)", elapsed)
}
if len(firstSnap) != 1 || firstSnap[0].ID != "node-immediate" {
t.Errorf("first snapshot = %+v, want node-immediate", firstSnap)
}
}
func TestNodeRepoWatch_StopsOnConsumerBreak(t *testing.T) {
withFastWatch(t, 10*time.Millisecond)
repo, cleanup := openTestDB(t)
defer cleanup()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
insertNode(t, repo, ctx, "node-break", "break")
done := make(chan struct{})
go func() {
defer close(done)
for range repo.Watch(ctx) {
break
}
}()
select {
case <-done:
case <-time.After(500 * time.Millisecond):
t.Fatal("watch did not stop on consumer break within 500ms")
}
}
func TestNodeRepoWatch_StopsOnCtxCancel(t *testing.T) {
withFastWatch(t, 10*time.Millisecond)
repo, cleanup := openTestDB(t)
defer cleanup()
ctx, cancel := context.WithCancel(context.Background())
insertNode(t, repo, ctx, "node-cancel", "cancel")
done := make(chan struct{})
go func() {
defer close(done)
for range repo.Watch(ctx) {
}
}()
time.Sleep(20 * time.Millisecond)
cancel()
select {
case <-done:
case <-time.After(500 * time.Millisecond):
t.Fatal("watch did not stop on ctx cancel within 500ms")
}
}