orca/.ciagent/PROJECT.md

# Project: Orca

## What This Is

A minimalist, offline-first, CLI-first orchestration engine inspired by HashiCorp Nomad, prioritizing stability, security, and simplicity over feature richness. Single-binary distribution, no container runtime, no cloud dependencies, no K8s-level complexity.

## Vision

A minimalist, offline-first, CLI-first orchestration engine inspired by HashiCorp Nomad, prioritizing stability, security, and simplicity over feature richness.

## Objective
Build a lightweight system to manage and execute workloads across a set of nodes, keeping complexity far below that of Kubernetes.

## Requirements
- **CLI First**: Primary interaction through a CLI tool.
- **Offline First**: Functional without constant internet connectivity.
- **AI First**: Designed to be easily discoverable and manageable by AI agents.
- **Stability & Security**: Prioritize security fixes and bug fixes over new features.
- **Language**: Written in Go 1.25+.
- **Simplicity**: Minimalist implementation, avoiding the "K8s complexity trap".

## Constraints
- No web UI as a primary requirement.
- Must not implement K8s-level complexity.
- Feature development must move slowly to ensure stability.
- Only CI system allowed: CoreCI (git.cloudinit.dev/coreci/coreci).
- Gitea remote: git.cloudinit.dev/coreci/orca.

## Clarified Decisions (D-series, full autonomy)

| ID | Question | Decision | Rationale | Confidence |
|----|----------|----------|-----------|------------|
| D-001 | Single binary or multi-binary distribution? | **Single binary** | Simpler distribution; subcommands baked into one `orca` binary. Aligns with simplicity pillar. | 0.95 |
| D-002 | Local state store technology? | **modernc/sqlite (pure Go, CGO-free)** | Cross-compile friendly, no CGO dependency, single file on disk, mature. | 0.92 |
| D-003 | Inter-node communication? | **Embedded HTTP (net/http) over loopback, mTLS for cross-node** | No external RPC framework needed for v0.1. HTTP suffices. | 0.85 |
| D-004 | Scheduling algorithm for v0.1? | **Single-node only (no scheduling)** | Multi-node scheduling is out of scope for v0.1. Tasks run on the node they're submitted to. | 0.90 |
| D-005 | CLI output format? | **Human-readable by default, `--json` flag for machine consumption** | Serves both humans and AI agents. | 0.95 |
| D-006 | Job/task definition format? | **HCL or YAML in `.hcl`/`.yaml` files** | Familiar to Nomad/HashiCorp users; simpler than JSON for humans. | 0.88 |
| D-007 | Authentication? | **mTLS for v0.1, token-based deferred** | mTLS is the most secure default. Tokens can be added later if needed. | 0.80 |
| D-008 | Container runtime? | **Direct process execution (no container runtime) for v0.1** | Avoids the Docker/container dependency. Pure process management. | 0.85 |
| D-009 | Configuration file location? | **`~/.orca/config.hcl` and `/etc/orca/orca.hcl`** | Standard XDG-style paths. | 0.90 |
| D-010 | Logging format? | **Structured JSON via `log/slog`** | Native Go 1.21+ slog, no external dependency. | 0.95 |
| D-011 | v0.2 mTLS cert authority model? | **Internal CA with CSR join** | One node bootstraps a local CA; peers generate CSRs and submit them to the CA for signing. CA cert is the trust anchor. More secure than self-signed per-node (single trust root) without the operational complexity of an external PKI. | 0.92 |
| D-012 | v0.2 CA bootstrap & cert distribution? | **Operator-mediated, fingerprint-verified** | Bootstrap node writes `~/.orca/ca.crt` and `~/.orca/ca.key` (mode 0600). Operator copies `ca.crt` to peers; peers verify by SHA-256 fingerprint at `orca node join --ca-fingerprint <sha256>`. No automated secret distribution. | 0.85 |
| D-013 | v0.2 cert validity & rotation? | **Server certs 90 days, CA cert 10 years, rotate 30 days before expiry** | Server certs are short-lived (compromise window small); CA is long-lived (manual rotation is expensive). `orca cert renew` reissues server certs automatically. | 0.90 |
| D-014 | v0.2 mTLS handshake timing? | **Eager — at `orca node join` time** | Fail fast on bad certs, misconfigurations, or CA mismatches. Lazy handshake would let stale configs run until first request, complicating debugging. | 0.88 |
| D-015 | v0.2 minimum TLS version & cipher suites? | **TLS 1.3 only; AEAD cipher allowlist** | MinVersion=tls.VersionTLS13, CipherSuites limited to TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_AES_128_GCM_SHA256. No TLS 1.2 fallback. | 0.92 |
| D-016 | v0.2 security-scanning placement? | **`validate` pipeline of `.coreci.yml`, gates merges to main** | `gosec` baseline JSON checked into repo; new findings fail the build. `govulncheck ./...` exit-on-known. Pre-commit hook with `gitleaks` is opt-in (developer machine). | 0.85 |
| D-017 | v0.2 `iter.Seq` API surface? | **`orca job list --watch` and `orca node list --watch`** | Pull-based `iter.Seq[Job]` / `iter.Seq[Node]`; cancellation via `context.Context`; `signal.NotifyContext` on ctrl-c. Backpressure is implicit (consumer-driven). | 0.90 |
| D-018 | v0.2 multi-node scheduling algorithm? | **Bin-packing by available CPU/memory, FIFO within a node** | Simple, deterministic, matches D-004 minimalism. Cross-node dispatch via ConnectRPC `orca.v1.Dispatch` service. Retry on transient failures with exponential backoff. | 0.85 |

## Out of Scope
- Full-blown Kubernetes-compatible API.
- Complex cloud-provider integrations.
- GUI-based management consoles.
- Container runtime integration.
- Service mesh / sidecar injection.
- Auto-scaling / horizontal pod autoscaler.
- External PKI / Let's Encrypt / cert transparency logs.
- gRPC framework dependency (ConnectRPC in `config.json` frameworks but
  not in `go.mod`; v0.2 uses stdlib `net/http` with h2c for
  `orca.v1.Dispatch` — see ARCHITECTURE.md AD-014).

## v0.2 Scope Summary

v0.2 is a focused 4-phase milestone that turns Orca from a single-node
process executor into a small cluster engine with strong transport
security and richer I/O. The 4 phases are:

- **P01 — mTLS handshake + internal CA with CSR join.** Internal CA, CSR
  join, eager handshake at `node join`, TLS 1.3 + AEAD allowlist.
  See ARCHITECTURE.md Flow 1 + Flow 2.
- **P02 — Multi-node scheduling & job dispatch.** Best-fit bin-packing by
  CPU/memory, FIFO within a node, `orca.v1.Dispatch` over mTLS. See
  ARCHITECTURE.md Flow 3.
- **P03 — `gosec` + `govulncheck` + `gitleaks` in CI.** `gosec` baseline
  JSON in repo, `govulncheck ./...` in `validate` pipeline, `gitleaks`
  in pre-commit (opt-in).
- **P04 — `iter.Seq` streaming for `--watch` flags.** Go 1.25+ range-over-func
  semantics, `context.Context` cancellation, `signal.NotifyContext` on
  ctrl-c. See ARCHITECTURE.md Flow 4.

The vision ("minimalist, offline-first, CLI-first orchestration engine")
is unchanged. v0.2 is a hardening + small-cluster extension, not a
direction change.

## Key Decisions

The 18 D-series decisions (D-001..D-018) are recorded in the "Clarified
Decisions" table above. The 10 v0.1 decisions (D-001..D-010) are stable
and unchanged in v0.2. The 8 v0.2 decisions (D-011..D-018) were
auto-resolved under full autonomy and are summarized here:

- **D-011: Internal CA with CSR join** (vs. self-signed per-node or SPIFFE).
  Single trust root, no external PKI, CSR workflow.
- **D-012: Operator-mediated CA cert distribution with fingerprint verify**
  (no automated secret distribution — matches offline-first principle).
- **D-013: 90d server certs, 10y CA cert, 30d pre-expiry rotation.**
- **D-014: Eager mTLS handshake at `orca node join` time** (fail fast).
- **D-015: TLS 1.3 only, AEAD cipher allowlist** (no TLS 1.2 fallback).
- **D-016: `gosec`+`govulncheck` in `validate` pipeline of `.coreci.yml`**
  (gates merges to main). `gitleaks` in pre-commit (opt-in).
- **D-017: `iter.Seq` for `orca job list --watch` and `orca node list --watch`**
  (pull-based, ctx cancellation, ctrl-c via `signal.NotifyContext`).
- **D-018: Bin-packing by CPU/memory with FIFO within node; JSON-over-HTTP
  orca.v1.Dispatch for cross-node** (no ConnectRPC dep).

## v0.3 Clarified Decisions (D-series, full autonomy)

v0.3 is a lean 2-execution-phase milestone completing the streaming and
doctor work deferred from v0.2. The 6 v0.3 decisions (D-019..D-024)
were auto-resolved under full autonomy:

| ID | Question | Decision | Rationale | Confidence |
|----|----------|----------|-----------|------------|
| D-019 | Watch refresh mechanism? | **Poll-based, 1s ticker** | Simpler than event channel; no daemon coupling for CLI; matches offline-first. | 0.90 |
| D-020 | Watch output format (REQ-030)? | **Table by default; `--watch --json` streams one-line JSON per event** | Consistent with D-005 `--json` convention; serves humans + AI agents. | 0.92 |
| D-021 | Doctor network check scope? | **Probe configured peer addresses via mTLS `/healthz` handshake; PASS/WARN/FAIL per peer** | Reuses existing transport client; read-only. | 0.85 |
| D-022 | Doctor db check scope? | **`PRAGMA integrity_check` + migration version query** | Already specced in ARCHITECTURE.md §5; minimal surface. | 0.95 |
| D-023 | iter.Seq cancellation? | **`signal.NotifyContext` on SIGINT/SIGTERM** | Per D-017 + ARCHITECTURE Flow 4. | 0.95 |
| D-024 | `--watch` applies to job list only, or node list too? | **Both `orca job list --watch` and `orca node list --watch`** | Per ARCHITECTURE.md CLI layer + D-017. | 0.92 |

## v0.3 Scope Summary

v0.3 is a focused 2-execution-phase milestone completing the work
deferred from v0.2 that was NOT already shipped in P08-P10. A codebase
audit during re-init SPECIFY confirmed that REQ-014, REQ-027, REQ-028,
REQ-029, REQ-031, REQ-037, REQ-039, REQ-040 all shipped in P08-P10
despite stale REQUIREMENTS.md marking them Pending. The remaining work:

- **P01 — `iter.Seq` streaming for `--watch` flags.** Go 1.25+
  range-over-func semantics, pull-based `iter.Seq[Job]` /
  `iter.Seq[Node]`, `context.Context` cancellation,
  `signal.NotifyContext` on ctrl-c. Applies to both `orca job list
  --watch` and `orca node list --watch`. Covers REQ-022, REQ-030.
- **P02 — `orca doctor` network + db full implementation.** Replaces
  the P01 stubs (`NetworkStub`, `DBStub`) with real checks: peer
  reachability via mTLS `/healthz` probe; SQLite `PRAGMA
  integrity_check` + migration version. Covers REQ-032 (completion).

The vision ("minimalist, offline-first, CLI-first orchestration
engine") is unchanged. v0.3 is a completion milestone, not a direction
change.