orca/.ciagent/PROJECT.md

Project: Orca

What This Is

A minimalist, offline-first, CLI-first orchestration engine inspired by HashiCorp Nomad, prioritizing stability, security, and simplicity over feature richness. Single-binary distribution, no container runtime, no cloud dependencies, no K8s-level complexity.

Vision

A minimalist, offline-first, CLI-first orchestration engine inspired by HashiCorp Nomad, prioritizing stability, security, and simplicity over feature richness.

Objective

Build a lightweight system to manage and execute workloads across a set of nodes, keeping complexity far below that of Kubernetes.

Requirements

• CLI First: Primary interaction through a CLI tool.
• Offline First: Functional without constant internet connectivity.
• AI First: Designed to be easily discoverable and manageable by AI agents.
• Stability & Security: Prioritize security fixes and bug fixes over new features.
• Language: Written in Go 1.25+.
• Simplicity: Minimalist implementation, avoiding the "K8s complexity trap".

Constraints

• No web UI as a primary requirement.
• Must not implement K8s-level complexity.
• Feature development must move slowly to ensure stability.
• Only CI system allowed: CoreCI (git.cloudinit.dev/coreci/coreci).
• Gitea remote: git.cloudinit.dev/coreci/orca.

Clarified Decisions (D-series, full autonomy)

ID	Question	Decision	Rationale	Confidence
D-001	Single binary or multi-binary distribution?	Single binary	Simpler distribution; subcommands baked into one orca binary. Aligns with simplicity pillar.	0.95
D-002	Local state store technology?	modernc/sqlite (pure Go, CGO-free)	Cross-compile friendly, no CGO dependency, single file on disk, mature.	0.92
D-003	Inter-node communication?	Embedded HTTP (net/http) over loopback, mTLS for cross-node	No external RPC framework needed for v0.1. HTTP suffices.	0.85
D-004	Scheduling algorithm for v0.1?	Single-node only (no scheduling)	Multi-node scheduling is out of scope for v0.1. Tasks run on the node they're submitted to.	0.90
D-005	CLI output format?	Human-readable by default, --json flag for machine consumption	Serves both humans and AI agents.	0.95
D-006	Job/task definition format?	HCL or YAML in .hcl/.yaml files	Familiar to Nomad/HashiCorp users; simpler than JSON for humans.	0.88
D-007	Authentication?	mTLS for v0.1, token-based deferred	mTLS is the most secure default. Tokens can be added later if needed.	0.80
D-008	Container runtime?	Direct process execution (no container runtime) for v0.1	Avoids the Docker/container dependency. Pure process management.	0.85
D-009	Configuration file location?	~/.orca/config.hcl and /etc/orca/orca.hcl	Standard XDG-style paths.	0.90
D-010	Logging format?	Structured JSON via log/slog	Native Go 1.21+ slog, no external dependency.	0.95
D-011	v0.2 mTLS cert authority model?	Internal CA with CSR join	One node bootstraps a local CA; peers generate CSRs and submit them to the CA for signing. CA cert is the trust anchor. More secure than self-signed per-node (single trust root) without the operational complexity of an external PKI.	0.92
D-012	v0.2 CA bootstrap & cert distribution?	Operator-mediated, fingerprint-verified	Bootstrap node writes ~/.orca/ca.crt and ~/.orca/ca.key (mode 0600). Operator copies ca.crt to peers; peers verify by SHA-256 fingerprint at orca node join --ca-fingerprint <sha256>. No automated secret distribution.	0.85
D-013	v0.2 cert validity & rotation?	Server certs 90 days, CA cert 10 years, rotate 30 days before expiry	Server certs are short-lived (compromise window small); CA is long-lived (manual rotation is expensive). orca cert renew reissues server certs automatically.	0.90
D-014	v0.2 mTLS handshake timing?	Eager — at orca node join time	Fail fast on bad certs, misconfigurations, or CA mismatches. Lazy handshake would let stale configs run until first request, complicating debugging.	0.88
D-015	v0.2 minimum TLS version & cipher suites?	TLS 1.3 only; AEAD cipher allowlist	MinVersion=tls.VersionTLS13, CipherSuites limited to TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_AES_128_GCM_SHA256. No TLS 1.2 fallback.	0.92
D-016	v0.2 security-scanning placement?	validate pipeline of .coreci.yml, gates merges to main	gosec baseline JSON checked into repo; new findings fail the build. govulncheck ./... exit-on-known. Pre-commit hook with gitleaks is opt-in (developer machine).	0.85
D-017	v0.2 iter.Seq API surface?	orca job list --watch and orca node list --watch	Pull-based iter.Seq[Job] / iter.Seq[Node]; cancellation via context.Context; signal.NotifyContext on ctrl-c. Backpressure is implicit (consumer-driven).	0.90
D-018	v0.2 multi-node scheduling algorithm?	Bin-packing by available CPU/memory, FIFO within a node	Simple, deterministic, matches D-004 minimalism. Cross-node dispatch via ConnectRPC orca.v1.Dispatch service. Retry on transient failures with exponential backoff.	0.85

Out of Scope

• Full-blown Kubernetes-compatible API.
• Complex cloud-provider integrations.
• GUI-based management consoles.
• Container runtime integration.
• Service mesh / sidecar injection.
• Auto-scaling / horizontal pod autoscaler.
• External PKI / Let's Encrypt / cert transparency logs.
• gRPC framework dependency (ConnectRPC in config.json frameworks but not in go.mod; v0.2 uses stdlib net/http with h2c for orca.v1.Dispatch — see ARCHITECTURE.md AD-014).

v0.2 Scope Summary

v0.2 is a focused 4-phase milestone that turns Orca from a single-node process executor into a small cluster engine with strong transport security and richer I/O. The 4 phases are:

• P01 — mTLS handshake + internal CA with CSR join. Internal CA, CSR join, eager handshake at node join, TLS 1.3 + AEAD allowlist. See ARCHITECTURE.md Flow 1 + Flow 2.
• P02 — Multi-node scheduling & job dispatch. Best-fit bin-packing by CPU/memory, FIFO within a node, orca.v1.Dispatch over mTLS. See ARCHITECTURE.md Flow 3.
• P03 — gosec + govulncheck + gitleaks in CI. gosec baseline JSON in repo, govulncheck ./... in validate pipeline, gitleaks in pre-commit (opt-in).
• P04 — iter.Seq streaming for --watch flags. Go 1.25+ range-over-func semantics, context.Context cancellation, signal.NotifyContext on ctrl-c. See ARCHITECTURE.md Flow 4.

The vision ("minimalist, offline-first, CLI-first orchestration engine") is unchanged. v0.2 is a hardening + small-cluster extension, not a direction change.

Key Decisions

The 18 D-series decisions (D-001..D-018) are recorded in the "Clarified Decisions" table above. The 10 v0.1 decisions (D-001..D-010) are stable and unchanged in v0.2. The 8 v0.2 decisions (D-011..D-018) were auto-resolved under full autonomy and are summarized here:

• D-011: Internal CA with CSR join (vs. self-signed per-node or SPIFFE). Single trust root, no external PKI, CSR workflow.
• D-012: Operator-mediated CA cert distribution with fingerprint verify (no automated secret distribution — matches offline-first principle).
• D-013: 90d server certs, 10y CA cert, 30d pre-expiry rotation.
• D-014: Eager mTLS handshake at orca node join time (fail fast).
• D-015: TLS 1.3 only, AEAD cipher allowlist (no TLS 1.2 fallback).
• D-016: gosec+govulncheck in validate pipeline of .coreci.yml (gates merges to main). gitleaks in pre-commit (opt-in).
• D-017: iter.Seq for orca job list --watch and orca node list --watch (pull-based, ctx cancellation, ctrl-c via signal.NotifyContext).
• D-018: Bin-packing by CPU/memory with FIFO within node; JSON-over-HTTP orca.v1.Dispatch for cross-node (no ConnectRPC dep).

v0.3 Clarified Decisions (D-series, full autonomy)

v0.3 is a lean 2-execution-phase milestone completing the streaming and doctor work deferred from v0.2. The 6 v0.3 decisions (D-019..D-024) were auto-resolved under full autonomy:

ID	Question	Decision	Rationale	Confidence
D-019	Watch refresh mechanism?	Poll-based, 1s ticker	Simpler than event channel; no daemon coupling for CLI; matches offline-first.	0.90
D-020	Watch output format (REQ-030)?	Table by default; --watch --json streams one-line JSON per event	Consistent with D-005 --json convention; serves humans + AI agents.	0.92
D-021	Doctor network check scope?	Probe configured peer addresses via mTLS /healthz handshake; PASS/WARN/FAIL per peer	Reuses existing transport client; read-only.	0.85
D-022	Doctor db check scope?	PRAGMA integrity_check + migration version query	Already specced in ARCHITECTURE.md §5; minimal surface.	0.95
D-023	iter.Seq cancellation?	signal.NotifyContext on SIGINT/SIGTERM	Per D-017 + ARCHITECTURE Flow 4.	0.95
D-024	--watch applies to job list only, or node list too?	Both orca job list --watch and orca node list --watch	Per ARCHITECTURE.md CLI layer + D-017.	0.92

v0.3 Scope Summary

v0.3 is a focused 2-execution-phase milestone completing the work deferred from v0.2 that was NOT already shipped in P08-P10. A codebase audit during re-init SPECIFY confirmed that REQ-014, REQ-027, REQ-028, REQ-029, REQ-031, REQ-037, REQ-039, REQ-040 all shipped in P08-P10 despite stale REQUIREMENTS.md marking them Pending. The remaining work:

• P01 — iter.Seq streaming for --watch flags. Go 1.25+ range-over-func semantics, pull-based iter.Seq[Job] / iter.Seq[Node], context.Context cancellation, signal.NotifyContext on ctrl-c. Applies to both orca job list --watch and orca node list --watch. Covers REQ-022, REQ-030.
• P02 — orca doctor network + db full implementation. Replaces the P01 stubs (NetworkStub, DBStub) with real checks: peer reachability via mTLS /healthz probe; SQLite PRAGMA integrity_check + migration version. Covers REQ-032 (completion).

The vision ("minimalist, offline-first, CLI-first orchestration engine") is unchanged. v0.3 is a completion milestone, not a direction change.

v0.5 Scope Summary — Distribution

v0.5 is a 3-execution-phase milestone that makes Orca installable, distributable, and containerized. The engine functionality from v0.1–v0.3 is unchanged; this milestone is purely about delivery surface:

• P01 — Namespace unification. A single ORCA_HOME environment variable becomes the namespace root for all on-disk state (db, certs, init, daemon). A --system flag on the root command selects the system-level namespace root /root/.orca. Backward compatible: empty ORCA_HOME → ~/.orca. Covers REQ-041, REQ-042.
• P02 — install.sh + in-place update. A 1-liner installer pulls the release binary from the public Gitea release URL, installs at user level by default (~/.local/bin/orca) or system level (/usr/local/bin/orca) with --system. Re-running updates the binary in place while preserving config/db/certs in the namespace dir. Idempotent. Covers REQ-043, REQ-044. Also updates README quickstart (REQ-016 completion).
• P03 — Docker release. A multi-stage Dockerfile builds a distroless image; scripts/release.sh and .coreci.yml publish the image to the Gitea container registry per release. Covers REQ-046.
• P04 — Final review + ship + audit. Milestone release.

The vision ("minimalist, offline-first, CLI-first orchestration engine") is unchanged. v0.5 is a distribution milestone, not a direction change.

v0.5 Clarified Decisions (D-series, full autonomy)

The 5 v0.5 decisions (D-025..D-029) were auto-resolved under full autonomy during the CLARIFY stage:

ID	Question	Decision	Rationale	Confidence
D-025	System-level namespace path layout?	/root/.orca (mirror of user-level ~/.orca)	Consistent shape with user-level; just a different root. Matches the user's "starts at /root" wording. Single dir keeps it simple.	0.90
D-026	Namespace override mechanism at runtime?	Unify on ORCA_HOME as single namespace root for all components (db, certs, init, daemon). Add --system flag that sets root to /root/.orca.	ORCA_HOME already exists for certs; extend to all components. Backward compatible (empty → ~/.orca). One knob, not many.	0.92
D-027	Docker registry target?	Gitea built-in container registry (git.cloudinit.dev/coreci/orca)	Keeps everything in one forge; uses Gitea's native registry. Consistent with REQ-045 (public repo → public image pulls).	0.88
D-028	How to make releases publicly accessible (REQ-045)?	Flip repo visibility to public via tea repos edit coreci/orca --private=false during P0 ship	Simplest path to anonymous downloads; enables both install.sh pulls and docker pulls. Pre-existing .env leak already suppressed via gitleaks baseline + rotate-forward (commit 00127ce).	0.85
D-029	install.sh default version?	Latest release (query Gitea releases API), optional --version vX.Y.Z to pin	Matches typical 1-liner installer UX; users get newest by default, can pin for reproducibility.	0.92

v0.5 Operational prerequisite (P0 ship)

The Gitea repo coreci/orca is currently private (returns 404 unauthenticated). P0 ship flips visibility to public via tea repos edit coreci/orca --private=false so that install.sh can pull release binaries unauthenticated (REQ-045). This is an operational step performed during the P0 ship, verified by an unauth curl against the releases API.

v0.6 Scope Summary — Node Bootstrap & Proxmox

v0.6 is a 3-execution-phase milestone that turns orca init from a bare mkdir into a full single-node cluster bootstrap, and adds Proxmox 8 & 9 as a first-class remote node type joined over SSH with least-privilege role delegation. The engine functionality from v0.1–v0.5 is unchanged; this milestone is about bootstrap ergonomics and heterogeneous node support:

• P01 — orca init full bootstrap. A single orca init call now: (a) creates the namespace dir (~/.orca or /root/.orca with --system); (b) runs all DB migrations including the new 0006 (nodes.kind, nodes.os — backward-compatible nullable columns); (c) bootstraps the internal CA via security.CAInit if ca.crt is absent; (d) generates the server cert via security.GenerateCSR + ca.SignCSR if server.crt is absent; (e) auto-detects the local OS via /etc/os-release ID= field (ubuntu/debian/alpine); (f) registers a localhost node with kind=localhost, os=<detected>, addr=localhost:8443 if no localhost node exists yet. After orca init, orca doctor MUST pass with zero FAILs. Idempotent: re-running orca init is a no-op (or refresh) for already-provisioned artifacts. Covers REQ-047, REQ-048, REQ-049.
• P02 — Proxmox SSH join. orca node join --type proxmox --host <addr> --user root --password <pw> (password via flag or $ORCA_PROXMOX_PASSWORD, never persisted) bootstraps a remote Proxmox 8/9 host via golang.org/x/crypto/ssh (new direct dep). Steps: (1) SSH password-auth; (2) generate or load orca's SSH keypair (~/.orca/orca_ssh_key / .pub, 0600/0644); (3) deploy pubkey to remote ~orca/.ssh/authorized_keys; (4) create orca user (config-overridable name via --proxmox-user, default orca); (5) create PVE custom role OrcaOperator (config-overridable via --proxmox-role) with privileges VM.Audit, Datastore.AllocateSpace, SDN.Use; (6) assign role to orca user on /; (7) drop /etc/sudoers.d/orca allowlist (pct, qm, pvesh, apt-get, dpkg — no shell-escape commands); (8) record node row kind=proxmox, os=pve, audit log. Idempotent re-run. Covers REQ-050, REQ-051.
• P03 — doctor os + doctor proxmox. Extends orca doctor with two new checks: doctor os re-runs /etc/os-release detection and verifies it matches the stored localhost node row's os field (drift = WARN); doctor proxmox iterates kind=proxmox nodes and SSH-probes each with pveversion / pvecmd status (3s timeout per peer per D-038 pattern), reporting PASS/WARN/FAIL per node. All bootstrap + join actions emit structured audit-log entries. Covers REQ-052.
• P04 — Final review + ship + audit. Milestone release.

The vision ("minimalist, offline-first, CLI-first orchestration engine") is unchanged. v0.6 is a bootstrap-ergonomics + heterogeneous- nodes milestone, not a direction change.