coreci/orca
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 91 Wiki Activity
Files
f31bed2dc30b7e983ad366cea77eb078cd203ea0
orca/.ciagent/PROJECT.md
T
Jon Chery 08d321f57f docs(P00): research findings 
v0.2 RESEARCH stage. Synthesizes the 4-phase v0.2 scope (P01-P04) into
updated static docs. No code changes. Decisions are derived from
CLARIFY D-011..D-018 (already on main) and direct investigation of
go.mod, the codebase, and ecosystem docs (Go 1.25+ iter.Seq, govulncheck,
gosec, gitleaks, step-ca).

Key research conclusions logged here:

- ConnectRPC is NOT in go.mod (.ciagent/config.json lists it in
  frameworks but the dependency was never added). v0.2 falls back to
  stdlib net/http with h2c for the orca.v1.Dispatch service. Zero new
  direct deps. (ARCHITECTURE.md AD-014)

- Roll-our-own CA via crypto/x509 (not step-ca/cfssl/vault-pki) keeps
  the binary single, dependency-free, and aligned with offline-first
  (no external PKI network calls). (ARCHITECTURE.md AD-010)

- govulncheck default mode requires network access to vuln.go.dev. CI
  step must use -format json (always exits 0) + a wrapper that gates
  on findings via jq/cat, OR pre-mirror the database. Caller to decide
  in PLAN. Logged as REQ candidate for IDEATE.

- gosec exit codes: 0 clean, 1 unsuppressed finding. -no-fail always
  returns 0. Baseline JSON via -track-suppressions + exclude=. We
  adopt -no-fail on initial run, baseline suppressed findings, then
  tighten to fail-on-finding once baseline is empty.

- gitleaks default config covers most cases; we extend .gitleaks.toml
  with stopwords for our test data paths and CA cert PEM (which would
  otherwise trigger the generic-api-key rule).

- iter.Seq: yield func(V) bool, iter.Pull for pull-style, range over
  function types since Go 1.25. Cancellation flows through ctx
  (consumer-driven backpressure). Single-use vs multi-use semantics
  documented in Go spec; we use multi-use for repo.Watch() since
  callers can re-iterate.

- mTLS hot-swap via tls.Config.GetCertificate callback enables cert
  rotation without daemon restart. tls.Config is read on every
  handshake; reload picks up new server.crt/server.key.

ARCHITECTURE.md changes:
- Added Transport Layer (internal/transport) and Dispatcher
  (internal/engine/dispatcher.go) components.
- Added Security Manager (internal/security) component with full cert
  lifecycle API.
- Added certs table schema (migration 0004) and Cert Go struct.
- Extended Node with NodeCapacity (CPU/memory) for bin-packing.
- Added v0.2 Component Graph ASCII diagram.
- Added 4 named flows: cert issuance, mTLS handshake, job dispatch,
  iter.Seq streaming.
- Added 8 new AD-009..AD-016 decisions and AD-014 notes the
  ConnectRPC-not-in-go.mod reality.

PERSONAS.md changes:
- Added network-engineer (custom, NEW in v0.2) for transport/dispatcher.
- security-engineer marked phase_specific: [P01, P02] (off after P02).
- network-engineer marked phase_specific: [P02].
- cli-engineer marked phase_specific: [P04] (--watch is a CLI concern).
- data-engineer.territory extended to include
  internal/store/migrations/0004_certs.sql.
- security-engineer.territory extended to TLS-config portion of
  internal/transport.
- Frontmatter updated: active_personas, phase_specific, reason.

PROJECT.md changes:
- Moved "Multi-node scheduling" out of "Out of Scope" (it ships in P02).
- Added "External PKI / Let's Encrypt / cert transparency logs" to
  Out of Scope (per D-011).
- Added "gRPC framework dependency" to Out of Scope (per AD-014).
- Added v0.2 Scope Summary section (4 phases) with cross-refs to
  ARCHITECTURE.md flows.

REQ candidates surfaced for IDEATE stage (not added to REQUIREMENTS.md
in this commit — that's the IDEATE stage's job):
- REQ-cand-A: Bounded cert rotation history (retain last N=3 server
  certs per node for rollback; documented in ARCHITECTURE.md certs
  table as "retention" implication of the schema).
- REQ-cand-B: Trusted-CA fingerprint pinning (D-012 requires operator
  to pass --ca-fingerprint at join; the daemon should refuse to start
  if the on-disk CA's fingerprint doesn't match a config-pinned value,
  to protect against operator typos).
- REQ-cand-C: govulncheck offline mode (CI must not call vuln.go.dev
  by default; either pre-mirror the DB or set GOVULNCHECK_DB env to
  a local file).
- REQ-cand-D: HCL/YAML schema for NodeCapacity declaration (where
  does the operator declare a node's CPU/RAM? Current v0.1 Node model
  has no capacity field. P02 will add this — needs a config file
  surface, e.g. ~/.orca/node.hcl or flag on `orca node join`).
- REQ-cand-E: gitleaks baseline for pre-existing secrets in history
  (the v0.1 .env leak was rotated forward but git history still has
  a SHA-1 leak — gitleaks/git filter-repo remediation may need a
  baseline file to avoid the same class of false positive recurring).
- REQ-cand-F: --watch output format mode (iter.Seq stream is
  table-style by default; users may want --watch --json one-line-per-
  event for piping). P04 scope decision; log for IDEATE.

---ci---
project: orca
phase: 0
milestone: v0.2
status: research
---/ci---
2026-06-03 20:59:32 +00:00

6.4 KiB
Raw Blame History

Project: Orca

Vision

A minimalist, offline-first, CLI-first orchestration engine inspired by HashiCorp Nomad, prioritizing stability, security, and simplicity over feature richness.

Objective

Build a lightweight system to manage and execute workloads across a set of nodes, keeping complexity far below that of Kubernetes.

Requirements

  • CLI First: Primary interaction through a CLI tool.
  • Offline First: Functional without constant internet connectivity.
  • AI First: Designed to be easily discoverable and manageable by AI agents.
  • Stability & Security: Prioritize security fixes and bug fixes over new features.
  • Language: Written in Go 1.25+.
  • Simplicity: Minimalist implementation, avoiding the "K8s complexity trap".

Constraints

  • No web UI as a primary requirement.
  • Must not implement K8s-level complexity.
  • Feature development must move slowly to ensure stability.
  • Only CI system allowed: CoreCI (git.cloudinit.dev/coreci/coreci).
  • Gitea remote: git.cloudinit.dev/coreci/orca.

Clarified Decisions (D-series, full autonomy)

ID Question Decision Rationale Confidence
D-001 Single binary or multi-binary distribution? Single binary Simpler distribution; subcommands baked into one orca binary. Aligns with simplicity pillar. 0.95
D-002 Local state store technology? modernc/sqlite (pure Go, CGO-free) Cross-compile friendly, no CGO dependency, single file on disk, mature. 0.92
D-003 Inter-node communication? Embedded HTTP (net/http) over loopback, mTLS for cross-node No external RPC framework needed for v0.1. HTTP suffices. 0.85
D-004 Scheduling algorithm for v0.1? Single-node only (no scheduling) Multi-node scheduling is out of scope for v0.1. Tasks run on the node they're submitted to. 0.90
D-005 CLI output format? Human-readable by default, --json flag for machine consumption Serves both humans and AI agents. 0.95
D-006 Job/task definition format? HCL or YAML in .hcl/.yaml files Familiar to Nomad/HashiCorp users; simpler than JSON for humans. 0.88
D-007 Authentication? mTLS for v0.1, token-based deferred mTLS is the most secure default. Tokens can be added later if needed. 0.80
D-008 Container runtime? Direct process execution (no container runtime) for v0.1 Avoids the Docker/container dependency. Pure process management. 0.85
D-009 Configuration file location? ~/.orca/config.hcl and /etc/orca/orca.hcl Standard XDG-style paths. 0.90
D-010 Logging format? Structured JSON via log/slog Native Go 1.21+ slog, no external dependency. 0.95
D-011 v0.2 mTLS cert authority model? Internal CA with CSR join One node bootstraps a local CA; peers generate CSRs and submit them to the CA for signing. CA cert is the trust anchor. More secure than self-signed per-node (single trust root) without the operational complexity of an external PKI. 0.92
D-012 v0.2 CA bootstrap & cert distribution? Operator-mediated, fingerprint-verified Bootstrap node writes ~/.orca/ca.crt and ~/.orca/ca.key (mode 0600). Operator copies ca.crt to peers; peers verify by SHA-256 fingerprint at orca node join --ca-fingerprint <sha256>. No automated secret distribution. 0.85
D-013 v0.2 cert validity & rotation? Server certs 90 days, CA cert 10 years, rotate 30 days before expiry Server certs are short-lived (compromise window small); CA is long-lived (manual rotation is expensive). orca cert renew reissues server certs automatically. 0.90
D-014 v0.2 mTLS handshake timing? Eager — at orca node join time Fail fast on bad certs, misconfigurations, or CA mismatches. Lazy handshake would let stale configs run until first request, complicating debugging. 0.88
D-015 v0.2 minimum TLS version & cipher suites? TLS 1.3 only; AEAD cipher allowlist MinVersion=tls.VersionTLS13, CipherSuites limited to TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_AES_128_GCM_SHA256. No TLS 1.2 fallback. 0.92
D-016 v0.2 security-scanning placement? validate pipeline of .coreci.yml, gates merges to main gosec baseline JSON checked into repo; new findings fail the build. govulncheck ./... exit-on-known. Pre-commit hook with gitleaks is opt-in (developer machine). 0.85
D-017 v0.2 iter.Seq API surface? orca job list --watch and orca node list --watch Pull-based iter.Seq[Job] / iter.Seq[Node]; cancellation via context.Context; signal.NotifyContext on ctrl-c. Backpressure is implicit (consumer-driven). 0.90
D-018 v0.2 multi-node scheduling algorithm? Bin-packing by available CPU/memory, FIFO within a node Simple, deterministic, matches D-004 minimalism. Cross-node dispatch via ConnectRPC orca.v1.Dispatch service. Retry on transient failures with exponential backoff. 0.85

Out of Scope

  • Full-blown Kubernetes-compatible API.
  • Complex cloud-provider integrations.
  • GUI-based management consoles.
  • Container runtime integration.
  • Service mesh / sidecar injection.
  • Auto-scaling / horizontal pod autoscaler.
  • External PKI / Let's Encrypt / cert transparency logs.
  • gRPC framework dependency (ConnectRPC in config.json frameworks but not in go.mod; v0.2 uses stdlib net/http with h2c for orca.v1.Dispatch — see ARCHITECTURE.md AD-014).

v0.2 Scope Summary

v0.2 is a focused 4-phase milestone that turns Orca from a single-node process executor into a small cluster engine with strong transport security and richer I/O. The 4 phases are:

  • P01 — mTLS handshake + internal CA with CSR join. Internal CA, CSR join, eager handshake at node join, TLS 1.3 + AEAD allowlist. See ARCHITECTURE.md Flow 1 + Flow 2.
  • P02 — Multi-node scheduling & job dispatch. Best-fit bin-packing by CPU/memory, FIFO within a node, orca.v1.Dispatch over mTLS. See ARCHITECTURE.md Flow 3.
  • P03 — gosec + govulncheck + gitleaks in CI. gosec baseline JSON in repo, govulncheck ./... in validate pipeline, gitleaks in pre-commit (opt-in).
  • P04 — iter.Seq streaming for --watch flags. Go 1.25+ range-over-func semantics, context.Context cancellation, signal.NotifyContext on ctrl-c. See ARCHITECTURE.md Flow 4.

The vision ("minimalist, offline-first, CLI-first orchestration engine") is unchanged. v0.2 is a hardening + small-cluster extension, not a direction change.

Reference in New Issue View Git Blame Copy Permalink