From 289e5cf6e1c9afbd433b472e04ebf09aefdbbd47 Mon Sep 17 00:00:00 2001 From: Jon Chery Date: Wed, 5 Aug 2026 20:57:05 +0000 Subject: [PATCH] docs(P02): CLI reference + jobspec reference + ingress guide MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit P02 — operator-facing documentation (REQ-091, REQ-092, REQ-093; gate C-22). docs/cli.md (REQ-091): - Full CLI command/flag reference: every command/subcommand with synopsis, flag tables (name/type/default/description), one-line examples. - Global flags (--json, --system, --config, --no-deprecation-warnings). - Output modes (text/json/watch), env vars, exit codes. - Deprecated surface callout boxes (daemon, cert, node-join-mTLS, HCL jobspec) pointing to v0.11 removal. docs/jobspec.md (REQ-092): - Markdown frontmatter schema reference: all top-level keys, block reference (runtime/ports/env-secrets/volumes/restart/update/service/ health/lifecycle/constraints/affinity/tasks), kinds matrix (Job/Service/DaemonSet required vs allowed), CEL subset grammar, body byte-exact preservation (R-015), deprecated HCL callout. docs/ingress.md (REQ-093): - Traefik ingress reference: service->Traefik mapping (D-175), R-007 socket-vs-TCP-bind semantics, generated YAML shape (routers/services/ healthCheck), atomic reload (C-10), drain (weight:0), TLS (certResolver, trust domain, step-ca), worked-example pointer to examples/full-stack/, v0.11 forward limitations. All factual claims grounded in live codebase (gate C-22). Cross-links verified to resolve. ---ci--- project: orca phase: 2 milestone: v0.10 status: execute ---/ci--- --- .ciagent/CHECKPOINT.json | 10 +- docs/cli.md | 522 +++++++++++++++++++++++++++++++++++++++ docs/ingress.md | 209 ++++++++++++++++ docs/jobspec.md | 442 +++++++++++++++++++++++++++++++++ 4 files changed, 1178 insertions(+), 5 deletions(-) create mode 100644 docs/cli.md create mode 100644 docs/ingress.md create mode 100644 docs/jobspec.md diff --git a/.ciagent/CHECKPOINT.json b/.ciagent/CHECKPOINT.json index fdadc7a..6e4f041 100644 --- a/.ciagent/CHECKPOINT.json +++ b/.ciagent/CHECKPOINT.json @@ -1,20 +1,20 @@ { - "phase": 1, - "stage": "verify", + "phase": 2, + "stage": "execute", "milestone": "v0.10", "milestone_slug": "docs-cli-examples", "phase_role": "execution", "attempts": 0, - "updated_at": "2026-08-05T20:30:00Z", + "updated_at": "2026-08-05T20:45:00Z", "milestone_complete": false, "ship": { - "tag": "v0.9.0", + "tag": "v0.9.1", "merged_to_main": false, "milestone_branch_deleted": false, "all_phase_branches_deleted": true }, "requirements": { - "covered": [], + "covered": ["REQ-097", "REQ-098"], "partial": [] }, "gates": { diff --git a/docs/cli.md b/docs/cli.md new file mode 100644 index 0000000..e8cd261 --- /dev/null +++ b/docs/cli.md @@ -0,0 +1,522 @@ +# Orca CLI Reference + +This document is the complete reference for the `orca` command-line +interface. Every command, subcommand, and flag is documented here. + +> **Canonical path (v0.9)**: The v0.9 re-architecture introduced the +> SSH-push deployment model, markdown jobspec, multi-namespace layout, +> and CLI-side scheduler. Commands marked **deprecated** below are from +> the v0.8 daemon/mTLS model and will be removed in v0.11. Use the +> v0.9 canonical path for all new work. + +## Global flags + +These flags are available on every `orca` command. + +| Flag | Type | Default | Description | +|------|------|---------|-------------| +| `--json` | bool | `false` | Output in JSON format (machine-readable) | +| `--system` | bool | `false` | Use system-level namespace root (`/root/.orca`) instead of user-level (`~/.orca`). Errors if `ORCA_HOME` is already set to a conflicting value. | +| `--config` | string | `""` | Path to config file (overrides `~/.orca/config.hcl`). Supports `.hcl` (legacy) and `.md` (v0.9 canonical) formats. | +| `--no-deprecation-warnings` | bool | `false` | Suppress v0.9 deprecation warnings. Use during `orca upgrade` migrations. | + +### Output modes + +- **Text** (default): human-readable tables and messages. +- **JSON** (`--json`): structured JSON output for machine consumption + and AI agents. +- **Watch** (`--watch` on list commands): table refresh (text default) + or NDJSON streaming (`--json`), one line per event until Ctrl-C. + +### Environment variables + +| Variable | Description | +|----------|-------------| +| `ORCA_HOME` | Namespace root directory (default `~/.orca`). Overrides all on-disk paths. | +| `ORCA_DB` | Fine-grained database path override. | +| `ORCA_PROXMOX_PASSWORD` | SSH password for `orca node join --type proxmox` (never persisted). | +| `ORCA_LISTEN_ADDR` | Daemon listen address (deprecated). | +| `ORCA_CA_PATH` | CA certificate path override. | +| `ORCA_SERVER_CERT_PATH` | Server certificate path override. | +| `ORCA_SERVER_KEY_PATH` | Server key path override. | +| `ORCA_NODE_CPU` | Node CPU capacity override (millicores). | +| `ORCA_NODE_MEMORY_MB` | Node memory capacity override (MiB). | + +### Exit codes + +| Code | Meaning | +|------|---------| +| `0` | Success | +| `1` | Error (printed to stderr) | + +--- + +## `orca init` + +Initialize local orca state with full bootstrap. + +``` +orca init +``` + +Performs a 6-step idempotent bootstrap: + +1. Create the namespace directory (honors `$ORCA_HOME`; defaults to `~/.orca`) +2. Open and migrate the SQLite database (migrations 0001–0006) +3. Bootstrap the internal CA (`ca.crt` + `ca.key`) if not already present +4. Generate the server cert (`server.crt` + `server.key`) if not already present +5. Auto-detect the local OS via `/etc/os-release` +6. Register a localhost node (kind=localhost, os=\) + +Re-running `orca init` is safe — it refreshes `last_seen` and `os` on +the localhost node without regenerating certs or changing the node ID. + +**Flags**: none. + +**Example**: +```bash +orca init +orca --system init # system-level bootstrap at /root/.orca +``` + +--- + +## `orca job` + +Manage orca jobs — run, list, stop, and inspect. + +### `orca job run` + +Run a job from a spec file. + +``` +orca job run [flags] +``` + +Dispatches by file extension: +- `.md` → Markdown frontmatter parser (v0.9 canonical) +- `.yaml` / `.yml` → YAML frontmatter parser +- `.hcl` → Legacy HCL adapter (deprecated, see callout below) + +| Flag | Type | Default | Description | +|------|------|---------|-------------| +| `--target` | string | `""` | Pin job to a specific node ID (overrides bin-packing scheduler) | +| `--idempotency-key` | string | `""` | Idempotency key for cross-node dispatch dedupe | + +**Examples**: +```bash +orca job run web-app.md +orca job run api.yaml --target node-abc-123 +orca job run worker.md --idempotency-key deploy-2026-08-05 +``` + +> **Deprecated**: `orca job run ` (legacy HCL jobspec) still +> works via the adapter but emits a deprecation warning. Migrate `.hcl` +> specs to `.md` (see [docs/jobspec.md](jobspec.md)). Removed in v0.11. + +### `orca job list` + +List all jobs. + +``` +orca job list [flags] +``` + +| Flag | Type | Default | Description | +|------|------|---------|-------------| +| `--watch` | bool | `false` | Stream jobs until Ctrl-C (table refresh or `--json` per-event) | + +**Output columns**: `ID NAME STATUS EXIT` + +**Examples**: +```bash +orca job list +orca job list --watch # table refresh +orca job list --watch --json # NDJSON: {"event":"update","job":{...}} +``` + +### `orca job stop` + +Stop a running job (soft stop). + +``` +orca job stop [job-id] [flags] +``` + +| Flag | Type | Default | Description | +|------|------|---------|-------------| +| `--id` | string | `""` | Job ID (alternative to positional argument) | + +**Example**: +```bash +orca job stop abc-123-def +orca job stop --id abc-123-def +``` + +### `orca job logs` + +Show task output for a job. + +``` +orca job logs [job-id] [flags] +``` + +| Flag | Type | Default | Description | +|------|------|---------|-------------| +| `--id` | string | `""` | Job ID (alternative to positional argument) | + +**Example**: +```bash +orca job logs abc-123-def +``` + +--- + +## `orca node` + +Manage orca nodes — join, leave, or list nodes in the registry. + +### `orca node join` + +Join a node to the orca registry. + +``` +orca node join [flags] +``` + +Node types (via `--type`): +- `localhost` (default): register a local or Linux node +- `proxmox`: SSH-bootstrap a remote Proxmox VE 8/9 host (deploys orca + pubkey, creates orca user + PVE role + sudoers allowlist; requires + `--host` + `--password`) + +| Flag | Type | Default | Description | +|------|------|---------|-------------| +| `--name` | string | `""` | Node name (required for `--type localhost`) | +| `--addr` | string | `""` | Node address (default `localhost:8443`) | +| `--ca-fingerprint` | string | `""` | Pin CA cert SHA-256 (fails if on-disk CA doesn't match) | +| `--type` | string | `"localhost"` | Node type: `localhost` or `proxmox` | +| `--host` | string | `""` | Proxmox host address (IP/hostname; required for `--type proxmox`) | +| `--ssh-user` | string | `"root"` | SSH username for proxmox bootstrap | +| `--password` | string | `""` | SSH password for proxmox bootstrap (never persisted; prefer `$ORCA_PROXMOX_PASSWORD`) | +| `--ssh-port` | int | `22` | SSH port for proxmox bootstrap | +| `--proxmox-user` | string | `"orca"` | Linux system user to create on the proxmox host | +| `--proxmox-role` | string | `"OrcaOperator"` | PVE custom role to create | +| `--host-key-fingerprint` | string | `""` | SSH host key `SHA256:base64` fingerprint (pre-pin; supersedes TOFU for `--type proxmox`) | + +**Examples**: +```bash +# Localhost (deprecated mTLS path) +orca node join --name my-node + +# Proxmox (v0.9 canonical SSH-push path) +orca node join --type proxmox --host 192.168.1.100 --ssh-user root +ORCA_PROXMOX_PASSWORD=secret orca node join --type proxmox --host 192.168.1.100 + +# Proxmox with pre-pinned host key +orca node join --type proxmox --host 192.168.1.100 --host-key-fingerprint SHA256:abc123... +``` + +> **Deprecated**: `orca node join` without `--type proxmox` (the +> localhost mTLS join path) is deprecated in v0.9. The v0.9 canonical +> path is SSH-push (`--type proxmox`) or local execution (no join +> needed). Removed in v0.11. + +### `orca node leave` + +Remove a node from the orca registry. + +``` +orca node leave [node-id] [flags] +``` + +| Flag | Type | Default | Description | +|------|------|---------|-------------| +| `--id` | string | `""` | Node ID (alternative to positional argument) | + +### `orca node list` + +List all nodes in the orca registry. + +``` +orca node list [flags] +``` + +| Flag | Type | Default | Description | +|------|------|---------|-------------| +| `--watch` | bool | `false` | Stream nodes until Ctrl-C (table refresh or `--json` per-event) | + +**Output columns**: `ID NAME ADDRESS STATE` + +### `orca node key-reset` + +Reset the SSH known_hosts entry for a node. + +``` +orca node key-reset +``` + +Removes the pinned SSH host key for `` from the local +`known_hosts` file. The next connect re-pins the key via TOFU or +`--host-key-fingerprint`. Local only — does not touch the remote +host's `authorized_keys`. + +`` is the node name (for proxmox nodes, this is the host address). + +**Example**: +```bash +orca node key-reset 192.168.1.100 +``` + +### `orca node capacity` + +Manage node capacity declarations (bin-packing scheduler input). + +``` +orca node capacity +``` + +#### `orca node capacity show` + +Show capacity for a node (defaults to `self`). + +``` +orca node capacity show [node-id] [flags] +``` + +| Flag | Type | Default | Description | +|------|------|---------|-------------| +| `--node` | string | `""` | Node ID (defaults to `self`) | + +**Output**: `Node:`, `CPU:` (millicores), `Memory:` (MiB), `Disk:` (MiB), `Updated:` + +#### `orca node capacity set` + +Declare capacity for a node. + +``` +orca node capacity set [flags] +``` + +| Flag | Type | Default | Description | +|------|------|---------|-------------| +| `--cpu` | int64 | `0` | CPU capacity in millicores (1000 = 1 vCPU) | +| `--memory` | int64 | `0` | Memory capacity in MiB | +| `--disk` | int64 | `0` | Disk capacity in MiB | +| `--node` | string | `""` | Node ID (defaults to `self`) | + +**Example**: +```bash +orca node capacity set --cpu 4000 --memory 8192 --disk 100000 +orca node capacity set --cpu 2000 --memory 4096 --node web-1 +``` + +#### `orca node capacity list` + +List all node capacity declarations. + +``` +orca node capacity list +``` + +**Output columns**: `NODE CPU(mc) MEM(MiB) DISK(MiB) UPDATED` + +--- + +## `orca ns` + +Manage orca namespaces under `ORCA_HOME` (R-002). + +Each namespace is a directory with `ns.md`, `.env`, `.env.secrets`, +`db/`, `jobs/`, `alloc/`. The implicit root namespace `_defaults` +always exists; every namespace inherits from `_defaults` and cannot +opt out. + +### `orca ns list` + +List all namespaces under `ORCA_HOME`. + +``` +orca ns list +``` + +**Output columns**: `NAME DEFAULT PATH` (`_defaults` marked `*`) + +### `orca ns create` + +Create a namespace directory + `ns.md`. + +``` +orca ns create [flags] +``` + +| Flag | Type | Default | Description | +|------|------|---------|-------------| +| `--parent` | string | `""` | Parent namespace (default `_defaults`; implicit root always appended last) | +| `--inherits-env` | bool | `true` | Inherit env from parents | +| `--inherits-secrets` | bool | `true` | Inherit secrets from parents | + +**Example**: +```bash +orca ns create prod --parent _defaults +orca ns create staging --parent prod +``` + +### `orca ns delete` + +Remove an empty namespace directory. + +``` +orca ns delete +``` + +Refuses if `jobs/` or `alloc/` contain files. The implicit root +`_defaults` cannot be deleted. + +### `orca ns inspect` + +Print the effective inheritance chain, merged env, and constraints. + +``` +orca ns inspect +``` + +**Output**: `Namespace:`, `Chain:` (e.g., `prod -> _defaults`), `Env:` +(sorted keys), `Constraints:` (unioned CEL expressions). + +### `orca ns validate` + +Run cycle + missing-parent + schema checks on a namespace. + +``` +orca ns validate +``` + +Exits 0 if valid, 1 on error. Runs over ALL namespaces under +`ORCA_HOME` (parsing + resolving validates cycles and missing parents +across the set). + +--- + +## `orca doctor` + +Run self-checks on the orca installation. + +``` +orca doctor [subcommand] +``` + +Without a subcommand, runs all checks and prints a PASS/WARN/FAIL +report per check. + +### Subcommands + +| Command | Description | +|---------|-------------| +| `orca doctor cert` | CA, server cert, expiry, fingerprint checks | +| `orca doctor network` | Network reachability via mTLS `/healthz` probe | +| `orca doctor db` | Database integrity (`PRAGMA integrity_check` + migration version) | +| `orca doctor os` | OS detection self-check (verifies `/etc/os-release` matches stored node) | +| `orca doctor proxmox` | Proxmox node reachability via SSH `pveversion`/`pvecmd status` probe | + +**Example**: +```bash +orca doctor +orca doctor cert +orca doctor proxmox --json +``` + +--- + +## `orca audit` + +View orca audit log (security-first observability). + +### `orca audit list` + +List recent audit log entries. + +``` +orca audit list [flags] +``` + +| Flag | Type | Default | Description | +|------|------|---------|-------------| +| `--limit` | int | `50` | Max entries to show | + +**Output columns**: `TIMESTAMP ACTOR ACTION RESOURCE RESULT` + +--- + +## `orca version` + +Print version information. + +``` +orca version +``` + +**Output**: +``` +orca version v0.9.1 + git commit: abc1234 + build time: 2026-08-05T20:30:00Z +``` + +--- + +## `orca status` + +Show orca daemon status. + +``` +orca status +``` + +> **Deprecated**: The daemon model is deprecated in v0.9 (replaced by +> SSH-push, R-001). This command returns a stub status. Removed in +> v0.11. + +--- + +## Deprecated commands + +The following commands are from the v0.8 daemon/mTLS model and are +**deprecated in v0.9**. They still work during the dual-write window +but emit `slog.Warn` deprecation warnings. They will be **removed in +v0.11**. + +> **`orca daemon`** — Run the orca daemon (HTTP API + health checks). +> The v0.9 re-architecture replaces the daemon with SSH-push (R-001). +> The daemon is repurposed to `drain-and-stop` in v0.11-P05 and deleted +> in v0.11-P14. Flags: `--addr` (default `:8080`), `--pprof` (pprof +> endpoint, default disabled). + +> **`orca cert`** — Manage orca certificates (CA, server, rotation). +> The v0.9 re-architecture replaces the internal CA with step-ca +> (D-101). Subcommands: `ca-init`, `gen`, `show`, `renew`, +> `fingerprint`. Removed in v0.11. + +> **`orca node join` (mTLS path)** — The localhost mTLS join path +> (without `--type proxmox`) is deprecated. The v0.9 canonical path is +> SSH-push (`--type proxmox`) or local execution (no join needed). + +> **`orca job run `** — Legacy HCL jobspec. Migrate to `.md` +> (see [docs/jobspec.md](jobspec.md)). The HCL adapter preserves +> `orca job run old-spec.hcl` during the migration window. + +To suppress deprecation warnings during migration, use +`--no-deprecation-warnings`: +```bash +orca --no-deprecation-warnings daemon +``` + +--- + +## See also + +- [docs/jobspec.md](jobspec.md) — Markdown frontmatter jobspec reference +- [docs/ingress.md](ingress.md) — Traefik ingress configuration guide +- [docs/namespace.md](namespace.md) — Namespace and path layout +- [docs/install.md](install.md) — Installation guide +- [examples/full-stack/](../examples/full-stack/) — Full-stack example with ingress \ No newline at end of file diff --git a/docs/ingress.md b/docs/ingress.md new file mode 100644 index 0000000..850057a --- /dev/null +++ b/docs/ingress.md @@ -0,0 +1,209 @@ +# Orca Ingress & Traefik Guide + +This document explains how Orca configures ingress via Traefik dynamic +configuration. It covers the service→Traefik mapping, the R-007 +socket-vs-TCP-bind model, atomic reload, drain, TLS, and a worked +example. + +> **Canonical path (v0.9)**: Orca generates Traefik dynamic +> configuration files via the `TraefikEmitter`. The `kind: Service` +> workload implies a Traefik route. The emitter renders one YAML file +> per Service; Traefik watches the dynamic config directory and reloads +> atomically on change. + +## The model + +A `kind: Service` jobspec **implies** a Traefik route (D-175). `Job` +and `DaemonSet` do **not** carry a Traefik route by default — a +`service:` block on a `Job` is rejected by the validator. + +When `orca job run` submits a `kind: Service` workload, the +`TraefikEmitter` renders a Traefik dynamic config file at: + +``` +/etc/traefik/dynamic/orca-.yaml +``` + +This file contains: +- One **router** (`orca-`) with a `PathPrefix` rule and TLS config. +- One **service** (`orca-`) as a `loadBalancer` with one **server** + per port, pointing at the workload's Unix socket (or TCP port). +- A **healthCheck** stanza when the `health:` block is present. + +Traefik watches `/etc/traefik/dynamic/` via `fsnotify` and reloads +whenever a file changes. Orca writes config atomically (write-tmp + +rename) so Traefik sees a single `IN_MOVED_TO` event and never observes +a half-written file. + +## R-007: socket vs TCP bind + +Orca workloads bind to a **Unix socket** by default, not a TCP port. +This is the R-007 security model: loopback-only by default, no network +exposure. + +### Default: Unix socket + +When `service.bind` is empty (default), the workload binds a Unix +socket at: + +``` +/run/orca/alloc-/port-.sock +``` + +systemd creates `/run/orca/alloc-/` via +`RuntimeDirectory=orca/alloc-` (mode 0750, owned by +`orca:orca`). The Traefik backend server URL is: + +```yaml +servers: + - url: "unix:///run/orca/alloc-/port-.sock" +``` + +### TCP opt-in: `service.bind: 127.0.0.1` + +When `service.bind: 127.0.0.1` is set, the workload binds a TCP port +directly (loopback only). The emitter adds an `ExecStartPre` marker to +the systemd unit so the bind mode is visible: + +```ini +ExecStartPre=/bin/echo orca: bind 127.0.0.1 port (tcp, R-007 opt-in) +``` + +`service.bind` must be a valid IP address. Empty (socket default) or +`127.0.0.1` (TCP opt-in) are the documented values; any other valid IP +is accepted but the bind happens in the process, not the emitter. + +## Generated Traefik YAML + +For a Service named `web` with port `http`: + +```yaml +http: + routers: + orca-web: + rule: PathPrefix("/web") + service: orca-web + tls: + certResolver: orca + domains: + - main: "cluster.orca.local" + services: + orca-web: + loadBalancer: + servers: + - url: "unix:///run/orca/alloc-/port-http.sock" + healthCheck: + path: /healthz + interval: 5s + timeout: 1s +``` + +- One router per Service, named `orca-`. +- Router rule: `PathPrefix("/")`. +- TLS: `certResolver: orca`, trust domain `cluster.orca.local` + (placeholder; step-ca provisioner overrides in v0.11). +- One service per Service, named `orca-`. +- One server per port, URL is `unix://`. +- `healthCheck` stanza present when `health:` block is set (required + for Service). Path is `/healthz`; interval and timeout come from the + `health:` block. + +## Atomic reload (gate C-10) + +Orca writes Traefik config atomically to avoid Traefik observing a +half-written file: + +1. Write to `.tmp` via `WriteFileIdempotent` (write + fsync). +2. `mv -f .tmp ` (atomic POSIX rename). + +Traefik's `fsnotify` watcher sees a single `IN_MOVED_TO` event and +reloads. If the new config is malformed, Traefik logs an error and +**holds last-good config** — the cluster keeps serving traffic on the +previous config. + +## Drain + +`RenderDrain` produces the same Traefik YAML with `weight: 0` on every +server in the load balancer: + +```yaml +servers: + - url: "unix:///run/orca/alloc-/port-http.sock" + weight: 0 +``` + +Traefik stops sending traffic to the drained backend. The workload +keeps running; drain is reversible (re-submit the normal config to +restore traffic). + +## TLS + +- **certResolver**: `orca` (references the Traefik ACME/step-ca + certificate resolver configured in Traefik's static config). +- **Trust domain**: `cluster.orca.local` (placeholder in v0.9; step-ca + provisioner in v0.11 overrides with the real cluster trust domain). +- **SPIFFE SVIDs**: workload identity via SPIFFE SVIDs minted at submit + time via step-ca (v0.11-P01.5, gate C-08). The SVID is a URI SAN in + the workload's X.509 cert. + +## Health checks + +The `health:` block (required for `Service`) maps to the Traefik +`healthCheck` stanza: + +```yaml +health: + check_type: http + interval: 5s + timeout: 1s + unhealthy_threshold: 2 +``` + +→ + +```yaml +healthCheck: + path: /healthz + interval: 5s + timeout: 1s +``` + +Traefik polls each backend's `/healthz` at the configured interval. An +unhealthy backend is removed from the load balancer pool until it +passes the health check again. + +## Worked example + +See [examples/full-stack/](../examples/full-stack/) for a complete +multi-service stack with ingress configured: +- `web-app.md` — frontend Service (socket bind, PathPrefix route) +- `api.md` — backend API Service (TCP opt-in, `127.0.0.1` bind) +- `examples/full-stack/rendered/traefik-dynamic-web-app.yaml` — the + Traefik config Orca generates + +## v0.11 forward (limitations) + +The following are not yet implemented in v0.9 and will land in v0.11: + +- **`service.host` / `service.route_id`**: stored on the `ServiceBlock` + but not yet consumed by the `TraefikEmitter`. The router rule is + hardcoded `PathPrefix("/")`. Custom host-based routing lands in + v0.11. +- **Socket activation**: real socket-activation (socket unit files, fd + passing) lands in v0.11-P08. The current emitter renders the + `RuntimeDirectory` + socket path comments but does not create socket + units. +- **Transactional update execution**: the `update:` block's rolling/ + canary/blue-green plan is computed by the emitter but not yet + executed transactionally. Transactional execution lands in + v0.11-P10. +- **SPIFFE SVID minting**: workload identity via step-ca SVIDs lands in + v0.11-P01.5 (gate C-08). +- **Secrets in env**: `env: { KEY: { from: "secret:..." } }` resolution + to `EnvironmentFile=`/`LoadCredential=` lands in v0.11-P03. + +## See also + +- [docs/cli.md](cli.md) — CLI reference +- [docs/jobspec.md](jobspec.md) — Jobspec reference (`service:`, `health:`, `ports:` blocks) +- [examples/full-stack/](../examples/full-stack/) — Full-stack example with ingress \ No newline at end of file diff --git a/docs/jobspec.md b/docs/jobspec.md new file mode 100644 index 0000000..3b8b787 --- /dev/null +++ b/docs/jobspec.md @@ -0,0 +1,442 @@ +# Orca Jobspec Reference + +This document is the complete reference for the Orca jobspec format — +the Markdown-with-frontmatter specification that describes workloads. + +> **Canonical format (v0.9)**: Orca uses Markdown with YAML frontmatter +> as the canonical jobspec format (R-013/R-014). The legacy HCL format +> is supported via an adapter during the migration window but is +> deprecated (see [HCL jobspec](#deprecated-hcl-jobspec) below). + +## File formats + +The `orca job run` command dispatches by file extension: + +| Extension | Parser | Body | +|-----------|--------|------| +| `.md` | `ParseMarkdown` (canonical) | Verbatim after closing `---` (R-015 byte-exact) | +| `.yaml` / `.yml` | `parseYAMLFile` | Whole file as frontmatter; body empty | +| `.hcl` | `ParseHCL` (legacy adapter) | Empty (deprecated) | + +## Minimal example + +```yaml +--- +kind: Job +name: my-job +runtime: + one_of: process + command: /bin/echo hello +--- +# My Job + +This body is preserved byte-exact and carried to the target node. +``` + +## Top-level keys + +| Key | Type | Default | Required | Notes | +|-----|------|---------|----------|-------| +| `orca-spec-version` | string | `""` | no | Free-form version tag (e.g. `"1"`) | +| `kind` | enum | — | **yes** | One of `Job`, `Service`, `DaemonSet` | +| `name` | string | — | **yes** | Workload name (trimmed, non-empty) | +| `count` | int | `1` | no | Job: must be 1; Service: ≥1; DaemonSet: not allowed | +| `runtime` | block | nil | see kinds | Runtime block (or per-task runtimes in a task group) | +| `ports` | block list | nil | Service: **yes** | Array of port mappings | +| `env` | block map | nil | no | Environment variables | +| `secrets` | inline/block list | nil | no | Secret names (resolution in v0.11) | +| `volumes` | block list | nil | no | Volume mounts | +| `restart` | block | nil | Service/DaemonSet: **yes** | Restart policy | +| `update` | block | nil | Service: **yes** | Update strategy | +| `service` | block | nil | no | Traefik route definition (implied for Service; not allowed for Job/DaemonSet) | +| `health` | block | nil | Service: **yes** | Health check | +| `lifecycle` | block | nil | no | Pre-stop / post-start hooks | +| `constraints` | list | nil | no | CEL expressions (node selection) | +| `affinity` | block list | nil | no | Co-location / anti-affinity rules | +| `tasks` | block list | nil | no | Task group (multi-process alloc) | +| `timeout` | duration string | `""` | no | Job timeout | +| `schedule` | block | nil | DaemonSet: **yes** | Schedule mode | + +## Kinds + +### `Job` + +A one-shot batch task. Runs once and exits. + +- `count` must be 1 (or unset). Use `Service` for replicas. +- `service` block is **not allowed** (no Traefik route for Jobs). +- `restart` optional (defaults to `never` / `on-failure`). +- `timeout` optional. + +**Example**: +```yaml +--- +kind: Job +name: data-migration +runtime: + one_of: process + command: /usr/bin/python3 migrate.py +timeout: 300s +env: + DB_URL: postgres://localhost/mydb +--- +``` + +### `Service` + +A long-running, load-balanced workload with a Traefik route. + +- `count` ≥ 1 (number of replicas). +- `ports` required (at least one). +- `restart` required; `mode` one of `service`, `on-failure`, `never`. +- `update` required; `strategy` one of `rolling`, `canary`, `blue-green`. +- `runtime` required (or a task group with per-task runtimes). +- `health` required (Traefik routing requires health checks). +- `service` block optional (implied for Service; use for `bind` override). +- `service.bind` if present must be a valid IP (`127.0.0.1` = TCP opt-in; + default = Unix socket). + +**Example**: +```yaml +--- +kind: Service +name: web +count: 3 +runtime: + one_of: process + command: /usr/bin/httpd +ports: + - name: http + port: 8080 +restart: + mode: service + attempts: 5 + delay: 2s +update: + strategy: rolling + max_parallel: 1 +health: + check_type: http + interval: 5s + timeout: 1s + unhealthy_threshold: 2 +constraints: + - node.role == "web" +--- +``` + +### `DaemonSet` + +A workload that runs on every matching node. + +- `schedule` required; `mode` one of `every-node`, `matching`, `mandatory`. +- `ports` **not allowed** (no Traefik route by default). +- `count` **not allowed** (implicit = matching nodes). +- `restart` required. + +**Example**: +```yaml +--- +kind: DaemonSet +name: log-shipper +schedule: + mode: every-node +runtime: + one_of: process + command: /usr/bin/fluent-bit +restart: + mode: service +--- +``` + +## Block reference + +### `runtime` + +The runtime backend for the workload. + +| Field | Key | Type | Default | Notes | +|-------|-----|------|---------|-------| +| `one_of` | `one_of` | string | — | Runtime type (see below) | +| `image` | `image` | string | `""` | Container image (for `podman`) | +| `command` | `command` | string | — | ExecStart command | + +**Supported runtime types** (`one_of`): + +| Type | Description | Requires | +|------|-------------|----------| +| `process` | Direct process execution via systemd (default) | systemd on target | +| `wasm` / `wasmtime` | WASM via wasmtime CLI (apt-installed on peer, SSH exec) | wasmtime on target | +| `podman` | Container via podman | podman on target | +| `pve-vm` | Proxmox VM via `qm` | Proxmox node | +| `pve-ct` | Proxmox container via `pct` | Proxmox node | +| `proxmox` | Alias for Proxmox runtime | Proxmox node | + +An empty/missing `Runtime` or `OneOf` is runtime-agnostic (always fits +the runtime axis in the scheduler). + +### `ports` + +Array of port mappings. Required for `Service`. + +| Field | Key | Type | Default | Notes | +|-------|-----|------|---------|-------| +| `name` | `name` | string | — | Port name (used in socket path) | +| `port` | `port` | int | — | Container port | +| `host_port` | `host_port` | int | `0` | Host port | +| `protocol` | `protocol` | string | `""` | Protocol (e.g. `tcp`) | +| `host_ip` | `host_ip` | string | `""` | Host IP | + +**Example**: +```yaml +ports: + - name: http + port: 8080 + host_port: 80 + protocol: tcp + - name: https + port: 8443 + host_port: 443 +``` + +### `env` + +Environment variables. Scalar values or secret references. + +```yaml +env: + FOO: bar + BAZ: "qux" + SECRET_REF: + from: "secret:db-password" + INLINE: {from: "secret:token"} +``` + +> Secret resolution (`from: "secret:..."`) lands in v0.11-P03. The +> parser stores the reference; the emitter will emit +> `EnvironmentFile=`/`LoadCredential=` in v0.11. + +### `secrets` + +List of secret names. Inline array or block list. + +```yaml +secrets: ["db-password", "api-token"] +# or +secrets: + - db-password + - api-token +``` + +### `volumes` + +Array of volume mounts. + +| Field | Key | Type | Default | Notes | +|-------|-----|------|---------|-------| +| `name` | `name` | string | — | Volume name | +| `type` | `type` | string | — | Volume type (e.g. `host`) | +| `source` | `source` | string | — | Source path (or `replicate:,` for Syncthing) | +| `target` | `target` | string | — | Mount target | +| `read_only` | `read_only` | bool | `false` | Read-only mount (`true`/`yes`/`on`/`1`) | + +**Example**: +```yaml +volumes: + - name: data + type: host + source: /data + target: /data + read_only: true +``` + +### `restart` + +Restart policy. + +| Field | Key | Type | Default | Notes | +|-------|-----|------|---------|-------| +| `mode` | `mode` | enum | — | `never`, `on-failure`, `service` | +| `attempts` / `max_retries` | `attempts` or `max_retries` | int | `0` | Max retries (both keys accepted) | +| `delay` | `delay` | duration string | `""` | Retry delay (e.g. `2s`) | + +### `update` + +Update strategy. Required for `Service`. + +| Field | Key | Type | Default | Notes | +|-------|-----|------|---------|-------| +| `strategy` | `strategy` | enum | — | `rolling`, `canary`, `blue-green` | +| `max_surge` | `max_surge` | int | `0` | Max surge | +| `max_parallel` | `max_parallel` | int | `1` (clamped to `count`) | Max parallel updates | +| `min_healthy_time` | `min_healthy_time` | duration | `""` | Min time healthy before next batch | +| `healthy_deadline` | `healthy_deadline` | duration | `""` | Deadline for health | +| `canary` | `canary` | int or `"%"` | — | Canary size (int count or percentage) | +| `auto_promote` | `auto_promote` | bool | `false` | Auto-promote canary (`true`/`yes`/`on`/`1`) | + +**Strategies**: +- **rolling**: batches of `max_parallel`, each batch waits for healthy. +- **canary**: canary batch first, then `promote` (manual or `auto_promote`), then remaining in `max_parallel` batches. +- **blue-green**: all new allocs start in parallel, wait healthy, then `cutover`. + +> Transactional update execution lands in v0.11-P10. The current +> emitter computes the plan; execution is a v0.11 deliverable. + +### `service` + +Traefik route definition. Implied for `Service`; not allowed for +`Job`/`DaemonSet`. See [docs/ingress.md](ingress.md) for details. + +| Field | Key | Type | Default | Notes | +|-------|-----|------|---------|-------| +| `name` | `name` | string | — | Service name | +| `port` | `port` | int | — | Service port | +| `bind` | `bind` | string (IP) | `""` | Bind mode: empty = Unix socket (default); `127.0.0.1` = TCP opt-in (R-007) | +| `host` | `host` | string | `""` | Host (stored, not yet consumed by emitter) | +| `route_id` | `route_id` | string | `""` | Route ID (stored, not yet consumed by emitter) | + +### `health` + +Health check. Required for `Service`. + +| Field | Key | Type | Default | Notes | +|-------|-----|------|---------|-------| +| `check_type` | `check_type` | string | — | Check type (e.g. `http`) | +| `interval` | `interval` | duration string | — | Check interval (e.g. `5s`) | +| `timeout` | `timeout` | duration string | — | Check timeout | +| `unhealthy_threshold` | `unhealthy_threshold` | int | `0` | Failures before unhealthy | + +Maps to Traefik `healthCheck` stanza (`path: /healthz`). + +### `lifecycle` + +Lifecycle hooks. Maps to systemd `ExecStartPost` / `ExecStop`. + +| Field | Key | Type | Default | systemd mapping | +|-------|-----|------|---------|-----------------| +| `post_start` | `post_start` | string list | nil | `ExecStartPost=` (runs after main starts) | +| `pre_stop` | `pre_stop` | string list | nil | `ExecStop=` (runs before kill) | + +**Example**: +```yaml +lifecycle: + pre_stop: + - /bin/sh -c 'sleep 5' + - /usr/local/bin/drain.sh + post_start: + - /usr/local/bin/warm-cache.sh +``` + +### `constraints` + +CEL-subset expressions for node selection. Inline array or block list. + +```yaml +constraints: + - node.role == "web" + - region == "us" +# or inline +constraints: ['node.role == "web"', 'region == "us"'] +``` + +**CEL subset grammar** (hand-rolled, no CEL dependency): +- Node attributes: `node.hostname`, `node.kind`, `node.cpus`, + `node.memory`, `node.tags`, `node.runtimes` +- Bare identifiers: equivalent to `node.` +- Literals: string (`"..."`), int +- Comparisons: `==`, `!=`, `>=`, `<=`, `>`, `<` +- Membership: `in`, `not in` +- Boolean: `and`, `or`, `not`, parentheses +- Anything outside the subset returns an error (node skipped, not + silently mis-evaluated) + +### `affinity` + +Co-location / anti-affinity rules. + +```yaml +affinity: + - target: zone == "a" + weight: 80 + - target: web + weight: -50 # anti-affinity (negative weight) +``` + +- `target`: CEL expression or bare workload name (for name-based + co-location). +- `weight`: positive = co-locate, negative = anti-affinity. +- Affinity is a **hint** (not a gate); evaluation failures are ignored. + +### `tasks` (task group) + +Multi-process alloc (P06). When `tasks` is non-empty, the alloc runs +multiple processes, each as its own systemd unit, grouped under a +systemd target. + +```yaml +tasks: + - name: app + runtime: + one_of: process + command: /usr/bin/httpd -f + env: + LOG_LEVEL: debug + - name: sidecar + runtime: + one_of: wasm + command: /bin/wasm-runner sidecar.wasm +``` + +- A task with no `runtime:` inherits the top-level `spec.Runtime`. +- Each task can have its own `env:` overlay. +- `command` falls back: `task.Command` → `task.Runtime.Command` → + `spec.Runtime.Command`. +- Task names must be unique within the group. + +## Kinds matrix + +| Feature | Job | Service | DaemonSet | +|---------|-----|---------|-----------| +| `count` | must be 1 | ≥ 1 | not allowed | +| `ports` | optional | **required** | not allowed | +| `service` block | not allowed | optional (implied) | not allowed | +| `restart` | optional | **required** | **required** | +| `update` | optional | **required** | optional | +| `health` | optional | **required** | optional | +| `runtime` | optional | **required** (or task group) | optional | +| `schedule` | optional | optional | **required** | +| `tasks` | optional | optional | optional | +| Traefik route | no | yes (implied) | no (by default) | + +## Body semantics + +The body after the closing `---` is preserved **byte-exact** (R-015) — +including trailing newlines, CRLF, BOM in body, and `---` inside code +fences. The body is carried verbatim to the target node. It is not +interpreted as commands/scripts by the parser today. + +## Deprecated: HCL jobspec + +The legacy HCL jobspec format is supported via an adapter during the +migration window. It is deprecated in v0.9 and will be removed in +v0.11. + +```hcl +job "hello-orca" { +} + +task "greet" { + command = "/bin/echo" + args = ["hello", "from", "orca"] +} +``` + +The adapter converts this to a `*WorkloadSpec{Kind: "Job", Name: +"hello-orca", Count: 1, Runtime: {OneOf: "process", Command: +"/bin/echo"}}`. Use `.md` for all new jobspecs. + +## See also + +- [docs/cli.md](cli.md) — CLI reference (including `orca job run`) +- [docs/ingress.md](ingress.md) — Traefik ingress configuration +- [examples/full-stack/](../examples/full-stack/) — Full-stack example jobspecs \ No newline at end of file