Jon Chery
4c2e59cf3f
Root cause: orca job run <example>.md failed with fork/exec: no such
file or directory on every example. Two compounding problems:
1. workloadToTaskSpecs (internal/cli/job.go:340) passed the entire
runtime.command string (e.g. "/usr/bin/httpd -f /etc/orca/web-app/
httpd.conf") as a single binary path to exec.Command, which then
looked for a file literally named "/usr/bin/httpd -f ..." and
failed. The v0.9 markdown parser stores command: as a raw string;
the legacy HCL path had separate command+args fields. Fix: add
splitCommand helper that splits on strings.Fields into binary+args,
with /bin/true fallback for empty commands.
2. The example commands referenced binaries that don't exist on a bare
Linux machine (/usr/bin/httpd, postgres, api-server, fluent-bit).
Fix: rewrite the 5 example runtime.command values to use /bin/sleep
3600 (long-running services) or /bin/echo (one-shot job) so they
run out-of-the-box. Each file has a Production substitution note
showing the real binary to use in deployment.
Verified: orca job run examples/full-stack/worker.md now succeeds
(exit 0). All 4 services (web-app, api, log-shipper, postgres) start
correctly (task started, pid assigned). 12 new unit tests pass
(splitCommand: 7 cases, workloadToTaskSpecs: 5 cases). All 5 example
jobspecs still parse + validate (gate C-20). make lint clean.
---ci---
project: orca
phase: 6
milestone: v0.10
status: execute
decisions:
- id: D-195
decision: split command string via strings.Fields in workloadToTaskSpecs
rationale: exec.Command expects binary path + args as separate elements;
the v0.9 markdown parser stores command: as a single string with no
args field (unlike legacy HCL). strings.Fields is dep-free and handles
multiple spaces/tabs. Shell quoting (single/double quotes inside the
command) is not handled — examples avoid sh -c with quoted strings.
confidence: 0.95
alternatives: [shellquote.Split from mvdan/sh (adds dependency)]
lessons:
- The v0.9 markdown jobspec path needs the same command+args split that
the legacy HCL path had via separate command/args fields. The parser
stores command: as a raw string; the CLI must split it before passing
to exec.Command.
- Example jobspecs should use /bin/sleep and /bin/echo (binaries that
exist on every Linux machine) so they run out-of-the-box. Descriptive
production commands belong in a comment block, not in runtime.command.
---/ci---
191 lines
6.5 KiB
Markdown
191 lines
6.5 KiB
Markdown
# Full-Stack Example with Ingress
|
|
|
|
This directory contains a complete multi-service stack deployed with
|
|
Orca, including Traefik ingress configuration. Each file is a valid
|
|
Orca jobspec (`.md` frontmatter) that passes the v0.9 parser and schema
|
|
validators.
|
|
|
|
> **Runnable out-of-the-box**: The `runtime.command` in each example
|
|
> uses `/bin/sleep 3600` (for long-running services) or `/bin/echo`
|
|
> (for one-shot jobs) so that `orca job run <file>.md` succeeds on any
|
|
> Linux machine without installing any software. Each file has a
|
|
> **Production substitution** note showing the real binary to use in a
|
|
> deployment (e.g. `/usr/bin/httpd`,
|
|
> `/usr/lib/postgresql/16/bin/postgres`).
|
|
|
|
## Stack overview
|
|
|
|
| File | Kind | Runtime | Ingress | Description |
|
|
|------|------|---------|---------|-------------|
|
|
| `web-app.md` | Service | process | Unix socket (default) | Frontend HTTP server, 3 replicas, rolling update |
|
|
| `api.md` | Service | process | TCP `127.0.0.1:9090` (R-007 opt-in) | Backend API, 2 replicas, canary update |
|
|
| `worker.md` | Job | process | none | One-shot batch worker with lifecycle hooks |
|
|
| `log-shipper.md` | Service | process | Unix socket (metrics) | Log shipper on a dedicated node |
|
|
| `postgres.md` | Service | process | Unix socket | Database with volume replication, blue-green update |
|
|
|
|
## Rendered artifacts
|
|
|
|
The `rendered/` directory shows what Orca generates on the target nodes
|
|
when you submit these jobspecs:
|
|
|
|
| File | Description |
|
|
|------|-------------|
|
|
| `traefik-dynamic-web-app.yaml` | Traefik dynamic config for the web-app Service |
|
|
| `traefik-dynamic-api.yaml` | Traefik dynamic config for the api Service (TCP bind) |
|
|
| `systemd-web-app.service` | Systemd unit for the web-app alloc |
|
|
| `systemd-api.service` | Systemd unit for the api alloc (with TCP bind marker) |
|
|
| `systemd-log-shipper.service` | Systemd unit for the log-shipper alloc |
|
|
|
|
## Walkthrough
|
|
|
|
### Prerequisites
|
|
|
|
- Orca installed (`orca version` works)
|
|
- 2+ Linux nodes reachable over SSH (for multi-node scheduling)
|
|
- Traefik installed on the lead node (watches `/etc/traefik/dynamic/`)
|
|
|
|
### Step 1: Initialize the cluster
|
|
|
|
```bash
|
|
# On the operator laptop
|
|
orca init
|
|
```
|
|
|
|
This creates `~/.orca/` (or `/root/.orca` with `--system`), bootstraps
|
|
the CA, generates the server cert, auto-detects the OS, and registers
|
|
a localhost node.
|
|
|
|
### Step 2: Join remote nodes
|
|
|
|
```bash
|
|
# Join a Proxmox node (v0.9 canonical SSH-push path)
|
|
orca node join --type proxmox --host 192.168.1.100 --ssh-user root
|
|
|
|
# Join a second node
|
|
ORCA_PROXMOX_PASSWORD=secret orca node join --type proxmox --host 192.168.1.101
|
|
```
|
|
|
|
### Step 3: Declare node capacity
|
|
|
|
The CLI-side scheduler uses capacity declarations for bin-packing:
|
|
|
|
```bash
|
|
orca node capacity set --cpu 4000 --memory 8192 --disk 100000 --node 192.168.1.100
|
|
orca node capacity set --cpu 4000 --memory 8192 --disk 100000 --node 192.168.1.101
|
|
```
|
|
|
|
### Step 4: Create a namespace
|
|
|
|
```bash
|
|
orca ns create prod --parent _defaults
|
|
```
|
|
|
|
This creates `~/.orca/prod/` with `db/`, `jobs/`, `alloc/`, and `ns.md`.
|
|
|
|
### Step 5: Submit the stack
|
|
|
|
```bash
|
|
orca job run web-app.md
|
|
orca job run api.md
|
|
orca job run worker.md
|
|
orca job run log-shipper.md
|
|
orca job run postgres.md
|
|
```
|
|
|
|
Each `orca job run` parses the `.md` jobspec, validates it against the
|
|
schema, schedules it via the CLI-side bin-packing scheduler, and
|
|
generates the systemd + Traefik artifacts on the target node via
|
|
SSH-push.
|
|
|
|
### Step 6: Observe placements
|
|
|
|
```bash
|
|
orca job list --watch
|
|
|
|
# Output:
|
|
# ID NAME STATUS EXIT
|
|
# abc-123... web-app running 0
|
|
# def-456... api running 0
|
|
# ghi-789... worker complete 0
|
|
# jkl-012... log-shipper running 0
|
|
# mno-345... postgres running 0
|
|
```
|
|
|
|
### Step 7: Inspect rendered artifacts
|
|
|
|
After submission, the target nodes have:
|
|
|
|
```
|
|
/etc/systemd/system/orca-v1-web-app.service # systemd unit
|
|
/etc/systemd/system/orca-v1-api.service # systemd unit (TCP bind)
|
|
/etc/traefik/dynamic/orca-web-app.yaml # Traefik dynamic config
|
|
/etc/traefik/dynamic/orca-api.yaml # Traefik dynamic config
|
|
/run/orca/alloc-web-app-0/port-http.sock # Unix socket (R-007 default)
|
|
```
|
|
|
|
See the `rendered/` directory in this example for the exact file
|
|
contents.
|
|
|
|
### Step 8: Verify ingress
|
|
|
|
Traefik watches `/etc/traefik/dynamic/` and atomically reloads when a
|
|
file changes (write-tmp + rename, gate C-10). The web-app is reachable
|
|
at `https://<cluster-domain>/web-app` and the API at
|
|
`https://<cluster-domain>/api`.
|
|
|
|
Health checks (`/healthz` on each backend) ensure Traefik only routes
|
|
to healthy instances.
|
|
|
|
### Step 9: Drain and rollback
|
|
|
|
To drain a service (stop traffic, keep the workload running):
|
|
|
|
```bash
|
|
# Orca writes a Traefik config with weight:0 on every backend
|
|
# (RenderDrain). Traefik stops sending traffic.
|
|
```
|
|
|
|
To roll back, re-submit the normal jobspec — Orca writes the
|
|
non-drained Traefik config and Traefik resumes routing.
|
|
|
|
## Ingress model
|
|
|
|
See [docs/ingress.md](../../docs/ingress.md) for the full Traefik
|
|
ingress reference. Key points:
|
|
|
|
- `kind: Service` **implies** a Traefik route (D-175).
|
|
- Default bind is a **Unix socket** at
|
|
`/run/orca/alloc-<id>/port-<name>.sock` (R-007).
|
|
- `service.bind: 127.0.0.1` opts in to **TCP** (loopback only).
|
|
- One Traefik dynamic file per Service at
|
|
`/etc/traefik/dynamic/orca-<name>.yaml`.
|
|
- Atomic reload via write-tmp + rename (gate C-10).
|
|
- Drain sets `weight: 0` per backend.
|
|
|
|
## Validation
|
|
|
|
All jobspecs in this directory are validated by a Go test:
|
|
|
|
```bash
|
|
go test ./examples/full-stack/ -v -run TestExamplesValidate
|
|
```
|
|
|
|
This test parses each `.md` file with `jobspec.ParseFile` and validates
|
|
it against `schema.ValidatorFor(kind)` — ensuring every field used in
|
|
the examples exists in the current `WorkloadSpec` struct and passes the
|
|
per-kind validators (gate C-20).
|
|
|
|
## v0.11 forward
|
|
|
|
The following are not yet implemented in v0.9 and will land in v0.11:
|
|
|
|
- **DaemonSet `schedule:` block**: the parser does not yet populate the
|
|
`schedule:` frontmatter block (v0.9 parser gap). The `log-shipper`
|
|
example uses `kind: Service` with `count: 1` and a `node.role`
|
|
constraint as a workaround.
|
|
- **Secret resolution**: `env: { KEY: { from: "secret:..." } }` is
|
|
parsed but not resolved to `EnvironmentFile=`/`LoadCredential=` until
|
|
v0.11-P03.
|
|
- **Transactional update execution**: the `update:` block's plan is
|
|
computed but not executed transactionally until v0.11-P10.
|
|
- **Socket activation**: real socket unit files land in v0.11-P08. |