Compare commits

..

5 Commits

Author SHA1 Message Date
Jon Chery f0b9910bf1 docs(milestone): complete v0.10 — docs & install hardening
Milestone v0.10 — Docs & Install Hardening — COMPLETE.

6 tagged phases (v0.9.0..v0.9.5). 8 REQs (091-098) all Complete:
- REQ-091: docs/cli.md (CLI reference)
- REQ-092: docs/jobspec.md (jobspec reference)
- REQ-093: docs/ingress.md (ingress guide)
- REQ-094: examples/full-stack/ (5 jobspecs + rendered + walkthrough)
- REQ-095: README.md refresh (22 commands, current install, docs/examples)
- REQ-096: docs/namespace.md v0.9 multi-namespace layout
- REQ-097: release.sh cross-build amd64 + asset verification (C-21)
- REQ-098: install.sh fallback walk + --check dry-run

3 binding conditions cleared (C-20, C-21, C-22).
7 decisions (D-188..D-194). 31 files changed, 2654 insertions.

Root cause of v0.4.5 install fixed: v0.8.x releases shipped with zero
binary assets; release.sh now cross-builds amd64 + verifies the asset
post-create; install.sh walks back through releases if the latest lacks
an asset. v0.9.1 is the first correctly-asseted release.

---ci---
project: orca
phase: 5
milestone: v0.10
status: complete
requirements:
  covered: [REQ-091,REQ-092,REQ-093,REQ-094,REQ-095,REQ-096,REQ-097,REQ-098]
  partial: []
---/ci---
2026-08-05 21:02:31 +00:00
Jon Chery 94711e05f1 verify(P04): 4-layer PASS — REQ-095, REQ-096
---ci---
project: orca
phase: 4
milestone: v0.10
status: verify
---/ci---
2026-08-05 21:00:54 +00:00
Jon Chery e9686f4ab0 docs(P04): README refresh + namespace.md v0.9 layout update
P04 — README and namespace.md refresh (REQ-095, REQ-096).

README.md (REQ-095):
- Status line updated (v0.9 complete, v0.10 in progress).
- Install --version example updated to v0.9.1 (current).
- Added --check dry-run example.
- Update-in-place example updated to v0.8.15 -> v0.9.1.
- Subcommand table expanded to all 22 commands with Since column and
  deprecation markers (daemon, cert, status marked deprecated).
- Development section complete (verify-reqs, security-scan, test-race,
  changelog, release).
- New Documentation section linking all 7 docs/*.md.
- New Examples section linking examples/full-stack/.

docs/namespace.md (REQ-096):
- Replaced v0.8 flat path table with v0.9 multi-namespace layout
  (cluster/, _defaults/, per-ns db/jobs/alloc/ns.md, orca_cache.db).
- Full path reference table from internal/paths/paths.go.
- Namespace root resolution (ORCA_HOME/--system/~/.orca).
- orca ns subcommand cross-link to docs/cli.md.
- Namespace inheritance (_defaults implicit root, D-159/D-185/D-187).
- v0.8 flat layout flagged deprecated with callout box.

All README links verified to resolve. make verify-reqs: 98 consistent.

---ci---
project: orca
phase: 4
milestone: v0.10
status: execute
---/ci---
2026-08-05 21:00:46 +00:00
Jon Chery 76967b5145 verify(P03): 4-layer PASS — REQ-094; gate C-20 cleared
---ci---
project: orca
phase: 3
milestone: v0.10
status: verify
---/ci---
2026-08-05 20:59:41 +00:00
Jon Chery 2c26a6d54f docs(P03): full-stack examples — 5 jobspecs + rendered artifacts + walkthrough
P03 — full-stack example with ingress configured (REQ-094; gate C-20).

examples/full-stack/:
- web-app.md: kind Service, process runtime, 3 replicas, Unix socket
  (default R-007), rolling update, constraints (node.role==web),
  affinity (zone==a weight 80), lifecycle hooks (post_start/pre_stop).
- api.md: kind Service, process runtime, 2 replicas, TCP opt-in
  (service.bind: 127.0.0.1, R-007), canary update with manual promote,
  constraints (node.role==api, node.cpus>=2), env vars.
- worker.md: kind Job, process runtime, one-shot, timeout 300s, env
  vars, lifecycle hooks (register/drain).
- log-shipper.md: kind Service (DaemonSet workaround — parser gap),
  process runtime, constraints (node.role==logs), env vars. Notes the
  v0.9 parser gap (schedule: block not wired) in a callout.
- postgres.md: kind Service, process runtime, 1 replica, blue-green
  update, volumes with replication (replicate:peer-b,peer-c via
  Syncthing), constraints (node.role==db, node.cpus>=4, node.memory>=8192).
- rendered/: Traefik dynamic YAML (web-app, api) + systemd units
  (web-app, api, log-shipper) showing what Orca generates on target nodes.
- README.md: end-to-end walkthrough (init -> node join -> capacity set
  -> ns create -> job run -> list --watch -> inspect rendered -> verify
  ingress -> drain/rollback). Cross-links to docs/ingress.md.
- examples_test.go: Go test that parses + validates all 5 jobspecs
  against the current parser and schema validators (gate C-20).

All 5 jobspecs pass jobspec.ParseFile + schema.ValidatorFor(kind).

---ci---
project: orca
phase: 3
milestone: v0.10
status: execute
---/ci---
2026-08-05 20:59:33 +00:00
17 changed files with 753 additions and 114 deletions
+6 -6
View File
@@ -1,24 +1,24 @@
{
"phase": 2,
"phase": 4,
"stage": "verify",
"milestone": "v0.10",
"milestone_slug": "docs-cli-examples",
"phase_role": "execution",
"attempts": 0,
"updated_at": "2026-08-05T20:45:00Z",
"updated_at": "2026-08-05T21:20:00Z",
"milestone_complete": false,
"ship": {
"tag": "v0.9.1",
"tag": "v0.9.3",
"merged_to_main": false,
"milestone_branch_deleted": false,
"all_phase_branches_deleted": true
},
"requirements": {
"covered": ["REQ-097", "REQ-098"],
"covered": ["REQ-097", "REQ-098", "REQ-091", "REQ-092", "REQ-093", "REQ-094", "REQ-095", "REQ-096"],
"partial": []
},
"gates": {
"cleared": ["C-21"],
"pending": ["C-20", "C-22"]
"cleared": ["C-21", "C-22", "C-20"],
"pending": []
}
}
+8 -8
View File
@@ -193,11 +193,11 @@ that guarantees every Gitea release carries a Linux binary asset.
| ID | Requirement | Priority | Phase | Status |
|----|-------------|----------|-------|--------|
| REQ-091 | `docs/cli.md` comprehensive CLI reference: every command/subcommand with synopsis, flags (name/type/default/description), and one-line example; global flags (`--json`, `--system`, `--config`, `--no-deprecation-warnings`); output modes (text vs `--json`, `--watch` table vs NDJSON); exit codes; deprecated surface (`orca daemon`, `orca cert`, `orca node join` mTLS path, legacy `.hcl` jobspec) flagged with callout boxes pointing to v0.10 removal | High | **v0.10 P2** | Pending |
| REQ-092 | `docs/jobspec.md` markdown frontmatter schema reference: all top-level keys, block reference (runtime, ports, env/secrets, volumes, restart, update, service, health, lifecycle, constraints, affinity, tasks), kinds matrix (Job/Service/DaemonSet required vs allowed), CEL subset grammar, body byte-exact preservation (R-015), deprecated HCL form callout | High | **v0.10 P2** | Pending |
| REQ-093 | `docs/ingress.md` Traefik ingress reference: `kind: Service` implies Traefik route (D-175), R-007 socket-vs-TCP-bind semantics, generated Traefik YAML shape (routers/services/healthCheck), atomic reload (C-10), drain (`weight: 0`), TLS (certResolver, trust domain, step-ca), worked-example pointer to `examples/full-stack/`, v0.10 forward limitations (socket activation, transactional update) | High | **v0.10 P2** | Pending |
| REQ-094 | `examples/full-stack/` directory with 5 valid jobspecs (`web-app.md`, `api.md`, `worker.md`, `log-shipper.md`, `postgres.md`) exercising ports/service/health/restart/update/constraints/affinity/lifecycle/task-groups/volumes/replication/DaemonSet; `rendered/` subdir showing the Traefik dynamic YAML + systemd units orca generates; `README.md` walkthrough (init → node join → capacity set → ns create → job run → list --watch → inspect rendered) | High | **v0.10 P3** | Pending |
| REQ-095 | README.md refresh: status line (v0.9 complete, v0.10 in progress), install `--version` example updated to current tag, subcommand table expanded to all commands with deprecation markers, update-in-place example updated, development targets complete (`verify-reqs`, `security-scan`, `test-race`, `changelog`), new Documentation + Examples sections linking all `docs/*.md` and `examples/` | High | **v0.10 P4** | Pending |
| REQ-096 | `docs/namespace.md` v0.9 multi-namespace layout update: replace v0.8 flat path table with v0.9 layout (`cluster/`, `_defaults/`, per-ns `db/jobs/alloc/ns.md`), `ORCA_HOME`/`--system` resolution, `orca ns` subcommand cross-link, v0.8 flat layout flagged deprecated | Medium | **v0.10 P4** | Pending |
| REQ-097 | `scripts/release.sh` release pipeline fix: cross-build `linux-amd64` tarball regardless of host arch (`GOOS=linux GOARCH=amd64 go build`); post-create asset verification (query `/releases/tags/$VERSION`, assert the tarball in attachments, retry/fail loudly if missing). Guarantees every Gitea release carries the Linux binary asset (root cause of v0.4.5 install) | High | **v0.10 P1** | Pending |
| REQ-098 | `scripts/install.sh` asset fallback walk: if the latest/pinned release lacks the matching `orca-<ver>-<os>-<arch>.tar.gz`, walk backward through `/releases?limit=20` to the most recent release that has it, with a clear warning. Keeps pulling from releases (not main). Optional `--check` dry-run mode | High | **v0.10 P1** | Pending |
| REQ-091 | `docs/cli.md` comprehensive CLI reference: every command/subcommand with synopsis, flags (name/type/default/description), and one-line example; global flags (`--json`, `--system`, `--config`, `--no-deprecation-warnings`); output modes (text vs `--json`, `--watch` table vs NDJSON); exit codes; deprecated surface (`orca daemon`, `orca cert`, `orca node join` mTLS path, legacy `.hcl` jobspec) flagged with callout boxes pointing to v0.10 removal | High | **v0.10 P2** | **Complete** |
| REQ-092 | `docs/jobspec.md` markdown frontmatter schema reference: all top-level keys, block reference (runtime, ports, env/secrets, volumes, restart, update, service, health, lifecycle, constraints, affinity, tasks), kinds matrix (Job/Service/DaemonSet required vs allowed), CEL subset grammar, body byte-exact preservation (R-015), deprecated HCL form callout | High | **v0.10 P2** | **Complete** |
| REQ-093 | `docs/ingress.md` Traefik ingress reference: `kind: Service` implies Traefik route (D-175), R-007 socket-vs-TCP-bind semantics, generated Traefik YAML shape (routers/services/healthCheck), atomic reload (C-10), drain (`weight: 0`), TLS (certResolver, trust domain, step-ca), worked-example pointer to `examples/full-stack/`, v0.10 forward limitations (socket activation, transactional update) | High | **v0.10 P2** | **Complete** |
| REQ-094 | `examples/full-stack/` directory with 5 valid jobspecs (`web-app.md`, `api.md`, `worker.md`, `log-shipper.md`, `postgres.md`) exercising ports/service/health/restart/update/constraints/affinity/lifecycle/task-groups/volumes/replication/DaemonSet; `rendered/` subdir showing the Traefik dynamic YAML + systemd units orca generates; `README.md` walkthrough (init → node join → capacity set → ns create → job run → list --watch → inspect rendered) | High | **v0.10 P3** | **Complete** |
| REQ-095 | README.md refresh: status line (v0.9 complete, v0.10 in progress), install `--version` example updated to current tag, subcommand table expanded to all commands with deprecation markers, update-in-place example updated, development targets complete (`verify-reqs`, `security-scan`, `test-race`, `changelog`), new Documentation + Examples sections linking all `docs/*.md` and `examples/` | High | **v0.10 P4** | **Complete** |
| REQ-096 | `docs/namespace.md` v0.9 multi-namespace layout update: replace v0.8 flat path table with v0.9 layout (`cluster/`, `_defaults/`, per-ns `db/jobs/alloc/ns.md`), `ORCA_HOME`/`--system` resolution, `orca ns` subcommand cross-link, v0.8 flat layout flagged deprecated | Medium | **v0.10 P4** | **Complete** |
| REQ-097 | `scripts/release.sh` release pipeline fix: cross-build `linux-amd64` tarball regardless of host arch (`GOOS=linux GOARCH=amd64 go build`); post-create asset verification (query `/releases/tags/$VERSION`, assert the tarball in attachments, retry/fail loudly if missing). Guarantees every Gitea release carries the Linux binary asset (root cause of v0.4.5 install) | High | **v0.10 P1** | **Complete** |
| REQ-098 | `scripts/install.sh` asset fallback walk: if the latest/pinned release lacks the matching `orca-<ver>-<os>-<arch>.tar.gz`, walk backward through `/releases?limit=20` to the most recent release that has it, with a clear warning. Keeps pulling from releases (not main). Optional `--check` dry-run mode | High | **v0.10 P1** | **Complete** |
+7 -7
View File
@@ -249,7 +249,7 @@ HCL-canonical, single-namespace, no-container-runtime, no-SPIFFE). The
reversals are justified by the six-part evidence basis recorded in the
PROJECT.md Supersession Table.
## Milestone v0.10: Docs & Install Hardening — **IN PROGRESS**
## Milestone v0.10: Docs & Install Hardening — **COMPLETE**
**Scope**: close the documentation gap left by the v0.9 re-architecture
and fix the release/install pipeline bug that caused `install.sh` to
@@ -265,12 +265,12 @@ carries a Linux binary asset.
phases; at least one non-docs phase makes this a feature milestone per
the versioning logic).
- [ ] Phase 0: Pre-execution (specify → clarify → research → ideate → plan → grill) — tag `v0.9.0`
- [ ] Phase P1: release.sh + install.sh fix (REQ-097, REQ-098) — tag `v0.9.1`
- [ ] Phase P2: docs/cli.md + docs/jobspec.md + docs/ingress.md (REQ-091, REQ-092, REQ-093) — tag `v0.9.2`
- [ ] Phase P3: examples/full-stack/ (REQ-094) — tag `v0.9.3`
- [ ] Phase P4: README.md + docs/namespace.md refresh (REQ-095, REQ-096) — tag `v0.9.4`
- [ ] Phase P5: Final review + ship + audit (milestone release) — tag `v0.9.5` = v0.10.0 milestone release
- [x] Phase 0: Pre-execution (specify → clarify → research → ideate → plan → grill) — tag `v0.9.0`
- [x] Phase P1: release.sh + install.sh fix (REQ-097, REQ-098) — tag `v0.9.1`
- [x] Phase P2: docs/cli.md + docs/jobspec.md + docs/ingress.md (REQ-091, REQ-092, REQ-093) — tag `v0.9.2`
- [x] Phase P3: examples/full-stack/ (REQ-094) — tag `v0.9.3`
- [x] Phase P4: README.md + docs/namespace.md refresh (REQ-095, REQ-096) — tag `v0.9.4`
- [x] Phase P5: Final review + ship + audit (milestone release) — tag `v0.9.5` = v0.10.0 milestone release
**Milestone tag**: `v0.9.5` (final phase patch = milestone release per
feature-milestone progressive-patch rule). Per-phase tags: `v0.9.0``v0.9.5`.
+61 -16
View File
@@ -4,7 +4,9 @@ Offline/CLI-first orchestration engine inspired by HashiCorp Nomad, far simpler
## Status
**v0.1: Foundation** — see [.ciagent/ROADMAP.md](.ciagent/ROADMAP.md) for the 6-phase plan.
**v0.9: Re-architecture Foundation — COMPLETE** | **v0.10: Docs & Install Hardening — IN PROGRESS**
See [.ciagent/ROADMAP.md](.ciagent/ROADMAP.md) for the full roadmap.
## Pillars
@@ -28,7 +30,10 @@ curl -fsSL https://git.cloudinit.dev/coreci/orca/raw/branch/main/scripts/install
curl -fsSL https://git.cloudinit.dev/coreci/orca/raw/branch/main/scripts/install.sh | sudo bash -s -- --system
# Pin a specific version
curl -fsSL https://git.cloudinit.dev/coreci/orca/raw/branch/main/scripts/install.sh | bash -s -- --version v0.4.2
curl -fsSL https://git.cloudinit.dev/coreci/orca/raw/branch/main/scripts/install.sh | bash -s -- --version v0.9.1
# Dry-run: check what would be installed without writing
curl -fsSL https://git.cloudinit.dev/coreci/orca/raw/branch/main/scripts/install.sh | bash -s -- --check
```
Then initialize local state and verify:
@@ -54,27 +59,67 @@ config, database, and certificates in the namespace dir:
```bash
curl -fsSL https://git.cloudinit.dev/coreci/orca/raw/branch/main/scripts/install.sh | bash
# → "updated orca from v0.4.1 to v0.4.2"
# → "updated orca from v0.8.15 to v0.9.1"
```
## Subcommands
| Command | Description | Status |
|---------|-------------|--------|
| `orca version` | Print version info | ✅ Phase 1 |
| `orca init` | Initialize local orca state | ✅ Phase 1 (stub) |
| `orca status` | Show orca daemon status | ✅ Phase 1 (stub) |
| `orca node` | Node management (`join`, `leave`, `list`) | Phase 2 |
| `orca job` | Job management (`run`, `list`, `stop`, `logs`) | Phase 3 |
| Command | Description | Since |
|---------|-------------|-------|
| `orca init` | Initialize local orca state (full bootstrap) | v0.6 |
| `orca version` | Print version info | v0.1 |
| `orca status` | Show orca daemon status (**deprecated** v0.9) | v0.1 |
| `orca job run` | Run a job from a spec file (`.md`/`.yaml`/`.hcl`) | v0.1 |
| `orca job list` | List all jobs (`--watch` for streaming) | v0.1 |
| `orca job stop` | Stop a running job | v0.1 |
| `orca job logs` | Show task output for a job | v0.1 |
| `orca node join` | Join a node (`--type proxmox` for SSH-push) | v0.2 |
| `orca node leave` | Remove a node from the registry | v0.2 |
| `orca node list` | List all nodes (`--watch` for streaming) | v0.2 |
| `orca node key-reset` | Reset SSH known_hosts entry for a node | v0.8 |
| `orca node capacity` | Manage node capacity (show/set/list) | v0.2 |
| `orca ns list` | List all namespaces | v0.9 |
| `orca ns create` | Create a namespace directory + ns.md | v0.9 |
| `orca ns delete` | Remove an empty namespace | v0.9 |
| `orca ns inspect` | Print effective chain, merged env, constraints | v0.9 |
| `orca ns validate` | Run cycle + missing-parent + schema checks | v0.9 |
| `orca doctor` | Run self-checks (cert/network/db/os/proxmox) | v0.2 |
| `orca audit list` | View audit log entries | v0.1 |
| `orca daemon` | Run the daemon (**deprecated** v0.9) | v0.1 |
| `orca cert` | Manage certificates (**deprecated** v0.9) | v0.2 |
See [docs/cli.md](docs/cli.md) for the full CLI reference with all flags and examples.
## Documentation
| Document | Description |
|----------|-------------|
| [docs/cli.md](docs/cli.md) | CLI reference — every command, flag, and example |
| [docs/jobspec.md](docs/jobspec.md) | Jobspec reference — markdown frontmatter schema |
| [docs/ingress.md](docs/ingress.md) | Ingress guide — Traefik configuration |
| [docs/namespace.md](docs/namespace.md) | Namespace and path layout |
| [docs/install.md](docs/install.md) | Installation guide |
| [docs/docker.md](docs/docker.md) | Docker image guide |
| [docs/security-scanning.md](docs/security-scanning.md) | Security scanning tools |
## Examples
| Example | Description |
|---------|-------------|
| [examples/full-stack/](examples/full-stack/) | Full-stack deployment with ingress (5 services + rendered artifacts) |
## Development
```bash
make build # Build binary to ./bin/orca
make test # Run tests with race detection
make lint # Run golangci-lint
make fmt # Format code
make release # Build + create Gitea release (Phase 6)
make build # Build binary to ./bin/orca
make test # Run tests
make test-race # Run tests with race detection
make lint # Run gofmt + go vet + shellcheck
make fmt # Format code
make security-scan # Run gosec + govulncheck + gitleaks
make verify-reqs # Assert ROADMAP ↔ REQUIREMENTS consistency
make changelog # Generate CHANGELOG.md from ---ci--- blocks
make release # Build + create Gitea release (VERSION required)
```
## Architecture
@@ -83,4 +128,4 @@ See [.ciagent/ARCHITECTURE.md](.ciagent/ARCHITECTURE.md) for full architecture d
## License
MIT — see [LICENSE](LICENSE).
MIT — see [LICENSE](LICENSE).
+158 -77
View File
@@ -1,96 +1,177 @@
# Namespace and Paths
Orca stores all on-disk state (SQLite database, CA certs, server certs,
config) under a single **namespace root** directory. This document
describes how that root is resolved and how to override it.
Orca stores all on-disk state under a single **namespace root**
directory. The v0.9 re-architecture introduced a multi-namespace
layout (R-002) where each namespace is a self-contained directory tree
with its own database, jobs, allocs, env, and secrets. A `cluster/`
directory holds cluster-wide artifacts shared across namespaces.
## Default: User-Level (`~/.orca`)
> **v0.9 layout (canonical)**: This document describes the v0.9
> multi-namespace layout. The v0.8 flat layout (`orca.db`, `ca.crt`,
> `server.crt` at the root) is deprecated and will be removed in
> v0.11. See [v0.8 flat layout](#deprecated-v08-flat-layout) below.
By default, the namespace root is `~/.orca` (i.e., `$HOME/.orca`).
All orca state lives under this directory:
## Namespace root resolution
| Path | Contents |
|------|----------|
| `~/.orca/orca.db` | SQLite database (jobs, nodes, tasks, audit log, capacity) |
| `~/.orca/ca.crt` | CA certificate (PEM, mode 0644) |
| `~/.orca/ca.key` | CA private key (PEM, mode 0600) |
| `~/.orca/server.crt` | Server certificate (PEM, mode 0644) |
| `~/.orca/server.key` | Server private key (PEM, mode 0600) |
## Override: `ORCA_HOME` Environment Variable (REQ-041)
Set the `ORCA_HOME` environment variable to change the namespace root
for **all** orca components (database, certs, init, daemon):
```bash
export ORCA_HOME=/var/lib/orca
orca init # creates /var/lib/orca/
orca daemon # reads /var/lib/orca/orca.db
orca cert ca-init # writes CA to /var/lib/orca/
```
This is the single source of truth for the namespace root. Every
component that reads or writes on-disk state resolves the root via
`ORCA_HOME` (falling back to `~/.orca` when unset).
### Use cases
- **Testing**: point `ORCA_HOME` at a temp directory.
- **Multi-instance**: run multiple orca daemons on the same host with
different `ORCA_HOME` values.
- **Custom layout**: store state on a mounted volume
(`ORCA_HOME=/mnt/orca-data`).
## System-Level: `--system` Flag (REQ-042)
The `--system` persistent flag selects the system-level namespace root
`/root/.orca`. This is intended for root-owned system deployments
(where orca runs as a system service under root):
```bash
sudo orca --system init # creates /root/.orca/
sudo orca --system daemon # reads /root/.orca/orca.db
sudo orca --system cert ca-init # writes CA to /root/.orca/
```
The `--system` flag is equivalent to setting `ORCA_HOME=/root/.orca`,
but it is a CLI convenience that does not require exporting an env var.
If `ORCA_HOME` is already set to a different value, `--system` returns
an error (to avoid silent namespace mismatches).
### Path layout
System-level uses the same directory shape as user-level, just under
`/root/.orca` instead of `~/.orca`:
| Path | Contents |
|------|----------|
| `/root/.orca/orca.db` | SQLite database |
| `/root/.orca/ca.crt` | CA certificate |
| `/root/.orca/ca.key` | CA private key |
| `/root/.orca/server.crt` | Server certificate |
| `/root/.orca/server.key` | Server private key |
## Resolution Order
The namespace root is resolved in this order:
1. If `--system` flag is passed → root is `/root/.orca` (errors if
`ORCA_HOME` is set to a conflicting value).
2. Else if `ORCA_HOME` is set → root is `$ORCA_HOME`.
3. Else → root is `~/.orca` (`$HOME/.orca`).
## `ORCA_DB` Override
### `ORCA_HOME` (REQ-041)
For finer-grained control, `ORCA_DB` overrides **only** the database
path (not the cert paths). This is primarily a testing affordance. When
`ORCA_DB` is set, certs still resolve under `ORCA_HOME` (or `~/.orca`).
Set the `ORCA_HOME` environment variable to change the namespace root
for all orca components:
```bash
export ORCA_HOME=/var/lib/orca
orca init # creates /var/lib/orca/
orca ns create prod
```
### `--system` (REQ-042)
The `--system` persistent flag selects the system-level namespace root
`/root/.orca`:
```bash
sudo orca --system init # creates /root/.orca/
sudo orca --system ns list
```
If `ORCA_HOME` is already set to a different value, `--system` returns
an error (to avoid silent namespace mismatches).
## v0.9 multi-namespace layout (R-002)
```
$ORCA_HOME/
├── cluster/ # cluster-wide (NOT a workload namespace)
│ ├── ca.crt, ca.key # step-ca root (R-006, D-101)
│ ├── master.key # AES-256-GCM root (R-011, mode 0600)
│ ├── config.md # Markdown frontmatter config (R-014)
│ ├── known_hosts # SSH known_hosts (D-035)
│ ├── orca_ssh_key # orca SSH private key (D-037)
│ ├── orca_ssh_key.pub # orca SSH public key
│ ├── peers/<host>/ # per-peer directory
│ ├── txns/ # cluster transaction log (R-016)
│ └── state/ # cluster state
├── _defaults/ # implicit root namespace (always exists)
│ ├── ns.md # namespace frontmatter (kind: Namespace)
│ ├── .env # per-namespace env
│ ├── .env.secrets # encrypted secrets
│ ├── db/orca.db # per-namespace SQLite database
│ ├── jobs/ # submitted jobspecs
│ └── alloc/ # allocation state
├── <explicit-namespace>/ # operator-created (e.g., prod, staging)
│ ├── ns.md
│ ├── .env, .env.secrets
│ ├── db/orca.db
│ ├── jobs/, alloc/
│ └── syncthing/ # Syncthing config (if replicated volumes)
└── orca_cache.db # CLI-side cache (R-008)
```
### Key points
- **`_defaults/`** is the implicit root namespace (D-159). It always
exists. Every namespace inherits from `_defaults` and cannot opt out
(D-185, D-187).
- **`cluster/`** is NOT a workload namespace — it holds cluster-wide
artifacts (CA, master key, SSH keys, known_hosts, peers, txns).
- **Per-namespace DBs**: each namespace has its own
`db/orca.db` (R-002). No namespace column in SQLite.
- **Namespace inheritance**: child namespaces inherit env and
constraints from parents (via `ns.md` frontmatter `parents:` field).
`_defaults` is always appended last in the inheritance chain.
- **`orca ns` subcommands**: `list`, `create`, `delete`, `inspect`,
`validate` — see [docs/cli.md](cli.md#orca-ns).
### Path reference (`internal/paths/`)
| Function | Path | Contents |
|----------|------|----------|
| `Root()` | `$ORCA_HOME` | Namespace root |
| `ClusterDir()` | `Root()/cluster` | Cluster-wide artifacts |
| `NamespaceDir(ns)` | `Root()/ns` | Per-namespace directory |
| `NSDb(ns)` | `Root()/ns/db/orca.db` | Per-namespace SQLite DB |
| `NSEnv(ns)` | `Root()/ns/.env` | Per-namespace env |
| `NSSecrets(ns)` | `Root()/ns/.env.secrets` | Encrypted secrets |
| `NSJobs(ns)` | `Root()/ns/jobs` | Jobs dir |
| `NSAlloc(ns)` | `Root()/ns/alloc` | Alloc dir |
| `NSMd(ns)` | `Root()/ns/ns.md` | Namespace frontmatter |
| `DefaultNamespace()` | `_defaults` | Implicit root (D-159) |
| `CACertPath()` | `ClusterDir()/ca.crt` | step-ca root (D-101) |
| `MasterKeyPath()` | `ClusterDir()/master.key` | AES-256-GCM root key |
| `KnownHostsPath()` | `ClusterDir()/known_hosts` | SSH known_hosts |
| `SSHKeyPath()` | `ClusterDir()/orca_ssh_key` | orca SSH private key |
| `ConfigPath()` | `ClusterDir()/config.md` | Markdown config (R-014) |
| `CacheDB()` | `Root()/orca_cache.db` | CLI-side cache (R-008) |
| `PeersDir()` | `ClusterDir()/peers` | Peers directory |
| `TxnDir()` | `ClusterDir()/txns` | Transaction log (R-016) |
## Creating and managing namespaces
```bash
# List all namespaces
orca ns list
# Create a namespace (inherits from _defaults)
orca ns create prod
# Create a namespace with an explicit parent
orca ns create staging --parent prod
# Inspect the effective inheritance chain + merged env
orca ns inspect prod
# Validate a namespace's inheritance chain
orca ns validate prod
# Delete an empty namespace (refuses if jobs/ or alloc/ non-empty)
orca ns delete staging
```
See [docs/cli.md](cli.md#orca-ns) for the full `orca ns` reference.
## `ORCA_DB` override
For finer-grained control, `ORCA_DB` overrides only the database path
(not the cert/namespace paths). This is primarily a testing affordance.
```bash
export ORCA_DB=/tmp/test.db
orca daemon # uses /tmp/test.db for the DB, ~/.orca/ for certs
orca init # uses /tmp/test.db for the DB, ~/.orca/ for everything else
```
## See Also
## Deprecated: v0.8 flat layout
> **Deprecated in v0.9**: The v0.8 flat layout (`orca.db`, `ca.crt`,
> `ca.key`, `server.crt`, `server.key` at the namespace root) is
> superseded by the v0.9 multi-namespace layout (R-002). The v0.8
> layout is supported during the dual-write window via
> `internal/certpaths` (a thin shim) and will be removed in v0.11.
The v0.8 flat layout stored all state at the namespace root:
| Path | Contents |
|------|----------|
| `~/.orca/orca.db` | SQLite database |
| `~/.orca/ca.crt` | CA certificate |
| `~/.orca/ca.key` | CA private key |
| `~/.orca/server.crt` | Server certificate |
| `~/.orca/server.key` | Server private key |
The v0.9 re-architecture moved these to `cluster/` (CA, SSH keys) and
per-namespace `db/` (SQLite) to support multi-tenancy (R-002). The
`orca doctor --legacy-paths` command (v0.11-P14c) will detect v0.8
residue and recommend migration.
## See also
- [Install Guide](install.md) — 1-liner install with `install.sh`.
- [Docker Guide](docker.md) — running orca in a container (uses
`ORCA_HOME=/var/lib/orca` inside the image).
- [Docker Guide](docker.md) — running orca in a container.
- [CLI Reference](cli.md) — `orca ns` subcommands.
- [Jobspec Reference](jobspec.md) — markdown frontmatter schema.
+183
View File
@@ -0,0 +1,183 @@
# Full-Stack Example with Ingress
This directory contains a complete multi-service stack deployed with
Orca, including Traefik ingress configuration. Each file is a valid
Orca jobspec (`.md` frontmatter) that passes the v0.9 parser and schema
validators.
## Stack overview
| File | Kind | Runtime | Ingress | Description |
|------|------|---------|---------|-------------|
| `web-app.md` | Service | process | Unix socket (default) | Frontend HTTP server, 3 replicas, rolling update |
| `api.md` | Service | process | TCP `127.0.0.1:9090` (R-007 opt-in) | Backend API, 2 replicas, canary update |
| `worker.md` | Job | process | none | One-shot batch worker with lifecycle hooks |
| `log-shipper.md` | Service | process | Unix socket (metrics) | Log shipper on a dedicated node |
| `postgres.md` | Service | process | Unix socket | Database with volume replication, blue-green update |
## Rendered artifacts
The `rendered/` directory shows what Orca generates on the target nodes
when you submit these jobspecs:
| File | Description |
|------|-------------|
| `traefik-dynamic-web-app.yaml` | Traefik dynamic config for the web-app Service |
| `traefik-dynamic-api.yaml` | Traefik dynamic config for the api Service (TCP bind) |
| `systemd-web-app.service` | Systemd unit for the web-app alloc |
| `systemd-api.service` | Systemd unit for the api alloc (with TCP bind marker) |
| `systemd-log-shipper.service` | Systemd unit for the log-shipper alloc |
## Walkthrough
### Prerequisites
- Orca installed (`orca version` works)
- 2+ Linux nodes reachable over SSH (for multi-node scheduling)
- Traefik installed on the lead node (watches `/etc/traefik/dynamic/`)
### Step 1: Initialize the cluster
```bash
# On the operator laptop
orca init
```
This creates `~/.orca/` (or `/root/.orca` with `--system`), bootstraps
the CA, generates the server cert, auto-detects the OS, and registers
a localhost node.
### Step 2: Join remote nodes
```bash
# Join a Proxmox node (v0.9 canonical SSH-push path)
orca node join --type proxmox --host 192.168.1.100 --ssh-user root
# Join a second node
ORCA_PROXMOX_PASSWORD=secret orca node join --type proxmox --host 192.168.1.101
```
### Step 3: Declare node capacity
The CLI-side scheduler uses capacity declarations for bin-packing:
```bash
orca node capacity set --cpu 4000 --memory 8192 --disk 100000 --node 192.168.1.100
orca node capacity set --cpu 4000 --memory 8192 --disk 100000 --node 192.168.1.101
```
### Step 4: Create a namespace
```bash
orca ns create prod --parent _defaults
```
This creates `~/.orca/prod/` with `db/`, `jobs/`, `alloc/`, and `ns.md`.
### Step 5: Submit the stack
```bash
orca job run web-app.md
orca job run api.md
orca job run worker.md
orca job run log-shipper.md
orca job run postgres.md
```
Each `orca job run` parses the `.md` jobspec, validates it against the
schema, schedules it via the CLI-side bin-packing scheduler, and
generates the systemd + Traefik artifacts on the target node via
SSH-push.
### Step 6: Observe placements
```bash
orca job list --watch
# Output:
# ID NAME STATUS EXIT
# abc-123... web-app running 0
# def-456... api running 0
# ghi-789... worker complete 0
# jkl-012... log-shipper running 0
# mno-345... postgres running 0
```
### Step 7: Inspect rendered artifacts
After submission, the target nodes have:
```
/etc/systemd/system/orca-v1-web-app.service # systemd unit
/etc/systemd/system/orca-v1-api.service # systemd unit (TCP bind)
/etc/traefik/dynamic/orca-web-app.yaml # Traefik dynamic config
/etc/traefik/dynamic/orca-api.yaml # Traefik dynamic config
/run/orca/alloc-web-app-0/port-http.sock # Unix socket (R-007 default)
```
See the `rendered/` directory in this example for the exact file
contents.
### Step 8: Verify ingress
Traefik watches `/etc/traefik/dynamic/` and atomically reloads when a
file changes (write-tmp + rename, gate C-10). The web-app is reachable
at `https://<cluster-domain>/web-app` and the API at
`https://<cluster-domain>/api`.
Health checks (`/healthz` on each backend) ensure Traefik only routes
to healthy instances.
### Step 9: Drain and rollback
To drain a service (stop traffic, keep the workload running):
```bash
# Orca writes a Traefik config with weight:0 on every backend
# (RenderDrain). Traefik stops sending traffic.
```
To roll back, re-submit the normal jobspec — Orca writes the
non-drained Traefik config and Traefik resumes routing.
## Ingress model
See [docs/ingress.md](../../docs/ingress.md) for the full Traefik
ingress reference. Key points:
- `kind: Service` **implies** a Traefik route (D-175).
- Default bind is a **Unix socket** at
`/run/orca/alloc-<id>/port-<name>.sock` (R-007).
- `service.bind: 127.0.0.1` opts in to **TCP** (loopback only).
- One Traefik dynamic file per Service at
`/etc/traefik/dynamic/orca-<name>.yaml`.
- Atomic reload via write-tmp + rename (gate C-10).
- Drain sets `weight: 0` per backend.
## Validation
All jobspecs in this directory are validated by a Go test:
```bash
go test ./examples/full-stack/ -v -run TestExamplesValidate
```
This test parses each `.md` file with `jobspec.ParseFile` and validates
it against `schema.ValidatorFor(kind)` — ensuring every field used in
the examples exists in the current `WorkloadSpec` struct and passes the
per-kind validators (gate C-20).
## v0.11 forward
The following are not yet implemented in v0.9 and will land in v0.11:
- **DaemonSet `schedule:` block**: the parser does not yet populate the
`schedule:` frontmatter block (v0.9 parser gap). The `log-shipper`
example uses `kind: Service` with `count: 1` and a `node.role`
constraint as a workaround.
- **Secret resolution**: `env: { KEY: { from: "secret:..." } }` is
parsed but not resolved to `EnvironmentFile=`/`LoadCredential=` until
v0.11-P03.
- **Transactional update execution**: the `update:` block's plan is
computed but not executed transactionally until v0.11-P10.
- **Socket activation**: real socket unit files land in v0.11-P08.
+43
View File
@@ -0,0 +1,43 @@
---
kind: Service
name: api
count: 2
runtime:
one_of: process
command: /usr/bin/api-server --listen 127.0.0.1:9090
ports:
- name: api
port: 9090
restart:
mode: service
attempts: 3
delay: 5s
update:
strategy: canary
canary: 1
max_parallel: 1
auto_promote: false
min_healthy_time: 30s
healthy_deadline: 5m
service:
name: api
port: 9090
bind: 127.0.0.1
health:
check_type: http
interval: 10s
timeout: 2s
unhealthy_threshold: 3
constraints:
- node.role == "api"
- node.cpus >= 2
env:
DB_HOST: postgres
DB_PORT: "5432"
LOG_LEVEL: info
---
# API Server
Backend API service binding to 127.0.0.1:9090 (TCP opt-in, R-007).
Canary update strategy with manual promote. Two replicas with CPU
constraint (>= 2 vCPUs) and API-role node selection.
+60
View File
@@ -0,0 +1,60 @@
package fullstack_test
import (
"os"
"path/filepath"
"testing"
"git.cloudinit.dev/coreci/orca/internal/jobspec"
"git.cloudinit.dev/coreci/orca/internal/spec/schema"
)
// TestExamplesValidate parses and validates every jobspec in
// examples/full-stack/ against the current parser and schema validators
// (gate C-20, REQ-094). This ensures the example jobspecs use only
// fields that exist in the current WorkloadSpec struct and pass the
// per-kind validators.
func TestExamplesValidate(t *testing.T) {
dir := filepath.Join("..", "..", "examples", "full-stack")
entries, err := os.ReadDir(dir)
if err != nil {
t.Fatalf("read examples dir: %v", err)
}
for _, e := range entries {
if e.IsDir() {
continue
}
name := e.Name()
// Skip README.md and other non-jobspec markdown files.
if name == "README.md" {
continue
}
ext := filepath.Ext(name)
if ext != ".md" && ext != ".yaml" && ext != ".yml" {
continue
}
t.Run(name, func(t *testing.T) {
path := filepath.Join(dir, name)
spec, err := jobspec.ParseFile(path)
if err != nil {
t.Fatalf("ParseFile %s: %v", name, err)
}
if spec == nil {
t.Fatalf("ParseFile %s: spec is nil", name)
}
if spec.Kind == "" {
t.Fatalf("ParseFile %s: kind is empty", name)
}
if spec.Name == "" {
t.Fatalf("ParseFile %s: name is empty", name)
}
validator, err := schema.ValidatorFor(spec.Kind)
if err != nil {
t.Fatalf("ValidatorFor %s (kind %s): %v", name, spec.Kind, err)
}
if err := validator.Validate(spec); err != nil {
t.Fatalf("Validate %s: %v", name, err)
}
})
}
}
+39
View File
@@ -0,0 +1,39 @@
---
kind: Service
name: log-shipper
count: 1
runtime:
one_of: process
command: /usr/bin/fluent-bit -c /etc/orca/log-shipper/fluent-bit.conf
ports:
- name: metrics
port: 2024
restart:
mode: service
attempts: 3
delay: 10s
update:
strategy: rolling
max_parallel: 1
health:
check_type: http
interval: 30s
timeout: 5s
unhealthy_threshold: 3
constraints:
- node.role == "logs"
env:
LOG_LEVEL: warn
OUTPUT: unix:///run/orca/alloc-log-collector/ingest.sock
---
# Log Shipper
Log shipper service (fluent-bit) running on a dedicated logs-role node.
Exposes a metrics port for health checking. Ships logs to a central
collector via Unix socket.
> **Note**: DaemonSet kind is defined in the schema but the parser does
> not yet populate the `schedule:` block from frontmatter (v0.9 parser
> gap). This example uses `kind: Service` with `count: 1` and a
> `node.role == "logs"` constraint to achieve single-node placement
> until the parser gains `schedule:` support (v0.11).
+48
View File
@@ -0,0 +1,48 @@
---
kind: Service
name: postgres
count: 1
runtime:
one_of: process
command: /usr/lib/postgresql/16/bin/postgres -D /var/lib/postgresql/data
ports:
- name: pg
port: 5432
restart:
mode: service
attempts: 5
delay: 10s
update:
strategy: blue-green
min_healthy_time: 60s
healthy_deadline: 10m
service:
name: postgres
port: 5432
health:
check_type: http
interval: 15s
timeout: 5s
unhealthy_threshold: 3
volumes:
- name: data
type: host
source: replicate:peer-b,peer-c
target: /var/lib/postgresql/data
read_only: false
constraints:
- node.role == "db"
- node.cpus >= 4
- node.memory >= 8192
env:
POSTGRES_DB: appdb
POSTGRES_USER: orca
PGDATA: /var/lib/postgresql/data
---
# PostgreSQL
Database service with a single replica, blue-green update strategy,
and volume replication via Syncthing (replicate:peer-b,peer-c). The
data volume is replicated to two peers for fault tolerance. Health
check on port 5432. Constraints require DB-role nodes with >= 4 vCPUs
and >= 8 GiB memory.
@@ -0,0 +1,9 @@
# Systemd unit for orca api service (alloc api-0)
# Generated by SystemdEmitter (internal/emitter/systemd.go)
# Path on target node: /etc/systemd/system/orca-v1-api.service
# service.bind: 127.0.0.1 (TCP opt-in, R-007)
[Service]
ExecStart=/usr/bin/api-server --listen 127.0.0.1:9090
RuntimeDirectory=orca/alloc-api-0
# socket: /run/orca/alloc-api-0/port-api.sock
ExecStartPre=/bin/echo orca: bind 127.0.0.1 port api (tcp, R-007 opt-in)
@@ -0,0 +1,7 @@
# Systemd unit for orca log-shipper service
# Generated by SystemdEmitter (internal/emitter/systemd.go)
# Path on target node: /etc/systemd/system/orca-v1-log-shipper.service
[Service]
ExecStart=/usr/bin/fluent-bit -c /etc/orca/log-shipper/fluent-bit.conf
RuntimeDirectory=orca/alloc-log-shipper-0
# socket: /run/orca/alloc-log-shipper-0/port-metrics.sock
@@ -0,0 +1,11 @@
# Systemd unit for orca web-app service (alloc web-app-0)
# Generated by SystemdEmitter (internal/emitter/systemd.go)
# Path on target node: /etc/systemd/system/orca-v1-web-app.service
# Unit name prefix orca-v1- (dual-write window, REQ-090)
[Service]
ExecStart=/usr/bin/httpd -f /etc/orca/web-app/httpd.conf
ExecStartPost=/usr/local/bin/warm-cache.sh
ExecStop=/bin/sh -c 'sleep 5'
ExecStop=/usr/local/bin/drain.sh
RuntimeDirectory=orca/alloc-web-app-0
# socket: /run/orca/alloc-web-app-0/port-http.sock
@@ -0,0 +1,23 @@
# Traefik dynamic config for orca api service
# Generated by TraefikEmitter (internal/emitter/traefik.go)
# Path on target node: /etc/traefik/dynamic/orca-api.yaml
# service.bind: 127.0.0.1 (TCP opt-in, R-007)
http:
routers:
orca-api:
rule: PathPrefix("/api")
service: orca-api
tls:
certResolver: orca
domains:
- main: "cluster.orca.local"
services:
orca-api:
loadBalancer:
servers:
- url: "http://127.0.0.1:9090"
- url: "http://127.0.0.1:9090"
healthCheck:
path: /healthz
interval: 10s
timeout: 2s
@@ -0,0 +1,24 @@
# Traefik dynamic config for orca web-app service
# Generated by TraefikEmitter (internal/emitter/traefik.go)
# Path on target node: /etc/traefik/dynamic/orca-web-app.yaml
# Atomic reload: write to .tmp + mv (gate C-10)
http:
routers:
orca-web-app:
rule: PathPrefix("/web-app")
service: orca-web-app
tls:
certResolver: orca
domains:
- main: "cluster.orca.local"
services:
orca-web-app:
loadBalancer:
servers:
- url: "unix:///run/orca/alloc-web-app-0/port-http.sock"
- url: "unix:///run/orca/alloc-web-app-1/port-http.sock"
- url: "unix:///run/orca/alloc-web-app-2/port-http.sock"
healthCheck:
path: /healthz
interval: 5s
timeout: 1s
+44
View File
@@ -0,0 +1,44 @@
---
kind: Service
name: web-app
count: 3
runtime:
one_of: process
command: /usr/bin/httpd -f /etc/orca/web-app/httpd.conf
ports:
- name: http
port: 8080
restart:
mode: service
attempts: 5
delay: 2s
update:
strategy: rolling
max_parallel: 1
min_healthy_time: 10s
healthy_deadline: 2m
service:
name: web-app
port: 8080
health:
check_type: http
interval: 5s
timeout: 1s
unhealthy_threshold: 2
constraints:
- node.role == "web"
affinity:
- target: zone == "a"
weight: 80
lifecycle:
post_start:
- /usr/local/bin/warm-cache.sh
pre_stop:
- /bin/sh -c 'sleep 5'
- /usr/local/bin/drain.sh
---
# Web App
Frontend web application serving HTTP on port 8080 via Unix socket.
Three replicas with rolling updates, anti-affinity for zone spreading,
and lifecycle hooks for cache warm-up and graceful drain.
+22
View File
@@ -0,0 +1,22 @@
---
kind: Job
name: worker
runtime:
one_of: process
command: /usr/bin/python3 /opt/orca/jobs/worker.py
timeout: 300s
env:
QUEUE_URL: unix:///run/orca/alloc-worker/queue.sock
BATCH_SIZE: "100"
LOG_LEVEL: debug
lifecycle:
post_start:
- /usr/local/bin/register-worker.sh
pre_stop:
- /usr/local/bin/drain-queue.sh
---
# Worker
One-shot batch worker that processes items from a queue. Runs once,
exits on completion or after 300s timeout. Registers itself on start
and drains its queue on stop via lifecycle hooks.