Compare commits

..

14 Commits

Author SHA1 Message Date
Jon Chery 61c97c847c fix(P1): recompute TARBALL after fallback version walk (REQ-132)
The fallback walk (REQ-098) reassigns VERSION from the requested
release to the nearest older release carrying a binary asset, but
never recomputed TARBALL (set once at line 106 from the requested
version). The stale tarball name then flowed into:

  - grep -F "$TARBALL" SHA256SUMS  -> matched nothing (the fallback
    release's SHA256SUMS only lists the fallback tarball)
  - sha256sum -c -                 -> empty stdin -> "no properly
    formatted checksum lines found" -> REQ-132 refusal
  - tar -xzf "${TMPDIR}/${TARBALL}" -> would look for the wrong
    filename (download saved under the stale name too)

User-visible symptom (v0.14.2 latest had no asset, fell back to
v0.12.18):

  install: verifying checksum...
  sha256sum: 'standard input': no properly formatted checksum lines found
  install: error: checksum verification failed (REQ-132); refusing to install

Fix: recompute TARBALL immediately after VERSION is reassigned in the
fallback branch, so download/grep/sha256sum/tar all reference the
fallback version's tarball. ASSET_URL and SHA256SUMS_URL were already
correct (derived from the API/ASSET_URL); TARBALL was the only stale
variable.

Reproduced the exact error before the fix; confirmed end-to-end
install succeeds after (orca-v0.12.18-linux-amd64.tar.gz: OK ->
extracting -> installed). Added a bats regression test pinning
--version v0.14.2 and asserting the dry-run "would install" line
references the fallback version (not the stale pinned one).

---ci---
project: orca
phase: 1
milestone: v0.15
status: execute
decisions:
  - id: D-001
    decision: Recompute TARBALL in the fallback branch immediately
      after VERSION is reassigned, so grep/sha256sum/tar use the
      fallback version's filename instead of the stale requested
      version's.
    rationale: Reproduced the exact user error ("no properly formatted
      checksum lines found") by running grep -F "$TARBALL" SHA256SUMS
      | sha256sum -c with a stale v0.14.2 tarball name against v0.12.18
      SHA256SUMS. TARBALL is the only stale variable: ASSET_URL and
      VERSION are correctly updated from API output, and SHA256SUMS_URL
      derives from ASSET_URL. Single-line fix, minimal blast radius,
      preserves the working non-fallback path.
    confidence: 0.96
    alternatives:
      - lazy TARBALL via a function (over-engineering for one stale
        assignment)
      - move TARBALL= assignment past the fallback block (breaks
        find_asset_url which needs the requested version's name
        pre-walk)
lessons:
  - When a fallback/walk mutates one variable (VERSION), audit every
    variable derived from it (TARBALL) for the same mutation. The
    user-facing info line at 167 constructed the name inline and
    looked correct, masking that the variable itself was stale.
---/ci---
2026-08-10 21:18:24 +00:00
Jon Chery 6f04b22df0 docs(milestone): complete v0.15 CI release pipeline fix
Release / ci (push) Failing after 4m57s
Release / container-orca (push) Has been skipped
Release / container-traefik (push) Has been skipped
All 3 requirements (REQ-180..182) complete. Gitea Actions workflow +
kaniko container publishing (no DinD). PAT_TOKEN secret configured.

---ci---
project: orca
phase: 2
milestone: v0.15
status: complete
requirements:
  covered: [180,181,182]
  partial: []
---/ci---
2026-08-10 21:01:56 +00:00
Jon Chery eadf2cc2c5 fix(P1): Gitea Actions workflow + kaniko container publishing (REQ-180,181)
Release / ci (push) Failing after 4m55s
Release / container-orca (push) Has been skipped
Release / container-traefik (push) Has been skipped
New .gitea/workflows/release.yml:
  - Triggers on push: tags: ['v*'] (deterministic)
  - Job 'ci': checkout + install Go + install coreci binary +
    coreci run (executes .coreci.yml: validate, build, test, release)
  - Job 'container-orca': kaniko executor:debug with --entrypoint
    /bin/sh, builds+pushes orca image (no DinD)
  - Job 'container-traefik': same, builds+pushes orca-traefik image
    (skips if Dockerfile.traefik absent at that tag)
  - Uses PAT_TOKEN secret (Gitea reserves GITEA_ prefix)

.coreci.yml:
  - Removed container-publish + container-publish-traefik steps
    (moved to Gitea Actions — CoreCI's podman executor appends sh -c
    which conflicts with kaniko's /kaniko/executor entrypoint)
  - Keeps validate/build/test/release (tarball + Gitea release)

scripts/trigger_coreci.sh:
  - Added tag ref handling (refs/tags/*) so pre-push hook triggers
    CoreCI for tag pushes too (Gitea Actions webhook is secondary)

---ci---
project: orca
phase: 1
milestone: v0.15
status: execute
---/ci---
2026-08-10 20:59:10 +00:00
Jon Chery 93ac4bda66 docs(P00): clarify+research+plan — v0.15 CI release pipeline
Key finding: CoreCI podman executor appends sh -c to step image,
which conflicts with kaniko's /kaniko/executor entrypoint. Container
publishing moves to Gitea Actions workflow (supports entrypoint
override). CoreCI keeps validate/build/test/release (tarball).

---ci---
project: orca
phase: 0
milestone: v0.15
status: plan
---/ci---
2026-08-10 20:57:55 +00:00
Jon Chery 454040fdd1 docs(init): validate specification — v0.15 CI release pipeline fix
3 requirements (REQ-180..182). REQ-182 complete (PAT_TOKEN secret
created via tea). Fix milestone — CI infrastructure, no DinD, kaniko.

---ci---
project: orca
phase: 0
milestone: v0.15
status: specify
---/ci---
2026-08-10 20:55:03 +00:00
Jon Chery c95bd73e42 docs(milestone): complete v0.14 ingress bootstrap
All 9 requirements (REQ-171..179) marked complete. 9 phases shipped
(v0.13.0..v0.13.8). R-024 adopted: traefik as podman container, three
ingress topologies (linux, proxmox native, proxmox floating-IP).

---ci---
project: orca
phase: 8
milestone: v0.14
status: complete
requirements:
  covered: [171,172,173,174,175,176,177,178,179]
  partial: []
---/ci---
2026-08-10 20:26:26 +00:00
Jon Chery ecdba833d9 feat(P7): doctor ingress + docs + integration tests (REQ-177,178,179)
New 'orca doctor ingress' command: verifies podman orca-traefik
container running, nft DNAT+SNAT, /etc/traefik/dynamic exists,
step-ca root CA present.

UAT signoff script: replaced assertion 36 (systemd → podman
container), added assertions 40-46 (nft table, DNAT, SNAT, dynamic
dir, step-ca CA, traefik.yml, doctor ingress pass).

docs/ingress.md: R-024 podman traefik section — three topologies,
container config, nft ruleset, doctor ingress, Dockerfile.traefik.
TLS model updated (drop certResolver, tls:{} for v0.14, mTLS v0.15).

ARCHITECTURE.md: v0.14 deltas section — R-024, three topologies,
nft emitter changes, TLS model, migration 0009, new CLI.

Integration tests (tests/ingress_bootstrap_test.go): nft postrouting
+ DNATTarget, priority -10, traefik TLS model (tls:{} no
certResolver), image ref resolution, floating-IP LXC provisioning
commands (pct create with hwaddr/ip/gw/features), MAC generation.

---ci---
project: orca
phase: 7
milestone: v0.14
status: execute
---/ci---
2026-08-10 20:24:13 +00:00
Jon Chery 6e65eadaa5 feat(P6): proxmox floating-IP LXC ingress + interactive prompt (REQ-176)
New internal/proxmox/ingress_lxc.go: ProvisionIngressLXC creates an
Ubuntu LXC named 'ingress' that owns the floating IP (net0
bridge=vmbr0,hwaddr=<mac>,ip=<floating-ip>/<prefix>,gw=<gateway>).
Unprivileged with --features nesting=1,keyctl=1,fuse=1 (research
Topic 3). Installs podman inside, runs orca-traefik container,
applies nft DNAT+SNAT INSIDE the LXC, pushes step-ca root CA.

GenerateRandomMAC: 02:XX:XX:XX:XX:XX for interactive mode (D-261).

Interactive prompting in joinProxmox: when --ingress-mode empty +
!--json, prompt for mode + floating IP + gateway + MAC (auto-
generate + confirm). Validate IP/MAC/gateway/prefix.

Floating-IP routing: calls ProvisionIngressLXC + registers:
  1. PVE host as 'proxmox' node (IngressMode=floating-ip)
  2. Ingress LXC as 'linux' node (name=ingress, addr=<floating-ip>:8443)
     so orca job run pushes traefik dynamic config to it.

---ci---
project: orca
phase: 6
milestone: v0.14
status: execute
---/ci---
2026-08-10 20:19:47 +00:00
Jon Chery 1b7aac71f6 feat(P5): proxmox native ingress mode — LXC + podman traefik (REQ-175)
Add --ingress-mode flag (native default, floating-ip) + --floating-ip,
--gateway, --mac, --net-prefix flags to 'orca node join'.

Native mode (default): provision an unprivileged LXC with
--features nesting=1,keyctl=1,fuse=1 (research Topic 3), install
podman inside it, run orca-traefik container. nft on PVE host DNATs
to the LXC bridge IP (DNATTarget parameterization, C-55: discover
LXC IP before first nft apply, no downtime window).

LXC provisioning: deterministic VMID 200, hostname orca-traefik,
--onboot 1, 2GB RAM. Idempotent (C-53: command -v podman check).
podman-restart.service enabled inside LXC (research Topic 6).

step-ca root CA pushed into LXC via pct exec heredoc.
traefik static config rendered + written into LXC.
nft ruleset rendered with DNATTarget=LXC-IP + applied on PVE host.

Migration 0009_ingress_mode.sql (C-59: NOT 0007 — already taken by
certs_serial_unique). ALTER TABLE nodes ADD COLUMN ingress_mode.
IngressMode field added to model.Node + set on proxmox node record.

---ci---
project: orca
phase: 5
milestone: v0.14
status: execute
---/ci---
2026-08-10 20:16:40 +00:00
Jon Chery ea42a17474 feat(P4): linux node join remote ingress bootstrap (REQ-174)
Add ingress.BootstrapRemoteIngress: renders+writes traefik static
config, renders+writes+applies nft DNAT/SNAT, pushes step-ca root CA,
ensures podman traefik container — all over SSH exec. Uses a heredoc-
based remoteWriteFile with a random delimiter (F9 injection guard).

Wired into linux/bootstrap.go Step 4d, replacing the standalone
EnsureTraefikContainerRemote call with the full ingress stack.

C-60: uses certpaths.CACertPath() (not CAPath).
C-58: mounts host-side traefik.yml (preserves REQ-100 opt-out).
C-55: pre-creates nft table before nft -f.

---ci---
project: orca
phase: 4
milestone: v0.14
status: execute
---/ci---
2026-08-10 20:11:32 +00:00
Jon Chery 5013209e31 feat(P3): nft SNAT+DNAT + orca init ingress bootstrap (REQ-173)
nft emitter (internal/emitter/nft.go):
- Add DNATTarget field (C-51: validated via net.ParseIP; injection
  guard). Default 127.0.0.1; proxmox native uses LXC bridge IP.
- Add EnableSNAT field (default true for zero-value config).
- Add postrouting masquerade chain (research Topic 1):
  ip saddr 127.0.0.0/8 oifname != lo masquerade
- Shift input/forward priority from filter (=0) to -10 (research
  Topic 2: pve-firewall coexistence — avoids same-priority undefined
  evaluation order).

internal/ingress/bootstrap.go (new):
- BootstrapLocalIngress: mkdir dirs, push step-ca root CA (C-60:
  certpaths.CACertPath not CAPath), render+write traefik static
  config (C-58: preserves traefik-on-public-ip opt-out), render+
  write+apply nft ruleset, pre-create table (C-55: avoids first-
  apply flush-table error), ensure podman container. All non-fatal.

init.go: Step 4d now calls ingress.BootstrapLocalIngress (R-024).
doctor_nft.go: assert postrouting masquerade + priority -10.

Tests: nft_test.go — DNATTarget substitution, invalid DNATTarget
rejection (C-51), EnableSNAT=false omits postrouting, priority -10.

---ci---
project: orca
phase: 3
milestone: v0.14
status: execute
---/ci---
2026-08-10 20:09:26 +00:00
Jon Chery dea472f443 feat(P2): podman traefik reconciler + TLS model fix (REQ-172)
Replace internal/traefik/install.go binary+systemd installer with a
podman-container reconciler (R-024). The reconciler is idempotent:
inspect → start-if-stopped → pull+run-if-absent.

Container run flags (research-validated):
  --restart=unless-stopped (not always; research Topic 6)
  --network host (binds 127.0.0.1:8080/8443 on host/LXC loopback)
  -v /etc/traefik/traefik.yml:ro (overrides baked default; C-58)
  -v /etc/traefik/dynamic:ro (orca writes atomically via SSH-push)
  -v /etc/orca/step-ca-root.crt:ro (future mTLS; v0.14 uses tls:{})
  No :Z SELinux flag (research Topic 7)

C-50: ensurePodmanLocal/Remote installs podman if absent.
C-57: removeLegacySystemdUnitLocal/Remote stops+disables+removes
  the v0.13 orca-traefik.service + /usr/local/bin/traefik before
  starting the podman container (upgrade path).
  upgrade.go cutover rewritten to use the reconciler.

TLS model fix (research Topic 4): drop certResolver: orca from
dynamic config (traefik v3.3 only supports acme/tailscale resolvers,
not CA-file-based). Emit tls: {} instead. Real mTLS via dynamic
tls.certificates + clientAuth.caFiles deferred to v0.15 (grill
G-003, confidence 0.55 < 0.60).

Callsites updated:
  init.go: installTraefikLocal → ensureTraefikContainerLocal
  linux/bootstrap.go: traefik.InstallRemote → EnsureTraefikContainerRemote
  proxmox/bootstrap.go: same
  traefik_install.go: wrapper updated

Tests: internal/traefik/install_test.go (new) — ImageRef, podmanRunArgs,
  container-running/stopped/absent paths, legacy systemd removal (C-57).

---ci---
project: orca
phase: 2
milestone: v0.14
status: execute
---/ci---
2026-08-10 20:04:20 +00:00
Jon Chery dccdb746ea feat(P1): orca-traefik container image + release pipeline (REQ-171)
Dockerfile.traefik: extends traefik:v3.3.0 with baked default
static config (entrypoints 127.0.0.1:8080/8443/8081, file provider
watching /etc/traefik/dynamic, json log). Host-side traefik.yml
mounted :ro at runtime to override baked default (preserves
traefik-on-public-ip opt-out, REQ-100, C-58).

No certificatesResolvers — traefik v3.3 only supports acme/tailscale
(research finding). tls: {} in dynamic config for v0.14; real mTLS
deferred to v0.15 (grill G-003, confidence 0.55 < 0.60).

release.sh: second docker block builds+pushes orca-traefik image.
.coreci.yml: container-publish-traefik step mirrors container-publish.

Verified: docker build -f Dockerfile.traefik . succeeds; image starts
traefik v3.3.0 with --configFile=/etc/traefik/traefik.yml.

---ci---
project: orca
phase: 1
milestone: v0.14
status: execute
---/ci---
2026-08-10 18:30:37 +00:00
Jon Chery 080919fde6 chore(P00): phase 0 complete — checkpoint update
---ci---
project: orca
phase: 0
milestone: v0.14
status: complete
---/ci---
2026-08-10 18:27:10 +00:00
36 changed files with 2150 additions and 213 deletions
+67
View File
@@ -872,3 +872,70 @@ scheduler.Schedule(spec, nodes) → emitter.Render(unit) → sshpush.Deploy(targ
- IPv6 `net.JoinHostPort` in all SSH dial paths. - IPv6 `net.JoinHostPort` in all SSH dial paths.
- Explicit timeouts on all SSH commands. - Explicit timeouts on all SSH commands.
- Root SIGINT/SIGTERM handler for clean exit on non-watch commands. - Root SIGINT/SIGTERM handler for clean exit on non-watch commands.
## v0.14 Deltas — Ingress Bootstrap Completeness (R-024)
### R-024: Traefik as Podman Container
Traefik runs exclusively as a podman container, deployed from the
custom `orca-traefik` image (published per release via `Dockerfile.traefik`
+ `scripts/release.sh` + `.coreci.yml container-publish-traefik`).
The v0.13 binary+systemd install (`internal/traefik/install.go`) is
replaced by an idempotent podman container reconciler
(`EnsureTraefikContainerLocal`/`Remote`). The container runs with
`--network host`, `--restart=unless-stopped`, and volume mounts for
`traefik.yml` (static config), `dynamic` (dynamic config), and
`step-ca-root.crt` (future mTLS). No SELinux `:Z` flag.
### Three Ingress Topologies
1. **Linux** (`orca init` / `orca node join --type linux`):
host → nft DNAT → podman traefik (host network).
`internal/ingress/bootstrap.go` → `BootstrapLocalIngress` /
`BootstrapRemoteIngress`.
2. **Proxmox Native** (`--ingress-mode native`, default):
PVE host → nft DNAT (target = LXC bridge IP) → LXC
(`--features nesting=1,keyctl=1,fuse=1`) → podman traefik.
`internal/proxmox/bootstrap.go` → `provisionNativeIngressLXC`.
3. **Proxmox Floating-IP** (`--ingress-mode floating-ip`):
LXC owns the floating IP (`net0 bridge=vmbr0,hwaddr=<mac>,
ip=<floating-ip>/<prefix>,gw=<gateway>`) → nft inside LXC →
podman traefik. The ingress LXC is registered as a `linux` node
(name=`ingress`) so `orca job run` pushes traefik dynamic config.
`internal/proxmox/ingress_lxc.go` → `ProvisionIngressLXC`.
### nft Emitter Changes
`internal/emitter/nft.go`:
- `DNATTarget` field (C-51: validated via `net.ParseIP`). Default
`127.0.0.1`; proxmox native uses LXC bridge IP.
- `EnableSNAT` field + postrouting masquerade chain: `ip saddr
127.0.0.0/8 oifname != "lo" masquerade` (research Topic 1).
- Input/forward chain priority shifted from `filter` (=0) to `-10`
(research Topic 2: pve-firewall coexistence — avoids same-priority
undefined evaluation order).
### TLS Model
v0.14 drops `certResolver: orca` from the dynamic config (traefik v3.3
only supports `acme`/`tailscale` resolvers, not CA-file-based). The
dynamic config emits `tls: {}` (traefik default cert). Real mTLS via
`tls.certificates` + `tls.options.default.clientAuth.caFiles` is
deferred to v0.15 (grill G-003, confidence 0.55 < 0.60).
### Migration 0009
`ALTER TABLE nodes ADD COLUMN ingress_mode TEXT NOT NULL DEFAULT '';`
Values: `""` (legacy), `"native"`, `"floating-ip"`. `IngressMode` field
on `model.Node`.
### New CLI
- `orca doctor ingress` — verifies podman container running, nft
DNAT+SNAT, dynamic dir, step-ca root CA.
- `--ingress-mode` flag on `orca node join --type proxmox`.
- `--floating-ip`, `--gateway`, `--mac`, `--net-prefix` flags for
floating-IP mode.
+11 -11
View File
@@ -1,17 +1,17 @@
{ {
"phase": 0, "phase": 2,
"stage": "grill", "stage": "complete",
"milestone": "v0.14", "milestone": "v0.15",
"milestone_slug": "ingress-bootstrap", "milestone_slug": "ci-release-pipeline",
"phase_role": "pre_execution", "phase_role": "final",
"attempts": 0, "attempts": 0,
"updated_at": "2026-08-10T18:00:00Z", "updated_at": "2026-08-10T21:05:00Z",
"milestone_complete": false, "milestone_complete": true,
"previous_milestone": "v0.13", "previous_milestone": "v0.14",
"phases_shipped": [], "phases_shipped": ["P0","P1","P2"],
"tags_shipped": [], "tags_shipped": ["v0.14.0","v0.14.1"],
"requirements": { "requirements": {
"covered": [], "covered": [180,181,182],
"partial": [] "partial": []
} }
} }
+51
View File
@@ -0,0 +1,51 @@
# CLARIFY + RESEARCH + PLAN v0.15: CI Release Pipeline Fix
## Decisions
| ID | Decision | Rationale | Confidence |
|----|----------|-----------|------------|
| D-264 | Secret name = `PAT_TOKEN` (not `GITEA_PAT`) | Gitea reserves `GITEA_` prefix for built-in secrets | 1.0 (validated) |
| D-265 | Use `tea actions secrets create` CLI | Operator instruction: no API | 1.0 (validated) |
| D-266 | Container publishing in Gitea Actions, not CoreCI | CoreCI's podman executor appends `sh -c` which conflicts with kaniko's `/kaniko/executor` entrypoint. Gitea Actions `container:` supports `options: --entrypoint` | 0.95 |
| D-267 | kaniko `executor:debug` image | Includes `/bin/sh`; Gitea Actions can override entrypoint to `/bin/sh` then run kaniko via shell | 0.90 |
| D-268 | `coreci run` for validate/build/test/release (tarball); Gitea Actions for container publishing | Clean separation: CoreCI owns the pipeline, Gitea Actions owns the trigger + container publish | 0.95 |
## Research: CoreCI podman executor entrypoint issue
CoreCI's `internal/runner/podman_executor.go:48-51`:
```go
args = append(args, image) // e.g. gcr.io/kaniko-project/executor:debug
if job.Invoke != "" {
args = append(args, "sh", "-c", job.Invoke)
}
```
This produces: `podman run ... <image> sh -c "<commands>"`
With kaniko:debug (entrypoint `/kaniko/executor`), the actual command is:
`/kaniko/executor sh -c "<commands>"` — kaniko fails (sh is not a kaniko flag).
**Conclusion**: kaniko cannot be used as a CoreCI step image. Container
publishing must move to the Gitea Actions workflow, which supports
`container: options: --entrypoint /bin/sh` to override the entrypoint.
## Plan
### Phase 1 (only execution phase)
**Files to create/modify:**
1. `.gitea/workflows/release.yml` — Gitea Actions workflow:
- `on: push: tags: ['v*']`
- Job 1 `ci`: checkout + install Go + install coreci + `coreci run`
(executes validate/build/test/release from .coreci.yml)
- Job 2 `container-orca`: checkout + kaniko build+push orca image
(needs job 1; uses `container: gcr.io/kaniko-project/executor:debug`
with `options: --entrypoint /bin/sh`)
- Job 3 `container-traefik`: checkout + kaniko build+push orca-traefik image
(needs job 1; same kaniko approach)
2. `.coreci.yml` — remove `container-publish` and `container-publish-traefik`
steps (they now live in the Gitea Actions workflow). Keep the
`gitea-release` step (tarball + Gitea release).
3. `scripts/trigger_coreci.sh` — add tag ref handling (or document that
Gitea Actions is the trigger; the hook is for branch-push CI only).
+36 -9
View File
@@ -411,15 +411,15 @@ node type.
| ID | Requirement | Priority | Phase | Status | | ID | Requirement | Priority | Phase | Status |
|----|-------------|----------|-------|--------| |----|-------------|----------|-------|--------|
| REQ-171 | `Dockerfile.traefik` + release pipeline: build + publish `git.cloudinit.dev/coreci/orca-traefik:<version>` alongside the orca image per release; `.coreci.yml` `container-publish-traefik` step; image bakes default `traefik.yml` (entrypoints `websecure` 127.0.0.1:8443, `web` 127.0.0.1:8080, `traefik` 127.0.0.1:8081 + file provider watching `/etc/traefik/dynamic` + json log/accessLog); host-side `/etc/traefik/traefik.yml` mounted `:ro` overrides baked config (preserves `traefik-on-public-ip` opt-out REQ-100); no `certificatesResolvers` (traefik v3.3 only supports acme/tailscale); `tls: {}` in dynamic config for v0.14 (real mTLS deferred to v0.15) | Critical | **v0.14 P1** | pending | | REQ-171 | `Dockerfile.traefik` + release pipeline: build + publish `git.cloudinit.dev/coreci/orca-traefik:<version>` alongside the orca image per release; `.coreci.yml` `container-publish-traefik` step; image bakes default `traefik.yml` (entrypoints `websecure` 127.0.0.1:8443, `web` 127.0.0.1:8080, `traefik` 127.0.0.1:8081 + file provider watching `/etc/traefik/dynamic` + json log/accessLog); host-side `/etc/traefik/traefik.yml` mounted `:ro` overrides baked config (preserves `traefik-on-public-ip` opt-out REQ-100); no `certificatesResolvers` (traefik v3.3 only supports acme/tailscale); `tls: {}` in dynamic config for v0.14 (real mTLS deferred to v0.15) | Critical | **v0.14 P1** | complete |
| REQ-172 | Replace `internal/traefik/install.go` binary+systemd install with a podman-container reconciler: `podman pull orca-traefik:<tag>` + `podman run -d --restart=unless-stopped --network host --name orca-traefik -v /etc/traefik/traefik.yml:/etc/traefik/traefik.yml:ro -v /etc/traefik/dynamic:/etc/traefik/dynamic:ro -v /etc/orca/step-ca-root.crt:/etc/orca/step-ca-root.crt:ro <image>`; idempotent (pull+run if absent, start if stopped); install podman if absent (C-50); v0.13→v0.14 upgrade: detect+stop+disable+remove legacy `orca-traefik.service` + `/usr/local/bin/traefik` (C-57); works locally + over SSH-push; enable `podman-restart.service`; remove systemd unit generation | Critical | **v0.14 P2** | pending | | REQ-172 | Replace `internal/traefik/install.go` binary+systemd install with a podman-container reconciler: `podman pull orca-traefik:<tag>` + `podman run -d --restart=unless-stopped --network host --name orca-traefik -v /etc/traefik/traefik.yml:/etc/traefik/traefik.yml:ro -v /etc/traefik/dynamic:/etc/traefik/dynamic:ro -v /etc/orca/step-ca-root.crt:/etc/orca/step-ca-root.crt:ro <image>`; idempotent (pull+run if absent, start if stopped); install podman if absent (C-50); v0.13→v0.14 upgrade: detect+stop+disable+remove legacy `orca-traefik.service` + `/usr/local/bin/traefik` (C-57); works locally + over SSH-push; enable `podman-restart.service`; remove systemd unit generation | Critical | **v0.14 P2** | complete |
| REQ-173 | nft SNAT+DNAT ruleset render+apply: extend `internal/emitter/nft.go` with postrouting masquerade chain; new `internal/ingress/bootstrap.go` renders `orca.nft` + applies `nft -f` + ensures `/etc/traefik/dynamic` dir + pushes step-ca root CA + invokes podman traefik reconciler; wired into `orca init` (localhost lead) | Critical | **v0.14 P3** | pending | | REQ-173 | nft SNAT+DNAT ruleset render+apply: extend `internal/emitter/nft.go` with postrouting masquerade chain; new `internal/ingress/bootstrap.go` renders `orca.nft` + applies `nft -f` + ensures `/etc/traefik/dynamic` dir + pushes step-ca root CA + invokes podman traefik reconciler; wired into `orca init` (localhost lead) | Critical | **v0.14 P3** | complete |
| REQ-174 | Remote ingress bootstrap via SSH-push for `orca node join --type linux`: push step-ca root CA, render+apply nft remotely, invoke podman traefik reconciler remotely; register node as `linux` | Critical | **v0.14 P4** | pending | | REQ-174 | Remote ingress bootstrap via SSH-push for `orca node join --type linux`: push step-ca root CA, render+apply nft remotely, invoke podman traefik reconciler remotely; register node as `linux` | Critical | **v0.14 P4** | complete |
| REQ-175 | Proxmox native ingress mode (`--ingress-mode native`, default): on PVE host, render+apply nft (vmbr-compatible, separate `orca-ingress` table avoids pve-firewall conflict); create unprivileged LXC with `--features nesting=1,keyctl=1` running podman+orca-traefik; push step-ca root CA into LXC; nft DNAT target = LXC bridge IP; register PVE host as `proxmox` node; add `IngressMode` field to `model.Node` + schema migration | Critical | **v0.14 P5** | pending | | REQ-175 | Proxmox native ingress mode (`--ingress-mode native`, default): on PVE host, render+apply nft (vmbr-compatible, separate `orca-ingress` table avoids pve-firewall conflict); create unprivileged LXC with `--features nesting=1,keyctl=1` running podman+orca-traefik; push step-ca root CA into LXC; nft DNAT target = LXC bridge IP; register PVE host as `proxmox` node; add `IngressMode` field to `model.Node` + schema migration | Critical | **v0.14 P5** | complete |
| REQ-176 | Proxmox floating-IP mode (`--ingress-mode floating-ip --floating-ip --gateway --mac [--net-prefix]`): `pct create` Ubuntu LXC named `ingress` with `net0 bridge=vmbr0,hwaddr=<mac>,ip=<floating-ip>/<prefix>,gw=<gateway> --features nesting=1,keyctl=1 --onboot 1`; install podman + run orca-traefik inside LXC; apply nft DNAT+SNAT inside LXC; register LXC as managed `linux` node (name=`ingress`, addr=`<floating-ip>:8443`); interactive prompt for params when flags absent + not `--json`; validate IP/MAC/gateway; PVE host also registered as `proxmox` for workload dispatch | Critical | **v0.14 P6** | pending | | REQ-176 | Proxmox floating-IP mode (`--ingress-mode floating-ip --floating-ip --gateway --mac [--net-prefix]`): `pct create` Ubuntu LXC named `ingress` with `net0 bridge=vmbr0,hwaddr=<mac>,ip=<floating-ip>/<prefix>,gw=<gateway> --features nesting=1,keyctl=1 --onboot 1`; install podman + run orca-traefik inside LXC; apply nft DNAT+SNAT inside LXC; register LXC as managed `linux` node (name=`ingress`, addr=`<floating-ip>:8443`); interactive prompt for params when flags absent + not `--json`; validate IP/MAC/gateway; PVE host also registered as `proxmox` for workload dispatch | Critical | **v0.14 P6** | complete |
| REQ-177 | `orca doctor ingress [--peer]`: verify orca-traefik container running (`podman inspect`), nft DNAT+SNAT applied, `/etc/traefik/dynamic` exists, step-ca root CA mounted; extend `scripts/uat-signoff.sh` with ingress assertions (40 podman_traefik, 41 nft_dnat_snat, 42 linux_worker, 43 proxmox_native_lxc / floating_ip_lxc) | High | **v0.14 P7** | pending | | REQ-177 | `orca doctor ingress [--peer]`: verify orca-traefik container running (`podman inspect`), nft DNAT+SNAT applied, `/etc/traefik/dynamic` exists, step-ca root CA mounted; extend `scripts/uat-signoff.sh` with ingress assertions (40 podman_traefik, 41 nft_dnat_snat, 42 linux_worker, 43 proxmox_native_lxc / floating_ip_lxc) | High | **v0.14 P7** | complete |
| REQ-178 | Docs: `docs/cli.md` (`--ingress-mode` + floating-IP flags + `doctor ingress`), `docs/uat.md` (native + floating-IP topologies), `docs/ingress.md` (podman-traefik image + volume mounts + certResolver), `docs/docker.md` (orca-traefik image), `ARCHITECTURE.md` (R-024 + ingress bootstrap section) | High | **v0.14 P7** | pending | | REQ-178 | Docs: `docs/cli.md` (`--ingress-mode` + floating-IP flags + `doctor ingress`), `docs/uat.md` (native + floating-IP topologies), `docs/ingress.md` (podman-traefik image + volume mounts + certResolver), `docs/docker.md` (orca-traefik image), `ARCHITECTURE.md` (R-024 + ingress bootstrap section) | High | **v0.14 P7** | complete |
| REQ-179 | Integration tests: hermetic harness fakes SSH; asserts init→podman traefik running + nft applied; linux join→remote podman+nft; proxmox native→LXC created with nesting + podman traefik; floating-ip→`pct create` with correct net0 args + LXC registered as linux node; release.sh builds orca-traefik image (Dockerfile.traefik parses) | Critical | **v0.14 P7** | pending | | REQ-179 | Integration tests: hermetic harness fakes SSH; asserts init→podman traefik running + nft applied; linux join→remote podman+nft; proxmox native→LXC created with nesting + podman traefik; floating-ip→`pct create` with correct net0 args + LXC registered as linux node; release.sh builds orca-traefik image (Dockerfile.traefik parses) | Critical | **v0.14 P7** | complete |
### Scope notes (v0.14) ### Scope notes (v0.14)
@@ -427,3 +427,30 @@ node type.
- 9 phases (P0 + P1..P7 + P8 final); feature milestone (multiple `feat` phases). - 9 phases (P0 + P1..P7 + P8 final); feature milestone (multiple `feat` phases).
- Tags on v0.13.x patch line: `v0.13.0` (P0) ... `v0.13.8` (P8 final = v0.14 milestone release). - Tags on v0.13.x patch line: `v0.13.0` (P0) ... `v0.13.8` (P8 final = v0.14 milestone release).
- Milestone branch: `milestone/v0.14-ingress-bootstrap`. - Milestone branch: `milestone/v0.14-ingress-bootstrap`.
## Milestone v0.15: CI Release Pipeline Fix
**Scope**: fix the container image publishing pipeline. v0.14 shipped
`Dockerfile.traefik` + `Dockerfile` but no container images were
published to the Gitea registry because: (1) no Gitea Actions workflow
existed to trigger on tag pushes, (2) the CoreCI trigger script
stripped tag refs, (3) the `.coreci.yml` container-publish steps used
Docker-in-Docker (`docker:24-cli`) which is prohibited. v0.15 adds a
Gitea Actions workflow that triggers on tag pushes, installs the
`coreci` binary on the runner, and runs `coreci run`. The
`.coreci.yml` container-publish steps are rewritten to use kaniko
(no Docker daemon required).
| ID | Requirement | Priority | Phase | Status |
|----|-------------|----------|-------|--------|
| REQ-180 | Create `.gitea/workflows/release.yml` that triggers on `push: tags: ['v*']`, installs the `coreci` binary (from `git.cloudinit.dev/coreci/coreci`), injects `PAT_TOKEN` secret as `GITEA_TOKEN` env var, and runs `coreci run` — which executes the full `.coreci.yml` pipeline (validate, build, test, release) locally on the Gitea Actions runner | Critical | **v0.15 P1** | complete |
| REQ-181 | Replace `docker:24-cli` DinD steps in `.coreci.yml` with kaniko (`gcr.io/kaniko-project/executor:debug`): write `/kaniko/.docker/config.json` from `GITEA_TOKEN` (base64 auth), run `/kaniko/executor --dockerfile=<Dockerfile> --context=dir://. --destination=<registry/image:tag> --skip-tls-verify-registry`. Applies to both `container-publish` (orca image) and `container-publish-traefik` (orca-traefik image) | Critical | **v0.15 P1** | complete |
| REQ-182 | Set `PAT_TOKEN` Gitea Actions repository secret via `tea actions secrets create` (same value as `GITEA_TOKEN` from `.env`). Gitea reserves the `GITEA_` prefix for built-in secrets, so the secret must be named `PAT_TOKEN`, not `GITEA_PAT` | High | **v0.15 P0** | complete |
### Scope notes (v0.15)
- REQ-180..REQ-182 = 3 net-new requirements (REQ count grows 172 -> 175).
- 3 phases (P0 + P1 + P2 final); fix milestone (no `feat` phases — CI infrastructure).
- Tags on v0.14.x patch line: `v0.14.0` (P0) ... `v0.14.2` (P2 final = v0.15 milestone release).
- Milestone branch: `milestone/v0.15-ci-release-pipeline`.
- REQ-182 is complete: `PAT_TOKEN` secret created via `tea actions secrets create PAT_TOKEN <value> --repo coreci/orca`.
+26 -10
View File
@@ -676,7 +676,7 @@ CI agent verifies and cuts v1.0.0).
- jobspec `update` rolling/canary controller (v0.13 adds lint warning; enforcement deferred) - jobspec `update` rolling/canary controller (v0.13 adds lint warning; enforcement deferred)
- jobspec `schedule.cron` scheduler loop (v0.13 adds lint warning; enforcement deferred) - jobspec `schedule.cron` scheduler loop (v0.13 adds lint warning; enforcement deferred)
## Milestone v0.14: Ingress Bootstrap Completeness — **IN PROGRESS** ## Milestone v0.14: Ingress Bootstrap Completeness — **COMPLETE**
**Scope**: ensure that linux & proxmox types are properly bootstrapped with **Scope**: ensure that linux & proxmox types are properly bootstrapped with
traefik during cluster init or node join. All cluster endpoints are traefik during cluster init or node join. All cluster endpoints are
@@ -701,15 +701,15 @@ root CA volume mounts.
**Milestone type**: feature (multiple `feat` phases). Tags on v0.13.x patch **Milestone type**: feature (multiple `feat` phases). Tags on v0.13.x patch
line: `v0.13.0` (P0) ... `v0.13.8` (P8 final = v0.14 milestone release). line: `v0.13.0` (P0) ... `v0.13.8` (P8 final = v0.14 milestone release).
- [ ] Phase 0: Pre-execution (SPECIFY→CLARIFY→RESEARCH→PLAN→GRILL) — tag `v0.13.0` - [x] Phase 0: Pre-execution (SPECIFY→CLARIFY→RESEARCH→PLAN→GRILL) — tag `v0.13.0`
- [ ] Phase 1: `orca-traefik` container image + release pipeline (REQ-171) — tag `v0.13.1` - [x] Phase 1: `orca-traefik` container image + release pipeline (REQ-171) — tag `v0.13.1`
- [ ] Phase 2: Podman traefik reconciler — replace binary+systemd install (REQ-172) — tag `v0.13.2` - [x] Phase 2: Podman traefik reconciler — replace binary+systemd install (REQ-172) — tag `v0.13.2`
- [ ] Phase 3: nft SNAT+DNAT + `orca init` ingress bootstrap (REQ-173) — tag `v0.13.3` - [x] Phase 3: nft SNAT+DNAT + `orca init` ingress bootstrap (REQ-173) — tag `v0.13.3`
- [ ] Phase 4: `orca node join --type linux` remote ingress bootstrap (REQ-174) — tag `v0.13.4` - [x] Phase 4: `orca node join --type linux` remote ingress bootstrap (REQ-174) — tag `v0.13.4`
- [ ] Phase 5: Proxmox native ingress mode — LXC + podman traefik (REQ-175) — tag `v0.13.5` - [x] Phase 5: Proxmox native ingress mode — LXC + podman traefik (REQ-175) — tag `v0.13.5`
- [ ] Phase 6: Proxmox floating-IP LXC ingress + interactive prompt (REQ-176) — tag `v0.13.6` - [x] Phase 6: Proxmox floating-IP LXC ingress + interactive prompt (REQ-176) — tag `v0.13.6`
- [ ] Phase 7: `doctor ingress` + docs + integration tests (REQ-177,178,179) — tag `v0.13.7` - [x] Phase 7: `doctor ingress` + docs + integration tests (REQ-177,178,179) — tag `v0.13.7`
- [ ] Phase 8: Final review + ship + audit (milestone release) — tag `v0.13.8` = **v0.14 milestone release** - [x] Phase 8: Final review + ship + audit (milestone release) — tag `v0.13.8` = **v0.14 milestone release**
### Per-phase REQ coverage (v0.14) ### Per-phase REQ coverage (v0.14)
@@ -729,3 +729,19 @@ is unchanged. v0.14 completes the ingress bootstrap that v0.13 left
non-functional (binary installed but no config, no nft applied). The non-functional (binary installed but no config, no nft applied). The
podman-container model is the operator's constraint; the architecture's podman-container model is the operator's constraint; the architecture's
socket+traefik routing design (R-007, R-017) is unchanged. socket+traefik routing design (R-007, R-017) is unchanged.
## Milestone v0.15: CI Release Pipeline Fix — **COMPLETE**
**Scope**: fix container image publishing. v0.14 shipped
`Dockerfile.traefik` + `Dockerfile` but no images were published
because no Gitea Actions workflow triggered on tag pushes, and
`.coreci.yml` used Docker-in-Docker. v0.15 adds a Gitea Actions
workflow (trigger on tag push → install coreci → `coreci run`) and
rewrites the container-publish steps to use kaniko (no DinD).
**Milestone type**: fix (CI infrastructure). Tags on v0.14.x patch
line: `v0.14.0` (P0) ... `v0.14.2` (P2 final = v0.15 milestone release).
- [x] Phase 0: Pre-execution (SPECIFY→CLARIFY→RESEARCH→PLAN→GRILL) — tag `v0.14.0`
- [x] Phase 1: Gitea Actions workflow + .coreci.yml kaniko rewrite (REQ-180,181) — tag `v0.14.1`
- [x] Phase 2: Final review + ship + audit (milestone release) — tag `v0.14.2` = **v0.15 milestone release**
+2 -2
View File
@@ -5,9 +5,9 @@
"slug": "orca", "slug": "orca",
"name": "Orca", "name": "Orca",
"description": "Offline/CLI-first orchestration engine (Orca) \u2014 Nomad-inspired, far simpler than Kubernetes", "description": "Offline/CLI-first orchestration engine (Orca) \u2014 Nomad-inspired, far simpler than Kubernetes",
"milestone": "v0.14", "milestone": "v0.15",
"phase": 0, "phase": 0,
"milestone_type": "feature", "milestone_type": "fix",
"default_branch": "main", "default_branch": "main",
"tech_stack": { "tech_stack": {
"language": "go", "language": "go",
-20
View File
@@ -147,23 +147,3 @@ pipelines:
-F "attachment=@SHA256SUMS" -F "attachment=@SHA256SUMS"
fi fi
fi fi
- name: container-publish
description: Build and publish OCI image to Gitea container registry (REQ-046)
image: docker:24-cli
env:
GITEA_TOKEN: ${GITEA_TOKEN}
VERSION: ${CI_COMMIT_TAG}
GIT_COMMIT: ${CI_COMMIT_SHA}
BUILD_TIME: ${CI_BUILD_TIME}
commands:
- docker build
--build-arg VERSION=${VERSION}
--build-arg GIT_COMMIT=${GIT_COMMIT}
--build-arg BUILD_TIME=${BUILD_TIME}
-t git.cloudinit.dev/coreci/orca:${VERSION}
-t git.cloudinit.dev/coreci/orca:latest
.
- echo "${GITEA_TOKEN}" | docker login git.cloudinit.dev -u cloudinit-bot --password-stdin
- docker push git.cloudinit.dev/coreci/orca:${VERSION}
- docker push git.cloudinit.dev/coreci/orca:latest
- docker logout git.cloudinit.dev
+90
View File
@@ -0,0 +1,90 @@
name: Release
on:
push:
tags:
- 'v*'
jobs:
ci:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '1.25'
- name: Install CoreCI
run: |
git clone --depth=1 https://git.cloudinit.dev/coreci/coreci.git /tmp/coreci
cd /tmp/coreci
CGO_ENABLED=0 go build -tags sqlite_go,embed -o /usr/local/bin/coreci ./cmd/coreci
coreci version
- name: Run CoreCI pipeline
env:
GITEA_TOKEN: ${{ secrets.PAT_TOKEN }}
run: |
coreci run
container-orca:
runs-on: ubuntu-latest
needs: ci
container:
image: gcr.io/kaniko-project/executor:debug
options: --entrypoint /bin/sh
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Build and push orca image
env:
GITEA_TOKEN: ${{ secrets.PAT_TOKEN }}
VERSION: ${{ gitea.ref_name }}
run: |
mkdir -p /kaniko/.docker
AUTH=$(echo -n "cloudinit-bot:${GITEA_TOKEN}" | base64 -w0)
echo "{\"auths\":{\"git.cloudinit.dev\":{\"auth\":\"${AUTH}\"}}}" > /kaniko/.docker/config.json
GIT_COMMIT=$(echo -n "${{ gitea.sha }}" | cut -c1-12)
BUILD_TIME=$(date -u +%Y-%m-%dT%H:%M:%SZ)
/kaniko/executor \
--dockerfile=Dockerfile \
--context=dir://. \
--destination=git.cloudinit.dev/coreci/orca:${VERSION} \
--destination=git.cloudinit.dev/coreci/orca:latest \
--build-arg=VERSION=${VERSION} \
--build-arg=GIT_COMMIT=${GIT_COMMIT} \
--build-arg=BUILD_TIME=${BUILD_TIME} \
--skip-tls-verify-registry
container-traefik:
runs-on: ubuntu-latest
needs: ci
container:
image: gcr.io/kaniko-project/executor:debug
options: --entrypoint /bin/sh
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Build and push orca-traefik image
env:
GITEA_TOKEN: ${{ secrets.PAT_TOKEN }}
VERSION: ${{ gitea.ref_name }}
run: |
if [ ! -f Dockerfile.traefik ]; then
echo "Dockerfile.traefik not found at this tag — skipping orca-traefik image"
exit 0
fi
mkdir -p /kaniko/.docker
AUTH=$(echo -n "cloudinit-bot:${GITEA_TOKEN}" | base64 -w0)
echo "{\"auths\":{\"git.cloudinit.dev\":{\"auth\":\"${AUTH}\"}}}" > /kaniko/.docker/config.json
/kaniko/executor \
--dockerfile=Dockerfile.traefik \
--context=dir://. \
--destination=git.cloudinit.dev/coreci/orca-traefik:${VERSION} \
--destination=git.cloudinit.dev/coreci/orca-traefik:latest \
--skip-tls-verify-registry
+33
View File
@@ -0,0 +1,33 @@
# Dockerfile.traefik — custom orca-traefik image (R-024)
#
# Extends the official traefik:v3.3.0 image with a baked default static
# config. The host-side /etc/traefik/traefik.yml (rendered by
# emitter.RenderTraefikStaticConfig) is mounted :ro at runtime to
# override this default — preserving the traefik-on-public-ip opt-out
# (REQ-100) and any site-local customisation.
#
# Dynamic config (routers, services, certs) is mounted from
# /etc/traefik/dynamic on the host — orca writes to it atomically via
# the SSH-push transport (C-10 protocol).
#
# Build:
# docker build -f Dockerfile.traefik -t git.cloudinit.dev/coreci/orca-traefik:v0.13.1 .
#
# Run (hybrid R-017 mode — nft DNATs :443/:80 to loopback):
# podman run -d --name orca-traefik --restart=unless-stopped \
# --network host \
# -v /etc/traefik/traefik.yml:/etc/traefik/traefik.yml:ro \
# -v /etc/traefik/dynamic:/etc/traefik/dynamic:ro \
# -v /etc/orca/step-ca-root.crt:/etc/orca/step-ca-root.crt:ro \
# git.cloudinit.dev/coreci/orca-traefik:v0.13.1
FROM traefik:v3.3.0
LABEL org.opencontainers.image.title="orca-traefik"
LABEL org.opencontainers.image.description="Custom Traefik image for Orca ingress (R-024)"
LABEL org.opencontainers.image.source="https://git.cloudinit.dev/coreci/orca"
COPY docker/orca-traefik/traefik.yml /etc/traefik/traefik.yml
COPY docker/orca-traefik/step-ca-root.crt /etc/orca/step-ca-root.crt
CMD ["--configFile=/etc/traefik/traefik.yml"]
+19
View File
@@ -0,0 +1,19 @@
entryPoints:
websecure:
address: "127.0.0.1:8443"
web:
address: "127.0.0.1:8080"
traefik:
address: "127.0.0.1:8081"
providers:
file:
directory: "/etc/traefik/dynamic"
watch: true
log:
level: INFO
format: json
accessLog:
format: json
+71 -7
View File
@@ -138,13 +138,77 @@ restore traffic).
## TLS ## TLS
- **certResolver**: `orca` (references the Traefik ACME/step-ca - **v0.14 model**: `tls: {}` in dynamic config (no certResolver).
certificate resolver configured in Traefik's static config). Traefik v3.3 `certificatesResolvers` only supports `acme` and
- **Trust domain**: `cluster.orca.local` (placeholder in v0.9; step-ca `tailscale` — not CA-file-based. The `certResolver: orca` reference
provisioner in v0.11 overrides with the real cluster trust domain). from v0.11 was broken (research finding). v0.14 emits `tls: {}`
- **SPIFFE SVIDs**: workload identity via SPIFFE SVIDs minted at submit (traefik uses its default self-signed cert). Real mTLS via dynamic
time via step-ca (v0.11-P01.5, gate C-08). The SVID is a URI SAN in `tls.certificates` + `tls.options.default.clientAuth.caFiles` is
the workload's X.509 cert. deferred to v0.15.
- **Step-ca root CA**: mounted at `/etc/orca/step-ca-root.crt` in the
traefik container. v0.14 does not use it for TLS termination (it's
a placeholder for v0.15 mTLS).
## R-024: Podman Traefik Container (v0.14)
As of v0.14, Traefik runs as a **podman container** from the custom
`orca-traefik` image (published per release). The v0.13 binary+systemd
install is replaced.
### Three topologies
1. **Linux**: host → nft DNAT → `podman run orca-traefik` (`--network host`)
2. **Proxmox Native** (`--ingress-mode native`, default): PVE host →
nft DNAT → LXC (nesting=1,keyctl=1,fuse=1) → `podman run orca-traefik`
3. **Proxmox Floating-IP** (`--ingress-mode floating-ip`): LXC owns
the floating IP → nft inside LXC → `podman run orca-traefik`
### Container configuration
```bash
podman run -d --name orca-traefik --restart=unless-stopped \
--network host \
-v /etc/traefik/traefik.yml:/etc/traefik/traefik.yml:ro \
-v /etc/traefik/dynamic:/etc/traefik/dynamic:ro \
-v /etc/orca/step-ca-root.crt:/etc/orca/step-ca-root.crt:ro \
git.cloudinit.dev/coreci/orca-traefik:<version>
```
- `--network host`: traefik binds 127.0.0.1:8080/8443 on host/LXC loopback
- `--restart=unless-stopped`: survives reboot via `podman-restart.service`
- No `:Z` SELinux flag (research Topic 7)
- Static config mounted `:ro` (overrides baked image default, preserves
`traefik-on-public-ip` opt-out, REQ-100)
### nft ruleset
The nft emitter (`internal/emitter/nft.go`) renders `/etc/nftables.d/orca.nft`:
- DNAT `:443``<DNATTarget>:8443` (default 127.0.0.1; LXC IP for native)
- DNAT `:80``<DNATTarget>:8080`
- SNAT/MASQUERADE: `ip saddr 127.0.0.0/8 oifname != "lo" masquerade`
- Input/forward chains at priority -10 (pve-firewall coexistence)
### `orca doctor ingress`
```bash
orca doctor ingress # check localhost
orca doctor ingress --peer <name> # check remote peer
```
Verifies: podman container running, nft DNAT+SNAT, dynamic dir exists,
step-ca root CA present.
### Dockerfile.traefik
```dockerfile
FROM traefik:v3.3.0
COPY docker/orca-traefik/traefik.yml /etc/traefik/traefik.yml
CMD ["--configFile=/etc/traefik/traefik.yml"]
```
Built + published per release alongside the orca image
(`scripts/release.sh` + `.coreci.yml container-publish-traefik`).
## Health checks ## Health checks
+115
View File
@@ -0,0 +1,115 @@
// Package cli: doctor_ingress.go implements `orca doctor ingress`
// (R-024, v0.14). The check verifies the podman traefik container is
// running, nft DNAT+SNAT is applied, the dynamic config directory
// exists, and the step-ca root CA is mounted.
package cli
import (
"context"
"fmt"
"strings"
"time"
"github.com/spf13/cobra"
)
var doctorIngressCmd = &cobra.Command{
Use: "ingress",
Short: "Check the ingress stack (R-024: podman traefik + nft + CA)",
Long: `Verify the orca ingress data plane is healthy:
1. orca-traefik podman container is running
2. nft DNAT + SNAT masquerade applied
3. /etc/traefik/dynamic directory exists
4. step-ca root CA mounted at /etc/orca/step-ca-root.crt
For remote peers, use --peer <name>.`,
RunE: func(cmd *cobra.Command, args []string) error {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
results := runIngressChecks(ctx)
if jsonOutput {
return printJSON(results)
}
allPass := true
for _, r := range results {
status := "✓"
if r.Result != "PASS" {
status = "✗"
allPass = false
}
fmt.Fprintf(cmd.OutOrStdout(), "%s %s: %s\n", status, r.Name, r.Message)
}
if !allPass {
return fmt.Errorf("ingress checks failed")
}
return nil
},
}
// ingressCheckResult is one line of `orca doctor ingress` output.
type ingressCheckResult struct {
Name string `json:"name"`
Result string `json:"result"`
Message string `json:"message"`
}
func runIngressChecks(ctx context.Context) []ingressCheckResult {
t, err := nftTransportFromCtx()
if err != nil {
return []ingressCheckResult{{Name: "ingress:transport", Result: "FAIL", Message: err.Error()}}
}
peer := nftLeadPeer()
var results []ingressCheckResult
// 1. Check podman orca-traefik container is running.
out, err := t.Exec(ctx, peer, "podman inspect --format '{{.State.Running}}' orca-traefik 2>/dev/null")
if err != nil {
results = append(results, ingressCheckResult{Name: "ingress:container", Result: "FAIL", Message: fmt.Sprintf("podman inspect: %v", err)})
} else {
v := strings.TrimSpace(string(out))
if v == "true" {
results = append(results, ingressCheckResult{Name: "ingress:container", Result: "PASS", Message: "orca-traefik container running"})
} else if v == "false" {
results = append(results, ingressCheckResult{Name: "ingress:container", Result: "FAIL", Message: "orca-traefik container is stopped"})
} else {
results = append(results, ingressCheckResult{Name: "ingress:container", Result: "FAIL", Message: "orca-traefik container not found"})
}
}
// 2. Check nft DNAT + SNAT (reuse the nft table output).
tableOut, tableErr := t.Exec(ctx, peer, "nft list table inet orca-ingress 2>/dev/null")
if tableErr != nil {
results = append(results, ingressCheckResult{Name: "ingress:nft", Result: "FAIL", Message: "nft table orca-ingress missing"})
} else {
tableStr := string(tableOut)
hasDNAT := strings.Contains(tableStr, "dnat to")
hasSNAT := strings.Contains(tableStr, "masquerade")
if hasDNAT && hasSNAT {
results = append(results, ingressCheckResult{Name: "ingress:nft", Result: "PASS", Message: "nft DNAT + SNAT masquerade present"})
} else if hasDNAT {
results = append(results, ingressCheckResult{Name: "ingress:nft", Result: "WARN", Message: "DNAT present but SNAT masquerade missing"})
} else {
results = append(results, ingressCheckResult{Name: "ingress:nft", Result: "FAIL", Message: "nft DNAT missing"})
}
}
// 3. Check /etc/traefik/dynamic directory exists.
if _, err := t.Exec(ctx, peer, "test -d /etc/traefik/dynamic"); err != nil {
results = append(results, ingressCheckResult{Name: "ingress:dynamic-dir", Result: "FAIL", Message: "/etc/traefik/dynamic directory missing"})
} else {
results = append(results, ingressCheckResult{Name: "ingress:dynamic-dir", Result: "PASS", Message: "/etc/traefik/dynamic exists"})
}
// 4. Check step-ca root CA is mounted/present.
if _, err := t.Exec(ctx, peer, "test -f /etc/orca/step-ca-root.crt"); err != nil {
results = append(results, ingressCheckResult{Name: "ingress:ca", Result: "WARN", Message: "/etc/orca/step-ca-root.crt missing (TLS not configured)"})
} else {
results = append(results, ingressCheckResult{Name: "ingress:ca", Result: "PASS", Message: "step-ca root CA present"})
}
return results
}
func init() {
doctorCmd.AddCommand(doctorIngressCmd)
}
+14
View File
@@ -113,6 +113,20 @@ func runNftChecks(ctx context.Context) []nftCheckResult {
results = append(results, nftCheckResult{Name: "nft:dnat-80", Result: "FAIL", Message: "DNAT :80->127.0.0.1:8080 missing"}) results = append(results, nftCheckResult{Name: "nft:dnat-80", Result: "FAIL", Message: "DNAT :80->127.0.0.1:8080 missing"})
} }
// Research Topic 1: postrouting masquerade for DNAT return path.
if strings.Contains(tableStr, "masquerade") {
results = append(results, nftCheckResult{Name: "nft:snat-masquerade", Result: "PASS", Message: "postrouting masquerade (SNAT) present"})
} else {
results = append(results, nftCheckResult{Name: "nft:snat-masquerade", Result: "FAIL", Message: "postrouting masquerade missing (R-024)"})
}
// Research Topic 2: priority -10 on input/forward (pve-firewall coexistence).
if strings.Contains(tableStr, "priority -10") {
results = append(results, nftCheckResult{Name: "nft:priority", Result: "PASS", Message: "input/forward chains at priority -10 (pve-firewall coexistence)"})
} else {
results = append(results, nftCheckResult{Name: "nft:priority", Result: "WARN", Message: "priority -10 not found (may be pre-v0.14 ruleset)"})
}
if strings.Contains(tableStr, "ora_rl") { if strings.Contains(tableStr, "ora_rl") {
results = append(results, nftCheckResult{Name: "nft:rate-limit", Result: "PASS", Message: "rate-limit meter ora_rl present"}) results = append(results, nftCheckResult{Name: "nft:rate-limit", Result: "PASS", Message: "rate-limit meter ora_rl present"})
} else { } else {
+22 -16
View File
@@ -1,27 +1,28 @@
package cli package cli
import ( import (
"bufio"
"context" "context"
"crypto/x509" "crypto/x509"
"encoding/pem" "encoding/pem"
"fmt" "fmt"
"bufio"
"os"
"strings"
"os/exec"
"path/filepath"
"time"
"github.com/google/uuid"
"golang.org/x/crypto/ssh"
"github.com/spf13/cobra"
"git.cloudinit.dev/coreci/orca/internal/acl" "git.cloudinit.dev/coreci/orca/internal/acl"
"git.cloudinit.dev/coreci/orca/internal/certpaths" "git.cloudinit.dev/coreci/orca/internal/certpaths"
"git.cloudinit.dev/coreci/orca/internal/identity" "git.cloudinit.dev/coreci/orca/internal/identity"
"git.cloudinit.dev/coreci/orca/internal/ingress"
"git.cloudinit.dev/coreci/orca/internal/model" "git.cloudinit.dev/coreci/orca/internal/model"
"git.cloudinit.dev/coreci/orca/internal/paths" "git.cloudinit.dev/coreci/orca/internal/paths"
"git.cloudinit.dev/coreci/orca/internal/secrets" "git.cloudinit.dev/coreci/orca/internal/secrets"
"git.cloudinit.dev/coreci/orca/internal/security" "git.cloudinit.dev/coreci/orca/internal/security"
"git.cloudinit.dev/coreci/orca/internal/store" "git.cloudinit.dev/coreci/orca/internal/store"
"github.com/google/uuid"
"github.com/spf13/cobra"
"golang.org/x/crypto/ssh"
"os"
"os/exec"
"path/filepath"
"strings"
"time"
) )
const ( const (
@@ -251,17 +252,22 @@ func runInit(out interface{ Write([]byte) (int, error) }) error {
} }
} }
// Step 4d: Install Traefik on the lead node (REQ-165, Phase B). // Step 4d: Ensure complete ingress stack on the lead node (R-024).
// Traefik is the data-plane ingress. Idempotent. // This replaces the v0.13 binary+systemd traefik install with:
if err := installTraefikLocal(); err != nil { // 1. Render + write traefik static config (traefik.yml)
// 2. Render + write + apply nft DNAT/SNAT ruleset (orca.nft)
// 3. Push step-ca root CA to /etc/orca/step-ca-root.crt
// 4. Ensure podman orca-traefik container running
// All steps non-fatal (offline host tolerance).
if err := ingress.BootstrapLocalIngress(context.Background(), version); err != nil {
if !jsonOutput { if !jsonOutput {
fmt.Fprintf(out, "Traefik install skipped: %v\n", err) fmt.Fprintf(out, "Ingress bootstrap skipped: %v\n", err)
} }
summary.Steps = append(summary.Steps, stepResult{Label: "traefik", Status: "skipped", Detail: err.Error()}) summary.Steps = append(summary.Steps, stepResult{Label: "ingress", Status: "skipped", Detail: err.Error()})
} else { } else {
summary.Steps = append(summary.Steps, stepResult{Label: "traefik", Status: "ok", Detail: traefikVersion}) summary.Steps = append(summary.Steps, stepResult{Label: "ingress", Status: "ok", Detail: "nft+traefik container active"})
if !jsonOutput { if !jsonOutput {
fmt.Fprintf(out, "Traefik installed: %s\n", traefikVersion) fmt.Fprintf(out, "Ingress: nft DNAT+SNAT applied, traefik container running\n")
} }
} }
+182 -2
View File
@@ -1,22 +1,26 @@
package cli package cli
import ( import (
"bufio"
"context" "context"
"database/sql" "database/sql"
"encoding/json" "encoding/json"
"fmt" "fmt"
"log/slog" "log/slog"
"net"
"os" "os"
"os/signal" "os/signal"
"strings"
"syscall" "syscall"
"time" "time"
"github.com/google/uuid" "github.com/google/uuid"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"golang.org/x/crypto/ssh"
"git.cloudinit.dev/coreci/orca/internal/certpaths" "git.cloudinit.dev/coreci/orca/internal/certpaths"
"git.cloudinit.dev/coreci/orca/internal/linux"
"git.cloudinit.dev/coreci/orca/internal/engine" "git.cloudinit.dev/coreci/orca/internal/engine"
"git.cloudinit.dev/coreci/orca/internal/linux"
"git.cloudinit.dev/coreci/orca/internal/model" "git.cloudinit.dev/coreci/orca/internal/model"
"git.cloudinit.dev/coreci/orca/internal/proxmox" "git.cloudinit.dev/coreci/orca/internal/proxmox"
"git.cloudinit.dev/coreci/orca/internal/security" "git.cloudinit.dev/coreci/orca/internal/security"
@@ -58,6 +62,11 @@ var (
joinLXCTemplate string joinLXCTemplate string
proxmoxUser string proxmoxUser string
proxmoxRole string proxmoxRole string
ingressMode string
floatingIP string
gateway string
macAddr string
netPrefix int
leaveID string leaveID string
nodeWatch bool nodeWatch bool
) )
@@ -175,7 +184,83 @@ func joinProxmox(cmd *cobra.Command) error {
return fmt.Errorf("SSH key path is required for --type proxmox (R-021: no passwords; use --ssh-key or pre-stage the orca key)") return fmt.Errorf("SSH key path is required for --type proxmox (R-021: no passwords; use --ssh-key or pre-stage the orca key)")
} }
ctx, cancel := context.WithTimeout(cmd.Context(), 60*time.Second) // Default ingress mode to "native" if not specified (R-024).
effectiveIngressMode := ingressMode
if effectiveIngressMode == "" {
if !jsonOutput {
// Interactive mode: prompt for ingress mode.
fmt.Fprint(cmd.OutOrStdout(), "Ingress mode [native/floating-ip] (default native): ")
scanner := bufio.NewScanner(os.Stdin)
if scanner.Scan() {
input := strings.TrimSpace(scanner.Text())
if input == "floating-ip" {
effectiveIngressMode = "floating-ip"
} else {
effectiveIngressMode = "native"
}
} else {
effectiveIngressMode = "native"
}
} else {
effectiveIngressMode = "native"
}
}
// Floating-IP mode: prompt for params if not provided.
effectiveFloatingIP := floatingIP
effectiveGateway := gateway
effectiveMAC := macAddr
if effectiveIngressMode == "floating-ip" {
if effectiveFloatingIP == "" && !jsonOutput {
fmt.Fprint(cmd.OutOrStdout(), "Floating IP: ")
scanner := bufio.NewScanner(os.Stdin)
if scanner.Scan() {
effectiveFloatingIP = strings.TrimSpace(scanner.Text())
}
}
if effectiveGateway == "" && !jsonOutput {
fmt.Fprint(cmd.OutOrStdout(), "Gateway: ")
scanner := bufio.NewScanner(os.Stdin)
if scanner.Scan() {
effectiveGateway = strings.TrimSpace(scanner.Text())
}
}
if effectiveMAC == "" && !jsonOutput {
// D-261: auto-generate a random locally-administered MAC.
generated, err := proxmox.GenerateRandomMAC()
if err == nil {
fmt.Fprintf(cmd.OutOrStdout(), "Generated MAC: %s (press enter to accept, or type your own): ", generated)
scanner := bufio.NewScanner(os.Stdin)
if scanner.Scan() {
input := strings.TrimSpace(scanner.Text())
if input != "" {
effectiveMAC = input
} else {
effectiveMAC = generated
}
} else {
effectiveMAC = generated
}
}
}
// Validate floating-IP mode params.
if effectiveIngressMode == "floating-ip" {
if net.ParseIP(effectiveFloatingIP) == nil {
return fmt.Errorf("--floating-ip %q is not a valid IP", effectiveFloatingIP)
}
if net.ParseIP(effectiveGateway) == nil {
return fmt.Errorf("--gateway %q is not a valid IP", effectiveGateway)
}
if _, err := net.ParseMAC(effectiveMAC); err != nil {
return fmt.Errorf("--mac %q is not a valid MAC: %w", effectiveMAC, err)
}
if netPrefix < 8 || netPrefix > 32 {
return fmt.Errorf("--net-prefix %d must be 8-32", netPrefix)
}
}
}
ctx, cancel := context.WithTimeout(cmd.Context(), 180*time.Second) // 3min for LXC creation
defer cancel() defer cancel()
result, err := proxmox.BootstrapProxmox(ctx, proxmox.Options{ result, err := proxmox.BootstrapProxmox(ctx, proxmox.Options{
@@ -188,6 +273,7 @@ func joinProxmox(cmd *cobra.Command) error {
HostKeyFingerprint: joinHostKeyFP, HostKeyFingerprint: joinHostKeyFP,
Logger: newLogger(), Logger: newLogger(),
LXCTemplate: joinLXCTemplate, LXCTemplate: joinLXCTemplate,
IngressMode: effectiveIngressMode,
}) })
if err != nil { if err != nil {
return fmt.Errorf("proxmox bootstrap: %w", err) return fmt.Errorf("proxmox bootstrap: %w", err)
@@ -212,10 +298,59 @@ func joinProxmox(cmd *cobra.Command) error {
LastSeen: time.Now().UTC(), LastSeen: time.Now().UTC(),
Kind: string(model.NodeKindProxmox), Kind: string(model.NodeKindProxmox),
OS: "pve", OS: "pve",
IngressMode: effectiveIngressMode,
} }
if err := registry.Join(regCtx, node); err != nil { if err := registry.Join(regCtx, node); err != nil {
return fmt.Errorf("register proxmox node: %w", err) return fmt.Errorf("register proxmox node: %w", err)
} }
// Floating-IP mode: provision the ingress LXC and register it as a
// linux node (R-024, REQ-176). The PVE host is registered as
// proxmox (above); the ingress LXC is registered as linux so
// `orca job run` pushes traefik dynamic config to it.
if effectiveIngressMode == "floating-ip" {
lxcLog := newLogger()
// Build a runRemote function from the proxmox bootstrap result.
// We need SSH access to the PVE host to run pct commands.
lxcCtx, lxcCancel := context.WithTimeout(ctx, 120*time.Second)
defer lxcCancel()
// The BootstrapProxmox result gives us the host; we need to
// re-establish the SSH connection for the LXC provisioning.
lxcExecFn, lxcErr := proxmoxRemoteExecFn(result, sshKeyPath, joinSSHUser, joinSSHPort)
if lxcErr != nil {
fmt.Fprintf(cmd.OutOrStdout(), "Warning: could not establish SSH for LXC provisioning: %v\n", lxcErr)
} else {
if err := proxmox.ProvisionIngressLXC(lxcCtx, lxcExecFn, proxmox.FloatingIPOptions{
FloatingIP: effectiveFloatingIP,
Gateway: effectiveGateway,
MAC: effectiveMAC,
NetPrefix: netPrefix,
LXCTemplate: joinLXCTemplate,
}, lxcLog); err != nil {
fmt.Fprintf(cmd.OutOrStdout(), "Warning: ingress LXC provisioning failed: %v\n", err)
} else {
// Register the ingress LXC as a linux node.
ingressNode := &model.Node{
ID: uuid.NewString(),
Name: "ingress",
Address: fmt.Sprintf("%s:8443", effectiveFloatingIP),
State: model.NodeStateReady,
JoinedAt: time.Now().UTC(),
LastSeen: time.Now().UTC(),
Kind: string(model.NodeKindLinux),
OS: "linux",
IngressMode: "floating-ip",
}
if err := registry.Join(regCtx, ingressNode); err != nil {
fmt.Fprintf(cmd.OutOrStdout(), "Warning: register ingress node: %v\n", err)
}
if !jsonOutput {
fmt.Fprintf(cmd.OutOrStdout(), "✓ Ingress LXC joined: %s (%s) at %s\n", ingressNode.ID, ingressNode.Name, ingressNode.Address)
}
}
}
}
// REQ-156 / P07 T5: invalidate the nodes cache. // REQ-156 / P07 T5: invalidate the nodes cache.
cacheInvalidate(cacheNodeClass) cacheInvalidate(cacheNodeClass)
if jsonOutput { if jsonOutput {
@@ -226,6 +361,46 @@ func joinProxmox(cmd *cobra.Command) error {
return nil return nil
} }
// proxmoxRemoteExecFn creates a RemoteExecFunc (func(string) ([]byte,
// error)) that runs commands on the PVE host via SSH. Used by the
// floating-IP LXC provisioning path (ProvisionIngressLXC).
func proxmoxRemoteExecFn(result *proxmox.Result, sshKeyPath, sshUser string, sshPort int) (func(string) ([]byte, error), error) {
cfg := &ssh.ClientConfig{
User: sshUser,
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
Timeout: 10 * time.Second,
}
if sshKeyPath != "" {
keyData, err := os.ReadFile(sshKeyPath)
if err != nil {
return nil, fmt.Errorf("read SSH key: %w", err)
}
signer, err := ssh.ParsePrivateKey(keyData)
if err != nil {
return nil, fmt.Errorf("parse SSH key: %w", err)
}
cfg.Auth = []ssh.AuthMethod{ssh.PublicKeys(signer)}
}
addr := result.NodeName
if sshPort != 22 {
addr = fmt.Sprintf("%s:%d", result.NodeName, sshPort)
} else {
addr = fmt.Sprintf("%s:%d", result.NodeName, sshPort)
}
client, err := ssh.Dial("tcp", addr, cfg)
if err != nil {
return nil, fmt.Errorf("ssh dial %s: %w", addr, err)
}
return func(cmd string) ([]byte, error) {
session, err := client.NewSession()
if err != nil {
return nil, err
}
defer session.Close()
return session.CombinedOutput(cmd)
}, nil
}
// joinLinux bootstraps a remote generic Linux worker via SSH and // joinLinux bootstraps a remote generic Linux worker via SSH and
// registers it as an orca node (REQ-161, P12). Uses SSH key auth // registers it as an orca node (REQ-161, P12). Uses SSH key auth
// (R-021: no passwords). // (R-021: no passwords).
@@ -514,6 +689,11 @@ func init() {
nodeJoinCmd.Flags().StringVar(&proxmoxRole, "proxmox-role", "OrcaOperator", "PVE custom role to create (config-overridable)") nodeJoinCmd.Flags().StringVar(&proxmoxRole, "proxmox-role", "OrcaOperator", "PVE custom role to create (config-overridable)")
nodeJoinCmd.Flags().StringVar(&joinHostKeyFP, "host-key-fingerprint", "", "SSH host key SHA256:base64 fingerprint (pre-pin; supersedes TOFU for --type proxmox or --type linux)") nodeJoinCmd.Flags().StringVar(&joinHostKeyFP, "host-key-fingerprint", "", "SSH host key SHA256:base64 fingerprint (pre-pin; supersedes TOFU for --type proxmox or --type linux)")
nodeJoinCmd.Flags().StringVar(&joinLXCTemplate, "lxc-template", "ubuntu-24.04", "LXC template for Proxmox (default ubuntu-24.04; alternatives: alpine-3.20, debian-12)") nodeJoinCmd.Flags().StringVar(&joinLXCTemplate, "lxc-template", "ubuntu-24.04", "LXC template for Proxmox (default ubuntu-24.04; alternatives: alpine-3.20, debian-12)")
nodeJoinCmd.Flags().StringVar(&ingressMode, "ingress-mode", "", "proxmox ingress mode: native (default, traefik in LXC) or floating-ip (ingress LXC owns floating IP)")
nodeJoinCmd.Flags().StringVar(&floatingIP, "floating-ip", "", "floating public IP for the ingress LXC (required for --ingress-mode floating-ip)")
nodeJoinCmd.Flags().StringVar(&gateway, "gateway", "", "gateway for the ingress LXC (required for --ingress-mode floating-ip)")
nodeJoinCmd.Flags().StringVar(&macAddr, "mac", "", "MAC address for the ingress LXC net0 (required for --ingress-mode floating-ip in --json mode; auto-generated in interactive mode)")
nodeJoinCmd.Flags().IntVar(&netPrefix, "net-prefix", 24, "network prefix (CIDR) for the ingress LXC IP (default 24; valid 8-32)")
nodeLeaveCmd.Flags().StringVar(&leaveID, "id", "", "node id") nodeLeaveCmd.Flags().StringVar(&leaveID, "id", "", "node id")
nodeListCmd.Flags().BoolVar(&nodeWatch, "watch", false, "stream nodes until Ctrl-C (table refresh or --json per-event)") nodeListCmd.Flags().BoolVar(&nodeWatch, "watch", false, "stream nodes until Ctrl-C (table refresh or --json per-event)")
+9 -4
View File
@@ -1,11 +1,16 @@
package cli package cli
import ( import (
"context"
"git.cloudinit.dev/coreci/orca/internal/traefik" "git.cloudinit.dev/coreci/orca/internal/traefik"
) )
var traefikVersion = traefik.DefaultVersion // ensureTraefikContainerLocal ensures the orca-traefik podman container
// is running on the local host (R-024). Replaces the v0.13
func installTraefikLocal() error { // installTraefikLocal binary+systemd installer.
return traefik.InstallLocal(traefikVersion) func ensureTraefikContainerLocal() error {
ctx, cancel := context.WithTimeout(context.Background(), 120_000_000_000) // 2min for pull
defer cancel()
return traefik.EnsureTraefikContainerLocal(ctx, version)
} }
+5 -4
View File
@@ -379,10 +379,11 @@ func performCutover(ctx context.Context, runner commandRunner, out interface{ Wr
return false, fmt.Errorf("cutover: atomic rename %s → %s: %w", tmpPath, traefikYml, err) return false, fmt.Errorf("cutover: atomic rename %s → %s: %w", tmpPath, traefikYml, err)
} }
if _, err := runner.Run(ctx, "systemctl", "restart", "traefik"); err != nil { if _, err := runner.Run(ctx, "bash", "-c", "systemctl stop orca-traefik.service 2>/dev/null; systemctl disable orca-traefik.service 2>/dev/null; rm -f /etc/systemd/system/orca-traefik.service /usr/local/bin/traefik; systemctl daemon-reload; true"); err != nil {
// Restart failed — restore from backup. slog.Warn("cutover: legacy systemd unit removal failed (non-fatal if already removed)", "err", err)
_ = cfs.Rename(backupPath, traefikYml) }
return false, fmt.Errorf("cutover: restart traefik: %w", err) if err := ensureTraefikContainerLocal(); err != nil {
slog.Warn("cutover: podman traefik container ensure failed", "err", err)
} }
nftCmd := `nft add table inet orca_redirect; nft 'add chain inet orca_redirect prerouting { type nat hook prerouting priority -100; }'; nft add rule inet orca_redirect prerouting tcp dport 443 dnat to 127.0.0.1:8443` nftCmd := `nft add table inet orca_redirect; nft 'add chain inet orca_redirect prerouting { type nat hook prerouting priority -100; }'; nft add rule inet orca_redirect prerouting tcp dport 443 dnat to 127.0.0.1:8443`
if _, err := runner.Run(ctx, "bash", "-c", nftCmd); err != nil { if _, err := runner.Run(ctx, "bash", "-c", nftCmd); err != nil {
+63 -4
View File
@@ -53,6 +53,16 @@ type NftClusterConfig struct {
// RateBurst is the per-source burst (packets) for the meter. // RateBurst is the per-source burst (packets) for the meter.
// Defaults to 200. // Defaults to 200.
RateBurst int RateBurst int
// DNATTarget is the destination IP for DNAT rules. Defaults to
// "127.0.0.1" (hybrid R-017 model — traefik on loopback). For
// Proxmox native mode where traefik runs inside an LXC, set this
// to the LXC's bridge IP so the PVE host DNATs to the LXC.
// Must be a valid IPv4 address (C-51: injection guard).
DNATTarget string
// EnableSNAT controls whether the postrouting masquerade chain
// is rendered. Defaults to true (R-024: SNAT/MASQUERADE for the
// DNAT return path). Set to false to omit the postrouting chain.
EnableSNAT bool
} }
// withDefaults returns a copy of c with zero values replaced by the // withDefaults returns a copy of c with zero values replaced by the
@@ -68,6 +78,33 @@ func (c NftClusterConfig) withDefaults() NftClusterConfig {
if out.RateBurst <= 0 { if out.RateBurst <= 0 {
out.RateBurst = 200 out.RateBurst = 200
} }
if out.DNATTarget == "" {
out.DNATTarget = "127.0.0.1"
}
// EnableSNAT defaults to true — use a sentinel: if the field was
// not explicitly set (false) and DNATTarget is the default, enable
// it. This is a Go zero-value compromise; callers who want to
// disable SNAT must set it to false explicitly after construction.
// Actually, since we want SNAT on by default, we flip it here:
// the zero value is false, but we want true. So we always set true
// unless the caller explicitly set it to a non-zero sentinel.
// Simpler: treat EnableSNAT as "opt-out" — default true, set false
// to disable. Since Go zero-value is false, we invert: use
// DisableSNAT instead. But the plan says EnableSNAT. To keep the
// plan naming and have default-true, we check if it's the zero
// value and set true:
// NOTE: since bool zero value is false, we can't distinguish "not
// set" from "set to false". So we use a pointer or invert. The
// simplest fix: the field is "EnableSNAT" and defaults to true via
// this logic: if the caller didn't set DNATTarget (still ""),
// they used a zero-value config, so enable SNAT. If they set
// DNATTarget explicitly, they should also set EnableSNAT.
// For now: always enable SNAT unless the caller sets it to false
// AND sets a non-default DNATTarget. This is pragmatic:
if !out.EnableSNAT && out.DNATTarget == "127.0.0.1" {
// Zero-value config (both fields unset) → enable SNAT.
out.EnableSNAT = true
}
return out return out
} }
@@ -94,6 +131,12 @@ func (NftEmitter) RenderNftConfig(clusterConfig NftClusterConfig) ([]File, error
if err != nil { if err != nil {
return nil, err return nil, err
} }
// C-51: validate DNATTarget as a valid IP before rendering. An
// unvalidated DNATTarget is an nft-syntax injection vector (same
// risk as TrustedProbes — the value is written raw into `dnat to`).
if net.ParseIP(cfg.DNATTarget) == nil {
return nil, fmt.Errorf("emitter/nft: DNATTarget %q is not a valid IP (C-51: ruleset injection guard)", cfg.DNATTarget)
}
content := renderNftRuleset(cfg, v4, v6) content := renderNftRuleset(cfg, v4, v6)
return []File{{Path: nftConfigPath, Content: content, Mode: "0644"}}, nil return []File{{Path: nftConfigPath, Content: content, Mode: "0644"}}, nil
} }
@@ -171,19 +214,35 @@ func renderNftRuleset(cfg NftClusterConfig, v4, v6 []string) string {
b.WriteString(" }\n") b.WriteString(" }\n")
b.WriteString("\t}\n\n") b.WriteString("\t}\n\n")
// Research Topic 2: shift input/forward priority from `filter`
// (=0) to -10 to avoid same-priority undefined evaluation order
// with pve-firewall's iptables chains (also at priority 0). This
// ensures orca's SYN-flood filter runs deterministically before
// pve-firewall on Proxmox hosts.
b.WriteString("\tchain input {\n") b.WriteString("\tchain input {\n")
b.WriteString("\t\ttype filter hook input priority filter; policy accept;\n") b.WriteString("\t\ttype filter hook input priority -10; policy accept;\n")
b.WriteString("\t\tct state invalid drop\n") b.WriteString("\t\tct state invalid drop\n")
b.WriteString("\t\tct state established,related accept\n") b.WriteString("\t\tct state established,related accept\n")
b.WriteString("\t\ttcp dport 443 tcp-flags != syn,rst,ack,fin notrack drop\n") b.WriteString("\t\ttcp dport 443 tcp-flags != syn,rst,ack,fin notrack drop\n")
b.WriteString("\t}\n\n") b.WriteString("\t}\n\n")
b.WriteString("\tchain prerouting {\n") b.WriteString("\tchain prerouting {\n")
b.WriteString("\t\ttype nat hook prerouting priority -100; policy accept;\n") b.WriteString("\t\ttype nat hook prerouting priority -100; policy accept;\n")
b.WriteString("\t\ttcp dport 443 dnat to 127.0.0.1:8443\n") b.WriteString(fmt.Sprintf("\t\ttcp dport 443 dnat to %s:8443\n", cfg.DNATTarget))
b.WriteString("\t\ttcp dport 80 dnat to 127.0.0.1:8080\n") b.WriteString(fmt.Sprintf("\t\ttcp dport 80 dnat to %s:8080\n", cfg.DNATTarget))
b.WriteString("\t}\n\n") b.WriteString("\t}\n\n")
// Research Topic 1: postrouting masquerade for the DNAT return
// path. Scoped to `ip saddr 127.0.0.0/8 oifname != "lo"` so only
// loopback-DNAT'd traffic is masqueraded (not all egress). This is
// the canonical "hairpin NAT" / "loopback DNAT return path" rule.
// Priority 100 = NF_IP_PRI_SRCNAT (standard srcnat priority).
if cfg.EnableSNAT {
b.WriteString("\tchain postrouting {\n")
b.WriteString("\t\ttype nat hook postrouting priority 100; policy accept;\n")
b.WriteString("\t\tip saddr 127.0.0.0/8 oifname != \"lo\" masquerade\n")
b.WriteString("\t}\n\n")
}
b.WriteString("\tchain forward {\n") b.WriteString("\tchain forward {\n")
b.WriteString("\t\ttype filter hook forward priority filter; policy accept;\n") b.WriteString("\t\ttype filter hook forward priority -10; policy accept;\n")
b.WriteString(fmt.Sprintf("\t\ttcp dport 443 ct state new meter { ora_rl { rate %d/second burst %d packets } } accept\n", cfg.RateLimit, cfg.RateBurst)) b.WriteString(fmt.Sprintf("\t\ttcp dport 443 ct state new meter { ora_rl { rate %d/second burst %d packets } } accept\n", cfg.RateLimit, cfg.RateBurst))
b.WriteString("\t}\n") b.WriteString("\t}\n")
b.WriteString("}\n") b.WriteString("}\n")
+90
View File
@@ -30,10 +30,13 @@ func TestNftEmitter_RenderBasic(t *testing.T) {
"127.0.0.1", "127.0.0.1",
"::1", "::1",
"chain input", "chain input",
"priority -10; policy accept;", // research Topic 2: pve-firewall coexistence
"tcp dport 443 tcp-flags != syn,rst,ack,fin notrack drop", "tcp dport 443 tcp-flags != syn,rst,ack,fin notrack drop",
"chain prerouting", "chain prerouting",
"tcp dport 443 dnat to 127.0.0.1:8443", "tcp dport 443 dnat to 127.0.0.1:8443",
"tcp dport 80 dnat to 127.0.0.1:8080", "tcp dport 80 dnat to 127.0.0.1:8080",
"chain postrouting", // research Topic 1: SNAT masquerade
"ip saddr 127.0.0.0/8 oifname != \"lo\" masquerade", // scoped to loopback DNAT return
"chain forward", "chain forward",
"rate 100/second burst 200 packets", "rate 100/second burst 200 packets",
"ora_rl", "ora_rl",
@@ -135,3 +138,90 @@ func TestNftEmitter_TrustedProbesSplitV4V6(t *testing.T) {
t.Errorf("missing ::1 in v6 set:\n%s", c) t.Errorf("missing ::1 in v6 set:\n%s", c)
} }
} }
// TestNftEmitter_CustomDNATTarget verifies the DNATTarget field
// substitutes into the dnat rules (C-51, D-262 — proxmox native mode
// DNATs to the LXC bridge IP instead of 127.0.0.1).
func TestNftEmitter_CustomDNATTarget(t *testing.T) {
files, err := (NftEmitter{}).RenderNftConfig(NftClusterConfig{DNATTarget: "10.99.0.10"})
if err != nil {
t.Fatalf("Render: %v", err)
}
c := files[0].Content
if !strings.Contains(c, "dnat to 10.99.0.10:8443") {
t.Errorf("missing custom DNAT target :8443:\n%s", c)
}
if !strings.Contains(c, "dnat to 10.99.0.10:8080") {
t.Errorf("missing custom DNAT target :8080:\n%s", c)
}
if strings.Contains(c, "127.0.0.1:8443") {
t.Errorf("default 127.0.0.1:8443 should not be present when custom DNATTarget set:\n%s", c)
}
}
// TestNftEmitter_RejectsInvalidDNATTarget verifies that an invalid
// DNATTarget is rejected (C-51: nft-syntax injection guard).
func TestNftEmitter_RejectsInvalidDNATTarget(t *testing.T) {
bad := []string{
"not-an-ip",
"127.0.0.1; flush ruleset",
"$(whoami)",
"10.0.0.0/33",
"",
}
for _, b := range bad {
// Empty string gets defaulted to 127.0.0.1, so it won't error.
// Test only non-empty invalid values.
if b == "" {
continue
}
_, err := (NftEmitter{}).RenderNftConfig(NftClusterConfig{DNATTarget: b})
if err == nil {
t.Errorf("expected error for invalid DNATTarget %q, got nil", b)
}
}
}
// TestNftEmitter_DisableSNAT verifies that EnableSNAT=false omits the
// postrouting chain entirely.
func TestNftEmitter_DisableSNAT(t *testing.T) {
// To explicitly disable SNAT, set DNATTarget to a non-default
// value AND EnableSNAT to false. The withDefaults logic only
// auto-enables SNAT for the zero-value config.
files, err := (NftEmitter{}).RenderNftConfig(NftClusterConfig{
DNATTarget: "10.99.0.10",
EnableSNAT: false,
})
if err != nil {
t.Fatalf("Render: %v", err)
}
c := files[0].Content
if strings.Contains(c, "chain postrouting") {
t.Errorf("postrouting chain should be absent when EnableSNAT=false:\n%s", c)
}
if strings.Contains(c, "masquerade") {
t.Errorf("masquerade rule should be absent when EnableSNAT=false:\n%s", c)
}
}
// TestNftEmitter_PriorityMinus10 verifies the input and forward chains
// use priority -10 (research Topic 2: pve-firewall coexistence — avoids
// same-priority undefined evaluation order with pve-firewall's
// iptables chains at priority 0).
func TestNftEmitter_PriorityMinus10(t *testing.T) {
files, _ := (NftEmitter{}).RenderNftConfig(NftClusterConfig{})
c := files[0].Content
if !strings.Contains(c, "hook input priority -10;") {
t.Errorf("input chain should use priority -10:\n%s", c)
}
if !strings.Contains(c, "hook forward priority -10;") {
t.Errorf("forward chain should use priority -10:\n%s", c)
}
// Nat chains should stay at standard priorities.
if !strings.Contains(c, "hook prerouting priority -100;") {
t.Errorf("prerouting chain should use priority -100:\n%s", c)
}
if !strings.Contains(c, "hook postrouting priority 100;") {
t.Errorf("postrouting chain should use priority 100:\n%s", c)
}
}
+14 -13
View File
@@ -46,16 +46,21 @@ type TraefikEmitter struct{}
// /etc/traefik/dynamic/orca-<spec.Name>.yaml. // /etc/traefik/dynamic/orca-<spec.Name>.yaml.
const traefikDynamicDir = "/etc/traefik/dynamic" const traefikDynamicDir = "/etc/traefik/dynamic"
// traefikRouterTLSCertResolver is the Traefik cert-resolver name that // traefikRouterTLSCertResolver is the Traefik cert-resolver name from
// the orca step-ca integration configures on the Traefik static config // v0.11. As of v0.14 (RESEARCH_v0.14 Topic 4), traefik v3.3 only
// (P10 / v0.10 wires the step-ca root into this resolver). The // supports acme/tailscale certResolvers — CA-file-based resolvers do
// dynamic-config file references it by name. // not exist. The dynamic config now emits `tls: {}` instead. Real
// mTLS via dynamic tls.certificates + clientAuth.caFiles is deferred
// to v0.15. This constant is retained for documentation.
//
// Deprecated: v0.14 removed certResolver from the dynamic config.
const traefikRouterTLSCertResolver = "orca" const traefikRouterTLSCertResolver = "orca"
// defaultTrustDomain is the SPIFFE trust domain used in the rendered // defaultTrustDomain is the SPIFFE trust domain. v0.14 removed the
// TLS stanza when the spec does not carry an explicit trust domain. // TLS domains stanza from the dynamic config (replaced with tls: {}).
// The step-ca provisioner (P10) overrides this at render time via the // Retained for documentation; will be used by v0.15 mTLS.
// node argument; for P02 the emitter renders the placeholder. //
// Deprecated: v0.14 removed TLS domains from the dynamic config.
const defaultTrustDomain = "cluster.orca.local" const defaultTrustDomain = "cluster.orca.local"
// Render renders the Traefik dynamic-config YAML for a Service // Render renders the Traefik dynamic-config YAML for a Service
@@ -175,17 +180,13 @@ func renderTraefikYAMLWeighted(spec *jobspec.WorkloadSpec, node *Node, drain boo
routerName := "orca-" + spec.Name routerName := "orca-" + spec.Name
serviceName := "orca-" + spec.Name serviceName := "orca-" + spec.Name
rule := fmt.Sprintf("PathPrefix(\"/%s\")", spec.Name) rule := fmt.Sprintf("PathPrefix(\"/%s\")", spec.Name)
trustDomain := defaultTrustDomain
b.WriteString("http:\n") b.WriteString("http:\n")
b.WriteString(" routers:\n") b.WriteString(" routers:\n")
b.WriteString(fmt.Sprintf(" %s:\n", routerName)) b.WriteString(fmt.Sprintf(" %s:\n", routerName))
b.WriteString(fmt.Sprintf(" rule: %s\n", rule)) b.WriteString(fmt.Sprintf(" rule: %s\n", rule))
b.WriteString(fmt.Sprintf(" service: %s\n", serviceName)) b.WriteString(fmt.Sprintf(" service: %s\n", serviceName))
b.WriteString(" tls:\n") b.WriteString(" tls: {}\n")
b.WriteString(fmt.Sprintf(" certResolver: %s\n", traefikRouterTLSCertResolver))
b.WriteString(" domains:\n")
b.WriteString(fmt.Sprintf(" - main: %q\n", trustDomain))
b.WriteString(" services:\n") b.WriteString(" services:\n")
b.WriteString(fmt.Sprintf(" %s:\n", serviceName)) b.WriteString(fmt.Sprintf(" %s:\n", serviceName))
b.WriteString(" loadBalancer:\n") b.WriteString(" loadBalancer:\n")
+2 -5
View File
@@ -53,11 +53,8 @@ func TestTraefikEmitter_RenderBasic(t *testing.T) {
if !strings.Contains(c, "unix:///run/orca/alloc-node-1/port-http.sock") { if !strings.Contains(c, "unix:///run/orca/alloc-node-1/port-http.sock") {
t.Errorf("content missing socket server URL\n%s", c) t.Errorf("content missing socket server URL\n%s", c)
} }
if !strings.Contains(c, "certResolver: orca") { if !strings.Contains(c, "tls: {}") {
t.Errorf("content missing 'certResolver: orca'\n%s", c) t.Errorf("content missing 'tls: {}' (v0.14: certResolver dropped, tls empty stanza)\n%s", c)
}
if !strings.Contains(c, "domains:") {
t.Errorf("content missing TLS domains\n%s", c)
} }
if !strings.Contains(c, "healthCheck:") { if !strings.Contains(c, "healthCheck:") {
t.Errorf("content missing 'healthCheck:'\n%s", c) t.Errorf("content missing 'healthCheck:'\n%s", c)
+240
View File
@@ -0,0 +1,240 @@
// Package ingress implements the R-024 ingress bootstrap: nft DNAT
// + SNAT/MASQUERADE + traefik podman container + step-ca root CA on
// every orca-managed node. The bootstrap is idempotent and non-fatal
// on each step (offline host tolerance — same as v0.13 traefik install).
//
// BootstrapLocalIngress runs on the lead (during `orca init`).
// BootstrapRemoteIngress runs on workers (during `orca node join`).
package ingress
import (
"context"
"crypto/rand"
"encoding/hex"
"fmt"
"log/slog"
"os"
"os/exec"
"path/filepath"
"strings"
"git.cloudinit.dev/coreci/orca/internal/certpaths"
"git.cloudinit.dev/coreci/orca/internal/emitter"
"git.cloudinit.dev/coreci/orca/internal/traefik"
)
// BootstrapLocalIngress ensures the complete ingress stack is running
// on the local host (lead node). It is called from `orca init` after
// EnsureTraefikContainerLocal.
//
// Steps (each non-fatal — logs a warning and continues):
// 1. mkdir -p /etc/traefik/dynamic /etc/orca
// 2. Push cluster root CA to /etc/orca/step-ca-root.crt (C-60:
// certpaths.CACertPath(), not CAPath)
// 3. Render static config via emitter.RenderTraefikStaticConfig to
// /etc/traefik/traefik.yml (preserves traefik-on-public-ip opt-out,
// C-58)
// 4. Render orca.nft via emitter.NftEmitter.RenderNftConfig + write
// to /etc/nftables.d/orca.nft
// 5. Pre-create nft table (C-55: avoids flush-table error on first
// apply)
// 6. Apply: nft -f /etc/nftables.d/orca.nft
func BootstrapLocalIngress(ctx context.Context, version string) error {
var errs []error
log := slog.Default()
// Step 1: ensure directories.
for _, dir := range []string{"/etc/traefik/dynamic", "/etc/orca"} {
if err := os.MkdirAll(dir, 0o755); err != nil {
log.Warn("ingress: mkdir failed", "dir", dir, "err", err)
errs = append(errs, fmt.Errorf("mkdir %s: %w", dir, err))
}
}
// Step 2: push cluster root CA (C-60: CACertPath, not CAPath).
caPath := certpaths.CACertPath()
if caData, err := os.ReadFile(caPath); err == nil {
if err := os.WriteFile("/etc/orca/step-ca-root.crt", caData, 0o644); err != nil {
log.Warn("ingress: step-ca root CA write failed", "err", err)
errs = append(errs, fmt.Errorf("write step-ca-root.crt: %w", err))
}
} else {
// CA may not exist yet (fresh init before step-ca). Write a
// placeholder so the podman volume mount doesn't fail.
_ = os.WriteFile("/etc/orca/step-ca-root.crt", []byte{}, 0o644)
log.Warn("ingress: step-ca root CA not found, wrote placeholder", "path", caPath)
}
// Step 3: render + write static config (C-58).
staticFiles, err := emitter.TraefikEmitter{}.RenderTraefikStaticConfig(emitter.TraefikStaticOpts{})
if err != nil {
log.Warn("ingress: render traefik static config failed", "err", err)
errs = append(errs, fmt.Errorf("render traefik static: %w", err))
} else {
for _, f := range staticFiles {
if err := os.WriteFile(f.Path, []byte(f.Content), 0o644); err != nil {
log.Warn("ingress: write traefik static config failed", "path", f.Path, "err", err)
errs = append(errs, fmt.Errorf("write %s: %w", f.Path, err))
}
}
}
// Step 4: render + write nft ruleset.
nftFiles, err := emitter.NftEmitter{}.RenderNftConfig(emitter.NftClusterConfig{})
if err != nil {
log.Warn("ingress: render nft config failed", "err", err)
errs = append(errs, fmt.Errorf("render nft: %w", err))
} else {
for _, f := range nftFiles {
dir := filepath.Dir(f.Path)
_ = os.MkdirAll(dir, 0o755)
if err := os.WriteFile(f.Path, []byte(f.Content), 0o644); err != nil {
log.Warn("ingress: write nft config failed", "path", f.Path, "err", err)
errs = append(errs, fmt.Errorf("write %s: %w", f.Path, err))
}
}
// Step 5: pre-create nft table (C-55: flush table on non-existent
// table errors — pre-create avoids the first-apply failure).
_ = exec.CommandContext(ctx, "nft", "add", "table", "inet", "orca-ingress").Run()
// Step 6: apply nft ruleset.
if out, err := exec.CommandContext(ctx, "nft", "-f", nftConfigPath).CombinedOutput(); err != nil {
log.Warn("ingress: nft apply failed", "err", err, "output", string(out))
errs = append(errs, fmt.Errorf("nft -f: %w (output: %s)", err, string(out)))
}
}
// Ensure the podman traefik container is running (Step 7 — C-50:
// installs podman if absent).
if err := traefik.EnsureTraefikContainerLocal(ctx, version); err != nil {
log.Warn("ingress: ensure traefik container failed", "err", err)
errs = append(errs, fmt.Errorf("ensure traefik container: %w", err))
}
if len(errs) > 0 {
return fmt.Errorf("ingress bootstrap: %d errors (first: %w)", len(errs), errs[0])
}
return nil
}
// nftConfigPath mirrors the emitter constant.
const nftConfigPath = "/etc/nftables.d/orca.nft"
// RemoteExecFunc runs a command on a remote host and returns combined
// output. Same signature as traefik.RemoteExecFunc.
type RemoteExecFunc func(cmd string) ([]byte, error)
// BootstrapRemoteIngress ensures the complete ingress stack is running
// on a remote host (linux worker). It is called from
// `orca node join --type linux` after the user setup.
//
// Steps (each non-fatal — logs a warning and continues):
// 1. mkdir -p /etc/traefik/dynamic /etc/orca (remote)
// 2. Push step-ca root CA to remote /etc/orca/step-ca-root.crt
// (C-60: certpaths.CACertPath)
// 3. Render + write static config to remote /etc/traefik/traefik.yml
// (C-58: preserves traefik-on-public-ip opt-out)
// 4. Render + write nft ruleset to remote /etc/nftables.d/orca.nft
// 5. Pre-create nft table (C-55: avoids first-apply flush-table error)
// 6. Apply: nft -f (remote)
// 7. Ensure podman traefik container running (remote)
func BootstrapRemoteIngress(ctx context.Context, version string, execFn RemoteExecFunc) error {
var errs []error
log := slog.Default()
// Step 1: ensure directories.
if _, err := execFn("mkdir -p /etc/traefik/dynamic /etc/orca"); err != nil {
log.Warn("ingress: remote mkdir failed", "err", err)
errs = append(errs, fmt.Errorf("remote mkdir: %w", err))
}
// Step 2: push step-ca root CA (C-60: CACertPath, not CAPath).
if caData, err := os.ReadFile(certpaths.CACertPath()); err == nil {
if err := remoteWriteFile(execFn, "/etc/orca/step-ca-root.crt", caData, "0644"); err != nil {
log.Warn("ingress: remote step-ca CA write failed", "err", err)
errs = append(errs, fmt.Errorf("remote write step-ca-root.crt: %w", err))
}
} else {
// Write a placeholder so the podman volume mount doesn't fail.
_ = remoteWriteFile(execFn, "/etc/orca/step-ca-root.crt", []byte{}, "0644")
log.Warn("ingress: step-ca root CA not found locally, wrote remote placeholder")
}
// Step 3: render + write static config (C-58).
staticFiles, err := emitter.TraefikEmitter{}.RenderTraefikStaticConfig(emitter.TraefikStaticOpts{})
if err != nil {
log.Warn("ingress: render traefik static config failed", "err", err)
errs = append(errs, fmt.Errorf("render traefik static: %w", err))
} else {
for _, f := range staticFiles {
_, _ = execFn(fmt.Sprintf("mkdir -p %s", filepath.Dir(f.Path)))
if err := remoteWriteFile(execFn, f.Path, []byte(f.Content), f.Mode); err != nil {
log.Warn("ingress: remote write traefik static config failed", "path", f.Path, "err", err)
errs = append(errs, fmt.Errorf("remote write %s: %w", f.Path, err))
}
}
}
// Step 4: render + write nft ruleset.
nftFiles, err := emitter.NftEmitter{}.RenderNftConfig(emitter.NftClusterConfig{})
if err != nil {
log.Warn("ingress: render nft config failed", "err", err)
errs = append(errs, fmt.Errorf("render nft: %w", err))
} else {
for _, f := range nftFiles {
_, _ = execFn(fmt.Sprintf("mkdir -p %s", filepath.Dir(f.Path)))
if err := remoteWriteFile(execFn, f.Path, []byte(f.Content), f.Mode); err != nil {
log.Warn("ingress: remote write nft config failed", "path", f.Path, "err", err)
errs = append(errs, fmt.Errorf("remote write %s: %w", f.Path, err))
}
}
// Step 5: pre-create nft table (C-55).
_, _ = execFn("nft add table inet orca-ingress 2>/dev/null || true")
// Step 6: apply nft ruleset.
if out, err := execFn("nft -f /etc/nftables.d/orca.nft 2>&1"); err != nil {
log.Warn("ingress: remote nft apply failed", "err", err, "output", string(out))
errs = append(errs, fmt.Errorf("remote nft -f: %w (output: %s)", err, string(out)))
}
}
// Step 7: ensure podman traefik container (C-50).
traefikExecFn := traefik.RemoteExecFunc(execFn)
if err := traefik.EnsureTraefikContainerRemote(ctx, version, traefikExecFn); err != nil {
log.Warn("ingress: remote ensure traefik container failed", "err", err)
errs = append(errs, fmt.Errorf("remote ensure traefik container: %w", err))
}
if len(errs) > 0 {
return fmt.Errorf("remote ingress bootstrap: %d errors (first: %w)", len(errs), errs[0])
}
return nil
}
// remoteWriteFile writes content to a remote path via a heredoc
// (same pattern as sshpush.idempotency.writeFile). The heredoc
// delimiter is a random hex string verified absent from the content
// (F9 injection guard).
func remoteWriteFile(execFn RemoteExecFunc, path string, content []byte, mode string) error {
// Generate a random delimiter unlikely to be in the content.
delim := "EOF_"
for {
b := make([]byte, 8)
if _, err := rand.Read(b); err != nil {
return fmt.Errorf("rand: %w", err)
}
delim = "EOF_" + hex.EncodeToString(b)
if !strings.Contains(string(content), delim) {
break
}
}
dir := filepath.Dir(path)
cmd := fmt.Sprintf("mkdir -p %s && cat > %s <<'%s'\n%s\n%s\nchmod %s %s",
dir, path, delim, string(content), delim, mode, path)
if out, err := execFn(cmd); err != nil {
return fmt.Errorf("remote write %s: %w (output: %s)", path, err, string(out))
}
return nil
}
+9 -6
View File
@@ -32,9 +32,9 @@ import (
"golang.org/x/crypto/ssh" "golang.org/x/crypto/ssh"
"git.cloudinit.dev/coreci/orca/internal/certpaths" "git.cloudinit.dev/coreci/orca/internal/certpaths"
"git.cloudinit.dev/coreci/orca/internal/ingress"
"git.cloudinit.dev/coreci/orca/internal/proxmox" "git.cloudinit.dev/coreci/orca/internal/proxmox"
"git.cloudinit.dev/coreci/orca/internal/security" "git.cloudinit.dev/coreci/orca/internal/security"
"git.cloudinit.dev/coreci/orca/internal/traefik"
) )
// DefaultSSHUser is the default SSH username for the initial connection. // DefaultSSHUser is the default SSH username for the initial connection.
@@ -157,8 +157,10 @@ func BootstrapLinux(ctx context.Context, opts Options) (*Result, error) {
} }
opts.Logger.Info("linux bootstrap: user created", "user", opts.OrcaUser) opts.Logger.Info("linux bootstrap: user created", "user", opts.OrcaUser)
// Step 4d: Install Traefik on the remote host (REQ-165, Phase B). // Step 4d: Ensure complete ingress stack on the remote host (R-024).
// Traefik is the data-plane ingress; SSH is control plane only. // Renders+applies nft DNAT/SNAT, pushes step-ca root CA, renders+
// writes traefik static config, ensures podman container running.
// All non-fatal (offline host tolerance).
sshExecFn := func(cmd string) ([]byte, error) { sshExecFn := func(cmd string) ([]byte, error) {
session, err := client.NewSession() session, err := client.NewSession()
if err != nil { if err != nil {
@@ -167,8 +169,10 @@ func BootstrapLinux(ctx context.Context, opts Options) (*Result, error) {
defer session.Close() defer session.Close()
return session.CombinedOutput(cmd) return session.CombinedOutput(cmd)
} }
if err := traefik.InstallRemote("", sshExecFn); err != nil { ctx, cancelIngress := context.WithTimeout(ctx, 120*time.Second)
opts.Logger.Warn("linux bootstrap: traefik install failed", "err", err) defer cancelIngress()
if err := ingress.BootstrapRemoteIngress(ctx, "", ingress.RemoteExecFunc(sshExecFn)); err != nil {
opts.Logger.Warn("linux bootstrap: ingress bootstrap failed", "err", err)
} }
// Step 5: Create the drift-events directory. // Step 5: Create the drift-events directory.
@@ -226,5 +230,4 @@ func pinnedHostKeyCallback(expectedSHA256Base64 string, capturedKey *ssh.PublicK
return cb, nil return cb, nil
} }
var _ = security.WriteAtomic var _ = security.WriteAtomic
+5
View File
@@ -39,4 +39,9 @@ type Node struct {
// OS is the auto-detected OS identifier from /etc/os-release ID= // OS is the auto-detected OS identifier from /etc/os-release ID=
// (ubuntu|debian|alpine|pve|linux). Empty for pre-0006 rows. // (ubuntu|debian|alpine|pve|linux). Empty for pre-0006 rows.
OS string `json:"os,omitempty"` OS string `json:"os,omitempty"`
// IngressMode is the ingress configuration for the node (R-024,
// v0.14). Values: "" (legacy/default for linux/localhost),
// "native" (proxmox native — traefik in LXC), "floating-ip"
// (proxmox floating-IP — ingress LXC owns the floating IP).
IngressMode string `json:"ingress_mode,omitempty"`
} }
+133 -9
View File
@@ -37,6 +37,7 @@ import (
"golang.org/x/crypto/ssh/knownhosts" "golang.org/x/crypto/ssh/knownhosts"
"git.cloudinit.dev/coreci/orca/internal/certpaths" "git.cloudinit.dev/coreci/orca/internal/certpaths"
"git.cloudinit.dev/coreci/orca/internal/emitter"
"git.cloudinit.dev/coreci/orca/internal/security" "git.cloudinit.dev/coreci/orca/internal/security"
"git.cloudinit.dev/coreci/orca/internal/traefik" "git.cloudinit.dev/coreci/orca/internal/traefik"
) )
@@ -86,6 +87,13 @@ type Options struct {
// LXCTemplate is the LXC template to download during bootstrap // LXCTemplate is the LXC template to download during bootstrap
// (default "ubuntu-24.04"; alternatives: "alpine-3.20", "debian-12"). // (default "ubuntu-24.04"; alternatives: "alpine-3.20", "debian-12").
LXCTemplate string LXCTemplate string
// IngressMode is the proxmox ingress mode (R-024, v0.14).
// "native" (default): traefik runs in an unprivileged LXC with
// nesting=1,keyctl=1,fuse=1 on the PVE host. nft on the PVE host
// DNATs to the LXC bridge IP.
// "floating-ip": a separate ingress LXC owns the floating IP;
// nft runs inside that LXC. See ProvisionIngressLXC (P6).
IngressMode string
} }
// Result is the outcome of a successful bootstrap. // Result is the outcome of a successful bootstrap.
@@ -247,21 +255,24 @@ func BootstrapProxmox(ctx context.Context, opts Options) (*Result, error) {
return nil, fmt.Errorf("validate sudoers: %w", err) return nil, fmt.Errorf("validate sudoers: %w", err)
} }
// Step 9a: Install Traefik on the Proxmox host (REQ-165, Phase B). // Step 9a: Proxmox native ingress mode (R-024, REQ-175).
// Traefik runs on the PVE OS as the data-plane ingress; SSH is // Create an unprivileged LXC with nesting=1,keyctl=1,fuse=1 (research
// control plane only. Idempotent: skips if binary already exists. // Topic 3), install podman inside it, and run the orca-traefik
if err := traefik.InstallRemote("", runRemote); err != nil { // container. nft on the PVE host DNATs to the LXC bridge IP.
log.Warn("proxmox.traefik_install_failed", "err", err) // Default mode is "native"; floating-ip mode is handled separately
} // (P6 — ProvisionIngressLXC).
// Step 9b: Download default LXC template (REQ-167, Phase C).
// Default: ubuntu-24.04. Configurable via --lxc-template.
template := opts.LXCTemplate template := opts.LXCTemplate
if template == "" { if template == "" {
template = "ubuntu-24.04" template = "ubuntu-24.04"
} }
_, _ = runRemote(fmt.Sprintf("pveam download local %s 2>/dev/null || true", shellQuote(template))) _, _ = runRemote(fmt.Sprintf("pveam download local %s 2>/dev/null || true", shellQuote(template)))
if opts.IngressMode != "floating-ip" {
if err := provisionNativeIngressLXC(ctx, runRemote, template, log); err != nil {
log.Warn("proxmox.native_ingress_lxc_failed", "err", err)
}
}
log.Info("proxmox.bootstrap_ok", log.Info("proxmox.bootstrap_ok",
slog.String("event", "proxmox.bootstrap_ok"), slog.String("event", "proxmox.bootstrap_ok"),
slog.String("host", opts.Host), slog.String("host", opts.Host),
@@ -276,6 +287,119 @@ func BootstrapProxmox(ctx context.Context, opts Options) (*Result, error) {
}, nil }, nil
} }
// provisionNativeIngressLXC creates an unprivileged LXC with
// nesting=1,keyctl=1,fuse=1 (research Topic 3), installs podman inside
// it, runs the orca-traefik container, and applies nft DNAT on the PVE
// host targeting the LXC's bridge IP (R-024, REQ-175).
//
// The LXC is named "orca-traefik" and uses a deterministic VMID derived
// from the host. It is idempotent: if the LXC already exists, it is
// not re-created (C-53: apt-get install is skipped if podman present).
func provisionNativeIngressLXC(ctx context.Context, runRemote func(string) ([]byte, error), template string, log *slog.Logger) error {
// Deterministic VMID for the native ingress LXC.
// Use a fixed VMID in the 200-299 range (Proxmox convention for CTs).
const vmid = "200"
const lxcName = "orca-traefik"
// Check if the LXC already exists.
existOut, _ := runRemote(fmt.Sprintf("pct status %s 2>/dev/null || echo absent", vmid))
existStr := strings.TrimSpace(string(existOut))
if existStr == "absent" {
// Create the LXC (research Topic 3: nesting=1,keyctl=1,fuse=1).
log.Info("proxmox.creating_native_ingress_lxc", "vmid", vmid, "name", lxcName)
createCmd := fmt.Sprintf(
"pct create %s local:vztmpl/%s --hostname %s --unprivileged 1 --features nesting=1,keyctl=1,fuse=1 --onboot 1 --memory 2048 --swap 0 --rootfs local:8 2>&1",
vmid, shellQuote(template), lxcName,
)
if out, err := runRemote(createCmd); err != nil {
return fmt.Errorf("pct create native ingress LXC: %w (output: %s)", err, string(out))
}
if out, err := runRemote(fmt.Sprintf("pct start %s", vmid)); err != nil {
return fmt.Errorf("pct start native ingress LXC: %w (output: %s)", err, string(out))
}
}
// Wait for LXC network (retry for up to 60s).
lxcIP := ""
for i := 0; i < 12; i++ {
ipOut, _ := runRemote(fmt.Sprintf("pct exec %s -- hostname -I 2>/dev/null", vmid))
ipStr := strings.TrimSpace(string(ipOut))
if ipStr != "" {
fields := strings.Fields(ipStr)
if len(fields) > 0 {
lxcIP = fields[0]
break
}
}
time.Sleep(5 * time.Second)
}
if lxcIP == "" {
return fmt.Errorf("native ingress LXC: could not discover IP after 60s")
}
log.Info("proxmox.native_ingress_lxc_ip", "vmid", vmid, "ip", lxcIP)
// Install podman inside the LXC (C-53: idempotent — check first).
_, _ = runRemote(fmt.Sprintf(
"pct exec %s -- bash -c 'command -v podman >/dev/null 2>&1 || (apt-get update -qq && apt-get install -y -qq podman conmon crun fuse-overlayfs nftables 2>&1)' 2>&1",
vmid,
))
// Enable podman-restart.service inside the LXC (research Topic 6).
_, _ = runRemote(fmt.Sprintf("pct exec %s -- systemctl enable --now podman-restart.service 2>/dev/null", vmid))
// Push step-ca root CA into the LXC (placeholder if absent locally).
caPath := certpaths.CACertPath()
caData, caErr := os.ReadFile(caPath)
if caErr != nil {
caData = []byte{}
}
// Write CA via pct exec heredoc.
caDelim := "EOF_CA"
_, _ = runRemote(fmt.Sprintf(
"pct exec %s -- bash -c 'mkdir -p /etc/orca && cat > /etc/orca/step-ca-root.crt <<%s\\n%s\\n%s'",
vmid, caDelim, string(caData), caDelim,
))
// Render + write traefik static config inside the LXC.
staticFiles, err := emitter.TraefikEmitter{}.RenderTraefikStaticConfig(emitter.TraefikStaticOpts{})
if err == nil {
for _, f := range staticFiles {
delim := "EOF_TF"
_, _ = runRemote(fmt.Sprintf(
"pct exec %s -- bash -c 'mkdir -p /etc/traefik/dynamic && cat > %s <<%s\\n%s\\n%s'",
vmid, f.Path, delim, f.Content, delim,
))
}
}
// Ensure podman orca-traefik container inside the LXC.
traefikExecFn := func(cmd string) ([]byte, error) {
return runRemote(fmt.Sprintf("pct exec %s -- bash -c %s 2>&1", vmid, shellQuote(cmd)))
}
if err := traefik.EnsureTraefikContainerRemote(ctx, "", traefikExecFn); err != nil {
log.Warn("proxmox.native_ingress_lxc_traefik_failed", "err", err)
}
// Render + apply nft on the PVE host with DNATTarget = LXC IP.
nftFiles, err := emitter.NftEmitter{}.RenderNftConfig(emitter.NftClusterConfig{
DNATTarget: lxcIP,
})
if err == nil {
for _, f := range nftFiles {
nftDelim := "EOF_NF"
_, _ = runRemote(fmt.Sprintf("mkdir -p /etc/nftables.d && cat > %s <<%s\\n%s\\n%s",
f.Path, nftDelim, f.Content, nftDelim))
}
_, _ = runRemote("nft add table inet orca-ingress 2>/dev/null || true")
if out, err := runRemote("nft -f /etc/nftables.d/orca.nft 2>&1"); err != nil {
log.Warn("proxmox.native_ingress_nft_apply_failed", "err", err, "output", string(out))
}
}
log.Info("proxmox.native_ingress_lxc_ok", "vmid", vmid, "ip", lxcIP)
return nil
}
// sshDialer is the dialer used by BootstrapProxmox. It's a package-level // sshDialer is the dialer used by BootstrapProxmox. It's a package-level
// variable so tests can override it with a fake SSH server. // variable so tests can override it with a fake SSH server.
var sshDialer sshDialerType = defaultSSHDialer{} var sshDialer sshDialerType = defaultSSHDialer{}
+171
View File
@@ -0,0 +1,171 @@
package proxmox
import (
"context"
"crypto/rand"
"fmt"
"log/slog"
"os"
"strings"
"time"
"git.cloudinit.dev/coreci/orca/internal/certpaths"
"git.cloudinit.dev/coreci/orca/internal/emitter"
"git.cloudinit.dev/coreci/orca/internal/traefik"
)
// FloatingIPOptions carries the parameters for provisioning a
// floating-IP ingress LXC (R-024, REQ-176).
type FloatingIPOptions struct {
// FloatingIP is the public IP assigned to the LXC's eth0.
FloatingIP string
// Gateway is the default gateway for the LXC.
Gateway string
// MAC is the MAC address for the LXC's net0 interface.
MAC string
// NetPrefix is the CIDR prefix for the floating IP (8-32).
NetPrefix int
// LXCTemplate is the LXC template (default "ubuntu-24.04").
LXCTemplate string
// VMID is the LXC container ID (default "201" for the ingress LXC).
VMID string
}
// ProvisionIngressLXC creates an Ubuntu LXC named "ingress" that owns
// the floating IP, installs podman + orca-traefik inside it, applies nft
// DNAT+SNAT inside the LXC, and returns the LXC's IP for node
// registration (R-024, REQ-176).
//
// The LXC is created with:
//
// --unprivileged 1 --features nesting=1,keyctl=1,fuse=1
// --net0 name=eth0,bridge=vmbr0,hwaddr=<mac>,ip=<floating-ip>/<prefix>,gw=<gateway>
// --onboot 1
//
// Inside the LXC, the complete ingress stack is set up: podman
// installed, traefik static config written, nft DNAT:443→127.0.0.1:8443
// + postrouting masquerade applied, orca-traefik podman container
// running with --network host.
//
// Idempotent: if the LXC already exists, it is not re-created (C-53).
func ProvisionIngressLXC(ctx context.Context, runRemote func(string) ([]byte, error), opts FloatingIPOptions, log *slog.Logger) error {
template := opts.LXCTemplate
if template == "" {
template = "ubuntu-24.04"
}
vmid := opts.VMID
if vmid == "" {
vmid = "201"
}
if opts.NetPrefix == 0 {
opts.NetPrefix = 24
}
// Validate required fields.
if opts.FloatingIP == "" || opts.Gateway == "" || opts.MAC == "" {
return fmt.Errorf("ingress_lxc: floating-ip, gateway, and mac are required")
}
// Ensure the template is downloaded.
_, _ = runRemote(fmt.Sprintf("pveam download local %s 2>/dev/null || true", shellQuote(template)))
// Check if the LXC already exists (idempotent — C-53).
existOut, _ := runRemote(fmt.Sprintf("pct status %s 2>/dev/null || echo absent", vmid))
existStr := strings.TrimSpace(string(existOut))
if existStr == "absent" {
log.Info("ingress_lxc.creating", "vmid", vmid, "hostname", "ingress", "ip", opts.FloatingIP)
net0 := fmt.Sprintf("name=eth0,bridge=vmbr0,hwaddr=%s,ip=%s/%d,gw=%s",
opts.MAC, opts.FloatingIP, opts.NetPrefix, opts.Gateway)
createCmd := fmt.Sprintf(
"pct create %s local:vztmpl/%s --hostname ingress --unprivileged 1 --features nesting=1,keyctl=1,fuse=1 --net0 %s --onboot 1 --memory 2048 --swap 0 --rootfs local:8 2>&1",
vmid, shellQuote(template), net0,
)
if out, err := runRemote(createCmd); err != nil {
return fmt.Errorf("pct create ingress LXC: %w (output: %s)", err, string(out))
}
if out, err := runRemote(fmt.Sprintf("pct start %s", vmid)); err != nil {
return fmt.Errorf("pct start ingress LXC: %w (output: %s)", err, string(out))
}
}
// Wait for LXC network (retry for up to 60s).
for i := 0; i < 12; i++ {
ipOut, _ := runRemote(fmt.Sprintf("pct exec %s -- hostname -I 2>/dev/null", vmid))
ipStr := strings.TrimSpace(string(ipOut))
if ipStr != "" {
break
}
time.Sleep(5 * time.Second)
}
log.Info("ingress_lxc.network_ready", "vmid", vmid, "ip", opts.FloatingIP)
// Install podman inside the LXC (C-53: idempotent).
_, _ = runRemote(fmt.Sprintf(
"pct exec %s -- bash -c 'command -v podman >/dev/null 2>&1 || (apt-get update -qq && apt-get install -y -qq podman conmon crun fuse-overlayfs nftables 2>&1)' 2>&1",
vmid,
))
// Enable podman-restart.service inside the LXC (research Topic 6).
_, _ = runRemote(fmt.Sprintf("pct exec %s -- systemctl enable --now podman-restart.service 2>/dev/null", vmid))
// Push step-ca root CA into the LXC (C-60: CACertPath).
caPath := certpaths.CACertPath()
caData, caErr := os.ReadFile(caPath)
if caErr != nil {
caData = []byte{}
}
caDelim := "EOF_CA"
_, _ = runRemote(fmt.Sprintf(
"pct exec %s -- bash -c 'mkdir -p /etc/orca && cat > /etc/orca/step-ca-root.crt <<%s\\n%s\\n%s'",
vmid, caDelim, string(caData), caDelim,
))
// Render + write traefik static config inside the LXC (C-58).
staticFiles, err := emitter.TraefikEmitter{}.RenderTraefikStaticConfig(emitter.TraefikStaticOpts{})
if err == nil {
for _, f := range staticFiles {
delim := "EOF_TF"
_, _ = runRemote(fmt.Sprintf(
"pct exec %s -- bash -c 'mkdir -p /etc/traefik/dynamic && cat > %s <<%s\\n%s\\n%s'",
vmid, f.Path, delim, f.Content, delim,
))
}
}
// Render + apply nft DNAT+SNAT INSIDE the LXC (DNATTarget =
// 127.0.0.1 — traefik runs with --network host inside the LXC).
nftFiles, err := emitter.NftEmitter{}.RenderNftConfig(emitter.NftClusterConfig{})
if err == nil {
for _, f := range nftFiles {
delim := "EOF_NF"
_, _ = runRemote(fmt.Sprintf(
"pct exec %s -- bash -c 'mkdir -p /etc/nftables.d && cat > %s <<%s\\n%s\\n%s'",
vmid, f.Path, delim, f.Content, delim,
))
}
_, _ = runRemote(fmt.Sprintf("pct exec %s -- nft add table inet orca-ingress 2>/dev/null || true", vmid))
_, _ = runRemote(fmt.Sprintf("pct exec %s -- nft -f /etc/nftables.d/orca.nft 2>&1", vmid))
}
// Ensure podman orca-traefik container inside the LXC.
traefikExecFn := func(cmd string) ([]byte, error) {
return runRemote(fmt.Sprintf("pct exec %s -- bash -c %s 2>&1", vmid, shellQuote(cmd)))
}
if err := traefik.EnsureTraefikContainerRemote(ctx, "", traefikExecFn); err != nil {
log.Warn("ingress_lxc.traefik_failed", "err", err)
}
log.Info("ingress_lxc.provisioned", "vmid", vmid, "ip", opts.FloatingIP)
return nil
}
// GenerateRandomMAC generates a random locally-administered MAC address
// (02:XX:XX:XX:XX:XX) for use as the LXC net0 hardware address when the
// operator does not provide one (D-261).
func GenerateRandomMAC() (string, error) {
b := make([]byte, 5)
if _, err := rand.Read(b); err != nil {
return "", fmt.Errorf("generate MAC: %w", err)
}
return fmt.Sprintf("02:%02x:%02x:%02x:%02x:%02x", b[0], b[1], b[2], b[3], b[4]), nil
}
@@ -0,0 +1,6 @@
-- REQ-175 / R-024: add ingress_mode column to nodes.
-- Values: '' (legacy/default), 'native' (proxmox native — traefik
-- in LXC), 'floating-ip' (proxmox floating-IP — ingress LXC owns
-- the floating IP). Defaults to empty string for backward
-- compatibility with pre-v0.14 nodes.
ALTER TABLE nodes ADD COLUMN ingress_mode TEXT NOT NULL DEFAULT '';
+202 -52
View File
@@ -1,78 +1,228 @@
package traefik package traefik
import ( import (
"context"
"fmt" "fmt"
"os/exec" "os/exec"
"strings" "strings"
) )
// DefaultVersion is the traefik version tag for the orca-traefik image.
const DefaultVersion = "v3.3.0" const DefaultVersion = "v3.3.0"
func downloadURL(version string) string { // DefaultImage is the full image reference for the orca-traefik container.
return fmt.Sprintf("https://github.com/traefik/traefik/releases/download/%s/traefik_%s_linux_amd64.tar.gz", version, version) // The tag is resolved at runtime from the orca version (or "latest" for
} // dev builds). The image is built from Dockerfile.traefik and published
// per release (REQ-171).
const DefaultImage = "git.cloudinit.dev/coreci/orca-traefik"
func InstallLocal(version string) error { // ContainerName is the podman container name for the traefik data plane.
if version == "" { const ContainerName = "orca-traefik"
version = DefaultVersion
}
if _, err := exec.LookPath("traefik"); err == nil {
ensureDirs()
return nil
}
url := downloadURL(version)
cmd := exec.Command("bash", "-c",
fmt.Sprintf(`curl -fsSL %s | tar -xzf - -C /usr/local/bin/ traefik && chmod +x /usr/local/bin/traefik`, url))
if out, err := cmd.CombinedOutput(); err != nil {
return fmt.Errorf("download traefik %s: %w (output: %s)", version, err, string(out))
}
ensureDirs()
writeSystemdUnit()
_ = exec.Command("systemctl", "daemon-reload").Run()
_ = exec.Command("systemctl", "enable", "--now", "orca-traefik").Run()
return nil
}
// RemoteExecFunc runs a command on a remote host and returns combined
// output. It is the same signature used by the v0.13 binary installer
// and by the proxmox/linux bootstrap SSH sessions.
type RemoteExecFunc func(cmd string) ([]byte, error) type RemoteExecFunc func(cmd string) ([]byte, error)
func InstallRemote(version string, execFn RemoteExecFunc) error { // ImageRef returns the full image:tag reference for the orca-traefik
if version == "" { // container. If version is empty or "dev"/"0.1.0-dev", it falls back to
version = DefaultVersion // "latest" (dev builds don't have a published tag).
func ImageRef(version string) string {
tag := version
if tag == "" || tag == "dev" || tag == "0.1.0-dev" || strings.HasSuffix(tag, "-dev") {
return fmt.Sprintf("%s:latest", DefaultImage)
} }
if out, err := execFn("command -v traefik"); err == nil && len(strings.TrimSpace(string(out))) > 0 { tag = strings.TrimPrefix(tag, "v")
_, _ = execFn("mkdir -p /etc/traefik/dynamic") return fmt.Sprintf("%s:v%s", DefaultImage, tag)
}
// podmanRunArgs returns the podman run arguments for the traefik
// container. The container uses --network host so traefik binds
// 127.0.0.1:8080/8443 directly on the host (or LXC) loopback. nft
// DNATs public :443/:80 to those loopback ports (R-017/R-024).
//
// Volume mounts (no SELinux :Z flag — research finding Topic 7):
// - /etc/traefik/traefik.yml:ro — static config (overrides baked default; C-58)
// - /etc/traefik/dynamic:ro — dynamic config (orca writes atomically via SSH-push)
// - /etc/orca/step-ca-root.crt:ro — step-ca root CA (for future mTLS; v0.14 uses tls:{})
func podmanRunArgs(imageRef string) []string {
return []string{
"run", "-d",
"--name", ContainerName,
"--restart=unless-stopped",
"--network", "host",
"-v", "/etc/traefik/traefik.yml:/etc/traefik/traefik.yml:ro",
"-v", "/etc/traefik/dynamic:/etc/traefik/dynamic:ro",
"-v", "/etc/orca/step-ca-root.crt:/etc/orca/step-ca-root.crt:ro",
imageRef,
}
}
// EnsureTraefikContainerLocal ensures the orca-traefik podman container
// is running on the local host. It is idempotent:
// 1. If the container is running → no-op.
// 2. If the container exists but is stopped → start it.
// 3. If the container does not exist → pull the image + run it.
//
// C-50: if podman is not installed, attempts apt-get install. If that
// fails, returns an error with install instructions.
//
// C-57: if a legacy v0.13 systemd service exists (orca-traefik.service),
// it is stopped, disabled, and removed before starting the container.
func EnsureTraefikContainerLocal(ctx context.Context, version string) error {
imageRef := ImageRef(version)
if err := ensurePodmanLocal(ctx); err != nil {
return err
}
if err := removeLegacySystemdUnitLocal(ctx); err != nil {
// Non-fatal: legacy unit may not exist on fresh installs.
_ = err
}
if err := ensureDirsLocal(); err != nil {
return fmt.Errorf("traefik: ensure dirs: %w", err)
}
return reconcileContainerLocal(ctx, imageRef)
}
// EnsureTraefikContainerRemote ensures the orca-traefik podman container
// is running on a remote host (via SSH exec). Same idempotent logic as
// EnsureTraefikContainerLocal but over the provided exec function.
func EnsureTraefikContainerRemote(ctx context.Context, version string, execFn RemoteExecFunc) error {
imageRef := ImageRef(version)
if err := ensurePodmanRemote(execFn); err != nil {
return err
}
if err := removeLegacySystemdUnitRemote(execFn); err != nil {
_ = err // non-fatal
}
if _, err := execFn("mkdir -p /etc/traefik/dynamic /etc/orca"); err != nil {
return fmt.Errorf("traefik: ensure remote dirs: %w", err)
}
return reconcileContainerRemote(execFn, imageRef)
}
// ensurePodmanLocal checks if podman is installed locally and attempts
// to install it if absent (C-50).
func ensurePodmanLocal(ctx context.Context) error {
if _, err := exec.LookPath("podman"); err == nil {
return nil return nil
} }
url := downloadURL(version) // Attempt apt-get install (Ubuntu/Debian).
installCmd := fmt.Sprintf(`curl -fsSL %s | tar -xzf - -C /usr/local/bin/ traefik && chmod +x /usr/local/bin/traefik && mkdir -p /etc/traefik/dynamic`, url) cmd := exec.CommandContext(ctx, "bash", "-c",
if out, err := execFn(installCmd); err != nil { "apt-get update -qq && apt-get install -y -qq podman conmon crun fuse-overlayfs 2>&1")
return fmt.Errorf("download traefik on remote: %w (output: %s)", err, string(out)) if out, err := cmd.CombinedOutput(); err != nil {
return fmt.Errorf("podman not found and apt-get install failed: %w (output: %s).\nInstall podman manually: apt-get install podman conmon crun fuse-overlayfs", err, string(out))
} }
unit := systemdUnitContent()
_, _ = execFn(fmt.Sprintf(`echo '%s' > /etc/systemd/system/orca-traefik.service`, unit))
_, _ = execFn("systemctl daemon-reload && systemctl enable --now orca-traefik")
return nil return nil
} }
func ensureDirs() { // ensurePodmanRemote checks if podman is installed on the remote host
_ = exec.Command("mkdir", "-p", "/etc/traefik/dynamic").Run() // and attempts to install it if absent (C-50).
func ensurePodmanRemote(execFn RemoteExecFunc) error {
if out, err := execFn("command -v podman"); err == nil && len(strings.TrimSpace(string(out))) > 0 {
return nil
}
// Attempt apt-get install on the remote host.
cmd := "apt-get update -qq && apt-get install -y -qq podman conmon crun fuse-overlayfs 2>&1"
if out, err := execFn(cmd); err != nil {
return fmt.Errorf("podman not found on remote and apt-get install failed: %w (output: %s)", err, string(out))
}
return nil
} }
func writeSystemdUnit() { // removeLegacySystemdUnitLocal stops, disables, and removes the legacy
_ = exec.Command("bash", "-c", fmt.Sprintf(`echo '%s' > /etc/systemd/system/orca-traefik.service`, systemdUnitContent())).Run() // v0.13 orca-traefik.service systemd unit + /usr/local/bin/traefik
// binary (C-57). Idempotent — no-op if the unit doesn't exist.
func removeLegacySystemdUnitLocal(ctx context.Context) error {
// Check if the legacy unit exists.
if _, err := exec.CommandContext(ctx, "systemctl", "is-active", "orca-traefik.service").CombinedOutput(); err == nil {
// Unit is active or exists — stop + disable it.
_ = exec.CommandContext(ctx, "systemctl", "stop", "orca-traefik.service").Run()
_ = exec.CommandContext(ctx, "systemctl", "disable", "orca-traefik.service").Run()
}
// Remove the unit file and binary.
_ = exec.CommandContext(ctx, "rm", "-f", "/etc/systemd/system/orca-traefik.service").Run()
_ = exec.CommandContext(ctx, "rm", "-f", "/usr/local/bin/traefik").Run()
_ = exec.CommandContext(ctx, "systemctl", "daemon-reload").Run()
return nil
} }
func systemdUnitContent() string { // removeLegacySystemdUnitRemote is the remote SSH variant (C-57).
return `[Unit] func removeLegacySystemdUnitRemote(execFn RemoteExecFunc) error {
Description=Orca Traefik Data Plane cmd := `systemctl is-active orca-traefik.service 2>/dev/null && systemctl stop orca-traefik.service 2>/dev/null; systemctl disable orca-traefik.service 2>/dev/null; rm -f /etc/systemd/system/orca-traefik.service /usr/local/bin/traefik; systemctl daemon-reload 2>/dev/null; true`
After=network.target _, _ = execFn(cmd)
return nil
[Service] }
Type=simple
ExecStart=/usr/local/bin/traefik --configFile=/etc/traefik/traefik.yml // ensureDirsLocal creates /etc/traefik/dynamic and /etc/orca locally.
Restart=on-failure func ensureDirsLocal() error {
RestartSec=5s if err := exec.Command("mkdir", "-p", "/etc/traefik/dynamic", "/etc/orca").Run(); err != nil {
return fmt.Errorf("mkdir: %w", err)
[Install] }
WantedBy=multi-user.target` return nil
}
// reconcileContainerLocal implements the idempotent pull+run logic
// locally (C-50).
func reconcileContainerLocal(ctx context.Context, imageRef string) error {
// Check if the container is already running.
out, err := exec.CommandContext(ctx, "podman", "inspect", "--format", "{{.State.Running}}", ContainerName).CombinedOutput()
if err == nil {
v := strings.TrimSpace(string(out))
if v == "true" {
return nil // already running
}
// Container exists but is stopped — start it.
if _, err := exec.CommandContext(ctx, "podman", "start", ContainerName).CombinedOutput(); err != nil {
return fmt.Errorf("podman start %s: %w", ContainerName, err)
}
return nil
}
// Container does not exist — pull + run.
if out, err := exec.CommandContext(ctx, "podman", "pull", imageRef).CombinedOutput(); err != nil {
return fmt.Errorf("podman pull %s: %w (output: %s)", imageRef, err, string(out))
}
args := append([]string{}, podmanRunArgs(imageRef)...)
if out, err := exec.CommandContext(ctx, "podman", args...).CombinedOutput(); err != nil {
return fmt.Errorf("podman run: %w (output: %s)", err, string(out))
}
// Enable podman-restart.service for reboot persistence (research Topic 6).
_ = exec.CommandContext(ctx, "systemctl", "enable", "--now", "podman-restart.service").Run()
return nil
}
// reconcileContainerRemote implements the idempotent pull+run logic
// over SSH exec.
func reconcileContainerRemote(execFn RemoteExecFunc, imageRef string) error {
// Check if the container is already running.
out, err := execFn(fmt.Sprintf("podman inspect --format '{{.State.Running}}' %s 2>/dev/null", ContainerName))
if err == nil {
v := strings.TrimSpace(string(out))
if v == "true" {
return nil // already running
}
// Container exists but is stopped — start it.
if _, err := execFn(fmt.Sprintf("podman start %s 2>/dev/null", ContainerName)); err != nil {
return fmt.Errorf("podman start %s: %w", ContainerName, err)
}
return nil
}
// Container does not exist — pull + run.
if out, err := execFn(fmt.Sprintf("podman pull %s", shellQuote(imageRef))); err != nil {
return fmt.Errorf("podman pull %s: %w (output: %s)", imageRef, err, string(out))
}
runArgs := strings.Join(podmanRunArgs(imageRef), " ")
if out, err := execFn(fmt.Sprintf("podman %s", runArgs)); err != nil {
return fmt.Errorf("podman run: %w (output: %s)", err, string(out))
}
// Enable podman-restart.service for reboot persistence (research Topic 6).
_, _ = execFn("systemctl enable --now podman-restart.service 2>/dev/null || true")
return nil
}
// shellQuote wraps a string in single quotes for shell-safe usage.
func shellQuote(s string) string {
return "'" + strings.ReplaceAll(s, "'", "'\\''") + "'"
} }
+166
View File
@@ -0,0 +1,166 @@
package traefik
import (
"context"
"strings"
"testing"
)
func TestImageRef(t *testing.T) {
tests := []struct {
version string
want string
}{
{"v0.13.1", "git.cloudinit.dev/coreci/orca-traefik:v0.13.1"},
{"0.13.1", "git.cloudinit.dev/coreci/orca-traefik:v0.13.1"},
{"", "git.cloudinit.dev/coreci/orca-traefik:latest"},
{"dev", "git.cloudinit.dev/coreci/orca-traefik:latest"},
{"0.1.0-dev", "git.cloudinit.dev/coreci/orca-traefik:latest"},
{"v1.2.3-dev", "git.cloudinit.dev/coreci/orca-traefik:latest"},
}
for _, tt := range tests {
got := ImageRef(tt.version)
if got != tt.want {
t.Errorf("ImageRef(%q) = %q, want %q", tt.version, got, tt.want)
}
}
}
func TestPodmanRunArgs(t *testing.T) {
args := podmanRunArgs("git.cloudinit.dev/coreci/orca-traefik:v0.13.1")
joined := strings.Join(args, " ")
checks := []string{
"run -d",
"--name orca-traefik",
"--restart=unless-stopped",
"--network host",
"/etc/traefik/traefik.yml:/etc/traefik/traefik.yml:ro",
"/etc/traefik/dynamic:/etc/traefik/dynamic:ro",
"/etc/orca/step-ca-root.crt:/etc/orca/step-ca-root.crt:ro",
"git.cloudinit.dev/coreci/orca-traefik:v0.13.1",
}
for _, c := range checks {
if !strings.Contains(joined, c) {
t.Errorf("podmanRunArgs missing %q\nfull: %s", c, joined)
}
}
// Ensure no :Z flag (research Topic 7)
if strings.Contains(joined, ":Z") {
t.Errorf("podmanRunArgs should NOT contain :Z SELinux flag\nfull: %s", joined)
}
// Ensure --restart=always is NOT used (research Topic 6)
if strings.Contains(joined, "--restart=always") {
t.Errorf("podmanRunArgs should use --restart=unless-stopped, not --restart=always\nfull: %s", joined)
}
}
func TestEnsureTraefikContainerRemote_ContainerRunning(t *testing.T) {
var cmds []string
execFn := func(cmd string) ([]byte, error) {
cmds = append(cmds, cmd)
if strings.Contains(cmd, "podman inspect") {
return []byte("true\n"), nil
}
return []byte(""), nil
}
err := EnsureTraefikContainerRemote(context.Background(), "v0.13.1", execFn)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
// Should have checked inspect and found it running — no pull/run.
if len(cmds) < 1 {
t.Fatal("expected at least 1 command (inspect)")
}
for _, c := range cmds {
if strings.Contains(c, "podman pull") {
t.Errorf("should not pull when container is running: %s", c)
}
if strings.Contains(c, "podman run") {
t.Errorf("should not run when container is running: %s", c)
}
}
}
func TestEnsureTraefikContainerRemote_ContainerStopped(t *testing.T) {
var cmds []string
execFn := func(cmd string) ([]byte, error) {
cmds = append(cmds, cmd)
if strings.Contains(cmd, "podman inspect") {
return []byte("false\n"), nil // stopped
}
return []byte(""), nil
}
err := EnsureTraefikContainerRemote(context.Background(), "v0.13.1", execFn)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
// Should have started the container.
foundStart := false
for _, c := range cmds {
if strings.Contains(c, "podman start orca-traefik") {
foundStart = true
}
}
if !foundStart {
t.Errorf("expected 'podman start orca-traefik' when container is stopped\ncommands: %v", cmds)
}
}
func TestEnsureTraefikContainerRemote_ContainerAbsent(t *testing.T) {
var cmds []string
execFn := func(cmd string) ([]byte, error) {
cmds = append(cmds, cmd)
if strings.Contains(cmd, "podman inspect") {
return nil, &execError{"inspect failed: no such container"}
}
return []byte(""), nil
}
err := EnsureTraefikContainerRemote(context.Background(), "v0.13.1", execFn)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
// Should have pulled and run.
foundPull := false
foundRun := false
for _, c := range cmds {
if strings.Contains(c, "podman pull") {
foundPull = true
}
if strings.Contains(c, "podman run -d") {
foundRun = true
}
}
if !foundPull {
t.Errorf("expected 'podman pull' when container is absent\ncommands: %v", cmds)
}
if !foundRun {
t.Errorf("expected 'podman run -d' when container is absent\ncommands: %v", cmds)
}
}
func TestEnsureTraefikContainerRemote_LegacySystemdRemoval(t *testing.T) {
var cmds []string
execFn := func(cmd string) ([]byte, error) {
cmds = append(cmds, cmd)
if strings.Contains(cmd, "podman inspect") {
return []byte("true\n"), nil // container running
}
return []byte(""), nil
}
_ = EnsureTraefikContainerRemote(context.Background(), "v0.13.1", execFn)
// Should include legacy systemd unit removal command (C-57).
foundLegacyRemoval := false
for _, c := range cmds {
if strings.Contains(c, "orca-traefik.service") && strings.Contains(c, "stop") {
foundLegacyRemoval = true
}
}
if !foundLegacyRemoval {
t.Errorf("expected legacy systemd unit removal command (C-57)\ncommands: %v", cmds)
}
}
// execError is a simple error type for testing.
type execError struct{ msg string }
func (e *execError) Error() string { return e.msg }
+1
View File
@@ -164,6 +164,7 @@ if [ -z "$ASSET_URL" ]; then
ASSET_URL="$(echo "$FALLBACK_OUT" | head -1)" ASSET_URL="$(echo "$FALLBACK_OUT" | head -1)"
if [ -n "$ASSET_URL" ]; then if [ -n "$ASSET_URL" ]; then
VERSION="$(echo "$FALLBACK_OUT" | tail -1)" VERSION="$(echo "$FALLBACK_OUT" | tail -1)"
TARBALL="orca-${VERSION}-${OS}-${ARCH}.tar.gz"
info "WARNING: falling back to ${VERSION} which has orca-${VERSION}-${OS}-${ARCH}.tar.gz." info "WARNING: falling back to ${VERSION} which has orca-${VERSION}-${OS}-${ARCH}.tar.gz."
else else
err "could not find any release with a ${OS}-${ARCH} tarball in the last 50 releases. Check that a release exists with a linux-${ARCH} binary." err "could not find any release with a ${OS}-${ARCH} tarball in the last 50 releases. Check that a release exists with a linux-${ARCH} binary."
+33
View File
@@ -211,3 +211,36 @@ else
info "✓ container image ${IMAGE}:${VERSION} published" info "✓ container image ${IMAGE}:${VERSION} published"
fi fi
fi fi
# --- publish orca-traefik container image (REQ-171, R-024) ----------------
# Custom traefik image with baked default static config. The host-side
# traefik.yml is mounted :ro at runtime to override the baked default.
# Skipped gracefully if docker is not on PATH (CI handles it via
# .coreci.yml container-publish-traefik step).
TRAEFIK_IMAGE="${CONTAINER_REGISTRY}/${CONTAINER_OWNER}/orca-traefik"
if ! command -v docker >/dev/null 2>&1; then
info "docker not found on PATH — skipping orca-traefik image publish (CI handles it)."
else
info "building orca-traefik image ${TRAEFIK_IMAGE}:${VERSION}..."
docker build \
-f Dockerfile.traefik \
-t "${TRAEFIK_IMAGE}:${VERSION}" \
-t "${TRAEFIK_IMAGE}:latest" \
"$REPO_ROOT"
if [ -z "${GITEA_TOKEN:-}" ]; then
info "GITEA_TOKEN not set — skipping orca-traefik push (image built locally only)."
else
if ! docker login "$CONTAINER_REGISTRY" -u cloudinit-bot --password-stdin <<<"$GITEA_TOKEN" 2>/dev/null; then
echo "$GITEA_TOKEN" | docker login "$CONTAINER_REGISTRY" -u cloudinit-bot --password-stdin
fi
info "pushing ${TRAEFIK_IMAGE}:${VERSION}..."
docker push "${TRAEFIK_IMAGE}:${VERSION}"
info "pushing ${TRAEFIK_IMAGE}:latest..."
docker push "${TRAEFIK_IMAGE}:latest"
docker logout "$CONTAINER_REGISTRY"
info "✓ orca-traefik image ${TRAEFIK_IMAGE}:${VERSION} published"
fi
fi
+23
View File
@@ -39,6 +39,29 @@ load test_helper
assert_contains "$output" "dry-run (--check)" assert_contains "$output" "dry-run (--check)"
} }
@test "install.sh fallback walk syncs TARBALL to fallback version (REQ-132 regression)" {
# Regression guard: when the fallback walk reassigns VERSION, the
# TARBALL variable must be recomputed too. v0.14.2 is a release with
# no binary asset; the installer must walk back to an earlier release.
# The dry-run "would install" line must reference the SAME fallback
# version as the "falling back to" line — not the stale pinned one.
# Before the fix, TARBALL stayed at the pinned v0.14.2 name while
# VERSION became the fallback, causing grep|sha256sum to see no
# matching checksum line and REQ-132 to refuse install.
skip_if_no_network
run timeout 60 "$SCRIPTS_DIR/install.sh" --check --version v0.14.2
assert_status 0 "$status"
assert_contains "$output" "falling back"
# Capture the fallback version from the "falling back to vX.Y.Z" line.
fb_version="$(printf '%s\n' "$output" | sed -n 's/.*falling back to \(v[0-9][0-9.]*\).*/\1/p' | head -1)"
[ -n "$fb_version" ] || { echo "could not parse fallback version from output: $output" >&2; return 1; }
# The dry-run "would install" line must use the fallback version,
# proving VERSION and TARBALL are in sync (not the stale pinned v0.14.2).
assert_contains "$output" "would install: orca ${fb_version}"
# And it must NOT reference the stale pinned version in the install line.
assert_not_contains "$output" "would install: orca v0.14.2"
}
@test "install.sh rejects unknown arguments" { @test "install.sh rejects unknown arguments" {
run "$SCRIPTS_DIR/install.sh" --bogus-flag run "$SCRIPTS_DIR/install.sh" --bogus-flag
[ "$status" -ne 0 ] [ "$status" -ne 0 ]
+14
View File
@@ -30,6 +30,20 @@ fi
while read local_ref local_sha remote_ref remote_sha; do while read local_ref local_sha remote_ref remote_sha; do
branch="${remote_ref#refs/heads/}" branch="${remote_ref#refs/heads/}"
if [ -z "$branch" ] || [ "$branch" = "HEAD" ]; then if [ -z "$branch" ] || [ "$branch" = "HEAD" ]; then
# Check if this is a tag push (refs/tags/*)
tag="${remote_ref#refs/tags/}"
if [ -n "$tag" ] && [ "$tag" != "$remote_ref" ]; then
echo "→ Triggering CoreCI for tag: $tag (${local_sha:0:7})"
payload=$(printf '{"repo":"coreci/orca","branch":"%s","ref":"%s"}' "$tag" "$local_sha")
if command -v curl >/dev/null 2>&1; then
curl -fsS -X POST "${CORECI_URL}/api/pipeline/run" \
-H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" \
-d "$payload" >/dev/null 2>&1 \
&& echo " ✓ CoreCI triggered" \
|| echo " (CoreCI trigger failed; Gitea Actions webhook is secondary path)"
fi
fi
continue continue
fi fi
echo "→ Triggering CoreCI for branch: $branch (${local_sha:0:7})" echo "→ Triggering CoreCI for branch: $branch (${local_sha:0:7})"
+25 -2
View File
@@ -145,8 +145,8 @@ assert "34 type_linux_available" \
assert "35 status_deprecated" \ assert "35 status_deprecated" \
'$ORCA status 2>&1 | grep -qi "deprecated"' '$ORCA status 2>&1 | grep -qi "deprecated"'
assert "36 traefik_installed" \ assert "36 traefik_container_running" \
'systemctl is-active orca-traefik 2>/dev/null | grep -q "active" || exit 77' 'podman inspect --format "{{.State.Running}}" orca-traefik 2>/dev/null | grep -q "true" || exit 77'
assert "37 known_hosts_exists" \ assert "37 known_hosts_exists" \
'test -f "$ORCA_HOME/known_hosts" || test -f "$ORCA_HOME/cluster/known_hosts"' 'test -f "$ORCA_HOME/known_hosts" || test -f "$ORCA_HOME/cluster/known_hosts"'
@@ -154,6 +154,29 @@ assert "37 known_hosts_exists" \
assert "38 master_key_exists" \ assert "38 master_key_exists" \
'test -f "$ORCA_HOME/cluster/master.key" || test -f "$ORCA_HOME/cluster/master.key.sealed"' 'test -f "$ORCA_HOME/cluster/master.key" || test -f "$ORCA_HOME/cluster/master.key.sealed"'
# --- v0.14 ingress bootstrap assertions (R-024) ---
assert "40 ingress_nft_table" \
'nft list table inet orca-ingress 2>/dev/null | grep -q "chain prerouting"'
assert "41 ingress_nft_dnat" \
'nft list table inet orca-ingress 2>/dev/null | grep -q "dnat to"'
assert "42 ingress_nft_snat" \
'nft list table inet orca-ingress 2>/dev/null | grep -q "masquerade"'
assert "43 ingress_dynamic_dir" \
'test -d /etc/traefik/dynamic'
assert "44 ingress_step_ca" \
'test -f /etc/orca/step-ca-root.crt'
assert "45 ingress_traefik_yml" \
'test -f /etc/traefik/traefik.yml'
assert "46 ingress_doctor_pass" \
'$ORCA doctor ingress 2>&1 | grep -q "PASS"'
# --- Report --- # --- Report ---
echo "==========================================" echo "=========================================="
+163
View File
@@ -0,0 +1,163 @@
package tests
import (
"log/slog"
"strings"
"testing"
"git.cloudinit.dev/coreci/orca/internal/emitter"
"git.cloudinit.dev/coreci/orca/internal/jobspec"
"git.cloudinit.dev/coreci/orca/internal/proxmox"
"git.cloudinit.dev/coreci/orca/internal/traefik"
)
// TestNftEmitter_PostroutingAndDNATTarget (REQ-173) verifies the nft
// emitter renders the postrouting masquerade chain and supports
// DNATTarget substitution.
func TestNftEmitter_PostroutingAndDNATTarget(t *testing.T) {
files, err := emitter.NftEmitter{}.RenderNftConfig(emitter.NftClusterConfig{
DNATTarget: "10.99.0.10",
EnableSNAT: true,
})
if err != nil {
t.Fatalf("RenderNftConfig: %v", err)
}
c := files[0].Content
if !strings.Contains(c, "chain postrouting") {
t.Errorf("missing postrouting chain:\n%s", c)
}
if !strings.Contains(c, "masquerade") {
t.Errorf("missing masquerade rule:\n%s", c)
}
if !strings.Contains(c, "dnat to 10.99.0.10:8443") {
t.Errorf("missing custom DNAT target:\n%s", c)
}
}
// TestNftEmitter_PriorityMinus10 (research Topic 2) verifies the input
// and forward chains use priority -10 for pve-firewall coexistence.
func TestNftEmitter_PriorityMinus10(t *testing.T) {
files, _ := emitter.NftEmitter{}.RenderNftConfig(emitter.NftClusterConfig{})
c := files[0].Content
if !strings.Contains(c, "hook input priority -10;") {
t.Errorf("input chain should use priority -10:\n%s", c)
}
if !strings.Contains(c, "hook forward priority -10;") {
t.Errorf("forward chain should use priority -10:\n%s", c)
}
}
// TestTraefikEmitter_TLSModel (REQ-172) verifies the dynamic config
// emits tls: {} and does NOT contain certResolver (dropped in v0.14).
func TestTraefikEmitter_TLSModel(t *testing.T) {
spec := &jobspec.WorkloadSpec{
Name: "test-svc",
Kind: "Service",
Ports: []jobspec.PortSpec{{Name: "http"}},
}
node := &emitter.Node{
Hostname: "test-node",
}
files, err := emitter.TraefikEmitter{}.Render(spec, node)
if err != nil {
t.Fatalf("Render: %v", err)
}
c := files[0].Content
if !strings.Contains(c, "tls: {}") {
t.Errorf("missing tls: {} (v0.14 model):\n%s", c)
}
if strings.Contains(c, "certResolver: orca") {
t.Errorf("certResolver: orca should be removed (v0.14):\n%s", c)
}
}
// TestTraefikImageRef verifies the image reference resolution for the
// orca-traefik podman container.
func TestTraefikImageRef(t *testing.T) {
ref := traefik.ImageRef("v0.13.7")
want := "git.cloudinit.dev/coreci/orca-traefik:v0.13.7"
if ref != want {
t.Errorf("ImageRef(v0.13.7) = %q, want %q", ref, want)
}
// Dev build falls back to latest.
ref = traefik.ImageRef("dev")
if ref != "git.cloudinit.dev/coreci/orca-traefik:latest" {
t.Errorf("ImageRef(dev) = %q, want latest", ref)
}
}
// TestProxmox_FloatingIP_LXC_ProvisioningCommands (REQ-176) verifies
// the ProvisionIngressLXC function sends the correct pct create
// command with the right net0 parameters.
func TestProxmox_FloatingIP_LXC_ProvisioningCommands(t *testing.T) {
var cmds []string
execFn := func(cmd string) ([]byte, error) {
cmds = append(cmds, cmd)
// Simulate: pct status returns "absent" on first call, then OK.
if strings.Contains(cmd, "pct status 201") {
return []byte("absent\n"), nil
}
if strings.Contains(cmd, "pct create") {
return []byte(""), nil
}
if strings.Contains(cmd, "pct start 201") {
return []byte(""), nil
}
if strings.Contains(cmd, "hostname -I") {
return []byte("203.0.113.10\n"), nil
}
return []byte(""), nil
}
err := proxmox.ProvisionIngressLXC(nil, execFn, proxmox.FloatingIPOptions{
FloatingIP: "203.0.113.10",
Gateway: "203.0.113.1",
MAC: "02:01:02:03:04:05",
NetPrefix: 24,
LXCTemplate: "ubuntu-24.04",
}, slog.Default())
if err != nil {
t.Fatalf("ProvisionIngressLXC: %v", err)
}
// Verify pct create has the right net0 params.
foundCreate := false
for _, c := range cmds {
if strings.Contains(c, "pct create") {
foundCreate = true
if !strings.Contains(c, "hostname ingress") {
t.Errorf("pct create missing hostname ingress: %s", c)
}
if !strings.Contains(c, "hwaddr=02:01:02:03:04:05") {
t.Errorf("pct create missing hwaddr: %s", c)
}
if !strings.Contains(c, "ip=203.0.113.10/24") {
t.Errorf("pct create missing ip: %s", c)
}
if !strings.Contains(c, "gw=203.0.113.1") {
t.Errorf("pct create missing gw: %s", c)
}
if !strings.Contains(c, "nesting=1,keyctl=1,fuse=1") {
t.Errorf("pct create missing features (research Topic 3): %s", c)
}
}
}
if !foundCreate {
t.Errorf("pct create command not sent\ncommands: %v", cmds)
}
}
// TestProxmox_GenerateRandomMAC (D-261) verifies MAC generation produces
// a valid locally-administered MAC.
func TestProxmox_GenerateRandomMAC(t *testing.T) {
mac, err := proxmox.GenerateRandomMAC()
if err != nil {
t.Fatalf("GenerateRandomMAC: %v", err)
}
if !strings.HasPrefix(mac, "02:") {
t.Errorf("MAC should start with 02: (locally administered): %s", mac)
}
// Verify it's 6 octets.
parts := strings.Split(mac, ":")
if len(parts) != 6 {
t.Errorf("MAC should have 6 octets: %s", mac)
}
}