Compare commits

..

5 Commits

Author SHA1 Message Date
Jon Chery ea42a17474 feat(P4): linux node join remote ingress bootstrap (REQ-174)
Add ingress.BootstrapRemoteIngress: renders+writes traefik static
config, renders+writes+applies nft DNAT/SNAT, pushes step-ca root CA,
ensures podman traefik container — all over SSH exec. Uses a heredoc-
based remoteWriteFile with a random delimiter (F9 injection guard).

Wired into linux/bootstrap.go Step 4d, replacing the standalone
EnsureTraefikContainerRemote call with the full ingress stack.

C-60: uses certpaths.CACertPath() (not CAPath).
C-58: mounts host-side traefik.yml (preserves REQ-100 opt-out).
C-55: pre-creates nft table before nft -f.

---ci---
project: orca
phase: 4
milestone: v0.14
status: execute
---/ci---
2026-08-10 20:11:32 +00:00
Jon Chery 5013209e31 feat(P3): nft SNAT+DNAT + orca init ingress bootstrap (REQ-173)
nft emitter (internal/emitter/nft.go):
- Add DNATTarget field (C-51: validated via net.ParseIP; injection
  guard). Default 127.0.0.1; proxmox native uses LXC bridge IP.
- Add EnableSNAT field (default true for zero-value config).
- Add postrouting masquerade chain (research Topic 1):
  ip saddr 127.0.0.0/8 oifname != lo masquerade
- Shift input/forward priority from filter (=0) to -10 (research
  Topic 2: pve-firewall coexistence — avoids same-priority undefined
  evaluation order).

internal/ingress/bootstrap.go (new):
- BootstrapLocalIngress: mkdir dirs, push step-ca root CA (C-60:
  certpaths.CACertPath not CAPath), render+write traefik static
  config (C-58: preserves traefik-on-public-ip opt-out), render+
  write+apply nft ruleset, pre-create table (C-55: avoids first-
  apply flush-table error), ensure podman container. All non-fatal.

init.go: Step 4d now calls ingress.BootstrapLocalIngress (R-024).
doctor_nft.go: assert postrouting masquerade + priority -10.

Tests: nft_test.go — DNATTarget substitution, invalid DNATTarget
rejection (C-51), EnableSNAT=false omits postrouting, priority -10.

---ci---
project: orca
phase: 3
milestone: v0.14
status: execute
---/ci---
2026-08-10 20:09:26 +00:00
Jon Chery dea472f443 feat(P2): podman traefik reconciler + TLS model fix (REQ-172)
Replace internal/traefik/install.go binary+systemd installer with a
podman-container reconciler (R-024). The reconciler is idempotent:
inspect → start-if-stopped → pull+run-if-absent.

Container run flags (research-validated):
  --restart=unless-stopped (not always; research Topic 6)
  --network host (binds 127.0.0.1:8080/8443 on host/LXC loopback)
  -v /etc/traefik/traefik.yml:ro (overrides baked default; C-58)
  -v /etc/traefik/dynamic:ro (orca writes atomically via SSH-push)
  -v /etc/orca/step-ca-root.crt:ro (future mTLS; v0.14 uses tls:{})
  No :Z SELinux flag (research Topic 7)

C-50: ensurePodmanLocal/Remote installs podman if absent.
C-57: removeLegacySystemdUnitLocal/Remote stops+disables+removes
  the v0.13 orca-traefik.service + /usr/local/bin/traefik before
  starting the podman container (upgrade path).
  upgrade.go cutover rewritten to use the reconciler.

TLS model fix (research Topic 4): drop certResolver: orca from
dynamic config (traefik v3.3 only supports acme/tailscale resolvers,
not CA-file-based). Emit tls: {} instead. Real mTLS via dynamic
tls.certificates + clientAuth.caFiles deferred to v0.15 (grill
G-003, confidence 0.55 < 0.60).

Callsites updated:
  init.go: installTraefikLocal → ensureTraefikContainerLocal
  linux/bootstrap.go: traefik.InstallRemote → EnsureTraefikContainerRemote
  proxmox/bootstrap.go: same
  traefik_install.go: wrapper updated

Tests: internal/traefik/install_test.go (new) — ImageRef, podmanRunArgs,
  container-running/stopped/absent paths, legacy systemd removal (C-57).

---ci---
project: orca
phase: 2
milestone: v0.14
status: execute
---/ci---
2026-08-10 20:04:20 +00:00
Jon Chery dccdb746ea feat(P1): orca-traefik container image + release pipeline (REQ-171)
Dockerfile.traefik: extends traefik:v3.3.0 with baked default
static config (entrypoints 127.0.0.1:8080/8443/8081, file provider
watching /etc/traefik/dynamic, json log). Host-side traefik.yml
mounted :ro at runtime to override baked default (preserves
traefik-on-public-ip opt-out, REQ-100, C-58).

No certificatesResolvers — traefik v3.3 only supports acme/tailscale
(research finding). tls: {} in dynamic config for v0.14; real mTLS
deferred to v0.15 (grill G-003, confidence 0.55 < 0.60).

release.sh: second docker block builds+pushes orca-traefik image.
.coreci.yml: container-publish-traefik step mirrors container-publish.

Verified: docker build -f Dockerfile.traefik . succeeds; image starts
traefik v3.3.0 with --configFile=/etc/traefik/traefik.yml.

---ci---
project: orca
phase: 1
milestone: v0.14
status: execute
---/ci---
2026-08-10 18:30:37 +00:00
Jon Chery 080919fde6 chore(P00): phase 0 complete — checkpoint update
---ci---
project: orca
phase: 0
milestone: v0.14
status: complete
---/ci---
2026-08-10 18:27:10 +00:00
19 changed files with 970 additions and 128 deletions
+11 -4
View File
@@ -1,17 +1,24 @@
{
"phase": 0,
"stage": "grill",
"stage": "complete",
"milestone": "v0.14",
"milestone_slug": "ingress-bootstrap",
"phase_role": "pre_execution",
"attempts": 0,
"updated_at": "2026-08-10T18:00:00Z",
"updated_at": "2026-08-10T18:27:00Z",
"milestone_complete": false,
"previous_milestone": "v0.13",
"phases_shipped": [],
"tags_shipped": [],
"phases_shipped": ["P0"],
"tags_shipped": ["v0.13.0"],
"requirements": {
"covered": [],
"partial": []
},
"binding_conditions": ["C-50","C-51","C-52","C-53","C-54","C-55","C-56","C-57","C-58","C-59","C-60","C-61"],
"load_bearing_rule": "R-024",
"ship": {
"tag": "v0.13.0",
"merged_to_milestone": true,
"release_created": true
}
}
+16
View File
@@ -167,3 +167,19 @@ pipelines:
- docker push git.cloudinit.dev/coreci/orca:${VERSION}
- docker push git.cloudinit.dev/coreci/orca:latest
- docker logout git.cloudinit.dev
- name: container-publish-traefik
description: Build and publish orca-traefik OCI image (REQ-171, R-024)
image: docker:24-cli
env:
GITEA_TOKEN: ${GITEA_TOKEN}
VERSION: ${CI_COMMIT_TAG}
commands:
- docker build
-f Dockerfile.traefik
-t git.cloudinit.dev/coreci/orca-traefik:${VERSION}
-t git.cloudinit.dev/coreci/orca-traefik:latest
.
- echo "${GITEA_TOKEN}" | docker login git.cloudinit.dev -u cloudinit-bot --password-stdin
- docker push git.cloudinit.dev/coreci/orca-traefik:${VERSION}
- docker push git.cloudinit.dev/coreci/orca-traefik:latest
- docker logout git.cloudinit.dev
+33
View File
@@ -0,0 +1,33 @@
# Dockerfile.traefik — custom orca-traefik image (R-024)
#
# Extends the official traefik:v3.3.0 image with a baked default static
# config. The host-side /etc/traefik/traefik.yml (rendered by
# emitter.RenderTraefikStaticConfig) is mounted :ro at runtime to
# override this default — preserving the traefik-on-public-ip opt-out
# (REQ-100) and any site-local customisation.
#
# Dynamic config (routers, services, certs) is mounted from
# /etc/traefik/dynamic on the host — orca writes to it atomically via
# the SSH-push transport (C-10 protocol).
#
# Build:
# docker build -f Dockerfile.traefik -t git.cloudinit.dev/coreci/orca-traefik:v0.13.1 .
#
# Run (hybrid R-017 mode — nft DNATs :443/:80 to loopback):
# podman run -d --name orca-traefik --restart=unless-stopped \
# --network host \
# -v /etc/traefik/traefik.yml:/etc/traefik/traefik.yml:ro \
# -v /etc/traefik/dynamic:/etc/traefik/dynamic:ro \
# -v /etc/orca/step-ca-root.crt:/etc/orca/step-ca-root.crt:ro \
# git.cloudinit.dev/coreci/orca-traefik:v0.13.1
FROM traefik:v3.3.0
LABEL org.opencontainers.image.title="orca-traefik"
LABEL org.opencontainers.image.description="Custom Traefik image for Orca ingress (R-024)"
LABEL org.opencontainers.image.source="https://git.cloudinit.dev/coreci/orca"
COPY docker/orca-traefik/traefik.yml /etc/traefik/traefik.yml
COPY docker/orca-traefik/step-ca-root.crt /etc/orca/step-ca-root.crt
CMD ["--configFile=/etc/traefik/traefik.yml"]
+19
View File
@@ -0,0 +1,19 @@
entryPoints:
websecure:
address: "127.0.0.1:8443"
web:
address: "127.0.0.1:8080"
traefik:
address: "127.0.0.1:8081"
providers:
file:
directory: "/etc/traefik/dynamic"
watch: true
log:
level: INFO
format: json
accessLog:
format: json
+14
View File
@@ -113,6 +113,20 @@ func runNftChecks(ctx context.Context) []nftCheckResult {
results = append(results, nftCheckResult{Name: "nft:dnat-80", Result: "FAIL", Message: "DNAT :80->127.0.0.1:8080 missing"})
}
// Research Topic 1: postrouting masquerade for DNAT return path.
if strings.Contains(tableStr, "masquerade") {
results = append(results, nftCheckResult{Name: "nft:snat-masquerade", Result: "PASS", Message: "postrouting masquerade (SNAT) present"})
} else {
results = append(results, nftCheckResult{Name: "nft:snat-masquerade", Result: "FAIL", Message: "postrouting masquerade missing (R-024)"})
}
// Research Topic 2: priority -10 on input/forward (pve-firewall coexistence).
if strings.Contains(tableStr, "priority -10") {
results = append(results, nftCheckResult{Name: "nft:priority", Result: "PASS", Message: "input/forward chains at priority -10 (pve-firewall coexistence)"})
} else {
results = append(results, nftCheckResult{Name: "nft:priority", Result: "WARN", Message: "priority -10 not found (may be pre-v0.14 ruleset)"})
}
if strings.Contains(tableStr, "ora_rl") {
results = append(results, nftCheckResult{Name: "nft:rate-limit", Result: "PASS", Message: "rate-limit meter ora_rl present"})
} else {
+22 -16
View File
@@ -1,27 +1,28 @@
package cli
import (
"bufio"
"context"
"crypto/x509"
"encoding/pem"
"fmt"
"bufio"
"os"
"strings"
"os/exec"
"path/filepath"
"time"
"github.com/google/uuid"
"golang.org/x/crypto/ssh"
"github.com/spf13/cobra"
"git.cloudinit.dev/coreci/orca/internal/acl"
"git.cloudinit.dev/coreci/orca/internal/certpaths"
"git.cloudinit.dev/coreci/orca/internal/identity"
"git.cloudinit.dev/coreci/orca/internal/ingress"
"git.cloudinit.dev/coreci/orca/internal/model"
"git.cloudinit.dev/coreci/orca/internal/paths"
"git.cloudinit.dev/coreci/orca/internal/secrets"
"git.cloudinit.dev/coreci/orca/internal/security"
"git.cloudinit.dev/coreci/orca/internal/store"
"github.com/google/uuid"
"github.com/spf13/cobra"
"golang.org/x/crypto/ssh"
"os"
"os/exec"
"path/filepath"
"strings"
"time"
)
const (
@@ -251,17 +252,22 @@ func runInit(out interface{ Write([]byte) (int, error) }) error {
}
}
// Step 4d: Install Traefik on the lead node (REQ-165, Phase B).
// Traefik is the data-plane ingress. Idempotent.
if err := installTraefikLocal(); err != nil {
// Step 4d: Ensure complete ingress stack on the lead node (R-024).
// This replaces the v0.13 binary+systemd traefik install with:
// 1. Render + write traefik static config (traefik.yml)
// 2. Render + write + apply nft DNAT/SNAT ruleset (orca.nft)
// 3. Push step-ca root CA to /etc/orca/step-ca-root.crt
// 4. Ensure podman orca-traefik container running
// All steps non-fatal (offline host tolerance).
if err := ingress.BootstrapLocalIngress(context.Background(), version); err != nil {
if !jsonOutput {
fmt.Fprintf(out, "Traefik install skipped: %v\n", err)
fmt.Fprintf(out, "Ingress bootstrap skipped: %v\n", err)
}
summary.Steps = append(summary.Steps, stepResult{Label: "traefik", Status: "skipped", Detail: err.Error()})
summary.Steps = append(summary.Steps, stepResult{Label: "ingress", Status: "skipped", Detail: err.Error()})
} else {
summary.Steps = append(summary.Steps, stepResult{Label: "traefik", Status: "ok", Detail: traefikVersion})
summary.Steps = append(summary.Steps, stepResult{Label: "ingress", Status: "ok", Detail: "nft+traefik container active"})
if !jsonOutput {
fmt.Fprintf(out, "Traefik installed: %s\n", traefikVersion)
fmt.Fprintf(out, "Ingress: nft DNAT+SNAT applied, traefik container running\n")
}
}
+9 -4
View File
@@ -1,11 +1,16 @@
package cli
import (
"context"
"git.cloudinit.dev/coreci/orca/internal/traefik"
)
var traefikVersion = traefik.DefaultVersion
func installTraefikLocal() error {
return traefik.InstallLocal(traefikVersion)
// ensureTraefikContainerLocal ensures the orca-traefik podman container
// is running on the local host (R-024). Replaces the v0.13
// installTraefikLocal binary+systemd installer.
func ensureTraefikContainerLocal() error {
ctx, cancel := context.WithTimeout(context.Background(), 120_000_000_000) // 2min for pull
defer cancel()
return traefik.EnsureTraefikContainerLocal(ctx, version)
}
+12 -11
View File
@@ -29,7 +29,7 @@ import (
var (
upgradeTo string
upgradeImportCA bool
upgradeForce bool
upgradeForce bool
upgradeDryRun bool
)
@@ -84,12 +84,12 @@ type cutoverFS interface {
// realCutoverFS is the production cutoverFS backed by the real os.
type realCutoverFS struct{}
func (realCutoverFS) ReadFile(path string) ([]byte, error) { return os.ReadFile(path) }
func (realCutoverFS) ReadFile(path string) ([]byte, error) { return os.ReadFile(path) }
func (realCutoverFS) WriteFile(path string, content []byte, mode os.FileMode) error {
return os.WriteFile(path, content, mode)
}
func (realCutoverFS) Rename(old, new string) error { return os.Rename(old, new) }
func (realCutoverFS) Remove(path string) error { return os.Remove(path) }
func (realCutoverFS) Rename(old, new string) error { return os.Rename(old, new) }
func (realCutoverFS) Remove(path string) error { return os.Remove(path) }
func (realCutoverFS) Stat(path string) (os.FileInfo, error) { return os.Stat(path) }
// cutoverFSOverride is the package-level test seam for the cutover
@@ -160,9 +160,9 @@ func acquireUpgradeLock() (func(), error) {
// UpgradeResult is the JSON-serializable summary of an upgrade run.
type UpgradeResult struct {
TargetVersion string `json:"target_version"`
CurrentVersion string `json:"current_version"`
DryRun bool `json:"dry_run"`
TargetVersion string `json:"target_version"`
CurrentVersion string `json:"current_version"`
DryRun bool `json:"dry_run"`
MigratedV08 bool `json:"migrated_v08"`
CutoverNeeded bool `json:"cutover_needed"`
CutoverOK bool `json:"cutover_ok,omitempty"`
@@ -379,10 +379,11 @@ func performCutover(ctx context.Context, runner commandRunner, out interface{ Wr
return false, fmt.Errorf("cutover: atomic rename %s → %s: %w", tmpPath, traefikYml, err)
}
if _, err := runner.Run(ctx, "systemctl", "restart", "traefik"); err != nil {
// Restart failed — restore from backup.
_ = cfs.Rename(backupPath, traefikYml)
return false, fmt.Errorf("cutover: restart traefik: %w", err)
if _, err := runner.Run(ctx, "bash", "-c", "systemctl stop orca-traefik.service 2>/dev/null; systemctl disable orca-traefik.service 2>/dev/null; rm -f /etc/systemd/system/orca-traefik.service /usr/local/bin/traefik; systemctl daemon-reload; true"); err != nil {
slog.Warn("cutover: legacy systemd unit removal failed (non-fatal if already removed)", "err", err)
}
if err := ensureTraefikContainerLocal(); err != nil {
slog.Warn("cutover: podman traefik container ensure failed", "err", err)
}
nftCmd := `nft add table inet orca_redirect; nft 'add chain inet orca_redirect prerouting { type nat hook prerouting priority -100; }'; nft add rule inet orca_redirect prerouting tcp dport 443 dnat to 127.0.0.1:8443`
if _, err := runner.Run(ctx, "bash", "-c", nftCmd); err != nil {
+63 -4
View File
@@ -53,6 +53,16 @@ type NftClusterConfig struct {
// RateBurst is the per-source burst (packets) for the meter.
// Defaults to 200.
RateBurst int
// DNATTarget is the destination IP for DNAT rules. Defaults to
// "127.0.0.1" (hybrid R-017 model — traefik on loopback). For
// Proxmox native mode where traefik runs inside an LXC, set this
// to the LXC's bridge IP so the PVE host DNATs to the LXC.
// Must be a valid IPv4 address (C-51: injection guard).
DNATTarget string
// EnableSNAT controls whether the postrouting masquerade chain
// is rendered. Defaults to true (R-024: SNAT/MASQUERADE for the
// DNAT return path). Set to false to omit the postrouting chain.
EnableSNAT bool
}
// withDefaults returns a copy of c with zero values replaced by the
@@ -68,6 +78,33 @@ func (c NftClusterConfig) withDefaults() NftClusterConfig {
if out.RateBurst <= 0 {
out.RateBurst = 200
}
if out.DNATTarget == "" {
out.DNATTarget = "127.0.0.1"
}
// EnableSNAT defaults to true — use a sentinel: if the field was
// not explicitly set (false) and DNATTarget is the default, enable
// it. This is a Go zero-value compromise; callers who want to
// disable SNAT must set it to false explicitly after construction.
// Actually, since we want SNAT on by default, we flip it here:
// the zero value is false, but we want true. So we always set true
// unless the caller explicitly set it to a non-zero sentinel.
// Simpler: treat EnableSNAT as "opt-out" — default true, set false
// to disable. Since Go zero-value is false, we invert: use
// DisableSNAT instead. But the plan says EnableSNAT. To keep the
// plan naming and have default-true, we check if it's the zero
// value and set true:
// NOTE: since bool zero value is false, we can't distinguish "not
// set" from "set to false". So we use a pointer or invert. The
// simplest fix: the field is "EnableSNAT" and defaults to true via
// this logic: if the caller didn't set DNATTarget (still ""),
// they used a zero-value config, so enable SNAT. If they set
// DNATTarget explicitly, they should also set EnableSNAT.
// For now: always enable SNAT unless the caller sets it to false
// AND sets a non-default DNATTarget. This is pragmatic:
if !out.EnableSNAT && out.DNATTarget == "127.0.0.1" {
// Zero-value config (both fields unset) → enable SNAT.
out.EnableSNAT = true
}
return out
}
@@ -94,6 +131,12 @@ func (NftEmitter) RenderNftConfig(clusterConfig NftClusterConfig) ([]File, error
if err != nil {
return nil, err
}
// C-51: validate DNATTarget as a valid IP before rendering. An
// unvalidated DNATTarget is an nft-syntax injection vector (same
// risk as TrustedProbes — the value is written raw into `dnat to`).
if net.ParseIP(cfg.DNATTarget) == nil {
return nil, fmt.Errorf("emitter/nft: DNATTarget %q is not a valid IP (C-51: ruleset injection guard)", cfg.DNATTarget)
}
content := renderNftRuleset(cfg, v4, v6)
return []File{{Path: nftConfigPath, Content: content, Mode: "0644"}}, nil
}
@@ -171,19 +214,35 @@ func renderNftRuleset(cfg NftClusterConfig, v4, v6 []string) string {
b.WriteString(" }\n")
b.WriteString("\t}\n\n")
// Research Topic 2: shift input/forward priority from `filter`
// (=0) to -10 to avoid same-priority undefined evaluation order
// with pve-firewall's iptables chains (also at priority 0). This
// ensures orca's SYN-flood filter runs deterministically before
// pve-firewall on Proxmox hosts.
b.WriteString("\tchain input {\n")
b.WriteString("\t\ttype filter hook input priority filter; policy accept;\n")
b.WriteString("\t\ttype filter hook input priority -10; policy accept;\n")
b.WriteString("\t\tct state invalid drop\n")
b.WriteString("\t\tct state established,related accept\n")
b.WriteString("\t\ttcp dport 443 tcp-flags != syn,rst,ack,fin notrack drop\n")
b.WriteString("\t}\n\n")
b.WriteString("\tchain prerouting {\n")
b.WriteString("\t\ttype nat hook prerouting priority -100; policy accept;\n")
b.WriteString("\t\ttcp dport 443 dnat to 127.0.0.1:8443\n")
b.WriteString("\t\ttcp dport 80 dnat to 127.0.0.1:8080\n")
b.WriteString(fmt.Sprintf("\t\ttcp dport 443 dnat to %s:8443\n", cfg.DNATTarget))
b.WriteString(fmt.Sprintf("\t\ttcp dport 80 dnat to %s:8080\n", cfg.DNATTarget))
b.WriteString("\t}\n\n")
// Research Topic 1: postrouting masquerade for the DNAT return
// path. Scoped to `ip saddr 127.0.0.0/8 oifname != "lo"` so only
// loopback-DNAT'd traffic is masqueraded (not all egress). This is
// the canonical "hairpin NAT" / "loopback DNAT return path" rule.
// Priority 100 = NF_IP_PRI_SRCNAT (standard srcnat priority).
if cfg.EnableSNAT {
b.WriteString("\tchain postrouting {\n")
b.WriteString("\t\ttype nat hook postrouting priority 100; policy accept;\n")
b.WriteString("\t\tip saddr 127.0.0.0/8 oifname != \"lo\" masquerade\n")
b.WriteString("\t}\n\n")
}
b.WriteString("\tchain forward {\n")
b.WriteString("\t\ttype filter hook forward priority filter; policy accept;\n")
b.WriteString("\t\ttype filter hook forward priority -10; policy accept;\n")
b.WriteString(fmt.Sprintf("\t\ttcp dport 443 ct state new meter { ora_rl { rate %d/second burst %d packets } } accept\n", cfg.RateLimit, cfg.RateBurst))
b.WriteString("\t}\n")
b.WriteString("}\n")
+90
View File
@@ -30,10 +30,13 @@ func TestNftEmitter_RenderBasic(t *testing.T) {
"127.0.0.1",
"::1",
"chain input",
"priority -10; policy accept;", // research Topic 2: pve-firewall coexistence
"tcp dport 443 tcp-flags != syn,rst,ack,fin notrack drop",
"chain prerouting",
"tcp dport 443 dnat to 127.0.0.1:8443",
"tcp dport 80 dnat to 127.0.0.1:8080",
"chain postrouting", // research Topic 1: SNAT masquerade
"ip saddr 127.0.0.0/8 oifname != \"lo\" masquerade", // scoped to loopback DNAT return
"chain forward",
"rate 100/second burst 200 packets",
"ora_rl",
@@ -135,3 +138,90 @@ func TestNftEmitter_TrustedProbesSplitV4V6(t *testing.T) {
t.Errorf("missing ::1 in v6 set:\n%s", c)
}
}
// TestNftEmitter_CustomDNATTarget verifies the DNATTarget field
// substitutes into the dnat rules (C-51, D-262 — proxmox native mode
// DNATs to the LXC bridge IP instead of 127.0.0.1).
func TestNftEmitter_CustomDNATTarget(t *testing.T) {
files, err := (NftEmitter{}).RenderNftConfig(NftClusterConfig{DNATTarget: "10.99.0.10"})
if err != nil {
t.Fatalf("Render: %v", err)
}
c := files[0].Content
if !strings.Contains(c, "dnat to 10.99.0.10:8443") {
t.Errorf("missing custom DNAT target :8443:\n%s", c)
}
if !strings.Contains(c, "dnat to 10.99.0.10:8080") {
t.Errorf("missing custom DNAT target :8080:\n%s", c)
}
if strings.Contains(c, "127.0.0.1:8443") {
t.Errorf("default 127.0.0.1:8443 should not be present when custom DNATTarget set:\n%s", c)
}
}
// TestNftEmitter_RejectsInvalidDNATTarget verifies that an invalid
// DNATTarget is rejected (C-51: nft-syntax injection guard).
func TestNftEmitter_RejectsInvalidDNATTarget(t *testing.T) {
bad := []string{
"not-an-ip",
"127.0.0.1; flush ruleset",
"$(whoami)",
"10.0.0.0/33",
"",
}
for _, b := range bad {
// Empty string gets defaulted to 127.0.0.1, so it won't error.
// Test only non-empty invalid values.
if b == "" {
continue
}
_, err := (NftEmitter{}).RenderNftConfig(NftClusterConfig{DNATTarget: b})
if err == nil {
t.Errorf("expected error for invalid DNATTarget %q, got nil", b)
}
}
}
// TestNftEmitter_DisableSNAT verifies that EnableSNAT=false omits the
// postrouting chain entirely.
func TestNftEmitter_DisableSNAT(t *testing.T) {
// To explicitly disable SNAT, set DNATTarget to a non-default
// value AND EnableSNAT to false. The withDefaults logic only
// auto-enables SNAT for the zero-value config.
files, err := (NftEmitter{}).RenderNftConfig(NftClusterConfig{
DNATTarget: "10.99.0.10",
EnableSNAT: false,
})
if err != nil {
t.Fatalf("Render: %v", err)
}
c := files[0].Content
if strings.Contains(c, "chain postrouting") {
t.Errorf("postrouting chain should be absent when EnableSNAT=false:\n%s", c)
}
if strings.Contains(c, "masquerade") {
t.Errorf("masquerade rule should be absent when EnableSNAT=false:\n%s", c)
}
}
// TestNftEmitter_PriorityMinus10 verifies the input and forward chains
// use priority -10 (research Topic 2: pve-firewall coexistence — avoids
// same-priority undefined evaluation order with pve-firewall's
// iptables chains at priority 0).
func TestNftEmitter_PriorityMinus10(t *testing.T) {
files, _ := (NftEmitter{}).RenderNftConfig(NftClusterConfig{})
c := files[0].Content
if !strings.Contains(c, "hook input priority -10;") {
t.Errorf("input chain should use priority -10:\n%s", c)
}
if !strings.Contains(c, "hook forward priority -10;") {
t.Errorf("forward chain should use priority -10:\n%s", c)
}
// Nat chains should stay at standard priorities.
if !strings.Contains(c, "hook prerouting priority -100;") {
t.Errorf("prerouting chain should use priority -100:\n%s", c)
}
if !strings.Contains(c, "hook postrouting priority 100;") {
t.Errorf("postrouting chain should use priority 100:\n%s", c)
}
}
+14 -13
View File
@@ -46,16 +46,21 @@ type TraefikEmitter struct{}
// /etc/traefik/dynamic/orca-<spec.Name>.yaml.
const traefikDynamicDir = "/etc/traefik/dynamic"
// traefikRouterTLSCertResolver is the Traefik cert-resolver name that
// the orca step-ca integration configures on the Traefik static config
// (P10 / v0.10 wires the step-ca root into this resolver). The
// dynamic-config file references it by name.
// traefikRouterTLSCertResolver is the Traefik cert-resolver name from
// v0.11. As of v0.14 (RESEARCH_v0.14 Topic 4), traefik v3.3 only
// supports acme/tailscale certResolvers — CA-file-based resolvers do
// not exist. The dynamic config now emits `tls: {}` instead. Real
// mTLS via dynamic tls.certificates + clientAuth.caFiles is deferred
// to v0.15. This constant is retained for documentation.
//
// Deprecated: v0.14 removed certResolver from the dynamic config.
const traefikRouterTLSCertResolver = "orca"
// defaultTrustDomain is the SPIFFE trust domain used in the rendered
// TLS stanza when the spec does not carry an explicit trust domain.
// The step-ca provisioner (P10) overrides this at render time via the
// node argument; for P02 the emitter renders the placeholder.
// defaultTrustDomain is the SPIFFE trust domain. v0.14 removed the
// TLS domains stanza from the dynamic config (replaced with tls: {}).
// Retained for documentation; will be used by v0.15 mTLS.
//
// Deprecated: v0.14 removed TLS domains from the dynamic config.
const defaultTrustDomain = "cluster.orca.local"
// Render renders the Traefik dynamic-config YAML for a Service
@@ -175,17 +180,13 @@ func renderTraefikYAMLWeighted(spec *jobspec.WorkloadSpec, node *Node, drain boo
routerName := "orca-" + spec.Name
serviceName := "orca-" + spec.Name
rule := fmt.Sprintf("PathPrefix(\"/%s\")", spec.Name)
trustDomain := defaultTrustDomain
b.WriteString("http:\n")
b.WriteString(" routers:\n")
b.WriteString(fmt.Sprintf(" %s:\n", routerName))
b.WriteString(fmt.Sprintf(" rule: %s\n", rule))
b.WriteString(fmt.Sprintf(" service: %s\n", serviceName))
b.WriteString(" tls:\n")
b.WriteString(fmt.Sprintf(" certResolver: %s\n", traefikRouterTLSCertResolver))
b.WriteString(" domains:\n")
b.WriteString(fmt.Sprintf(" - main: %q\n", trustDomain))
b.WriteString(" tls: {}\n")
b.WriteString(" services:\n")
b.WriteString(fmt.Sprintf(" %s:\n", serviceName))
b.WriteString(" loadBalancer:\n")
+2 -5
View File
@@ -53,11 +53,8 @@ func TestTraefikEmitter_RenderBasic(t *testing.T) {
if !strings.Contains(c, "unix:///run/orca/alloc-node-1/port-http.sock") {
t.Errorf("content missing socket server URL\n%s", c)
}
if !strings.Contains(c, "certResolver: orca") {
t.Errorf("content missing 'certResolver: orca'\n%s", c)
}
if !strings.Contains(c, "domains:") {
t.Errorf("content missing TLS domains\n%s", c)
if !strings.Contains(c, "tls: {}") {
t.Errorf("content missing 'tls: {}' (v0.14: certResolver dropped, tls empty stanza)\n%s", c)
}
if !strings.Contains(c, "healthCheck:") {
t.Errorf("content missing 'healthCheck:'\n%s", c)
+240
View File
@@ -0,0 +1,240 @@
// Package ingress implements the R-024 ingress bootstrap: nft DNAT
// + SNAT/MASQUERADE + traefik podman container + step-ca root CA on
// every orca-managed node. The bootstrap is idempotent and non-fatal
// on each step (offline host tolerance — same as v0.13 traefik install).
//
// BootstrapLocalIngress runs on the lead (during `orca init`).
// BootstrapRemoteIngress runs on workers (during `orca node join`).
package ingress
import (
"context"
"crypto/rand"
"encoding/hex"
"fmt"
"log/slog"
"os"
"os/exec"
"path/filepath"
"strings"
"git.cloudinit.dev/coreci/orca/internal/certpaths"
"git.cloudinit.dev/coreci/orca/internal/emitter"
"git.cloudinit.dev/coreci/orca/internal/traefik"
)
// BootstrapLocalIngress ensures the complete ingress stack is running
// on the local host (lead node). It is called from `orca init` after
// EnsureTraefikContainerLocal.
//
// Steps (each non-fatal — logs a warning and continues):
// 1. mkdir -p /etc/traefik/dynamic /etc/orca
// 2. Push cluster root CA to /etc/orca/step-ca-root.crt (C-60:
// certpaths.CACertPath(), not CAPath)
// 3. Render static config via emitter.RenderTraefikStaticConfig to
// /etc/traefik/traefik.yml (preserves traefik-on-public-ip opt-out,
// C-58)
// 4. Render orca.nft via emitter.NftEmitter.RenderNftConfig + write
// to /etc/nftables.d/orca.nft
// 5. Pre-create nft table (C-55: avoids flush-table error on first
// apply)
// 6. Apply: nft -f /etc/nftables.d/orca.nft
func BootstrapLocalIngress(ctx context.Context, version string) error {
var errs []error
log := slog.Default()
// Step 1: ensure directories.
for _, dir := range []string{"/etc/traefik/dynamic", "/etc/orca"} {
if err := os.MkdirAll(dir, 0o755); err != nil {
log.Warn("ingress: mkdir failed", "dir", dir, "err", err)
errs = append(errs, fmt.Errorf("mkdir %s: %w", dir, err))
}
}
// Step 2: push cluster root CA (C-60: CACertPath, not CAPath).
caPath := certpaths.CACertPath()
if caData, err := os.ReadFile(caPath); err == nil {
if err := os.WriteFile("/etc/orca/step-ca-root.crt", caData, 0o644); err != nil {
log.Warn("ingress: step-ca root CA write failed", "err", err)
errs = append(errs, fmt.Errorf("write step-ca-root.crt: %w", err))
}
} else {
// CA may not exist yet (fresh init before step-ca). Write a
// placeholder so the podman volume mount doesn't fail.
_ = os.WriteFile("/etc/orca/step-ca-root.crt", []byte{}, 0o644)
log.Warn("ingress: step-ca root CA not found, wrote placeholder", "path", caPath)
}
// Step 3: render + write static config (C-58).
staticFiles, err := emitter.TraefikEmitter{}.RenderTraefikStaticConfig(emitter.TraefikStaticOpts{})
if err != nil {
log.Warn("ingress: render traefik static config failed", "err", err)
errs = append(errs, fmt.Errorf("render traefik static: %w", err))
} else {
for _, f := range staticFiles {
if err := os.WriteFile(f.Path, []byte(f.Content), 0o644); err != nil {
log.Warn("ingress: write traefik static config failed", "path", f.Path, "err", err)
errs = append(errs, fmt.Errorf("write %s: %w", f.Path, err))
}
}
}
// Step 4: render + write nft ruleset.
nftFiles, err := emitter.NftEmitter{}.RenderNftConfig(emitter.NftClusterConfig{})
if err != nil {
log.Warn("ingress: render nft config failed", "err", err)
errs = append(errs, fmt.Errorf("render nft: %w", err))
} else {
for _, f := range nftFiles {
dir := filepath.Dir(f.Path)
_ = os.MkdirAll(dir, 0o755)
if err := os.WriteFile(f.Path, []byte(f.Content), 0o644); err != nil {
log.Warn("ingress: write nft config failed", "path", f.Path, "err", err)
errs = append(errs, fmt.Errorf("write %s: %w", f.Path, err))
}
}
// Step 5: pre-create nft table (C-55: flush table on non-existent
// table errors — pre-create avoids the first-apply failure).
_ = exec.CommandContext(ctx, "nft", "add", "table", "inet", "orca-ingress").Run()
// Step 6: apply nft ruleset.
if out, err := exec.CommandContext(ctx, "nft", "-f", nftConfigPath).CombinedOutput(); err != nil {
log.Warn("ingress: nft apply failed", "err", err, "output", string(out))
errs = append(errs, fmt.Errorf("nft -f: %w (output: %s)", err, string(out)))
}
}
// Ensure the podman traefik container is running (Step 7 — C-50:
// installs podman if absent).
if err := traefik.EnsureTraefikContainerLocal(ctx, version); err != nil {
log.Warn("ingress: ensure traefik container failed", "err", err)
errs = append(errs, fmt.Errorf("ensure traefik container: %w", err))
}
if len(errs) > 0 {
return fmt.Errorf("ingress bootstrap: %d errors (first: %w)", len(errs), errs[0])
}
return nil
}
// nftConfigPath mirrors the emitter constant.
const nftConfigPath = "/etc/nftables.d/orca.nft"
// RemoteExecFunc runs a command on a remote host and returns combined
// output. Same signature as traefik.RemoteExecFunc.
type RemoteExecFunc func(cmd string) ([]byte, error)
// BootstrapRemoteIngress ensures the complete ingress stack is running
// on a remote host (linux worker). It is called from
// `orca node join --type linux` after the user setup.
//
// Steps (each non-fatal — logs a warning and continues):
// 1. mkdir -p /etc/traefik/dynamic /etc/orca (remote)
// 2. Push step-ca root CA to remote /etc/orca/step-ca-root.crt
// (C-60: certpaths.CACertPath)
// 3. Render + write static config to remote /etc/traefik/traefik.yml
// (C-58: preserves traefik-on-public-ip opt-out)
// 4. Render + write nft ruleset to remote /etc/nftables.d/orca.nft
// 5. Pre-create nft table (C-55: avoids first-apply flush-table error)
// 6. Apply: nft -f (remote)
// 7. Ensure podman traefik container running (remote)
func BootstrapRemoteIngress(ctx context.Context, version string, execFn RemoteExecFunc) error {
var errs []error
log := slog.Default()
// Step 1: ensure directories.
if _, err := execFn("mkdir -p /etc/traefik/dynamic /etc/orca"); err != nil {
log.Warn("ingress: remote mkdir failed", "err", err)
errs = append(errs, fmt.Errorf("remote mkdir: %w", err))
}
// Step 2: push step-ca root CA (C-60: CACertPath, not CAPath).
if caData, err := os.ReadFile(certpaths.CACertPath()); err == nil {
if err := remoteWriteFile(execFn, "/etc/orca/step-ca-root.crt", caData, "0644"); err != nil {
log.Warn("ingress: remote step-ca CA write failed", "err", err)
errs = append(errs, fmt.Errorf("remote write step-ca-root.crt: %w", err))
}
} else {
// Write a placeholder so the podman volume mount doesn't fail.
_ = remoteWriteFile(execFn, "/etc/orca/step-ca-root.crt", []byte{}, "0644")
log.Warn("ingress: step-ca root CA not found locally, wrote remote placeholder")
}
// Step 3: render + write static config (C-58).
staticFiles, err := emitter.TraefikEmitter{}.RenderTraefikStaticConfig(emitter.TraefikStaticOpts{})
if err != nil {
log.Warn("ingress: render traefik static config failed", "err", err)
errs = append(errs, fmt.Errorf("render traefik static: %w", err))
} else {
for _, f := range staticFiles {
_, _ = execFn(fmt.Sprintf("mkdir -p %s", filepath.Dir(f.Path)))
if err := remoteWriteFile(execFn, f.Path, []byte(f.Content), f.Mode); err != nil {
log.Warn("ingress: remote write traefik static config failed", "path", f.Path, "err", err)
errs = append(errs, fmt.Errorf("remote write %s: %w", f.Path, err))
}
}
}
// Step 4: render + write nft ruleset.
nftFiles, err := emitter.NftEmitter{}.RenderNftConfig(emitter.NftClusterConfig{})
if err != nil {
log.Warn("ingress: render nft config failed", "err", err)
errs = append(errs, fmt.Errorf("render nft: %w", err))
} else {
for _, f := range nftFiles {
_, _ = execFn(fmt.Sprintf("mkdir -p %s", filepath.Dir(f.Path)))
if err := remoteWriteFile(execFn, f.Path, []byte(f.Content), f.Mode); err != nil {
log.Warn("ingress: remote write nft config failed", "path", f.Path, "err", err)
errs = append(errs, fmt.Errorf("remote write %s: %w", f.Path, err))
}
}
// Step 5: pre-create nft table (C-55).
_, _ = execFn("nft add table inet orca-ingress 2>/dev/null || true")
// Step 6: apply nft ruleset.
if out, err := execFn("nft -f /etc/nftables.d/orca.nft 2>&1"); err != nil {
log.Warn("ingress: remote nft apply failed", "err", err, "output", string(out))
errs = append(errs, fmt.Errorf("remote nft -f: %w (output: %s)", err, string(out)))
}
}
// Step 7: ensure podman traefik container (C-50).
traefikExecFn := traefik.RemoteExecFunc(execFn)
if err := traefik.EnsureTraefikContainerRemote(ctx, version, traefikExecFn); err != nil {
log.Warn("ingress: remote ensure traefik container failed", "err", err)
errs = append(errs, fmt.Errorf("remote ensure traefik container: %w", err))
}
if len(errs) > 0 {
return fmt.Errorf("remote ingress bootstrap: %d errors (first: %w)", len(errs), errs[0])
}
return nil
}
// remoteWriteFile writes content to a remote path via a heredoc
// (same pattern as sshpush.idempotency.writeFile). The heredoc
// delimiter is a random hex string verified absent from the content
// (F9 injection guard).
func remoteWriteFile(execFn RemoteExecFunc, path string, content []byte, mode string) error {
// Generate a random delimiter unlikely to be in the content.
delim := "EOF_"
for {
b := make([]byte, 8)
if _, err := rand.Read(b); err != nil {
return fmt.Errorf("rand: %w", err)
}
delim = "EOF_" + hex.EncodeToString(b)
if !strings.Contains(string(content), delim) {
break
}
}
dir := filepath.Dir(path)
cmd := fmt.Sprintf("mkdir -p %s && cat > %s <<'%s'\n%s\n%s\nchmod %s %s",
dir, path, delim, string(content), delim, mode, path)
if out, err := execFn(cmd); err != nil {
return fmt.Errorf("remote write %s: %w (output: %s)", path, err, string(out))
}
return nil
}
+17 -14
View File
@@ -32,9 +32,9 @@ import (
"golang.org/x/crypto/ssh"
"git.cloudinit.dev/coreci/orca/internal/certpaths"
"git.cloudinit.dev/coreci/orca/internal/ingress"
"git.cloudinit.dev/coreci/orca/internal/proxmox"
"git.cloudinit.dev/coreci/orca/internal/security"
"git.cloudinit.dev/coreci/orca/internal/traefik"
)
// DefaultSSHUser is the default SSH username for the initial connection.
@@ -48,13 +48,13 @@ const DefaultSSHPort = 22
// Options configures a Linux worker bootstrap run.
type Options struct {
Host string
SSHUser string
SSHKeyPath string
OrcaUser string
SSHPort int
HostKeyFingerprint string
Logger *slog.Logger
Host string
SSHUser string
SSHKeyPath string
OrcaUser string
SSHPort int
HostKeyFingerprint string
Logger *slog.Logger
}
// Result is the outcome of a successful bootstrap.
@@ -157,8 +157,10 @@ func BootstrapLinux(ctx context.Context, opts Options) (*Result, error) {
}
opts.Logger.Info("linux bootstrap: user created", "user", opts.OrcaUser)
// Step 4d: Install Traefik on the remote host (REQ-165, Phase B).
// Traefik is the data-plane ingress; SSH is control plane only.
// Step 4d: Ensure complete ingress stack on the remote host (R-024).
// Renders+applies nft DNAT/SNAT, pushes step-ca root CA, renders+
// writes traefik static config, ensures podman container running.
// All non-fatal (offline host tolerance).
sshExecFn := func(cmd string) ([]byte, error) {
session, err := client.NewSession()
if err != nil {
@@ -167,8 +169,10 @@ func BootstrapLinux(ctx context.Context, opts Options) (*Result, error) {
defer session.Close()
return session.CombinedOutput(cmd)
}
if err := traefik.InstallRemote("", sshExecFn); err != nil {
opts.Logger.Warn("linux bootstrap: traefik install failed", "err", err)
ctx, cancelIngress := context.WithTimeout(ctx, 120*time.Second)
defer cancelIngress()
if err := ingress.BootstrapRemoteIngress(ctx, "", ingress.RemoteExecFunc(sshExecFn)); err != nil {
opts.Logger.Warn("linux bootstrap: ingress bootstrap failed", "err", err)
}
// Step 5: Create the drift-events directory.
@@ -188,7 +192,7 @@ func BootstrapLinux(ctx context.Context, opts Options) (*Result, error) {
return &Result{
NodeName: opts.Host,
NodeAddress: fmt.Sprintf("%s:8443", opts.Host),
NodeAddress: fmt.Sprintf("%s:8443", opts.Host),
HostKeyFingerprint: hostKeyFP,
}, nil
}
@@ -226,5 +230,4 @@ func pinnedHostKeyCallback(expectedSHA256Base64 string, capturedKey *ssh.PublicK
return cb, nil
}
var _ = security.WriteAtomic
+7 -5
View File
@@ -247,11 +247,13 @@ func BootstrapProxmox(ctx context.Context, opts Options) (*Result, error) {
return nil, fmt.Errorf("validate sudoers: %w", err)
}
// Step 9a: Install Traefik on the Proxmox host (REQ-165, Phase B).
// Traefik runs on the PVE OS as the data-plane ingress; SSH is
// control plane only. Idempotent: skips if binary already exists.
if err := traefik.InstallRemote("", runRemote); err != nil {
log.Warn("proxmox.traefik_install_failed", "err", err)
// Step 9a: Ensure orca-traefik podman container on the Proxmox host
// (REQ-172, R-024). Replaces v0.13 binary+systemd install.
// In native mode (default for v0.14 P5), the container runs inside
// an LXC with nesting. For now, this installs on the PVE host OS.
// Idempotent: no-op if container already running.
if err := traefik.EnsureTraefikContainerRemote(ctx, "", runRemote); err != nil {
log.Warn("proxmox.traefik_container_failed", "err", err)
}
// Step 9b: Download default LXC template (REQ-167, Phase C).
+202 -52
View File
@@ -1,78 +1,228 @@
package traefik
import (
"context"
"fmt"
"os/exec"
"strings"
)
// DefaultVersion is the traefik version tag for the orca-traefik image.
const DefaultVersion = "v3.3.0"
func downloadURL(version string) string {
return fmt.Sprintf("https://github.com/traefik/traefik/releases/download/%s/traefik_%s_linux_amd64.tar.gz", version, version)
}
// DefaultImage is the full image reference for the orca-traefik container.
// The tag is resolved at runtime from the orca version (or "latest" for
// dev builds). The image is built from Dockerfile.traefik and published
// per release (REQ-171).
const DefaultImage = "git.cloudinit.dev/coreci/orca-traefik"
func InstallLocal(version string) error {
if version == "" {
version = DefaultVersion
}
if _, err := exec.LookPath("traefik"); err == nil {
ensureDirs()
return nil
}
url := downloadURL(version)
cmd := exec.Command("bash", "-c",
fmt.Sprintf(`curl -fsSL %s | tar -xzf - -C /usr/local/bin/ traefik && chmod +x /usr/local/bin/traefik`, url))
if out, err := cmd.CombinedOutput(); err != nil {
return fmt.Errorf("download traefik %s: %w (output: %s)", version, err, string(out))
}
ensureDirs()
writeSystemdUnit()
_ = exec.Command("systemctl", "daemon-reload").Run()
_ = exec.Command("systemctl", "enable", "--now", "orca-traefik").Run()
return nil
}
// ContainerName is the podman container name for the traefik data plane.
const ContainerName = "orca-traefik"
// RemoteExecFunc runs a command on a remote host and returns combined
// output. It is the same signature used by the v0.13 binary installer
// and by the proxmox/linux bootstrap SSH sessions.
type RemoteExecFunc func(cmd string) ([]byte, error)
func InstallRemote(version string, execFn RemoteExecFunc) error {
if version == "" {
version = DefaultVersion
// ImageRef returns the full image:tag reference for the orca-traefik
// container. If version is empty or "dev"/"0.1.0-dev", it falls back to
// "latest" (dev builds don't have a published tag).
func ImageRef(version string) string {
tag := version
if tag == "" || tag == "dev" || tag == "0.1.0-dev" || strings.HasSuffix(tag, "-dev") {
return fmt.Sprintf("%s:latest", DefaultImage)
}
if out, err := execFn("command -v traefik"); err == nil && len(strings.TrimSpace(string(out))) > 0 {
_, _ = execFn("mkdir -p /etc/traefik/dynamic")
tag = strings.TrimPrefix(tag, "v")
return fmt.Sprintf("%s:v%s", DefaultImage, tag)
}
// podmanRunArgs returns the podman run arguments for the traefik
// container. The container uses --network host so traefik binds
// 127.0.0.1:8080/8443 directly on the host (or LXC) loopback. nft
// DNATs public :443/:80 to those loopback ports (R-017/R-024).
//
// Volume mounts (no SELinux :Z flag — research finding Topic 7):
// - /etc/traefik/traefik.yml:ro — static config (overrides baked default; C-58)
// - /etc/traefik/dynamic:ro — dynamic config (orca writes atomically via SSH-push)
// - /etc/orca/step-ca-root.crt:ro — step-ca root CA (for future mTLS; v0.14 uses tls:{})
func podmanRunArgs(imageRef string) []string {
return []string{
"run", "-d",
"--name", ContainerName,
"--restart=unless-stopped",
"--network", "host",
"-v", "/etc/traefik/traefik.yml:/etc/traefik/traefik.yml:ro",
"-v", "/etc/traefik/dynamic:/etc/traefik/dynamic:ro",
"-v", "/etc/orca/step-ca-root.crt:/etc/orca/step-ca-root.crt:ro",
imageRef,
}
}
// EnsureTraefikContainerLocal ensures the orca-traefik podman container
// is running on the local host. It is idempotent:
// 1. If the container is running → no-op.
// 2. If the container exists but is stopped → start it.
// 3. If the container does not exist → pull the image + run it.
//
// C-50: if podman is not installed, attempts apt-get install. If that
// fails, returns an error with install instructions.
//
// C-57: if a legacy v0.13 systemd service exists (orca-traefik.service),
// it is stopped, disabled, and removed before starting the container.
func EnsureTraefikContainerLocal(ctx context.Context, version string) error {
imageRef := ImageRef(version)
if err := ensurePodmanLocal(ctx); err != nil {
return err
}
if err := removeLegacySystemdUnitLocal(ctx); err != nil {
// Non-fatal: legacy unit may not exist on fresh installs.
_ = err
}
if err := ensureDirsLocal(); err != nil {
return fmt.Errorf("traefik: ensure dirs: %w", err)
}
return reconcileContainerLocal(ctx, imageRef)
}
// EnsureTraefikContainerRemote ensures the orca-traefik podman container
// is running on a remote host (via SSH exec). Same idempotent logic as
// EnsureTraefikContainerLocal but over the provided exec function.
func EnsureTraefikContainerRemote(ctx context.Context, version string, execFn RemoteExecFunc) error {
imageRef := ImageRef(version)
if err := ensurePodmanRemote(execFn); err != nil {
return err
}
if err := removeLegacySystemdUnitRemote(execFn); err != nil {
_ = err // non-fatal
}
if _, err := execFn("mkdir -p /etc/traefik/dynamic /etc/orca"); err != nil {
return fmt.Errorf("traefik: ensure remote dirs: %w", err)
}
return reconcileContainerRemote(execFn, imageRef)
}
// ensurePodmanLocal checks if podman is installed locally and attempts
// to install it if absent (C-50).
func ensurePodmanLocal(ctx context.Context) error {
if _, err := exec.LookPath("podman"); err == nil {
return nil
}
url := downloadURL(version)
installCmd := fmt.Sprintf(`curl -fsSL %s | tar -xzf - -C /usr/local/bin/ traefik && chmod +x /usr/local/bin/traefik && mkdir -p /etc/traefik/dynamic`, url)
if out, err := execFn(installCmd); err != nil {
return fmt.Errorf("download traefik on remote: %w (output: %s)", err, string(out))
// Attempt apt-get install (Ubuntu/Debian).
cmd := exec.CommandContext(ctx, "bash", "-c",
"apt-get update -qq && apt-get install -y -qq podman conmon crun fuse-overlayfs 2>&1")
if out, err := cmd.CombinedOutput(); err != nil {
return fmt.Errorf("podman not found and apt-get install failed: %w (output: %s).\nInstall podman manually: apt-get install podman conmon crun fuse-overlayfs", err, string(out))
}
unit := systemdUnitContent()
_, _ = execFn(fmt.Sprintf(`echo '%s' > /etc/systemd/system/orca-traefik.service`, unit))
_, _ = execFn("systemctl daemon-reload && systemctl enable --now orca-traefik")
return nil
}
func ensureDirs() {
_ = exec.Command("mkdir", "-p", "/etc/traefik/dynamic").Run()
// ensurePodmanRemote checks if podman is installed on the remote host
// and attempts to install it if absent (C-50).
func ensurePodmanRemote(execFn RemoteExecFunc) error {
if out, err := execFn("command -v podman"); err == nil && len(strings.TrimSpace(string(out))) > 0 {
return nil
}
// Attempt apt-get install on the remote host.
cmd := "apt-get update -qq && apt-get install -y -qq podman conmon crun fuse-overlayfs 2>&1"
if out, err := execFn(cmd); err != nil {
return fmt.Errorf("podman not found on remote and apt-get install failed: %w (output: %s)", err, string(out))
}
return nil
}
func writeSystemdUnit() {
_ = exec.Command("bash", "-c", fmt.Sprintf(`echo '%s' > /etc/systemd/system/orca-traefik.service`, systemdUnitContent())).Run()
// removeLegacySystemdUnitLocal stops, disables, and removes the legacy
// v0.13 orca-traefik.service systemd unit + /usr/local/bin/traefik
// binary (C-57). Idempotent — no-op if the unit doesn't exist.
func removeLegacySystemdUnitLocal(ctx context.Context) error {
// Check if the legacy unit exists.
if _, err := exec.CommandContext(ctx, "systemctl", "is-active", "orca-traefik.service").CombinedOutput(); err == nil {
// Unit is active or exists — stop + disable it.
_ = exec.CommandContext(ctx, "systemctl", "stop", "orca-traefik.service").Run()
_ = exec.CommandContext(ctx, "systemctl", "disable", "orca-traefik.service").Run()
}
// Remove the unit file and binary.
_ = exec.CommandContext(ctx, "rm", "-f", "/etc/systemd/system/orca-traefik.service").Run()
_ = exec.CommandContext(ctx, "rm", "-f", "/usr/local/bin/traefik").Run()
_ = exec.CommandContext(ctx, "systemctl", "daemon-reload").Run()
return nil
}
func systemdUnitContent() string {
return `[Unit]
Description=Orca Traefik Data Plane
After=network.target
[Service]
Type=simple
ExecStart=/usr/local/bin/traefik --configFile=/etc/traefik/traefik.yml
Restart=on-failure
RestartSec=5s
[Install]
WantedBy=multi-user.target`
// removeLegacySystemdUnitRemote is the remote SSH variant (C-57).
func removeLegacySystemdUnitRemote(execFn RemoteExecFunc) error {
cmd := `systemctl is-active orca-traefik.service 2>/dev/null && systemctl stop orca-traefik.service 2>/dev/null; systemctl disable orca-traefik.service 2>/dev/null; rm -f /etc/systemd/system/orca-traefik.service /usr/local/bin/traefik; systemctl daemon-reload 2>/dev/null; true`
_, _ = execFn(cmd)
return nil
}
// ensureDirsLocal creates /etc/traefik/dynamic and /etc/orca locally.
func ensureDirsLocal() error {
if err := exec.Command("mkdir", "-p", "/etc/traefik/dynamic", "/etc/orca").Run(); err != nil {
return fmt.Errorf("mkdir: %w", err)
}
return nil
}
// reconcileContainerLocal implements the idempotent pull+run logic
// locally (C-50).
func reconcileContainerLocal(ctx context.Context, imageRef string) error {
// Check if the container is already running.
out, err := exec.CommandContext(ctx, "podman", "inspect", "--format", "{{.State.Running}}", ContainerName).CombinedOutput()
if err == nil {
v := strings.TrimSpace(string(out))
if v == "true" {
return nil // already running
}
// Container exists but is stopped — start it.
if _, err := exec.CommandContext(ctx, "podman", "start", ContainerName).CombinedOutput(); err != nil {
return fmt.Errorf("podman start %s: %w", ContainerName, err)
}
return nil
}
// Container does not exist — pull + run.
if out, err := exec.CommandContext(ctx, "podman", "pull", imageRef).CombinedOutput(); err != nil {
return fmt.Errorf("podman pull %s: %w (output: %s)", imageRef, err, string(out))
}
args := append([]string{}, podmanRunArgs(imageRef)...)
if out, err := exec.CommandContext(ctx, "podman", args...).CombinedOutput(); err != nil {
return fmt.Errorf("podman run: %w (output: %s)", err, string(out))
}
// Enable podman-restart.service for reboot persistence (research Topic 6).
_ = exec.CommandContext(ctx, "systemctl", "enable", "--now", "podman-restart.service").Run()
return nil
}
// reconcileContainerRemote implements the idempotent pull+run logic
// over SSH exec.
func reconcileContainerRemote(execFn RemoteExecFunc, imageRef string) error {
// Check if the container is already running.
out, err := execFn(fmt.Sprintf("podman inspect --format '{{.State.Running}}' %s 2>/dev/null", ContainerName))
if err == nil {
v := strings.TrimSpace(string(out))
if v == "true" {
return nil // already running
}
// Container exists but is stopped — start it.
if _, err := execFn(fmt.Sprintf("podman start %s 2>/dev/null", ContainerName)); err != nil {
return fmt.Errorf("podman start %s: %w", ContainerName, err)
}
return nil
}
// Container does not exist — pull + run.
if out, err := execFn(fmt.Sprintf("podman pull %s", shellQuote(imageRef))); err != nil {
return fmt.Errorf("podman pull %s: %w (output: %s)", imageRef, err, string(out))
}
runArgs := strings.Join(podmanRunArgs(imageRef), " ")
if out, err := execFn(fmt.Sprintf("podman %s", runArgs)); err != nil {
return fmt.Errorf("podman run: %w (output: %s)", err, string(out))
}
// Enable podman-restart.service for reboot persistence (research Topic 6).
_, _ = execFn("systemctl enable --now podman-restart.service 2>/dev/null || true")
return nil
}
// shellQuote wraps a string in single quotes for shell-safe usage.
func shellQuote(s string) string {
return "'" + strings.ReplaceAll(s, "'", "'\\''") + "'"
}
+166
View File
@@ -0,0 +1,166 @@
package traefik
import (
"context"
"strings"
"testing"
)
func TestImageRef(t *testing.T) {
tests := []struct {
version string
want string
}{
{"v0.13.1", "git.cloudinit.dev/coreci/orca-traefik:v0.13.1"},
{"0.13.1", "git.cloudinit.dev/coreci/orca-traefik:v0.13.1"},
{"", "git.cloudinit.dev/coreci/orca-traefik:latest"},
{"dev", "git.cloudinit.dev/coreci/orca-traefik:latest"},
{"0.1.0-dev", "git.cloudinit.dev/coreci/orca-traefik:latest"},
{"v1.2.3-dev", "git.cloudinit.dev/coreci/orca-traefik:latest"},
}
for _, tt := range tests {
got := ImageRef(tt.version)
if got != tt.want {
t.Errorf("ImageRef(%q) = %q, want %q", tt.version, got, tt.want)
}
}
}
func TestPodmanRunArgs(t *testing.T) {
args := podmanRunArgs("git.cloudinit.dev/coreci/orca-traefik:v0.13.1")
joined := strings.Join(args, " ")
checks := []string{
"run -d",
"--name orca-traefik",
"--restart=unless-stopped",
"--network host",
"/etc/traefik/traefik.yml:/etc/traefik/traefik.yml:ro",
"/etc/traefik/dynamic:/etc/traefik/dynamic:ro",
"/etc/orca/step-ca-root.crt:/etc/orca/step-ca-root.crt:ro",
"git.cloudinit.dev/coreci/orca-traefik:v0.13.1",
}
for _, c := range checks {
if !strings.Contains(joined, c) {
t.Errorf("podmanRunArgs missing %q\nfull: %s", c, joined)
}
}
// Ensure no :Z flag (research Topic 7)
if strings.Contains(joined, ":Z") {
t.Errorf("podmanRunArgs should NOT contain :Z SELinux flag\nfull: %s", joined)
}
// Ensure --restart=always is NOT used (research Topic 6)
if strings.Contains(joined, "--restart=always") {
t.Errorf("podmanRunArgs should use --restart=unless-stopped, not --restart=always\nfull: %s", joined)
}
}
func TestEnsureTraefikContainerRemote_ContainerRunning(t *testing.T) {
var cmds []string
execFn := func(cmd string) ([]byte, error) {
cmds = append(cmds, cmd)
if strings.Contains(cmd, "podman inspect") {
return []byte("true\n"), nil
}
return []byte(""), nil
}
err := EnsureTraefikContainerRemote(context.Background(), "v0.13.1", execFn)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
// Should have checked inspect and found it running — no pull/run.
if len(cmds) < 1 {
t.Fatal("expected at least 1 command (inspect)")
}
for _, c := range cmds {
if strings.Contains(c, "podman pull") {
t.Errorf("should not pull when container is running: %s", c)
}
if strings.Contains(c, "podman run") {
t.Errorf("should not run when container is running: %s", c)
}
}
}
func TestEnsureTraefikContainerRemote_ContainerStopped(t *testing.T) {
var cmds []string
execFn := func(cmd string) ([]byte, error) {
cmds = append(cmds, cmd)
if strings.Contains(cmd, "podman inspect") {
return []byte("false\n"), nil // stopped
}
return []byte(""), nil
}
err := EnsureTraefikContainerRemote(context.Background(), "v0.13.1", execFn)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
// Should have started the container.
foundStart := false
for _, c := range cmds {
if strings.Contains(c, "podman start orca-traefik") {
foundStart = true
}
}
if !foundStart {
t.Errorf("expected 'podman start orca-traefik' when container is stopped\ncommands: %v", cmds)
}
}
func TestEnsureTraefikContainerRemote_ContainerAbsent(t *testing.T) {
var cmds []string
execFn := func(cmd string) ([]byte, error) {
cmds = append(cmds, cmd)
if strings.Contains(cmd, "podman inspect") {
return nil, &execError{"inspect failed: no such container"}
}
return []byte(""), nil
}
err := EnsureTraefikContainerRemote(context.Background(), "v0.13.1", execFn)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
// Should have pulled and run.
foundPull := false
foundRun := false
for _, c := range cmds {
if strings.Contains(c, "podman pull") {
foundPull = true
}
if strings.Contains(c, "podman run -d") {
foundRun = true
}
}
if !foundPull {
t.Errorf("expected 'podman pull' when container is absent\ncommands: %v", cmds)
}
if !foundRun {
t.Errorf("expected 'podman run -d' when container is absent\ncommands: %v", cmds)
}
}
func TestEnsureTraefikContainerRemote_LegacySystemdRemoval(t *testing.T) {
var cmds []string
execFn := func(cmd string) ([]byte, error) {
cmds = append(cmds, cmd)
if strings.Contains(cmd, "podman inspect") {
return []byte("true\n"), nil // container running
}
return []byte(""), nil
}
_ = EnsureTraefikContainerRemote(context.Background(), "v0.13.1", execFn)
// Should include legacy systemd unit removal command (C-57).
foundLegacyRemoval := false
for _, c := range cmds {
if strings.Contains(c, "orca-traefik.service") && strings.Contains(c, "stop") {
foundLegacyRemoval = true
}
}
if !foundLegacyRemoval {
t.Errorf("expected legacy systemd unit removal command (C-57)\ncommands: %v", cmds)
}
}
// execError is a simple error type for testing.
type execError struct{ msg string }
func (e *execError) Error() string { return e.msg }
+33
View File
@@ -211,3 +211,36 @@ else
info "✓ container image ${IMAGE}:${VERSION} published"
fi
fi
# --- publish orca-traefik container image (REQ-171, R-024) ----------------
# Custom traefik image with baked default static config. The host-side
# traefik.yml is mounted :ro at runtime to override the baked default.
# Skipped gracefully if docker is not on PATH (CI handles it via
# .coreci.yml container-publish-traefik step).
TRAEFIK_IMAGE="${CONTAINER_REGISTRY}/${CONTAINER_OWNER}/orca-traefik"
if ! command -v docker >/dev/null 2>&1; then
info "docker not found on PATH — skipping orca-traefik image publish (CI handles it)."
else
info "building orca-traefik image ${TRAEFIK_IMAGE}:${VERSION}..."
docker build \
-f Dockerfile.traefik \
-t "${TRAEFIK_IMAGE}:${VERSION}" \
-t "${TRAEFIK_IMAGE}:latest" \
"$REPO_ROOT"
if [ -z "${GITEA_TOKEN:-}" ]; then
info "GITEA_TOKEN not set — skipping orca-traefik push (image built locally only)."
else
if ! docker login "$CONTAINER_REGISTRY" -u cloudinit-bot --password-stdin <<<"$GITEA_TOKEN" 2>/dev/null; then
echo "$GITEA_TOKEN" | docker login "$CONTAINER_REGISTRY" -u cloudinit-bot --password-stdin
fi
info "pushing ${TRAEFIK_IMAGE}:${VERSION}..."
docker push "${TRAEFIK_IMAGE}:${VERSION}"
info "pushing ${TRAEFIK_IMAGE}:latest..."
docker push "${TRAEFIK_IMAGE}:latest"
docker logout "$CONTAINER_REGISTRY"
info "✓ orca-traefik image ${TRAEFIK_IMAGE}:${VERSION} published"
fi
fi