Compare commits
6 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| ecdba833d9 | |||
| 6e65eadaa5 | |||
| 1b7aac71f6 | |||
| ea42a17474 | |||
| 5013209e31 | |||
| dea472f443 |
@@ -872,3 +872,70 @@ scheduler.Schedule(spec, nodes) → emitter.Render(unit) → sshpush.Deploy(targ
|
|||||||
- IPv6 `net.JoinHostPort` in all SSH dial paths.
|
- IPv6 `net.JoinHostPort` in all SSH dial paths.
|
||||||
- Explicit timeouts on all SSH commands.
|
- Explicit timeouts on all SSH commands.
|
||||||
- Root SIGINT/SIGTERM handler for clean exit on non-watch commands.
|
- Root SIGINT/SIGTERM handler for clean exit on non-watch commands.
|
||||||
|
|
||||||
|
## v0.14 Deltas — Ingress Bootstrap Completeness (R-024)
|
||||||
|
|
||||||
|
### R-024: Traefik as Podman Container
|
||||||
|
|
||||||
|
Traefik runs exclusively as a podman container, deployed from the
|
||||||
|
custom `orca-traefik` image (published per release via `Dockerfile.traefik`
|
||||||
|
+ `scripts/release.sh` + `.coreci.yml container-publish-traefik`).
|
||||||
|
|
||||||
|
The v0.13 binary+systemd install (`internal/traefik/install.go`) is
|
||||||
|
replaced by an idempotent podman container reconciler
|
||||||
|
(`EnsureTraefikContainerLocal`/`Remote`). The container runs with
|
||||||
|
`--network host`, `--restart=unless-stopped`, and volume mounts for
|
||||||
|
`traefik.yml` (static config), `dynamic` (dynamic config), and
|
||||||
|
`step-ca-root.crt` (future mTLS). No SELinux `:Z` flag.
|
||||||
|
|
||||||
|
### Three Ingress Topologies
|
||||||
|
|
||||||
|
1. **Linux** (`orca init` / `orca node join --type linux`):
|
||||||
|
host → nft DNAT → podman traefik (host network).
|
||||||
|
`internal/ingress/bootstrap.go` → `BootstrapLocalIngress` /
|
||||||
|
`BootstrapRemoteIngress`.
|
||||||
|
|
||||||
|
2. **Proxmox Native** (`--ingress-mode native`, default):
|
||||||
|
PVE host → nft DNAT (target = LXC bridge IP) → LXC
|
||||||
|
(`--features nesting=1,keyctl=1,fuse=1`) → podman traefik.
|
||||||
|
`internal/proxmox/bootstrap.go` → `provisionNativeIngressLXC`.
|
||||||
|
|
||||||
|
3. **Proxmox Floating-IP** (`--ingress-mode floating-ip`):
|
||||||
|
LXC owns the floating IP (`net0 bridge=vmbr0,hwaddr=<mac>,
|
||||||
|
ip=<floating-ip>/<prefix>,gw=<gateway>`) → nft inside LXC →
|
||||||
|
podman traefik. The ingress LXC is registered as a `linux` node
|
||||||
|
(name=`ingress`) so `orca job run` pushes traefik dynamic config.
|
||||||
|
`internal/proxmox/ingress_lxc.go` → `ProvisionIngressLXC`.
|
||||||
|
|
||||||
|
### nft Emitter Changes
|
||||||
|
|
||||||
|
`internal/emitter/nft.go`:
|
||||||
|
- `DNATTarget` field (C-51: validated via `net.ParseIP`). Default
|
||||||
|
`127.0.0.1`; proxmox native uses LXC bridge IP.
|
||||||
|
- `EnableSNAT` field + postrouting masquerade chain: `ip saddr
|
||||||
|
127.0.0.0/8 oifname != "lo" masquerade` (research Topic 1).
|
||||||
|
- Input/forward chain priority shifted from `filter` (=0) to `-10`
|
||||||
|
(research Topic 2: pve-firewall coexistence — avoids same-priority
|
||||||
|
undefined evaluation order).
|
||||||
|
|
||||||
|
### TLS Model
|
||||||
|
|
||||||
|
v0.14 drops `certResolver: orca` from the dynamic config (traefik v3.3
|
||||||
|
only supports `acme`/`tailscale` resolvers, not CA-file-based). The
|
||||||
|
dynamic config emits `tls: {}` (traefik default cert). Real mTLS via
|
||||||
|
`tls.certificates` + `tls.options.default.clientAuth.caFiles` is
|
||||||
|
deferred to v0.15 (grill G-003, confidence 0.55 < 0.60).
|
||||||
|
|
||||||
|
### Migration 0009
|
||||||
|
|
||||||
|
`ALTER TABLE nodes ADD COLUMN ingress_mode TEXT NOT NULL DEFAULT '';`
|
||||||
|
Values: `""` (legacy), `"native"`, `"floating-ip"`. `IngressMode` field
|
||||||
|
on `model.Node`.
|
||||||
|
|
||||||
|
### New CLI
|
||||||
|
|
||||||
|
- `orca doctor ingress` — verifies podman container running, nft
|
||||||
|
DNAT+SNAT, dynamic dir, step-ca root CA.
|
||||||
|
- `--ingress-mode` flag on `orca node join --type proxmox`.
|
||||||
|
- `--floating-ip`, `--gateway`, `--mac`, `--net-prefix` flags for
|
||||||
|
floating-IP mode.
|
||||||
|
|||||||
+71
-7
@@ -138,13 +138,77 @@ restore traffic).
|
|||||||
|
|
||||||
## TLS
|
## TLS
|
||||||
|
|
||||||
- **certResolver**: `orca` (references the Traefik ACME/step-ca
|
- **v0.14 model**: `tls: {}` in dynamic config (no certResolver).
|
||||||
certificate resolver configured in Traefik's static config).
|
Traefik v3.3 `certificatesResolvers` only supports `acme` and
|
||||||
- **Trust domain**: `cluster.orca.local` (placeholder in v0.9; step-ca
|
`tailscale` — not CA-file-based. The `certResolver: orca` reference
|
||||||
provisioner in v0.11 overrides with the real cluster trust domain).
|
from v0.11 was broken (research finding). v0.14 emits `tls: {}`
|
||||||
- **SPIFFE SVIDs**: workload identity via SPIFFE SVIDs minted at submit
|
(traefik uses its default self-signed cert). Real mTLS via dynamic
|
||||||
time via step-ca (v0.11-P01.5, gate C-08). The SVID is a URI SAN in
|
`tls.certificates` + `tls.options.default.clientAuth.caFiles` is
|
||||||
the workload's X.509 cert.
|
deferred to v0.15.
|
||||||
|
- **Step-ca root CA**: mounted at `/etc/orca/step-ca-root.crt` in the
|
||||||
|
traefik container. v0.14 does not use it for TLS termination (it's
|
||||||
|
a placeholder for v0.15 mTLS).
|
||||||
|
|
||||||
|
## R-024: Podman Traefik Container (v0.14)
|
||||||
|
|
||||||
|
As of v0.14, Traefik runs as a **podman container** from the custom
|
||||||
|
`orca-traefik` image (published per release). The v0.13 binary+systemd
|
||||||
|
install is replaced.
|
||||||
|
|
||||||
|
### Three topologies
|
||||||
|
|
||||||
|
1. **Linux**: host → nft DNAT → `podman run orca-traefik` (`--network host`)
|
||||||
|
2. **Proxmox Native** (`--ingress-mode native`, default): PVE host →
|
||||||
|
nft DNAT → LXC (nesting=1,keyctl=1,fuse=1) → `podman run orca-traefik`
|
||||||
|
3. **Proxmox Floating-IP** (`--ingress-mode floating-ip`): LXC owns
|
||||||
|
the floating IP → nft inside LXC → `podman run orca-traefik`
|
||||||
|
|
||||||
|
### Container configuration
|
||||||
|
|
||||||
|
```bash
|
||||||
|
podman run -d --name orca-traefik --restart=unless-stopped \
|
||||||
|
--network host \
|
||||||
|
-v /etc/traefik/traefik.yml:/etc/traefik/traefik.yml:ro \
|
||||||
|
-v /etc/traefik/dynamic:/etc/traefik/dynamic:ro \
|
||||||
|
-v /etc/orca/step-ca-root.crt:/etc/orca/step-ca-root.crt:ro \
|
||||||
|
git.cloudinit.dev/coreci/orca-traefik:<version>
|
||||||
|
```
|
||||||
|
|
||||||
|
- `--network host`: traefik binds 127.0.0.1:8080/8443 on host/LXC loopback
|
||||||
|
- `--restart=unless-stopped`: survives reboot via `podman-restart.service`
|
||||||
|
- No `:Z` SELinux flag (research Topic 7)
|
||||||
|
- Static config mounted `:ro` (overrides baked image default, preserves
|
||||||
|
`traefik-on-public-ip` opt-out, REQ-100)
|
||||||
|
|
||||||
|
### nft ruleset
|
||||||
|
|
||||||
|
The nft emitter (`internal/emitter/nft.go`) renders `/etc/nftables.d/orca.nft`:
|
||||||
|
|
||||||
|
- DNAT `:443` → `<DNATTarget>:8443` (default 127.0.0.1; LXC IP for native)
|
||||||
|
- DNAT `:80` → `<DNATTarget>:8080`
|
||||||
|
- SNAT/MASQUERADE: `ip saddr 127.0.0.0/8 oifname != "lo" masquerade`
|
||||||
|
- Input/forward chains at priority -10 (pve-firewall coexistence)
|
||||||
|
|
||||||
|
### `orca doctor ingress`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
orca doctor ingress # check localhost
|
||||||
|
orca doctor ingress --peer <name> # check remote peer
|
||||||
|
```
|
||||||
|
|
||||||
|
Verifies: podman container running, nft DNAT+SNAT, dynamic dir exists,
|
||||||
|
step-ca root CA present.
|
||||||
|
|
||||||
|
### Dockerfile.traefik
|
||||||
|
|
||||||
|
```dockerfile
|
||||||
|
FROM traefik:v3.3.0
|
||||||
|
COPY docker/orca-traefik/traefik.yml /etc/traefik/traefik.yml
|
||||||
|
CMD ["--configFile=/etc/traefik/traefik.yml"]
|
||||||
|
```
|
||||||
|
|
||||||
|
Built + published per release alongside the orca image
|
||||||
|
(`scripts/release.sh` + `.coreci.yml container-publish-traefik`).
|
||||||
|
|
||||||
## Health checks
|
## Health checks
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,115 @@
|
|||||||
|
// Package cli: doctor_ingress.go implements `orca doctor ingress`
|
||||||
|
// (R-024, v0.14). The check verifies the podman traefik container is
|
||||||
|
// running, nft DNAT+SNAT is applied, the dynamic config directory
|
||||||
|
// exists, and the step-ca root CA is mounted.
|
||||||
|
package cli
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/spf13/cobra"
|
||||||
|
)
|
||||||
|
|
||||||
|
var doctorIngressCmd = &cobra.Command{
|
||||||
|
Use: "ingress",
|
||||||
|
Short: "Check the ingress stack (R-024: podman traefik + nft + CA)",
|
||||||
|
Long: `Verify the orca ingress data plane is healthy:
|
||||||
|
1. orca-traefik podman container is running
|
||||||
|
2. nft DNAT + SNAT masquerade applied
|
||||||
|
3. /etc/traefik/dynamic directory exists
|
||||||
|
4. step-ca root CA mounted at /etc/orca/step-ca-root.crt
|
||||||
|
|
||||||
|
For remote peers, use --peer <name>.`,
|
||||||
|
RunE: func(cmd *cobra.Command, args []string) error {
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
results := runIngressChecks(ctx)
|
||||||
|
if jsonOutput {
|
||||||
|
return printJSON(results)
|
||||||
|
}
|
||||||
|
allPass := true
|
||||||
|
for _, r := range results {
|
||||||
|
status := "✓"
|
||||||
|
if r.Result != "PASS" {
|
||||||
|
status = "✗"
|
||||||
|
allPass = false
|
||||||
|
}
|
||||||
|
fmt.Fprintf(cmd.OutOrStdout(), "%s %s: %s\n", status, r.Name, r.Message)
|
||||||
|
}
|
||||||
|
if !allPass {
|
||||||
|
return fmt.Errorf("ingress checks failed")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
// ingressCheckResult is one line of `orca doctor ingress` output.
|
||||||
|
type ingressCheckResult struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Result string `json:"result"`
|
||||||
|
Message string `json:"message"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func runIngressChecks(ctx context.Context) []ingressCheckResult {
|
||||||
|
t, err := nftTransportFromCtx()
|
||||||
|
if err != nil {
|
||||||
|
return []ingressCheckResult{{Name: "ingress:transport", Result: "FAIL", Message: err.Error()}}
|
||||||
|
}
|
||||||
|
peer := nftLeadPeer()
|
||||||
|
var results []ingressCheckResult
|
||||||
|
|
||||||
|
// 1. Check podman orca-traefik container is running.
|
||||||
|
out, err := t.Exec(ctx, peer, "podman inspect --format '{{.State.Running}}' orca-traefik 2>/dev/null")
|
||||||
|
if err != nil {
|
||||||
|
results = append(results, ingressCheckResult{Name: "ingress:container", Result: "FAIL", Message: fmt.Sprintf("podman inspect: %v", err)})
|
||||||
|
} else {
|
||||||
|
v := strings.TrimSpace(string(out))
|
||||||
|
if v == "true" {
|
||||||
|
results = append(results, ingressCheckResult{Name: "ingress:container", Result: "PASS", Message: "orca-traefik container running"})
|
||||||
|
} else if v == "false" {
|
||||||
|
results = append(results, ingressCheckResult{Name: "ingress:container", Result: "FAIL", Message: "orca-traefik container is stopped"})
|
||||||
|
} else {
|
||||||
|
results = append(results, ingressCheckResult{Name: "ingress:container", Result: "FAIL", Message: "orca-traefik container not found"})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 2. Check nft DNAT + SNAT (reuse the nft table output).
|
||||||
|
tableOut, tableErr := t.Exec(ctx, peer, "nft list table inet orca-ingress 2>/dev/null")
|
||||||
|
if tableErr != nil {
|
||||||
|
results = append(results, ingressCheckResult{Name: "ingress:nft", Result: "FAIL", Message: "nft table orca-ingress missing"})
|
||||||
|
} else {
|
||||||
|
tableStr := string(tableOut)
|
||||||
|
hasDNAT := strings.Contains(tableStr, "dnat to")
|
||||||
|
hasSNAT := strings.Contains(tableStr, "masquerade")
|
||||||
|
if hasDNAT && hasSNAT {
|
||||||
|
results = append(results, ingressCheckResult{Name: "ingress:nft", Result: "PASS", Message: "nft DNAT + SNAT masquerade present"})
|
||||||
|
} else if hasDNAT {
|
||||||
|
results = append(results, ingressCheckResult{Name: "ingress:nft", Result: "WARN", Message: "DNAT present but SNAT masquerade missing"})
|
||||||
|
} else {
|
||||||
|
results = append(results, ingressCheckResult{Name: "ingress:nft", Result: "FAIL", Message: "nft DNAT missing"})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 3. Check /etc/traefik/dynamic directory exists.
|
||||||
|
if _, err := t.Exec(ctx, peer, "test -d /etc/traefik/dynamic"); err != nil {
|
||||||
|
results = append(results, ingressCheckResult{Name: "ingress:dynamic-dir", Result: "FAIL", Message: "/etc/traefik/dynamic directory missing"})
|
||||||
|
} else {
|
||||||
|
results = append(results, ingressCheckResult{Name: "ingress:dynamic-dir", Result: "PASS", Message: "/etc/traefik/dynamic exists"})
|
||||||
|
}
|
||||||
|
|
||||||
|
// 4. Check step-ca root CA is mounted/present.
|
||||||
|
if _, err := t.Exec(ctx, peer, "test -f /etc/orca/step-ca-root.crt"); err != nil {
|
||||||
|
results = append(results, ingressCheckResult{Name: "ingress:ca", Result: "WARN", Message: "/etc/orca/step-ca-root.crt missing (TLS not configured)"})
|
||||||
|
} else {
|
||||||
|
results = append(results, ingressCheckResult{Name: "ingress:ca", Result: "PASS", Message: "step-ca root CA present"})
|
||||||
|
}
|
||||||
|
|
||||||
|
return results
|
||||||
|
}
|
||||||
|
|
||||||
|
func init() {
|
||||||
|
doctorCmd.AddCommand(doctorIngressCmd)
|
||||||
|
}
|
||||||
@@ -113,6 +113,20 @@ func runNftChecks(ctx context.Context) []nftCheckResult {
|
|||||||
results = append(results, nftCheckResult{Name: "nft:dnat-80", Result: "FAIL", Message: "DNAT :80->127.0.0.1:8080 missing"})
|
results = append(results, nftCheckResult{Name: "nft:dnat-80", Result: "FAIL", Message: "DNAT :80->127.0.0.1:8080 missing"})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Research Topic 1: postrouting masquerade for DNAT return path.
|
||||||
|
if strings.Contains(tableStr, "masquerade") {
|
||||||
|
results = append(results, nftCheckResult{Name: "nft:snat-masquerade", Result: "PASS", Message: "postrouting masquerade (SNAT) present"})
|
||||||
|
} else {
|
||||||
|
results = append(results, nftCheckResult{Name: "nft:snat-masquerade", Result: "FAIL", Message: "postrouting masquerade missing (R-024)"})
|
||||||
|
}
|
||||||
|
|
||||||
|
// Research Topic 2: priority -10 on input/forward (pve-firewall coexistence).
|
||||||
|
if strings.Contains(tableStr, "priority -10") {
|
||||||
|
results = append(results, nftCheckResult{Name: "nft:priority", Result: "PASS", Message: "input/forward chains at priority -10 (pve-firewall coexistence)"})
|
||||||
|
} else {
|
||||||
|
results = append(results, nftCheckResult{Name: "nft:priority", Result: "WARN", Message: "priority -10 not found (may be pre-v0.14 ruleset)"})
|
||||||
|
}
|
||||||
|
|
||||||
if strings.Contains(tableStr, "ora_rl") {
|
if strings.Contains(tableStr, "ora_rl") {
|
||||||
results = append(results, nftCheckResult{Name: "nft:rate-limit", Result: "PASS", Message: "rate-limit meter ora_rl present"})
|
results = append(results, nftCheckResult{Name: "nft:rate-limit", Result: "PASS", Message: "rate-limit meter ora_rl present"})
|
||||||
} else {
|
} else {
|
||||||
|
|||||||
+22
-16
@@ -1,27 +1,28 @@
|
|||||||
package cli
|
package cli
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"bufio"
|
||||||
"context"
|
"context"
|
||||||
"crypto/x509"
|
"crypto/x509"
|
||||||
"encoding/pem"
|
"encoding/pem"
|
||||||
"fmt"
|
"fmt"
|
||||||
"bufio"
|
|
||||||
"os"
|
|
||||||
"strings"
|
|
||||||
"os/exec"
|
|
||||||
"path/filepath"
|
|
||||||
"time"
|
|
||||||
"github.com/google/uuid"
|
|
||||||
"golang.org/x/crypto/ssh"
|
|
||||||
"github.com/spf13/cobra"
|
|
||||||
"git.cloudinit.dev/coreci/orca/internal/acl"
|
"git.cloudinit.dev/coreci/orca/internal/acl"
|
||||||
"git.cloudinit.dev/coreci/orca/internal/certpaths"
|
"git.cloudinit.dev/coreci/orca/internal/certpaths"
|
||||||
"git.cloudinit.dev/coreci/orca/internal/identity"
|
"git.cloudinit.dev/coreci/orca/internal/identity"
|
||||||
|
"git.cloudinit.dev/coreci/orca/internal/ingress"
|
||||||
"git.cloudinit.dev/coreci/orca/internal/model"
|
"git.cloudinit.dev/coreci/orca/internal/model"
|
||||||
"git.cloudinit.dev/coreci/orca/internal/paths"
|
"git.cloudinit.dev/coreci/orca/internal/paths"
|
||||||
"git.cloudinit.dev/coreci/orca/internal/secrets"
|
"git.cloudinit.dev/coreci/orca/internal/secrets"
|
||||||
"git.cloudinit.dev/coreci/orca/internal/security"
|
"git.cloudinit.dev/coreci/orca/internal/security"
|
||||||
"git.cloudinit.dev/coreci/orca/internal/store"
|
"git.cloudinit.dev/coreci/orca/internal/store"
|
||||||
|
"github.com/google/uuid"
|
||||||
|
"github.com/spf13/cobra"
|
||||||
|
"golang.org/x/crypto/ssh"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -251,17 +252,22 @@ func runInit(out interface{ Write([]byte) (int, error) }) error {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Step 4d: Install Traefik on the lead node (REQ-165, Phase B).
|
// Step 4d: Ensure complete ingress stack on the lead node (R-024).
|
||||||
// Traefik is the data-plane ingress. Idempotent.
|
// This replaces the v0.13 binary+systemd traefik install with:
|
||||||
if err := installTraefikLocal(); err != nil {
|
// 1. Render + write traefik static config (traefik.yml)
|
||||||
|
// 2. Render + write + apply nft DNAT/SNAT ruleset (orca.nft)
|
||||||
|
// 3. Push step-ca root CA to /etc/orca/step-ca-root.crt
|
||||||
|
// 4. Ensure podman orca-traefik container running
|
||||||
|
// All steps non-fatal (offline host tolerance).
|
||||||
|
if err := ingress.BootstrapLocalIngress(context.Background(), version); err != nil {
|
||||||
if !jsonOutput {
|
if !jsonOutput {
|
||||||
fmt.Fprintf(out, "Traefik install skipped: %v\n", err)
|
fmt.Fprintf(out, "Ingress bootstrap skipped: %v\n", err)
|
||||||
}
|
}
|
||||||
summary.Steps = append(summary.Steps, stepResult{Label: "traefik", Status: "skipped", Detail: err.Error()})
|
summary.Steps = append(summary.Steps, stepResult{Label: "ingress", Status: "skipped", Detail: err.Error()})
|
||||||
} else {
|
} else {
|
||||||
summary.Steps = append(summary.Steps, stepResult{Label: "traefik", Status: "ok", Detail: traefikVersion})
|
summary.Steps = append(summary.Steps, stepResult{Label: "ingress", Status: "ok", Detail: "nft+traefik container active"})
|
||||||
if !jsonOutput {
|
if !jsonOutput {
|
||||||
fmt.Fprintf(out, "Traefik installed: %s\n", traefikVersion)
|
fmt.Fprintf(out, "Ingress: nft DNAT+SNAT applied, traefik container running\n")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+182
-2
@@ -1,22 +1,26 @@
|
|||||||
package cli
|
package cli
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"bufio"
|
||||||
"context"
|
"context"
|
||||||
"database/sql"
|
"database/sql"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"fmt"
|
"fmt"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
|
"net"
|
||||||
"os"
|
"os"
|
||||||
"os/signal"
|
"os/signal"
|
||||||
|
"strings"
|
||||||
"syscall"
|
"syscall"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/google/uuid"
|
"github.com/google/uuid"
|
||||||
"github.com/spf13/cobra"
|
"github.com/spf13/cobra"
|
||||||
|
"golang.org/x/crypto/ssh"
|
||||||
|
|
||||||
"git.cloudinit.dev/coreci/orca/internal/certpaths"
|
"git.cloudinit.dev/coreci/orca/internal/certpaths"
|
||||||
"git.cloudinit.dev/coreci/orca/internal/linux"
|
|
||||||
"git.cloudinit.dev/coreci/orca/internal/engine"
|
"git.cloudinit.dev/coreci/orca/internal/engine"
|
||||||
|
"git.cloudinit.dev/coreci/orca/internal/linux"
|
||||||
"git.cloudinit.dev/coreci/orca/internal/model"
|
"git.cloudinit.dev/coreci/orca/internal/model"
|
||||||
"git.cloudinit.dev/coreci/orca/internal/proxmox"
|
"git.cloudinit.dev/coreci/orca/internal/proxmox"
|
||||||
"git.cloudinit.dev/coreci/orca/internal/security"
|
"git.cloudinit.dev/coreci/orca/internal/security"
|
||||||
@@ -58,6 +62,11 @@ var (
|
|||||||
joinLXCTemplate string
|
joinLXCTemplate string
|
||||||
proxmoxUser string
|
proxmoxUser string
|
||||||
proxmoxRole string
|
proxmoxRole string
|
||||||
|
ingressMode string
|
||||||
|
floatingIP string
|
||||||
|
gateway string
|
||||||
|
macAddr string
|
||||||
|
netPrefix int
|
||||||
leaveID string
|
leaveID string
|
||||||
nodeWatch bool
|
nodeWatch bool
|
||||||
)
|
)
|
||||||
@@ -175,7 +184,83 @@ func joinProxmox(cmd *cobra.Command) error {
|
|||||||
return fmt.Errorf("SSH key path is required for --type proxmox (R-021: no passwords; use --ssh-key or pre-stage the orca key)")
|
return fmt.Errorf("SSH key path is required for --type proxmox (R-021: no passwords; use --ssh-key or pre-stage the orca key)")
|
||||||
}
|
}
|
||||||
|
|
||||||
ctx, cancel := context.WithTimeout(cmd.Context(), 60*time.Second)
|
// Default ingress mode to "native" if not specified (R-024).
|
||||||
|
effectiveIngressMode := ingressMode
|
||||||
|
if effectiveIngressMode == "" {
|
||||||
|
if !jsonOutput {
|
||||||
|
// Interactive mode: prompt for ingress mode.
|
||||||
|
fmt.Fprint(cmd.OutOrStdout(), "Ingress mode [native/floating-ip] (default native): ")
|
||||||
|
scanner := bufio.NewScanner(os.Stdin)
|
||||||
|
if scanner.Scan() {
|
||||||
|
input := strings.TrimSpace(scanner.Text())
|
||||||
|
if input == "floating-ip" {
|
||||||
|
effectiveIngressMode = "floating-ip"
|
||||||
|
} else {
|
||||||
|
effectiveIngressMode = "native"
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
effectiveIngressMode = "native"
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
effectiveIngressMode = "native"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Floating-IP mode: prompt for params if not provided.
|
||||||
|
effectiveFloatingIP := floatingIP
|
||||||
|
effectiveGateway := gateway
|
||||||
|
effectiveMAC := macAddr
|
||||||
|
if effectiveIngressMode == "floating-ip" {
|
||||||
|
if effectiveFloatingIP == "" && !jsonOutput {
|
||||||
|
fmt.Fprint(cmd.OutOrStdout(), "Floating IP: ")
|
||||||
|
scanner := bufio.NewScanner(os.Stdin)
|
||||||
|
if scanner.Scan() {
|
||||||
|
effectiveFloatingIP = strings.TrimSpace(scanner.Text())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if effectiveGateway == "" && !jsonOutput {
|
||||||
|
fmt.Fprint(cmd.OutOrStdout(), "Gateway: ")
|
||||||
|
scanner := bufio.NewScanner(os.Stdin)
|
||||||
|
if scanner.Scan() {
|
||||||
|
effectiveGateway = strings.TrimSpace(scanner.Text())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if effectiveMAC == "" && !jsonOutput {
|
||||||
|
// D-261: auto-generate a random locally-administered MAC.
|
||||||
|
generated, err := proxmox.GenerateRandomMAC()
|
||||||
|
if err == nil {
|
||||||
|
fmt.Fprintf(cmd.OutOrStdout(), "Generated MAC: %s (press enter to accept, or type your own): ", generated)
|
||||||
|
scanner := bufio.NewScanner(os.Stdin)
|
||||||
|
if scanner.Scan() {
|
||||||
|
input := strings.TrimSpace(scanner.Text())
|
||||||
|
if input != "" {
|
||||||
|
effectiveMAC = input
|
||||||
|
} else {
|
||||||
|
effectiveMAC = generated
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
effectiveMAC = generated
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Validate floating-IP mode params.
|
||||||
|
if effectiveIngressMode == "floating-ip" {
|
||||||
|
if net.ParseIP(effectiveFloatingIP) == nil {
|
||||||
|
return fmt.Errorf("--floating-ip %q is not a valid IP", effectiveFloatingIP)
|
||||||
|
}
|
||||||
|
if net.ParseIP(effectiveGateway) == nil {
|
||||||
|
return fmt.Errorf("--gateway %q is not a valid IP", effectiveGateway)
|
||||||
|
}
|
||||||
|
if _, err := net.ParseMAC(effectiveMAC); err != nil {
|
||||||
|
return fmt.Errorf("--mac %q is not a valid MAC: %w", effectiveMAC, err)
|
||||||
|
}
|
||||||
|
if netPrefix < 8 || netPrefix > 32 {
|
||||||
|
return fmt.Errorf("--net-prefix %d must be 8-32", netPrefix)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx, cancel := context.WithTimeout(cmd.Context(), 180*time.Second) // 3min for LXC creation
|
||||||
defer cancel()
|
defer cancel()
|
||||||
|
|
||||||
result, err := proxmox.BootstrapProxmox(ctx, proxmox.Options{
|
result, err := proxmox.BootstrapProxmox(ctx, proxmox.Options{
|
||||||
@@ -188,6 +273,7 @@ func joinProxmox(cmd *cobra.Command) error {
|
|||||||
HostKeyFingerprint: joinHostKeyFP,
|
HostKeyFingerprint: joinHostKeyFP,
|
||||||
Logger: newLogger(),
|
Logger: newLogger(),
|
||||||
LXCTemplate: joinLXCTemplate,
|
LXCTemplate: joinLXCTemplate,
|
||||||
|
IngressMode: effectiveIngressMode,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("proxmox bootstrap: %w", err)
|
return fmt.Errorf("proxmox bootstrap: %w", err)
|
||||||
@@ -212,10 +298,59 @@ func joinProxmox(cmd *cobra.Command) error {
|
|||||||
LastSeen: time.Now().UTC(),
|
LastSeen: time.Now().UTC(),
|
||||||
Kind: string(model.NodeKindProxmox),
|
Kind: string(model.NodeKindProxmox),
|
||||||
OS: "pve",
|
OS: "pve",
|
||||||
|
IngressMode: effectiveIngressMode,
|
||||||
}
|
}
|
||||||
if err := registry.Join(regCtx, node); err != nil {
|
if err := registry.Join(regCtx, node); err != nil {
|
||||||
return fmt.Errorf("register proxmox node: %w", err)
|
return fmt.Errorf("register proxmox node: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Floating-IP mode: provision the ingress LXC and register it as a
|
||||||
|
// linux node (R-024, REQ-176). The PVE host is registered as
|
||||||
|
// proxmox (above); the ingress LXC is registered as linux so
|
||||||
|
// `orca job run` pushes traefik dynamic config to it.
|
||||||
|
if effectiveIngressMode == "floating-ip" {
|
||||||
|
lxcLog := newLogger()
|
||||||
|
// Build a runRemote function from the proxmox bootstrap result.
|
||||||
|
// We need SSH access to the PVE host to run pct commands.
|
||||||
|
lxcCtx, lxcCancel := context.WithTimeout(ctx, 120*time.Second)
|
||||||
|
defer lxcCancel()
|
||||||
|
// The BootstrapProxmox result gives us the host; we need to
|
||||||
|
// re-establish the SSH connection for the LXC provisioning.
|
||||||
|
lxcExecFn, lxcErr := proxmoxRemoteExecFn(result, sshKeyPath, joinSSHUser, joinSSHPort)
|
||||||
|
if lxcErr != nil {
|
||||||
|
fmt.Fprintf(cmd.OutOrStdout(), "Warning: could not establish SSH for LXC provisioning: %v\n", lxcErr)
|
||||||
|
} else {
|
||||||
|
if err := proxmox.ProvisionIngressLXC(lxcCtx, lxcExecFn, proxmox.FloatingIPOptions{
|
||||||
|
FloatingIP: effectiveFloatingIP,
|
||||||
|
Gateway: effectiveGateway,
|
||||||
|
MAC: effectiveMAC,
|
||||||
|
NetPrefix: netPrefix,
|
||||||
|
LXCTemplate: joinLXCTemplate,
|
||||||
|
}, lxcLog); err != nil {
|
||||||
|
fmt.Fprintf(cmd.OutOrStdout(), "Warning: ingress LXC provisioning failed: %v\n", err)
|
||||||
|
} else {
|
||||||
|
// Register the ingress LXC as a linux node.
|
||||||
|
ingressNode := &model.Node{
|
||||||
|
ID: uuid.NewString(),
|
||||||
|
Name: "ingress",
|
||||||
|
Address: fmt.Sprintf("%s:8443", effectiveFloatingIP),
|
||||||
|
State: model.NodeStateReady,
|
||||||
|
JoinedAt: time.Now().UTC(),
|
||||||
|
LastSeen: time.Now().UTC(),
|
||||||
|
Kind: string(model.NodeKindLinux),
|
||||||
|
OS: "linux",
|
||||||
|
IngressMode: "floating-ip",
|
||||||
|
}
|
||||||
|
if err := registry.Join(regCtx, ingressNode); err != nil {
|
||||||
|
fmt.Fprintf(cmd.OutOrStdout(), "Warning: register ingress node: %v\n", err)
|
||||||
|
}
|
||||||
|
if !jsonOutput {
|
||||||
|
fmt.Fprintf(cmd.OutOrStdout(), "✓ Ingress LXC joined: %s (%s) at %s\n", ingressNode.ID, ingressNode.Name, ingressNode.Address)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// REQ-156 / P07 T5: invalidate the nodes cache.
|
// REQ-156 / P07 T5: invalidate the nodes cache.
|
||||||
cacheInvalidate(cacheNodeClass)
|
cacheInvalidate(cacheNodeClass)
|
||||||
if jsonOutput {
|
if jsonOutput {
|
||||||
@@ -226,6 +361,46 @@ func joinProxmox(cmd *cobra.Command) error {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// proxmoxRemoteExecFn creates a RemoteExecFunc (func(string) ([]byte,
|
||||||
|
// error)) that runs commands on the PVE host via SSH. Used by the
|
||||||
|
// floating-IP LXC provisioning path (ProvisionIngressLXC).
|
||||||
|
func proxmoxRemoteExecFn(result *proxmox.Result, sshKeyPath, sshUser string, sshPort int) (func(string) ([]byte, error), error) {
|
||||||
|
cfg := &ssh.ClientConfig{
|
||||||
|
User: sshUser,
|
||||||
|
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
|
||||||
|
Timeout: 10 * time.Second,
|
||||||
|
}
|
||||||
|
if sshKeyPath != "" {
|
||||||
|
keyData, err := os.ReadFile(sshKeyPath)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("read SSH key: %w", err)
|
||||||
|
}
|
||||||
|
signer, err := ssh.ParsePrivateKey(keyData)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("parse SSH key: %w", err)
|
||||||
|
}
|
||||||
|
cfg.Auth = []ssh.AuthMethod{ssh.PublicKeys(signer)}
|
||||||
|
}
|
||||||
|
addr := result.NodeName
|
||||||
|
if sshPort != 22 {
|
||||||
|
addr = fmt.Sprintf("%s:%d", result.NodeName, sshPort)
|
||||||
|
} else {
|
||||||
|
addr = fmt.Sprintf("%s:%d", result.NodeName, sshPort)
|
||||||
|
}
|
||||||
|
client, err := ssh.Dial("tcp", addr, cfg)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("ssh dial %s: %w", addr, err)
|
||||||
|
}
|
||||||
|
return func(cmd string) ([]byte, error) {
|
||||||
|
session, err := client.NewSession()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer session.Close()
|
||||||
|
return session.CombinedOutput(cmd)
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
// joinLinux bootstraps a remote generic Linux worker via SSH and
|
// joinLinux bootstraps a remote generic Linux worker via SSH and
|
||||||
// registers it as an orca node (REQ-161, P12). Uses SSH key auth
|
// registers it as an orca node (REQ-161, P12). Uses SSH key auth
|
||||||
// (R-021: no passwords).
|
// (R-021: no passwords).
|
||||||
@@ -514,6 +689,11 @@ func init() {
|
|||||||
nodeJoinCmd.Flags().StringVar(&proxmoxRole, "proxmox-role", "OrcaOperator", "PVE custom role to create (config-overridable)")
|
nodeJoinCmd.Flags().StringVar(&proxmoxRole, "proxmox-role", "OrcaOperator", "PVE custom role to create (config-overridable)")
|
||||||
nodeJoinCmd.Flags().StringVar(&joinHostKeyFP, "host-key-fingerprint", "", "SSH host key SHA256:base64 fingerprint (pre-pin; supersedes TOFU for --type proxmox or --type linux)")
|
nodeJoinCmd.Flags().StringVar(&joinHostKeyFP, "host-key-fingerprint", "", "SSH host key SHA256:base64 fingerprint (pre-pin; supersedes TOFU for --type proxmox or --type linux)")
|
||||||
nodeJoinCmd.Flags().StringVar(&joinLXCTemplate, "lxc-template", "ubuntu-24.04", "LXC template for Proxmox (default ubuntu-24.04; alternatives: alpine-3.20, debian-12)")
|
nodeJoinCmd.Flags().StringVar(&joinLXCTemplate, "lxc-template", "ubuntu-24.04", "LXC template for Proxmox (default ubuntu-24.04; alternatives: alpine-3.20, debian-12)")
|
||||||
|
nodeJoinCmd.Flags().StringVar(&ingressMode, "ingress-mode", "", "proxmox ingress mode: native (default, traefik in LXC) or floating-ip (ingress LXC owns floating IP)")
|
||||||
|
nodeJoinCmd.Flags().StringVar(&floatingIP, "floating-ip", "", "floating public IP for the ingress LXC (required for --ingress-mode floating-ip)")
|
||||||
|
nodeJoinCmd.Flags().StringVar(&gateway, "gateway", "", "gateway for the ingress LXC (required for --ingress-mode floating-ip)")
|
||||||
|
nodeJoinCmd.Flags().StringVar(&macAddr, "mac", "", "MAC address for the ingress LXC net0 (required for --ingress-mode floating-ip in --json mode; auto-generated in interactive mode)")
|
||||||
|
nodeJoinCmd.Flags().IntVar(&netPrefix, "net-prefix", 24, "network prefix (CIDR) for the ingress LXC IP (default 24; valid 8-32)")
|
||||||
nodeLeaveCmd.Flags().StringVar(&leaveID, "id", "", "node id")
|
nodeLeaveCmd.Flags().StringVar(&leaveID, "id", "", "node id")
|
||||||
nodeListCmd.Flags().BoolVar(&nodeWatch, "watch", false, "stream nodes until Ctrl-C (table refresh or --json per-event)")
|
nodeListCmd.Flags().BoolVar(&nodeWatch, "watch", false, "stream nodes until Ctrl-C (table refresh or --json per-event)")
|
||||||
|
|
||||||
|
|||||||
@@ -1,11 +1,16 @@
|
|||||||
package cli
|
package cli
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
|
|
||||||
"git.cloudinit.dev/coreci/orca/internal/traefik"
|
"git.cloudinit.dev/coreci/orca/internal/traefik"
|
||||||
)
|
)
|
||||||
|
|
||||||
var traefikVersion = traefik.DefaultVersion
|
// ensureTraefikContainerLocal ensures the orca-traefik podman container
|
||||||
|
// is running on the local host (R-024). Replaces the v0.13
|
||||||
func installTraefikLocal() error {
|
// installTraefikLocal binary+systemd installer.
|
||||||
return traefik.InstallLocal(traefikVersion)
|
func ensureTraefikContainerLocal() error {
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 120_000_000_000) // 2min for pull
|
||||||
|
defer cancel()
|
||||||
|
return traefik.EnsureTraefikContainerLocal(ctx, version)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -379,10 +379,11 @@ func performCutover(ctx context.Context, runner commandRunner, out interface{ Wr
|
|||||||
return false, fmt.Errorf("cutover: atomic rename %s → %s: %w", tmpPath, traefikYml, err)
|
return false, fmt.Errorf("cutover: atomic rename %s → %s: %w", tmpPath, traefikYml, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
if _, err := runner.Run(ctx, "systemctl", "restart", "traefik"); err != nil {
|
if _, err := runner.Run(ctx, "bash", "-c", "systemctl stop orca-traefik.service 2>/dev/null; systemctl disable orca-traefik.service 2>/dev/null; rm -f /etc/systemd/system/orca-traefik.service /usr/local/bin/traefik; systemctl daemon-reload; true"); err != nil {
|
||||||
// Restart failed — restore from backup.
|
slog.Warn("cutover: legacy systemd unit removal failed (non-fatal if already removed)", "err", err)
|
||||||
_ = cfs.Rename(backupPath, traefikYml)
|
}
|
||||||
return false, fmt.Errorf("cutover: restart traefik: %w", err)
|
if err := ensureTraefikContainerLocal(); err != nil {
|
||||||
|
slog.Warn("cutover: podman traefik container ensure failed", "err", err)
|
||||||
}
|
}
|
||||||
nftCmd := `nft add table inet orca_redirect; nft 'add chain inet orca_redirect prerouting { type nat hook prerouting priority -100; }'; nft add rule inet orca_redirect prerouting tcp dport 443 dnat to 127.0.0.1:8443`
|
nftCmd := `nft add table inet orca_redirect; nft 'add chain inet orca_redirect prerouting { type nat hook prerouting priority -100; }'; nft add rule inet orca_redirect prerouting tcp dport 443 dnat to 127.0.0.1:8443`
|
||||||
if _, err := runner.Run(ctx, "bash", "-c", nftCmd); err != nil {
|
if _, err := runner.Run(ctx, "bash", "-c", nftCmd); err != nil {
|
||||||
|
|||||||
+63
-4
@@ -53,6 +53,16 @@ type NftClusterConfig struct {
|
|||||||
// RateBurst is the per-source burst (packets) for the meter.
|
// RateBurst is the per-source burst (packets) for the meter.
|
||||||
// Defaults to 200.
|
// Defaults to 200.
|
||||||
RateBurst int
|
RateBurst int
|
||||||
|
// DNATTarget is the destination IP for DNAT rules. Defaults to
|
||||||
|
// "127.0.0.1" (hybrid R-017 model — traefik on loopback). For
|
||||||
|
// Proxmox native mode where traefik runs inside an LXC, set this
|
||||||
|
// to the LXC's bridge IP so the PVE host DNATs to the LXC.
|
||||||
|
// Must be a valid IPv4 address (C-51: injection guard).
|
||||||
|
DNATTarget string
|
||||||
|
// EnableSNAT controls whether the postrouting masquerade chain
|
||||||
|
// is rendered. Defaults to true (R-024: SNAT/MASQUERADE for the
|
||||||
|
// DNAT return path). Set to false to omit the postrouting chain.
|
||||||
|
EnableSNAT bool
|
||||||
}
|
}
|
||||||
|
|
||||||
// withDefaults returns a copy of c with zero values replaced by the
|
// withDefaults returns a copy of c with zero values replaced by the
|
||||||
@@ -68,6 +78,33 @@ func (c NftClusterConfig) withDefaults() NftClusterConfig {
|
|||||||
if out.RateBurst <= 0 {
|
if out.RateBurst <= 0 {
|
||||||
out.RateBurst = 200
|
out.RateBurst = 200
|
||||||
}
|
}
|
||||||
|
if out.DNATTarget == "" {
|
||||||
|
out.DNATTarget = "127.0.0.1"
|
||||||
|
}
|
||||||
|
// EnableSNAT defaults to true — use a sentinel: if the field was
|
||||||
|
// not explicitly set (false) and DNATTarget is the default, enable
|
||||||
|
// it. This is a Go zero-value compromise; callers who want to
|
||||||
|
// disable SNAT must set it to false explicitly after construction.
|
||||||
|
// Actually, since we want SNAT on by default, we flip it here:
|
||||||
|
// the zero value is false, but we want true. So we always set true
|
||||||
|
// unless the caller explicitly set it to a non-zero sentinel.
|
||||||
|
// Simpler: treat EnableSNAT as "opt-out" — default true, set false
|
||||||
|
// to disable. Since Go zero-value is false, we invert: use
|
||||||
|
// DisableSNAT instead. But the plan says EnableSNAT. To keep the
|
||||||
|
// plan naming and have default-true, we check if it's the zero
|
||||||
|
// value and set true:
|
||||||
|
// NOTE: since bool zero value is false, we can't distinguish "not
|
||||||
|
// set" from "set to false". So we use a pointer or invert. The
|
||||||
|
// simplest fix: the field is "EnableSNAT" and defaults to true via
|
||||||
|
// this logic: if the caller didn't set DNATTarget (still ""),
|
||||||
|
// they used a zero-value config, so enable SNAT. If they set
|
||||||
|
// DNATTarget explicitly, they should also set EnableSNAT.
|
||||||
|
// For now: always enable SNAT unless the caller sets it to false
|
||||||
|
// AND sets a non-default DNATTarget. This is pragmatic:
|
||||||
|
if !out.EnableSNAT && out.DNATTarget == "127.0.0.1" {
|
||||||
|
// Zero-value config (both fields unset) → enable SNAT.
|
||||||
|
out.EnableSNAT = true
|
||||||
|
}
|
||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -94,6 +131,12 @@ func (NftEmitter) RenderNftConfig(clusterConfig NftClusterConfig) ([]File, error
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
// C-51: validate DNATTarget as a valid IP before rendering. An
|
||||||
|
// unvalidated DNATTarget is an nft-syntax injection vector (same
|
||||||
|
// risk as TrustedProbes — the value is written raw into `dnat to`).
|
||||||
|
if net.ParseIP(cfg.DNATTarget) == nil {
|
||||||
|
return nil, fmt.Errorf("emitter/nft: DNATTarget %q is not a valid IP (C-51: ruleset injection guard)", cfg.DNATTarget)
|
||||||
|
}
|
||||||
content := renderNftRuleset(cfg, v4, v6)
|
content := renderNftRuleset(cfg, v4, v6)
|
||||||
return []File{{Path: nftConfigPath, Content: content, Mode: "0644"}}, nil
|
return []File{{Path: nftConfigPath, Content: content, Mode: "0644"}}, nil
|
||||||
}
|
}
|
||||||
@@ -171,19 +214,35 @@ func renderNftRuleset(cfg NftClusterConfig, v4, v6 []string) string {
|
|||||||
b.WriteString(" }\n")
|
b.WriteString(" }\n")
|
||||||
b.WriteString("\t}\n\n")
|
b.WriteString("\t}\n\n")
|
||||||
|
|
||||||
|
// Research Topic 2: shift input/forward priority from `filter`
|
||||||
|
// (=0) to -10 to avoid same-priority undefined evaluation order
|
||||||
|
// with pve-firewall's iptables chains (also at priority 0). This
|
||||||
|
// ensures orca's SYN-flood filter runs deterministically before
|
||||||
|
// pve-firewall on Proxmox hosts.
|
||||||
b.WriteString("\tchain input {\n")
|
b.WriteString("\tchain input {\n")
|
||||||
b.WriteString("\t\ttype filter hook input priority filter; policy accept;\n")
|
b.WriteString("\t\ttype filter hook input priority -10; policy accept;\n")
|
||||||
b.WriteString("\t\tct state invalid drop\n")
|
b.WriteString("\t\tct state invalid drop\n")
|
||||||
b.WriteString("\t\tct state established,related accept\n")
|
b.WriteString("\t\tct state established,related accept\n")
|
||||||
b.WriteString("\t\ttcp dport 443 tcp-flags != syn,rst,ack,fin notrack drop\n")
|
b.WriteString("\t\ttcp dport 443 tcp-flags != syn,rst,ack,fin notrack drop\n")
|
||||||
b.WriteString("\t}\n\n")
|
b.WriteString("\t}\n\n")
|
||||||
b.WriteString("\tchain prerouting {\n")
|
b.WriteString("\tchain prerouting {\n")
|
||||||
b.WriteString("\t\ttype nat hook prerouting priority -100; policy accept;\n")
|
b.WriteString("\t\ttype nat hook prerouting priority -100; policy accept;\n")
|
||||||
b.WriteString("\t\ttcp dport 443 dnat to 127.0.0.1:8443\n")
|
b.WriteString(fmt.Sprintf("\t\ttcp dport 443 dnat to %s:8443\n", cfg.DNATTarget))
|
||||||
b.WriteString("\t\ttcp dport 80 dnat to 127.0.0.1:8080\n")
|
b.WriteString(fmt.Sprintf("\t\ttcp dport 80 dnat to %s:8080\n", cfg.DNATTarget))
|
||||||
b.WriteString("\t}\n\n")
|
b.WriteString("\t}\n\n")
|
||||||
|
// Research Topic 1: postrouting masquerade for the DNAT return
|
||||||
|
// path. Scoped to `ip saddr 127.0.0.0/8 oifname != "lo"` so only
|
||||||
|
// loopback-DNAT'd traffic is masqueraded (not all egress). This is
|
||||||
|
// the canonical "hairpin NAT" / "loopback DNAT return path" rule.
|
||||||
|
// Priority 100 = NF_IP_PRI_SRCNAT (standard srcnat priority).
|
||||||
|
if cfg.EnableSNAT {
|
||||||
|
b.WriteString("\tchain postrouting {\n")
|
||||||
|
b.WriteString("\t\ttype nat hook postrouting priority 100; policy accept;\n")
|
||||||
|
b.WriteString("\t\tip saddr 127.0.0.0/8 oifname != \"lo\" masquerade\n")
|
||||||
|
b.WriteString("\t}\n\n")
|
||||||
|
}
|
||||||
b.WriteString("\tchain forward {\n")
|
b.WriteString("\tchain forward {\n")
|
||||||
b.WriteString("\t\ttype filter hook forward priority filter; policy accept;\n")
|
b.WriteString("\t\ttype filter hook forward priority -10; policy accept;\n")
|
||||||
b.WriteString(fmt.Sprintf("\t\ttcp dport 443 ct state new meter { ora_rl { rate %d/second burst %d packets } } accept\n", cfg.RateLimit, cfg.RateBurst))
|
b.WriteString(fmt.Sprintf("\t\ttcp dport 443 ct state new meter { ora_rl { rate %d/second burst %d packets } } accept\n", cfg.RateLimit, cfg.RateBurst))
|
||||||
b.WriteString("\t}\n")
|
b.WriteString("\t}\n")
|
||||||
b.WriteString("}\n")
|
b.WriteString("}\n")
|
||||||
|
|||||||
@@ -30,10 +30,13 @@ func TestNftEmitter_RenderBasic(t *testing.T) {
|
|||||||
"127.0.0.1",
|
"127.0.0.1",
|
||||||
"::1",
|
"::1",
|
||||||
"chain input",
|
"chain input",
|
||||||
|
"priority -10; policy accept;", // research Topic 2: pve-firewall coexistence
|
||||||
"tcp dport 443 tcp-flags != syn,rst,ack,fin notrack drop",
|
"tcp dport 443 tcp-flags != syn,rst,ack,fin notrack drop",
|
||||||
"chain prerouting",
|
"chain prerouting",
|
||||||
"tcp dport 443 dnat to 127.0.0.1:8443",
|
"tcp dport 443 dnat to 127.0.0.1:8443",
|
||||||
"tcp dport 80 dnat to 127.0.0.1:8080",
|
"tcp dport 80 dnat to 127.0.0.1:8080",
|
||||||
|
"chain postrouting", // research Topic 1: SNAT masquerade
|
||||||
|
"ip saddr 127.0.0.0/8 oifname != \"lo\" masquerade", // scoped to loopback DNAT return
|
||||||
"chain forward",
|
"chain forward",
|
||||||
"rate 100/second burst 200 packets",
|
"rate 100/second burst 200 packets",
|
||||||
"ora_rl",
|
"ora_rl",
|
||||||
@@ -135,3 +138,90 @@ func TestNftEmitter_TrustedProbesSplitV4V6(t *testing.T) {
|
|||||||
t.Errorf("missing ::1 in v6 set:\n%s", c)
|
t.Errorf("missing ::1 in v6 set:\n%s", c)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestNftEmitter_CustomDNATTarget verifies the DNATTarget field
|
||||||
|
// substitutes into the dnat rules (C-51, D-262 — proxmox native mode
|
||||||
|
// DNATs to the LXC bridge IP instead of 127.0.0.1).
|
||||||
|
func TestNftEmitter_CustomDNATTarget(t *testing.T) {
|
||||||
|
files, err := (NftEmitter{}).RenderNftConfig(NftClusterConfig{DNATTarget: "10.99.0.10"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Render: %v", err)
|
||||||
|
}
|
||||||
|
c := files[0].Content
|
||||||
|
if !strings.Contains(c, "dnat to 10.99.0.10:8443") {
|
||||||
|
t.Errorf("missing custom DNAT target :8443:\n%s", c)
|
||||||
|
}
|
||||||
|
if !strings.Contains(c, "dnat to 10.99.0.10:8080") {
|
||||||
|
t.Errorf("missing custom DNAT target :8080:\n%s", c)
|
||||||
|
}
|
||||||
|
if strings.Contains(c, "127.0.0.1:8443") {
|
||||||
|
t.Errorf("default 127.0.0.1:8443 should not be present when custom DNATTarget set:\n%s", c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestNftEmitter_RejectsInvalidDNATTarget verifies that an invalid
|
||||||
|
// DNATTarget is rejected (C-51: nft-syntax injection guard).
|
||||||
|
func TestNftEmitter_RejectsInvalidDNATTarget(t *testing.T) {
|
||||||
|
bad := []string{
|
||||||
|
"not-an-ip",
|
||||||
|
"127.0.0.1; flush ruleset",
|
||||||
|
"$(whoami)",
|
||||||
|
"10.0.0.0/33",
|
||||||
|
"",
|
||||||
|
}
|
||||||
|
for _, b := range bad {
|
||||||
|
// Empty string gets defaulted to 127.0.0.1, so it won't error.
|
||||||
|
// Test only non-empty invalid values.
|
||||||
|
if b == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
_, err := (NftEmitter{}).RenderNftConfig(NftClusterConfig{DNATTarget: b})
|
||||||
|
if err == nil {
|
||||||
|
t.Errorf("expected error for invalid DNATTarget %q, got nil", b)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestNftEmitter_DisableSNAT verifies that EnableSNAT=false omits the
|
||||||
|
// postrouting chain entirely.
|
||||||
|
func TestNftEmitter_DisableSNAT(t *testing.T) {
|
||||||
|
// To explicitly disable SNAT, set DNATTarget to a non-default
|
||||||
|
// value AND EnableSNAT to false. The withDefaults logic only
|
||||||
|
// auto-enables SNAT for the zero-value config.
|
||||||
|
files, err := (NftEmitter{}).RenderNftConfig(NftClusterConfig{
|
||||||
|
DNATTarget: "10.99.0.10",
|
||||||
|
EnableSNAT: false,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Render: %v", err)
|
||||||
|
}
|
||||||
|
c := files[0].Content
|
||||||
|
if strings.Contains(c, "chain postrouting") {
|
||||||
|
t.Errorf("postrouting chain should be absent when EnableSNAT=false:\n%s", c)
|
||||||
|
}
|
||||||
|
if strings.Contains(c, "masquerade") {
|
||||||
|
t.Errorf("masquerade rule should be absent when EnableSNAT=false:\n%s", c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestNftEmitter_PriorityMinus10 verifies the input and forward chains
|
||||||
|
// use priority -10 (research Topic 2: pve-firewall coexistence — avoids
|
||||||
|
// same-priority undefined evaluation order with pve-firewall's
|
||||||
|
// iptables chains at priority 0).
|
||||||
|
func TestNftEmitter_PriorityMinus10(t *testing.T) {
|
||||||
|
files, _ := (NftEmitter{}).RenderNftConfig(NftClusterConfig{})
|
||||||
|
c := files[0].Content
|
||||||
|
if !strings.Contains(c, "hook input priority -10;") {
|
||||||
|
t.Errorf("input chain should use priority -10:\n%s", c)
|
||||||
|
}
|
||||||
|
if !strings.Contains(c, "hook forward priority -10;") {
|
||||||
|
t.Errorf("forward chain should use priority -10:\n%s", c)
|
||||||
|
}
|
||||||
|
// Nat chains should stay at standard priorities.
|
||||||
|
if !strings.Contains(c, "hook prerouting priority -100;") {
|
||||||
|
t.Errorf("prerouting chain should use priority -100:\n%s", c)
|
||||||
|
}
|
||||||
|
if !strings.Contains(c, "hook postrouting priority 100;") {
|
||||||
|
t.Errorf("postrouting chain should use priority 100:\n%s", c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
+14
-13
@@ -46,16 +46,21 @@ type TraefikEmitter struct{}
|
|||||||
// /etc/traefik/dynamic/orca-<spec.Name>.yaml.
|
// /etc/traefik/dynamic/orca-<spec.Name>.yaml.
|
||||||
const traefikDynamicDir = "/etc/traefik/dynamic"
|
const traefikDynamicDir = "/etc/traefik/dynamic"
|
||||||
|
|
||||||
// traefikRouterTLSCertResolver is the Traefik cert-resolver name that
|
// traefikRouterTLSCertResolver is the Traefik cert-resolver name from
|
||||||
// the orca step-ca integration configures on the Traefik static config
|
// v0.11. As of v0.14 (RESEARCH_v0.14 Topic 4), traefik v3.3 only
|
||||||
// (P10 / v0.10 wires the step-ca root into this resolver). The
|
// supports acme/tailscale certResolvers — CA-file-based resolvers do
|
||||||
// dynamic-config file references it by name.
|
// not exist. The dynamic config now emits `tls: {}` instead. Real
|
||||||
|
// mTLS via dynamic tls.certificates + clientAuth.caFiles is deferred
|
||||||
|
// to v0.15. This constant is retained for documentation.
|
||||||
|
//
|
||||||
|
// Deprecated: v0.14 removed certResolver from the dynamic config.
|
||||||
const traefikRouterTLSCertResolver = "orca"
|
const traefikRouterTLSCertResolver = "orca"
|
||||||
|
|
||||||
// defaultTrustDomain is the SPIFFE trust domain used in the rendered
|
// defaultTrustDomain is the SPIFFE trust domain. v0.14 removed the
|
||||||
// TLS stanza when the spec does not carry an explicit trust domain.
|
// TLS domains stanza from the dynamic config (replaced with tls: {}).
|
||||||
// The step-ca provisioner (P10) overrides this at render time via the
|
// Retained for documentation; will be used by v0.15 mTLS.
|
||||||
// node argument; for P02 the emitter renders the placeholder.
|
//
|
||||||
|
// Deprecated: v0.14 removed TLS domains from the dynamic config.
|
||||||
const defaultTrustDomain = "cluster.orca.local"
|
const defaultTrustDomain = "cluster.orca.local"
|
||||||
|
|
||||||
// Render renders the Traefik dynamic-config YAML for a Service
|
// Render renders the Traefik dynamic-config YAML for a Service
|
||||||
@@ -175,17 +180,13 @@ func renderTraefikYAMLWeighted(spec *jobspec.WorkloadSpec, node *Node, drain boo
|
|||||||
routerName := "orca-" + spec.Name
|
routerName := "orca-" + spec.Name
|
||||||
serviceName := "orca-" + spec.Name
|
serviceName := "orca-" + spec.Name
|
||||||
rule := fmt.Sprintf("PathPrefix(\"/%s\")", spec.Name)
|
rule := fmt.Sprintf("PathPrefix(\"/%s\")", spec.Name)
|
||||||
trustDomain := defaultTrustDomain
|
|
||||||
|
|
||||||
b.WriteString("http:\n")
|
b.WriteString("http:\n")
|
||||||
b.WriteString(" routers:\n")
|
b.WriteString(" routers:\n")
|
||||||
b.WriteString(fmt.Sprintf(" %s:\n", routerName))
|
b.WriteString(fmt.Sprintf(" %s:\n", routerName))
|
||||||
b.WriteString(fmt.Sprintf(" rule: %s\n", rule))
|
b.WriteString(fmt.Sprintf(" rule: %s\n", rule))
|
||||||
b.WriteString(fmt.Sprintf(" service: %s\n", serviceName))
|
b.WriteString(fmt.Sprintf(" service: %s\n", serviceName))
|
||||||
b.WriteString(" tls:\n")
|
b.WriteString(" tls: {}\n")
|
||||||
b.WriteString(fmt.Sprintf(" certResolver: %s\n", traefikRouterTLSCertResolver))
|
|
||||||
b.WriteString(" domains:\n")
|
|
||||||
b.WriteString(fmt.Sprintf(" - main: %q\n", trustDomain))
|
|
||||||
b.WriteString(" services:\n")
|
b.WriteString(" services:\n")
|
||||||
b.WriteString(fmt.Sprintf(" %s:\n", serviceName))
|
b.WriteString(fmt.Sprintf(" %s:\n", serviceName))
|
||||||
b.WriteString(" loadBalancer:\n")
|
b.WriteString(" loadBalancer:\n")
|
||||||
|
|||||||
@@ -53,11 +53,8 @@ func TestTraefikEmitter_RenderBasic(t *testing.T) {
|
|||||||
if !strings.Contains(c, "unix:///run/orca/alloc-node-1/port-http.sock") {
|
if !strings.Contains(c, "unix:///run/orca/alloc-node-1/port-http.sock") {
|
||||||
t.Errorf("content missing socket server URL\n%s", c)
|
t.Errorf("content missing socket server URL\n%s", c)
|
||||||
}
|
}
|
||||||
if !strings.Contains(c, "certResolver: orca") {
|
if !strings.Contains(c, "tls: {}") {
|
||||||
t.Errorf("content missing 'certResolver: orca'\n%s", c)
|
t.Errorf("content missing 'tls: {}' (v0.14: certResolver dropped, tls empty stanza)\n%s", c)
|
||||||
}
|
|
||||||
if !strings.Contains(c, "domains:") {
|
|
||||||
t.Errorf("content missing TLS domains\n%s", c)
|
|
||||||
}
|
}
|
||||||
if !strings.Contains(c, "healthCheck:") {
|
if !strings.Contains(c, "healthCheck:") {
|
||||||
t.Errorf("content missing 'healthCheck:'\n%s", c)
|
t.Errorf("content missing 'healthCheck:'\n%s", c)
|
||||||
|
|||||||
@@ -0,0 +1,240 @@
|
|||||||
|
// Package ingress implements the R-024 ingress bootstrap: nft DNAT
|
||||||
|
// + SNAT/MASQUERADE + traefik podman container + step-ca root CA on
|
||||||
|
// every orca-managed node. The bootstrap is idempotent and non-fatal
|
||||||
|
// on each step (offline host tolerance — same as v0.13 traefik install).
|
||||||
|
//
|
||||||
|
// BootstrapLocalIngress runs on the lead (during `orca init`).
|
||||||
|
// BootstrapRemoteIngress runs on workers (during `orca node join`).
|
||||||
|
package ingress
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/rand"
|
||||||
|
"encoding/hex"
|
||||||
|
"fmt"
|
||||||
|
"log/slog"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"git.cloudinit.dev/coreci/orca/internal/certpaths"
|
||||||
|
"git.cloudinit.dev/coreci/orca/internal/emitter"
|
||||||
|
"git.cloudinit.dev/coreci/orca/internal/traefik"
|
||||||
|
)
|
||||||
|
|
||||||
|
// BootstrapLocalIngress ensures the complete ingress stack is running
|
||||||
|
// on the local host (lead node). It is called from `orca init` after
|
||||||
|
// EnsureTraefikContainerLocal.
|
||||||
|
//
|
||||||
|
// Steps (each non-fatal — logs a warning and continues):
|
||||||
|
// 1. mkdir -p /etc/traefik/dynamic /etc/orca
|
||||||
|
// 2. Push cluster root CA to /etc/orca/step-ca-root.crt (C-60:
|
||||||
|
// certpaths.CACertPath(), not CAPath)
|
||||||
|
// 3. Render static config via emitter.RenderTraefikStaticConfig to
|
||||||
|
// /etc/traefik/traefik.yml (preserves traefik-on-public-ip opt-out,
|
||||||
|
// C-58)
|
||||||
|
// 4. Render orca.nft via emitter.NftEmitter.RenderNftConfig + write
|
||||||
|
// to /etc/nftables.d/orca.nft
|
||||||
|
// 5. Pre-create nft table (C-55: avoids flush-table error on first
|
||||||
|
// apply)
|
||||||
|
// 6. Apply: nft -f /etc/nftables.d/orca.nft
|
||||||
|
func BootstrapLocalIngress(ctx context.Context, version string) error {
|
||||||
|
var errs []error
|
||||||
|
log := slog.Default()
|
||||||
|
|
||||||
|
// Step 1: ensure directories.
|
||||||
|
for _, dir := range []string{"/etc/traefik/dynamic", "/etc/orca"} {
|
||||||
|
if err := os.MkdirAll(dir, 0o755); err != nil {
|
||||||
|
log.Warn("ingress: mkdir failed", "dir", dir, "err", err)
|
||||||
|
errs = append(errs, fmt.Errorf("mkdir %s: %w", dir, err))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 2: push cluster root CA (C-60: CACertPath, not CAPath).
|
||||||
|
caPath := certpaths.CACertPath()
|
||||||
|
if caData, err := os.ReadFile(caPath); err == nil {
|
||||||
|
if err := os.WriteFile("/etc/orca/step-ca-root.crt", caData, 0o644); err != nil {
|
||||||
|
log.Warn("ingress: step-ca root CA write failed", "err", err)
|
||||||
|
errs = append(errs, fmt.Errorf("write step-ca-root.crt: %w", err))
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// CA may not exist yet (fresh init before step-ca). Write a
|
||||||
|
// placeholder so the podman volume mount doesn't fail.
|
||||||
|
_ = os.WriteFile("/etc/orca/step-ca-root.crt", []byte{}, 0o644)
|
||||||
|
log.Warn("ingress: step-ca root CA not found, wrote placeholder", "path", caPath)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 3: render + write static config (C-58).
|
||||||
|
staticFiles, err := emitter.TraefikEmitter{}.RenderTraefikStaticConfig(emitter.TraefikStaticOpts{})
|
||||||
|
if err != nil {
|
||||||
|
log.Warn("ingress: render traefik static config failed", "err", err)
|
||||||
|
errs = append(errs, fmt.Errorf("render traefik static: %w", err))
|
||||||
|
} else {
|
||||||
|
for _, f := range staticFiles {
|
||||||
|
if err := os.WriteFile(f.Path, []byte(f.Content), 0o644); err != nil {
|
||||||
|
log.Warn("ingress: write traefik static config failed", "path", f.Path, "err", err)
|
||||||
|
errs = append(errs, fmt.Errorf("write %s: %w", f.Path, err))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 4: render + write nft ruleset.
|
||||||
|
nftFiles, err := emitter.NftEmitter{}.RenderNftConfig(emitter.NftClusterConfig{})
|
||||||
|
if err != nil {
|
||||||
|
log.Warn("ingress: render nft config failed", "err", err)
|
||||||
|
errs = append(errs, fmt.Errorf("render nft: %w", err))
|
||||||
|
} else {
|
||||||
|
for _, f := range nftFiles {
|
||||||
|
dir := filepath.Dir(f.Path)
|
||||||
|
_ = os.MkdirAll(dir, 0o755)
|
||||||
|
if err := os.WriteFile(f.Path, []byte(f.Content), 0o644); err != nil {
|
||||||
|
log.Warn("ingress: write nft config failed", "path", f.Path, "err", err)
|
||||||
|
errs = append(errs, fmt.Errorf("write %s: %w", f.Path, err))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 5: pre-create nft table (C-55: flush table on non-existent
|
||||||
|
// table errors — pre-create avoids the first-apply failure).
|
||||||
|
_ = exec.CommandContext(ctx, "nft", "add", "table", "inet", "orca-ingress").Run()
|
||||||
|
|
||||||
|
// Step 6: apply nft ruleset.
|
||||||
|
if out, err := exec.CommandContext(ctx, "nft", "-f", nftConfigPath).CombinedOutput(); err != nil {
|
||||||
|
log.Warn("ingress: nft apply failed", "err", err, "output", string(out))
|
||||||
|
errs = append(errs, fmt.Errorf("nft -f: %w (output: %s)", err, string(out)))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ensure the podman traefik container is running (Step 7 — C-50:
|
||||||
|
// installs podman if absent).
|
||||||
|
if err := traefik.EnsureTraefikContainerLocal(ctx, version); err != nil {
|
||||||
|
log.Warn("ingress: ensure traefik container failed", "err", err)
|
||||||
|
errs = append(errs, fmt.Errorf("ensure traefik container: %w", err))
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(errs) > 0 {
|
||||||
|
return fmt.Errorf("ingress bootstrap: %d errors (first: %w)", len(errs), errs[0])
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// nftConfigPath mirrors the emitter constant.
|
||||||
|
const nftConfigPath = "/etc/nftables.d/orca.nft"
|
||||||
|
|
||||||
|
// RemoteExecFunc runs a command on a remote host and returns combined
|
||||||
|
// output. Same signature as traefik.RemoteExecFunc.
|
||||||
|
type RemoteExecFunc func(cmd string) ([]byte, error)
|
||||||
|
|
||||||
|
// BootstrapRemoteIngress ensures the complete ingress stack is running
|
||||||
|
// on a remote host (linux worker). It is called from
|
||||||
|
// `orca node join --type linux` after the user setup.
|
||||||
|
//
|
||||||
|
// Steps (each non-fatal — logs a warning and continues):
|
||||||
|
// 1. mkdir -p /etc/traefik/dynamic /etc/orca (remote)
|
||||||
|
// 2. Push step-ca root CA to remote /etc/orca/step-ca-root.crt
|
||||||
|
// (C-60: certpaths.CACertPath)
|
||||||
|
// 3. Render + write static config to remote /etc/traefik/traefik.yml
|
||||||
|
// (C-58: preserves traefik-on-public-ip opt-out)
|
||||||
|
// 4. Render + write nft ruleset to remote /etc/nftables.d/orca.nft
|
||||||
|
// 5. Pre-create nft table (C-55: avoids first-apply flush-table error)
|
||||||
|
// 6. Apply: nft -f (remote)
|
||||||
|
// 7. Ensure podman traefik container running (remote)
|
||||||
|
func BootstrapRemoteIngress(ctx context.Context, version string, execFn RemoteExecFunc) error {
|
||||||
|
var errs []error
|
||||||
|
log := slog.Default()
|
||||||
|
|
||||||
|
// Step 1: ensure directories.
|
||||||
|
if _, err := execFn("mkdir -p /etc/traefik/dynamic /etc/orca"); err != nil {
|
||||||
|
log.Warn("ingress: remote mkdir failed", "err", err)
|
||||||
|
errs = append(errs, fmt.Errorf("remote mkdir: %w", err))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 2: push step-ca root CA (C-60: CACertPath, not CAPath).
|
||||||
|
if caData, err := os.ReadFile(certpaths.CACertPath()); err == nil {
|
||||||
|
if err := remoteWriteFile(execFn, "/etc/orca/step-ca-root.crt", caData, "0644"); err != nil {
|
||||||
|
log.Warn("ingress: remote step-ca CA write failed", "err", err)
|
||||||
|
errs = append(errs, fmt.Errorf("remote write step-ca-root.crt: %w", err))
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// Write a placeholder so the podman volume mount doesn't fail.
|
||||||
|
_ = remoteWriteFile(execFn, "/etc/orca/step-ca-root.crt", []byte{}, "0644")
|
||||||
|
log.Warn("ingress: step-ca root CA not found locally, wrote remote placeholder")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 3: render + write static config (C-58).
|
||||||
|
staticFiles, err := emitter.TraefikEmitter{}.RenderTraefikStaticConfig(emitter.TraefikStaticOpts{})
|
||||||
|
if err != nil {
|
||||||
|
log.Warn("ingress: render traefik static config failed", "err", err)
|
||||||
|
errs = append(errs, fmt.Errorf("render traefik static: %w", err))
|
||||||
|
} else {
|
||||||
|
for _, f := range staticFiles {
|
||||||
|
_, _ = execFn(fmt.Sprintf("mkdir -p %s", filepath.Dir(f.Path)))
|
||||||
|
if err := remoteWriteFile(execFn, f.Path, []byte(f.Content), f.Mode); err != nil {
|
||||||
|
log.Warn("ingress: remote write traefik static config failed", "path", f.Path, "err", err)
|
||||||
|
errs = append(errs, fmt.Errorf("remote write %s: %w", f.Path, err))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 4: render + write nft ruleset.
|
||||||
|
nftFiles, err := emitter.NftEmitter{}.RenderNftConfig(emitter.NftClusterConfig{})
|
||||||
|
if err != nil {
|
||||||
|
log.Warn("ingress: render nft config failed", "err", err)
|
||||||
|
errs = append(errs, fmt.Errorf("render nft: %w", err))
|
||||||
|
} else {
|
||||||
|
for _, f := range nftFiles {
|
||||||
|
_, _ = execFn(fmt.Sprintf("mkdir -p %s", filepath.Dir(f.Path)))
|
||||||
|
if err := remoteWriteFile(execFn, f.Path, []byte(f.Content), f.Mode); err != nil {
|
||||||
|
log.Warn("ingress: remote write nft config failed", "path", f.Path, "err", err)
|
||||||
|
errs = append(errs, fmt.Errorf("remote write %s: %w", f.Path, err))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 5: pre-create nft table (C-55).
|
||||||
|
_, _ = execFn("nft add table inet orca-ingress 2>/dev/null || true")
|
||||||
|
|
||||||
|
// Step 6: apply nft ruleset.
|
||||||
|
if out, err := execFn("nft -f /etc/nftables.d/orca.nft 2>&1"); err != nil {
|
||||||
|
log.Warn("ingress: remote nft apply failed", "err", err, "output", string(out))
|
||||||
|
errs = append(errs, fmt.Errorf("remote nft -f: %w (output: %s)", err, string(out)))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 7: ensure podman traefik container (C-50).
|
||||||
|
traefikExecFn := traefik.RemoteExecFunc(execFn)
|
||||||
|
if err := traefik.EnsureTraefikContainerRemote(ctx, version, traefikExecFn); err != nil {
|
||||||
|
log.Warn("ingress: remote ensure traefik container failed", "err", err)
|
||||||
|
errs = append(errs, fmt.Errorf("remote ensure traefik container: %w", err))
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(errs) > 0 {
|
||||||
|
return fmt.Errorf("remote ingress bootstrap: %d errors (first: %w)", len(errs), errs[0])
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// remoteWriteFile writes content to a remote path via a heredoc
|
||||||
|
// (same pattern as sshpush.idempotency.writeFile). The heredoc
|
||||||
|
// delimiter is a random hex string verified absent from the content
|
||||||
|
// (F9 injection guard).
|
||||||
|
func remoteWriteFile(execFn RemoteExecFunc, path string, content []byte, mode string) error {
|
||||||
|
// Generate a random delimiter unlikely to be in the content.
|
||||||
|
delim := "EOF_"
|
||||||
|
for {
|
||||||
|
b := make([]byte, 8)
|
||||||
|
if _, err := rand.Read(b); err != nil {
|
||||||
|
return fmt.Errorf("rand: %w", err)
|
||||||
|
}
|
||||||
|
delim = "EOF_" + hex.EncodeToString(b)
|
||||||
|
if !strings.Contains(string(content), delim) {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
dir := filepath.Dir(path)
|
||||||
|
cmd := fmt.Sprintf("mkdir -p %s && cat > %s <<'%s'\n%s\n%s\nchmod %s %s",
|
||||||
|
dir, path, delim, string(content), delim, mode, path)
|
||||||
|
if out, err := execFn(cmd); err != nil {
|
||||||
|
return fmt.Errorf("remote write %s: %w (output: %s)", path, err, string(out))
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -32,9 +32,9 @@ import (
|
|||||||
"golang.org/x/crypto/ssh"
|
"golang.org/x/crypto/ssh"
|
||||||
|
|
||||||
"git.cloudinit.dev/coreci/orca/internal/certpaths"
|
"git.cloudinit.dev/coreci/orca/internal/certpaths"
|
||||||
|
"git.cloudinit.dev/coreci/orca/internal/ingress"
|
||||||
"git.cloudinit.dev/coreci/orca/internal/proxmox"
|
"git.cloudinit.dev/coreci/orca/internal/proxmox"
|
||||||
"git.cloudinit.dev/coreci/orca/internal/security"
|
"git.cloudinit.dev/coreci/orca/internal/security"
|
||||||
"git.cloudinit.dev/coreci/orca/internal/traefik"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// DefaultSSHUser is the default SSH username for the initial connection.
|
// DefaultSSHUser is the default SSH username for the initial connection.
|
||||||
@@ -157,8 +157,10 @@ func BootstrapLinux(ctx context.Context, opts Options) (*Result, error) {
|
|||||||
}
|
}
|
||||||
opts.Logger.Info("linux bootstrap: user created", "user", opts.OrcaUser)
|
opts.Logger.Info("linux bootstrap: user created", "user", opts.OrcaUser)
|
||||||
|
|
||||||
// Step 4d: Install Traefik on the remote host (REQ-165, Phase B).
|
// Step 4d: Ensure complete ingress stack on the remote host (R-024).
|
||||||
// Traefik is the data-plane ingress; SSH is control plane only.
|
// Renders+applies nft DNAT/SNAT, pushes step-ca root CA, renders+
|
||||||
|
// writes traefik static config, ensures podman container running.
|
||||||
|
// All non-fatal (offline host tolerance).
|
||||||
sshExecFn := func(cmd string) ([]byte, error) {
|
sshExecFn := func(cmd string) ([]byte, error) {
|
||||||
session, err := client.NewSession()
|
session, err := client.NewSession()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -167,8 +169,10 @@ func BootstrapLinux(ctx context.Context, opts Options) (*Result, error) {
|
|||||||
defer session.Close()
|
defer session.Close()
|
||||||
return session.CombinedOutput(cmd)
|
return session.CombinedOutput(cmd)
|
||||||
}
|
}
|
||||||
if err := traefik.InstallRemote("", sshExecFn); err != nil {
|
ctx, cancelIngress := context.WithTimeout(ctx, 120*time.Second)
|
||||||
opts.Logger.Warn("linux bootstrap: traefik install failed", "err", err)
|
defer cancelIngress()
|
||||||
|
if err := ingress.BootstrapRemoteIngress(ctx, "", ingress.RemoteExecFunc(sshExecFn)); err != nil {
|
||||||
|
opts.Logger.Warn("linux bootstrap: ingress bootstrap failed", "err", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Step 5: Create the drift-events directory.
|
// Step 5: Create the drift-events directory.
|
||||||
@@ -226,5 +230,4 @@ func pinnedHostKeyCallback(expectedSHA256Base64 string, capturedKey *ssh.PublicK
|
|||||||
return cb, nil
|
return cb, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
var _ = security.WriteAtomic
|
var _ = security.WriteAtomic
|
||||||
|
|||||||
@@ -39,4 +39,9 @@ type Node struct {
|
|||||||
// OS is the auto-detected OS identifier from /etc/os-release ID=
|
// OS is the auto-detected OS identifier from /etc/os-release ID=
|
||||||
// (ubuntu|debian|alpine|pve|linux). Empty for pre-0006 rows.
|
// (ubuntu|debian|alpine|pve|linux). Empty for pre-0006 rows.
|
||||||
OS string `json:"os,omitempty"`
|
OS string `json:"os,omitempty"`
|
||||||
|
// IngressMode is the ingress configuration for the node (R-024,
|
||||||
|
// v0.14). Values: "" (legacy/default for linux/localhost),
|
||||||
|
// "native" (proxmox native — traefik in LXC), "floating-ip"
|
||||||
|
// (proxmox floating-IP — ingress LXC owns the floating IP).
|
||||||
|
IngressMode string `json:"ingress_mode,omitempty"`
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -37,6 +37,7 @@ import (
|
|||||||
"golang.org/x/crypto/ssh/knownhosts"
|
"golang.org/x/crypto/ssh/knownhosts"
|
||||||
|
|
||||||
"git.cloudinit.dev/coreci/orca/internal/certpaths"
|
"git.cloudinit.dev/coreci/orca/internal/certpaths"
|
||||||
|
"git.cloudinit.dev/coreci/orca/internal/emitter"
|
||||||
"git.cloudinit.dev/coreci/orca/internal/security"
|
"git.cloudinit.dev/coreci/orca/internal/security"
|
||||||
"git.cloudinit.dev/coreci/orca/internal/traefik"
|
"git.cloudinit.dev/coreci/orca/internal/traefik"
|
||||||
)
|
)
|
||||||
@@ -86,6 +87,13 @@ type Options struct {
|
|||||||
// LXCTemplate is the LXC template to download during bootstrap
|
// LXCTemplate is the LXC template to download during bootstrap
|
||||||
// (default "ubuntu-24.04"; alternatives: "alpine-3.20", "debian-12").
|
// (default "ubuntu-24.04"; alternatives: "alpine-3.20", "debian-12").
|
||||||
LXCTemplate string
|
LXCTemplate string
|
||||||
|
// IngressMode is the proxmox ingress mode (R-024, v0.14).
|
||||||
|
// "native" (default): traefik runs in an unprivileged LXC with
|
||||||
|
// nesting=1,keyctl=1,fuse=1 on the PVE host. nft on the PVE host
|
||||||
|
// DNATs to the LXC bridge IP.
|
||||||
|
// "floating-ip": a separate ingress LXC owns the floating IP;
|
||||||
|
// nft runs inside that LXC. See ProvisionIngressLXC (P6).
|
||||||
|
IngressMode string
|
||||||
}
|
}
|
||||||
|
|
||||||
// Result is the outcome of a successful bootstrap.
|
// Result is the outcome of a successful bootstrap.
|
||||||
@@ -247,21 +255,24 @@ func BootstrapProxmox(ctx context.Context, opts Options) (*Result, error) {
|
|||||||
return nil, fmt.Errorf("validate sudoers: %w", err)
|
return nil, fmt.Errorf("validate sudoers: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Step 9a: Install Traefik on the Proxmox host (REQ-165, Phase B).
|
// Step 9a: Proxmox native ingress mode (R-024, REQ-175).
|
||||||
// Traefik runs on the PVE OS as the data-plane ingress; SSH is
|
// Create an unprivileged LXC with nesting=1,keyctl=1,fuse=1 (research
|
||||||
// control plane only. Idempotent: skips if binary already exists.
|
// Topic 3), install podman inside it, and run the orca-traefik
|
||||||
if err := traefik.InstallRemote("", runRemote); err != nil {
|
// container. nft on the PVE host DNATs to the LXC bridge IP.
|
||||||
log.Warn("proxmox.traefik_install_failed", "err", err)
|
// Default mode is "native"; floating-ip mode is handled separately
|
||||||
}
|
// (P6 — ProvisionIngressLXC).
|
||||||
|
|
||||||
// Step 9b: Download default LXC template (REQ-167, Phase C).
|
|
||||||
// Default: ubuntu-24.04. Configurable via --lxc-template.
|
|
||||||
template := opts.LXCTemplate
|
template := opts.LXCTemplate
|
||||||
if template == "" {
|
if template == "" {
|
||||||
template = "ubuntu-24.04"
|
template = "ubuntu-24.04"
|
||||||
}
|
}
|
||||||
_, _ = runRemote(fmt.Sprintf("pveam download local %s 2>/dev/null || true", shellQuote(template)))
|
_, _ = runRemote(fmt.Sprintf("pveam download local %s 2>/dev/null || true", shellQuote(template)))
|
||||||
|
|
||||||
|
if opts.IngressMode != "floating-ip" {
|
||||||
|
if err := provisionNativeIngressLXC(ctx, runRemote, template, log); err != nil {
|
||||||
|
log.Warn("proxmox.native_ingress_lxc_failed", "err", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
log.Info("proxmox.bootstrap_ok",
|
log.Info("proxmox.bootstrap_ok",
|
||||||
slog.String("event", "proxmox.bootstrap_ok"),
|
slog.String("event", "proxmox.bootstrap_ok"),
|
||||||
slog.String("host", opts.Host),
|
slog.String("host", opts.Host),
|
||||||
@@ -276,6 +287,119 @@ func BootstrapProxmox(ctx context.Context, opts Options) (*Result, error) {
|
|||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// provisionNativeIngressLXC creates an unprivileged LXC with
|
||||||
|
// nesting=1,keyctl=1,fuse=1 (research Topic 3), installs podman inside
|
||||||
|
// it, runs the orca-traefik container, and applies nft DNAT on the PVE
|
||||||
|
// host targeting the LXC's bridge IP (R-024, REQ-175).
|
||||||
|
//
|
||||||
|
// The LXC is named "orca-traefik" and uses a deterministic VMID derived
|
||||||
|
// from the host. It is idempotent: if the LXC already exists, it is
|
||||||
|
// not re-created (C-53: apt-get install is skipped if podman present).
|
||||||
|
func provisionNativeIngressLXC(ctx context.Context, runRemote func(string) ([]byte, error), template string, log *slog.Logger) error {
|
||||||
|
// Deterministic VMID for the native ingress LXC.
|
||||||
|
// Use a fixed VMID in the 200-299 range (Proxmox convention for CTs).
|
||||||
|
const vmid = "200"
|
||||||
|
const lxcName = "orca-traefik"
|
||||||
|
|
||||||
|
// Check if the LXC already exists.
|
||||||
|
existOut, _ := runRemote(fmt.Sprintf("pct status %s 2>/dev/null || echo absent", vmid))
|
||||||
|
existStr := strings.TrimSpace(string(existOut))
|
||||||
|
if existStr == "absent" {
|
||||||
|
// Create the LXC (research Topic 3: nesting=1,keyctl=1,fuse=1).
|
||||||
|
log.Info("proxmox.creating_native_ingress_lxc", "vmid", vmid, "name", lxcName)
|
||||||
|
createCmd := fmt.Sprintf(
|
||||||
|
"pct create %s local:vztmpl/%s --hostname %s --unprivileged 1 --features nesting=1,keyctl=1,fuse=1 --onboot 1 --memory 2048 --swap 0 --rootfs local:8 2>&1",
|
||||||
|
vmid, shellQuote(template), lxcName,
|
||||||
|
)
|
||||||
|
if out, err := runRemote(createCmd); err != nil {
|
||||||
|
return fmt.Errorf("pct create native ingress LXC: %w (output: %s)", err, string(out))
|
||||||
|
}
|
||||||
|
if out, err := runRemote(fmt.Sprintf("pct start %s", vmid)); err != nil {
|
||||||
|
return fmt.Errorf("pct start native ingress LXC: %w (output: %s)", err, string(out))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Wait for LXC network (retry for up to 60s).
|
||||||
|
lxcIP := ""
|
||||||
|
for i := 0; i < 12; i++ {
|
||||||
|
ipOut, _ := runRemote(fmt.Sprintf("pct exec %s -- hostname -I 2>/dev/null", vmid))
|
||||||
|
ipStr := strings.TrimSpace(string(ipOut))
|
||||||
|
if ipStr != "" {
|
||||||
|
fields := strings.Fields(ipStr)
|
||||||
|
if len(fields) > 0 {
|
||||||
|
lxcIP = fields[0]
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
time.Sleep(5 * time.Second)
|
||||||
|
}
|
||||||
|
if lxcIP == "" {
|
||||||
|
return fmt.Errorf("native ingress LXC: could not discover IP after 60s")
|
||||||
|
}
|
||||||
|
log.Info("proxmox.native_ingress_lxc_ip", "vmid", vmid, "ip", lxcIP)
|
||||||
|
|
||||||
|
// Install podman inside the LXC (C-53: idempotent — check first).
|
||||||
|
_, _ = runRemote(fmt.Sprintf(
|
||||||
|
"pct exec %s -- bash -c 'command -v podman >/dev/null 2>&1 || (apt-get update -qq && apt-get install -y -qq podman conmon crun fuse-overlayfs nftables 2>&1)' 2>&1",
|
||||||
|
vmid,
|
||||||
|
))
|
||||||
|
|
||||||
|
// Enable podman-restart.service inside the LXC (research Topic 6).
|
||||||
|
_, _ = runRemote(fmt.Sprintf("pct exec %s -- systemctl enable --now podman-restart.service 2>/dev/null", vmid))
|
||||||
|
|
||||||
|
// Push step-ca root CA into the LXC (placeholder if absent locally).
|
||||||
|
caPath := certpaths.CACertPath()
|
||||||
|
caData, caErr := os.ReadFile(caPath)
|
||||||
|
if caErr != nil {
|
||||||
|
caData = []byte{}
|
||||||
|
}
|
||||||
|
// Write CA via pct exec heredoc.
|
||||||
|
caDelim := "EOF_CA"
|
||||||
|
_, _ = runRemote(fmt.Sprintf(
|
||||||
|
"pct exec %s -- bash -c 'mkdir -p /etc/orca && cat > /etc/orca/step-ca-root.crt <<%s\\n%s\\n%s'",
|
||||||
|
vmid, caDelim, string(caData), caDelim,
|
||||||
|
))
|
||||||
|
|
||||||
|
// Render + write traefik static config inside the LXC.
|
||||||
|
staticFiles, err := emitter.TraefikEmitter{}.RenderTraefikStaticConfig(emitter.TraefikStaticOpts{})
|
||||||
|
if err == nil {
|
||||||
|
for _, f := range staticFiles {
|
||||||
|
delim := "EOF_TF"
|
||||||
|
_, _ = runRemote(fmt.Sprintf(
|
||||||
|
"pct exec %s -- bash -c 'mkdir -p /etc/traefik/dynamic && cat > %s <<%s\\n%s\\n%s'",
|
||||||
|
vmid, f.Path, delim, f.Content, delim,
|
||||||
|
))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ensure podman orca-traefik container inside the LXC.
|
||||||
|
traefikExecFn := func(cmd string) ([]byte, error) {
|
||||||
|
return runRemote(fmt.Sprintf("pct exec %s -- bash -c %s 2>&1", vmid, shellQuote(cmd)))
|
||||||
|
}
|
||||||
|
if err := traefik.EnsureTraefikContainerRemote(ctx, "", traefikExecFn); err != nil {
|
||||||
|
log.Warn("proxmox.native_ingress_lxc_traefik_failed", "err", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Render + apply nft on the PVE host with DNATTarget = LXC IP.
|
||||||
|
nftFiles, err := emitter.NftEmitter{}.RenderNftConfig(emitter.NftClusterConfig{
|
||||||
|
DNATTarget: lxcIP,
|
||||||
|
})
|
||||||
|
if err == nil {
|
||||||
|
for _, f := range nftFiles {
|
||||||
|
nftDelim := "EOF_NF"
|
||||||
|
_, _ = runRemote(fmt.Sprintf("mkdir -p /etc/nftables.d && cat > %s <<%s\\n%s\\n%s",
|
||||||
|
f.Path, nftDelim, f.Content, nftDelim))
|
||||||
|
}
|
||||||
|
_, _ = runRemote("nft add table inet orca-ingress 2>/dev/null || true")
|
||||||
|
if out, err := runRemote("nft -f /etc/nftables.d/orca.nft 2>&1"); err != nil {
|
||||||
|
log.Warn("proxmox.native_ingress_nft_apply_failed", "err", err, "output", string(out))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
log.Info("proxmox.native_ingress_lxc_ok", "vmid", vmid, "ip", lxcIP)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
// sshDialer is the dialer used by BootstrapProxmox. It's a package-level
|
// sshDialer is the dialer used by BootstrapProxmox. It's a package-level
|
||||||
// variable so tests can override it with a fake SSH server.
|
// variable so tests can override it with a fake SSH server.
|
||||||
var sshDialer sshDialerType = defaultSSHDialer{}
|
var sshDialer sshDialerType = defaultSSHDialer{}
|
||||||
|
|||||||
@@ -0,0 +1,171 @@
|
|||||||
|
package proxmox
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/rand"
|
||||||
|
"fmt"
|
||||||
|
"log/slog"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.cloudinit.dev/coreci/orca/internal/certpaths"
|
||||||
|
"git.cloudinit.dev/coreci/orca/internal/emitter"
|
||||||
|
"git.cloudinit.dev/coreci/orca/internal/traefik"
|
||||||
|
)
|
||||||
|
|
||||||
|
// FloatingIPOptions carries the parameters for provisioning a
|
||||||
|
// floating-IP ingress LXC (R-024, REQ-176).
|
||||||
|
type FloatingIPOptions struct {
|
||||||
|
// FloatingIP is the public IP assigned to the LXC's eth0.
|
||||||
|
FloatingIP string
|
||||||
|
// Gateway is the default gateway for the LXC.
|
||||||
|
Gateway string
|
||||||
|
// MAC is the MAC address for the LXC's net0 interface.
|
||||||
|
MAC string
|
||||||
|
// NetPrefix is the CIDR prefix for the floating IP (8-32).
|
||||||
|
NetPrefix int
|
||||||
|
// LXCTemplate is the LXC template (default "ubuntu-24.04").
|
||||||
|
LXCTemplate string
|
||||||
|
// VMID is the LXC container ID (default "201" for the ingress LXC).
|
||||||
|
VMID string
|
||||||
|
}
|
||||||
|
|
||||||
|
// ProvisionIngressLXC creates an Ubuntu LXC named "ingress" that owns
|
||||||
|
// the floating IP, installs podman + orca-traefik inside it, applies nft
|
||||||
|
// DNAT+SNAT inside the LXC, and returns the LXC's IP for node
|
||||||
|
// registration (R-024, REQ-176).
|
||||||
|
//
|
||||||
|
// The LXC is created with:
|
||||||
|
//
|
||||||
|
// --unprivileged 1 --features nesting=1,keyctl=1,fuse=1
|
||||||
|
// --net0 name=eth0,bridge=vmbr0,hwaddr=<mac>,ip=<floating-ip>/<prefix>,gw=<gateway>
|
||||||
|
// --onboot 1
|
||||||
|
//
|
||||||
|
// Inside the LXC, the complete ingress stack is set up: podman
|
||||||
|
// installed, traefik static config written, nft DNAT:443→127.0.0.1:8443
|
||||||
|
// + postrouting masquerade applied, orca-traefik podman container
|
||||||
|
// running with --network host.
|
||||||
|
//
|
||||||
|
// Idempotent: if the LXC already exists, it is not re-created (C-53).
|
||||||
|
func ProvisionIngressLXC(ctx context.Context, runRemote func(string) ([]byte, error), opts FloatingIPOptions, log *slog.Logger) error {
|
||||||
|
template := opts.LXCTemplate
|
||||||
|
if template == "" {
|
||||||
|
template = "ubuntu-24.04"
|
||||||
|
}
|
||||||
|
vmid := opts.VMID
|
||||||
|
if vmid == "" {
|
||||||
|
vmid = "201"
|
||||||
|
}
|
||||||
|
if opts.NetPrefix == 0 {
|
||||||
|
opts.NetPrefix = 24
|
||||||
|
}
|
||||||
|
|
||||||
|
// Validate required fields.
|
||||||
|
if opts.FloatingIP == "" || opts.Gateway == "" || opts.MAC == "" {
|
||||||
|
return fmt.Errorf("ingress_lxc: floating-ip, gateway, and mac are required")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ensure the template is downloaded.
|
||||||
|
_, _ = runRemote(fmt.Sprintf("pveam download local %s 2>/dev/null || true", shellQuote(template)))
|
||||||
|
|
||||||
|
// Check if the LXC already exists (idempotent — C-53).
|
||||||
|
existOut, _ := runRemote(fmt.Sprintf("pct status %s 2>/dev/null || echo absent", vmid))
|
||||||
|
existStr := strings.TrimSpace(string(existOut))
|
||||||
|
if existStr == "absent" {
|
||||||
|
log.Info("ingress_lxc.creating", "vmid", vmid, "hostname", "ingress", "ip", opts.FloatingIP)
|
||||||
|
net0 := fmt.Sprintf("name=eth0,bridge=vmbr0,hwaddr=%s,ip=%s/%d,gw=%s",
|
||||||
|
opts.MAC, opts.FloatingIP, opts.NetPrefix, opts.Gateway)
|
||||||
|
createCmd := fmt.Sprintf(
|
||||||
|
"pct create %s local:vztmpl/%s --hostname ingress --unprivileged 1 --features nesting=1,keyctl=1,fuse=1 --net0 %s --onboot 1 --memory 2048 --swap 0 --rootfs local:8 2>&1",
|
||||||
|
vmid, shellQuote(template), net0,
|
||||||
|
)
|
||||||
|
if out, err := runRemote(createCmd); err != nil {
|
||||||
|
return fmt.Errorf("pct create ingress LXC: %w (output: %s)", err, string(out))
|
||||||
|
}
|
||||||
|
if out, err := runRemote(fmt.Sprintf("pct start %s", vmid)); err != nil {
|
||||||
|
return fmt.Errorf("pct start ingress LXC: %w (output: %s)", err, string(out))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Wait for LXC network (retry for up to 60s).
|
||||||
|
for i := 0; i < 12; i++ {
|
||||||
|
ipOut, _ := runRemote(fmt.Sprintf("pct exec %s -- hostname -I 2>/dev/null", vmid))
|
||||||
|
ipStr := strings.TrimSpace(string(ipOut))
|
||||||
|
if ipStr != "" {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
time.Sleep(5 * time.Second)
|
||||||
|
}
|
||||||
|
log.Info("ingress_lxc.network_ready", "vmid", vmid, "ip", opts.FloatingIP)
|
||||||
|
|
||||||
|
// Install podman inside the LXC (C-53: idempotent).
|
||||||
|
_, _ = runRemote(fmt.Sprintf(
|
||||||
|
"pct exec %s -- bash -c 'command -v podman >/dev/null 2>&1 || (apt-get update -qq && apt-get install -y -qq podman conmon crun fuse-overlayfs nftables 2>&1)' 2>&1",
|
||||||
|
vmid,
|
||||||
|
))
|
||||||
|
|
||||||
|
// Enable podman-restart.service inside the LXC (research Topic 6).
|
||||||
|
_, _ = runRemote(fmt.Sprintf("pct exec %s -- systemctl enable --now podman-restart.service 2>/dev/null", vmid))
|
||||||
|
|
||||||
|
// Push step-ca root CA into the LXC (C-60: CACertPath).
|
||||||
|
caPath := certpaths.CACertPath()
|
||||||
|
caData, caErr := os.ReadFile(caPath)
|
||||||
|
if caErr != nil {
|
||||||
|
caData = []byte{}
|
||||||
|
}
|
||||||
|
caDelim := "EOF_CA"
|
||||||
|
_, _ = runRemote(fmt.Sprintf(
|
||||||
|
"pct exec %s -- bash -c 'mkdir -p /etc/orca && cat > /etc/orca/step-ca-root.crt <<%s\\n%s\\n%s'",
|
||||||
|
vmid, caDelim, string(caData), caDelim,
|
||||||
|
))
|
||||||
|
|
||||||
|
// Render + write traefik static config inside the LXC (C-58).
|
||||||
|
staticFiles, err := emitter.TraefikEmitter{}.RenderTraefikStaticConfig(emitter.TraefikStaticOpts{})
|
||||||
|
if err == nil {
|
||||||
|
for _, f := range staticFiles {
|
||||||
|
delim := "EOF_TF"
|
||||||
|
_, _ = runRemote(fmt.Sprintf(
|
||||||
|
"pct exec %s -- bash -c 'mkdir -p /etc/traefik/dynamic && cat > %s <<%s\\n%s\\n%s'",
|
||||||
|
vmid, f.Path, delim, f.Content, delim,
|
||||||
|
))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Render + apply nft DNAT+SNAT INSIDE the LXC (DNATTarget =
|
||||||
|
// 127.0.0.1 — traefik runs with --network host inside the LXC).
|
||||||
|
nftFiles, err := emitter.NftEmitter{}.RenderNftConfig(emitter.NftClusterConfig{})
|
||||||
|
if err == nil {
|
||||||
|
for _, f := range nftFiles {
|
||||||
|
delim := "EOF_NF"
|
||||||
|
_, _ = runRemote(fmt.Sprintf(
|
||||||
|
"pct exec %s -- bash -c 'mkdir -p /etc/nftables.d && cat > %s <<%s\\n%s\\n%s'",
|
||||||
|
vmid, f.Path, delim, f.Content, delim,
|
||||||
|
))
|
||||||
|
}
|
||||||
|
_, _ = runRemote(fmt.Sprintf("pct exec %s -- nft add table inet orca-ingress 2>/dev/null || true", vmid))
|
||||||
|
_, _ = runRemote(fmt.Sprintf("pct exec %s -- nft -f /etc/nftables.d/orca.nft 2>&1", vmid))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ensure podman orca-traefik container inside the LXC.
|
||||||
|
traefikExecFn := func(cmd string) ([]byte, error) {
|
||||||
|
return runRemote(fmt.Sprintf("pct exec %s -- bash -c %s 2>&1", vmid, shellQuote(cmd)))
|
||||||
|
}
|
||||||
|
if err := traefik.EnsureTraefikContainerRemote(ctx, "", traefikExecFn); err != nil {
|
||||||
|
log.Warn("ingress_lxc.traefik_failed", "err", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
log.Info("ingress_lxc.provisioned", "vmid", vmid, "ip", opts.FloatingIP)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// GenerateRandomMAC generates a random locally-administered MAC address
|
||||||
|
// (02:XX:XX:XX:XX:XX) for use as the LXC net0 hardware address when the
|
||||||
|
// operator does not provide one (D-261).
|
||||||
|
func GenerateRandomMAC() (string, error) {
|
||||||
|
b := make([]byte, 5)
|
||||||
|
if _, err := rand.Read(b); err != nil {
|
||||||
|
return "", fmt.Errorf("generate MAC: %w", err)
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("02:%02x:%02x:%02x:%02x:%02x", b[0], b[1], b[2], b[3], b[4]), nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
-- REQ-175 / R-024: add ingress_mode column to nodes.
|
||||||
|
-- Values: '' (legacy/default), 'native' (proxmox native — traefik
|
||||||
|
-- in LXC), 'floating-ip' (proxmox floating-IP — ingress LXC owns
|
||||||
|
-- the floating IP). Defaults to empty string for backward
|
||||||
|
-- compatibility with pre-v0.14 nodes.
|
||||||
|
ALTER TABLE nodes ADD COLUMN ingress_mode TEXT NOT NULL DEFAULT '';
|
||||||
+202
-52
@@ -1,78 +1,228 @@
|
|||||||
package traefik
|
package traefik
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
"fmt"
|
"fmt"
|
||||||
"os/exec"
|
"os/exec"
|
||||||
"strings"
|
"strings"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// DefaultVersion is the traefik version tag for the orca-traefik image.
|
||||||
const DefaultVersion = "v3.3.0"
|
const DefaultVersion = "v3.3.0"
|
||||||
|
|
||||||
func downloadURL(version string) string {
|
// DefaultImage is the full image reference for the orca-traefik container.
|
||||||
return fmt.Sprintf("https://github.com/traefik/traefik/releases/download/%s/traefik_%s_linux_amd64.tar.gz", version, version)
|
// The tag is resolved at runtime from the orca version (or "latest" for
|
||||||
}
|
// dev builds). The image is built from Dockerfile.traefik and published
|
||||||
|
// per release (REQ-171).
|
||||||
|
const DefaultImage = "git.cloudinit.dev/coreci/orca-traefik"
|
||||||
|
|
||||||
func InstallLocal(version string) error {
|
// ContainerName is the podman container name for the traefik data plane.
|
||||||
if version == "" {
|
const ContainerName = "orca-traefik"
|
||||||
version = DefaultVersion
|
|
||||||
}
|
|
||||||
if _, err := exec.LookPath("traefik"); err == nil {
|
|
||||||
ensureDirs()
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
url := downloadURL(version)
|
|
||||||
cmd := exec.Command("bash", "-c",
|
|
||||||
fmt.Sprintf(`curl -fsSL %s | tar -xzf - -C /usr/local/bin/ traefik && chmod +x /usr/local/bin/traefik`, url))
|
|
||||||
if out, err := cmd.CombinedOutput(); err != nil {
|
|
||||||
return fmt.Errorf("download traefik %s: %w (output: %s)", version, err, string(out))
|
|
||||||
}
|
|
||||||
ensureDirs()
|
|
||||||
writeSystemdUnit()
|
|
||||||
_ = exec.Command("systemctl", "daemon-reload").Run()
|
|
||||||
_ = exec.Command("systemctl", "enable", "--now", "orca-traefik").Run()
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
|
// RemoteExecFunc runs a command on a remote host and returns combined
|
||||||
|
// output. It is the same signature used by the v0.13 binary installer
|
||||||
|
// and by the proxmox/linux bootstrap SSH sessions.
|
||||||
type RemoteExecFunc func(cmd string) ([]byte, error)
|
type RemoteExecFunc func(cmd string) ([]byte, error)
|
||||||
|
|
||||||
func InstallRemote(version string, execFn RemoteExecFunc) error {
|
// ImageRef returns the full image:tag reference for the orca-traefik
|
||||||
if version == "" {
|
// container. If version is empty or "dev"/"0.1.0-dev", it falls back to
|
||||||
version = DefaultVersion
|
// "latest" (dev builds don't have a published tag).
|
||||||
|
func ImageRef(version string) string {
|
||||||
|
tag := version
|
||||||
|
if tag == "" || tag == "dev" || tag == "0.1.0-dev" || strings.HasSuffix(tag, "-dev") {
|
||||||
|
return fmt.Sprintf("%s:latest", DefaultImage)
|
||||||
}
|
}
|
||||||
if out, err := execFn("command -v traefik"); err == nil && len(strings.TrimSpace(string(out))) > 0 {
|
tag = strings.TrimPrefix(tag, "v")
|
||||||
_, _ = execFn("mkdir -p /etc/traefik/dynamic")
|
return fmt.Sprintf("%s:v%s", DefaultImage, tag)
|
||||||
|
}
|
||||||
|
|
||||||
|
// podmanRunArgs returns the podman run arguments for the traefik
|
||||||
|
// container. The container uses --network host so traefik binds
|
||||||
|
// 127.0.0.1:8080/8443 directly on the host (or LXC) loopback. nft
|
||||||
|
// DNATs public :443/:80 to those loopback ports (R-017/R-024).
|
||||||
|
//
|
||||||
|
// Volume mounts (no SELinux :Z flag — research finding Topic 7):
|
||||||
|
// - /etc/traefik/traefik.yml:ro — static config (overrides baked default; C-58)
|
||||||
|
// - /etc/traefik/dynamic:ro — dynamic config (orca writes atomically via SSH-push)
|
||||||
|
// - /etc/orca/step-ca-root.crt:ro — step-ca root CA (for future mTLS; v0.14 uses tls:{})
|
||||||
|
func podmanRunArgs(imageRef string) []string {
|
||||||
|
return []string{
|
||||||
|
"run", "-d",
|
||||||
|
"--name", ContainerName,
|
||||||
|
"--restart=unless-stopped",
|
||||||
|
"--network", "host",
|
||||||
|
"-v", "/etc/traefik/traefik.yml:/etc/traefik/traefik.yml:ro",
|
||||||
|
"-v", "/etc/traefik/dynamic:/etc/traefik/dynamic:ro",
|
||||||
|
"-v", "/etc/orca/step-ca-root.crt:/etc/orca/step-ca-root.crt:ro",
|
||||||
|
imageRef,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// EnsureTraefikContainerLocal ensures the orca-traefik podman container
|
||||||
|
// is running on the local host. It is idempotent:
|
||||||
|
// 1. If the container is running → no-op.
|
||||||
|
// 2. If the container exists but is stopped → start it.
|
||||||
|
// 3. If the container does not exist → pull the image + run it.
|
||||||
|
//
|
||||||
|
// C-50: if podman is not installed, attempts apt-get install. If that
|
||||||
|
// fails, returns an error with install instructions.
|
||||||
|
//
|
||||||
|
// C-57: if a legacy v0.13 systemd service exists (orca-traefik.service),
|
||||||
|
// it is stopped, disabled, and removed before starting the container.
|
||||||
|
func EnsureTraefikContainerLocal(ctx context.Context, version string) error {
|
||||||
|
imageRef := ImageRef(version)
|
||||||
|
if err := ensurePodmanLocal(ctx); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := removeLegacySystemdUnitLocal(ctx); err != nil {
|
||||||
|
// Non-fatal: legacy unit may not exist on fresh installs.
|
||||||
|
_ = err
|
||||||
|
}
|
||||||
|
if err := ensureDirsLocal(); err != nil {
|
||||||
|
return fmt.Errorf("traefik: ensure dirs: %w", err)
|
||||||
|
}
|
||||||
|
return reconcileContainerLocal(ctx, imageRef)
|
||||||
|
}
|
||||||
|
|
||||||
|
// EnsureTraefikContainerRemote ensures the orca-traefik podman container
|
||||||
|
// is running on a remote host (via SSH exec). Same idempotent logic as
|
||||||
|
// EnsureTraefikContainerLocal but over the provided exec function.
|
||||||
|
func EnsureTraefikContainerRemote(ctx context.Context, version string, execFn RemoteExecFunc) error {
|
||||||
|
imageRef := ImageRef(version)
|
||||||
|
if err := ensurePodmanRemote(execFn); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := removeLegacySystemdUnitRemote(execFn); err != nil {
|
||||||
|
_ = err // non-fatal
|
||||||
|
}
|
||||||
|
if _, err := execFn("mkdir -p /etc/traefik/dynamic /etc/orca"); err != nil {
|
||||||
|
return fmt.Errorf("traefik: ensure remote dirs: %w", err)
|
||||||
|
}
|
||||||
|
return reconcileContainerRemote(execFn, imageRef)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ensurePodmanLocal checks if podman is installed locally and attempts
|
||||||
|
// to install it if absent (C-50).
|
||||||
|
func ensurePodmanLocal(ctx context.Context) error {
|
||||||
|
if _, err := exec.LookPath("podman"); err == nil {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
url := downloadURL(version)
|
// Attempt apt-get install (Ubuntu/Debian).
|
||||||
installCmd := fmt.Sprintf(`curl -fsSL %s | tar -xzf - -C /usr/local/bin/ traefik && chmod +x /usr/local/bin/traefik && mkdir -p /etc/traefik/dynamic`, url)
|
cmd := exec.CommandContext(ctx, "bash", "-c",
|
||||||
if out, err := execFn(installCmd); err != nil {
|
"apt-get update -qq && apt-get install -y -qq podman conmon crun fuse-overlayfs 2>&1")
|
||||||
return fmt.Errorf("download traefik on remote: %w (output: %s)", err, string(out))
|
if out, err := cmd.CombinedOutput(); err != nil {
|
||||||
|
return fmt.Errorf("podman not found and apt-get install failed: %w (output: %s).\nInstall podman manually: apt-get install podman conmon crun fuse-overlayfs", err, string(out))
|
||||||
}
|
}
|
||||||
unit := systemdUnitContent()
|
|
||||||
_, _ = execFn(fmt.Sprintf(`echo '%s' > /etc/systemd/system/orca-traefik.service`, unit))
|
|
||||||
_, _ = execFn("systemctl daemon-reload && systemctl enable --now orca-traefik")
|
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func ensureDirs() {
|
// ensurePodmanRemote checks if podman is installed on the remote host
|
||||||
_ = exec.Command("mkdir", "-p", "/etc/traefik/dynamic").Run()
|
// and attempts to install it if absent (C-50).
|
||||||
|
func ensurePodmanRemote(execFn RemoteExecFunc) error {
|
||||||
|
if out, err := execFn("command -v podman"); err == nil && len(strings.TrimSpace(string(out))) > 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
// Attempt apt-get install on the remote host.
|
||||||
|
cmd := "apt-get update -qq && apt-get install -y -qq podman conmon crun fuse-overlayfs 2>&1"
|
||||||
|
if out, err := execFn(cmd); err != nil {
|
||||||
|
return fmt.Errorf("podman not found on remote and apt-get install failed: %w (output: %s)", err, string(out))
|
||||||
|
}
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func writeSystemdUnit() {
|
// removeLegacySystemdUnitLocal stops, disables, and removes the legacy
|
||||||
_ = exec.Command("bash", "-c", fmt.Sprintf(`echo '%s' > /etc/systemd/system/orca-traefik.service`, systemdUnitContent())).Run()
|
// v0.13 orca-traefik.service systemd unit + /usr/local/bin/traefik
|
||||||
|
// binary (C-57). Idempotent — no-op if the unit doesn't exist.
|
||||||
|
func removeLegacySystemdUnitLocal(ctx context.Context) error {
|
||||||
|
// Check if the legacy unit exists.
|
||||||
|
if _, err := exec.CommandContext(ctx, "systemctl", "is-active", "orca-traefik.service").CombinedOutput(); err == nil {
|
||||||
|
// Unit is active or exists — stop + disable it.
|
||||||
|
_ = exec.CommandContext(ctx, "systemctl", "stop", "orca-traefik.service").Run()
|
||||||
|
_ = exec.CommandContext(ctx, "systemctl", "disable", "orca-traefik.service").Run()
|
||||||
|
}
|
||||||
|
// Remove the unit file and binary.
|
||||||
|
_ = exec.CommandContext(ctx, "rm", "-f", "/etc/systemd/system/orca-traefik.service").Run()
|
||||||
|
_ = exec.CommandContext(ctx, "rm", "-f", "/usr/local/bin/traefik").Run()
|
||||||
|
_ = exec.CommandContext(ctx, "systemctl", "daemon-reload").Run()
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func systemdUnitContent() string {
|
// removeLegacySystemdUnitRemote is the remote SSH variant (C-57).
|
||||||
return `[Unit]
|
func removeLegacySystemdUnitRemote(execFn RemoteExecFunc) error {
|
||||||
Description=Orca Traefik Data Plane
|
cmd := `systemctl is-active orca-traefik.service 2>/dev/null && systemctl stop orca-traefik.service 2>/dev/null; systemctl disable orca-traefik.service 2>/dev/null; rm -f /etc/systemd/system/orca-traefik.service /usr/local/bin/traefik; systemctl daemon-reload 2>/dev/null; true`
|
||||||
After=network.target
|
_, _ = execFn(cmd)
|
||||||
|
return nil
|
||||||
[Service]
|
}
|
||||||
Type=simple
|
|
||||||
ExecStart=/usr/local/bin/traefik --configFile=/etc/traefik/traefik.yml
|
// ensureDirsLocal creates /etc/traefik/dynamic and /etc/orca locally.
|
||||||
Restart=on-failure
|
func ensureDirsLocal() error {
|
||||||
RestartSec=5s
|
if err := exec.Command("mkdir", "-p", "/etc/traefik/dynamic", "/etc/orca").Run(); err != nil {
|
||||||
|
return fmt.Errorf("mkdir: %w", err)
|
||||||
[Install]
|
}
|
||||||
WantedBy=multi-user.target`
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// reconcileContainerLocal implements the idempotent pull+run logic
|
||||||
|
// locally (C-50).
|
||||||
|
func reconcileContainerLocal(ctx context.Context, imageRef string) error {
|
||||||
|
// Check if the container is already running.
|
||||||
|
out, err := exec.CommandContext(ctx, "podman", "inspect", "--format", "{{.State.Running}}", ContainerName).CombinedOutput()
|
||||||
|
if err == nil {
|
||||||
|
v := strings.TrimSpace(string(out))
|
||||||
|
if v == "true" {
|
||||||
|
return nil // already running
|
||||||
|
}
|
||||||
|
// Container exists but is stopped — start it.
|
||||||
|
if _, err := exec.CommandContext(ctx, "podman", "start", ContainerName).CombinedOutput(); err != nil {
|
||||||
|
return fmt.Errorf("podman start %s: %w", ContainerName, err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Container does not exist — pull + run.
|
||||||
|
if out, err := exec.CommandContext(ctx, "podman", "pull", imageRef).CombinedOutput(); err != nil {
|
||||||
|
return fmt.Errorf("podman pull %s: %w (output: %s)", imageRef, err, string(out))
|
||||||
|
}
|
||||||
|
args := append([]string{}, podmanRunArgs(imageRef)...)
|
||||||
|
if out, err := exec.CommandContext(ctx, "podman", args...).CombinedOutput(); err != nil {
|
||||||
|
return fmt.Errorf("podman run: %w (output: %s)", err, string(out))
|
||||||
|
}
|
||||||
|
// Enable podman-restart.service for reboot persistence (research Topic 6).
|
||||||
|
_ = exec.CommandContext(ctx, "systemctl", "enable", "--now", "podman-restart.service").Run()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// reconcileContainerRemote implements the idempotent pull+run logic
|
||||||
|
// over SSH exec.
|
||||||
|
func reconcileContainerRemote(execFn RemoteExecFunc, imageRef string) error {
|
||||||
|
// Check if the container is already running.
|
||||||
|
out, err := execFn(fmt.Sprintf("podman inspect --format '{{.State.Running}}' %s 2>/dev/null", ContainerName))
|
||||||
|
if err == nil {
|
||||||
|
v := strings.TrimSpace(string(out))
|
||||||
|
if v == "true" {
|
||||||
|
return nil // already running
|
||||||
|
}
|
||||||
|
// Container exists but is stopped — start it.
|
||||||
|
if _, err := execFn(fmt.Sprintf("podman start %s 2>/dev/null", ContainerName)); err != nil {
|
||||||
|
return fmt.Errorf("podman start %s: %w", ContainerName, err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Container does not exist — pull + run.
|
||||||
|
if out, err := execFn(fmt.Sprintf("podman pull %s", shellQuote(imageRef))); err != nil {
|
||||||
|
return fmt.Errorf("podman pull %s: %w (output: %s)", imageRef, err, string(out))
|
||||||
|
}
|
||||||
|
runArgs := strings.Join(podmanRunArgs(imageRef), " ")
|
||||||
|
if out, err := execFn(fmt.Sprintf("podman %s", runArgs)); err != nil {
|
||||||
|
return fmt.Errorf("podman run: %w (output: %s)", err, string(out))
|
||||||
|
}
|
||||||
|
// Enable podman-restart.service for reboot persistence (research Topic 6).
|
||||||
|
_, _ = execFn("systemctl enable --now podman-restart.service 2>/dev/null || true")
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// shellQuote wraps a string in single quotes for shell-safe usage.
|
||||||
|
func shellQuote(s string) string {
|
||||||
|
return "'" + strings.ReplaceAll(s, "'", "'\\''") + "'"
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,166 @@
|
|||||||
|
package traefik
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestImageRef(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
version string
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{"v0.13.1", "git.cloudinit.dev/coreci/orca-traefik:v0.13.1"},
|
||||||
|
{"0.13.1", "git.cloudinit.dev/coreci/orca-traefik:v0.13.1"},
|
||||||
|
{"", "git.cloudinit.dev/coreci/orca-traefik:latest"},
|
||||||
|
{"dev", "git.cloudinit.dev/coreci/orca-traefik:latest"},
|
||||||
|
{"0.1.0-dev", "git.cloudinit.dev/coreci/orca-traefik:latest"},
|
||||||
|
{"v1.2.3-dev", "git.cloudinit.dev/coreci/orca-traefik:latest"},
|
||||||
|
}
|
||||||
|
for _, tt := range tests {
|
||||||
|
got := ImageRef(tt.version)
|
||||||
|
if got != tt.want {
|
||||||
|
t.Errorf("ImageRef(%q) = %q, want %q", tt.version, got, tt.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPodmanRunArgs(t *testing.T) {
|
||||||
|
args := podmanRunArgs("git.cloudinit.dev/coreci/orca-traefik:v0.13.1")
|
||||||
|
joined := strings.Join(args, " ")
|
||||||
|
checks := []string{
|
||||||
|
"run -d",
|
||||||
|
"--name orca-traefik",
|
||||||
|
"--restart=unless-stopped",
|
||||||
|
"--network host",
|
||||||
|
"/etc/traefik/traefik.yml:/etc/traefik/traefik.yml:ro",
|
||||||
|
"/etc/traefik/dynamic:/etc/traefik/dynamic:ro",
|
||||||
|
"/etc/orca/step-ca-root.crt:/etc/orca/step-ca-root.crt:ro",
|
||||||
|
"git.cloudinit.dev/coreci/orca-traefik:v0.13.1",
|
||||||
|
}
|
||||||
|
for _, c := range checks {
|
||||||
|
if !strings.Contains(joined, c) {
|
||||||
|
t.Errorf("podmanRunArgs missing %q\nfull: %s", c, joined)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Ensure no :Z flag (research Topic 7)
|
||||||
|
if strings.Contains(joined, ":Z") {
|
||||||
|
t.Errorf("podmanRunArgs should NOT contain :Z SELinux flag\nfull: %s", joined)
|
||||||
|
}
|
||||||
|
// Ensure --restart=always is NOT used (research Topic 6)
|
||||||
|
if strings.Contains(joined, "--restart=always") {
|
||||||
|
t.Errorf("podmanRunArgs should use --restart=unless-stopped, not --restart=always\nfull: %s", joined)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnsureTraefikContainerRemote_ContainerRunning(t *testing.T) {
|
||||||
|
var cmds []string
|
||||||
|
execFn := func(cmd string) ([]byte, error) {
|
||||||
|
cmds = append(cmds, cmd)
|
||||||
|
if strings.Contains(cmd, "podman inspect") {
|
||||||
|
return []byte("true\n"), nil
|
||||||
|
}
|
||||||
|
return []byte(""), nil
|
||||||
|
}
|
||||||
|
err := EnsureTraefikContainerRemote(context.Background(), "v0.13.1", execFn)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
// Should have checked inspect and found it running — no pull/run.
|
||||||
|
if len(cmds) < 1 {
|
||||||
|
t.Fatal("expected at least 1 command (inspect)")
|
||||||
|
}
|
||||||
|
for _, c := range cmds {
|
||||||
|
if strings.Contains(c, "podman pull") {
|
||||||
|
t.Errorf("should not pull when container is running: %s", c)
|
||||||
|
}
|
||||||
|
if strings.Contains(c, "podman run") {
|
||||||
|
t.Errorf("should not run when container is running: %s", c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnsureTraefikContainerRemote_ContainerStopped(t *testing.T) {
|
||||||
|
var cmds []string
|
||||||
|
execFn := func(cmd string) ([]byte, error) {
|
||||||
|
cmds = append(cmds, cmd)
|
||||||
|
if strings.Contains(cmd, "podman inspect") {
|
||||||
|
return []byte("false\n"), nil // stopped
|
||||||
|
}
|
||||||
|
return []byte(""), nil
|
||||||
|
}
|
||||||
|
err := EnsureTraefikContainerRemote(context.Background(), "v0.13.1", execFn)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
// Should have started the container.
|
||||||
|
foundStart := false
|
||||||
|
for _, c := range cmds {
|
||||||
|
if strings.Contains(c, "podman start orca-traefik") {
|
||||||
|
foundStart = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !foundStart {
|
||||||
|
t.Errorf("expected 'podman start orca-traefik' when container is stopped\ncommands: %v", cmds)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnsureTraefikContainerRemote_ContainerAbsent(t *testing.T) {
|
||||||
|
var cmds []string
|
||||||
|
execFn := func(cmd string) ([]byte, error) {
|
||||||
|
cmds = append(cmds, cmd)
|
||||||
|
if strings.Contains(cmd, "podman inspect") {
|
||||||
|
return nil, &execError{"inspect failed: no such container"}
|
||||||
|
}
|
||||||
|
return []byte(""), nil
|
||||||
|
}
|
||||||
|
err := EnsureTraefikContainerRemote(context.Background(), "v0.13.1", execFn)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
// Should have pulled and run.
|
||||||
|
foundPull := false
|
||||||
|
foundRun := false
|
||||||
|
for _, c := range cmds {
|
||||||
|
if strings.Contains(c, "podman pull") {
|
||||||
|
foundPull = true
|
||||||
|
}
|
||||||
|
if strings.Contains(c, "podman run -d") {
|
||||||
|
foundRun = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !foundPull {
|
||||||
|
t.Errorf("expected 'podman pull' when container is absent\ncommands: %v", cmds)
|
||||||
|
}
|
||||||
|
if !foundRun {
|
||||||
|
t.Errorf("expected 'podman run -d' when container is absent\ncommands: %v", cmds)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnsureTraefikContainerRemote_LegacySystemdRemoval(t *testing.T) {
|
||||||
|
var cmds []string
|
||||||
|
execFn := func(cmd string) ([]byte, error) {
|
||||||
|
cmds = append(cmds, cmd)
|
||||||
|
if strings.Contains(cmd, "podman inspect") {
|
||||||
|
return []byte("true\n"), nil // container running
|
||||||
|
}
|
||||||
|
return []byte(""), nil
|
||||||
|
}
|
||||||
|
_ = EnsureTraefikContainerRemote(context.Background(), "v0.13.1", execFn)
|
||||||
|
// Should include legacy systemd unit removal command (C-57).
|
||||||
|
foundLegacyRemoval := false
|
||||||
|
for _, c := range cmds {
|
||||||
|
if strings.Contains(c, "orca-traefik.service") && strings.Contains(c, "stop") {
|
||||||
|
foundLegacyRemoval = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !foundLegacyRemoval {
|
||||||
|
t.Errorf("expected legacy systemd unit removal command (C-57)\ncommands: %v", cmds)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// execError is a simple error type for testing.
|
||||||
|
type execError struct{ msg string }
|
||||||
|
|
||||||
|
func (e *execError) Error() string { return e.msg }
|
||||||
+25
-2
@@ -145,8 +145,8 @@ assert "34 type_linux_available" \
|
|||||||
assert "35 status_deprecated" \
|
assert "35 status_deprecated" \
|
||||||
'$ORCA status 2>&1 | grep -qi "deprecated"'
|
'$ORCA status 2>&1 | grep -qi "deprecated"'
|
||||||
|
|
||||||
assert "36 traefik_installed" \
|
assert "36 traefik_container_running" \
|
||||||
'systemctl is-active orca-traefik 2>/dev/null | grep -q "active" || exit 77'
|
'podman inspect --format "{{.State.Running}}" orca-traefik 2>/dev/null | grep -q "true" || exit 77'
|
||||||
|
|
||||||
assert "37 known_hosts_exists" \
|
assert "37 known_hosts_exists" \
|
||||||
'test -f "$ORCA_HOME/known_hosts" || test -f "$ORCA_HOME/cluster/known_hosts"'
|
'test -f "$ORCA_HOME/known_hosts" || test -f "$ORCA_HOME/cluster/known_hosts"'
|
||||||
@@ -154,6 +154,29 @@ assert "37 known_hosts_exists" \
|
|||||||
assert "38 master_key_exists" \
|
assert "38 master_key_exists" \
|
||||||
'test -f "$ORCA_HOME/cluster/master.key" || test -f "$ORCA_HOME/cluster/master.key.sealed"'
|
'test -f "$ORCA_HOME/cluster/master.key" || test -f "$ORCA_HOME/cluster/master.key.sealed"'
|
||||||
|
|
||||||
|
# --- v0.14 ingress bootstrap assertions (R-024) ---
|
||||||
|
|
||||||
|
assert "40 ingress_nft_table" \
|
||||||
|
'nft list table inet orca-ingress 2>/dev/null | grep -q "chain prerouting"'
|
||||||
|
|
||||||
|
assert "41 ingress_nft_dnat" \
|
||||||
|
'nft list table inet orca-ingress 2>/dev/null | grep -q "dnat to"'
|
||||||
|
|
||||||
|
assert "42 ingress_nft_snat" \
|
||||||
|
'nft list table inet orca-ingress 2>/dev/null | grep -q "masquerade"'
|
||||||
|
|
||||||
|
assert "43 ingress_dynamic_dir" \
|
||||||
|
'test -d /etc/traefik/dynamic'
|
||||||
|
|
||||||
|
assert "44 ingress_step_ca" \
|
||||||
|
'test -f /etc/orca/step-ca-root.crt'
|
||||||
|
|
||||||
|
assert "45 ingress_traefik_yml" \
|
||||||
|
'test -f /etc/traefik/traefik.yml'
|
||||||
|
|
||||||
|
assert "46 ingress_doctor_pass" \
|
||||||
|
'$ORCA doctor ingress 2>&1 | grep -q "PASS"'
|
||||||
|
|
||||||
# --- Report ---
|
# --- Report ---
|
||||||
|
|
||||||
echo "=========================================="
|
echo "=========================================="
|
||||||
|
|||||||
@@ -0,0 +1,163 @@
|
|||||||
|
package tests
|
||||||
|
|
||||||
|
import (
|
||||||
|
"log/slog"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"git.cloudinit.dev/coreci/orca/internal/emitter"
|
||||||
|
"git.cloudinit.dev/coreci/orca/internal/jobspec"
|
||||||
|
"git.cloudinit.dev/coreci/orca/internal/proxmox"
|
||||||
|
"git.cloudinit.dev/coreci/orca/internal/traefik"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TestNftEmitter_PostroutingAndDNATTarget (REQ-173) verifies the nft
|
||||||
|
// emitter renders the postrouting masquerade chain and supports
|
||||||
|
// DNATTarget substitution.
|
||||||
|
func TestNftEmitter_PostroutingAndDNATTarget(t *testing.T) {
|
||||||
|
files, err := emitter.NftEmitter{}.RenderNftConfig(emitter.NftClusterConfig{
|
||||||
|
DNATTarget: "10.99.0.10",
|
||||||
|
EnableSNAT: true,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("RenderNftConfig: %v", err)
|
||||||
|
}
|
||||||
|
c := files[0].Content
|
||||||
|
if !strings.Contains(c, "chain postrouting") {
|
||||||
|
t.Errorf("missing postrouting chain:\n%s", c)
|
||||||
|
}
|
||||||
|
if !strings.Contains(c, "masquerade") {
|
||||||
|
t.Errorf("missing masquerade rule:\n%s", c)
|
||||||
|
}
|
||||||
|
if !strings.Contains(c, "dnat to 10.99.0.10:8443") {
|
||||||
|
t.Errorf("missing custom DNAT target:\n%s", c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestNftEmitter_PriorityMinus10 (research Topic 2) verifies the input
|
||||||
|
// and forward chains use priority -10 for pve-firewall coexistence.
|
||||||
|
func TestNftEmitter_PriorityMinus10(t *testing.T) {
|
||||||
|
files, _ := emitter.NftEmitter{}.RenderNftConfig(emitter.NftClusterConfig{})
|
||||||
|
c := files[0].Content
|
||||||
|
if !strings.Contains(c, "hook input priority -10;") {
|
||||||
|
t.Errorf("input chain should use priority -10:\n%s", c)
|
||||||
|
}
|
||||||
|
if !strings.Contains(c, "hook forward priority -10;") {
|
||||||
|
t.Errorf("forward chain should use priority -10:\n%s", c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestTraefikEmitter_TLSModel (REQ-172) verifies the dynamic config
|
||||||
|
// emits tls: {} and does NOT contain certResolver (dropped in v0.14).
|
||||||
|
func TestTraefikEmitter_TLSModel(t *testing.T) {
|
||||||
|
spec := &jobspec.WorkloadSpec{
|
||||||
|
Name: "test-svc",
|
||||||
|
Kind: "Service",
|
||||||
|
Ports: []jobspec.PortSpec{{Name: "http"}},
|
||||||
|
}
|
||||||
|
node := &emitter.Node{
|
||||||
|
Hostname: "test-node",
|
||||||
|
}
|
||||||
|
files, err := emitter.TraefikEmitter{}.Render(spec, node)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Render: %v", err)
|
||||||
|
}
|
||||||
|
c := files[0].Content
|
||||||
|
if !strings.Contains(c, "tls: {}") {
|
||||||
|
t.Errorf("missing tls: {} (v0.14 model):\n%s", c)
|
||||||
|
}
|
||||||
|
if strings.Contains(c, "certResolver: orca") {
|
||||||
|
t.Errorf("certResolver: orca should be removed (v0.14):\n%s", c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestTraefikImageRef verifies the image reference resolution for the
|
||||||
|
// orca-traefik podman container.
|
||||||
|
func TestTraefikImageRef(t *testing.T) {
|
||||||
|
ref := traefik.ImageRef("v0.13.7")
|
||||||
|
want := "git.cloudinit.dev/coreci/orca-traefik:v0.13.7"
|
||||||
|
if ref != want {
|
||||||
|
t.Errorf("ImageRef(v0.13.7) = %q, want %q", ref, want)
|
||||||
|
}
|
||||||
|
// Dev build falls back to latest.
|
||||||
|
ref = traefik.ImageRef("dev")
|
||||||
|
if ref != "git.cloudinit.dev/coreci/orca-traefik:latest" {
|
||||||
|
t.Errorf("ImageRef(dev) = %q, want latest", ref)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestProxmox_FloatingIP_LXC_ProvisioningCommands (REQ-176) verifies
|
||||||
|
// the ProvisionIngressLXC function sends the correct pct create
|
||||||
|
// command with the right net0 parameters.
|
||||||
|
func TestProxmox_FloatingIP_LXC_ProvisioningCommands(t *testing.T) {
|
||||||
|
var cmds []string
|
||||||
|
execFn := func(cmd string) ([]byte, error) {
|
||||||
|
cmds = append(cmds, cmd)
|
||||||
|
// Simulate: pct status returns "absent" on first call, then OK.
|
||||||
|
if strings.Contains(cmd, "pct status 201") {
|
||||||
|
return []byte("absent\n"), nil
|
||||||
|
}
|
||||||
|
if strings.Contains(cmd, "pct create") {
|
||||||
|
return []byte(""), nil
|
||||||
|
}
|
||||||
|
if strings.Contains(cmd, "pct start 201") {
|
||||||
|
return []byte(""), nil
|
||||||
|
}
|
||||||
|
if strings.Contains(cmd, "hostname -I") {
|
||||||
|
return []byte("203.0.113.10\n"), nil
|
||||||
|
}
|
||||||
|
return []byte(""), nil
|
||||||
|
}
|
||||||
|
err := proxmox.ProvisionIngressLXC(nil, execFn, proxmox.FloatingIPOptions{
|
||||||
|
FloatingIP: "203.0.113.10",
|
||||||
|
Gateway: "203.0.113.1",
|
||||||
|
MAC: "02:01:02:03:04:05",
|
||||||
|
NetPrefix: 24,
|
||||||
|
LXCTemplate: "ubuntu-24.04",
|
||||||
|
}, slog.Default())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ProvisionIngressLXC: %v", err)
|
||||||
|
}
|
||||||
|
// Verify pct create has the right net0 params.
|
||||||
|
foundCreate := false
|
||||||
|
for _, c := range cmds {
|
||||||
|
if strings.Contains(c, "pct create") {
|
||||||
|
foundCreate = true
|
||||||
|
if !strings.Contains(c, "hostname ingress") {
|
||||||
|
t.Errorf("pct create missing hostname ingress: %s", c)
|
||||||
|
}
|
||||||
|
if !strings.Contains(c, "hwaddr=02:01:02:03:04:05") {
|
||||||
|
t.Errorf("pct create missing hwaddr: %s", c)
|
||||||
|
}
|
||||||
|
if !strings.Contains(c, "ip=203.0.113.10/24") {
|
||||||
|
t.Errorf("pct create missing ip: %s", c)
|
||||||
|
}
|
||||||
|
if !strings.Contains(c, "gw=203.0.113.1") {
|
||||||
|
t.Errorf("pct create missing gw: %s", c)
|
||||||
|
}
|
||||||
|
if !strings.Contains(c, "nesting=1,keyctl=1,fuse=1") {
|
||||||
|
t.Errorf("pct create missing features (research Topic 3): %s", c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !foundCreate {
|
||||||
|
t.Errorf("pct create command not sent\ncommands: %v", cmds)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestProxmox_GenerateRandomMAC (D-261) verifies MAC generation produces
|
||||||
|
// a valid locally-administered MAC.
|
||||||
|
func TestProxmox_GenerateRandomMAC(t *testing.T) {
|
||||||
|
mac, err := proxmox.GenerateRandomMAC()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GenerateRandomMAC: %v", err)
|
||||||
|
}
|
||||||
|
if !strings.HasPrefix(mac, "02:") {
|
||||||
|
t.Errorf("MAC should start with 02: (locally administered): %s", mac)
|
||||||
|
}
|
||||||
|
// Verify it's 6 octets.
|
||||||
|
parts := strings.Split(mac, ":")
|
||||||
|
if len(parts) != 6 {
|
||||||
|
t.Errorf("MAC should have 6 octets: %s", mac)
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user