Compare commits
24 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Jon Chery
|
7d453ac5a6 | ||
|
Jon Chery
|
cc09351185 | ||
|
Jon Chery
|
405877ee27 | ||
|
Jon Chery
|
5af2b1d660 | ||
|
Jon Chery
|
075d8bbc5f | ||
|
Jon Chery
|
a2738f56c4 | ||
|
Jon Chery
|
df3f980fa0 | ||
|
Jon Chery
|
7e26490b5f | ||
|
Jon Chery
|
8ca5ffd0fc | ||
|
Jon Chery
|
5c07fafa18 | ||
|
Jon Chery
|
fb89c30d91 | ||
|
Jon Chery
|
b8f766de03 | ||
|
Jon Chery
|
566145d45a | ||
|
Jon Chery
|
4a97cb1ea2 | ||
|
Jon Chery
|
e55dfed716 | ||
|
Jon Chery
|
1b71e0515f | ||
|
Jon Chery
|
d7896e5287 | ||
|
Jon Chery
|
fed24b93e9 | ||
|
Jon Chery
|
3be86e6daf | ||
|
Jon Chery
|
437908662f | ||
|
Jon Chery
|
1c2843a39d | ||
|
Jon Chery
|
82dfe7a941 | ||
|
Jon Chery
|
699196f368 | ||
|
Jon Chery
|
61c97c847c |
@@ -1,17 +1,17 @@
|
||||
{
|
||||
"phase": 2,
|
||||
"stage": "complete",
|
||||
"milestone": "v0.15",
|
||||
"milestone_slug": "ci-release-pipeline",
|
||||
"milestone": "v0.16",
|
||||
"milestone_slug": "release-binary-fix",
|
||||
"phase_role": "final",
|
||||
"attempts": 0,
|
||||
"updated_at": "2026-08-10T21:05:00Z",
|
||||
"updated_at": "2026-08-12T23:15:00Z",
|
||||
"milestone_complete": true,
|
||||
"previous_milestone": "v0.14",
|
||||
"phases_shipped": ["P0","P1","P2"],
|
||||
"tags_shipped": ["v0.14.0","v0.14.1"],
|
||||
"previous_milestone": "v0.15",
|
||||
"phases_shipped": ["P0", "P1", "P2"],
|
||||
"tags_shipped": ["v0.15.0", "v0.15.1", "v0.15.2"],
|
||||
"requirements": {
|
||||
"covered": [180,181,182],
|
||||
"covered": [183, 184],
|
||||
"partial": []
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,117 @@
|
||||
# CLARIFY v0.16: Release Binary Asset Fix
|
||||
|
||||
## Decisions
|
||||
|
||||
| ID | Decision | Rationale | Confidence |
|
||||
|----|----------|-----------|------------|
|
||||
| D-269 | Auth the coreci clone via URL embedding | The `actions/checkout@v4` step injects auth only for the orca repo. The bare `git clone https://git.cloudinit.dev/coreci/coreci.git` has no credentials and fails with `fatal: could not read Username`. Embedding `https://cloudinit-bot:${GITEA_TOKEN}@git.cloudinit.dev/...` in the clone URL is the simplest fix — no git credential helper config needed. | 0.95 |
|
||||
| D-270 | Rewrite .coreci.yml to CoreCI native `jobs:` format | CoreCI's `Pipeline` struct (`workflows/pipeline/pipeline.go`) only has `Jobs`/`Services`/`Env` fields. The orca `.coreci.yml` uses `pipelines:`/`steps:`/`image:`/`commands:` which are silently dropped by `yaml.Unmarshal` → empty `Jobs` map → zero jobs execute. `validate()` does not reject empty jobs. | 0.97 (validated against CoreCI source) |
|
||||
| D-271 | No tag-conditional rules in .coreci.yml | The Gitea Actions workflow already gates on `on: push: tags: ['v*']`. Every `coreci run` invocation is already a release run. CoreCI's `cicontext/github.go` doesn't even set `CI_COMMIT_TAG` for Gitea Actions (it maps `GITHUB_REF_NAME` → `CI_COMMIT_BRANCH`). | 0.90 |
|
||||
| D-272 | Use `CI_COMMIT_BRANCH` for tag name | On a tag push, Gitea Actions sets `GITHUB_REF_NAME=v0.15.1`. CoreCI's `github.go` maps this to `CI_COMMIT_BRANCH`. So `CI_COMMIT_BRANCH` contains the tag name on tag pushes. This is the env var to use for version injection in `invoke:` scripts. | 0.90 |
|
||||
| D-273 | Handle duplicate release gracefully | The CIAgent ship workflow creates releases (title+body, no binary) via the Gitea API. The `coreci run` release job runs later (after the Gitea Actions workflow triggers). `tea releases create` fails if the release exists. Fallback: query the release ID by tag and attach assets via the Gitea API `POST /releases/{id}/assets` endpoint. | 0.92 |
|
||||
| D-274 | Shell-friendly jobs (no `apk add`) | The Gitea Actions runner runs the `ci` job inside `docker.gitea.com/runner-images:ubuntu-latest` (ubuntu, not alpine). CoreCI's shell-isolated executor (the likely fallback if podman/docker aren't in the runner container) runs `sh -c <invoke>` directly. `apk add` won't work on ubuntu. Use `curl` (pre-installed) for tool downloads. | 0.85 |
|
||||
| D-275 | `GITEA_TOKEN` via PassThroughEnv | CoreCI's `PassThroughEnv()` forwards env vars not in the deny-list. `GITEA_TOKEN` is not in `systemVars` or `predefinedCIVars`, so it IS forwarded when `IsRunningInCI()` is true (Gitea Actions sets `GITHUB_ACTIONS=true`). The Gitea Actions workflow sets `GITEA_TOKEN: ${{ secrets.PAT_TOKEN }}` in the `coreci run` step env. So `GITEA_TOKEN` is available in job `invoke:` scripts. Belt-and-suspenders: also declare `vars: { GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }} }` so CoreCI's secret resolver picks it up via `os.Getenv` fallback. | 0.88 |
|
||||
|
||||
## Research: CoreCI .coreci.yml format (validated against source)
|
||||
|
||||
CoreCI's `Pipeline` struct (`workflows/pipeline/pipeline.go`):
|
||||
```go
|
||||
type Pipeline struct {
|
||||
Jobs map[string]Job `yaml:"jobs"`
|
||||
Services map[string]Service `yaml:"services,omitempty"`
|
||||
Env EnvBlock `yaml:"env,omitempty"`
|
||||
}
|
||||
type Job struct {
|
||||
Needs []string `yaml:"needs,omitempty"`
|
||||
Plugin string `yaml:"plugin,omitempty"`
|
||||
Invoke string `yaml:"invoke,omitempty"`
|
||||
Vars map[string]string `yaml:"vars,omitempty"`
|
||||
MemoryLimitMb int `yaml:"memory_limit_mb,omitempty"`
|
||||
TimeoutMs int `yaml:"timeout_ms,omitempty"`
|
||||
Rules []Rule `yaml:"rules,omitempty"`
|
||||
Tags []string `yaml:"tags,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
Key facts (from `docs/CORECI-YML.md` + source):
|
||||
- `plugin` and `invoke` are mutually exclusive on the same job — BUT the
|
||||
isolated_shell executor checks `invoke` first (line 52) and only falls
|
||||
back to `plugin` if `invoke` is empty. So a job with BOTH `plugin:
|
||||
docker://golang:1.25` AND `invoke: "go build ..."` works: container
|
||||
executors run the invoke inside the container; shell-isolated runs
|
||||
the invoke directly (ignoring the image). This is the correct pattern.
|
||||
- Unknown YAML fields are silently dropped (no strict decode). This is
|
||||
why the current `pipelines:`/`steps:`/`image:`/`commands:` format
|
||||
produces an empty `Jobs` map with no error.
|
||||
- `validate()` does NOT reject empty `Jobs` maps — it only checks for
|
||||
duplicate names and plugin/invoke mutual exclusivity within existing
|
||||
jobs.
|
||||
- Secret interpolation `${{ secrets.KEY }}` works only in job `vars:`
|
||||
values. The resolver (run.go:146-153) checks the local secret store
|
||||
first, then falls back to `os.Getenv(key)`.
|
||||
- Shell `${VAR}` expansion works inside `invoke:` strings at runtime
|
||||
(via `sh -c`), but NOT in YAML field values at parse time.
|
||||
- `env.from_ci` controls which CI vars are injected; if empty, all
|
||||
detected CI vars are merged.
|
||||
|
||||
## Research: Gitea Actions runner environment
|
||||
|
||||
The `ci` job (no `container:` field) runs inside
|
||||
`docker.gitea.com/runner-images:ubuntu-latest`. The `Set up Go` step
|
||||
installs Go 1.25. CoreCI's executor chain is `podman,docker,shell-isolated`.
|
||||
If podman/docker aren't in the runner container, jobs fall back to
|
||||
`shell-isolated` which runs `sh -c <invoke>` directly. Go commands work
|
||||
in shell-isolated mode (Go is on PATH). Tool installation via `go install`
|
||||
works (needs Go + network). `gitleaks` binary download via `curl` works.
|
||||
|
||||
## Research: Gitea Actions CI context (CoreCI detection)
|
||||
|
||||
CoreCI's `cicontext/github.go`:
|
||||
```go
|
||||
func isGitHub() bool { return os.Getenv("GITHUB_ACTIONS") == "true" }
|
||||
func normalizeGitHub() map[string]string {
|
||||
return map[string]string{
|
||||
"CI": "true",
|
||||
"CI_COMMIT_SHA": envOrDefault("GITHUB_SHA", ""),
|
||||
"CI_COMMIT_BRANCH": envOrDefault("GITHUB_REF_NAME", ""),
|
||||
// ... (no CI_COMMIT_TAG)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
On a tag push: `GITHUB_REF_NAME=v0.15.1` → `CI_COMMIT_BRANCH=v0.15.1`.
|
||||
`CI_COMMIT_TAG` is NOT set — CoreCI doesn't populate it for Gitea Actions.
|
||||
|
||||
## Plan
|
||||
|
||||
### Phase 1 (only execution phase)
|
||||
|
||||
**Files to modify:**
|
||||
|
||||
1. `.gitea/workflows/release.yml` — move `GITEA_TOKEN` env to the
|
||||
"Install CoreCI" step and embed it in the clone URL:
|
||||
```yaml
|
||||
- name: Install CoreCI
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.PAT_TOKEN }}
|
||||
run: |
|
||||
git clone --depth=1 https://cloudinit-bot:${GITEA_TOKEN}@git.cloudinit.dev/coreci/coreci.git /tmp/coreci
|
||||
cd /tmp/coreci
|
||||
CGO_ENABLED=0 go build -tags sqlite_go,embed -o /usr/local/bin/coreci ./cmd/coreci
|
||||
coreci version
|
||||
```
|
||||
|
||||
2. `.coreci.yml` — full rewrite to CoreCI native `jobs:` format:
|
||||
- DAG: `go-vet` → fan-out to `verify-reqs`, `gosec`, `govulncheck`,
|
||||
`gitleaks` → `build` → `test` → `release`
|
||||
- Each job: `plugin: docker://golang:1.25.12` + `invoke: |` (multi-line)
|
||||
- `build` job: version injection via `CI_COMMIT_BRANCH` (tag) +
|
||||
`CI_COMMIT_SHA` + `date` for build time
|
||||
- `release` job: build tarball + SHA256SUMS, install `tea` via curl,
|
||||
create release with assets (fallback to API asset attachment if
|
||||
release exists), verify asset count ≥ 2 (REQ-097 gate C-21)
|
||||
- `GITEA_TOKEN` via `vars: { GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }} }`
|
||||
- No tag-conditional rules (workflow already gates on tags)
|
||||
|
||||
3. `scripts/trigger_coreci.sh` — no changes needed (Gitea Actions is the
|
||||
trigger; the hook is for branch-push CI only).
|
||||
@@ -0,0 +1,84 @@
|
||||
# PLAN v0.16: Release Binary Asset Fix
|
||||
|
||||
## Milestone Summary
|
||||
|
||||
**Milestone**: v0.16 (fix type — tags on v0.15.x patch line)
|
||||
**Phases**: P0 (pre-execution) → P1 (fix) → P2 (final review+ship)
|
||||
**Requirements**: REQ-183 (clone auth), REQ-184 (.coreci.yml rewrite)
|
||||
|
||||
## Phase 1: Fix Gitea Actions clone auth + rewrite .coreci.yml
|
||||
|
||||
### Wave 1: Both fixes (single wave — they are independent files)
|
||||
|
||||
**Task 1.1 (REQ-183): Fix `.gitea/workflows/release.yml` — auth the coreci clone**
|
||||
|
||||
File: `.gitea/workflows/release.yml`
|
||||
|
||||
Current failing step:
|
||||
```yaml
|
||||
- name: Install CoreCI
|
||||
run: |
|
||||
git clone --depth=1 https://git.cloudinit.dev/coreci/coreci.git /tmp/coreci
|
||||
cd /tmp/coreci
|
||||
CGO_ENABLED=0 go build -tags sqlite_go,embed -o /usr/local/bin/coreci ./cmd/coreci
|
||||
coreci version
|
||||
```
|
||||
|
||||
Fix: add `GITEA_TOKEN` env and embed in clone URL:
|
||||
```yaml
|
||||
- name: Install CoreCI
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.PAT_TOKEN }}
|
||||
run: |
|
||||
git clone --depth=1 https://cloudinit-bot:${GITEA_TOKEN}@git.cloudinit.dev/coreci/coreci.git /tmp/coreci
|
||||
cd /tmp/coreci
|
||||
CGO_ENABLED=0 go build -tags sqlite_go,embed -o /usr/local/bin/coreci ./cmd/coreci
|
||||
coreci version
|
||||
```
|
||||
|
||||
**Task 1.2 (REQ-184): Rewrite `.coreci.yml` to CoreCI native `jobs:` format**
|
||||
|
||||
File: `.coreci.yml`
|
||||
|
||||
Convert from `pipelines:`/`steps:`/`image:`/`commands:` to `jobs:`/`plugin:`/`invoke:`/`vars:` with a DAG.
|
||||
|
||||
DAG structure:
|
||||
```
|
||||
go-vet ──→ verify-reqs ──┐
|
||||
├──→ gosec ────────┤
|
||||
├──→ govulncheck ──┤──→ build ──→ test ──→ release
|
||||
└──→ gitleaks ─────┘
|
||||
```
|
||||
|
||||
Key adaptations:
|
||||
- `plugin: docker://golang:1.25.12` on each job (container if available, shell-isolated fallback)
|
||||
- `invoke: |` for multi-line commands (shell expansion works via `sh -c`)
|
||||
- `GITEA_TOKEN` via `vars: { GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }} }` (resolved from env)
|
||||
- `CI_COMMIT_BRANCH` (tag name on tag push) and `CI_COMMIT_SHA` for version injection
|
||||
- No tag-conditional rules (workflow gates on `on: push: tags: ['v*']`)
|
||||
- `release` job: build tarball + SHA256SUMS, install tea via curl, create release
|
||||
with assets (fallback to Gitea API asset attachment if release exists),
|
||||
verify asset count ≥ 2 (REQ-097 gate C-21)
|
||||
- No `apk add` (runner is ubuntu, not alpine)
|
||||
|
||||
### Must-haves (verification gates)
|
||||
|
||||
- [ ] `.gitea/workflows/release.yml` "Install CoreCI" step has `GITEA_TOKEN` env and token in clone URL
|
||||
- [ ] `.coreci.yml` uses `jobs:` top-level key (not `pipelines:`)
|
||||
- [ ] Each job has `plugin:` and/or `invoke:` (mutually exclusive rule)
|
||||
- [ ] DAG via `needs:` (validate → build → test → release)
|
||||
- [ ] `GITEA_TOKEN` passed via job `vars:` with `${{ secrets.GITEA_TOKEN }}`
|
||||
- [ ] Release job handles duplicate release (fallback to API asset attachment)
|
||||
- [ ] Release job verifies asset count ≥ 2 (REQ-097)
|
||||
- [ ] No `apk add` commands (ubuntu runner, not alpine)
|
||||
- [ ] No `${VAR}` interpolation in YAML fields (only in `invoke:` via sh -c)
|
||||
- [ ] `make verify-reqs` passes (ROADMAP ↔ REQUIREMENTS consistency)
|
||||
|
||||
### Verification
|
||||
|
||||
1. `make verify-reqs` — ROADMAP/REQUIREMENTS consistency
|
||||
2. `go vet ./...` — no vet errors
|
||||
3. `gofmt -l .` — no formatting issues
|
||||
4. YAML validity check for `.coreci.yml` and `.gitea/workflows/release.yml`
|
||||
5. Confirm `.coreci.yml` has `jobs:` key and at least 5 jobs (go-vet, verify-reqs, gosec, govulncheck, gitleaks, build, test, release)
|
||||
6. Confirm `.gitea/workflows/release.yml` "Install CoreCI" step references `GITEA_TOKEN`
|
||||
@@ -454,3 +454,28 @@ Gitea Actions workflow that triggers on tag pushes, installs the
|
||||
- Tags on v0.14.x patch line: `v0.14.0` (P0) ... `v0.14.2` (P2 final = v0.15 milestone release).
|
||||
- Milestone branch: `milestone/v0.15-ci-release-pipeline`.
|
||||
- REQ-182 is complete: `PAT_TOKEN` secret created via `tea actions secrets create PAT_TOKEN <value> --repo coreci/orca`.
|
||||
|
||||
## Milestone v0.16: Release Binary Asset Fix
|
||||
|
||||
**Scope**: fix the root cause of releases shipping with zero binary
|
||||
assets. v0.15 added a Gitea Actions workflow but it never executed
|
||||
successfully due to two compounding bugs: (1) the `git clone` of the
|
||||
private `coreci` repo in the workflow had no credentials, causing the
|
||||
"Install CoreCI" step to fail; (2) the `.coreci.yml` used an invalid
|
||||
`pipelines:`/`steps:`/`image:`/`commands:` format that CoreCI does not
|
||||
recognize (CoreCI's native format is `jobs:` with `plugin:`/`invoke:`
|
||||
/`vars:` and a DAG via `needs:`). Both issues must be fixed for the
|
||||
release pipeline to actually build and upload binaries.
|
||||
|
||||
| ID | Requirement | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-183 | Fix `.gitea/workflows/release.yml` "Install CoreCI" step: the `git clone` of the private `coreci` repo fails because the clone command has no credentials. The `actions/checkout@v4` step only injects auth for the orca repo (via `http.https://git.cloudinit.dev/.extraheader`), not for the subsequent bare `git clone` of the coreci repo. Fix: embed the `PAT_TOKEN` in the clone URL (`https://cloudinit-bot:${GITEA_TOKEN}@git.cloudinit.dev/coreci/coreci.git`) and pass `GITEA_TOKEN: ${{ secrets.PAT_TOKEN }}` as env to the "Install CoreCI" step | Critical | **v0.16 P1** | complete |
|
||||
| REQ-184 | Rewrite `.coreci.yml` from the invalid `pipelines:`/`steps:`/`image:`/`commands:` format to CoreCI's native `jobs:`/`plugin:`/`invoke:`/`vars:` format with a proper DAG (`needs:`). CoreCI's `Pipeline` struct only has `Jobs`/`Services`/`Env` fields — unknown top-level keys and unknown job fields are silently dropped by `yaml.Unmarshal`, producing an empty `Jobs` map. `coreci run` then executes zero jobs (validate does not reject empty jobs). The rewrite must: (a) convert each pipeline to a job with `plugin: docker://golang:1.25.12` and `invoke:` for the commands, (b) use `needs:` for DAG ordering (validate→build→test→release), (c) pass `GITEA_TOKEN` via `vars: { GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }} }` (resolved from env via CoreCI's secret resolver `os.Getenv` fallback), (d) use `CI_COMMIT_BRANCH` (tag name on tag push, from CoreCI's github.go CI context) and `CI_COMMIT_SHA` for version injection, (e) handle the case where the release already exists (created by the CIAgent ship workflow with title+body but no binary) by falling back to Gitea API asset attachment, (f) verify assets are actually attached after release creation (REQ-097 gate C-21) | Critical | **v0.16 P1** | complete |
|
||||
|
||||
### Scope notes (v0.16)
|
||||
|
||||
- REQ-183..REQ-184 = 2 net-new requirements (REQ count grows 175 -> 177).
|
||||
- 3 phases (P0 + P1 + P2 final); fix milestone (no `feat` phases — CI infrastructure).
|
||||
- Tags on v0.15.x patch line: `v0.15.0` (P0) ... `v0.15.2` (P2 final = v0.16 milestone release).
|
||||
- Milestone branch: `milestone/v0.16-release-binary-fix`.
|
||||
- Root cause analysis confirmed: all 87 releases in the repo's history have zero binary assets — this has never worked. The releases are created by the CIAgent ship workflow (via Gitea API, title+body only); the binary upload is exclusively the `.coreci.yml` release job's job, and that job has never executed.
|
||||
|
||||
@@ -0,0 +1,159 @@
|
||||
# RESEARCH v0.16: Release Binary Asset Fix
|
||||
|
||||
## Root Cause: Two Compounding Bugs
|
||||
|
||||
### Bug 1: Gitea Actions workflow cannot clone private coreci repo
|
||||
|
||||
`.gitea/workflows/release.yml` step "Install CoreCI" runs:
|
||||
```bash
|
||||
git clone --depth=1 https://git.cloudinit.dev/coreci/coreci.git /tmp/coreci
|
||||
```
|
||||
|
||||
The `coreci` repo is **private** (confirmed: `curl -s -o /dev/null -w "%{http_code}" https://git.cloudinit.dev/coreci/coreci` returns 404 without auth; the Gitea API reports `"private": true`).
|
||||
|
||||
The `actions/checkout@v4` step injects auth only for the orca repo:
|
||||
```
|
||||
[command]/usr/bin/git config --local http.https://git.cloudinit.dev/.extraheader AUTHORIZATION: basic ***
|
||||
```
|
||||
|
||||
The subsequent bare `git clone` of the coreci repo has no credentials. Runner logs confirm the failure:
|
||||
```
|
||||
fatal: could not read Username for 'https://git.cloudinit.dev': No such device or address
|
||||
❌ Failure - Main Install CoreCI
|
||||
exitcode '128': failure
|
||||
```
|
||||
|
||||
Both Gitea Actions runs (v0.14.1 run #3540, v0.14.2 run #3544) failed at this step. The `container-orca` and `container-traefik` jobs (with `needs: ci`) were skipped. `coreci run` never executed.
|
||||
|
||||
**Fix (REQ-183):** Pass `GITEA_TOKEN: ${{ secrets.PAT_TOKEN }}` as env to the "Install CoreCI" step and embed it in the clone URL:
|
||||
```bash
|
||||
git clone --depth=1 https://cloudinit-bot:${GITEA_TOKEN}@git.cloudinit.dev/coreci/coreci.git /tmp/coreci
|
||||
```
|
||||
|
||||
### Bug 2: .coreci.yml uses a format CoreCI does not understand
|
||||
|
||||
The orca `.coreci.yml` uses:
|
||||
```yaml
|
||||
pipelines:
|
||||
validate:
|
||||
steps:
|
||||
- name: go-version
|
||||
image: golang:1.25.12
|
||||
commands: [go version, gofmt -l ., go vet ./...]
|
||||
release:
|
||||
when:
|
||||
ref: "refs/tags/v*"
|
||||
steps: [...]
|
||||
```
|
||||
|
||||
CoreCI's actual format (validated against `workflows/pipeline/pipeline.go`):
|
||||
```go
|
||||
type Pipeline struct {
|
||||
Jobs map[string]Job `yaml:"jobs"`
|
||||
Services map[string]Service `yaml:"services,omitempty"`
|
||||
Env EnvBlock `yaml:"env,omitempty"`
|
||||
}
|
||||
type Job struct {
|
||||
Needs []string `yaml:"needs,omitempty"`
|
||||
Plugin string `yaml:"plugin,omitempty"`
|
||||
Invoke string `yaml:"invoke,omitempty"`
|
||||
Vars map[string]string `yaml:"vars,omitempty"`
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
`yaml.Unmarshal` into a struct silently drops unknown fields (no `KnownFields(true)`). The `pipelines:` top-level key doesn't map to any struct field → `Jobs` is an empty map. CoreCI's `validate()` does NOT reject empty jobs:
|
||||
```go
|
||||
func validate(p *Pipeline) error {
|
||||
seen := make(map[string]struct{}, len(p.Jobs))
|
||||
for name, job := range p.Jobs { ... }
|
||||
return nil // empty Jobs → no iterations → nil error
|
||||
}
|
||||
```
|
||||
|
||||
So `coreci run` loads an empty pipeline, runs zero jobs, and exits successfully — no build, no tarball, no asset upload.
|
||||
|
||||
Additional format mismatches in the current `.coreci.yml`:
|
||||
- `when: ref: "refs/tags/v*"` — CoreCI has no `when` at pipeline/job level; it uses `rules: [{if: "CI_COMMIT_TAG"}]` on jobs. And CoreCI's `github.go` doesn't set `CI_COMMIT_TAG` for Gitea Actions. But the Gitea Actions workflow already gates on `on: push: tags: ['v*']`, so no conditional is needed.
|
||||
- `${CI_COMMIT_TAG:-dev}` in `env:` blocks — CoreCI does not expand `${VAR}` at YAML parse time. Only `${{ secrets.KEY }}` is interpolated. Shell expansion works inside `invoke:` via `sh -c`.
|
||||
- `GITEA_TOKEN: ${GITEA_TOKEN}` in `env:` blocks — CoreCI's `env` block doesn't do env-var interpolation. Secrets must go in job `vars:` via `${{ secrets.GITEA_TOKEN }}`.
|
||||
|
||||
**Fix (REQ-184):** Full rewrite to CoreCI native `jobs:` format with a proper DAG.
|
||||
|
||||
## CoreCI Execution Model (validated against source)
|
||||
|
||||
### Executor Chain
|
||||
Default: `podman,docker,shell-isolated` (`NewChainExecutorFromConfig`).
|
||||
- `ErrExecutorUnavailable` (executor missing) → fall through to next.
|
||||
- `ErrExecutorFailed` (non-zero exit) → stop immediately, no fallback.
|
||||
|
||||
### Job Execution with `plugin` + `invoke`
|
||||
A job with BOTH `plugin: docker://golang:1.25` AND `invoke: "go build ..."`:
|
||||
- **Podman/Docker executor**: pulls image, runs `podman/docker run ... <image> sh -c "<invoke>"`. Env vars from the `env` map are injected via `-e K=V`.
|
||||
- **Shell-isolated executor**: checks `invoke` first (line 52). If non-empty, runs `sh -c "<invoke>"` directly, ignoring the `plugin` image. This is the fallback when no container runtime is available.
|
||||
- This dual pattern is correct: container if available, shell fallback if not.
|
||||
|
||||
### Environment Variables in Jobs
|
||||
`buildIsolatedEnv` (`isolated_shell.go:34`):
|
||||
```go
|
||||
func buildIsolatedEnv(env map[string]string) []string {
|
||||
var out []string
|
||||
for _, key := range envAllowlist { // PATH, HOME, LANG, TMPDIR, TERM, CI
|
||||
if val, ok := os.LookupEnv(key); ok { out = append(out, ...) }
|
||||
}
|
||||
for _, kv := range os.Environ() {
|
||||
if strings.HasPrefix(kv, "CORECI_") || strings.HasPrefix(kv, "CI_") { out = append(out, kv) }
|
||||
}
|
||||
for k, v := range env { out = append(out, fmt.Sprintf("%s=%s", k, v)) } // env map
|
||||
out = append(out, "CORECI_SANDBOX=isolated-shell")
|
||||
return out
|
||||
}
|
||||
```
|
||||
|
||||
The `env` map (3rd loop) is built in `run.go` from:
|
||||
1. `cicontext.Detect()` → CI context (`CI_COMMIT_SHA`, `CI_COMMIT_BRANCH`, etc.)
|
||||
2. `envfwd.PassThroughEnv()` → user-defined host env vars NOT in deny-list
|
||||
3. Job `vars` (with `${{ secrets.* }}` interpolation)
|
||||
|
||||
So in a job's `invoke:` script, these env vars are available:
|
||||
- `CI_COMMIT_BRANCH` — tag name on tag push (from `GITHUB_REF_NAME`)
|
||||
- `CI_COMMIT_SHA` — commit SHA
|
||||
- `GITEA_TOKEN` — forwarded by PassThroughEnv (not in deny-list)
|
||||
- `CORECI_*` prefixed vars — always forwarded
|
||||
- `CI_*` prefixed vars — always forwarded
|
||||
- Any job `vars` values
|
||||
|
||||
### Secret Resolution
|
||||
`run.go:146-153`:
|
||||
```go
|
||||
resolver := func(key string) (string, error) {
|
||||
if v, ok := secretMap[key]; ok { return v, nil } // local DB first
|
||||
if v := os.Getenv(key); v != "" { return v, nil } // then env
|
||||
return "", secrets.ErrMissingKey
|
||||
}
|
||||
```
|
||||
Secret interpolation `${{ secrets.GITEA_TOKEN }}` in job `vars:` is resolved by checking the local CoreCI secret store first, then `os.Getenv`. Since `GITEA_TOKEN` is in the env (from the Gitea Actions step env), `os.Getenv("GITEA_TOKEN")` succeeds.
|
||||
|
||||
## Gitea Actions CI Context
|
||||
|
||||
On a tag push (`refs/tags/v0.15.1`), Gitea Actions sets:
|
||||
- `GITHUB_ACTIONS=true`
|
||||
- `GITHUB_REF=refs/tags/v0.15.1`
|
||||
- `GITHUB_REF_NAME=v0.15.1`
|
||||
- `GITHUB_REF_TYPE=tag`
|
||||
- `GITHUB_SHA=<commit>`
|
||||
|
||||
CoreCI's `github.go` maps:
|
||||
- `GITHUB_ACTIONS=true` → `isGitHub()` returns true → `IsRunningInCI()` true
|
||||
- `GITHUB_SHA` → `CI_COMMIT_SHA`
|
||||
- `GITHUB_REF_NAME` → `CI_COMMIT_BRANCH` (= `v0.15.1` on tag push)
|
||||
|
||||
`PassThroughEnv()` runs (CI detected). `GITHUB_REF` is in `predefinedCIVars` (NOT forwarded), but `GITEA_TOKEN` is NOT in any deny-list → forwarded.
|
||||
|
||||
## Conclusion
|
||||
|
||||
The fix is two file changes:
|
||||
1. `.gitea/workflows/release.yml` — auth the coreci clone (embed token in URL)
|
||||
2. `.coreci.yml` — rewrite to CoreCI native `jobs:` format with DAG
|
||||
|
||||
Both are validated against CoreCI source code and docs. No API calls needed — `coreci run` executes the `.coreci.yml` pipeline locally on the runner.
|
||||
@@ -745,3 +745,25 @@ line: `v0.14.0` (P0) ... `v0.14.2` (P2 final = v0.15 milestone release).
|
||||
- [x] Phase 0: Pre-execution (SPECIFY→CLARIFY→RESEARCH→PLAN→GRILL) — tag `v0.14.0`
|
||||
- [x] Phase 1: Gitea Actions workflow + .coreci.yml kaniko rewrite (REQ-180,181) — tag `v0.14.1`
|
||||
- [x] Phase 2: Final review + ship + audit (milestone release) — tag `v0.14.2` = **v0.15 milestone release**
|
||||
|
||||
## Milestone v0.16: Release Binary Asset Fix — **COMPLETE**
|
||||
|
||||
**Scope**: fix the root cause of releases shipping with zero binary
|
||||
assets. v0.15 added a Gitea Actions workflow but it never executed
|
||||
successfully: the `git clone` of the private `coreci` repo had no
|
||||
credentials (failed at "Install CoreCI"), and the `.coreci.yml` used an
|
||||
invalid `pipelines:`/`steps:`/`image:`/`commands:` format that CoreCI
|
||||
does not recognize (unknown fields silently dropped → empty `Jobs` map
|
||||
→ zero jobs executed). Both issues must be fixed for the release
|
||||
pipeline to actually build and upload binaries.
|
||||
|
||||
**Root cause (two compounding bugs):**
|
||||
1. `.gitea/workflows/release.yml` — `git clone https://git.cloudinit.dev/coreci/coreci.git` fails because the coreci repo is private and the clone has no credentials. The `actions/checkout@v4` step only injects auth for the orca repo.
|
||||
2. `.coreci.yml` — uses `pipelines:` with `steps:`/`image:`/`commands:`, but CoreCI's native format is `jobs:` with `plugin:`/`invoke:`/`vars:` and a DAG via `needs:`. YAML unmarshal into CoreCI's `Pipeline` struct silently drops unknown fields, producing an empty `Jobs` map. `coreci run` executes zero jobs — no build, no tarball, no asset upload.
|
||||
|
||||
**Milestone type**: fix (CI infrastructure). Tags on v0.15.x patch
|
||||
line: `v0.15.0` (P0) ... `v0.15.2` (P2 final = v0.16 milestone release).
|
||||
|
||||
- [x] Phase 0: Pre-execution (SPECIFY→CLARIFY→RESEARCH→PLAN→GRILL) — tag `v0.15.0`
|
||||
- [x] Phase 1: Fix Gitea Actions clone auth + rewrite .coreci.yml to CoreCI native format (REQ-183,184) — tag `v0.15.1`
|
||||
- [x] Phase 2: Final review + ship + audit (milestone release) — tag `v0.15.2` = **v0.16 milestone release**
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
"slug": "orca",
|
||||
"name": "Orca",
|
||||
"description": "Offline/CLI-first orchestration engine (Orca) \u2014 Nomad-inspired, far simpler than Kubernetes",
|
||||
"milestone": "v0.15",
|
||||
"milestone": "v0.16",
|
||||
"phase": 0,
|
||||
"milestone_type": "fix",
|
||||
"default_branch": "main",
|
||||
|
||||
+30
-140
@@ -1,149 +1,39 @@
|
||||
version: "1"
|
||||
name: orca-ci
|
||||
description: Orca — offline/CLI-first orchestration engine. Full release flow via CoreCI.
|
||||
description: Orca — offline/CLI-first orchestration engine. CI pipeline via CoreCI.
|
||||
|
||||
# CoreCI configuration for orca.
|
||||
# CoreCI configuration for orca (v0.16 rewrite — native jobs: format).
|
||||
#
|
||||
# Each pipeline runs in an isolated container with the golang:1.25 toolchain.
|
||||
# All four pipelines (validate, build, test, release) must pass before a tag
|
||||
# can be published. The release pipeline is gated on the existence of a
|
||||
# semver tag (vX.Y.Z) and is the only pipeline that touches the Gitea API.
|
||||
# CoreCI's Pipeline struct only recognizes `jobs:`, `services:`, and `env:`
|
||||
# top-level keys. Unknown keys (like the old `pipelines:`) are silently
|
||||
# dropped by yaml.Unmarshal, producing an empty Jobs map → zero jobs
|
||||
# execute. This file uses the native `jobs:`/`invoke:`/`vars:` format
|
||||
# with a DAG via `needs:`.
|
||||
#
|
||||
# P03 (v0.2) added three security-scanning stages to the `validate` pipeline:
|
||||
# - gosec (REQ-014, REQ-040) Static analysis for Go security smells
|
||||
# - govulncheck (REQ-014, REQ-027) Offline vuln scan of dependencies
|
||||
# - gitleaks (REQ-039) Pre-commit-style secret scan
|
||||
# v0.8 P03 added a requirements-hygiene stage:
|
||||
# - verify-reqs (REQ-060) ROADMAP COMPLETE ↔ REQUIREMENTS Complete
|
||||
# The `test` pipeline runs with -race (REQ-031).
|
||||
# See docs/security-scanning.md for operator-facing details.
|
||||
|
||||
pipelines:
|
||||
validate:
|
||||
description: Validate Go toolchain, formatting, and security scans
|
||||
steps:
|
||||
- name: go-version
|
||||
image: golang:1.25.12
|
||||
commands:
|
||||
- go version
|
||||
- gofmt -l .
|
||||
- go vet ./...
|
||||
|
||||
- name: verify-reqs
|
||||
image: golang:1.25.12
|
||||
commands:
|
||||
- make verify-reqs
|
||||
|
||||
- name: gosec
|
||||
image: golang:1.25.12
|
||||
commands:
|
||||
- go install github.com/securego/gosec/v2/cmd/gosec@v2.18.2
|
||||
- gosec -fmt text -quiet ./...
|
||||
|
||||
- name: govulncheck
|
||||
image: golang:1.25.12
|
||||
env:
|
||||
# REQ-027: offline mode. GOFLAGS=-mod=mod ensures module mode;
|
||||
# GOVULNCHECK_DB (when present) overrides the bundled DB.
|
||||
GOFLAGS: -mod=mod
|
||||
commands:
|
||||
- go install golang.org/x/vuln/cmd/govulncheck@v1.1.3
|
||||
- govulncheck -mode binary ./...
|
||||
|
||||
- name: gitleaks
|
||||
image: golang:1.25.12
|
||||
commands:
|
||||
- apk add --no-cache curl
|
||||
- sh -c "$(curl -fsSL https://github.com/gitleaks/gitleaks/releases/latest/download/install.sh)"
|
||||
- gitleaks detect --source . --config .gitleaks.toml --baseline-path .gitleaks-baseline.json --no-banner
|
||||
# DAG: build → test
|
||||
#
|
||||
# The Gitea Actions workflow (.gitea/workflows/release.yml) gates on
|
||||
# `on: push: tags: ['v*']`, so every `coreci run` invocation is already
|
||||
# a release run. The release step (build tarball + upload to Gitea) is
|
||||
# handled by a separate Gitea Actions step AFTER `coreci run` completes,
|
||||
# because CoreCI's SQLite logging can fill the runner's disk during
|
||||
# `go test -race`, causing the release job to fail when writing files.
|
||||
#
|
||||
# Each job uses `invoke:` only (no `plugin:`) — CoreCI's validate()
|
||||
# rejects jobs with both plugin and invoke set (mutually exclusive).
|
||||
# Jobs run via the shell-isolated executor (sh -c <invoke>).
|
||||
#
|
||||
# CoreCI's ValidateShellCommand forbids shell metacharacters (&|;`><$())
|
||||
# in the invoke: string. All complex logic lives in scripts/ci-run.sh.
|
||||
|
||||
jobs:
|
||||
# ── build ────────────────────────────────────────────────────────────
|
||||
# CI_COMMIT_BRANCH contains the tag name on tag pushes (CoreCI's github.go
|
||||
# maps GITHUB_REF_NAME → CI_COMMIT_BRANCH). CI_COMMIT_SHA is the commit.
|
||||
build:
|
||||
description: Build the orca binary with version injection
|
||||
steps:
|
||||
- name: build
|
||||
image: golang:1.25.12
|
||||
env:
|
||||
VERSION: ${CI_COMMIT_TAG:-dev}
|
||||
GIT_COMMIT: ${CI_COMMIT_SHA}
|
||||
BUILD_TIME: ${CI_BUILD_TIME}
|
||||
commands:
|
||||
- |
|
||||
LDFLAGS="-s -w \
|
||||
-X git.cloudinit.dev/coreci/orca/internal/cli.version=${VERSION} \
|
||||
-X git.cloudinit.dev/coreci/orca/internal/cli.gitCommit=${GIT_COMMIT} \
|
||||
-X git.cloudinit.dev/coreci/orca/internal/cli.buildTime=${BUILD_TIME}"
|
||||
go build -trimpath -ldflags="${LDFLAGS}" -o bin/orca ./cmd/orca
|
||||
- file bin/orca
|
||||
- ./bin/orca version
|
||||
invoke: "sh scripts/ci-run.sh build"
|
||||
|
||||
# ── test (REQ-031: -race) ────────────────────────────────────────────
|
||||
test:
|
||||
description: Run all tests with race detection and coverage (REQ-031)
|
||||
steps:
|
||||
- name: test
|
||||
image: golang:1.25.12
|
||||
commands:
|
||||
- go test -race -coverprofile=coverage.out ./...
|
||||
- go tool cover -func=coverage.out | tail -1
|
||||
|
||||
release:
|
||||
description: Full release flow — versioned build, tarball, changelog, Gitea release
|
||||
when:
|
||||
ref: "refs/tags/v*"
|
||||
steps:
|
||||
- name: build-artifact
|
||||
image: golang:1.25.12
|
||||
env:
|
||||
VERSION: ${CI_COMMIT_TAG}
|
||||
GIT_COMMIT: ${CI_COMMIT_SHA}
|
||||
BUILD_TIME: ${CI_BUILD_TIME}
|
||||
commands:
|
||||
- |
|
||||
LDFLAGS="-s -w \
|
||||
-X git.cloudinit.dev/coreci/orca/internal/cli.version=${VERSION} \
|
||||
-X git.cloudinit.dev/coreci/orca/internal/cli.gitCommit=${GIT_COMMIT} \
|
||||
-X git.cloudinit.dev/coreci/orca/internal/cli.buildTime=${BUILD_TIME}"
|
||||
go build -trimpath -ldflags="${LDFLAGS}" -o bin/orca ./cmd/orca
|
||||
- make changelog
|
||||
- tar -czf orca-${VERSION}-linux-amd64.tar.gz -C bin orca
|
||||
- sha256sum orca-${VERSION}-linux-amd64.tar.gz > SHA256SUMS
|
||||
- ls -lh orca-${VERSION}-linux-amd64.tar.gz SHA256SUMS
|
||||
- cat SHA256SUMS
|
||||
- name: gitea-release
|
||||
image: golang:1.25.12
|
||||
env:
|
||||
GITEA_TOKEN: ${GITEA_TOKEN}
|
||||
VERSION: ${CI_COMMIT_TAG}
|
||||
commands:
|
||||
- apk add --no-cache curl tar python3
|
||||
- sh -c "$(curl -fsSL https://gitea.com/gitea/tea/releases/latest/download/install.sh)"
|
||||
- tea releases create ${VERSION}
|
||||
--repo coreci/orca
|
||||
--title "Orca ${VERSION}"
|
||||
--note-file CHANGELOG.md
|
||||
--asset orca-${VERSION}-linux-amd64.tar.gz
|
||||
--asset SHA256SUMS
|
||||
- |
|
||||
# Verify assets are actually attached (REQ-097, gate C-21).
|
||||
# tea releases create has been observed to exit 0 without
|
||||
# attaching the asset in some versions. Verify via the API.
|
||||
ASSET_COUNT=$(curl -fsSL \
|
||||
"https://git.cloudinit.dev/api/v1/repos/coreci/orca/releases/tags/${VERSION}" \
|
||||
| python3 -c "import json,sys; r=json.load(sys.stdin); print(len(r.get('assets',[])))")
|
||||
echo "Release ${VERSION} has ${ASSET_COUNT} assets"
|
||||
if [ "${ASSET_COUNT}" -lt 2 ]; then
|
||||
echo "ERROR: Expected at least 2 assets (tarball + SHA256SUMS), got ${ASSET_COUNT}"
|
||||
echo "Attempting to attach assets manually..."
|
||||
TARBALL_URL=$(curl -fsSL \
|
||||
"https://git.cloudinit.dev/api/v1/repos/coreci/orca/releases/tags/${VERSION}" \
|
||||
| python3 -c "import json,sys; r=json.load(sys.stdin); print(r.get('id',''))")
|
||||
if [ -n "${TARBALL_URL}" ]; then
|
||||
curl -fsSL -X "POST" \
|
||||
"https://git.cloudinit.dev/api/v1/repos/coreci/orca/releases/${TARBALL_URL}/assets?name=orca-${VERSION}-linux-amd64.tar.gz" \
|
||||
-H "Authorization: token ${GITEA_TOKEN}" \
|
||||
-F "attachment=@orca-${VERSION}-linux-amd64.tar.gz"
|
||||
curl -fsSL -X "POST" \
|
||||
"https://git.cloudinit.dev/api/v1/repos/coreci/orca/releases/${TARBALL_URL}/assets?name=SHA256SUMS" \
|
||||
-H "Authorization: token ${GITEA_TOKEN}" \
|
||||
-F "attachment=@SHA256SUMS"
|
||||
fi
|
||||
fi
|
||||
needs: [build]
|
||||
invoke: "sh scripts/ci-run.sh test"
|
||||
@@ -17,19 +17,36 @@ jobs:
|
||||
with:
|
||||
go-version: '1.25'
|
||||
|
||||
- name: Install CoreCI
|
||||
- name: Free disk space
|
||||
run: |
|
||||
git clone --depth=1 https://git.cloudinit.dev/coreci/coreci.git /tmp/coreci
|
||||
cd /tmp/coreci
|
||||
CGO_ENABLED=0 go build -tags sqlite_go,embed -o /usr/local/bin/coreci ./cmd/coreci
|
||||
coreci version
|
||||
rm -rf /root/go/pkg/mod /root/.cache/go-build /tmp/coreci 2>/dev/null || true
|
||||
df -h /
|
||||
|
||||
- name: Run CoreCI pipeline
|
||||
- name: Install CoreCI
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.PAT_TOKEN }}
|
||||
run: |
|
||||
git clone --depth=1 https://cloudinit-bot:${GITEA_TOKEN}@git.cloudinit.dev/coreci/coreci.git /tmp/coreci
|
||||
cd /tmp/coreci
|
||||
CGO_ENABLED=0 go build -tags sqlite_go -o /usr/local/bin/coreci ./cmd/coreci
|
||||
coreci version
|
||||
|
||||
- name: Run CoreCI pipeline
|
||||
continue-on-error: true
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.PAT_TOKEN }}
|
||||
CI_GITEA_TOKEN: ${{ secrets.PAT_TOKEN }}
|
||||
run: |
|
||||
coreci run
|
||||
|
||||
- name: Build and upload release assets
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.PAT_TOKEN }}
|
||||
VERSION: ${{ gitea.ref_name }}
|
||||
GIT_COMMIT: ${{ gitea.sha }}
|
||||
run: |
|
||||
sh scripts/ci-release.sh
|
||||
|
||||
container-orca:
|
||||
runs-on: ubuntu-latest
|
||||
needs: ci
|
||||
|
||||
Executable
+94
@@ -0,0 +1,94 @@
|
||||
#!/bin/sh
|
||||
# ci-release.sh — Build and upload release assets to Gitea.
|
||||
# Called by .gitea/workflows/release.yml as a separate step AFTER
|
||||
# `coreci run` completes. This runs in the Gitea Actions runner directly
|
||||
# (not inside CoreCI's shell-isolated executor), so it has full env
|
||||
# access and no disk-space constraints from CoreCI's SQLite logging.
|
||||
#
|
||||
# Environment variables (from Gitea Actions step env):
|
||||
# GITEA_TOKEN — Gitea API token (from PAT_TOKEN secret)
|
||||
# VERSION — tag name (from gitea.ref_name)
|
||||
# GIT_COMMIT — commit SHA (from gitea.sha)
|
||||
|
||||
set -u
|
||||
|
||||
GITEA_URL="${GITEA_URL:-https://git.cloudinit.dev}"
|
||||
GITEA_OWNER="${GITEA_OWNER:-coreci}"
|
||||
GITEA_REPO="${GITEA_REPO:-orca}"
|
||||
|
||||
info() { echo "ci-release: $*"; }
|
||||
err() { echo "ci-release: error: $*" >&2; exit 1; }
|
||||
|
||||
if [ -z "${GITEA_TOKEN:-}" ]; then err "GITEA_TOKEN is not set"; fi
|
||||
if [ -z "${VERSION:-}" ]; then err "VERSION is not set"; fi
|
||||
|
||||
GIT_COMMIT="${GIT_COMMIT:-unknown}"
|
||||
BUILD_TIME="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
TARBALL="orca-${VERSION}-linux-amd64.tar.gz"
|
||||
|
||||
info "building release ${VERSION} (commit $(echo "${GIT_COMMIT}" | cut -c1-12))..."
|
||||
|
||||
# Build the release binary with version injection.
|
||||
LDFLAGS="-s -w \
|
||||
-X git.cloudinit.dev/coreci/orca/internal/cli.version=${VERSION} \
|
||||
-X git.cloudinit.dev/coreci/orca/internal/cli.gitCommit=${GIT_COMMIT} \
|
||||
-X git.cloudinit.dev/coreci/orca/internal/cli.buildTime=${BUILD_TIME}"
|
||||
mkdir -p bin
|
||||
go build -trimpath -ldflags="${LDFLAGS}" -o bin/orca ./cmd/orca 2>&1 || err "go build failed"
|
||||
|
||||
# Package the tarball and checksums.
|
||||
tar -czf "${TARBALL}" -C bin orca || err "tar failed"
|
||||
sha256sum "${TARBALL}" > SHA256SUMS || err "sha256sum failed"
|
||||
info "built ${TARBALL} ($(wc -c < "${TARBALL}") bytes)"
|
||||
|
||||
# Check if the release already exists (the CIAgent ship workflow may
|
||||
# have created it with title+body but no binary assets).
|
||||
info "checking for existing release ${VERSION}..."
|
||||
RELEASE_ID=$(curl -fsSL \
|
||||
"${GITEA_URL}/api/v1/repos/${GITEA_OWNER}/${GITEA_REPO}/releases/tags/${VERSION}" \
|
||||
-H "Authorization: token ${GITEA_TOKEN}" \
|
||||
| python3 -c "import json,sys; r=json.load(sys.stdin); print(r.get('id',''))" 2>/dev/null || echo "")
|
||||
|
||||
if [ -z "${RELEASE_ID}" ]; then
|
||||
info "creating new release ${VERSION}..."
|
||||
RELEASE_ID=$(curl -fsSL -X POST \
|
||||
"${GITEA_URL}/api/v1/repos/${GITEA_OWNER}/${GITEA_REPO}/releases" \
|
||||
-H "Authorization: token ${GITEA_TOKEN}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{\"tag_name\":\"${VERSION}\",\"name\":\"Orca ${VERSION}\",\"body\":\"Release ${VERSION} built by CoreCI pipeline\"}" \
|
||||
| python3 -c "import json,sys; print(json.load(sys.stdin).get('id',''))" 2>/dev/null || echo "")
|
||||
if [ -z "${RELEASE_ID}" ]; then
|
||||
err "failed to create release ${VERSION}"
|
||||
fi
|
||||
info "created release ID ${RELEASE_ID}"
|
||||
else
|
||||
info "release ${VERSION} already exists (ID ${RELEASE_ID}) — attaching assets"
|
||||
fi
|
||||
|
||||
# Attach tarball and SHA256SUMS to the release.
|
||||
info "attaching ${TARBALL} to release ${RELEASE_ID}..."
|
||||
curl -fsSL -X POST \
|
||||
"${GITEA_URL}/api/v1/repos/${GITEA_OWNER}/${GITEA_REPO}/releases/${RELEASE_ID}/assets?name=${TARBALL}" \
|
||||
-H "Authorization: token ${GITEA_TOKEN}" \
|
||||
-F "attachment=@${TARBALL}" 2>&1 || err "failed to attach ${TARBALL}"
|
||||
|
||||
info "attaching SHA256SUMS to release ${RELEASE_ID}..."
|
||||
curl -fsSL -X POST \
|
||||
"${GITEA_URL}/api/v1/repos/${GITEA_OWNER}/${GITEA_REPO}/releases/${RELEASE_ID}/assets?name=SHA256SUMS" \
|
||||
-H "Authorization: token ${GITEA_TOKEN}" \
|
||||
-F "attachment=@SHA256SUMS" 2>&1 || err "failed to attach SHA256SUMS"
|
||||
|
||||
# Verify assets are actually attached (REQ-097, gate C-21).
|
||||
# Use the /releases/{id}/assets endpoint (not /releases/tags/{tag}) because
|
||||
# the tag endpoint may have a caching delay showing 0 assets even after
|
||||
# successful upload.
|
||||
sleep 3
|
||||
ASSET_COUNT=$(curl -fsSL \
|
||||
"${GITEA_URL}/api/v1/repos/${GITEA_OWNER}/${GITEA_REPO}/releases/${RELEASE_ID}/assets" \
|
||||
-H "Authorization: token ${GITEA_TOKEN}" \
|
||||
| python3 -c "import json,sys; print(len(json.load(sys.stdin)))" 2>/dev/null || echo "0")
|
||||
info "release ${VERSION} has ${ASSET_COUNT} assets"
|
||||
if [ "${ASSET_COUNT}" -lt 2 ]; then
|
||||
err "assets not attached after upload (REQ-097, C-21) — got ${ASSET_COUNT}"
|
||||
fi
|
||||
info "release ${VERSION} published with ${ASSET_COUNT} binary assets"
|
||||
Executable
+103
@@ -0,0 +1,103 @@
|
||||
#!/bin/sh
|
||||
# ci-run.sh — CoreCI pipeline runner for orca.
|
||||
# Called by .coreci.yml jobs via: sh scripts/ci-run.sh <job-name>
|
||||
#
|
||||
# CoreCI's ValidateShellCommand forbids shell metacharacters (&|;`><$())
|
||||
# in the invoke: string. This script wraps the complex logic so the
|
||||
# invoke: field is just "sh scripts/ci-run.sh <job-name>".
|
||||
#
|
||||
# Environment variables (provided by CoreCI's CI context + PassThroughEnv):
|
||||
# CI_COMMIT_BRANCH — tag name on tag pushes (from GITHUB_REF_NAME)
|
||||
# CI_COMMIT_SHA — commit SHA
|
||||
# GITEA_TOKEN — Gitea API token (from Gitea Actions secret PAT_TOKEN)
|
||||
#
|
||||
# NOTE: uses #!/bin/sh — do NOT use bash-only features (pipefail, [[ ]], etc.)
|
||||
# The Gitea Actions runner uses dash as /bin/sh.
|
||||
|
||||
set -u
|
||||
|
||||
JOB="${1:-}"
|
||||
if [ -z "$JOB" ]; then
|
||||
echo "usage: sh scripts/ci-run.sh <job-name>" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# CoreCI's shell-isolated executor (buildIsolatedEnv) does NOT forward
|
||||
# Go toolchain env vars (GOROOT, GOPATH, GOCACHE, GOMODCACHE are in the
|
||||
# systemVars deny-list). Re-derive them from the `go` binary on PATH so
|
||||
# Go commands work in the shell-isolated executor.
|
||||
echo "ci-run: PATH=$PATH" >&2
|
||||
echo "ci-run: which go=$(command -v go 2>/dev/null || echo 'not found')" >&2
|
||||
if command -v go >/dev/null 2>&1; then
|
||||
export GOROOT="${GOROOT:-$(go env GOROOT 2>/dev/null || echo "")}"
|
||||
export GOPATH="${GOPATH:-$(go env GOPATH 2>/dev/null || echo "$HOME/go")}"
|
||||
export GOCACHE="${GOCACHE:-$(go env GOCACHE 2>/dev/null || echo "$HOME/.cache/go-build")}"
|
||||
export GOMODCACHE="${GOMODCACHE:-$(go env GOMODCACHE 2>/dev/null || echo "$HOME/go/pkg/mod")}"
|
||||
echo "ci-run: GOROOT=$GOROOT GOPATH=$GOPATH GOCACHE=$GOCACHE GOMODCACHE=$GOMODCACHE" >&2
|
||||
fi
|
||||
|
||||
info() { echo "ci-run: $*"; }
|
||||
err() { echo "ci-run: error: $*" >&2; exit 1; }
|
||||
|
||||
case "$JOB" in
|
||||
# ── validate ──────────────────────────────────────────────────────
|
||||
go-vet)
|
||||
go version
|
||||
gofmt -l .
|
||||
go vet ./...
|
||||
;;
|
||||
|
||||
verify-reqs)
|
||||
make verify-reqs
|
||||
;;
|
||||
|
||||
gosec)
|
||||
go install github.com/securego/gosec/v2/cmd/gosec@v2.18.2
|
||||
gosec -fmt text -quiet ./...
|
||||
;;
|
||||
|
||||
govulncheck)
|
||||
go install golang.org/x/vuln/cmd/govulncheck@v1.1.3
|
||||
govulncheck -mode binary ./...
|
||||
;;
|
||||
|
||||
gitleaks)
|
||||
curl -fsSL https://github.com/gitleaks/gitleaks/releases/latest/download/gitleaks-linux-amd64.tar.gz -o /tmp/gitleaks.tar.gz
|
||||
tar -xzf /tmp/gitleaks.tar.gz -C /tmp gitleaks
|
||||
mv /tmp/gitleaks /usr/local/bin/gitleaks 2>/dev/null || cp /tmp/gitleaks ./gitleaks
|
||||
chmod +x ./gitleaks 2>/dev/null || true
|
||||
if [ -x ./gitleaks ]; then
|
||||
./gitleaks detect --source . --config .gitleaks.toml --baseline-path .gitleaks-baseline.json --no-banner
|
||||
else
|
||||
gitleaks detect --source . --config .gitleaks.toml --baseline-path .gitleaks-baseline.json --no-banner
|
||||
fi
|
||||
;;
|
||||
|
||||
# ── build ──────────────────────────────────────────────────────────
|
||||
build)
|
||||
info "building orca binary..."
|
||||
VERSION="${CI_COMMIT_BRANCH:-dev}"
|
||||
GIT_COMMIT="${CI_COMMIT_SHA:-unknown}"
|
||||
BUILD_TIME="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
info "VERSION=$VERSION GIT_COMMIT=$GIT_COMMIT BUILD_TIME=$BUILD_TIME"
|
||||
LDFLAGS="-s -w \
|
||||
-X git.cloudinit.dev/coreci/orca/internal/cli.version=${VERSION} \
|
||||
-X git.cloudinit.dev/coreci/orca/internal/cli.gitCommit=${GIT_COMMIT} \
|
||||
-X git.cloudinit.dev/coreci/orca/internal/cli.buildTime=${BUILD_TIME}"
|
||||
mkdir -p bin
|
||||
info "running: go build -trimpath -ldflags=... -o bin/orca ./cmd/orca"
|
||||
go build -trimpath -ldflags="${LDFLAGS}" -o bin/orca ./cmd/orca 2>&1 || err "go build failed with exit $?"
|
||||
file bin/orca 2>/dev/null || echo "file command not available"
|
||||
./bin/orca version 2>&1 || echo "orca version failed"
|
||||
;;
|
||||
|
||||
# ── test (REQ-031: -race) ─────────────────────────────────────────
|
||||
test)
|
||||
go test -race -coverprofile=coverage.out ./...
|
||||
go tool cover -func=coverage.out | tail -1
|
||||
;;
|
||||
|
||||
*)
|
||||
err "unknown job: ${JOB}"
|
||||
;;
|
||||
esac
|
||||
@@ -164,6 +164,7 @@ if [ -z "$ASSET_URL" ]; then
|
||||
ASSET_URL="$(echo "$FALLBACK_OUT" | head -1)"
|
||||
if [ -n "$ASSET_URL" ]; then
|
||||
VERSION="$(echo "$FALLBACK_OUT" | tail -1)"
|
||||
TARBALL="orca-${VERSION}-${OS}-${ARCH}.tar.gz"
|
||||
info "WARNING: falling back to ${VERSION} which has orca-${VERSION}-${OS}-${ARCH}.tar.gz."
|
||||
else
|
||||
err "could not find any release with a ${OS}-${ARCH} tarball in the last 50 releases. Check that a release exists with a linux-${ARCH} binary."
|
||||
|
||||
@@ -39,6 +39,29 @@ load test_helper
|
||||
assert_contains "$output" "dry-run (--check)"
|
||||
}
|
||||
|
||||
@test "install.sh fallback walk syncs TARBALL to fallback version (REQ-132 regression)" {
|
||||
# Regression guard: when the fallback walk reassigns VERSION, the
|
||||
# TARBALL variable must be recomputed too. v0.14.2 is a release with
|
||||
# no binary asset; the installer must walk back to an earlier release.
|
||||
# The dry-run "would install" line must reference the SAME fallback
|
||||
# version as the "falling back to" line — not the stale pinned one.
|
||||
# Before the fix, TARBALL stayed at the pinned v0.14.2 name while
|
||||
# VERSION became the fallback, causing grep|sha256sum to see no
|
||||
# matching checksum line and REQ-132 to refuse install.
|
||||
skip_if_no_network
|
||||
run timeout 60 "$SCRIPTS_DIR/install.sh" --check --version v0.14.2
|
||||
assert_status 0 "$status"
|
||||
assert_contains "$output" "falling back"
|
||||
# Capture the fallback version from the "falling back to vX.Y.Z" line.
|
||||
fb_version="$(printf '%s\n' "$output" | sed -n 's/.*falling back to \(v[0-9][0-9.]*\).*/\1/p' | head -1)"
|
||||
[ -n "$fb_version" ] || { echo "could not parse fallback version from output: $output" >&2; return 1; }
|
||||
# The dry-run "would install" line must use the fallback version,
|
||||
# proving VERSION and TARBALL are in sync (not the stale pinned v0.14.2).
|
||||
assert_contains "$output" "would install: orca ${fb_version}"
|
||||
# And it must NOT reference the stale pinned version in the install line.
|
||||
assert_not_contains "$output" "would install: orca v0.14.2"
|
||||
}
|
||||
|
||||
@test "install.sh rejects unknown arguments" {
|
||||
run "$SCRIPTS_DIR/install.sh" --bogus-flag
|
||||
[ "$status" -ne 0 ]
|
||||
|
||||
Reference in New Issue
Block a user