Compare commits
4 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 72b359c9a9 | |||
| 711b61d63e | |||
| 3ea36ef3ab | |||
| 6e27df7404 |
+150
-1
@@ -49,6 +49,47 @@ The demo is a three-repo, stub-driven system that simulates an autonomous cloud
|
||||
| `acdl-evidence` repo | Pages host for `audit.json` + `index.html` timeline | Read-only for the pipeline; written at finalize stage | evidence_writer.py output |
|
||||
| Reusable pipeline workflow | Dev → QA → Prod → Finalize stages with environment gates | Gitea Actions; calls core scripts | All core scripts |
|
||||
|
||||
## Phase 04 pipeline topology (research)
|
||||
|
||||
Gitea Actions limitations (confirmed via research, supersedes any
|
||||
GitHub-Actions assumptions):
|
||||
|
||||
- `actions/upload-artifact@v3` / `download-artifact@v3` work; v4 is NOT supported by act_runner.
|
||||
- Artifacts are scoped to a single workflow run; **re-dispatch starts a new run, so artifacts do NOT survive between dispatches**.
|
||||
- `workflow_dispatch` API: `POST /api/v1/repos/{owner}/{repo}/actions/workflows/{filename}.yml/dispatches` with body `{ "ref": "<branch>", "inputs": {...} }`.
|
||||
- `on: workflow_call` + `uses: <owner>/<repo>/.gitea/workflows/<file>@<ref>` works; pin to `@milestone/v1.0-initial`.
|
||||
- `actions/checkout@v4` supports cross-repo (pass `repository:` + `ref:` + `token: ${{ secrets.GITEA_TOKEN }}` for private repos).
|
||||
- File-contents API: POST to create (201), PUT to update (must include current `sha`, obtained via GET).
|
||||
- `${{ secrets.GITEA_TOKEN }}` is a manually-created PAT secret on the `acdl` + `acdl-contracts` repos; the auto-injected token is current-repo only and cannot cross-repo.
|
||||
- No native approval-gate UI; gates are `workflow_dispatch` inputs (`approve_qa`, `approve_prod`).
|
||||
|
||||
### Approval-gate + state-persistence approach (D-027, D-028 refined)
|
||||
|
||||
Because re-dispatch starts a new run and artifacts do not survive:
|
||||
|
||||
1. The pipeline workflow has `workflow_dispatch` inputs:
|
||||
- `contract-ref` (string; default `main`) — the ref on `acdl-contracts` carrying the contract.
|
||||
- `approve_qa` (boolean; default `false`) — the human sets this to `true` to advance past QA.
|
||||
- `approve_prod` (boolean; default `false`) — the human sets this to `true` to advance past Prod.
|
||||
|
||||
2. Each stage job (`dev`, `qa-gate`, `prod-gate`, `finalize`) writes its evidence to `acdl-evidence` via the file-contents API (PUT `audit.json` with the new event appended). This is the persistent state across re-dispatches.
|
||||
|
||||
3. **Dev stage** (always runs on dispatch): check out `acdl` + `acdl-contracts@<contract-ref>`, run `policy_checker.py` + `confidence_signal.py`; if `score < 0.50`, write a `dev_rejected` evidence event and exit 1 (Act 4). Otherwise run `mock_executor.sh`, write a `dev_applied` evidence event, and exit 0. The run ends here.
|
||||
|
||||
4. **QA gate** (next dispatch with `approve_qa=true`): check out, run `evidence_writer.py --stage qa --event "qa approved"`, commit updated `audit.json` to `acdl-evidence`. Exit 0. The run ends.
|
||||
|
||||
5. **Prod gate** (next dispatch with `approve_prod=true`): same as QA but `--stage prod`.
|
||||
|
||||
6. **Finalize** (same dispatch as Prod, chained via `needs: prod-gate`): write the `finalize` evidence event, commit final `audit.json` to `acdl-evidence`. The raw URL now serves the updated timeline.
|
||||
|
||||
Because each stage is a separate dispatch, the workflow file uses `if:` conditions on each job:
|
||||
- `dev` runs when `inputs.approve_qa != true && inputs.approve_prod != true` (the initial dispatch).
|
||||
- `qa-gate` runs when `inputs.approve_qa == true && inputs.approve_prod != true`.
|
||||
- `prod-gate` runs when `inputs.approve_prod == true`.
|
||||
- `finalize` runs after `prod-gate` (`needs: prod-gate`).
|
||||
|
||||
This means a full pipeline = 3 dispatches (initial, qa-approve, prod-approve). The human drives each via the Gitea UI or the dispatch API.
|
||||
|
||||
## Data Flow
|
||||
|
||||
1. A `contract.yaml` arrives either by direct push (L3A) or by the issue workflow running `l3b_agent_stub.py` (L3B).
|
||||
@@ -143,4 +184,112 @@ Each L1 module lives at `modules/l1/<name>/` with exactly two files:
|
||||
| `l1-cloudwatch` | Observability primitive |
|
||||
|
||||
L1 modules are single-purpose, substrate-agnostic, max-depth-1 (per
|
||||
PROJECT.md Constraints). They do not compose with other L1s.
|
||||
PROJECT.md Constraints). They do not compose with other L1s.
|
||||
|
||||
## L2 module schema + core scripts (Phase 03 research)
|
||||
|
||||
### L2 manifest.yaml schema (D-020)
|
||||
|
||||
```yaml
|
||||
name: l2-commodity-price-feed # matches the folder name
|
||||
kind: l2 # literal "l2"
|
||||
description: <one-line>
|
||||
l1s: # ordered list of L1 references
|
||||
- name: l1-eks-fargate # MUST match an existing L1 folder name
|
||||
inputs:
|
||||
cluster_name: price-feed-cluster
|
||||
region: us-east-1
|
||||
cpu_arch: arm64
|
||||
- name: l1-lambda
|
||||
inputs:
|
||||
function_name: price-ingest
|
||||
runtime: python3.11
|
||||
handler: index.handler
|
||||
# ... up to 5 L1 references per L2 (max-depth-5 per REQ-05; L2->L1 is depth 1)
|
||||
```
|
||||
|
||||
L2s reference L1s **by name only** (no path); `mock_executor.sh` resolves
|
||||
the name to `modules/l1/<name>/`.
|
||||
|
||||
### L2 list (fixed per REQ-04)
|
||||
|
||||
| Folder | Description | L1s (per S&P Global Energy / Platts use cases) |
|
||||
|--------|-------------|------------------------------------------------|
|
||||
| `l2-invoice-service` | Billing + invoicing microservice | `l1-eks-fargate`, `l1-iam-role`, `l1-lambda`, `l1-sqs`, `l1-s3` |
|
||||
| `l2-commodity-price-feed` | Real-time price ingestion | `l1-eks-fargate`, `l1-lambda`, `l1-api-gateway`, `l1-eventbridge`, `l1-s3` |
|
||||
| `l2-energy-analytics-api` | Historical query API | `l1-eks-fargate`, `l1-api-gateway`, `l1-lambda`, `l1-s3`, `l1-cloudwatch` |
|
||||
| `l2-regulatory-reporting` | Compliance + reporting | `l1-eks-fargate`, `l1-iam-role`, `l1-lambda`, `l1-sqs`, `l1-s3` |
|
||||
|
||||
Each L2 references exactly 5 L1s (within the max-depth-5 constraint; L2→L1
|
||||
is depth 1, so depth-5 is generous but the spec caps composition depth at
|
||||
5 — the count is 5 to demonstrate a realistic composed stack).
|
||||
|
||||
### contract.yaml schema (D-021)
|
||||
|
||||
```yaml
|
||||
stack: l2-commodity-price-feed # MUST match an existing L2 folder name
|
||||
inputs: # top-level params for the L2 (optional)
|
||||
environment: dev
|
||||
owner: platform-team
|
||||
public-ingress: false # bool; true triggers POLICY_VIOLATION:PUBLIC_INGRESS
|
||||
```
|
||||
|
||||
The `public-ingress` key is the only policy-enforced field in Phase 03.
|
||||
Phase 04's pipeline reads `contract.yaml`, runs `policy_checker.py`, then
|
||||
`mock_executor.sh` to apply the L2.
|
||||
|
||||
### state.json shape (D-022)
|
||||
|
||||
`mock_executor.sh` writes `state.json` to its working directory:
|
||||
|
||||
```json
|
||||
{
|
||||
"l2": "l2-commodity-price-feed",
|
||||
"l1s": [
|
||||
{"name": "l1-eks-fargate", "applied": true, "exit_code": 0},
|
||||
{"name": "l1-lambda", "applied": true, "exit_code": 0},
|
||||
...
|
||||
],
|
||||
"contract": {
|
||||
"stack": "l2-commodity-price-feed",
|
||||
"inputs": {...},
|
||||
"public-ingress": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### audit.json event + hash chain (D-023)
|
||||
|
||||
`audit.json` is a JSON array of event objects. `evidence_writer.py`
|
||||
appends one event per call. Hash chain:
|
||||
|
||||
1. Construct the event dict with `hash` set to empty string.
|
||||
2. Serialize via `json.dumps(event, sort_keys=True, separators=(",", ":"))` — canonical JSON (deterministic key order, no whitespace).
|
||||
3. Compute `hash = sha256(canonical_json.encode("utf-8")).hexdigest()`.
|
||||
4. Set `event["hash"] = hash`.
|
||||
5. Append to `audit.json`.
|
||||
|
||||
Genesis event (when `audit.json` is empty or missing):
|
||||
|
||||
```json
|
||||
{
|
||||
"seq": 0,
|
||||
"ts": "2026-07-21T13:00:00Z",
|
||||
"stage": "genesis",
|
||||
"event": "audit log initialized",
|
||||
"prev_hash": "GENESIS",
|
||||
"hash": "<sha256 of the canonical json of this event with hash empty>"
|
||||
}
|
||||
```
|
||||
|
||||
Subsequent events: `seq = prev.seq + 1`, `prev_hash = prev.hash`.
|
||||
|
||||
### Core script I/O contracts
|
||||
|
||||
| Script | Input | Output | Exit |
|
||||
|--------|-------|--------|------|
|
||||
| `mock_executor.sh` | `<contract.yaml path>` (argv[1]); reads L2 manifest from `modules/l2/<contract.stack>/manifest.yaml` | writes `state.json` to cwd; prints per-L1 progress | 0 on all-L1s-pass; non-zero on any L1 failure |
|
||||
| `policy_checker.py` | `<contract.yaml path>` (argv[1]) | stdout: `POLICY_PASS` or `POLICY_VIOLATION:PUBLIC_INGRESS` | 0 on pass; 1 on violation |
|
||||
| `confidence_signal.py` | `<contract.yaml path>` (argv[1]); calls policy_checker | stdout: `{"score": 0.90|0.40, "reason": "..."}` | 0 always (per D-024; pipeline decides gate) |
|
||||
| `evidence_writer.py` | argv: `--stage <dev|qa|prod|finalize|genesis>` `--event "<text>"` `--audit <path to audit.json>` (default `./audit.json`) | appends event to audit.json; prints the new event's hash + seq | 0 on success; 1 on I/O error |
|
||||
| `l3b_agent_stub.py` | argv[1] = issue body text (or stdin if no argv); optional `-o <path>` (default stdout) | writes a `contract.yaml` (D-021 schema) with `stack` set by the D-008 keyword map | 0 on success; 1 on empty input |
|
||||
+53
-50
@@ -1,85 +1,88 @@
|
||||
---
|
||||
phase: 02
|
||||
name: l1-modules
|
||||
phase: 04
|
||||
name: pipeline-and-approval-gates
|
||||
milestone: v1.0
|
||||
milestone_type: feature
|
||||
status: planned
|
||||
requirements: [REQ-02, REQ-03]
|
||||
requirements: [REQ-10, REQ-12]
|
||||
must_haves:
|
||||
- "All 8 L1 module folders exist under modules/l1/ with the exact names from REQ-02"
|
||||
- "Each L1 has a manifest.yaml matching the schema in ARCHITECTURE.md (name, kind: l1, description, inputs: map of string keys)"
|
||||
- "Each L1 has a mock_apply.sh that echoes '[L1: <name>] applying...', sleeps 1s, echoes '[L1: <name>] OK', exits 0 (D-007)"
|
||||
- "All mock_apply.sh are executable (chmod +x) and bash -n clean"
|
||||
- "All manifest.yaml files parse as valid YAML"
|
||||
- "scripts/verify_phase02.sh passes: enumerates 8 L1s, validates each manifest, runs each mock_apply.sh, confirms exit 0 + expected output"
|
||||
- ".gitea/workflows/pipeline.yml is a real workflow (no placeholders) with workflow_dispatch inputs (contract-ref, approve_qa, approve_prod) and 4 jobs (dev, qa-gate, prod-gate, finalize) with correct if: conditions per D-027/D-028"
|
||||
- "The dev job checks out acdl + acdl-contracts@contract-ref, runs policy_checker.py + confidence_signal.py + mock_executor.sh, and writes a dev_applied or dev_rejected evidence event to acdl-evidence via the file-contents API"
|
||||
- "The qa-gate job (approve_qa=true) writes a qa_approved evidence event"
|
||||
- "The prod-gate + finalize jobs (approve_prod=true) write prod_approved + finalize evidence events, committing the final audit.json to acdl-evidence main"
|
||||
- "contracts-repo/.gitea/workflows/issue-to-contract.yml is a real workflow (no placeholders) that checks out l3b_agent_stub.py from acdl, parses the Issue body, commits contract.yaml to contract/<issue-number> on acdl-contracts, and dispatches pipeline.yml with contract-ref=contract/<issue-number>"
|
||||
- "scripts/finalize_evidence.py helper exists: given an audit.json file path, GETs the current file sha from acdl-evidence, PUTs the new content via the file-contents API. Used by the workflow steps to commit evidence updates."
|
||||
- "scripts/verify_phase04.sh passes: validates the workflow YAML syntax, confirms job if: conditions, confirms issue-to-contract.yml references the dispatch API, runs a dry-run of finalize_evidence.py against a temp audit.json (mock API)"
|
||||
verification:
|
||||
typecheck: "bash -n modules/l1/*/mock_apply.sh scripts/*.sh && python3 -c 'import yaml; [yaml.safe_load(open(f)) for f in glob.glob(\"modules/l1/*/manifest.yaml\")]'"
|
||||
test: "scripts/verify_phase02.sh"
|
||||
typecheck: "bash -n scripts/*.sh && python3 -m py_compile scripts/*.py && python3 -c 'import yaml; yaml.safe_load(open(\".gitea/workflows/pipeline.yml\")); yaml.safe_load(open(\"contracts-repo/.gitea/workflows/issue-to-contract.yml\"))'"
|
||||
test: "scripts/verify_phase04.sh"
|
||||
build: no-op
|
||||
---
|
||||
|
||||
# Phase 02 — l1-modules PLAN
|
||||
# Phase 04 — pipeline-and-approval-gates PLAN
|
||||
|
||||
## Goal
|
||||
|
||||
Create the 8 L1 stub modules under `modules/l1/`. Each module has a
|
||||
`manifest.yaml` (declared inputs, flat string map per D-017) and a uniform
|
||||
`mock_apply.sh` (echo + 1s sleep + exit 0 per D-007/D-018). After this phase,
|
||||
Phase 03 can compose L1s into L2 modules and `mock_executor.sh` can iterate
|
||||
over an L2's L1 references.
|
||||
Replace the Phase 01 workflow skeletons with real implementations. Wire
|
||||
the Dev → QA → Prod → Finalize pipeline with the 3-dispatch approval-gate
|
||||
pattern (D-027/D-028), and the issue-to-contract trigger (D-030).
|
||||
|
||||
## Requirements covered
|
||||
|
||||
- REQ-02: 8 L1 module folders exist (exact names)
|
||||
- REQ-03: each L1 has manifest.yaml + mock_apply.sh with the uniform behavior
|
||||
- REQ-10: reusable pipeline runs Dev (autonomous), QA (approval), Prod (approval), Finalize (commits audit.json to acdl-evidence)
|
||||
- REQ-12: opening an Issue in acdl-contracts runs l3b_agent_stub.py, commits a contract.yaml to a new branch, closes the Issue, and triggers the main pipeline
|
||||
|
||||
## Waves (vertical slices)
|
||||
## Waves (vertical slices, domain priority order)
|
||||
|
||||
### Wave 1 — infra-stub-engineer (creates the 8 L1s)
|
||||
### Wave 1 — backend-engineer (pipeline + finalize helper + issue trigger)
|
||||
|
||||
**Tasks:**
|
||||
|
||||
- **T-2.1** Create `modules/l1/l1-eks-fargate/{manifest.yaml, mock_apply.sh}`
|
||||
- **T-2.2** Create `modules/l1/l1-iam-role/{manifest.yaml, mock_apply.sh}`
|
||||
- **T-2.3** Create `modules/l1/l1-lambda/{manifest.yaml, mock_apply.sh}`
|
||||
- **T-2.4** Create `modules/l1/l1-api-gateway/{manifest.yaml, mock_apply.sh}`
|
||||
- **T-2.5** Create `modules/l1/l1-eventbridge/{manifest.yaml, mock_apply.sh}`
|
||||
- **T-2.6** Create `modules/l1/l1-sqs/{manifest.yaml, mock_apply.sh}`
|
||||
- **T-2.7** Create `modules/l1/l1-s3/{manifest.yaml, mock_apply.sh}`
|
||||
- **T-2.8** Create `modules/l1/l1-cloudwatch/{manifest.yaml, mock_apply.sh}`
|
||||
- **T-4.1** Create `scripts/finalize_evidence.py` — a helper that takes `--audit <path>` `--owner <org>` `--repo <name>` `--branch <name>` (defaults: continuous-intelligence, acdl-evidence, main) and `--token-env <name>` (default ACDL_GITEA_TOKEN). It GETs the current `audit.json` from the repo (to get the sha if it exists), reads the local `<audit>` file, PUTs (or POSTs if 404) the new content via the Gitea file-contents API with base64 + commit message "finalize: update audit.json (run <gitea.run_id>)". Exit 0 on success, 1 on API failure. Used by the pipeline finalize + each stage to persist evidence to acdl-evidence.
|
||||
- **T-4.2** Replace `.gitea/workflows/pipeline.yml` with the real implementation:
|
||||
- `on: workflow_dispatch` with inputs `contract-ref` (string, default `main`), `approve_qa` (boolean, default `false`), `approve_prod` (boolean, default `false`).
|
||||
- Job `dev`: `if: inputs.approve_qa != true && inputs.approve_prod != true`. Steps: checkout `acdl` (current repo) at `milestone/v1.0-initial`; checkout `acdl-contracts` at `inputs.contract-ref` into a sibling dir using `actions/checkout@v4` with `repository: continuous-intelligence/acdl-contracts`, `ref: ${{ inputs.contract-ref }}`, `token: ${{ secrets.GITEA_TOKEN }}`; run `python3 scripts/policy_checker.py ../acdl-contracts/contract.yaml`; run `python3 scripts/confidence_signal.py ../acdl-contracts/contract.yaml` and capture the score; if score < 0.50, run `python3 scripts/evidence_writer.py --stage dev --event "dev rejected: confidence < 0.50"` and `python3 scripts/finalize_evidence.py --audit audit.json` and `exit 1`; else run `bash scripts/mock_executor.sh ../acdl-contracts/contract.yaml`, then `python3 scripts/evidence_writer.py --stage dev --event "dev applied: <stack>"`, then `python3 scripts/finalize_evidence.py --audit audit.json`.
|
||||
- Job `qa-gate`: `if: inputs.approve_qa == true && inputs.approve_prod != true`. Steps: checkout acdl; run `python3 scripts/evidence_writer.py --stage qa --event "qa approved"`; `python3 scripts/finalize_evidence.py --audit audit.json`.
|
||||
- Job `prod-gate`: `if: inputs.approve_prod == true`. Steps: checkout acdl; run `python3 scripts/evidence_writer.py --stage prod --event "prod approved"`; `python3 scripts/finalize_evidence.py --audit audit.json`.
|
||||
- Job `finalize`: `needs: prod-gate`. Steps: checkout acdl; run `python3 scripts/evidence_writer.py --stage finalize --event "pipeline complete: audit.json committed to acdl-evidence"`; `python3 scripts/finalize_evidence.py --audit audit.json`.
|
||||
- All jobs `runs-on: ubuntu-latest`. Use `gitea.*` context where relevant.
|
||||
- **T-4.3** Replace `contracts-repo/.gitea/workflows/issue-to-contract.yml` with the real implementation:
|
||||
- `on: issues` with `types: [opened]`.
|
||||
- Job `parse-and-trigger`: `runs-on: ubuntu-latest`. Steps: checkout `acdl` (to get `l3b_agent_stub.py`) at `milestone/v1.0-initial`; run `python3 scripts/l3b_agent_stub.py "${{ gitea.event.issue.body }}" -o contract.yaml`; parse the contract to confirm a stack was set; commit `contract.yaml` to a new branch `contract/${{ gitea.event.issue.number }}` on `acdl-contracts` via the file-contents API (POST); comment on the Issue with the contract summary; close the Issue via the issues API; dispatch the pipeline workflow via `POST /api/v1/repos/continuous-intelligence/acdl/actions/workflows/pipeline.yml/dispatches` with body `{ "ref": "milestone/v1.0-initial", "inputs": { "contract-ref": "contract/${{ gitea.event.issue.number }}" } }` using `secrets.GITEA_TOKEN`.
|
||||
|
||||
Each L1's `manifest.yaml` declares 1-3 plausible inputs for that primitive
|
||||
(e.g., `l1-s3` declares `bucket_name`, `region`, `retention_days`; `l1-iam-role`
|
||||
declares `role_name`, `trust_policy`). Each `mock_apply.sh` follows the exact
|
||||
uniform template from ARCHITECTURE.md.
|
||||
**Files owned:** `scripts/finalize_evidence.py`, `.gitea/workflows/pipeline.yml`, `contracts-repo/.gitea/workflows/issue-to-contract.yml`
|
||||
|
||||
**Files owned (territory):** `modules/l1/**`
|
||||
**Commits:** one per task, `phase: 4, status: plan-as-execute, persona: backend-engineer, task: T-4.x, requirements.covered: [REQ-10 or REQ-12]`.
|
||||
|
||||
**Commits:** one per task, `---ci---` block has `phase: 2, status: plan-as-execute, persona: infra-stub-engineer, task: T-2.x, requirements.covered: [REQ-02, REQ-03]`.
|
||||
|
||||
### Wave 2 — lead-developer (verification script + traceability)
|
||||
### Wave 2 — lead-developer (verify script + traceability)
|
||||
|
||||
**Tasks:**
|
||||
|
||||
- **T-2.9** Create `scripts/verify_phase02.sh`. It:
|
||||
1. Enumerates `modules/l1/*/` and confirms exactly 8 folders with the 8 expected names.
|
||||
2. For each L1: confirms `manifest.yaml` exists and parses as YAML with `name` matching the folder, `kind: l1`, and an `inputs:` map.
|
||||
3. For each L1: confirms `mock_apply.sh` is executable, `bash -n` clean, runs in <2s, exits 0, and its stdout contains the `[L1: <name>] applying...` and `[L1: <name>] OK` markers.
|
||||
4. Prints a PASS/FAIL summary; exits 0 on full success.
|
||||
- **T-2.10** Update `.ciagent/REQUIREMENTS.md` (REQ-02/03 → covered pending VERIFY) and `.ciagent/ROADMAP.md` (Phase 02 → executing). No README change.
|
||||
- **T-4.4** Create `scripts/verify_phase04.sh`. Checks:
|
||||
1. `.gitea/workflows/pipeline.yml` parses as YAML; has `on.workflow_dispatch` with inputs `contract-ref`, `approve_qa`, `approve_prod`; has 4 jobs `dev`, `qa-gate`, `prod-gate`, `finalize`; `dev.if` excludes approve_qa/approve_prod; `qa-gate.if` requires approve_qa; `prod-gate.if` requires approve_prod; `finalize.needs == prod-gate`.
|
||||
2. `contracts-repo/.gitea/workflows/issue-to-contract.yml` parses as YAML; has `on.issues` with `[opened]`; the job references `l3b_agent_stub.py` and the dispatch API endpoint `/actions/workflows/pipeline.yml/dispatches`.
|
||||
3. `scripts/finalize_evidence.py` is py_compile clean; `--help` exits 0; called with a non-existent token env fails cleanly (exit 1, not a stack trace).
|
||||
4. `bash -n` + `py_compile` + `yaml.safe_load` on all phase files.
|
||||
5. End-to-end mini-test: run `evidence_writer.py` to produce a temp `audit.json`, then call `finalize_evidence.py` against a mock Gitea API (set `GITEA_HOST=http://127.0.0.1:0` so the connection fails; verify the script exits 1 cleanly without crashing).
|
||||
- **T-4.5** Update `.ciagent/REQUIREMENTS.md` (REQ-10/12 → covered pending VERIFY) and `.ciagent/ROADMAP.md` (Phase 04 → executing).
|
||||
|
||||
**Files owned (territory):** `scripts/verify_phase02.sh`, `.ciagent/REQUIREMENTS.md`, `.ciagent/ROADMAP.md`
|
||||
**Files owned:** `scripts/verify_phase04.sh`, `.ciagent/REQUIREMENTS.md`, `.ciagent/ROADMAP.md`
|
||||
|
||||
**Commits:** one per task, `---ci---` block has `phase: 2, status: plan-as-execute, persona: lead-developer, task: T-2.9/2.10`.
|
||||
**Commits:** one per task, `phase: 4, status: plan-as-execute, persona: lead-developer, task: T-4.4/4.5`.
|
||||
|
||||
## Wave ordering
|
||||
|
||||
- Wave 1 (infra-stub-engineer) creates all 8 L1s. A single subagent gets all 8 tasks; it commits per task.
|
||||
- Wave 2 (lead-developer) adds the verify script and traceability after the L1s exist.
|
||||
- Wave 1 (backend) builds the pipeline + helper + issue trigger.
|
||||
- Wave 2 (lead-developer) wires the verify script after the workflows exist.
|
||||
|
||||
`backend-engineer`, `data-engineer`, `frontend-engineer` have 0 tasks this phase.
|
||||
`infra-stub-engineer`, `data-engineer`, `frontend-engineer` have 0 tasks.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Depends on Phase 01 (the `modules/l1/.gitkeep` from T-1.1 is replaced by real folders).
|
||||
- Phase 03 depends on this phase for L1 references in L2 compositions.
|
||||
- Depends on Phase 03 (all 5 core scripts must exist for the workflow to invoke them).
|
||||
- Phase 05 depends on this phase for the finalize step to produce `audit.json` on `acdl-evidence` that the UI fetches.
|
||||
|
||||
## Risk notes
|
||||
|
||||
- The full pipeline cannot be exercised end-to-end without an act_runner registered to the `acdl` repo (Phase 04 ships the workflow but cannot trigger a real run in this environment). verify_phase04.sh validates structure + syntax + a dry-run of finalize_evidence against a dead host; the end-to-end Act 2/3/4 dry run is Phase 05.
|
||||
- `actions/checkout@v4` cross-repo requires the `GITEA_TOKEN` secret to be set on the `acdl` repo (out-of-band Gitea UI step). verify_phase04 cannot test this; documented in the workflow YAML comments.
|
||||
+12
-1
@@ -78,4 +78,15 @@ Build a runnable demo (Linux + GitHub/Gitea Actions) that walks executives throu
|
||||
| D-016 | Pages placeholder for Phase 01 is a minimal HTML stub (`<title>ACDL Evidence</title>` + "evidence stream coming soon"); full UI deferred to Phase 05 | Phase 01 success criterion is "Pages returns 200 with placeholder index.html" but Gitea has no Pages | Raw-URL HTTP 200 against `index.html` substitutes for the Pages check; full timeline UI built in Phase 05 |
|
||||
| D-017 | Each L1 `manifest.yaml` declares a single `inputs:` map of named string keys with descriptions; no nested types (substrate-agnostic, max-depth-1) | REQ-02/03 say "declared inputs"; spec forbids composition and cloud-specific types | Uniform, parseable schema that Phase 03's `mock_executor.sh` can read with python+yaml |
|
||||
| D-018 | L1 `mock_apply.sh` reads its own `manifest.yaml` for self-identification but ignores the input values (uniform stub per D-007) | D-007 mandates a literal echo + 1s sleep + exit 0; inputs are declared for traceability, not consumed | Predictable evidence events + clean separation from Phase 03 where L2s pass inputs to L1s |
|
||||
| D-019 | The 8 L1 names are fixed per REQ-02: `l1-eks-fargate`, `l1-iam-role`, `l1-lambda`, `l1-api-gateway`, `l1-eventbridge`, `l1-sqs`, `l1-s3`, `l1-cloudwatch` | REQ-02 literal | Phase 02 enumerates them exactly; no naming freedom |
|
||||
| D-019 | The 8 L1 names are fixed per REQ-02: `l1-eks-fargate`, `l1-iam-role`, `l1-lambda`, `l1-api-gateway`, `l1-eventbridge`, `l1-sqs`, `l1-s3`, `l1-cloudwatch` | REQ-02 literal | Phase 02 enumerates them exactly; no naming freedom |
|
||||
| D-020 | L2 `manifest.yaml` schema: `name`, `kind: l2`, `description`, `l1s:` (list of `{name, inputs: map}` entries). L2 references L1s by name (no path); inputs are string maps per L1 manifest declarations | REQ-04 says L2 "composes L1s"; REQ-05 caps depth at 5 (L2→L1 is depth 1) | mock_executor.sh reads `l1s:` and invokes each L1's `mock_apply.sh` |
|
||||
| D-021 | `contract.yaml` schema: `stack` (L2 name), `inputs` (string map for the L2's top-level params), optional `public-ingress: bool` (the policy violation key per REQ-07) | REQ-07 cites `public-ingress: true` as the forbidden key; REQ-08's confidence signal keys off policy pass/fail | Single flat schema drives both policy_checker and the mock_executor |
|
||||
| D-022 | `mock_executor.sh` writes `state.json` with shape `{"l2": "<name>", "l1s": [{"name":"...","applied":true,"exit_code":0}], "contract": <contract.yaml parsed>}` to the cwd; idempotent (overwrites) | REQ-06 says "writes state.json" but does not specify shape | Deterministic, parseable; Phase 05's evidence UI can include it in the audit trail |
|
||||
| D-023 | `evidence_writer.py` appends events to `audit.json` (a JSON array of event objects). Each event: `{"seq": N, "ts": <iso8601>, "stage": "dev|qa|prod|finalize", "event": "<string>", "prev_hash": "<sha256>", "hash": "<sha256 of canonical json of this event with hash field empty>"}`. The genesis event has `prev_hash: "GENESIS"` and `seq: 0` | D-005 mandates hash-chained ledger; canonical JSON for deterministic hashing | Visible tamper-evidence without overengineering; Phase 05 UI reads the array |
|
||||
| D-024 | `confidence_signal.py` reads `contract.yaml`, calls `policy_checker.py` (as a subprocess or import), returns base 0.90 on pass and 0.40 with reason code on policy failure; prints `{"score": 0.90|0.40, "reason": "<POLICY_VIOLATION:...|>"}` to stdout; exit 0 always | REQ-08 literal: base 0.90, drops to 0.40, gate ≥ 0.50 | Deterministic JSON output for the pipeline to consume |
|
||||
| D-025 | `policy_checker.py` reads `contract.yaml`, fails with exit code 1 and stdout `POLICY_VIOLATION:PUBLIC_INGRESS` if `public-ingress: true`; otherwise exits 0 with stdout `POLICY_PASS` | REQ-07 literal | Single source of policy truth; called by confidence_signal and the pipeline directly |
|
||||
| D-026 | `l3b_agent_stub.py` reads Issue body text from argv[1] (or stdin if no argv), applies the D-008 keyword map, writes a `contract.yaml` to stdout (or to `-o <path>`). Output contract uses the D-021 schema with `stack:` set to the mapped L2 name and a fixed `inputs:` map per L2 | D-008 + Act 3 example; L3B must produce the same contract format as L3A | Deterministic keyword parser; no external APIs |
|
||||
| D-027 | Phase 04 models the pipeline as TWO Gitea Actions workflows: (1) `acdl/.gitea/workflows/pipeline.yml` — `on: workflow_call` + `on: workflow_dispatch` (so it can be both called by the contracts-repo trigger AND manually re-dispatched for approvals); (2) `acdl-contracts/.gitea/workflows/issue-to-contract.yml` — `on: issues [opened]`. Approval gates are implemented as separate workflow_dispatch inputs (`approve_qa: bool`, `approve_prod: bool`) on the pipeline workflow, since Gitea ignores `environment:` blocks (D-013) | Gitea Actions has no environment reviewers, no `repository_dispatch`, no native approval UI | Pipeline can be re-dispatched by a human at each gate; the workflow_dispatch API call from a step (D-014) drives cross-repo triggering |
|
||||
| D-028 | The pipeline workflow runs all 4 stages (dev, qa-gate, prod-gate, finalize) in a single workflow run, with each gate job checking a workflow_dispatch input (`approve_qa`/`approve_prod`). When the input is false (the default), the gate job fails with a clear "awaiting approval" message; the human re-dispatches with `approve_qa=true` to advance. State (state.json, audit.json, contract ref) is passed via workflow artifacts (upload/download between jobs) because Gitea Actions artifacts work the same as GitHub Actions | Gitea Actions supports `actions/upload-artifact` and `actions/download-artifact`; the alternative is committing state between jobs, which is heavier | Deterministic, observable pipeline; artifacts keep the audit trail within one run |
|
||||
| D-029 | The finalize step commits `audit.json` to `acdl-evidence` main via the Gitea file-contents API (POST `/repos/{owner}/{repo}/contents/{path}` with the base64 content + a commit message referencing the pipeline run id), exactly like Phase 01's `gitea_setup.sh` does for `index.html`. It uses `${GITEA_TOKEN}` (a repo secret) for auth | D-012 raw-URL approach requires the file to be on main; the API is the only way to put it there from a workflow step | The evidence timeline (Phase 05 UI) fetches the raw URL after finalize completes |
|
||||
| D-030 | The issue-to-contract workflow in `acdl-contracts` checks out `l3b_agent_stub.py` from the `acdl` repo (pinned to `@milestone/v1.0-initial` per the branch-pin rule), parses the Issue body, commits `contract.yaml` to a new branch `contract/<issue-number>` on `acdl-contracts`, then dispatches the pipeline workflow on the `acdl` repo via `curl POST /actions/workflows/<id>/dispatches` with `inputs: {contract-ref: contract/<issue-number>}` (D-014). The pipeline workflow checks out `acdl-contracts` at that ref to read the contract | Gitea Actions cannot trigger across repos without an explicit API call; the branch carries the contract ref | Reproducible Act 3: Issue → contract.yaml → pipeline run with the same contract as Act 2 |
|
||||
+10
-10
@@ -58,17 +58,17 @@
|
||||
| Requirement | Phase | Status |
|
||||
|-------------|-------|--------|
|
||||
| REQ-01 | 1 | complete (v1.0.1) |
|
||||
| REQ-02 | 2 | covered (pending VERIFY) |
|
||||
| REQ-03 | 2 | covered (pending VERIFY) |
|
||||
| REQ-04 | 3 | pending |
|
||||
| REQ-05 | 3 | pending |
|
||||
| REQ-06 | 3 | pending |
|
||||
| REQ-07 | 3 | pending |
|
||||
| REQ-08 | 3 | pending |
|
||||
| REQ-02 | 2 | complete (v1.0.2) |
|
||||
| REQ-03 | 2 | complete (v1.0.2) |
|
||||
| REQ-04 | 3 | complete (v1.0.3) |
|
||||
| REQ-05 | 3 | complete (v1.0.3) |
|
||||
| REQ-06 | 3 | complete (v1.0.3) |
|
||||
| REQ-07 | 3 | complete (v1.0.3) |
|
||||
| REQ-08 | 3 | complete (v1.0.3) |
|
||||
| REQ-09 | 1 | complete (v1.0.1) |
|
||||
| REQ-10 | 4 | partial (skeleton in Phase 01 v1.0.1; full impl in Phase 04) |
|
||||
| REQ-11 | 3 | pending |
|
||||
| REQ-12 | 4 | partial (skeleton in Phase 01 v1.0.1; full impl in Phase 04) |
|
||||
| REQ-10 | 4 | covered (pending VERIFY) |
|
||||
| REQ-11 | 3 | complete (v1.0.3) |
|
||||
| REQ-12 | 4 | covered (pending VERIFY) |
|
||||
| REQ-13 | 5 | pending |
|
||||
| REQ-14 | 5 | pending |
|
||||
| REQ-15 | 5 | pending |
|
||||
+3
-3
@@ -18,7 +18,7 @@ Five-phase breakdown to take ACDL from empty repo to a reproducible 4-act execut
|
||||
|
||||
### Phase 02 — l1-modules
|
||||
- **Description:** Create all 8 L1 module folders under `acdl/modules/l1/`, each with `manifest.yaml` (declared inputs) and `mock_apply.sh` (uniform echo + 1s sleep + exit 0).
|
||||
- **Status:** executing
|
||||
- **Status:** complete (v1.0.2)
|
||||
- **Depends on:** [1]
|
||||
- **Requirements:** REQ-02, REQ-03
|
||||
- **Success Criteria:**
|
||||
@@ -27,7 +27,7 @@ Five-phase breakdown to take ACDL from empty repo to a reproducible 4-act execut
|
||||
|
||||
### Phase 03 — l2-modules-and-core-scripts
|
||||
- **Description:** Create the 4 L2 compositions under `acdl/modules/l2/` referencing L1s, plus the 5 core scripts in `acdl/scripts/` (`mock_executor.sh`, `policy_checker.py`, `confidence_signal.py`, `evidence_writer.py`, `l3b_agent_stub.py`).
|
||||
- **Status:** not_started
|
||||
- **Status:** complete (v1.0.3)
|
||||
- **Depends on:** [2]
|
||||
- **Requirements:** REQ-04, REQ-05, REQ-06, REQ-07
|
||||
- **Success Criteria:**
|
||||
@@ -39,7 +39,7 @@ Five-phase breakdown to take ACDL from empty repo to a reproducible 4-act execut
|
||||
|
||||
### Phase 04 — pipeline-and-approval-gates
|
||||
- **Description:** Build the reusable pipeline workflow in `acdl/.gitea/workflows/` (Dev → QA → Prod → Finalize) plus the issue-triggered L3B workflow in `acdl-contracts/.gitea/workflows/`. Wire environment protection for QA and Prod.
|
||||
- **Status:** not_started
|
||||
- **Status:** executing
|
||||
- **Depends on:** [3]
|
||||
- **Requirements:** REQ-08, REQ-09, REQ-10, REQ-12
|
||||
- **Success Criteria:**
|
||||
|
||||
+118
-40
@@ -1,75 +1,153 @@
|
||||
# ACDL reusable pipeline workflow (Phase 01 skeleton).
|
||||
# ACDL pipeline workflow (Phase 04 implementation).
|
||||
#
|
||||
# This workflow is called from acdl-contracts via:
|
||||
# uses: continuous-intelligence/acdl/.gitea/workflows/pipeline.yml@milestone/v1.0-initial
|
||||
# 3-dispatch approval-gate topology (D-027 / D-028; ARCHITECTURE.md
|
||||
# "Phase 04 pipeline topology"):
|
||||
#
|
||||
# Branch pinning rule (see .ciagent/ARCHITECTURE.md): the `acdl` repo's default
|
||||
# branch is `milestone/v1.0-initial`, so `uses:` references must pin to
|
||||
# `@milestone/v1.0-initial`, NOT `@main`.
|
||||
# Dispatch 1 (initial): approve_qa=false, approve_prod=false
|
||||
# -> runs the `dev` job (policy check, confidence
|
||||
# gate, mock_executor, evidence + finalize).
|
||||
# Dispatch 2 (QA approve): approve_qa=true, approve_prod=false
|
||||
# -> runs the `qa-gate` job (records QA approval
|
||||
# in the audit chain via evidence_writer +
|
||||
# finalize_evidence).
|
||||
# Dispatch 3 (Prod approve): approve_prod=true
|
||||
# -> runs the `prod-gate` job, then the `finalize`
|
||||
# job (needs: prod-gate) which writes the final
|
||||
# evidence event and commits audit.json to
|
||||
# acdl-evidence.
|
||||
#
|
||||
# Phase 04 will implement the actual stage logic + approval gates (D-013:
|
||||
# Gitea has no environments API; gates become workflow_dispatch approval
|
||||
# inputs).
|
||||
# Gitea Actions limitations driving this design:
|
||||
# - No `repository_dispatch` trigger (D-014).
|
||||
# - No environments API / `environment:` blocks are ignored (D-013).
|
||||
# - Re-dispatch starts a NEW run; artifacts do NOT survive between runs,
|
||||
# so state is persisted to acdl-evidence via the file-contents API
|
||||
# (D-028 / finalize_evidence.py) instead of via artifacts.
|
||||
#
|
||||
# Branch-pin rule (ARCHITECTURE.md "Branch pinning rule"):
|
||||
# This workflow lives on `acdl`'s default branch `milestone/v1.0-initial`.
|
||||
# Cross-repo `uses:` references (e.g. the issue-trigger's checkout of
|
||||
# l3b_agent_stub.py) MUST pin to `@milestone/v1.0-initial`, NOT `@main`
|
||||
# (the `acdl` repo has no `main` branch). This workflow is invoked via
|
||||
# the workflow_dispatch API (D-014), NOT via `workflow_call`, so the
|
||||
# `uses:` rule applies to the issue-trigger's checkout of the acdl repo,
|
||||
# not to this file itself.
|
||||
name: acdl-pipeline
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
"on":
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
contract-ref:
|
||||
description: "Ref on acdl-contracts that triggered the pipeline"
|
||||
description: "Ref on acdl-contracts that carries the contract"
|
||||
required: false
|
||||
type: string
|
||||
default: main
|
||||
approve_qa:
|
||||
description: "Human approval to advance past QA"
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
approve_prod:
|
||||
description: "Human approval to advance past Prod"
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
|
||||
jobs:
|
||||
dev:
|
||||
name: "Dev (autonomous)"
|
||||
if: inputs.approve_qa != true && inputs.approve_prod != true
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
# Phase 04 will implement: checkout acdl + acdl-contracts, run
|
||||
# policy_checker.py, mock_executor.sh, confidence_signal.py, write
|
||||
# evidence via evidence_writer.py.
|
||||
- name: "Dev stage placeholder"
|
||||
- name: "Checkout acdl (this repo, pinned to milestone/v1.0-initial)"
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
ref: milestone/v1.0-initial
|
||||
|
||||
- name: "Checkout acdl-contracts at contract-ref"
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
repository: continuous-intelligence/acdl-contracts
|
||||
ref: ${{ inputs.contract-ref }}
|
||||
token: ${{ secrets.GITEA_TOKEN }}
|
||||
path: acdl-contracts
|
||||
|
||||
- name: "Policy check"
|
||||
run: |
|
||||
echo "Dev stage placeholder (Phase 01 skeleton)"
|
||||
echo "Phase 04 will run policy_checker, mock_executor, confidence_signal, evidence_writer"
|
||||
exit 0
|
||||
python3 scripts/policy_checker.py acdl-contracts/contract.yaml
|
||||
|
||||
- name: "Confidence signal"
|
||||
id: confidence
|
||||
run: |
|
||||
set +e
|
||||
SCORE_JSON=$(python3 scripts/confidence_signal.py acdl-contracts/contract.yaml)
|
||||
echo "$SCORE_JSON"
|
||||
echo "score_json=$SCORE_JSON" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: "Apply or reject based on confidence (gate < 0.50)"
|
||||
run: |
|
||||
set +e
|
||||
SCORE=$(python3 -c "import json,sys; print(json.load(sys.stdin)['score'])" <<< '${{ steps.confidence.outputs.score_json }}')
|
||||
python3 -c "import sys; sys.exit(0 if float('${SCORE}') >= 0.50 else 1)"
|
||||
THRESHOLD_RC=$?
|
||||
if [ "$THRESHOLD_RC" -ne 0 ]; then
|
||||
python3 scripts/evidence_writer.py --stage dev --event "dev rejected: confidence < 0.50" --audit audit.json
|
||||
python3 scripts/finalize_evidence.py --audit audit.json
|
||||
exit 1
|
||||
fi
|
||||
STACK=$(python3 -c 'import yaml; print(yaml.safe_load(open("acdl-contracts/contract.yaml"))["stack"])')
|
||||
bash scripts/mock_executor.sh acdl-contracts/contract.yaml
|
||||
python3 scripts/evidence_writer.py --stage dev --event "dev applied: ${STACK}" --audit audit.json
|
||||
python3 scripts/finalize_evidence.py --audit audit.json
|
||||
|
||||
- name: "Upload dev state artifacts (best-effort)"
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
name: dev-state
|
||||
path: |
|
||||
audit.json
|
||||
state.json
|
||||
|
||||
qa-gate:
|
||||
name: "QA (manual approval)"
|
||||
needs: dev
|
||||
if: inputs.approve_qa == true && inputs.approve_prod != true
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
# Phase 04 will implement: gate via workflow_dispatch approval input
|
||||
# (D-013 fallback; Gitea ignores jobs.<id>.environment).
|
||||
- name: "QA gate placeholder"
|
||||
- name: "Checkout acdl (this repo, pinned to milestone/v1.0-initial)"
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
ref: milestone/v1.0-initial
|
||||
|
||||
- name: "Record QA approval in evidence"
|
||||
run: |
|
||||
echo "QA gate placeholder (Phase 01 skeleton)"
|
||||
echo "Phase 04 will pause here for human approval via workflow_dispatch"
|
||||
exit 0
|
||||
python3 scripts/evidence_writer.py --stage qa --event "qa approved" --audit audit.json
|
||||
python3 scripts/finalize_evidence.py --audit audit.json
|
||||
|
||||
prod-gate:
|
||||
name: "Prod (manual approval)"
|
||||
needs: qa-gate
|
||||
if: inputs.approve_prod == true
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
# Phase 04 will implement: same approval-input gate as qa-gate.
|
||||
- name: "Prod gate placeholder"
|
||||
- name: "Checkout acdl (this repo, pinned to milestone/v1.0-initial)"
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
ref: milestone/v1.0-initial
|
||||
|
||||
- name: "Record Prod approval in evidence"
|
||||
run: |
|
||||
echo "Prod gate placeholder (Phase 01 skeleton)"
|
||||
echo "Phase 04 will pause here for human approval via workflow_dispatch"
|
||||
exit 0
|
||||
python3 scripts/evidence_writer.py --stage prod --event "prod approved" --audit audit.json
|
||||
python3 scripts/finalize_evidence.py --audit audit.json
|
||||
|
||||
finalize:
|
||||
name: "Finalize (publish evidence)"
|
||||
needs: prod-gate
|
||||
needs: [prod-gate]
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
# Phase 04/05 will implement: commit audit.json to acdl-evidence main
|
||||
# via the Gitea file-contents API; raw URL republishes index.html +
|
||||
# audit.json for the timeline UI (D-012).
|
||||
- name: "Finalize placeholder"
|
||||
- name: "Checkout acdl (this repo, pinned to milestone/v1.0-initial)"
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
ref: milestone/v1.0-initial
|
||||
|
||||
- name: "Write finalize event + commit audit.json to acdl-evidence"
|
||||
run: |
|
||||
echo "Finalize placeholder (Phase 01 skeleton)"
|
||||
echo "Phase 04/05 will commit audit.json to acdl-evidence main"
|
||||
exit 0
|
||||
python3 scripts/evidence_writer.py --stage finalize --event "pipeline complete: audit.json committed to acdl-evidence" --audit audit.json
|
||||
python3 scripts/finalize_evidence.py --audit audit.json
|
||||
@@ -1,18 +1,35 @@
|
||||
# ACDL issue-to-contract workflow (Phase 01 skeleton, reference copy).
|
||||
#
|
||||
# This file is the source-of-truth copy kept in the `acdl` repo under
|
||||
# contracts-repo/.gitea/workflows/. Phase 04 will push it to the actual
|
||||
# `acdl-contracts` repo under .gitea/workflows/ and implement the real
|
||||
# step bodies.
|
||||
# ACDL issue-to-contract workflow (Phase 04 implementation).
|
||||
#
|
||||
# Trigger: a new Issue is opened in acdl-contracts. The workflow runs
|
||||
# l3b_agent_stub.py to map the Issue body to a contract.yaml, commits the
|
||||
# contract to a new branch, closes the Issue, and triggers the main
|
||||
# pipeline in the `acdl` repo via the workflow_dispatch API (D-014; Gitea
|
||||
# Actions does not support repository_dispatch).
|
||||
# l3b_agent_stub.py (checked out from the `acdl` repo, pinned to
|
||||
# @milestone/v1.0-initial) to map the Issue body to a contract.yaml, commits
|
||||
# the contract to a new branch `contract/<issue-number>` on acdl-contracts
|
||||
# via the Gitea file-contents API, closes the Issue with a comment, and
|
||||
# dispatches the main pipeline in the `acdl` repo via the workflow_dispatch
|
||||
# API (D-014; Gitea Actions does not support repository_dispatch).
|
||||
#
|
||||
# Cross-repo trigger (D-014):
|
||||
# The final step POSTs to
|
||||
# /api/v1/repos/continuous-intelligence/acdl/actions/workflows/pipeline.yml/dispatches
|
||||
# with body {"ref": "milestone/v1.0-initial",
|
||||
# "inputs": {"contract-ref": "contract/<issue-number>"}}.
|
||||
#
|
||||
# Branch-pin rule (ARCHITECTURE.md):
|
||||
# The `acdl` repo's default branch is `milestone/v1.0-initial`, so the
|
||||
# checkout step pins `ref: milestone/v1.0-initial`. The pipeline dispatch
|
||||
# also pins `ref: milestone/v1.0-initial` (the workflow file lives on
|
||||
# that branch). The new `contract/<n>` branch is created on acdl-contracts
|
||||
# (whose default branch is `main`, per D-015).
|
||||
#
|
||||
# File-contents POST with `new_branch` (D-030):
|
||||
# The POST to /repos/.../contents/contract.yaml includes
|
||||
# `new_branch: contract/<n>`, which tells Gitea to create the file on a
|
||||
# NEW branch off the current head of `branch: main` instead of committing
|
||||
# directly to main. This avoids a separate branch-create + commit round
|
||||
# trip.
|
||||
name: issue-to-contract
|
||||
|
||||
on:
|
||||
"on":
|
||||
issues:
|
||||
types: [opened]
|
||||
|
||||
@@ -20,21 +37,109 @@ jobs:
|
||||
parse-and-trigger:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
# Phase 04 will implement:
|
||||
# 1. checkout acdl-contracts (so l3b_agent_stub.py is available).
|
||||
# 2. run: python3 scripts/l3b_agent_stub.py "${{ gitea.event.issue.body }}" > contract.yaml
|
||||
# 3. parse the generated contract; commit it to a new branch
|
||||
# (e.g. contract/<issue-number>).
|
||||
# 4. push the branch.
|
||||
# 5. close the Issue with a comment linking to the pipeline run.
|
||||
# 6. trigger the main pipeline:
|
||||
# curl -X POST \
|
||||
# -H "Authorization: token ${GITEA_TOKEN}" \
|
||||
# https://git.cloudinit.dev/api/v1/repos/continuous-intelligence/acdl/actions/workflows/<id>/dispatches \
|
||||
# -d '{"ref":"milestone/v1.0-initial","inputs":{"contract-ref":"<branch>"}}'
|
||||
- name: "Issue-trigger placeholder"
|
||||
- name: "Checkout acdl (pinned to milestone/v1.0-initial for l3b_agent_stub.py)"
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
repository: continuous-intelligence/acdl
|
||||
ref: milestone/v1.0-initial
|
||||
token: ${{ secrets.GITEA_TOKEN }}
|
||||
|
||||
- name: "Parse Issue body into contract.yaml"
|
||||
env:
|
||||
ISSUE_BODY: ${{ gitea.event.issue.body }}
|
||||
run: |
|
||||
echo "issue-to-contract placeholder (Phase 01 skeleton)"
|
||||
echo "Issue body: ${{ gitea.event.issue.body }}"
|
||||
echo "Phase 04 will run l3b_agent_stub.py, commit contract.yaml, close issue, dispatch pipeline"
|
||||
exit 0
|
||||
# Pass the Issue body via an env var to avoid shell injection from
|
||||
# arbitrary Issue text. l3b_agent_stub.py reads argv[1]; we pass
|
||||
# the env var quoted so no metacharacter interpretation happens.
|
||||
python3 scripts/l3b_agent_stub.py "$ISSUE_BODY" -o contract.yaml
|
||||
echo "--- generated contract.yaml ---"
|
||||
cat contract.yaml
|
||||
|
||||
- name: "Commit contract.yaml to new branch contract/${{ gitea.event.issue.number }} on acdl-contracts"
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
STACK=$(python3 -c 'import yaml; print(yaml.safe_load(open("contract.yaml"))["stack"])')
|
||||
ISSUE_NUMBER="${{ gitea.event.issue.number }}"
|
||||
BRANCH="contract/${ISSUE_NUMBER}"
|
||||
HOST="https://git.cloudinit.dev"
|
||||
API="${HOST}/api/v1/repos/continuous-intelligence/acdl-contracts/contents/contract.yaml"
|
||||
B64=$(base64 -w 0 contract.yaml)
|
||||
BODY=$(python3 -c "
|
||||
import json
|
||||
print(json.dumps({
|
||||
'content': '${B64}',
|
||||
'message': 'l3b: contract for issue #${ISSUE_NUMBER}',
|
||||
'branch': 'main',
|
||||
'new_branch': '${BRANCH}'
|
||||
}))
|
||||
")
|
||||
STATUS=$(curl -sS -o /tmp/contract_post.json -w "%{http_code}" \
|
||||
-X POST \
|
||||
-H "Authorization: token ${GITEA_TOKEN}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "$BODY" \
|
||||
"$API")
|
||||
echo "POST contract.yaml -> HTTP ${STATUS}"
|
||||
cat /tmp/contract_post.json || true
|
||||
case "$STATUS" in
|
||||
201) echo "contract.yaml committed on branch ${BRANCH}" ;;
|
||||
*) echo "ERROR: file-contents POST failed (HTTP ${STATUS})" >&2; exit 1 ;;
|
||||
esac
|
||||
echo "STACK=${STACK}" >> "$GITHUB_ENV"
|
||||
echo "BRANCH=${BRANCH}" >> "$GITHUB_ENV"
|
||||
|
||||
- name: "Comment on Issue + close it"
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
ISSUE_NUMBER="${{ gitea.event.issue.number }}"
|
||||
HOST="https://git.cloudinit.dev"
|
||||
ISSUES_API="${HOST}/api/v1/repos/continuous-intelligence/acdl-contracts/issues/${ISSUE_NUMBER}"
|
||||
COMMENT_BODY=$(python3 -c "
|
||||
import json
|
||||
print(json.dumps({'body': 'Generated contract.yaml for stack \`' + '${STACK}' + '\` on branch \`' + '${BRANCH}' + '\`. Pipeline dispatched.'}))
|
||||
")
|
||||
curl -sS -o /tmp/comment.json -w "comment HTTP %{http_code}\n" \
|
||||
-X POST \
|
||||
-H "Authorization: token ${GITEA_TOKEN}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "$COMMENT_BODY" \
|
||||
"${ISSUES_API}/comments"
|
||||
CLOSE_BODY='{"state":"closed"}'
|
||||
curl -sS -o /tmp/close.json -w "close HTTP %{http_code}\n" \
|
||||
-X PATCH \
|
||||
-H "Authorization: token ${GITEA_TOKEN}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "$CLOSE_BODY" \
|
||||
"${ISSUES_API}"
|
||||
|
||||
- name: "Dispatch the pipeline on acdl (contract-ref = contract/${{ gitea.event.issue.number }})"
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
ISSUE_NUMBER="${{ gitea.event.issue.number }}"
|
||||
HOST="https://git.cloudinit.dev"
|
||||
DISPATCH_URL="${HOST}/api/v1/repos/continuous-intelligence/acdl/actions/workflows/pipeline.yml/dispatches"
|
||||
BODY=$(python3 -c "
|
||||
import json
|
||||
print(json.dumps({
|
||||
'ref': 'milestone/v1.0-initial',
|
||||
'inputs': {'contract-ref': 'contract/${ISSUE_NUMBER}'}
|
||||
}))
|
||||
")
|
||||
STATUS=$(curl -sS -o /tmp/dispatch.json -w "%{http_code}" \
|
||||
-X POST \
|
||||
-H "Authorization: token ${GITEA_TOKEN}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "$BODY" \
|
||||
"$DISPATCH_URL")
|
||||
echo "pipeline dispatch -> HTTP ${STATUS}"
|
||||
cat /tmp/dispatch.json || true
|
||||
case "$STATUS" in
|
||||
201|202|204) echo "pipeline dispatched (contract-ref=contract/${ISSUE_NUMBER})" ;;
|
||||
*) echo "ERROR: pipeline dispatch failed (HTTP ${STATUS})" >&2; exit 1 ;;
|
||||
esac
|
||||
@@ -0,0 +1,27 @@
|
||||
name: l2-commodity-price-feed
|
||||
kind: l2
|
||||
description: Real-time commodity price ingestion from Platts
|
||||
l1s:
|
||||
- name: l1-eks-fargate
|
||||
inputs:
|
||||
cluster_name: price-feed-cluster
|
||||
region: us-east-1
|
||||
cpu_arch: arm64
|
||||
- name: l1-lambda
|
||||
inputs:
|
||||
function_name: price-ingest
|
||||
runtime: python3.11
|
||||
handler: index.handler
|
||||
- name: l1-api-gateway
|
||||
inputs:
|
||||
api_name: platts-price-api
|
||||
stage_name: dev
|
||||
- name: l1-eventbridge
|
||||
inputs:
|
||||
bus_name: price-events
|
||||
rule_name: price-publish-rule
|
||||
- name: l1-s3
|
||||
inputs:
|
||||
bucket_name: acdl-price-archive
|
||||
region: us-east-1
|
||||
retention_days: "90"
|
||||
@@ -0,0 +1,27 @@
|
||||
name: l2-energy-analytics-api
|
||||
kind: l2
|
||||
description: Historical energy analytics query API
|
||||
l1s:
|
||||
- name: l1-eks-fargate
|
||||
inputs:
|
||||
cluster_name: analytics-cluster
|
||||
region: us-east-1
|
||||
cpu_arch: arm64
|
||||
- name: l1-api-gateway
|
||||
inputs:
|
||||
api_name: energy-analytics-api
|
||||
stage_name: dev
|
||||
- name: l1-lambda
|
||||
inputs:
|
||||
function_name: analytics-query
|
||||
runtime: python3.11
|
||||
handler: index.handler
|
||||
- name: l1-s3
|
||||
inputs:
|
||||
bucket_name: acdl-analytics-data
|
||||
region: us-east-1
|
||||
retention_days: "2555"
|
||||
- name: l1-cloudwatch
|
||||
inputs:
|
||||
log_group_name: /acdl/analytics-api
|
||||
metric_namespace: acdl/analytics
|
||||
@@ -0,0 +1,27 @@
|
||||
name: l2-invoice-service
|
||||
kind: l2
|
||||
description: Billing and invoicing microservice for energy trades
|
||||
l1s:
|
||||
- name: l1-eks-fargate
|
||||
inputs:
|
||||
cluster_name: invoice-cluster
|
||||
region: us-east-1
|
||||
cpu_arch: arm64
|
||||
- name: l1-iam-role
|
||||
inputs:
|
||||
role_name: invoice-service-role
|
||||
trust_policy: '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"eks.amazonaws.com"},"Action":"sts:AssumeRole"}]}'
|
||||
- name: l1-lambda
|
||||
inputs:
|
||||
function_name: invoice-generator
|
||||
runtime: python3.11
|
||||
handler: index.handler
|
||||
- name: l1-sqs
|
||||
inputs:
|
||||
queue_name: invoice-queue
|
||||
visibility_timeout: "60"
|
||||
- name: l1-s3
|
||||
inputs:
|
||||
bucket_name: acdl-invoice-archive
|
||||
region: us-east-1
|
||||
retention_days: "365"
|
||||
@@ -0,0 +1,27 @@
|
||||
name: l2-regulatory-reporting
|
||||
kind: l2
|
||||
description: Regulatory compliance and reporting for energy trading
|
||||
l1s:
|
||||
- name: l1-eks-fargate
|
||||
inputs:
|
||||
cluster_name: regulatory-cluster
|
||||
region: us-east-1
|
||||
cpu_arch: arm64
|
||||
- name: l1-iam-role
|
||||
inputs:
|
||||
role_name: regulatory-reporting-role
|
||||
trust_policy: '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"eks.amazonaws.com"},"Action":"sts:AssumeRole"}]}'
|
||||
- name: l1-lambda
|
||||
inputs:
|
||||
function_name: regulatory-reporter
|
||||
runtime: python3.11
|
||||
handler: index.handler
|
||||
- name: l1-sqs
|
||||
inputs:
|
||||
queue_name: regulatory-queue
|
||||
visibility_timeout: "120"
|
||||
- name: l1-s3
|
||||
inputs:
|
||||
bucket_name: acdl-regulatory-archive
|
||||
region: us-east-1
|
||||
retention_days: "2555"
|
||||
Executable
+55
@@ -0,0 +1,55 @@
|
||||
#!/usr/bin/env python3
|
||||
"""confidence_signal.py — REQ-08 / D-024
|
||||
|
||||
Reads a contract.yaml, invokes policy_checker.py as a subprocess, and emits
|
||||
a deterministic JSON confidence score.
|
||||
|
||||
policy pass -> {"score": 0.90, "reason": "POLICY_PASS"}
|
||||
policy fail -> {"score": 0.40, "reason": "<violation code>"}
|
||||
|
||||
Exit 0 ALWAYS (per D-024): the pipeline decides the gate, not this script's
|
||||
exit code.
|
||||
|
||||
Input: argv[1] = path to a contract.yaml file.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
|
||||
def main() -> int:
|
||||
if len(sys.argv) < 2:
|
||||
print("usage: confidence_signal.py <contract.yaml>", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
contract_path = sys.argv[1]
|
||||
|
||||
# Resolve policy_checker.py relative to this script so it works regardless
|
||||
# of cwd. Use python3 + script path (not ./) per the contract.
|
||||
here = os.path.dirname(os.path.abspath(__file__))
|
||||
policy_checker = os.path.join(here, "policy_checker.py")
|
||||
|
||||
proc = subprocess.run(
|
||||
["python3", policy_checker, contract_path],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
|
||||
if proc.returncode == 0:
|
||||
score = "0.90"
|
||||
# POLICY_PASS is the expected stdout; strip any trailing whitespace.
|
||||
reason = proc.stdout.strip() or "POLICY_PASS"
|
||||
else:
|
||||
score = "0.40"
|
||||
# The violation code (e.g. "POLICY_VIOLATION:PUBLIC_INGRESS") is on stdout.
|
||||
reason = proc.stdout.strip() or "POLICY_VIOLATION:UNKNOWN"
|
||||
|
||||
# Emit with literal score (two-decimal form per the contract) and a quoted
|
||||
# reason. Constructed manually so json.dumps does not collapse 0.90 -> 0.9.
|
||||
print('{"score": ' + score + ', "reason": ' + json.dumps(reason) + '}')
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Executable
+123
@@ -0,0 +1,123 @@
|
||||
#!/usr/bin/env python3
|
||||
"""evidence_writer.py — REQ-11 / D-023 / D-005
|
||||
|
||||
Appends a hash-chained event to audit.json.
|
||||
|
||||
Each event: {"seq": N, "ts": <iso8601 UTC>, "stage": "...", "event": "...",
|
||||
"prev_hash": "<sha256 or GENESIS>", "hash": "<sha256 of canonical json of this event with hash empty>"}
|
||||
|
||||
Hash chain (D-023):
|
||||
1. Build event dict with hash = "" (empty string).
|
||||
2. canonical = json.dumps(event, sort_keys=True, separators=(",", ":"))
|
||||
3. hash = sha256(canonical.encode("utf-8")).hexdigest()
|
||||
4. event["hash"] = hash
|
||||
5. append to audit.json
|
||||
|
||||
Auto-genesis: if audit.json is empty/missing and --stage is not "genesis",
|
||||
a genesis event (seq 0, prev_hash "GENESIS") is inserted first.
|
||||
|
||||
Input:
|
||||
--stage <dev|qa|prod|finalize|genesis> (required)
|
||||
--event "<text>" (required)
|
||||
--audit <path> (optional, default ./audit.json)
|
||||
Output: stdout {"seq": N, "hash": "..."}
|
||||
Exit: 0 on success, 1 on I/O error.
|
||||
"""
|
||||
import argparse
|
||||
import datetime
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
GENESIS_EVENT_TEXT = "audit log initialized"
|
||||
|
||||
|
||||
def now_iso8601_utc() -> str:
|
||||
return datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
|
||||
|
||||
def compute_hash(event: dict) -> str:
|
||||
"""Compute the sha256 hash of an event using canonical JSON (D-023)."""
|
||||
tmp = dict(event)
|
||||
tmp["hash"] = ""
|
||||
canonical = json.dumps(tmp, sort_keys=True, separators=(",", ":"))
|
||||
return hashlib.sha256(canonical.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def make_event(seq: int, stage: str, event_text: str, prev_hash: str) -> dict:
|
||||
event = {
|
||||
"seq": seq,
|
||||
"ts": now_iso8601_utc(),
|
||||
"stage": stage,
|
||||
"event": event_text,
|
||||
"prev_hash": prev_hash,
|
||||
"hash": "",
|
||||
}
|
||||
event["hash"] = compute_hash(event)
|
||||
return event
|
||||
|
||||
|
||||
def load_audit(audit_path: str) -> list:
|
||||
if not os.path.exists(audit_path):
|
||||
return []
|
||||
try:
|
||||
with open(audit_path, "r", encoding="utf-8") as fh:
|
||||
data = json.load(fh)
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
return []
|
||||
if not isinstance(data, list):
|
||||
return []
|
||||
return data
|
||||
|
||||
|
||||
def atomic_write(audit_path: str, data: list) -> None:
|
||||
tmp_path = audit_path + ".tmp"
|
||||
with open(tmp_path, "w", encoding="utf-8") as fh:
|
||||
json.dump(data, fh, indent=2)
|
||||
fh.write("\n")
|
||||
os.replace(tmp_path, audit_path)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description="Append a hash-chained event to audit.json")
|
||||
parser.add_argument("--stage", required=True,
|
||||
choices=["dev", "qa", "prod", "finalize", "genesis"])
|
||||
parser.add_argument("--event", required=True)
|
||||
parser.add_argument("--audit", default="./audit.json")
|
||||
args = parser.parse_args()
|
||||
|
||||
events = load_audit(args.audit)
|
||||
|
||||
# Auto-genesis: if the log is empty and the caller did not ask for a
|
||||
# genesis event, seed one first.
|
||||
if len(events) == 0 and args.stage != "genesis":
|
||||
genesis = make_event(seq=0, stage="genesis", event_text=GENESIS_EVENT_TEXT,
|
||||
prev_hash="GENESIS")
|
||||
events.append(genesis)
|
||||
|
||||
# Determine the new seq + prev_hash.
|
||||
if events:
|
||||
last = events[-1]
|
||||
seq = last["seq"] + 1
|
||||
prev_hash = last["hash"]
|
||||
else:
|
||||
seq = 0
|
||||
prev_hash = "GENESIS"
|
||||
|
||||
new_event = make_event(seq=seq, stage=args.stage, event_text=args.event,
|
||||
prev_hash=prev_hash)
|
||||
events.append(new_event)
|
||||
|
||||
try:
|
||||
atomic_write(args.audit, events)
|
||||
except OSError as exc:
|
||||
print(f"evidence_writer: I/O error: {exc}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
print(json.dumps({"seq": new_event["seq"], "hash": new_event["hash"]}))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Executable
+182
@@ -0,0 +1,182 @@
|
||||
#!/usr/bin/env python3
|
||||
"""finalize_evidence.py — REQ-10 / D-028 / D-029
|
||||
|
||||
Uploads (PUT or POST) a local `audit.json` to the `acdl-evidence` repo on
|
||||
Gitea via the file-contents API. Used by the pipeline workflow steps to
|
||||
persist the hash-chained audit trail to `acdl-evidence` between dispatches
|
||||
(D-028 state-persistence across re-dispatches; D-029 finalize step).
|
||||
|
||||
Uses only the Python standard library (urllib.request) so it has no
|
||||
external dependency on `requests`. Auth header: `Authorization: token <token>`.
|
||||
|
||||
Input (argv flags):
|
||||
--audit <path> (required) local audit.json file to upload
|
||||
--owner <org> (optional, default continuous-intelligence)
|
||||
--repo <name> (optional, default acdl-evidence)
|
||||
--branch <name> (optional, default main)
|
||||
--path <remote path> (optional, default audit.json) path in the repo
|
||||
--token-env <env var> (optional, default ACDL_GITEA_TOKEN)
|
||||
--host <url> (optional, default https://git.cloudinit.dev)
|
||||
--message <commit msg> (optional, default chore(evidence): update audit.json)
|
||||
|
||||
Behavior:
|
||||
1. Read the token from os.environ[token_env]. Missing -> stderr + exit 1.
|
||||
2. Read the local audit file; base64-encode it.
|
||||
3. GET the current file at .../contents/<path>?ref=<branch> to discover
|
||||
the existing `sha`. 200 -> capture sha (update mode). 404 -> no sha
|
||||
(create mode). Other errors -> exit 1.
|
||||
4. If sha set: PUT with body {content, message, branch, sha}.
|
||||
If no sha: POST with body {content, message, branch}.
|
||||
5. Print {"uploaded": true, "path": "<path>", "sha": "<new sha>"} to
|
||||
stdout and exit 0.
|
||||
6. On any HTTP error: print
|
||||
{"uploaded": false, "status": <code>, "body": "<body>"} to stdout
|
||||
and exit 1.
|
||||
"""
|
||||
import argparse
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
|
||||
def _request(method: str, url: str, token: str, body: dict = None):
|
||||
"""Perform an HTTP request with the Gitea auth header. Returns
|
||||
(status_code, response_body_text). Raises URLError on network failure."""
|
||||
data = None
|
||||
headers = {"Authorization": f"token {token}",
|
||||
"Accept": "application/json"}
|
||||
if body is not None:
|
||||
data = json.dumps(body).encode("utf-8")
|
||||
headers["Content-Type"] = "application/json"
|
||||
req = urllib.request.Request(url, data=data, method=method, headers=headers)
|
||||
try:
|
||||
with urllib.request.urlopen(req) as resp:
|
||||
return resp.getcode(), resp.read().decode("utf-8", "replace")
|
||||
except urllib.error.HTTPError as exc:
|
||||
# HTTPError carries the response body
|
||||
try:
|
||||
body_text = exc.read().decode("utf-8", "replace")
|
||||
except Exception:
|
||||
body_text = ""
|
||||
return exc.code, body_text
|
||||
except urllib.error.URLError as exc:
|
||||
# Network-level failure (connection refused, DNS, timeout). Return
|
||||
# a synthetic 0 status + the reason so callers can report cleanly
|
||||
# without a stack trace.
|
||||
return 0, f"URLError: {exc.reason}"
|
||||
|
||||
|
||||
def get_existing_sha(host: str, owner: str, repo: str, path: str,
|
||||
branch: str, token: str):
|
||||
"""Return (sha-or-None, error_status_or_None). On 200 returns the sha.
|
||||
On 404 returns (None, None). Other codes return (None, (status, body))."""
|
||||
qs = urllib.parse.urlencode({"ref": branch})
|
||||
url = f"{host}/api/v1/repos/{owner}/{repo}/contents/{path}?{qs}"
|
||||
status, body = _request("GET", url, token)
|
||||
if status == 200:
|
||||
try:
|
||||
data = json.loads(body)
|
||||
return data.get("sha"), None
|
||||
except (ValueError, TypeError):
|
||||
return None, (status, body)
|
||||
if status == 404:
|
||||
return None, None
|
||||
return None, (status, body)
|
||||
|
||||
|
||||
def upload(host: str, owner: str, repo: str, path: str, branch: str,
|
||||
message: str, content_b64: str, sha, token: str):
|
||||
"""PUT (update) or POST (create) the file. Returns (new_sha, None) on
|
||||
success or (None, (status, body)) on HTTP error."""
|
||||
url = f"{host}/api/v1/repos/{owner}/{repo}/contents/{path}"
|
||||
if sha:
|
||||
body = {"content": content_b64, "message": message,
|
||||
"branch": branch, "sha": sha}
|
||||
status, resp = _request("PUT", url, token, body)
|
||||
else:
|
||||
body = {"content": content_b64, "message": message, "branch": branch}
|
||||
status, resp = _request("POST", url, token, body)
|
||||
if status in (200, 201):
|
||||
try:
|
||||
data = json.loads(resp)
|
||||
# The file-contents API returns the new content object either at
|
||||
# top-level `content` (POST create) or `content` (PUT update).
|
||||
new_sha = None
|
||||
if isinstance(data, dict):
|
||||
content_obj = data.get("content") or data
|
||||
if isinstance(content_obj, dict):
|
||||
new_sha = content_obj.get("sha")
|
||||
return new_sha, None
|
||||
except (ValueError, TypeError):
|
||||
return None, None
|
||||
return None, (status, resp)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Upload a local audit.json to the acdl-evidence Gitea "
|
||||
"repo via the file-contents API (D-028/D-029).")
|
||||
parser.add_argument("--audit", required=True,
|
||||
help="Local audit.json file to upload")
|
||||
parser.add_argument("--owner", default="continuous-intelligence",
|
||||
help="Gitea org (default: continuous-intelligence)")
|
||||
parser.add_argument("--repo", default="acdl-evidence",
|
||||
help="Gitea repo (default: acdl-evidence)")
|
||||
parser.add_argument("--branch", default="main",
|
||||
help="Target branch (default: main)")
|
||||
parser.add_argument("--path", default="audit.json",
|
||||
help="Remote path in the repo (default: audit.json)")
|
||||
parser.add_argument("--token-env", default="ACDL_GITEA_TOKEN",
|
||||
help="Env var name holding the Gitea token "
|
||||
"(default: ACDL_GITEA_TOKEN)")
|
||||
parser.add_argument("--host", default="https://git.cloudinit.dev",
|
||||
help="Gitea host URL (default: https://git.cloudinit.dev)")
|
||||
parser.add_argument("--message", default="chore(evidence): update audit.json",
|
||||
help="Commit message (default: chore(evidence): "
|
||||
"update audit.json)")
|
||||
args = parser.parse_args()
|
||||
|
||||
token = os.environ.get(args.token_env)
|
||||
if not token:
|
||||
print(f"finalize_evidence: required env var {args.token_env} is not "
|
||||
f"set", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
# Read + base64-encode the local audit file. Missing/unreadable file is
|
||||
# a clean exit 1 (no stack trace).
|
||||
try:
|
||||
with open(args.audit, "rb") as fh:
|
||||
raw = fh.read()
|
||||
except OSError as exc:
|
||||
print(f"finalize_evidence: cannot read {args.audit}: {exc}",
|
||||
file=sys.stderr)
|
||||
return 1
|
||||
content_b64 = base64.b64encode(raw).decode("ascii")
|
||||
|
||||
# Discover existing sha (update vs create).
|
||||
sha, err = get_existing_sha(args.host, args.owner, args.repo,
|
||||
args.path, args.branch, token)
|
||||
if err is not None:
|
||||
status, body = err
|
||||
print(json.dumps({"uploaded": False, "status": status, "body": body}))
|
||||
return 1
|
||||
|
||||
# Upload (PUT if sha, POST otherwise).
|
||||
new_sha, err = upload(args.host, args.owner, args.repo, args.path,
|
||||
args.branch, args.message, content_b64, sha, token)
|
||||
if err is not None:
|
||||
status, body = err
|
||||
print(json.dumps({"uploaded": False, "status": status, "body": body}))
|
||||
return 1
|
||||
|
||||
print(json.dumps({"uploaded": True, "path": args.path,
|
||||
"sha": new_sha}))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Executable
+118
@@ -0,0 +1,118 @@
|
||||
#!/usr/bin/env python3
|
||||
"""l3b_agent_stub.py — D-008 / D-026 / D-021
|
||||
|
||||
Parses a GitHub/Gitea Issue body by keywords and emits a contract.yaml that
|
||||
selects an L2 stack. This is the agentic (L3B) entry surface: deterministic
|
||||
keyword matching, no external AI APIs.
|
||||
|
||||
D-008 keyword map (priority order — first match wins):
|
||||
gas, price, ingest, data-lake -> l2-commodity-price-feed
|
||||
invoice, billing -> l2-invoice-service
|
||||
analytics, historical, query -> l2-energy-analytics-api
|
||||
regulatory, compliance, reporting, trading
|
||||
-> l2-regulatory-reporting
|
||||
(no match) -> l2-invoice-service (fallback)
|
||||
|
||||
Output contract.yaml (D-021 schema):
|
||||
stack: <mapped L2 name>
|
||||
inputs:
|
||||
environment: dev
|
||||
owner: citizen-developer
|
||||
source: l3b-agent-stub
|
||||
public-ingress: false
|
||||
|
||||
Input:
|
||||
argv[1] = issue body text (or stdin if argv[1] absent/empty)
|
||||
-o <path> = write the contract to a file (default: stdout)
|
||||
Exit:
|
||||
0 on success, 1 on empty input
|
||||
"""
|
||||
import sys
|
||||
|
||||
|
||||
# Ordered keyword groups -> L2 stack mapping (D-008). First match wins.
|
||||
KEYWORD_MAP = [
|
||||
(("gas", "price", "ingest", "data-lake"), "l2-commodity-price-feed"),
|
||||
(("invoice", "billing"), "l2-invoice-service"),
|
||||
(("analytics", "historical", "query"), "l2-energy-analytics-api"),
|
||||
(("regulatory", "compliance", "reporting", "trading"), "l2-regulatory-reporting"),
|
||||
]
|
||||
|
||||
FALLBACK_STACK = "l2-invoice-service"
|
||||
|
||||
|
||||
def map_issue_to_stack(text: str) -> str:
|
||||
lowered = text.lower()
|
||||
for keywords, stack in KEYWORD_MAP:
|
||||
for kw in keywords:
|
||||
if kw in lowered:
|
||||
return stack
|
||||
return FALLBACK_STACK
|
||||
|
||||
|
||||
def render_contract(stack: str) -> str:
|
||||
# Fixed-schema YAML (D-021). Emitted as text (no yaml dependency needed).
|
||||
return (
|
||||
f"stack: {stack}\n"
|
||||
"inputs:\n"
|
||||
" environment: dev\n"
|
||||
" owner: citizen-developer\n"
|
||||
" source: l3b-agent-stub\n"
|
||||
"public-ingress: false\n"
|
||||
)
|
||||
|
||||
|
||||
def read_issue_body(args: list) -> str:
|
||||
"""Read issue body from args[0] (already-stripped argv, no script name)
|
||||
or stdin. Empty -> error."""
|
||||
if len(args) >= 1 and args[0].strip():
|
||||
return args[0]
|
||||
# Fall back to stdin if argv body is absent or empty.
|
||||
if not sys.stdin.isatty():
|
||||
data = sys.stdin.read()
|
||||
if data.strip():
|
||||
return data
|
||||
return ""
|
||||
|
||||
|
||||
def parse_output_flag(argv: list):
|
||||
"""Extract -o <path> from argv (returns (rest, output_path))."""
|
||||
output_path = None
|
||||
rest = []
|
||||
i = 1
|
||||
while i < len(argv):
|
||||
arg = argv[i]
|
||||
if arg == "-o":
|
||||
if i + 1 < len(argv):
|
||||
output_path = argv[i + 1]
|
||||
i += 2
|
||||
continue
|
||||
else:
|
||||
print("l3b_agent_stub: -o requires a path argument", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
rest.append(arg)
|
||||
i += 1
|
||||
return rest, output_path
|
||||
|
||||
|
||||
def main() -> int:
|
||||
rest, output_path = parse_output_flag(sys.argv)
|
||||
body = read_issue_body(rest)
|
||||
if not body.strip():
|
||||
print("l3b_agent_stub: empty issue body (no argv[1] and no stdin)", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
stack = map_issue_to_stack(body)
|
||||
contract = render_contract(stack)
|
||||
|
||||
if output_path:
|
||||
with open(output_path, "w", encoding="utf-8") as fh:
|
||||
fh.write(contract)
|
||||
else:
|
||||
sys.stdout.write(contract)
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Executable
+126
@@ -0,0 +1,126 @@
|
||||
#!/usr/bin/env bash
|
||||
# mock_executor.sh — REQ-06 / D-022
|
||||
#
|
||||
# Reads a contract.yaml, resolves the L2 composition, invokes each L1's
|
||||
# mock_apply.sh in order, and writes state.json to the current working
|
||||
# directory.
|
||||
#
|
||||
# Input: argv[1] = path to a contract.yaml file.
|
||||
# Output:
|
||||
# - stdout: per-L1 progress (echoed from each mock_apply.sh)
|
||||
# - state.json in cwd: {"l2": "...", "l1s": [...], "contract": {...}}
|
||||
# Exit:
|
||||
# 0 if all L1s exit 0; 1 if any L1 exited non-zero (state.json is still
|
||||
# written with the recorded exit codes).
|
||||
set -euo pipefail
|
||||
|
||||
if [[ $# -lt 1 ]]; then
|
||||
echo "usage: mock_executor.sh <contract.yaml>" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
CONTRACT_PATH="$1"
|
||||
|
||||
if [[ ! -f "$CONTRACT_PATH" ]]; then
|
||||
echo "contract not found: $CONTRACT_PATH" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# --- Parse the contract (stack + full contract dict) via python3 + yaml. ---
|
||||
# Emit stack on line 1 and the full contract JSON on line 2, then read both
|
||||
# lines into separate bash variables (so the JSON's internal spaces survive).
|
||||
CONTRACT_PARSED=$(python3 - "$CONTRACT_PATH" <<'PY'
|
||||
import sys, json, yaml
|
||||
path = sys.argv[1]
|
||||
with open(path, "r", encoding="utf-8") as fh:
|
||||
contract = yaml.safe_load(fh)
|
||||
if not isinstance(contract, dict):
|
||||
sys.stderr.write("contract is not a mapping\n")
|
||||
sys.exit(2)
|
||||
stack = contract.get("stack", "")
|
||||
# Use a compact JSON (no spaces) so the single-line contract survives bash
|
||||
# variable capture cleanly.
|
||||
print(stack)
|
||||
print(json.dumps(contract, sort_keys=True, separators=(",", ":")))
|
||||
PY
|
||||
)
|
||||
|
||||
STACK=$(printf '%s\n' "$CONTRACT_PARSED" | sed -n '1p')
|
||||
CONTRACT_JSON=$(printf '%s\n' "$CONTRACT_PARSED" | sed -n '2p')
|
||||
|
||||
if [[ -z "$STACK" ]]; then
|
||||
echo "contract missing 'stack' key" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# --- Resolve the L2 manifest. ---
|
||||
L2_MANIFEST="modules/l2/${STACK}/manifest.yaml"
|
||||
if [[ ! -f "$L2_MANIFEST" ]]; then
|
||||
echo "L2_NOT_FOUND: ${STACK}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# --- Read the L2's l1s: list (ordered names) via python. ---
|
||||
L1_NAMES_JSON=$(python3 - "$L2_MANIFEST" <<'PY'
|
||||
import sys, json, yaml
|
||||
path = sys.argv[1]
|
||||
with open(path, "r", encoding="utf-8") as fh:
|
||||
manifest = yaml.safe_load(fh)
|
||||
l1s = manifest.get("l1s", []) if isinstance(manifest, dict) else []
|
||||
names = [entry.get("name", "") for entry in l1s if isinstance(entry, dict)]
|
||||
print(json.dumps(names))
|
||||
PY
|
||||
)
|
||||
|
||||
# --- Invoke each L1's mock_apply.sh in order, recording exit codes. ---
|
||||
# Build the l1s results array in JSON via python, appending as we go.
|
||||
RESULTS_JSON="[]"
|
||||
|
||||
ALL_OK=0
|
||||
while IFS= read -r L1_NAME; do
|
||||
L1_SCRIPT="modules/l1/${L1_NAME}/mock_apply.sh"
|
||||
if [[ ! -f "$L1_SCRIPT" ]]; then
|
||||
echo "L1_NOT_FOUND: ${L1_NAME}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Capture stdout + exit code. stderr passes through.
|
||||
L1_OUT=$(bash "$L1_SCRIPT")
|
||||
L1_RC=$?
|
||||
|
||||
# Echo the L1's stdout so the pipeline sees the progress lines.
|
||||
printf '%s\n' "$L1_OUT"
|
||||
|
||||
# Record {"name": ..., "applied": true, "exit_code": ...}.
|
||||
RESULTS_JSON=$(python3 - "$RESULTS_JSON" "$L1_NAME" "$L1_RC" <<'PY'
|
||||
import sys, json
|
||||
results = json.loads(sys.argv[1])
|
||||
name = sys.argv[2]
|
||||
rc = int(sys.argv[3])
|
||||
results.append({"name": name, "applied": True, "exit_code": rc})
|
||||
print(json.dumps(results))
|
||||
PY
|
||||
)
|
||||
|
||||
if [[ $L1_RC -ne 0 ]]; then
|
||||
ALL_OK=1
|
||||
fi
|
||||
done < <(python3 -c "import sys, json; print('\n'.join(json.loads(sys.argv[1])))" "$L1_NAMES_JSON")
|
||||
|
||||
# --- Write state.json to the current working directory (D-022). ---
|
||||
python3 - "$RESULTS_JSON" "$STACK" "$CONTRACT_JSON" <<'PY'
|
||||
import sys, json
|
||||
results = json.loads(sys.argv[1])
|
||||
stack = sys.argv[2]
|
||||
contract = json.loads(sys.argv[3])
|
||||
state = {
|
||||
"l2": stack,
|
||||
"l1s": results,
|
||||
"contract": contract,
|
||||
}
|
||||
with open("state.json", "w", encoding="utf-8") as fh:
|
||||
json.dump(state, fh, indent=2)
|
||||
fh.write("\n")
|
||||
PY
|
||||
|
||||
exit "$ALL_OK"
|
||||
Executable
+51
@@ -0,0 +1,51 @@
|
||||
#!/usr/bin/env python3
|
||||
"""policy_checker.py — REQ-07 / D-025
|
||||
|
||||
Reads a contract.yaml and enforces the single Phase-03 policy rule:
|
||||
`public-ingress: true` is forbidden.
|
||||
|
||||
Input: argv[1] = path to a contract.yaml file.
|
||||
Output: stdout "POLICY_PASS" or "POLICY_VIOLATION:PUBLIC_INGRESS"
|
||||
Exit: 0 on pass, 1 on violation.
|
||||
|
||||
Idempotent, no side effects (no file writes). Treats an absent or falsy
|
||||
`public-ingress` key as a pass.
|
||||
"""
|
||||
import sys
|
||||
import yaml
|
||||
|
||||
|
||||
def main() -> int:
|
||||
if len(sys.argv) < 2:
|
||||
print("usage: policy_checker.py <contract.yaml>", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
contract_path = sys.argv[1]
|
||||
|
||||
try:
|
||||
with open(contract_path, "r", encoding="utf-8") as fh:
|
||||
contract = yaml.safe_load(fh)
|
||||
except FileNotFoundError:
|
||||
print(f"contract not found: {contract_path}", file=sys.stderr)
|
||||
return 2
|
||||
except yaml.YAMLError as exc:
|
||||
print(f"invalid yaml: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
# Treat missing/non-mapping as no policy violation.
|
||||
if not isinstance(contract, dict):
|
||||
print("POLICY_PASS")
|
||||
return 0
|
||||
|
||||
public_ingress = contract.get("public-ingress", False)
|
||||
|
||||
if public_ingress is True:
|
||||
print("POLICY_VIOLATION:PUBLIC_INGRESS")
|
||||
return 1
|
||||
|
||||
print("POLICY_PASS")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Executable
+240
@@ -0,0 +1,240 @@
|
||||
#!/usr/bin/env bash
|
||||
# Phase 03 verification script.
|
||||
# Confirms the 4 L2 modules and the 5 core scripts conform to their contracts.
|
||||
#
|
||||
# Usage: scripts/verify_phase03.sh
|
||||
# Exit codes: 0 = all checks passed; 1 = one or more checks failed.
|
||||
|
||||
set -uo pipefail
|
||||
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
cd "$ROOT"
|
||||
|
||||
fail_count=0
|
||||
pass() { printf ' [PASS] %s\n' "$1"; }
|
||||
fail() { printf ' [FAIL] %s\n' "$1"; fail_count=$((fail_count + 1)); }
|
||||
|
||||
# Expected L2 names per REQ-04.
|
||||
EXPECTED_L2S=(
|
||||
l2-invoice-service
|
||||
l2-commodity-price-feed
|
||||
l2-energy-analytics-api
|
||||
l2-regulatory-reporting
|
||||
)
|
||||
|
||||
echo "== Phase 03 verification =="
|
||||
echo "Root: ${ROOT}"
|
||||
echo
|
||||
|
||||
# --- Check 1: exactly 4 L2 folders with the expected names ---
|
||||
echo "-- Check 1: 4 L2 folders with expected names --"
|
||||
actual=$(ls modules/l2/ 2>/dev/null | sort | tr '\n' ' ')
|
||||
expected=$(printf '%s\n' "${EXPECTED_L2S[@]}" | sort | tr '\n' ' ')
|
||||
if [ "$actual" = "$expected" ]; then
|
||||
pass "exactly 4 L2 folders present and named correctly"
|
||||
else
|
||||
fail "L2 folder list mismatch"
|
||||
echo " expected: $expected"
|
||||
echo " actual: $actual"
|
||||
fi
|
||||
|
||||
# --- Check 2: each L2 manifest.yaml validates + references 5 existing L1s ---
|
||||
echo "-- Check 2: L2 manifests reference 5 existing L1s --"
|
||||
l2_validate=$(python3 << 'PYEOF' || true
|
||||
import yaml, glob, os, sys
|
||||
ok = True
|
||||
l1s = set(os.listdir('modules/l1'))
|
||||
for f in sorted(glob.glob('modules/l2/*/manifest.yaml')):
|
||||
d = yaml.safe_load(open(f))
|
||||
folder = os.path.basename(os.path.dirname(f))
|
||||
problems = []
|
||||
if d.get('name') != folder: problems.append(f"name != {folder}")
|
||||
if d.get('kind') != 'l2': problems.append("kind != l2")
|
||||
refs = [x.get('name') for x in d.get('l1s', [])]
|
||||
if len(refs) != 5: problems.append(f"expected 5 l1s, got {len(refs)}")
|
||||
unknown = [r for r in refs if r not in l1s]
|
||||
if unknown: problems.append(f"unknown L1 refs: {unknown}")
|
||||
# each l1 entry must have an inputs: map
|
||||
for x in d.get('l1s', []):
|
||||
if not isinstance(x.get('inputs'), dict): problems.append(f"l1 {x.get('name')} missing inputs map")
|
||||
status = 'OK' if not problems else 'FAIL: ' + '; '.join(problems)
|
||||
print(f' [{status}] {f}')
|
||||
if problems: ok = False
|
||||
sys.exit(0 if ok else 1)
|
||||
PYEOF
|
||||
)
|
||||
echo "$l2_validate"
|
||||
if [ "$l2_validate" = "" ] || echo "$l2_validate" | grep -q FAIL; then
|
||||
if ! echo "$l2_validate" | grep -q PASS; then
|
||||
fail "one or more L2 manifests invalid (see above)"
|
||||
fi
|
||||
else
|
||||
pass "all 4 L2 manifests valid"
|
||||
fi
|
||||
# Re-run for the explicit pass/fail count
|
||||
python3 << 'PYEOF' > /tmp/l2_check.txt 2>&1 || true
|
||||
import yaml, glob, os, sys
|
||||
ok = True
|
||||
l1s = set(os.listdir('modules/l1'))
|
||||
for f in sorted(glob.glob('modules/l2/*/manifest.yaml')):
|
||||
d = yaml.safe_load(open(f))
|
||||
folder = os.path.basename(os.path.dirname(f))
|
||||
if d.get('name') != folder: ok = False
|
||||
if d.get('kind') != 'l2': ok = False
|
||||
refs = [x.get('name') for x in d.get('l1s', [])]
|
||||
if len(refs) != 5: ok = False
|
||||
if any(r not in l1s for r in refs): ok = False
|
||||
for x in d.get('l1s', []):
|
||||
if not isinstance(x.get('inputs'), dict): ok = False
|
||||
sys.exit(0 if ok else 1)
|
||||
PYEOF
|
||||
if [ $? -eq 0 ]; then pass "all 4 L2 manifests pass structural + reference checks"; else fail "L2 manifest structural check"; fi
|
||||
|
||||
# --- Check 3: typecheck (bash -n + py_compile + yaml load) ---
|
||||
echo "-- Check 3: typecheck --"
|
||||
if bash -n scripts/mock_executor.sh; then pass "bash -n mock_executor.sh"; else fail "bash -n mock_executor.sh"; fi
|
||||
if python3 -m py_compile scripts/policy_checker.py scripts/confidence_signal.py scripts/evidence_writer.py scripts/l3b_agent_stub.py 2>/dev/null; then
|
||||
pass "py_compile all 4 python scripts"
|
||||
else
|
||||
fail "py_compile"
|
||||
fi
|
||||
if python3 -c "import yaml, glob; [yaml.safe_load(open(f)) for f in glob.glob('modules/l2/*/manifest.yaml')]" 2>/dev/null; then
|
||||
pass "yaml load all L2 manifests"
|
||||
else
|
||||
fail "yaml load L2 manifests"
|
||||
fi
|
||||
|
||||
# --- Check 4: policy_checker (D-025) ---
|
||||
echo "-- Check 4: policy_checker behavior (D-025) --"
|
||||
WORK="$(mktemp -d)"
|
||||
trap 'rm -rf "$WORK" "$ROOT/tmp_pass_contract.yaml" "$ROOT/tmp_fail_contract.yaml" "$ROOT/state.json" 2>/dev/null || true' EXIT
|
||||
printf 'stack: l2-commodity-price-feed\npublic-ingress: false\n' > "$WORK/pass.yaml"
|
||||
printf 'stack: l2-regulatory-reporting\npublic-ingress: true\n' > "$WORK/fail.yaml"
|
||||
out=$(python3 scripts/policy_checker.py "$WORK/pass.yaml" 2>&1); rc=$?
|
||||
if [ "$out" = "POLICY_PASS" ] && [ "$rc" = "0" ]; then
|
||||
pass "policy_checker pass contract -> POLICY_PASS exit 0"
|
||||
else
|
||||
fail "policy_checker pass contract: got '$out' exit=$rc"
|
||||
fi
|
||||
out=$(python3 scripts/policy_checker.py "$WORK/fail.yaml" 2>&1); rc=$?
|
||||
if [ "$out" = "POLICY_VIOLATION:PUBLIC_INGRESS" ] && [ "$rc" = "1" ]; then
|
||||
pass "policy_checker fail contract -> POLICY_VIOLATION:PUBLIC_INGRESS exit 1"
|
||||
else
|
||||
fail "policy_checker fail contract: got '$out' exit=$rc"
|
||||
fi
|
||||
|
||||
# --- Check 5: confidence_signal (D-024) ---
|
||||
echo "-- Check 5: confidence_signal behavior (D-024) --"
|
||||
out=$(python3 scripts/confidence_signal.py "$WORK/pass.yaml" 2>&1); rc=$?
|
||||
if echo "$out" | grep -q '"score": 0.90' && [ "$rc" = "0" ]; then
|
||||
pass "confidence_signal pass -> score 0.90 exit 0"
|
||||
else
|
||||
fail "confidence_signal pass: got '$out' exit=$rc"
|
||||
fi
|
||||
out=$(python3 scripts/confidence_signal.py "$WORK/fail.yaml" 2>&1); rc=$?
|
||||
if echo "$out" | grep -q '"score": 0.40' && [ "$rc" = "0" ]; then
|
||||
pass "confidence_signal fail -> score 0.40 exit 0"
|
||||
else
|
||||
fail "confidence_signal fail: got '$out' exit=$rc"
|
||||
fi
|
||||
|
||||
# --- Check 6: evidence_writer hash chain (D-023) ---
|
||||
echo "-- Check 6: evidence_writer hash chain (D-023) --"
|
||||
rm -f "$WORK/audit.json"
|
||||
python3 scripts/evidence_writer.py --stage dev --event "dev start" --audit "$WORK/audit.json" > /dev/null
|
||||
python3 scripts/evidence_writer.py --stage qa --event "qa approved" --audit "$WORK/audit.json" > /dev/null
|
||||
python3 scripts/evidence_writer.py --stage prod --event "prod approved" --audit "$WORK/audit.json" > /dev/null
|
||||
chain_ok=$(python3 << PYEOF
|
||||
import json, hashlib, sys
|
||||
try:
|
||||
events = json.load(open("$WORK/audit.json"))
|
||||
assert len(events) == 4, f"expected 4 (genesis + 3), got {len(events)}"
|
||||
assert events[0]['prev_hash'] == 'GENESIS', "genesis prev_hash"
|
||||
for i in range(1, len(events)):
|
||||
assert events[i]['prev_hash'] == events[i-1]['hash'], f"chain break at {i}"
|
||||
e = dict(events[i]); h = e.pop('hash'); e['hash'] = ''
|
||||
canon = json.dumps(e, sort_keys=True, separators=(',',':'))
|
||||
assert hashlib.sha256(canon.encode()).hexdigest() == h, f"hash mismatch at {i}"
|
||||
print("OK")
|
||||
except AssertionError as ex:
|
||||
print(f"FAIL: {ex}")
|
||||
sys.exit(1)
|
||||
PYEOF
|
||||
)
|
||||
if [ "$chain_ok" = "OK" ]; then
|
||||
pass "evidence_writer: 4 events, GENESIS + 3, chain links + hashes valid"
|
||||
else
|
||||
fail "evidence_writer chain: $chain_ok"
|
||||
fi
|
||||
|
||||
# --- Check 7: mock_executor (D-022) ---
|
||||
echo "-- Check 7: mock_executor writes state.json (D-022) --"
|
||||
rm -f "$ROOT/state.json"
|
||||
out=$(bash scripts/mock_executor.sh "$WORK/pass.yaml" 2>&1); rc=$?
|
||||
if [ "$rc" != "0" ]; then
|
||||
fail "mock_executor exit $rc (expected 0)"
|
||||
else
|
||||
me_ok=$(python3 << PYEOF
|
||||
import json, sys
|
||||
try:
|
||||
s = json.load(open("$ROOT/state.json"))
|
||||
assert s['l2'] == 'l2-commodity-price-feed', f"l2 mismatch: {s.get('l2')}"
|
||||
assert 'l1s' in s and len(s['l1s']) == 5, f"expected 5 l1s, got {len(s.get('l1s', []))}"
|
||||
assert all(x['applied'] is True and x['exit_code'] == 0 for x in s['l1s']), "l1 not all applied+0"
|
||||
assert 'contract' in s, "missing contract field"
|
||||
print("OK")
|
||||
except Exception as ex:
|
||||
print(f"FAIL: {ex}")
|
||||
sys.exit(1)
|
||||
PYEOF
|
||||
)
|
||||
if [ "$me_ok" = "OK" ]; then
|
||||
pass "mock_executor: state.json with l2 + 5 l1s (all exit 0) + contract"
|
||||
else
|
||||
fail "mock_executor state.json: $me_ok"
|
||||
fi
|
||||
fi
|
||||
rm -f "$ROOT/state.json"
|
||||
|
||||
# --- Check 8: l3b_agent_stub D-008 keyword map ---
|
||||
echo "-- Check 8: l3b_agent_stub keyword map (D-008) --"
|
||||
act3=$(python3 scripts/l3b_agent_stub.py "We need to ingest natural gas prices from Platts and report on compliance." 2>&1)
|
||||
if echo "$act3" | grep -q 'stack: l2-commodity-price-feed'; then
|
||||
pass "l3b Act 3 example -> l2-commodity-price-feed"
|
||||
else
|
||||
fail "l3b Act 3 example: got '$act3'"
|
||||
fi
|
||||
fallback=$(python3 scripts/l3b_agent_stub.py "please deploy something" 2>&1)
|
||||
if echo "$fallback" | grep -q 'stack: l2-invoice-service'; then
|
||||
pass "l3b fallback (no keywords) -> l2-invoice-service"
|
||||
else
|
||||
fail "l3b fallback: got '$fallback'"
|
||||
fi
|
||||
regulatory=$(python3 scripts/l3b_agent_stub.py "regulatory compliance reporting for trading desk" 2>&1)
|
||||
if echo "$regulatory" | grep -q 'stack: l2-regulatory-reporting'; then
|
||||
pass "l3b regulatory keywords -> l2-regulatory-reporting"
|
||||
else
|
||||
fail "l3b regulatory: got '$regulatory'"
|
||||
fi
|
||||
invoice=$(python3 scripts/l3b_agent_stub.py "monthly invoice and billing reconciliation" 2>&1)
|
||||
if echo "$invoice" | grep -q 'stack: l2-invoice-service'; then
|
||||
pass "l3b invoice keywords -> l2-invoice-service"
|
||||
else
|
||||
fail "l3b invoice: got '$invoice'"
|
||||
fi
|
||||
analytics=$(python3 scripts/l3b_agent_stub.py "historical analytics and query API" 2>&1)
|
||||
if echo "$analytics" | grep -q 'stack: l2-energy-analytics-api'; then
|
||||
pass "l3b analytics keywords -> l2-energy-analytics-api"
|
||||
else
|
||||
fail "l3b analytics: got '$analytics'"
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "== Summary =="
|
||||
if [ "$fail_count" -eq 0 ]; then
|
||||
echo "Phase 03 verification PASSED (4 L2s + 5 core scripts, all checks ok)"
|
||||
exit 0
|
||||
else
|
||||
echo "Phase 03 verification FAILED (${fail_count} check(s) failed)"
|
||||
exit 1
|
||||
fi
|
||||
Executable
+186
@@ -0,0 +1,186 @@
|
||||
#!/usr/bin/env bash
|
||||
# Phase 04 verification script.
|
||||
# Confirms the pipeline workflow + issue trigger + finalize_evidence.py
|
||||
# conform to the Phase 04 plan and the Gitea Actions topology in
|
||||
# ARCHITECTURE.md. Does NOT execute a real Gitea Actions run (act_runner
|
||||
# is not registered in this environment); validates structure + syntax
|
||||
# + a dry-run of finalize_evidence.py against a dead host.
|
||||
#
|
||||
# Usage: scripts/verify_phase04.sh
|
||||
# Exit codes: 0 = all checks passed; 1 = one or more checks failed.
|
||||
|
||||
set -uo pipefail
|
||||
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
cd "$ROOT"
|
||||
|
||||
fail_count=0
|
||||
pass() { printf ' [PASS] %s\n' "$1"; }
|
||||
fail() { printf ' [FAIL] %s\n' "$1"; fail_count=$((fail_count + 1)); }
|
||||
|
||||
echo "== Phase 04 verification =="
|
||||
echo "Root: ${ROOT}"
|
||||
echo
|
||||
|
||||
# --- Check 1: typecheck ---
|
||||
echo "-- Check 1: typecheck --"
|
||||
if bash -n scripts/finalize_evidence.py 2>/dev/null || python3 -m py_compile scripts/finalize_evidence.py 2>/dev/null; then
|
||||
pass "py_compile finalize_evidence.py"
|
||||
else
|
||||
fail "py_compile finalize_evidence.py"
|
||||
fi
|
||||
if python3 -c "import yaml; yaml.safe_load(open('.gitea/workflows/pipeline.yml')); yaml.safe_load(open('contracts-repo/.gitea/workflows/issue-to-contract.yml'))" 2>/dev/null; then
|
||||
pass "yaml load both workflows"
|
||||
else
|
||||
fail "yaml load workflows"
|
||||
fi
|
||||
|
||||
# --- Check 2: pipeline.yml structure ---
|
||||
echo "-- Check 2: pipeline.yml structure (D-027, D-028) --"
|
||||
p_struct=$(python3 << 'PYEOF'
|
||||
import yaml, sys
|
||||
try:
|
||||
d = yaml.safe_load(open('.gitea/workflows/pipeline.yml'))
|
||||
on = d.get('on', d.get(True)) or {}
|
||||
assert 'workflow_dispatch' in on, 'no workflow_dispatch trigger'
|
||||
inputs = on['workflow_dispatch']['inputs']
|
||||
assert set(inputs.keys()) == {'contract-ref', 'approve_qa', 'approve_prod'}, f'inputs: {set(inputs.keys())}'
|
||||
assert inputs['contract-ref']['type'] == 'string', 'contract-ref type'
|
||||
assert inputs['approve_qa']['type'] == 'boolean', 'approve_qa type'
|
||||
assert inputs['approve_prod']['type'] == 'boolean', 'approve_prod type'
|
||||
jobs = d['jobs']
|
||||
assert set(jobs.keys()) == {'dev', 'qa-gate', 'prod-gate', 'finalize'}, f'jobs: {set(jobs.keys())}'
|
||||
dev_if = jobs['dev'].get('if', '')
|
||||
assert 'approve_qa' in dev_if and 'approve_prod' in dev_if, f'dev.if: {dev_if}'
|
||||
qa_if = jobs['qa-gate'].get('if', '')
|
||||
assert 'approve_qa' in qa_if, f'qa-gate.if: {qa_if}'
|
||||
prod_if = jobs['prod-gate'].get('if', '')
|
||||
assert 'approve_prod' in prod_if, f'prod-gate.if: {prod_if}'
|
||||
fin_needs = jobs['finalize'].get('needs', [])
|
||||
assert fin_needs == ['prod-gate'] or fin_needs == 'prod-gate', f'finalize.needs: {fin_needs}'
|
||||
# All jobs runs-on ubuntu-latest
|
||||
for name, job in jobs.items():
|
||||
assert job.get('runs-on') == 'ubuntu-latest', f'{name} runs-on: {job.get("runs-on")}'
|
||||
print('OK')
|
||||
except AssertionError as ex:
|
||||
print(f'FAIL: {ex}')
|
||||
sys.exit(1)
|
||||
except Exception as ex:
|
||||
print(f'FAIL: {ex}')
|
||||
sys.exit(1)
|
||||
PYEOF
|
||||
)
|
||||
if [ "$p_struct" = "OK" ]; then
|
||||
pass "pipeline.yml: 3 inputs + 4 jobs + correct if: conditions + finalize.needs=prod-gate"
|
||||
else
|
||||
fail "pipeline.yml structure: $p_struct"
|
||||
fi
|
||||
|
||||
# --- Check 3: pipeline.yml references core scripts ---
|
||||
echo "-- Check 3: pipeline.yml references core scripts (D-029) --"
|
||||
text=$(cat .gitea/workflows/pipeline.yml)
|
||||
missing=""
|
||||
for ref in policy_checker.py confidence_signal.py mock_executor.sh evidence_writer.py finalize_evidence.py; do
|
||||
if ! echo "$text" | grep -qF "$ref"; then
|
||||
missing="$missing $ref"
|
||||
fi
|
||||
done
|
||||
if [ -z "$missing" ]; then
|
||||
pass "pipeline.yml references all 5 core scripts"
|
||||
else
|
||||
fail "pipeline.yml missing references:$missing"
|
||||
fi
|
||||
# Branch-pin documentation
|
||||
if echo "$text" | grep -q 'milestone/v1.0-initial'; then
|
||||
pass "pipeline.yml documents branch-pin to milestone/v1.0-initial"
|
||||
else
|
||||
fail "pipeline.yml missing branch-pin reference"
|
||||
fi
|
||||
|
||||
# --- Check 4: issue-to-contract.yml structure ---
|
||||
echo "-- Check 4: issue-to-contract.yml structure (D-030) --"
|
||||
i_struct=$(python3 << 'PYEOF'
|
||||
import yaml, sys
|
||||
try:
|
||||
d = yaml.safe_load(open('contracts-repo/.gitea/workflows/issue-to-contract.yml'))
|
||||
on = d.get('on', d.get(True)) or {}
|
||||
assert 'issues' in on, 'no issues trigger'
|
||||
assert on['issues']['types'] == ['opened'], f'types: {on["issues"]["types"]}'
|
||||
assert 'parse-and-trigger' in d['jobs'], 'no parse-and-trigger job'
|
||||
assert d['jobs']['parse-and-trigger'].get('runs-on') == 'ubuntu-latest', 'runs-on'
|
||||
print('OK')
|
||||
except AssertionError as ex:
|
||||
print(f'FAIL: {ex}')
|
||||
sys.exit(1)
|
||||
PYEOF
|
||||
)
|
||||
if [ "$i_struct" = "OK" ]; then
|
||||
pass "issue-to-contract.yml: issues[opened] + parse-and-trigger job"
|
||||
else
|
||||
fail "issue-to-contract.yml structure: $i_struct"
|
||||
fi
|
||||
|
||||
# --- Check 5: issue-to-contract.yml references + dispatch endpoint ---
|
||||
echo "-- Check 5: issue-to-contract.yml references + dispatch (D-014, D-030) --"
|
||||
text=$(cat contracts-repo/.gitea/workflows/issue-to-contract.yml)
|
||||
missing=""
|
||||
for ref in l3b_agent_stub.py 'actions/workflows/pipeline.yml/dispatches' 'contract-ref' 'gitea.event.issue.number' 'GITEA_TOKEN' 'new_branch'; do
|
||||
if ! echo "$text" | grep -qF "$ref"; then
|
||||
missing="$missing $ref"
|
||||
fi
|
||||
done
|
||||
if [ -z "$missing" ]; then
|
||||
pass "issue-to-contract.yml: l3b_agent_stub + dispatch + contract-ref + issue number + token + new_branch"
|
||||
else
|
||||
fail "issue-to-contract.yml missing references:$missing"
|
||||
fi
|
||||
|
||||
# --- Check 6: finalize_evidence.py --help + clean failure ---
|
||||
echo "-- Check 6: finalize_evidence.py CLI + clean failure modes ---"
|
||||
out=$(python3 scripts/finalize_evidence.py --help 2>&1); rc=$?
|
||||
if [ "$rc" = "0" ] && echo "$out" | grep -qi 'usage\|--audit\|--owner'; then
|
||||
pass "finalize_evidence.py --help exits 0 with usage"
|
||||
else
|
||||
fail "finalize_evidence.py --help: rc=$rc"
|
||||
fi
|
||||
|
||||
# Missing audit file (with a fake token so it gets past the env check) → exit 1, no stack trace
|
||||
out=$(ACDL_GITEA_TOKEN=fake python3 scripts/finalize_evidence.py --audit /tmp/definitely_nonexistent_audit.json 2>&1); rc=$?
|
||||
if [ "$rc" = "1" ] && ! echo "$out" | grep -q 'Traceback'; then
|
||||
pass "finalize_evidence.py missing file → exit 1, no stack trace"
|
||||
else
|
||||
fail "finalize_evidence.py missing file: rc=$rc, out='$out'"
|
||||
fi
|
||||
|
||||
# Missing token env (audit file present) → exit 1, no stack trace
|
||||
printf '[]\n' > /tmp/empty_audit.json
|
||||
out=$(env -u ACDL_GITEA_TOKEN python3 scripts/finalize_evidence.py --audit /tmp/empty_audit.json 2>&1); rc=$?
|
||||
if [ "$rc" = "1" ] && ! echo "$out" | grep -q 'Traceback'; then
|
||||
pass "finalize_evidence.py missing token env → exit 1, no stack trace"
|
||||
else
|
||||
fail "finalize_evidence.py missing token: rc=$rc, out='$out'"
|
||||
fi
|
||||
|
||||
# --- Check 7: finalize_evidence.py dry-run against a dead host (clean failure) ---
|
||||
echo "-- Check 7: finalize_evidence.py dry-run against dead host ---"
|
||||
# Use a real audit.json but point at a host that will refuse the connection.
|
||||
printf '[{"seq":0,"ts":"2026-07-21T00:00:00Z","stage":"genesis","event":"init","prev_hash":"GENESIS","hash":"x"}]\n' > /tmp/real_audit.json
|
||||
out=$(ACDL_GITEA_TOKEN=fake GITEA_HOST=http://127.0.0.1:0 python3 scripts/finalize_evidence.py --audit /tmp/real_audit.json --host http://127.0.0.1:0 2>&1); rc=$?
|
||||
if [ "$rc" = "1" ] && ! echo "$out" | grep -q 'Traceback'; then
|
||||
pass "finalize_evidence.py dead host → exit 1, no stack trace (clean API failure)"
|
||||
else
|
||||
fail "finalize_evidence.py dead host: rc=$rc, out='$out'"
|
||||
fi
|
||||
|
||||
# Cleanup
|
||||
rm -f /tmp/empty_audit.json /tmp/real_audit.json
|
||||
|
||||
echo
|
||||
echo "== Summary =="
|
||||
if [ "$fail_count" -eq 0 ]; then
|
||||
echo "Phase 04 verification PASSED (pipeline + issue trigger + finalize helper, all checks ok)"
|
||||
exit 0
|
||||
else
|
||||
echo "Phase 04 verification FAILED (${fail_count} check(s) failed)"
|
||||
exit 1
|
||||
fi
|
||||
Reference in New Issue
Block a user