acdl/.ciagent/ARCHITECTURE.md

ACDL — Architecture (initial)

Initial architecture for the ACDL demo. May be incomplete; refined at phase boundaries.

Overview

The demo is a three-repo, stub-driven system that simulates an autonomous cloud delivery platform. No real cloud or AI is used; every "infrastructure" action is a bash/Python stub that emits structured evidence. The platform is driven by either a developer-supplied contract.yaml (L3A) or a natural-language GitHub Issue parsed by a keyword script (L3B), then flows through an autonomous Dev stage, manual QA and Prod approval gates, and finally publishes a hash-chained audit trail to a Pages site.

                 ┌──────────────── acdl-contracts ─────────────────┐
  Developer ───▶ │ commit contract.yaml      Issue (NL intent)     │
                 └────────────┬───────────────────┬────────────────┘
                              │ (push)            │ (issue opened)
                              ▼                   ▼
                    ┌─────────────────┐  ┌──────────────────────┐
                    │ reusable        │  │ issue workflow →      │
                    │ pipeline        │  │ l3b_agent_stub.py →   │
                    │ (acdl repo)     │  │ contract.yaml → push  │
                    └────────┬────────┘  └──────────────────────┘
                             │
        ┌────────────────────┼────────────────────┐
        ▼                    ▼                    ▼
  Dev (autonomous)     QA (approval)        Prod (approval)
  mock_executor.sh     environment gate     environment gate
  policy_checker.py
  confidence_signal.py
        │
        ▼
  evidence_writer.py ──▶ audit.json (hash-chained) ──▶ acdl-evidence
                                                          │
                                                          ▼
                                                   index.html (Pages)
                                                   timeline UI

Components

Name	Description	Boundaries	Depends On
acdl repo	Platform meta repo: reusable workflows, L1/L2 stub modules, core scripts	Owns workflows + stubs; does not hold contracts or evidence	—
L1 modules	Single-purpose infra primitives (EKS Fargate, IAM, Lambda, API Gateway, EventBridge, SQS, S3, CloudWatch)	One folder per L1; manifest.yaml + mock_apply.sh; do not compose with other L1s	acdl repo
L2 modules	Composed stacks (invoice, commodity-price-feed, energy-analytics-api, regulatory-reporting)	Reference L1s by name; max depth 5; expressed as a composition manifest	L1 modules
mock_executor.sh	Reads an L2 composition, invokes each L1 mock_apply.sh, writes state.json	Bash; reads L2 manifest + L1 manifests	L1/L2 modules
policy_checker.py	Reads contract.yaml; fails on forbidden keys (e.g. public-ingress: true)	Python; emits POLICY_VIOLATION:<REASON> or pass	contract.yaml
confidence_signal.py	Base 0.90; on policy failure drops to 0.40 and echoes reason	Python; calls policy_checker	policy_checker.py
evidence_writer.py	Appends an event to audit.json, links to previous event via SHA-256 chain	Python; canonical-JSON hashing	audit.json
l3b_agent_stub.py	Parses Issue text by keywords, emits contract.yaml	Python keyword map; no external APIs	contract.yaml schema
acdl-contracts repo	Developer + agentic entry surface; holds contracts + issue workflow	Triggers main pipeline on push	acdl reusable workflow
acdl-evidence repo	Pages host for audit.json + index.html timeline	Read-only for the pipeline; written at finalize stage	evidence_writer.py output
Reusable pipeline workflow	Dev → QA → Prod → Finalize stages with environment gates	Gitea Actions; calls core scripts	All core scripts
evidence-ui/index.html	Vanilla-JS timeline UI (Phase 05); fetches ./audit.json and renders events	Single HTML file with inline CSS+JS; no frameworks (REQ-14)	audit.json on acdl-evidence main
scripts/run_demo.sh	Phase 05 dry-run simulation of the 4 demo acts; calls core scripts + writes evidence + uploads audit.json + index.html to acdl-evidence	Bash; uses evidence_writer.py + finalize_evidence.py + the file-contents API	All Phase 03/04 artifacts

Phase 04 pipeline topology (research)

Gitea Actions limitations (confirmed via research, supersedes any GitHub-Actions assumptions):

• actions/upload-artifact@v3 / download-artifact@v3 work; v4 is NOT supported by act_runner.
• Artifacts are scoped to a single workflow run; re-dispatch starts a new run, so artifacts do NOT survive between dispatches.
• workflow_dispatch API: POST /api/v1/repos/{owner}/{repo}/actions/workflows/{filename}.yml/dispatches with body { "ref": "<branch>", "inputs": {...} }.
• on: workflow_call + uses: <owner>/<repo>/.gitea/workflows/<file>@<ref> works; pin to @milestone/v1.0-initial.
• actions/checkout@v4 supports cross-repo (pass repository: + ref: + token: ${{ secrets.GITEA_TOKEN }} for private repos).
• File-contents API: POST to create (201), PUT to update (must include current sha, obtained via GET).
• ${{ secrets.GITEA_TOKEN }} is a manually-created PAT secret on the acdl + acdl-contracts repos; the auto-injected token is current-repo only and cannot cross-repo.
• No native approval-gate UI; gates are workflow_dispatch inputs (approve_qa, approve_prod).

Approval-gate + state-persistence approach (D-027, D-028 refined)

Because re-dispatch starts a new run and artifacts do not survive:

1. The pipeline workflow has workflow_dispatch inputs:

   • contract-ref (string; default main) — the ref on acdl-contracts carrying the contract.
   • approve_qa (boolean; default false) — the human sets this to true to advance past QA.
   • approve_prod (boolean; default false) — the human sets this to true to advance past Prod.

2. Each stage job (dev, qa-gate, prod-gate, finalize) writes its evidence to acdl-evidence via the file-contents API (PUT audit.json with the new event appended). This is the persistent state across re-dispatches.

3. Dev stage (always runs on dispatch): check out acdl + acdl-contracts@<contract-ref>, run policy_checker.py + confidence_signal.py; if score < 0.50, write a dev_rejected evidence event and exit 1 (Act 4). Otherwise run mock_executor.sh, write a dev_applied evidence event, and exit 0. The run ends here.

4. QA gate (next dispatch with approve_qa=true): check out, run evidence_writer.py --stage qa --event "qa approved", commit updated audit.json to acdl-evidence. Exit 0. The run ends.

5. Prod gate (next dispatch with approve_prod=true): same as QA but --stage prod.

6. Finalize (same dispatch as Prod, chained via needs: prod-gate): write the finalize evidence event, commit final audit.json to acdl-evidence. The raw URL now serves the updated timeline.

Because each stage is a separate dispatch, the workflow file uses if: conditions on each job:

• dev runs when inputs.approve_qa != true && inputs.approve_prod != true (the initial dispatch).
• qa-gate runs when inputs.approve_qa == true && inputs.approve_prod != true.
• prod-gate runs when inputs.approve_prod == true.
• finalize runs after prod-gate (needs: prod-gate).

This means a full pipeline = 3 dispatches (initial, qa-approve, prod-approve). The human drives each via the Gitea UI or the dispatch API.

Phase 05 dry-run + UI (research)

Phase 05 has no act_runner available in this environment, so the "dry run" is a local bash simulation (scripts/run_demo.sh) that produces the same audit.json shape a real pipeline run would, then uploads it (plus the UI) to acdl-evidence via the file-contents API. The simulation covers:

• Act 1 — Friction: a single evidence event "manual 2-week deployment (legacy process)" at stage: dev with a red-colored timeline marker.
• Act 2 — Developer Self-Service: l2-commodity-price-feed contract, full pipeline (dev → qa → prod → finalize), 4 evidence events.
• Act 3 — Citizen Developer: Issue body fed to l3b_agent_stub.py, generates the same l2-commodity-price-feed contract, identical pipeline, 4 evidence events.
• Act 4 — Safety Net: l2-regulatory-reporting contract with public-ingress: true, dev rejects (confidence 0.40 < 0.50), 1 evidence event "dev rejected: POLICY_VIOLATION:PUBLIC_INGRESS".

The audit.json after run_demo.sh contains the genesis + all act events (typically ~14 events). The UI fetches ./audit.json and renders a vertical timeline with stage-colored markers and a per-event hash preview.

UI rendering contract (D-032, D-033)

evidence-ui/index.html:

• Single file, inline CSS + JS, no external resources.
• Fetches ./audit.json (relative URL; works against any raw-URL origin).
• Renders events as a vertical timeline; each event card shows seq, ts, stage (color-coded: dev blue, qa yellow, prod orange, finalize green, genesis gray, rejected events red), event text, and a 12-char hash preview (hash.slice(0, 12)…).
• Handles fetch failure with a "No audit data yet" message.
• Refresh button to re-fetch.

Data Flow

1. A contract.yaml arrives either by direct push (L3A) or by the issue workflow running l3b_agent_stub.py (L3B).
2. Push to acdl-contracts triggers the reusable pipeline in the acdl repo.
3. Dev stage: policy_checker.py validates the contract; mock_executor.sh applies the L2 composition's L1s; confidence_signal.py computes the score; evidence_writer.py records each step. If score < 0.50, the stage fails and evidence records the rejection.
4. QA stage: the workflow pauses on the qa environment; a human approves.
5. Prod stage: same gate on the prod environment.
6. Finalize: the workflow commits the updated audit.json to acdl-evidence; Pages republishes index.html, which fetches and renders the timeline.

Build Order

1. Repo scaffolding: create acdl-contracts and acdl-evidence in the org; seed acdl directory layout.
2. L1 modules (8 stubs).
3. L2 modules (4 compositions).
4. Core scripts (mock_executor.sh, policy_checker.py, confidence_signal.py, evidence_writer.py, l3b_agent_stub.py).
5. Reusable pipeline workflow (Dev → QA → Prod → Finalize) + environment gates.
6. Issue-triggered L3B workflow in acdl-contracts.
7. Evidence UI (index.html + Pages config).
8. Demo dry-run + the four scripted acts.

Gitea API Surface (Phase 01 research)

Authoritative findings from the Gitea docs (added in RESEARCH; supersedes any GitHub-Pages / GitHub-Environments assumptions carried over from the spec):

Capability	Gitea support	ACDL approach
Org-scoped repo create	POST /api/v1/orgs/{org}/repos (CreateRepoOption)	Used to create acdl-contracts + acdl-evidence
Native Pages	None (no [pages] config section)	Serve acdl-evidence via raw file URLs: https://git.cloudinit.dev/continuous-intelligence/acdl-evidence/raw/branch/main/index.html; index.html fetches audit.json from the same raw path. Requires [cors] ENABLED=true on the server if the UI is loaded cross-origin.
Environments API	None; jobs.<id>.environment is ignored by act_runner	Model QA/Prod gates as workflow_dispatch approval inputs (D-004 / D-013); optionally create qa and prod branches as a visible stand-in
repository_dispatch trigger	Not supported	Cross-repo trigger via workflow_dispatch API: POST /api/v1/repos/{owner}/{repo}/actions/workflows/{workflow_id}/dispatches called from a step using $GITEA_TOKEN
Reusable workflows (workflow_call)	Supported	acdl/.gitea/workflows/pipeline.yml called via uses: continuous-intelligence/acdl/.gitea/workflows/pipeline.yml@milestone/v1.0-initial
workflow_dispatch	Supported (trigger + API)	Used for the manual-approval fallback and the issue workflow's cross-repo trigger
issues.opened trigger	Supported	Drives the L3B issue-trigger workflow in acdl-contracts
act_runner labels	Single label only (runs-on: ubuntu-latest)	All workflows use runs-on: ubuntu-latest
Context	${{ gitea.* }} and ${{ github.* }} both work	Workflows use gitea.* for clarity

Branch pinning rule

The reusable workflow in the acdl repo lives on milestone/v1.0-initial (that is the repo's default branch). uses: references from acdl-contracts must pin to @milestone/v1.0-initial, not @main (the acdl repo has no main branch). The new repos acdl-contracts and acdl-evidence use default_branch: "main" (D-015) so their default branch exists immediately for pushes.

Default verification toolchain

There is no package.json; ACDL is bash + python stubs. The verification gate substitutes bash -n and python -m py_compile for npm run typecheck, and per-phase scripts/verify_phaseNN.sh for npm test. npm run build is a no-op (no build step). See PERSONAS.md / VERIFICATION note.

L1 module schema (Phase 02 research)

Each L1 module lives at modules/l1/<name>/ with exactly two files:

• manifest.yaml — declares the L1's identity + a flat inputs: map. Schema (D-017):

  name: l1-eks-fargate        # matches the folder name
  kind: l1                    # literal "l1"; substrate-agnostic
  description: <one-line>
  inputs:
    <key>:
      description: <one-line>
      type: string            # only "string" allowed (flat, max-depth-1)
  

• mock_apply.sh — uniform stub per D-007 + D-018:

  #!/usr/bin/env bash
  set -euo pipefail
  echo "[L1: <name>] applying..."
  sleep 1
  echo "[L1: <name>] OK"
  exit 0
  

  mock_apply.sh does NOT read input values; the manifest is for traceability and for Phase 03's mock_executor.sh to enumerate the L1s in an L2.

L1 list (fixed per REQ-02 / D-019)

Folder	Description
l1-eks-fargate	Serverless container compute substrate
l1-iam-role	Identity and access role primitive
l1-lambda	Event-driven function primitive
l1-api-gateway	HTTP routing primitive
l1-eventbridge	Event bus primitive
l1-sqs	Queue primitive
l1-s3	Object store primitive
l1-cloudwatch	Observability primitive

L1 modules are single-purpose, substrate-agnostic, max-depth-1 (per PROJECT.md Constraints). They do not compose with other L1s.

L2 module schema + core scripts (Phase 03 research)

L2 manifest.yaml schema (D-020)

name: l2-commodity-price-feed      # matches the folder name
kind: l2                            # literal "l2"
description: <one-line>
l1s:                                # ordered list of L1 references
  - name: l1-eks-fargate            # MUST match an existing L1 folder name
    inputs:
      cluster_name: price-feed-cluster
      region: us-east-1
      cpu_arch: arm64
  - name: l1-lambda
    inputs:
      function_name: price-ingest
      runtime: python3.11
      handler: index.handler
  # ... up to 5 L1 references per L2 (max-depth-5 per REQ-05; L2->L1 is depth 1)

L2s reference L1s by name only (no path); mock_executor.sh resolves the name to modules/l1/<name>/.

L2 list (fixed per REQ-04)

Folder	Description	L1s (per S&P Global Energy / Platts use cases)
l2-invoice-service	Billing + invoicing microservice	l1-eks-fargate, l1-iam-role, l1-lambda, l1-sqs, l1-s3
l2-commodity-price-feed	Real-time price ingestion	l1-eks-fargate, l1-lambda, l1-api-gateway, l1-eventbridge, l1-s3
l2-energy-analytics-api	Historical query API	l1-eks-fargate, l1-api-gateway, l1-lambda, l1-s3, l1-cloudwatch
l2-regulatory-reporting	Compliance + reporting	l1-eks-fargate, l1-iam-role, l1-lambda, l1-sqs, l1-s3

Each L2 references exactly 5 L1s (within the max-depth-5 constraint; L2→L1 is depth 1, so depth-5 is generous but the spec caps composition depth at 5 — the count is 5 to demonstrate a realistic composed stack).

contract.yaml schema (D-021)

stack: l2-commodity-price-feed      # MUST match an existing L2 folder name
inputs:                              # top-level params for the L2 (optional)
  environment: dev
  owner: platform-team
public-ingress: false                # bool; true triggers POLICY_VIOLATION:PUBLIC_INGRESS

The public-ingress key is the only policy-enforced field in Phase 03. Phase 04's pipeline reads contract.yaml, runs policy_checker.py, then mock_executor.sh to apply the L2.

state.json shape (D-022)

mock_executor.sh writes state.json to its working directory:

{
  "l2": "l2-commodity-price-feed",
  "l1s": [
    {"name": "l1-eks-fargate", "applied": true, "exit_code": 0},
    {"name": "l1-lambda", "applied": true, "exit_code": 0},
    ...
  ],
  "contract": {
    "stack": "l2-commodity-price-feed",
    "inputs": {...},
    "public-ingress": false
  }
}

audit.json event + hash chain (D-023)

audit.json is a JSON array of event objects. evidence_writer.py appends one event per call. Hash chain:

1. Construct the event dict with hash set to empty string.
2. Serialize via json.dumps(event, sort_keys=True, separators=(",", ":")) — canonical JSON (deterministic key order, no whitespace).
3. Compute hash = sha256(canonical_json.encode("utf-8")).hexdigest().
4. Set event["hash"] = hash.
5. Append to audit.json.

Genesis event (when audit.json is empty or missing):

{
  "seq": 0,
  "ts": "2026-07-21T13:00:00Z",
  "stage": "genesis",
  "event": "audit log initialized",
  "prev_hash": "GENESIS",
  "hash": "<sha256 of the canonical json of this event with hash empty>"
}

Subsequent events: seq = prev.seq + 1, prev_hash = prev.hash.

Core script I/O contracts

Script	Input	Output	Exit
mock_executor.sh	<contract.yaml path> (argv[1]); reads L2 manifest from modules/l2/<contract.stack>/manifest.yaml	writes state.json to cwd; prints per-L1 progress	0 on all-L1s-pass; non-zero on any L1 failure
policy_checker.py	<contract.yaml path> (argv[1])	stdout: POLICY_PASS or POLICY_VIOLATION:PUBLIC_INGRESS	0 on pass; 1 on violation
confidence_signal.py	<contract.yaml path> (argv[1]); calls policy_checker	stdout: `{"score": 0.90	0.40, "reason": "..."}`
evidence_writer.py	argv: `--stage <dev	qa	prod
l3b_agent_stub.py	argv[1] = issue body text (or stdin if no argv); optional -o <path> (default stdout)	writes a contract.yaml (D-021 schema) with stack set by the D-008 keyword map	0 on success; 1 on empty input