Jon Chery e050e65158
acdl-ci / Lint (push) Successful in 8s
acdl-ci / Test (push) Successful in 14s
acdl-ci / Platform check-only (offline) (push) Successful in 9s
feat(P19): central pipeline contract + shell reproducibility + output streaming (v1.4.1)
---ci---
project: acdl
phase: 19
milestone: v1.4
status: execute
---

Add declarative pipeline contract (schemas/pipeline.schema.json +
pipelines/ci.yaml) as single source of truth for both Gitea Actions (dev)
and GitHub Actions (production) workflows. Both workflow files are
byte-identical and validated against the contract by 32 new tests.

Add scripts/run_ci.sh for shell reproducibility — mirrors the CI pipeline
locally (lint → test → check-only), exits 0 with 'CI PIPELINE OK'.

Update scripts/run_platform.sh to stream output by default: terraform
init/validate/plan via tee, Checkov compliance results with per-record
severity/rule/pass-fail, and emitted Terraform in --check-only. New
--quiet flag for log-only mode.

Requirements: REQ-43 (central pipeline contract), REQ-44 (shell
reproducibility), REQ-45 (output streaming). 122 tests pass (90 + 32).
2026-07-22 15:10:54 +00:00

ACDL — Agentic Cloud Delivery Platform

Consumers declare intent; the platform delivers safe production deployment through an agentic stack — automatically, safely, and with a complete audit trail. A merged change progresses through lower environments end-to-end without a platform engineer joining a thread; a non-technical consumer ships a production deployment by declaring intent, without authoring a workflow, a configuration file, or a Terraform module.

Status

  • v1.4 (active): central pipeline contract + shell reproducibility + output streaming. A declarative pipeline contract (schemas/pipeline.schema.json + pipelines/ci.yaml) binds the Gitea and GitHub workflows to a single source of truth. scripts/run_ci.sh mirrors the CI pipeline locally. scripts/run_platform.sh streams terraform/checkov output by default. Ship tag v1.4.1.
  • v1.3 (complete, tag v1.3.2): module documentation + thin-composition removal. The L2 composition layer is removed; module READMEs are built out. Testing + CI/CD pipelines (pytest, --check-only, Gitea + GitHub workflows).
  • v1.2 (complete, tag v1.3.0): platform hardening + first real consumer deployment. Harden the v1.1 spike's NFRs, simplify the setup, rewrite the docs, and prove the platform delivers real value by deploying a basic microservice to AWS ECS Fargate end-to-end (terraform apply, dev autonomous).
  • v1.1 (complete, tag v1.2.0): architecture finalization + v1 spike. Finalized the architecture to v1.0 (resolved all 11 open design decisions) and proved the IR commitments hold with one end-to-end spike (l1-s3 + l2-static-asset + Terraform adapter → real terraform plan against AWS). Gitea release id 202.
  • v1.0 demo (complete, archived under demo/, tag v1.1.0): the 30-minute stub-driven executive demo. Preserved as the intent reference; it is not the platform.

How the platform works

The platform is four layers + six cross-cutting concerns, bound by the vision's "Two Consumer Surfaces, One Platform" tenet: technical developers (L3A) and non-technical consumers (L3B) converge on the same contract schema, the same policy envelope, and the same evidence stream.

The v1.1 spike flow (end-to-end)

contracts/spike.yaml
        │  (contract schema validation)
        ▼
acdl_platform/contract_resolver.py  ──▶ Target Stack IR (JSON)
        │  (IR schema validation)
        ▼
adapters/terraform/adapter.py  ──▶ terraform/spike/{main,terraform,providers}.tf
        │  (the only substrate-specific code)
        ▼
terraform plan  (real AWS, via the rotated spike key — D-039/D-047)
        │
        ▼
adapters/terraform/policy/checkov_adapter.py  ──▶ PolicyCheckResult (JSON list)
        │  (normalized, engine-agnostic)
        ▼
acdl_platform/confidence_signal.py  ──▶ { score, band, perInput, reasonCodes }
        │  (6 inputs: policy, validation, freshness, source, history, nfrs)
        ▼
acdl_platform/outbox_writer.py  ──▶ DynamoDB outbox (acdl-outbox)
        │  (hash-chained evidence event)
        ▼
acdl-evidence timeline (acdl-evidence repo, raw-file served)

The spike validates the architecture's claim that the IR-shaped commitments do not require a polyglot mess: the adapter is the only substrate-specific code. modules-ir/, schemas/, contracts/, acdl_platform/confidence_signal.py, acdl_platform/contract_resolver.py, and acdl_platform/outbox_writer.py are all substrate-agnostic (no aws_s3_bucket / aws_ Terraform terms).

What's different in v1.2

v1.2 extends the spike to a real, simpler, better-documented platform that deploys a microservice to ECS Fargate:

  • Six new IR-typed L1s: l1-vpc, l1-ecs-cluster, l1-ecs-service, l1-iam-role, l1-alb, l1-ecr.
  • One new L2 thin-composition: l2-microservice (references the six L1s).
  • terraform apply (dev, autonomous per §10, confidence ≥ 0.50) — real provisioning, not just plan.
  • A new consumer repo acdl-consumer-microservice with a basic HTTP container + Dockerfile + ECR push + contract submission.
  • One scripts/run_platform.sh (consolidated from the v1.1 spike scripts).
  • NFR hardening: least-privilege IAM (expanded for ECS), idempotent bootstrap, proper error handling, P1-1 redaction.

How to run

Prerequisites

  • AWS account + the rotated spike key in .env.secrets (see scripts/rotate_spike_key.sh; the bootstrap root key was deactivated per D-034 closure).
  • terraform (pin 1.9.*), checkov (pin >=3.2,<4), python3 + boto3
    • jsonschema.

Run the platform pipeline end-to-end

# 1. Bootstrap the AWS state backend + spike IAM user (one-time, idempotent)
#    (requires the bootstrap root key in env — now deactivated; skip if
#    the state bucket + acdl-spike-runner already exist)
ACDL_BOOTSTRAP_AWS_ACCESS_KEY_ID=... ACDL_BOOTSTRAP_AWS_SECRET_ACCESS_KEY=... \
  python3 terraform/bootstrap/create_state_backend.py
ACDL_BOOTSTRAP_AWS_ACCESS_KEY_ID=... ACDL_BOOTSTRAP_AWS_SECRET_ACCESS_KEY=... \
  python3 terraform/bootstrap/create_iam_user.py  # prints the initial key

# 2. Rotate the spike key (writes .env.secrets, gitignored)
ACDL_BOOTSTRAP_AWS_ACCESS_KEY_ID=... ACDL_BOOTSTRAP_AWS_SECRET_ACCESS_KEY=... \
  bash scripts/rotate_spike_key.sh

# 3. Run the full platform pipeline (IR -> adapter -> plan -> Checkov ->
#    confidence -> outbox). Output is streamed to stdout by default.
bash scripts/run_platform.sh
# Expected: "=== PLATFORM E2E OK ==="

# Or plan-only (IR -> adapter -> terraform plan; no Checkov/outbox):
bash scripts/run_platform.sh --plan-only

# Add --quiet to suppress streaming (output to log files only):
bash scripts/run_platform.sh --quiet

Test the platform (offline, no AWS required)

# Install test dependencies
pip install -r requirements-test.txt

# Run the test suite (122 tests, all offline — uses moto for DynamoDB mocking)
python3 -m pytest tests/ -v

# Run the platform in check-only mode (offline — no AWS, no Checkov, no outbox)
# Streams the emitted Terraform to stdout by default; --quiet suppresses it
bash scripts/run_platform.sh --check-only
# Expected: "=== PLATFORM CHECK OK ==="

# Reproduce the full CI pipeline locally (lint → test → check-only)
bash scripts/run_ci.sh
# Expected: "=== CI PIPELINE OK ==="

CI/CD pipelines

The CI/CD pipeline is defined by a central pipeline contract — a declarative YAML instance (pipelines/ci.yaml) validated against a JSON Schema (schemas/pipeline.schema.json). Both forge workflows implement the same contract:

  • .gitea/workflows/ci.yml — Gitea Actions (dev environment)
  • .github/workflows/ci.yml — GitHub Actions (production)

Both workflow files are byte-identical — the only difference is the forge runtime. Both run three stages: lint (py_compile), test (pytest), and check-only (run_platform.sh --check-only). Both trigger on push to main and on pull requests. A test (tests/test_pipeline_contract.py) validates that both workflows conform to the contract.

scripts/run_ci.sh mirrors the CI pipeline locally — running the same three stages in sequence. This makes the pipeline fully reproducible from the shell, not just in CI:

bash scripts/run_ci.sh           # run all 3 stages (lint, test, check-only)
bash scripts/run_ci.sh --quiet    # suppress per-stage banners

Output streaming (run_platform.sh)

scripts/run_platform.sh streams output by default so the user can see what the platform is doing:

  • --check-only: streams the emitted Terraform file content to stdout
  • --plan-only and full mode: streams terraform init, terraform validate, and terraform plan output via tee (visible and logged)
  • Full mode: prints Checkov compliance results and each PolicyCheckResult record with severity, rule ID, and pass/fail status

A --quiet flag suppresses streaming (output to log files only) for backwards-compatible log-only mode.

Re-run the archived v1.0 demo (stubs only, no AWS)

bash demo/scripts/run_demo.sh --no-upload

The demo deck is at demo/ACDL_DEMO.md. It runs entirely on local stubs — no AWS, no AI — and shows intent and safety behavior rather than provisioning real cloud resources.

Repository layout

Path Purpose Status
acdl_platform/ Platform code: confidence signal, outbox writer, separation of duties, HITL/ledger designs v1.1 complete; v1.3 removes contract_resolver
schemas/ JSON Schemas: IR, PolicyCheckResult, pipeline contract (draft 2020-12) v1.1 complete; v1.4 adds pipeline schema
pipelines/ Central pipeline contract: ci.yaml (YAML instance validated against schemas/pipeline.schema.json) v1.4
adapters/ Substrate adapters — Terraform adapter (the only substrate-specific code per §12) + Checkov policy adapter v1.1 complete; v1.2 expands TYPE_MAP
terraform/ State backend (S3 + DynamoDB) + spike TF (terraform/spike/) + bootstrap scripts (terraform/bootstrap/) v1.1 complete; v1.2 adds ECS apply
modules-ir/ IR-typed L1/L2 modules + registry.json. v1.1: l1-s3. v1.2: + 6 ECS L1s. v1.3: L2 removed (placeholders) v1.3
scripts/ Platform run script (run_platform.sh with --check-only/--plan-only/--quiet), CI pipeline script (run_ci.sh), verify scripts, key rotation v1.4
tests/ Pytest suite (122 tests, all offline — adapter, confidence signal, checkov adapter, outbox writer, pipeline contract, streaming) v1.4
demo/ Archived v1.0 executive demo (tag v1.1.0); runs locally via demo/scripts/run_demo.sh --no-upload complete (archived)
.ciagent/ CIAgent metadata (config, project, architecture, requirements, roadmap, personas, plans, research, verify, review, audit) active
docs/ Upstream vision + architecture sources (vision.md, architecture.md) active

Environments

Environment Autonomy Gate Status
dev Full autonomy (no HITL) Confidence ≥ 0.50 v1.1 spike (plan); v1.2 (apply)
qa Held for attestation QA HITL + confidence ≥ 0.75 v1.3+
prod Held for attestation SRE HITL + confidence ≥ 0.90 v1.3+
dr Held for attestation SRE HITL + confidence ≥ 0.95 + dr-drill v1.3+

Staging does not exist (Path A locked).

Credentials

Long-lived AWS credentials are forbidden (§12.5). The v1.1 spike uses a temporary long-lived key once to bootstrap (waiver D-034, now closed — the root key was deactivated by the user), then rotates the spike key per-run via scripts/rotate_spike_key.sh (waiver D-039, extended for v1.2 as D-047). Real OIDC federation is deferred to v1.3+, blocked on go-gitea/gitea#36988 (still open as of 2026-07-21).

S
Description
Nova — The New Dawn of DevSecOps. Autonomous infrastructure delivery: consumers declare intent, the platform ships safely with an immutable audit trail.
Readme 69 MiB
Languages
Python 87.2%
Shell 8.6%
HCL 4.2%