acdl/README.md

ACDL — Agentic Cloud Delivery Platform

Consumers declare intent; the platform delivers safe production deployment through an agentic stack — automatically, safely, and with a complete audit trail. A merged change progresses through lower environments end-to-end without a platform engineer joining a thread; a non-technical consumer ships a production deployment by declaring intent, without authoring a workflow, a configuration file, or an infrastructure module.

• Consumer guide: docs/consumer-guide.md
• Modules: docs/modules/
• Contracts: docs/contracts/
• Pipeline: docs/pipeline/
• Versioning: docs/pipeline/versioning.md
• Environments: docs/environments/
• Architecture: docs/architecture.md
• Vision: docs/vision.md

Repository roles

There are two kinds of repository in the ACDL model:

• Platform repo (this one). This is the source code of the platform. It owns modules/, adapters/, core/, schemas/, pipelines/, scripts/, and the reusable workflow files. Platform engineers work here. A consumer never clones it.

• Consumer repo (yours). A consumer repo contains only:

  1. Its application code — the service or site being deployed.
  2. One or more contracts — small YAML files at .acdl/contract.yml that declare infrastructure (one or more modules by name + version), select an environment, and supply module-specific inputs.
  3. One or more CI definitions — thin .github/workflows/*.yml files that uses: the central reusable deploy workflow, pointing at the appropriate environment + contract.

  The consumer does not write infrastructure modules, workflow YAML beyond the thin uses: wrapper, or adapter code — they write a contract YAML file and the platform does the rest.

The rest of this README describes the platform repo (how the platform works, how to run it locally, how it's laid out). If you are a consumer, jump to the Consumer guide.

Features

A referenceable list of what the platform provides today, for consumers and platform engineers alike:

• Contract-driven deploys — a consumer writes a YAML contract; the platform resolves it to a stack, compiles it, and deploys it.
• Reusable versioned deploy workflow — consumer repos uses: a versioned central workflow; no platform code is cloned by the consumer.
• Module catalog — primitives (single resources) and modules (patterns of primitives) with self-documented inputs/outputs. See docs/modules/.
• Zero-trust credentials — OIDC federation + attribute-based authorization (ABAC) by default; no long-lived keys in consumer repos.
• Security + policy checks — a security-check stage and a policy-check stage run before any infrastructure is created.
• Confidence signal — a computed, explainable score gates promotion.
• Evidence outbox — every deployment writes a hash-chained evidence event to an audit outbox.
• Shell reproducibility — scripts/run_ci.sh mirrors the CI pipeline locally; scripts/run_platform.sh --check-only runs offline.
• Platform-managed environments — consumers provide no AWS account, VPC, subnet, or state bucket; the platform manages environments. See docs/environments/.
• Central pipeline contract — a declarative YAML instance is the single source of truth for both the CI and deploy workflows.

Roadmap

Planned future features (no dates; tracked in the internal roadmap):

• Dynamic module creation from a contract — an agentic flow where a consumer creates a module directly from the contract file (the "composition" mechanism, redesigned).
• Compliance milestone — per-module compliance extension points (GDPR, SOX, SOC2, DORA) wired into the pipeline.
• Additional engine adapters — beyond the Terraform adapter.
• Environment self-service — a consumer-facing flow to request and provision a new platform-managed environment (today it is a platform-team action).
• HITL gates for qa / prod / dr — human attestation + higher confidence thresholds for higher environments.
• OIDC for all platform runners — zero-trust credentials everywhere.

How the platform works

The platform is four layers + six cross-cutting concerns, bound by the vision's "Two Consumer Surfaces, One Platform" tenet: consumers declare intent via a contract; the platform delivers the deployment through the same contract schema, the same policy envelope, and the same evidence stream.

Consumers have their own repos and consume ACDL by writing a contract that declares infrastructure. A consumer declares a contract (id + name + environment + infrastructure); the platform resolves it to a stack instance, compiles it, runs security + policy checks, computes a confidence signal, writes an evidence event to the audit outbox, and applies the infrastructure.

The platform flow (end-to-end)

flowchart TD
    A["consumer contract<br/>(id + name + environment + infrastructure)"] --> B
    B["schema validation<br/>(contract schema)"] --> C
    C["resolve to Target Stack<br/>(contract resolver)"] --> D
    D["security checks<br/>(adapter)"] --> E
    E["infrastructure plan<br/>(adapter compiles the stack)"] --> F
    F["policy checks<br/>(adapter -&gt; PolicyCheckResult records)"] --> G
    G["confidence signal<br/>(6 inputs: policy, validation,<br/>freshness, source, history, NFRs)"] --> H
    H["evidence event<br/>(hash-chained, to the audit outbox)"] --> I
    I["infrastructure apply<br/>(dev only, autonomous)"]

The platform validates the architecture's claim that the stack commitments do not require a polyglot mess: the adapter is the only engine-specific code. modules/, schemas/, contracts/, core/confidence_signal.py, core/contract_resolver.py, and core/outbox_writer.py are all engine-agnostic (no aws_s3_bucket / aws_ infrastructure terms).

How to run

Prerequisites

These prerequisites are for running the platform repo locally. A consumer does not need any of these — see the Consumer guide for the consumer happy path.

• A platform-managed environment (see docs/environments/). For local testing, core/environments/dev.json is provided as the sample.
• AWS credentials for the dev environment (in .env.secrets, gitignored; see Credentials & zero-trust).
• terraform (pin 1.9.*), checkov (pin >=3.2,<4), python3 + boto3
  • jsonschema.

Run the platform pipeline end-to-end

# 1. Bootstrap the AWS state backend + runner IAM user (one-time, idempotent)
#    (requires the bootstrap root key in env — skip if the state bucket +
#    acdl-spike-runner already exist)
ACDL_BOOTSTRAP_AWS_ACCESS_KEY_ID=... ACDL_BOOTSTRAP_AWS_SECRET_ACCESS_KEY=... \
  python3 terraform/bootstrap/create_state_backend.py
ACDL_BOOTSTRAP_AWS_ACCESS_KEY_ID=... ACDL_BOOTSTRAP_AWS_SECRET_ACCESS_KEY=... \
  python3 terraform/bootstrap/create_iam_user.py  # prints the initial key

# 2. Rotate the runner key (writes .env.secrets, gitignored)
ACDL_BOOTSTRAP_AWS_ACCESS_KEY_ID=... ACDL_BOOTSTRAP_AWS_SECRET_ACCESS_KEY=... \
  bash scripts/rotate_spike_key.sh

# 3. Run the full platform pipeline (contract -> environment check -> stack ->
#    adapter -> security checks -> infrastructure plan -> policy checks ->
#    confidence -> evidence event -> apply). Output is streamed to stdout.
bash scripts/run_platform.sh contracts/static-assets.yml
# Expected: "=== PLATFORM E2E OK ==="

# Or plan-only (contract -> stack -> adapter -> infrastructure plan; no
# policy checks / outbox):
bash scripts/run_platform.sh --plan-only contracts/static-assets.yaml

# Add --quiet to suppress streaming (output to log files only):
bash scripts/run_platform.sh --quiet contracts/static-assets.yaml

Test the platform (offline, no AWS required)

# Install test dependencies
pip install -r requirements-test.txt

# Run the test suite (all offline — uses moto for DynamoDB mocking)
python3 -m pytest tests/ -v

# Run the platform in check-only mode (offline — no AWS, no policy checks,
# no outbox). Uses the default sample contract (contracts/static-assets.yaml)
# and the sample dev environment (core/environments/dev.json).
bash scripts/run_platform.sh --check-only
# Expected: "=== PLATFORM CHECK OK ==="

# Reproduce the full CI pipeline locally (lint -> test -> check-only)
bash scripts/run_ci.sh
# Expected: "=== CI PIPELINE OK ==="

CI/CD pipelines

The CI/CD pipeline is defined by a central pipeline contract — a declarative YAML instance (pipelines/ci.yml) validated against a JSON Schema (schemas/pipeline.schema.json). Both platform-runner workflows implement the same contract:

• .github/workflows/ci.yml — GitHub Actions (production)

Both run three stages: lint (py_compile), test (pytest), and check-only (run_platform.sh --check-only). Both trigger on push to main and on pull requests. A test (tests/test_pipeline_contract.py) validates that the workflow conforms to the contract.

scripts/run_ci.sh mirrors the CI pipeline locally — running the same three stages in sequence. This makes the pipeline fully reproducible from the shell, not just in CI:

bash scripts/run_ci.sh           # run all 3 stages (lint, test, check-only)
bash scripts/run_ci.sh --quiet    # suppress per-stage banners

Reusable deploy workflow

The deployment pipeline is defined by a central deployment pipeline contract (pipelines/contract.yml, validated against schemas/deploy-pipeline.schema.json) and exposed to consumer repos as a reusable workflow:

• .github/workflows/deploy.yml — GitHub Actions (production)

The workflow implements the same stages as pipelines/contract.yml (validate-contract → resolve-stack → security checks → infrastructure plan → policy checks → confidence → evidence event → apply). A consumer repo invokes the reusable workflow via a versioned tag (floating MAJOR + MINOR, e.g. acdl/.github/workflows/deploy.yml@v1.6). The workflow checks out the consumer repo, then checks out the ACDL platform repo into the runner workspace, and runs scripts/run_platform.sh against the consumer's contract — the consumer never clones the platform repo or invokes its scripts locally. See the Consumer guide for the end-to-end happy path.

Output streaming (run_platform.sh)

scripts/run_platform.sh streams output by default so the user can see what the platform is doing:

• --check-only: streams the emitted infrastructure file content to stdout.
• --plan-only and full mode: streams the infrastructure plan output via tee (visible and logged).
• Full mode: prints policy-check results and each PolicyCheckResult record with severity, rule ID, and pass/fail status.

A --quiet flag suppresses streaming (output to log files only) for backwards-compatible log-only mode.

Consumer guide

A step-by-step guide for a consumer to create their pipeline and define a contract that deploys any ACDL module to AWS is at docs/consumer-guide.md. The guide is generic across all modules; static-assets is the worked example.

Repository layout

Path	Purpose	Status
core/	Platform code: contract resolver, confidence signal, outbox writer, environment check, environments, separation of duties, HITL/ledger designs	active
schemas/	JSON Schemas: stack, contract, PolicyCheckResult, pipeline contract, deploy pipeline contract (draft 2020-12)	active
pipelines/	Central pipeline contracts: ci.yml (CI), contract.yml (deployment)	active
adapters/	Angine adapters — the engine adapter (the only engine-specific code per §12) + the policy adapter	active
terraform/	State backend (S3 + DynamoDB) + platform TF (terraform/spike/) + bootstrap scripts (terraform/bootstrap/)	active
modules/	Primitives + modules + registry.json. Primitives: s3, vpc, ecs-cluster, ecs-service, iam-role, alb, ecr, cloudfront, waf, rds. Modules: microservice, static-assets. Each module has a examples/ directory with validated contract examples	active
contracts/	Sample consumer contracts (static-assets.yaml, microservice.yaml)	active
scripts/	Platform run script (run_platform.sh with --check-only/--plan-only/--quiet), CI pipeline script (run_ci.sh), key rotation	active
tests/	Pytest suite (all offline — adapter, confidence signal, policy adapter, outbox writer, pipeline contract, contract resolver, streaming, environment check)	active
.github/workflows/	GitHub Actions workflows: ci.yml (CI), deploy.yml (reusable deploy, invoked by consumer repos)	active
docs/	GitHub Pages documentation site: consumer guide, modules, contracts, pipeline, versioning, environments, architecture, vision	active

Credentials & zero-trust

Default — zero-trust OIDC + attribute-based authorization

Consumer repos are zero-trust: they hold no long-lived AWS keys and no static credentials in repo secrets.

• Authentication is OIDC federation between the platform runners (GitHub Actions) and AWS. Each job mints a short-lived STS token; no credential is ever stored in the consumer repo or in a runner secret.

• Authorization is attribute-based (ABAC), not role-based (RBAC). AWS IAM roles and session policies are scoped by two attribute classes:

  • Repository identity — the runner claim (e.g. repo:org/consumer-repo:ref:refs/heads/main) binds the role's trust policy to the exact consumer repo + branch that invoked the workflow.
  • Resource-creation attributes — every resource the pipeline creates is tagged with acdl:owner=<consumer-repo> and acdl:contract=<contract-id>. The session policy grants view/update/delete only on resources whose tags match the calling repo.

  The effect: a consumer's pipeline can only view and update the resources it created. Blast radius is contained to that consumer's own stack instances — one consumer can never touch another consumer's resources, and the consumer cannot escape its own scope.

Alternative — static AWS key

Where OIDC is not yet available, a static AWS key may be used as a documented alternative:

• The key is stored in GitHub Secrets (consumer repo) for platform-runner runs, or in .env.secrets (gitignored, chmod 600) for local testing.
• The platform rotates platform-runner keys on a daily cadence — rotation is not the consumer's burden in the platform-runner path.
• When .env.secrets is used locally, rotating the key out of band is the consumer's responsibility. The platform guarantees daily rotation for platform-runner runs; it does not guarantee rotation for locally-held copies. The consumer must rotate a local key via scripts/rotate_spike_key.sh (or equivalent) on their own cadence.

No long-lived credential is permitted persistently — the platform-runner key's useful lifetime is one workflow run, and the local alternative is rotated at least daily (platform-runner) or out of band (local).