acdl/docs/pipeline/index.md

Pipeline

The platform runs two pipelines, both defined by declarative contracts that are the single source of truth for the workflow files.

CI pipeline

The CI pipeline runs on every push and pull request to main. It is defined by pipelines/ci.yaml, validated against schemas/pipeline.schema.json. Both platform-runner workflow files implement the same contract and are byte-identical:

• .github/workflows/ci.yml — GitHub Actions (production)

Three stages run in sequence:

1. lint — py_compile across the platform's Python files.
2. test — pytest across the offline test suite.
3. check-only — run_platform.sh --check-only (offline, no AWS).

scripts/run_ci.sh mirrors the CI pipeline locally so the pipeline is fully reproducible from the shell:

bash scripts/run_ci.sh           # run all 3 stages
bash scripts/run_ci.sh --quiet    # suppress per-stage banners

Deployment pipeline

The deployment pipeline runs when a consumer submits a contract. It is defined by pipelines/deploy.yaml, validated against schemas/deploy-pipeline.schema.json. It is exposed to consumer repos as a reusable workflow:

• .github/workflows/deploy.yml — GitHub Actions (production)

A consumer repo invokes the reusable workflow via a versioned tag (floating MAJOR + MINOR, e.g. acdl/.github/workflows/deploy.yml@v1.6). The workflow checks out the consumer repo, then checks out the ACDL platform repo into the runner workspace, and runs scripts/run_platform.sh against the consumer's contract. The consumer never clones the platform repo or invokes its scripts locally. See the Consumer Guide for the end-to-end happy path.

Deployment stages

flowchart TD
    S1["validate-contract<br/>schema check"] --> S2
    S2["resolve-stack<br/>contract -&gt; Target Stack"] --> S3
    S3["security checks<br/>(adapter)"] --> S4
    S4["infrastructure plan<br/>(adapter compiles the stack)"] --> S5
    S5["policy checks<br/>(adapter -&gt; PolicyCheckResult)"] --> S6
    S6["confidence<br/>score + band"] --> S7
    S7["evidence event<br/>to the audit outbox"] --> S8
    S8["infrastructure apply<br/>(dev only)"]

1. validate-contract — validates the contract YAML against the contract schema. Fails fast on missing fields, unknown modules, or wrong types.
2. resolve-stack — the contract resolver resolves the contract to a Target Stack instance (loads the module's pattern, expands its children, wires the contract inputs, emits a stack JSON instance).
3. security checks (adapter) — security checks run on the resolved stack before any infrastructure is planned.
4. infrastructure plan (adapter) — the substrate adapter compiles the stack to an infrastructure plan.
5. policy checks (adapter) — policy checks run on the plan. Results are normalized to PolicyCheckResult records (severity, rule ID, pass/fail).
6. confidence — the confidence signal computes a score from 6 inputs (policy, validation, freshness, source, history, NFRs). For dev, the threshold is ≥ 0.50. If the band is pass, the pipeline proceeds.
7. evidence event — a hash-chained evidence event is written to the audit outbox.
8. infrastructure apply (dev only) — the infrastructure plan is applied, creating the resources. An evidence event for the apply is recorded.

Higher environments hold for human attestation (see Environments).

Output streaming

scripts/run_platform.sh streams output by default so the user can see what the platform is doing:

• --check-only: streams the emitted infrastructure file content.
• --plan-only and full mode: streams the infrastructure plan output.
• Full mode: prints policy-check results with severity, rule ID, and pass/fail status.

A --quiet flag suppresses streaming (output to log files only).