diff --git a/README.md b/README.md index 895fbd3..563de91 100644 --- a/README.md +++ b/README.md @@ -5,61 +5,85 @@ through an agentic stack — automatically, safely, and with a complete audit trail. A merged change progresses through lower environments end-to-end without a platform engineer joining a thread; a non-technical consumer ships a production deployment by declaring intent, without authoring a workflow, -a configuration file, or a Terraform module. +a configuration file, or an infrastructure module. -- **Vision** (the why): [`docs/vision.md`](docs/vision.md) -- **Architecture** (the how): [`docs/architecture.md`](docs/architecture.md) + [`.ciagent/ARCHITECTURE.md`](.ciagent/ARCHITECTURE.md) -- **Decisions**: [`.ciagent/PROJECT.md`](.ciagent/PROJECT.md) -- **Phase plan**: [`.ciagent/ROADMAP.md`](.ciagent/ROADMAP.md) -- **Consumer guide**: [`docs/CONSUMER_GUIDE.md`](docs/CONSUMER_GUIDE.md) +- **Consumer guide:** [`docs/consumer-guide.md`](docs/consumer-guide.md) +- **Modules:** [`docs/modules/`](docs/modules/) +- **Contracts:** [`docs/contracts/`](docs/contracts/) +- **Pipeline:** [`docs/pipeline/`](docs/pipeline/) +- **Versioning:** [`docs/pipeline/versioning.md`](docs/pipeline/versioning.md) +- **Environments:** [`docs/environments/`](docs/environments/) +- **Architecture:** [`docs/architecture.md`](docs/architecture.md) +- **Vision:** [`docs/vision.md`](docs/vision.md) ## Repository roles There are two kinds of repository in the ACDL model: - **Platform repo (this one).** This is the **source code of the platform**. - It owns `modules/`, `adapters/`, `acdl_platform/`, `schemas/`, `pipelines/`, + It owns `modules/`, `adapters/`, `core/`, `schemas/`, `pipelines/`, `scripts/`, and the reusable workflow files. Platform engineers work here. A **consumer never clones it.** -- **Consumer repo (yours).** A consumer repo contains only its application - code and a single `contract.yaml` that references the central pipeline + - contract. The consumer does not write Terraform, workflow YAML, or adapter - code — they write a contract YAML file and the platform does the rest. +- **Consumer repo (yours).** A consumer repo contains only: + 1. **Its application code** — the service or site being deployed. + 2. **One or more contracts** — small YAML files at `.acdl/contract.yaml` + that reference the central pipeline, name a module, select an + environment, and supply module-specific inputs. + 3. **One or more CI definitions** — thin `.github/workflows/*.yml` files + that `uses:` the central reusable deploy workflow, pointing at the + appropriate environment + contract. + + The consumer does not write infrastructure modules, workflow YAML beyond + the thin `uses:` wrapper, or adapter code — they write a contract YAML + file and the platform does the rest. The rest of this README describes the **platform repo** (how the platform works, how to run it locally, how it's laid out). If you are a consumer, -jump to the [Consumer guide](docs/CONSUMER_GUIDE.md). +jump to the [Consumer guide](docs/consumer-guide.md). -## Status +## Features -- **v1.5 (active):** consumer happy path + zero-trust docs + reusable deploy - workflow. README rewritten so the consumer model is unambiguous. Platform - flow + consumer guide converted to mermaid. Legacy surface + implementation - nomenclature removed from docs. Credentials section rewritten for - zero-trust OIDC + ABAC. A generic `docs/CONSUMER_GUIDE.md` (all L2 modules, - versioned `uses:`, consumer-scoped prerequisites, run-time platform fetch) - replaces the module-specific guide. A byte-identical reusable `deploy.yml` - workflow (Gitea + GitHub) implements `pipelines/deploy.yaml` and is invoked - by consumer repos via a versioned tag. -- **v1.4 (complete, tag `v1.4.1`):** central pipeline contract + shell - reproducibility + output streaming. A declarative pipeline contract - (`schemas/pipeline.schema.json` + `pipelines/ci.yaml`) binds the Gitea - and GitHub workflows to a single source of truth. `scripts/run_ci.sh` - mirrors the CI pipeline locally. `scripts/run_platform.sh` streams - terraform/checkov output by default. L2 compositions re-introduced with - a `uses:`-based contract resolution mechanism. -- **v1.3 (complete, tag `v1.3.2`):** module documentation. Testing + CI/CD - pipelines (pytest, `--check-only`, Gitea + GitHub workflows). -- **v1.2 (complete, tag `v1.3.0`):** platform hardening + first real - consumer deployment. Harden the v1.1 implementation's NFRs, simplify the - setup, rewrite the docs, and prove the platform delivers real value by - deploying a basic microservice to AWS ECS Fargate end-to-end (`terraform - apply`, dev autonomous). -- **v1.1 (complete, tag `v1.2.0`):** architecture finalization + v1 - implementation. Finalized the architecture to v1.0 (resolved all 11 open - design decisions) and proved the stack commitments hold with one - end-to-end run (`s3` + `static-asset` + Terraform adapter → real - `terraform plan` against AWS). Gitea release id 202. +A referenceable list of what the platform provides today, for consumers and +platform engineers alike: + +- **Contract-driven deploys** — a consumer writes a YAML contract; the + platform resolves it to a stack, compiles it, and deploys it. +- **Reusable versioned deploy workflow** — consumer repos `uses:` a + versioned central workflow; no platform code is cloned by the consumer. +- **Module catalog** — primitives (single resources) and modules (patterns + of primitives) with self-documented inputs/outputs. See + [docs/modules/](docs/modules/). +- **Zero-trust credentials** — OIDC federation + attribute-based + authorization (ABAC) by default; no long-lived keys in consumer repos. +- **Security + policy checks** — a security-check stage and a policy-check + stage run before any infrastructure is created. +- **Confidence signal** — a computed, explainable score gates promotion. +- **Evidence outbox** — every deployment writes a hash-chained evidence + event to an audit outbox. +- **Shell reproducibility** — `scripts/run_ci.sh` mirrors the CI pipeline + locally; `scripts/run_platform.sh --check-only` runs offline. +- **Platform-managed environments** — consumers provide no AWS account, + VPC, subnet, or state bucket; the platform manages environments. See + [docs/environments/](docs/environments/). +- **Central pipeline contract** — a declarative YAML instance is the single + source of truth for both the CI and deploy workflows. + +## Roadmap + +Planned future features (no dates; tracked in the internal roadmap): + +- **Dynamic module creation from a contract** — an agentic flow where a + consumer creates a module directly from the contract file (the + "composition" mechanism, redesigned). +- **Compliance milestone** — per-module compliance extension points (GDPR, + SOX, SOC2, HIPAA, DORA) wired into the pipeline. +- **Additional substrate adapters** — beyond the Terraform adapter. +- **Environment self-service** — a consumer-facing flow to request and + provision a new platform-managed environment (today it is a platform-team + action). +- **HITL gates for qa / prod / dr** — human attestation + higher confidence + thresholds for higher environments. +- **OIDC for all platform runners** — zero-trust credentials everywhere. ## How the platform works @@ -72,31 +96,31 @@ stream. Consumers have their own repos and consume ACDL by referencing `uses:` the central pipeline definitions. A consumer declares a contract (module + environment + inputs); the platform resolves it to a stack instance, -compiles it to Terraform, runs policy checks, computes a confidence signal, -and writes an evidence event to the audit outbox. +compiles it, runs security + policy checks, computes a confidence signal, +writes an evidence event to the audit outbox, and applies the +infrastructure. ### The platform flow (end-to-end) ```mermaid flowchart TD - A["contracts/static-asset.yaml
(consumer contract: uses + module + inputs)"] --> B - B["schema validation
(schemas/contract.schema.json)"] --> C - C["acdl_platform/contract_resolver.py
→ Target Stack (JSON)"] --> D - D["stack schema validation
(schemas/stack.schema.json)"] --> E - E["adapters/terraform/adapter.py
→ terraform/spike/{main,terraform,providers}.tf
(the only substrate-specific code)"] --> F - F["terraform plan
(real AWS, via the rotated runner key — D-039/D-047)"] --> G - G["adapters/terraform/policy/checkov_adapter.py
→ PolicyCheckResult (JSON list)
(normalized, engine-agnostic)"] --> H - H["acdl_platform/confidence_signal.py
→ { score, band, perInput, reasonCodes }
(6 inputs: policy, validation, freshness, source, history, nfrs)"] --> I - I["acdl_platform/outbox_writer.py
→ DynamoDB outbox (acdl-outbox)
(hash-chained evidence event)"] --> J - J["acdl-evidence timeline
(acdl-evidence repo, raw-file served)"] + A["consumer contract
(uses + module + environment + inputs)"] --> B + B["schema validation
(contract schema)"] --> C + C["resolve to Target Stack
(contract resolver)"] --> D + D["security checks
(adapter)"] --> E + E["infrastructure plan
(adapter compiles the stack)"] --> F + F["policy checks
(adapter -> PolicyCheckResult records)"] --> G + G["confidence signal
(6 inputs: policy, validation,
freshness, source, history, NFRs)"] --> H + H["evidence event
(hash-chained, to the audit outbox)"] --> I + I["infrastructure apply
(dev only, autonomous)"] ``` The platform validates the architecture's claim that the **stack commitments do not require a polyglot mess**: the adapter is the only substrate-specific code. `modules/`, `schemas/`, `contracts/`, -`acdl_platform/confidence_signal.py`, `acdl_platform/contract_resolver.py`, -and `acdl_platform/outbox_writer.py` are all substrate-agnostic (no -`aws_s3_bucket` / `aws_` Terraform terms). +`core/confidence_signal.py`, `core/contract_resolver.py`, and +`core/outbox_writer.py` are all substrate-agnostic (no `aws_s3_bucket` / +`aws_` infrastructure terms). ## How to run @@ -104,11 +128,12 @@ and `acdl_platform/outbox_writer.py` are all substrate-agnostic (no > These prerequisites are for running the **platform repo** locally. A > consumer does not need any of these — see the -> [Consumer guide](docs/CONSUMER_GUIDE.md) for the consumer happy path. +> [Consumer guide](docs/consumer-guide.md) for the consumer happy path. -- AWS account + the rotated runner key in `.env.secrets` (see - `scripts/rotate_spike_key.sh`; the bootstrap root key was deactivated - per D-034 closure). +- A platform-managed environment (see [docs/environments/](docs/environments/)). + For local testing, `core/environments/dev.json` is provided as the sample. +- AWS credentials for the dev environment (in `.env.secrets`, gitignored; + see [Credentials & zero-trust](#credentials--zero-trust)). - `terraform` (pin `1.9.*`), `checkov` (pin `>=3.2,<4`), `python3` + `boto3` + `jsonschema`. @@ -116,8 +141,8 @@ and `acdl_platform/outbox_writer.py` are all substrate-agnostic (no ```bash # 1. Bootstrap the AWS state backend + runner IAM user (one-time, idempotent) -# (requires the bootstrap root key in env — now deactivated; skip if -# the state bucket + acdl-spike-runner already exist) +# (requires the bootstrap root key in env — skip if the state bucket + +# acdl-spike-runner already exist) ACDL_BOOTSTRAP_AWS_ACCESS_KEY_ID=... ACDL_BOOTSTRAP_AWS_SECRET_ACCESS_KEY=... \ python3 terraform/bootstrap/create_state_backend.py ACDL_BOOTSTRAP_AWS_ACCESS_KEY_ID=... ACDL_BOOTSTRAP_AWS_SECRET_ACCESS_KEY=... \ @@ -127,12 +152,14 @@ ACDL_BOOTSTRAP_AWS_ACCESS_KEY_ID=... ACDL_BOOTSTRAP_AWS_SECRET_ACCESS_KEY=... \ ACDL_BOOTSTRAP_AWS_ACCESS_KEY_ID=... ACDL_BOOTSTRAP_AWS_SECRET_ACCESS_KEY=... \ bash scripts/rotate_spike_key.sh -# 3. Run the full platform pipeline (contract -> stack -> adapter -> plan -> -# Checkov -> confidence -> outbox). Output is streamed to stdout by default. +# 3. Run the full platform pipeline (contract -> environment check -> stack -> +# adapter -> security checks -> infrastructure plan -> policy checks -> +# confidence -> evidence event -> apply). Output is streamed to stdout. bash scripts/run_platform.sh contracts/static-asset.yaml # Expected: "=== PLATFORM E2E OK ===" -# Or plan-only (contract -> stack -> adapter -> terraform plan; no Checkov/outbox): +# Or plan-only (contract -> stack -> adapter -> infrastructure plan; no +# policy checks / outbox): bash scripts/run_platform.sh --plan-only contracts/static-asset.yaml # Add --quiet to suppress streaming (output to log files only): @@ -148,12 +175,13 @@ pip install -r requirements-test.txt # Run the test suite (all offline — uses moto for DynamoDB mocking) python3 -m pytest tests/ -v -# Run the platform in check-only mode (offline — no AWS, no Checkov, no outbox) -# Uses the default sample contract (contracts/static-asset.yaml) +# Run the platform in check-only mode (offline — no AWS, no policy checks, +# no outbox). Uses the default sample contract (contracts/static-asset.yaml) +# and the sample dev environment (core/environments/dev.json). bash scripts/run_platform.sh --check-only # Expected: "=== PLATFORM CHECK OK ===" -# Reproduce the full CI pipeline locally (lint → test → check-only) +# Reproduce the full CI pipeline locally (lint -> test -> check-only) bash scripts/run_ci.sh # Expected: "=== CI PIPELINE OK ===" ``` @@ -162,18 +190,15 @@ bash scripts/run_ci.sh The CI/CD pipeline is defined by a **central pipeline contract** — a declarative YAML instance (`pipelines/ci.yaml`) validated against a JSON -Schema (`schemas/pipeline.schema.json`). Both forge workflows implement -the same contract: +Schema (`schemas/pipeline.schema.json`). Both platform-runner workflows +implement the same contract: -- `.gitea/workflows/ci.yml` — Gitea Actions (dev environment) - `.github/workflows/ci.yml` — GitHub Actions (production) -Both workflow files are **byte-identical** — the only difference is the -forge runtime. Both run three stages: **lint** (py_compile), **test** -(pytest), and **check-only** (`run_platform.sh --check-only`). Both -trigger on push to `main` and on pull requests. A test -(`tests/test_pipeline_contract.py`) validates that both workflows conform -to the contract. +Both run three stages: **lint** (py_compile), **test** (pytest), and +**check-only** (`run_platform.sh --check-only`). Both trigger on push to +`main` and on pull requests. A test (`tests/test_pipeline_contract.py`) +validates that the workflow conforms to the contract. `scripts/run_ci.sh` mirrors the CI pipeline locally — running the same three stages in sequence. This makes the pipeline fully reproducible from @@ -191,18 +216,17 @@ contract** (`pipelines/deploy.yaml`, validated against `schemas/deploy-pipeline.schema.json`) and exposed to consumer repos as a **reusable workflow**: -- `.gitea/workflows/deploy.yml` — Gitea Actions (dev environment) - `.github/workflows/deploy.yml` — GitHub Actions (production) -Both files are **byte-identical** and implement the same stages as -`pipelines/deploy.yaml` (validate-contract → resolve-stack → -terraform-plan → checkov → confidence → apply). A consumer repo invokes -the reusable workflow via a **versioned tag** (floating MAJOR + MINOR, e.g. -`acdl/.gitea/workflows/deploy.yml@v1.4`). The workflow checks out the -consumer repo, then checks out the ACDL platform repo into the runner -workspace, and runs `scripts/run_platform.sh` against the consumer's +The workflow implements the same stages as `pipelines/deploy.yaml` +(validate-contract → resolve-stack → security checks → infrastructure plan +→ policy checks → confidence → evidence event → apply). A consumer repo +invokes the reusable workflow via a **versioned tag** (floating MAJOR + +MINOR, e.g. `acdl/.github/workflows/deploy.yml@v1.4`). The workflow checks +out the consumer repo, then checks out the ACDL platform repo into the +runner workspace, and runs `scripts/run_platform.sh` against the consumer's contract — the consumer never clones the platform repo or invokes its -scripts locally. See the [Consumer guide](docs/CONSUMER_GUIDE.md) for the +scripts locally. See the [Consumer guide](docs/consumer-guide.md) for the end-to-end happy path. ### Output streaming (run_platform.sh) @@ -210,11 +234,12 @@ end-to-end happy path. `scripts/run_platform.sh` streams output by default so the user can see what the platform is doing: -- **`--check-only`**: streams the emitted Terraform file content to stdout -- **`--plan-only`** and **full mode**: streams `terraform init`, `terraform - validate`, and `terraform plan` output via `tee` (visible and logged) -- **Full mode**: prints Checkov compliance results and each - PolicyCheckResult record with severity, rule ID, and pass/fail status +- **`--check-only`**: streams the emitted infrastructure file content to + stdout. +- **`--plan-only`** and **full mode**: streams the infrastructure plan + output via `tee` (visible and logged). +- **Full mode**: prints policy-check results and each `PolicyCheckResult` + record with severity, rule ID, and pass/fail status. A `--quiet` flag suppresses streaming (output to log files only) for backwards-compatible log-only mode. @@ -223,51 +248,38 @@ backwards-compatible log-only mode. A step-by-step guide for a consumer to create their pipeline and define a contract that deploys any ACDL module to AWS is at -[`docs/CONSUMER_GUIDE.md`](docs/CONSUMER_GUIDE.md). The guide is generic -across all L2 modules; `static-asset` is the worked example. +[`docs/consumer-guide.md`](docs/consumer-guide.md). The guide is generic +across all modules; `static-asset` is the worked example. ## Repository layout | Path | Purpose | Status | |------|---------|--------| -| `acdl_platform/` | Platform code: contract resolver, confidence signal, outbox writer, separation of duties, HITL/ledger designs | active | +| `core/` | Platform code: contract resolver, confidence signal, outbox writer, environment check, environments, separation of duties, HITL/ledger designs | active | | `schemas/` | JSON Schemas: stack, contract, PolicyCheckResult, pipeline contract, deploy pipeline contract (draft 2020-12) | active | | `pipelines/` | Central pipeline contracts: `ci.yaml` (CI), `deploy.yaml` (deployment) | active | -| `adapters/` | Substrate adapters — Terraform adapter (the only substrate-specific code per §12) + Checkov policy adapter | active | +| `adapters/` | Substrate adapters — the substrate adapter (the only substrate-specific code per §12) + the policy adapter | active | | `terraform/` | State backend (S3 + DynamoDB) + platform TF (`terraform/spike/`) + bootstrap scripts (`terraform/bootstrap/`) | active | -| `modules/` | L1/L2 modules + `registry.json`. L1: s3, vpc, ecs-cluster, ecs-service, iam-role, alb, ecr. L2: microservice, static-asset | active | +| `modules/` | Primitives + modules + `registry.json`. Primitives: s3, vpc, ecs-cluster, ecs-service, iam-role, alb, ecr. Modules: microservice, static-asset | active | | `contracts/` | Sample consumer contracts (e.g. `static-asset.yaml`) | active | | `scripts/` | Platform run script (`run_platform.sh` with `--check-only`/`--plan-only`/`--quiet`), CI pipeline script (`run_ci.sh`), key rotation | active | -| `tests/` | Pytest suite (all offline — adapter, confidence signal, checkov adapter, outbox writer, pipeline contract, contract resolver, streaming) | active | -| `.gitea/workflows/` | Gitea Actions workflows: `ci.yml` (CI), `deploy.yml` (reusable deploy, invoked by consumer repos) | active | +| `tests/` | Pytest suite (all offline — adapter, confidence signal, policy adapter, outbox writer, pipeline contract, contract resolver, streaming, environment check) | active | | `.github/workflows/` | GitHub Actions workflows: `ci.yml` (CI), `deploy.yml` (reusable deploy, invoked by consumer repos) | active | -| `.ciagent/` | CIAgent metadata (config, project, architecture, requirements, roadmap, personas, plans, research, verify, review, audit) | active | -| `docs/` | Upstream vision + architecture sources (`vision.md`, `architecture.md`) + consumer guide | active | - -## Environments - -| Environment | Autonomy | Gate | Status | -|---|---|---|---| -| dev | Full autonomy (no HITL) | Confidence ≥ 0.50 | v1.1 (`plan`); v1.2 (`apply`) | -| qa | Held for attestation | QA HITL + confidence ≥ 0.75 | v1.3+ | -| prod | Held for attestation | SRE HITL + confidence ≥ 0.90 | v1.3+ | -| dr | Held for attestation | SRE HITL + confidence ≥ 0.95 + dr-drill | v1.3+ | - -**Staging does not exist** (Path A locked). +| `docs/` | GitHub Pages documentation site: consumer guide, modules, contracts, pipeline, versioning, environments, architecture, vision | active | ## Credentials & zero-trust -### Default — zero-trust OIDC + attribute-based authorization (the locked target) +### Default — zero-trust OIDC + attribute-based authorization -Consumer GitHub/Gitea repos are **zero-trust**: they hold **no long-lived -AWS keys** and no static credentials in repo secrets. +Consumer repos are **zero-trust**: they hold **no long-lived AWS keys** and +no static credentials in repo secrets. -- **Authentication** is **OIDC federation** between the forge (GitHub or - Gitea Actions) and AWS. Each job mints a short-lived STS token; no - credential is ever stored in the consumer repo or in a forge secret. +- **Authentication** is **OIDC federation** between the platform runners + (GitHub Actions) and AWS. Each job mints a short-lived STS token; no + credential is ever stored in the consumer repo or in a runner secret. - **Authorization** is **attribute-based (ABAC)**, not role-based (RBAC). AWS IAM roles and session policies are scoped by two attribute classes: - - **Repository identity** — the forge claim (e.g. + - **Repository identity** — the runner claim (e.g. `repo:org/consumer-repo:ref:refs/heads/main`) binds the role's trust policy to the exact consumer repo + branch that invoked the workflow. - **Resource-creation attributes** — every resource the pipeline creates @@ -281,27 +293,21 @@ AWS keys** and no static credentials in repo secrets. instances — one consumer can never touch another consumer's resources, and the consumer cannot escape its own scope. -### Override — static key + managed daily rotation +### Alternative — static AWS key -Where OIDC is not yet available (Gitea Actions OIDC is blocked on -[go-gitea/gitea#36988](https://github.com/go-gitea/gitea/pull/36988), still -open as of 2026-07-21), a static AWS key **may** be used as a documented -override: +Where OIDC is not yet available, a static AWS key **may** be used as a +documented alternative: -- The key is stored in **GitHub Secrets** (consumer repo) for forge runs, - or in **`.env.secrets`** (gitignored, chmod 600) for local testing. -- The key is rotated by a **platform-managed scheduled pipeline on a daily - cadence** — rotation is not the consumer's burden in the forge path. +- The key is stored in **GitHub Secrets** (consumer repo) for platform-runner + runs, or in **`.env.secrets`** (gitignored, chmod 600) for local testing. +- The platform rotates platform-runner keys on a **daily cadence** — + rotation is not the consumer's burden in the platform-runner path. - **When `.env.secrets` is used locally**, rotating the key **out of band is the consumer's responsibility**. The platform guarantees daily rotation - for forge runs; it does not guarantee rotation for locally-held copies. - The consumer must rotate a local key via `scripts/rotate_spike_key.sh` - (or equivalent) on their own cadence. + for platform-runner runs; it does not guarantee rotation for + locally-held copies. The consumer must rotate a local key via + `scripts/rotate_spike_key.sh` (or equivalent) on their own cadence. -The current per-run-rotated-key flow (waivers D-039 / D-047) is the -present-day instance of this override. The zero-trust OIDC + ABAC model -above is the locked target; the override is time-boxed until the Gitea -OIDC provider merges. `§12.5` forbids long-lived credentials; both the -target and the override satisfy its *intent* (no *persistently* long-lived -key — the forge key's useful lifetime is one workflow run, and the -override is rotated at least daily). \ No newline at end of file +No long-lived credential is permitted persistently — the platform-runner +key's useful lifetime is one workflow run, and the local alternative is +rotated at least daily (platform-runner) or out of band (local). \ No newline at end of file diff --git a/modules/README-TEMPLATE.md b/modules/README-TEMPLATE.md index 48cb0a4..f046285 100644 --- a/modules/README-TEMPLATE.md +++ b/modules/README-TEMPLATE.md @@ -1,6 +1,6 @@ # <module-name> — <plain-language description> -> **Module kind:** L1 primitive | **Version:** 1.0.0 +> **Module kind:** primitive | **Version:** 1.0.0 ## Overview diff --git a/modules/README.md b/modules/README.md index dfdb542..7616e1b 100644 --- a/modules/README.md +++ b/modules/README.md @@ -8,19 +8,19 @@ self-documented with a `README.md` following the There are two kinds of module: -- **L1 primitives** — a single cloud resource or a small group of - related resources (e.g. a VPC with subnets and routing). Each L1 has - an `interface.json` declaring its inputs and outputs, and a `README.md` - in plain language. -- **L2 compositions** — a composition that references multiple L1s to - deploy a complete stack (e.g. an ECS Fargate microservice). Each L2 +- **Primitives** — a single cloud resource or a small group of + related resources (e.g. a VPC with subnets and routing). Each primitive + has an `interface.json` declaring its inputs and outputs, and a + `README.md` in plain language. +- **Modules** — a pattern that references multiple primitives to + deploy a complete stack (e.g. an ECS Fargate microservice). Each module has a `composition.json` declaring its children and wires. -The Terraform adapter (`adapters/terraform/adapter.py`) compiles a -module instance to Terraform. Each module's README documents which -Terraform resources it creates. +The substrate adapter (`adapters/terraform/adapter.py`) compiles a +module instance to infrastructure. Each module's README documents which +resources it creates. -## L1 primitives +## Primitives | Module | What it creates | README | |--------|----------------|--------| @@ -32,19 +32,26 @@ Terraform resources it creates. | `alb` | `aws_lb` + `aws_lb_target_group` + `aws_lb_listener` — Application Load Balancer | [README](l1/alb/README.md) | | `ecr` | `aws_ecr_repository` — ECR container image repository | [README](l1/ecr/README.md) | -## L2 compositions +## Modules | Module | What it references | README | |--------|--------------------|--------| -| `microservice` | 6 L1s (vpc, cluster, ecr, iam-role, alb, ecs-service) | [README](l2/microservice/README.md) | -| `static-asset` | 1 L1 (s3) | [README](l2/static-asset/README.md) | +| `microservice` | 6 primitives (vpc, cluster, ecr, iam-role, alb, ecs-service) | [README](l2/microservice/README.md) | +| `static-asset` | 1 primitive (s3) | [README](l2/static-asset/README.md) | ## Registry -Module versions are tracked in `registry.json`. Both L1 and L2 entries -are registered. +Module versions are tracked in `registry.json`. Both primitives and +modules are registered. ## Template New modules should use [README-TEMPLATE.md](README-TEMPLATE.md) as -their starting point. \ No newline at end of file +their starting point. + +## Module patterns (roadmap) + +The current `composition.json` mechanism is a thin pattern layer. A future +redesign will let a consumer dynamically create a module directly from the +contract file (an agentic "composition" flow). That is on the roadmap, not +implemented today. \ No newline at end of file diff --git a/modules/l1/alb/README.md b/modules/l1/alb/README.md index 15a605a..37e9b6f 100644 --- a/modules/l1/alb/README.md +++ b/modules/l1/alb/README.md @@ -1,6 +1,6 @@ # alb — Application Load Balancer (load balancer + target group + listener) -> **Module kind:** L1 primitive | **Version:** 1.0.0 +> **Module kind:** primitive | **Version:** 1.0.0 An Application Load Balancer with a target group and a listener. This is a multi-resource module: it creates a load balancer, a target group, and diff --git a/modules/l1/ecr/README.md b/modules/l1/ecr/README.md index c630e30..77eb847 100644 --- a/modules/l1/ecr/README.md +++ b/modules/l1/ecr/README.md @@ -1,6 +1,6 @@ # ecr — ECR repository -> **Module kind:** L1 primitive | **Version:** 1.0.0 +> **Module kind:** primitive | **Version:** 1.0.0 A single ECR repository that hosts the container image for the ECS task. The simplest container-registry module — one resource, two diff --git a/modules/l1/ecs-cluster/README.md b/modules/l1/ecs-cluster/README.md index 7c9e984..73c8247 100644 --- a/modules/l1/ecs-cluster/README.md +++ b/modules/l1/ecs-cluster/README.md @@ -1,6 +1,6 @@ # ecs-cluster — ECS Fargate cluster -> **Module kind:** L1 primitive | **Version:** 1.0.0 +> **Module kind:** primitive | **Version:** 1.0.0 An ECS Fargate cluster. The simplest ECS module — one resource, two inputs, two outputs. The cluster is the container orchestration diff --git a/modules/l1/ecs-service/README.md b/modules/l1/ecs-service/README.md index d292d55..7ec1a0d 100644 --- a/modules/l1/ecs-service/README.md +++ b/modules/l1/ecs-service/README.md @@ -1,6 +1,6 @@ # ecs-service — ECS Fargate service (task definition + service) -> **Module kind:** L1 primitive | **Version:** 1.0.0 +> **Module kind:** primitive | **Version:** 1.0.0 An ECS Fargate service with its task definition. Runs a container image on Fargate, optionally behind an ALB target group. This is a diff --git a/modules/l1/iam-role/README.md b/modules/l1/iam-role/README.md index 638e1ef..91cc719 100644 --- a/modules/l1/iam-role/README.md +++ b/modules/l1/iam-role/README.md @@ -1,6 +1,6 @@ # iam-role — IAM role -> **Module kind:** L1 primitive | **Version:** 1.0.0 +> **Module kind:** primitive | **Version:** 1.0.0 A single IAM role with an assume-role policy and optional managed policy attachments. Used as the ECS task execution role. diff --git a/modules/l1/s3/README.md b/modules/l1/s3/README.md index ca14749..0a65c4e 100644 --- a/modules/l1/s3/README.md +++ b/modules/l1/s3/README.md @@ -1,6 +1,6 @@ # s3 — S3 bucket -> **Module kind:** L1 primitive | **Version:** 1.0.0 +> **Module kind:** primitive | **Version:** 1.0.0 A single S3 bucket for object storage. The simplest module — one resource, two inputs, two outputs. Versioning is enabled by default. diff --git a/modules/l1/vpc/README.md b/modules/l1/vpc/README.md index 52ab043..924049b 100644 --- a/modules/l1/vpc/README.md +++ b/modules/l1/vpc/README.md @@ -1,6 +1,6 @@ # vpc — VPC with subnets and routing -> **Module kind:** L1 primitive | **Version:** 1.0.0 +> **Module kind:** primitive | **Version:** 1.0.0 A VPC with one subnet per availability zone and a route table with a default route through an internet gateway. The networking foundation diff --git a/modules/l2/microservice/README.md b/modules/l2/microservice/README.md index 6e102c1..59984dd 100644 --- a/modules/l2/microservice/README.md +++ b/modules/l2/microservice/README.md @@ -1,16 +1,16 @@ # microservice — ECS Fargate microservice -> **Module kind:** L2 composition | **Version:** 1.0.0 +> **Module kind:** module pattern | **Version:** 1.0.0 -A composition that references multiple L1 primitives to deploy an ECS +A pattern that references multiple primitives to deploy an ECS Fargate microservice end-to-end (VPC, cluster, ECR, IAM role, ALB, ECS service). ## Resources -The composition references these L1 primitives: +The pattern references these primitives: -| L1 module | Purpose | README | +| Primitive | Purpose | README | |-----------|---------|--------| | `vpc` | VPC, subnets, routing | [README](../l1/vpc/README.md) | | `ecs-cluster` | ECS Fargate cluster | [README](../l1/ecs-cluster/README.md) | @@ -38,10 +38,10 @@ The composition references these L1 primitives: ## Usage -Define a contract referencing this composition: +Define a contract referencing this module: ```yaml -uses: acdl/pipelines/deploy.yaml@v1 +uses: acdl/pipelines/deploy.yaml@v1.4 module: microservice environment: dev inputs: @@ -52,7 +52,7 @@ inputs: ## Compliance extension points -The composition can wire compliance resources across L1s when the +The pattern can wire compliance resources across primitives when the compliance milestone (GDPR, SOX, SOC2, HIPAA, DORA) lands: - **KMS key** — shared encryption key referenced by S3, ECR, CloudWatch Logs, and Secrets Manager. @@ -61,7 +61,7 @@ compliance milestone (GDPR, SOX, SOC2, HIPAA, DORA) lands: - **Security groups** — proper network segmentation between ALB, service, and data tiers. - **Private subnets** — ECS tasks in private subnets with NAT egress. -See each L1 module's README for per-module compliance extension points. +See each primitive's README for per-module compliance extension points. ## Versioning diff --git a/modules/l2/static-asset/README.md b/modules/l2/static-asset/README.md index 39e4e2d..bbe0b9a 100644 --- a/modules/l2/static-asset/README.md +++ b/modules/l2/static-asset/README.md @@ -1,15 +1,15 @@ # static-asset — S3 static asset -> **Module kind:** L2 composition | **Version:** 1.0.0 +> **Module kind:** module pattern | **Version:** 1.0.0 -A composition that references the `s3` L1 primitive to deploy a single +A pattern that references the `s3` primitive to deploy a single S3 bucket for static asset hosting. ## Resources -The composition references this L1 primitive: +The pattern references this primitive: -| L1 module | Purpose | README | +| Primitive | Purpose | README | |-----------|---------|--------| | `s3` | S3 bucket | [README](../l1/s3/README.md) | @@ -29,10 +29,10 @@ The composition references this L1 primitive: ## Usage -Define a contract referencing this composition: +Define a contract referencing this module: ```yaml -uses: acdl/pipelines/deploy.yaml@v1 +uses: acdl/pipelines/deploy.yaml@v1.4 module: static-asset environment: dev inputs: @@ -40,13 +40,13 @@ inputs: region: us-east-1 ``` -See the [consumer guide](../../docs/CONSUMER_GUIDE.md) for a +See the [consumer guide](../../docs/consumer-guide.md) for a step-by-step walkthrough, and the [s3 README](../l1/s3/README.md) for the -underlying L1 primitive. +underlying primitive. ## Compliance extension points -The composition can wire compliance resources when the compliance +The pattern can wire compliance resources when the compliance milestone (GDPR, SOX, SOC2, HIPAA, DORA) lands: - **KMS key** — shared encryption key for S3 SSE.