acdl/docs/architecture.md

Architecture

Status: v1.0 (current). All design decisions are resolved. This is the source of truth for how the platform works; the Vision is the source of truth for why.

0. Purpose

This document encodes the architectural commitments that realize the vision. Every commitment is grounded in a vision tenet.

The platform is four layers + six cross-cutting concerns, bound by the vision's "Two Consumer Surfaces, One Platform" tenet: both surfaces converge on the same contract schema, the same policy envelope, and the same evidence stream.

1. Architectural Overview

flowchart TD
    A["Consumer surfaces"] --> B["Contract schema"]
    B --> C["Central pipeline"]
    C --> D["Modules + primitives"]
    C --> E["Substrate adapter"]
    C --> F["Confidence signal"]
    C --> G["Evidence stream"]
    D --> E
    E --> H["Infrastructure"]
    F --> G

The four layers:

1. Primitives — single-purpose, substrate-agnostic modules representing the smallest reusable infrastructure pieces (a VPC, an S3 bucket, an ECS cluster). A primitive does not reference other primitives; it takes its environment as input.
2. Modules — patterns that combine primitives into deployable infrastructure shapes (an ECS Fargate microservice, a static-assets site). A module references registered primitives (max depth 5).
3. Developer surface — the developer-owned workflow file + contract. The developer references the central pipeline via a versioned tag and owns their workflow file (no platform auto-sync).
4. Agentic surface — a hybrid runtime where a consumer declares intent in natural language and an agent resolves it to a contract submission. Trust model: trust and always verify on the platform side. Stateless agents; all state lives in the platform.

The developer and agentic surfaces are parallel paths, not a progression. Both end in a contract submission that enters the same pipeline.

2. Primitives

Single-purpose, substrate-agnostic modules. Locked commitments:

• No inter-primitive references. A primitive may call substrate data sources.
• Semver with three triggers: interface → MAJOR, behavior → MINOR, lifecycle → PATCH.
• Immutability on publication.
• 12-month deprecation window.
• AI refinement is a flag, triggered by a joint operational condition (N ≥ 50 consecutive zero-rollback changes, no primitive/module incident in 6 months, Infra & Ops unilateral override).
• A primitive's interface is defined against the Target Stack (substrate- agnostic), not against any substrate's variable block directly.

3. Modules

Patterns that combine primitives into deployable shapes. Locked commitments:

• One codebase maps to one canonical module (default); multiStack: true is permitted only for (a) a DR-region mirror, (b) a time-boxed experimental stack (TTL ≤ 30 days), or (c) explicit Infra & Ops approval with a documented justification.
• A module references registered primitives only (max depth 5).
• Pipeline quality checks: secrets-in-plaintext, public ingress, IAM wildcard, KMS key reference, tag compliance, naming convention.
• Restricted from module patterns: IAM principal creation, network boundary creation, key/secret creation, external data transfer.
• Auto-promote after 3 observed usages.
• A module's pattern tree wires field is defined against the stack's relationship type, not against any substrate's module block. The stack → substrate translation is the substrate adapter's job (§12). The pattern pipeline itself is substrate-agnostic.

4. Developer Surface

• Tag-based reference to the central pipeline template.
• Developer-owned workflow file, no platform auto-sync.
• Tag mutability for production-bound references: tag for dev/qa, SHA for prod. The platform provides a CLI command that resolves the current tag to its SHA for prod-bound workflows.

5. Agentic Surface

• Hybrid runtime: skill as markdown, agent as executor.
• Trust model: trust and always verify on the platform side.
• Skill envelope (4 dimensions).
• Stateless agents; all state in the platform.
• Initial skill catalog: web API, worker, scheduled job, static asset, basic observability bootstrap. Addition criteria: (a) reviewable for sensitive data, (b) expressible as a single contract submission, (c) documented use case.
• profile: agentic unlocks agentic-specific fields (naturalLanguageIntent, confidenceAtSubmission, agentTrace).

6. Cross-Cutting — Central Pipeline Template

• JSON Schema (draft 2020-12) with a thin domain-specific wrapper.
• Central repo + generated client libraries.
• Multi-stage validation pipeline: schema → policy → NFR → confidence.
• Distributed enrichment.
• GitOps reconciler + substrate execution layer.
• The pipeline emits a PolicyCheckResult record per policy rule evaluated; the confidence signal consumes these as one normalized input (§8).

7. Cross-Cutting — Contract Schema

• Central repo + generated client libraries.
• Strict fail-fast at the schema stage with reason codes from a published vocabulary.
• Per-environment mandatory fields: dev requires stack + environment; qa adds validation.e2eSuite + validation.loadTest; prod adds runbook + dashboard + oncall; dr adds drDrillRef. inputs is always optional. profile: agentic fields are optional everywhere (naturalLanguageIntent required when profile is agentic).

8. Cross-Cutting — Confidence Signal

• Six canonical inputs: policy, validation, freshness, source, history, NFRs.
• Weighted sum with per-input breakdown.
• Per-environment thresholds: dev ≥ 0.50, qa ≥ 0.75, prod ≥ 0.90, dr ≥ 0.95.
• Structured output: { score, band, perInput, reasonCodes }.
• 1-year storage; no algorithm retraining in v1.
• Halt with explicit reason on missing input.
• Severity → score penalty: critical → hard override to mandatory block, high → -0.2, medium → -0.05, low → -0.01, info → 0.0. One critical finding hard-overrides the score regardless of all other inputs.
• Thresholds frozen for v1; tuning begins post-v1 with quarterly FP/FN tracking per environment. Override authority = Infra & Ops + SRE joint sign-off; any override is itself a confidence-event in the audit stream.

9. Cross-Cutting — Audit and Evidence Stream

• Every delivery action produces an immutable, hash-chained evidence event.
• The audit stream is the platform's certified record of what happened, when, and why.
• Events are written to a DynamoDB outbox and rendered on an evidence timeline.

10. Cross-Cutting — HITL Matrix

Human-in-the-loop gates for higher environments:

Environment	Autonomy	Attester	Gate
dev	Full autonomy (no HITL)	—	Confidence ≥ 0.50, all six inputs present
qa	Held for attestation	QA	Platform-runner deployment approval + full QA matrix
prod	Held for attestation	SRE	Platform-runner deployment approval + full SRE matrix
dr	Held for attestation	SRE	Platform-runner deployment approval + dr-drill evidence

Staging does not exist. Dev is the only autonomous environment and absorbs integration, contract, security smoke, and performance smoke validation.

• Pre-execution gate model. 1 business day = warn + escalate; 2 business days = auto-freeze + re-submit. Rejection extends the audit chain; no partial deploy to roll back.
• Separation of duties: the platform-internal identity record in the DynamoDB outbox enforces qaApprover ≠ prodApprover for the same contract.

11. Cross-Cutting — Separation of Duties

• CODEOWNERS routes the right reviewer to the right environment.
• The DynamoDB outbox enforces identity distinctness across environment approvers.

12. Cross-Cutting — Substrate Execution

The technical execution layer. Primitives and modules are substrate-agnostic in shape; substrate adapters are the only substrate-specific component.

The architecture defines a Target Stack — a substrate-neutral description of:

• The resources to create (typed against the stack schema).
• Their relationships (the module's pattern tree).
• Their inputs (wired from the contract).
• Policy hooks (the points in the pattern where policy checks attach).

The registry, the module pattern tree, the contract schema, and the PolicyCheckResult schema are all defined against the stack schema. None is defined against any specific substrate.

v1 implementation reality: the stack is shaped to round-trip cleanly to Terraform because there is no other adapter to differentiate from. As additional adapters appear, the stack gets more expressive and the adapters gain translation logic, but the primitive content, the module pattern tree, and the contract schema do not change. This is the design that prevents a polyglot mess.

The substrate adapter:

• Translates the stack-typed module pattern tree to a substrate root module that calls the primitive modules.
• Is a thin layer. It does not own primitive/module content; it only translates.
• Is the only substrate-specific code in the platform.

Policy checks run on the substrate plan output. Results are normalized to PolicyCheckResult records by a policy adapter. The confidence signal consumes the union of all PolicyCheckResult records, regardless of engine — substrate-agnostic over its inputs, matching the module model's substrate-agnosticism over its outputs.

13. Cross-Cutting — Platform Runners

The platform runs on platform-managed runners (GitHub Actions in production). Runner-specific code = workflow YAML, OIDC trust, CODEOWNERS, environments. The contract schema, stack, PolicyCheckResult, confidence signal, and audit stream are portable (runner-agnostic); a second runner platform needs a runner adapter + workflow-template translator, with no change to the modules/stack/confidence/audit.

14. Versioning

• Primitives and modules use semver: interface → MAJOR, behavior → MINOR, lifecycle → PATCH.
• A module pins primitives by name@semver; the resolver picks the highest compatible.
• A MAJOR bump requires a new registry entry (immutable publication); the old entry enters a 12-month deprecation window.
• The central deploy pipeline is referenced by a floating MAJOR + MINOR tag (e.g. @v1.6); patch fixes flow within the tag, breaking changes land under the next MINOR tag.

See Versioning for the consumer-facing details.

15. OpenTofu

Not in v1. The substrate abstraction (§12) makes OpenTofu a future adapter, not an architecture change. Revisit when an OpenTofu adapter is requested.