acdl/docs/architecture.md

# Architecture

> **Status:** v1.0 (current). All design decisions are resolved. This is the
> source of truth for *how* the platform works; the [Vision](vision) is the
> source of truth for *why*.

## 0. Purpose

This document encodes the architectural commitments that realize the
[vision](vision). Every commitment is grounded in a vision tenet.

The platform is **four layers + six cross-cutting concerns**, bound by the
vision's "Two Consumer Surfaces, One Platform" tenet: both surfaces converge
on the same contract schema, the same policy envelope, and the same evidence
stream.

## 1. Architectural Overview

```mermaid
flowchart TD
    A["Consumer surfaces"] --> B["Contract schema"]
    B --> C["Central pipeline"]
    C --> D["Modules + primitives"]
    C --> E["Substrate adapter"]
    C --> F["Confidence signal"]
    C --> G["Evidence stream"]
    D --> E
    E --> H["Infrastructure"]
    F --> G
```

The four layers:

1. **Primitives** — single-purpose, substrate-agnostic modules representing
   the smallest reusable infrastructure pieces (a VPC, an S3 bucket, an ECS
   cluster). A primitive does not reference other primitives; it takes its
   environment as input.
2. **Modules** — patterns that combine primitives into deployable
   infrastructure shapes (an ECS Fargate microservice, a static-assets site).
   A module references registered primitives (max depth 5).
3. **Developer surface** — the developer-owned workflow file + contract. The
   developer references the central pipeline via a versioned tag and owns
   their workflow file (no platform auto-sync).
4. **Agentic surface** — a hybrid runtime where a consumer declares intent
   in natural language and an agent resolves it to a contract submission.
   Trust model: trust and always verify on the platform side. Stateless
   agents; all state lives in the platform.

The developer and agentic surfaces are parallel paths, not a progression.
Both end in a contract submission that enters the same pipeline.

## 2. Primitives

Single-purpose, substrate-agnostic modules. Locked commitments:

- No inter-primitive references. A primitive may call substrate data sources.
- Semver with three triggers: interface → MAJOR, behavior → MINOR,
  lifecycle → PATCH.
- Immutability on publication.
- 12-month deprecation window.
- AI refinement is a flag, triggered by a joint operational condition
  (N ≥ 50 consecutive zero-rollback changes, no primitive/module incident in
  6 months, Infra & Ops unilateral override).
- A primitive's interface is defined against the Target Stack (substrate-
  agnostic), not against any substrate's variable block directly.

## 3. Modules

Patterns that combine primitives into deployable shapes. Locked commitments:

- One codebase maps to one canonical module (default); `multiStack: true`
  is permitted only for (a) a DR-region mirror, (b) a time-boxed
  experimental stack (TTL ≤ 30 days), or (c) explicit Infra & Ops approval
  with a documented justification.
- A module references registered primitives only (max depth 5).
- Pipeline quality checks: secrets-in-plaintext, public ingress, IAM
  wildcard, KMS key reference, tag compliance, naming convention.
- Restricted from module patterns: IAM principal creation, network boundary
  creation, key/secret creation, external data transfer.
- Auto-promote after 3 observed usages.
- A module's pattern tree wires field is defined against the stack's
  relationship type, not against any substrate's module block. The stack →
  substrate translation is the substrate adapter's job (§12). The pattern
  pipeline itself is substrate-agnostic.

## 4. Developer Surface

- Tag-based reference to the central pipeline template.
- Developer-owned workflow file, no platform auto-sync.
- Tag mutability for production-bound references: tag for dev/qa, SHA for
  prod. The platform provides a CLI command that resolves the current tag
  to its SHA for prod-bound workflows.

## 5. Agentic Surface

- Hybrid runtime: skill as markdown, agent as executor.
- Trust model: trust and always verify on the platform side.
- Skill envelope (4 dimensions).
- Stateless agents; all state in the platform.
- Initial skill catalog: web API, worker, scheduled job, static asset,
  basic observability bootstrap. Addition criteria: (a) reviewable for
  sensitive data, (b) expressible as a single contract submission,
  (c) documented use case.
- `profile: agentic` unlocks agentic-specific fields
  (`naturalLanguageIntent`, `confidenceAtSubmission`, `agentTrace`).

## 6. Cross-Cutting — Central Pipeline Template

- JSON Schema (draft 2020-12) with a thin domain-specific wrapper.
- Central repo + generated client libraries.
- Multi-stage validation pipeline: schema → policy → NFR → confidence.
- Distributed enrichment.
- GitOps reconciler + substrate execution layer.
- The pipeline emits a `PolicyCheckResult` record per policy rule evaluated;
  the confidence signal consumes these as one normalized input (§8).

## 7. Cross-Cutting — Contract Schema

- Central repo + generated client libraries.
- Strict fail-fast at the schema stage with reason codes from a published
  vocabulary.
- Per-environment mandatory fields: dev requires stack + environment; qa
  adds `validation.e2eSuite` + `validation.loadTest`; prod adds runbook +
  dashboard + oncall; dr adds `drDrillRef`. `inputs` is always optional.
  `profile: agentic` fields are optional everywhere (`naturalLanguageIntent`
  required when profile is agentic).

## 8. Cross-Cutting — Confidence Signal

- Six canonical inputs: policy, validation, freshness, source, history, NFRs.
- Weighted sum with per-input breakdown.
- Per-environment thresholds: dev ≥ 0.50, qa ≥ 0.75, prod ≥ 0.90, dr ≥ 0.95.
- Structured output: `{ score, band, perInput, reasonCodes }`.
- 1-year storage; no algorithm retraining in v1.
- Halt with explicit reason on missing input.
- Severity → score penalty: critical → hard override to mandatory block,
  high → -0.2, medium → -0.05, low → -0.01, info → 0.0. One critical finding
  hard-overrides the score regardless of all other inputs.
- Thresholds frozen for v1; tuning begins post-v1 with quarterly FP/FN
  tracking per environment. Override authority = Infra & Ops + SRE joint
  sign-off; any override is itself a confidence-event in the audit stream.

## 9. Cross-Cutting — Audit and Evidence Stream

- Every delivery action produces an immutable, hash-chained evidence event.
- The audit stream is the platform's certified record of what happened, when,
  and why.
- Events are written to a DynamoDB outbox and rendered on an evidence
  timeline.

## 10. Cross-Cutting — HITL Matrix

Human-in-the-loop gates for higher environments:

| Environment | Autonomy | Attester | Gate |
|---|---|---|---|
| dev | Full autonomy (no HITL) | — | Confidence ≥ 0.50, all six inputs present |
| qa | Held for attestation | QA | Platform-runner deployment approval + full QA matrix |
| prod | Held for attestation | SRE | Platform-runner deployment approval + full SRE matrix |
| dr | Held for attestation | SRE | Platform-runner deployment approval + dr-drill evidence |

Staging does not exist. Dev is the only autonomous environment and absorbs
integration, contract, security smoke, and performance smoke validation.

- Pre-execution gate model. 1 business day = warn + escalate; 2 business
  days = auto-freeze + re-submit. Rejection extends the audit chain; no
  partial deploy to roll back.
- Separation of duties: the platform-internal identity record in the
  DynamoDB outbox enforces `qaApprover ≠ prodApprover` for the same contract.

## 11. Cross-Cutting — Separation of Duties

- CODEOWNERS routes the right reviewer to the right environment.
- The DynamoDB outbox enforces identity distinctness across environment
  approvers.

## 12. Cross-Cutting — Substrate Execution

The technical execution layer. Primitives and modules are substrate-agnostic
in shape; substrate adapters are the only substrate-specific component.

The architecture defines a **Target Stack** — a substrate-neutral
description of:

- The resources to create (typed against the stack schema).
- Their relationships (the module's pattern tree).
- Their inputs (wired from the contract).
- Policy hooks (the points in the pattern where policy checks attach).

The registry, the module pattern tree, the contract schema, and the
`PolicyCheckResult` schema are all defined against the stack schema. None is
defined against any specific substrate.

**v1 implementation reality:** the stack is shaped to round-trip cleanly to
Terraform because there is no other adapter to differentiate from. As
additional adapters appear, the stack gets more expressive and the adapters
gain translation logic, but the primitive content, the module pattern tree,
and the contract schema do not change. This is the design that prevents a
polyglot mess.

The substrate adapter:

- Translates the stack-typed module pattern tree to a substrate root module
  that calls the primitive modules.
- Is a thin layer. It does not own primitive/module content; it only
  translates.
- Is the only substrate-specific code in the platform.

Policy checks run on the substrate plan output. Results are normalized to
`PolicyCheckResult` records by a policy adapter. The confidence signal
consumes the union of all `PolicyCheckResult` records, regardless of engine
— substrate-agnostic over its inputs, matching the module model's
substrate-agnosticism over its outputs.

## 13. Cross-Cutting — Platform Runners

The platform runs on platform-managed runners (GitHub Actions in
production). Runner-specific code = workflow YAML, OIDC trust, CODEOWNERS,
environments. The contract schema, stack, `PolicyCheckResult`, confidence
signal, and audit stream are portable (runner-agnostic); a second runner
platform needs a runner adapter + workflow-template translator, with no
change to the modules/stack/confidence/audit.

## 14. Versioning

- Primitives and modules use semver: interface → MAJOR, behavior → MINOR,
  lifecycle → PATCH.
- A module pins primitives by `name@semver`; the resolver picks the highest
  compatible.
- A MAJOR bump requires a new registry entry (immutable publication); the
  old entry enters a 12-month deprecation window.
- The central deploy pipeline is referenced by a floating MAJOR + MINOR tag
  (e.g. `@v1.6`); patch fixes flow within the tag, breaking changes land
  under the next MINOR tag.

See [Versioning](pipeline/versioning) for the consumer-facing details.

## 15. OpenTofu

Not in v1. The substrate abstraction (§12) makes OpenTofu a future adapter,
not an architecture change. Revisit when an OpenTofu adapter is requested.