acdl/.ciagent/ARCHITECTURE.md

ACDL — Architecture (v1.1 target)

Target architecture for the real Agentic Cloud Delivery Platform. Source of truth for how: docs/architecture.md (v0.2) is the upstream draft; this file is the ACDL-repo operating copy, refined at phase boundaries. Where this file and docs/vision.md conflict, the vision wins.

Status

Architecture is at v0.2 upstream (docs/architecture.md). Milestone v1.1 finalizes it to v1.0 in Phase 07 by resolving the 11 open decisions (see PROJECT.md open-decision resolutions table). This file records the locked commitments and the v1.1 spike scope.

Overview

The platform is four layers + six cross-cutting concerns. The sixth concern — the substrate abstraction (§12) — is first-class, not an implementation detail. The vision's "Two Consumer Surfaces, One Platform" tenet binds everything: L3A and L3B converge on the same contract schema, the same policy envelope, and the same evidence stream.

                  ┌──────────── acdl-contracts ────────────┐
  Developer ───▶  │ commit contract.yaml                   │   (L3A)
  Citizen dev ──▶ │ Issue → agent → contract.yaml            │   (L3B)
                  └────────────────┬───────────────────────┘
                                   │ (push)
                                   ▼
                         ┌──────────────────────┐
                         │ central pipeline      │
                         │ (acdl repo, Gitea      │
                         │  Actions / act_runner) │
                         └────────┬─────────────┘
                                  │
        ┌─────────────────────────┼─────────────────────────┐
        ▼                         ▼                           ▼
  contract→IR resolution   policy (Checkov/Kyverno)    confidence signal
        │                         │                           │
        ▼                         ▼                           ▼
  Terraform adapter ──▶ terraform plan ──▶ PolicyCheckResult ──▶ {score,band}
        │                                                                │
        ▼                                                                ▼
  dev (autonomous, ≥0.50)   qa (HITL, ≥0.75)   prod (HITL, ≥0.90)   dr (HITL, ≥0.95)
        │
        ▼
  DynamoDB outbox ──▶ S3 Object Lock (7-yr, source of truth) ──▶ GitHub audit repo (hot index)
        │
        ▼
  acdl-evidence (timeline UI)

Layers

Layer 1 — Foundational Primitives

Single-purpose, substrate-agnostic primitive modules. L1 modules do not compose with other L1s; L1 takes its environment as input. The L1 interface is defined against the Target Stack IR, not against Terraform directly (the IR is shaped to round-trip to Terraform in v1, per §12.1).

• No inter-L1 references. L1 may call Terraform data sources.
• Semver: interface → MAJOR, behavior → MINOR, lifecycle → PATCH (W3.D).
• Immutability on publication. 12-month deprecation window.
• AI refinement is a flag; the trigger is the W1.A joint condition.

Layer 2 — Composed Stacks

Combine L1 primitives into deployable shapes. Each codebase maps to one canonical L2 stack (multiStack: true only per W1.B). Shape X (parameterized module) or Shape Y (thin-composition layer). Hierarchical composition, max depth 5, only registered L1s. The thin-composition tree's wires field is defined against the IR's relationship type, not a Terraform module block.

Pipeline quality checks: secrets-in-plaintext, public ingress, IAM wildcard, KMS key reference, tag compliance, naming convention. Restricted from thin-composition: IAM principal creation, network boundary creation, key/secret creation, external data transfer. Auto-promote after 3 observed usages.

Layer 3A — Developer Consumer Surface

Tag-based reference to the central pipeline template. Developer-owned workflow file, no platform auto-sync. L3A and L3B are parallel paths, not a progression. W2.A (Path B): tag for dev/qa, SHA for prod; platform CLI resolves tag→SHA for prod-bound workflows.

Layer 3B — Agentic Consumer Surface

Hybrid runtime, skill as markdown, agent as executor. Trust model: trust and always verify on the platform side. Skill envelope (4 dimensions). Stateless agents, all state in the platform. profile: agentic marker unlocks naturalLanguageIntent, confidenceAtSubmission, agentTrace. Initial skill catalog (BA.A): web API, worker, scheduled job, static asset, basic observability bootstrap.

Environment progression:

Environment	Autonomy	Attester	Gate
dev	Full autonomy (no HITL)	—	Confidence ≥ 0.50, all six inputs present
qa	Held for attestation	QA	GitHub Deployment approval + full QA matrix (§10)
prod	Held for attestation	SRE	GitHub Deployment approval + full SRE matrix (§10)
dr	Held for attestation	SRE	GitHub Deployment approval + dr-drill evidence

Staging is removed. Dev is the only autonomous environment.

Cross-cutting concerns

Central pipeline template (§6)

JSON Schema (draft 2020-12) with a thin domain wrapper. Central repo + generated client libraries. Multi-stage validation: schema → policy → NFR → confidence. Distributed enrichment. GitOps reconciler (K8s API; cdlc-gitops state → CRDs) + Terraform execution layer (§12.5). The pipeline emits one PolicyCheckResult per policy rule; the confidence signal consumes them as one normalized input.

Contract schema (§7)

Central repo + generated client libraries. Strict fail-fast at schema stage, multi-stage validation with reason codes from a published vocabulary. W3.E: per-env mandatory inputs —

• dev: stack, environment
• qa adds: validation.e2eSuite, validation.loadTest
• prod adds: runbook, dashboard, oncall
• dr adds: drDrillRef
• inputs always optional; profile: agentic fields optional everywhere.

Confidence signal (§8)

Six canonical inputs, weighted sum with per-input breakdown. Per-env thresholds: dev ≥ 0.50, qa ≥ 0.75, prod ≥ 0.90, dr ≥ 0.95. Structured output { score, band, perInput, reasonCodes }. 1-year storage, no retraining in v1. Halt with explicit reason on missing input.

Policy input = list of PolicyCheckResult records (engine-agnostic). Severity → penalty: critical → hard override to mandatory block; high → -0.2; medium → -0.05; low → -0.01; info → 0.0. One critical finding hard-overrides the score regardless of all other inputs.

BA.B: thresholds frozen for v1; tuning begins v1.2 (quarterly FP/FN tracking; override = Infra & Ops + SRE joint sign-off, itself a confidence-event).

Audit and evidence stream (§9)

Tiered ledger: S3 with Object Lock in compliance mode (cold, source of truth, 7-year retention) + GitHub audit repo (acdl-evidence, hot query index, not part of the chain). Daily checkpoints. Event schema: JWS detached signature, prev_event_hash chain, controlled-vocabulary event_type. Outbox pattern: local durable outbox + async worker.

Outbox database = DynamoDB. RPO = 0 (synchronous write to local outbox before contract submission ack); RTO = async worker's dead-letter recovery. Single-region in v1. The outbox also stores per-contract QA and prod approver identities (the only durable record outside GitHub's audit log).

Human-in-the-Loop mechanics (§10)

Pre-execution gates. qa, prod, dr are PR-based attestation gates backed by GitHub Environments with required reviewers. No partial deployment to roll back on rejection (qa, prod); dr is a separate GitHub Deployment against a separate cluster/region.

Reviewer routing: GitHub CODEOWNERS + Environment required reviewers (qa → QA; prod → SRE; dr → SRE). CODEOWNERS routes, does not enforce identity distinctness.

Separation of duties (platform-internal, not GitHub-native, not Kyverno in v1): on dev→qa promotion the platform writes the QA approver's GitHub identity to the DynamoDB outbox keyed by contractId; on qa→prod it reads the stored QA approver and the new SRE approver; if equal, it blocks, emits SEPARATION_OF_DUTIES_VIOLATION, and routes a halt artifact to SRE on-call.

Full 8-concern attestation matrix (functional, performance, security posture, contract NFRs, operational readiness, incident response, capacity/cost, resilience) — see docs/architecture.md §10.4.

Timeout: 1 business day = warn + escalate; 2 business days = auto-freeze + re-submit (linked via supersedes). Rejection returns the contract to HELD; the audit chain is extended, not torn up.

Agentic stack (§11)

Hybrid runtime: platform-managed control plane + consumer-owned agent. Versioned, signed skill catalog over MCP. Skill envelope enforced on invocation and result submission. Consumer-owned skill execution; the platform does not run the skill. Stateless agents, all state in the platform. Skills are reviewed for sensitive data before release (Infra & Ops owns the review; it is the mandatory release gate).

Substrate execution (§12) — the binding constraint

Target Stack IR (locked): a substrate-neutral description of resources (typed inputs/outputs/NFRs), relationships (single parent per child), composition (tree, max depth 5), and policy hooks. The L1 registry, L2 thin-composition tree, contract YML, and PolicyCheckResult schema are all defined against the IR — none against any specific substrate.

Substrate adapters are the only substrate-specific code. An adapter compiles the IR into a substrate execution plan. v1 ships exactly one adapter: the Terraform adapter. v2+ may add OpenTofu, Pulumi, K8s CRDs without architectural change.

v1 reality: the IR is shaped to round-trip cleanly to Terraform (nearly isomorphic). As more adapters appear, the IR gets more expressive and the adapters gain translation logic; the L1 content, the YML standard, and the thin-composition tree do not change.

Terraform adapter (v1): translates IR-typed L1 interface → Terraform variable/output blocks; IR-typed L2 thin-composition tree → Terraform root module; IR-typed relationships → module references; emits a terraform plan from the IR. The adapter is a thin layer; it does not own L1/L2 content.

State storage: S3 (state) + DynamoDB (locking), cloud-managed, single-region in v1.

Policy toolchain: Checkov for Terraform plan policy (the L2 checks + tag/naming); Kyverno for K8s-native/platform-internal policy; OPA reserved for cross-resource cases, explicitly last resort.

Policy result normalization (§12.6): the confidence signal consumes a normalized PolicyCheckResult schema, not raw engine output.

{
  "contractId": "uuid",
  "evaluatedAt": "ISO-8601",
  "engine": "checkov | kyverno | opa",
  "ruleId": "CKV_AWS_24 | KYVERNO_NO_PRIVILEGED | ...",
  "severity": "critical | high | medium | low | info",
  "result": "pass | fail | skipped | error",
  "message": "human-readable",
  "evidence": { "...engine-specific, opaque to the signal..." },
  "resourceRef": "IR-typed resource identifier"
}

Execution layer: GitHub/Gitea Actions in the central pipeline repo. State locking via DynamoDB. AWS credentials via OIDC federation — long-lived credentials are forbidden (§12.5). The platform does not run terraform apply against a developer's workstation; all execution is in the central pipeline.

Registry maintenance: L1 publication updates the L1 registry in the same PR. The registry is the IR-typed contract, not a Terraform-specific variable schema.

Contract→IR resolution: the contract declares intent in IR-typed terms; the pipeline resolves it to a target stack (list of L1 instances + inputs + relationships); the Terraform adapter compiles the target stack to a plan.

v1.1 spike scope

The spike (Phases 08–10) materializes the minimum that proves the IR commitments hold (no polyglot mess):

• One L1: l1-s3 (IR-typed interface; the only AWS resource in the spike).
• One L2 thin-composition: l2-static-asset (references l1-s3 only).
• Terraform adapter: IR → terraform plan against AWS via OIDC.
• One contract submission → contract→IR → terraform plan → Checkov PolicyCheckResult → confidence signal → evidence event to the DynamoDB outbox.
• State: S3 + DynamoDB (real AWS, single-region).

Out of spike scope: full HITL matrix wiring, Kyverno, OPA, MCP skill catalog, GitOps reconciler, multi-region, prod/dr environments, the 5-skill L3B catalog. Those are post-spike (v1.2+) platform build-out.

Gitea API surface (carried from v1.0, refined)

Capability	Gitea support	ACDL approach (v1.1)
Org-scoped repo create	POST /api/v1/orgs/{org}/repos	Used for any new repos
Native Pages	None	Serve acdl-evidence via raw file URLs (unchanged from v1.0)
Environments API	None; act_runner ignores environment:	Model HITL gates via workflow_dispatch approval inputs (v1.0 D-013 pattern) — refined in Phase 07 for the real pre-execution gate model
repository_dispatch	Not supported	Cross-repo trigger via workflow_dispatch API (unchanged)
Reusable workflows	Supported	acdl/.gitea/workflows/pipeline.yml via uses: ...@<ref>
id-token: write / OIDC	To be confirmed in RESEARCH (highest-priority unknown)	If act_runner supports OIDC, configure AWS trust; if not, use a documented spike-only waiver and revisit in v1.2
actions/configure-aws-credentials	Depends on OIDC support	If OIDC unavailable, fall back to short-lived aws sts assume-role-with-web-identity from a step

Branch pinning rule (refined for W2.A)

• Dev/qa contracts reference the reusable workflow by tag (@v1.1-spike).
• Prod-bound workflows reference by SHA; the platform CLI (platform/cli/resolve-tag.ts, Phase 07) resolves the current tag to its SHA. (Spike scope: the CLI is a stub; the real CLI lands in v1.2.)

Verification toolchain

ACDL has no package.json. The verification gate substitutes:

• typecheck: terraform validate, python3 -m py_compile, JSON Schema validation (ajv or python -m jsonschema) against schemas/.
• test: per-phase scripts/verify_phaseNN.sh (Phase 06: archive integrity; Phase 07: schema validation + decision-resolution completeness; Phase 08: OIDC assume-role + state backend; Phase 09: IR + L1 + adapter terraform plan; Phase 10: end-to-end contract submission).
• build: terraform init (real build for the spike).
• See PERSONAS.md verification_toolchain.

Build order (v1.1)

1. Phase 06 — archive demo, reorient repo.
2. Phase 07 — finalize architecture v1.0; author schemas + designs.
3. Phase 08 — AWS OIDC bootstrap (use temp key once, rotate).
4. Phase 09 — IR + l1-s3 + Terraform adapter → terraform plan.
5. Phase 10 — l2-static-asset + contract→IR → end-to-end spike.
6. COMPLETE gate — review → ship v1.2.0 → audit.