acdl/.ciagent/ARCHITECTURE.md

ACDL — Architecture (v1.1 target)

Target architecture for the real Agentic Cloud Delivery Platform. Source of truth for how: docs/architecture.md (v0.2) is the upstream draft; this file is the ACDL-repo operating copy, refined at phase boundaries. Where this file and docs/vision.md conflict, the vision wins.

Status

Architecture is at v0.2 upstream (docs/architecture.md). Milestone v1.1 finalizes it to v1.0 in Phase 07 by resolving the 11 open decisions (see PROJECT.md open-decision resolutions table). This file records the locked commitments and the v1.1 spike scope.

Overview

The platform is four layers + six cross-cutting concerns. The sixth concern — the substrate abstraction (§12) — is first-class, not an implementation detail. The vision's "Two Consumer Surfaces, One Platform" tenet binds everything: L3A and L3B converge on the same contract schema, the same policy envelope, and the same evidence stream.

                  ┌──────────── acdl-contracts ────────────┐
  Developer ───▶  │ commit contract.yaml                   │   (L3A)
  Citizen dev ──▶ │ Issue → agent → contract.yaml            │   (L3B)
                  └────────────────┬───────────────────────┘
                                   │ (push)
                                   ▼
                         ┌──────────────────────┐
                         │ central pipeline      │
                         │ (acdl repo, Gitea      │
                         │  Actions / act_runner) │
                         └────────┬─────────────┘
                                  │
        ┌─────────────────────────┼─────────────────────────┐
        ▼                         ▼                           ▼
  contract→IR resolution   policy (Checkov/Kyverno)    confidence signal
        │                         │                           │
        ▼                         ▼                           ▼
  Terraform adapter ──▶ terraform plan ──▶ PolicyCheckResult ──▶ {score,band}
        │                                                                │
        ▼                                                                ▼
  dev (autonomous, ≥0.50)   qa (HITL, ≥0.75)   prod (HITL, ≥0.90)   dr (HITL, ≥0.95)
        │
        ▼
  DynamoDB outbox ──▶ S3 Object Lock (7-yr, source of truth) ──▶ GitHub audit repo (hot index)
        │
        ▼
  acdl-evidence (timeline UI)

Layers

Layer 1 — Foundational Primitives

Single-purpose, substrate-agnostic primitive modules. L1 modules do not compose with other L1s; L1 takes its environment as input. The L1 interface is defined against the Target Stack IR, not against Terraform directly (the IR is shaped to round-trip to Terraform in v1, per §12.1).

• No inter-L1 references. L1 may call Terraform data sources.
• Semver: interface → MAJOR, behavior → MINOR, lifecycle → PATCH (W3.D).
• Immutability on publication. 12-month deprecation window.
• AI refinement is a flag; the trigger is the W1.A joint condition.

Layer 2 — Composed Stacks

Combine L1 primitives into deployable shapes. Each codebase maps to one canonical L2 stack (multiStack: true only per W1.B). Shape X (parameterized module) or Shape Y (thin-composition layer). Hierarchical composition, max depth 5, only registered L1s. The thin-composition tree's wires field is defined against the IR's relationship type, not a Terraform module block.

Pipeline quality checks: secrets-in-plaintext, public ingress, IAM wildcard, KMS key reference, tag compliance, naming convention. Restricted from thin-composition: IAM principal creation, network boundary creation, key/secret creation, external data transfer. Auto-promote after 3 observed usages.

Layer 3A — Developer Consumer Surface

Tag-based reference to the central pipeline template. Developer-owned workflow file, no platform auto-sync. L3A and L3B are parallel paths, not a progression. W2.A (Path B): tag for dev/qa, SHA for prod; platform CLI resolves tag→SHA for prod-bound workflows.

Layer 3B — Agentic Consumer Surface

Hybrid runtime, skill as markdown, agent as executor. Trust model: trust and always verify on the platform side. Skill envelope (4 dimensions). Stateless agents, all state in the platform. profile: agentic marker unlocks naturalLanguageIntent, confidenceAtSubmission, agentTrace. Initial skill catalog (BA.A): web API, worker, scheduled job, static asset, basic observability bootstrap.

Environment progression:

Environment	Autonomy	Attester	Gate
dev	Full autonomy (no HITL)	—	Confidence ≥ 0.50, all six inputs present
qa	Held for attestation	QA	GitHub Deployment approval + full QA matrix (§10)
prod	Held for attestation	SRE	GitHub Deployment approval + full SRE matrix (§10)
dr	Held for attestation	SRE	GitHub Deployment approval + dr-drill evidence

Staging is removed. Dev is the only autonomous environment.

Cross-cutting concerns

Central pipeline template (§6)

JSON Schema (draft 2020-12) with a thin domain wrapper. Central repo + generated client libraries. Multi-stage validation: schema → policy → NFR → confidence. Distributed enrichment. GitOps reconciler (K8s API; cdlc-gitops state → CRDs) + Terraform execution layer (§12.5). The pipeline emits one PolicyCheckResult per policy rule; the confidence signal consumes them as one normalized input.

Contract schema (§7)

Central repo + generated client libraries. Strict fail-fast at schema stage, multi-stage validation with reason codes from a published vocabulary. W3.E: per-env mandatory inputs —

• dev: stack, environment
• qa adds: validation.e2eSuite, validation.loadTest
• prod adds: runbook, dashboard, oncall
• dr adds: drDrillRef
• inputs always optional; profile: agentic fields optional everywhere.

Confidence signal (§8)

Six canonical inputs, weighted sum with per-input breakdown. Per-env thresholds: dev ≥ 0.50, qa ≥ 0.75, prod ≥ 0.90, dr ≥ 0.95. Structured output { score, band, perInput, reasonCodes }. 1-year storage, no retraining in v1. Halt with explicit reason on missing input.

Policy input = list of PolicyCheckResult records (engine-agnostic). Severity → penalty: critical → hard override to mandatory block; high → -0.2; medium → -0.05; low → -0.01; info → 0.0. One critical finding hard-overrides the score regardless of all other inputs.

BA.B: thresholds frozen for v1; tuning begins v1.2 (quarterly FP/FN tracking; override = Infra & Ops + SRE joint sign-off, itself a confidence-event).

Audit and evidence stream (§9)

Tiered ledger: S3 with Object Lock in compliance mode (cold, source of truth, 7-year retention) + GitHub audit repo (acdl-evidence, hot query index, not part of the chain). Daily checkpoints. Event schema: JWS detached signature, prev_event_hash chain, controlled-vocabulary event_type. Outbox pattern: local durable outbox + async worker.

Outbox database = DynamoDB. RPO = 0 (synchronous write to local outbox before contract submission ack); RTO = async worker's dead-letter recovery. Single-region in v1. The outbox also stores per-contract QA and prod approver identities (the only durable record outside GitHub's audit log).

Human-in-the-Loop mechanics (§10)

Pre-execution gates. qa, prod, dr are PR-based attestation gates backed by GitHub Environments with required reviewers. No partial deployment to roll back on rejection (qa, prod); dr is a separate GitHub Deployment against a separate cluster/region.

Reviewer routing: GitHub CODEOWNERS + Environment required reviewers (qa → QA; prod → SRE; dr → SRE). CODEOWNERS routes, does not enforce identity distinctness.

Separation of duties (platform-internal, not GitHub-native, not Kyverno in v1): on dev→qa promotion the platform writes the QA approver's GitHub identity to the DynamoDB outbox keyed by contractId; on qa→prod it reads the stored QA approver and the new SRE approver; if equal, it blocks, emits SEPARATION_OF_DUTIES_VIOLATION, and routes a halt artifact to SRE on-call.

Full 8-concern attestation matrix (functional, performance, security posture, contract NFRs, operational readiness, incident response, capacity/cost, resilience) — see docs/architecture.md §10.4.

Timeout: 1 business day = warn + escalate; 2 business days = auto-freeze + re-submit (linked via supersedes). Rejection returns the contract to HELD; the audit chain is extended, not torn up.

Agentic stack (§11)

Hybrid runtime: platform-managed control plane + consumer-owned agent. Versioned, signed skill catalog over MCP. Skill envelope enforced on invocation and result submission. Consumer-owned skill execution; the platform does not run the skill. Stateless agents, all state in the platform. Skills are reviewed for sensitive data before release (Infra & Ops owns the review; it is the mandatory release gate).

Substrate execution (§12) — the binding constraint

Target Stack IR (locked): a substrate-neutral description of resources (typed inputs/outputs/NFRs), relationships (single parent per child), composition (tree, max depth 5), and policy hooks. The L1 registry, L2 thin-composition tree, contract YML, and PolicyCheckResult schema are all defined against the IR — none against any specific substrate.

Substrate adapters are the only substrate-specific code. An adapter compiles the IR into a substrate execution plan. v1 ships exactly one adapter: the Terraform adapter. v2+ may add OpenTofu, Pulumi, K8s CRDs without architectural change.

v1 reality: the IR is shaped to round-trip cleanly to Terraform (nearly isomorphic). As more adapters appear, the IR gets more expressive and the adapters gain translation logic; the L1 content, the YML standard, and the thin-composition tree do not change.

Terraform adapter (v1): translates IR-typed L1 interface → Terraform variable/output blocks; IR-typed L2 thin-composition tree → Terraform root module; IR-typed relationships → module references; emits a terraform plan from the IR. The adapter is a thin layer; it does not own L1/L2 content.

State storage: S3 (state) + DynamoDB (locking), cloud-managed, single-region in v1.

Policy toolchain: Checkov for Terraform plan policy (the L2 checks + tag/naming); Kyverno for K8s-native/platform-internal policy; OPA reserved for cross-resource cases, explicitly last resort.

Policy result normalization (§12.6): the confidence signal consumes a normalized PolicyCheckResult schema, not raw engine output.

{
  "contractId": "uuid",
  "evaluatedAt": "ISO-8601",
  "engine": "checkov | kyverno | opa",
  "ruleId": "CKV_AWS_24 | KYVERNO_NO_PRIVILEGED | ...",
  "severity": "critical | high | medium | low | info",
  "result": "pass | fail | skipped | error",
  "message": "human-readable",
  "evidence": { "...engine-specific, opaque to the signal..." },
  "resourceRef": "IR-typed resource identifier"
}

Execution layer: GitHub/Gitea Actions in the central pipeline repo. State locking via DynamoDB. AWS credentials via OIDC federation — long-lived credentials are forbidden (§12.5). The platform does not run terraform apply against a developer's workstation; all execution is in the central pipeline.

Registry maintenance: L1 publication updates the L1 registry in the same PR. The registry is the IR-typed contract, not a Terraform-specific variable schema.

Contract→IR resolution: the contract declares intent in IR-typed terms; the pipeline resolves it to a target stack (list of L1 instances + inputs + relationships); the Terraform adapter compiles the target stack to a plan.

v1.1 spike scope

The spike (Phases 08–10) materializes the minimum that proves the IR commitments hold (no polyglot mess):

• One L1: l1-s3 (IR-typed interface; the only AWS resource in the spike).
• One L2 thin-composition: l2-static-assets (references l1-s3 only).
• Terraform adapter: IR → terraform plan against AWS via OIDC.
• One contract submission → contract→IR → terraform plan → Checkov PolicyCheckResult → confidence signal → evidence event to the DynamoDB outbox.
• State: S3 + DynamoDB (real AWS, single-region).

Out of spike scope: full HITL matrix wiring, Kyverno, OPA, MCP skill catalog, GitOps reconciler, multi-region, prod/dr environments, the 5-skill L3B catalog. Those are post-spike (v1.2+) platform build-out.

Gitea API surface (carried from v1.0, refined)

Capability	Gitea support	ACDL approach (v1.1)
Org-scoped repo create	POST /api/v1/orgs/{org}/repos	Used for any new repos
Native Pages	None	Serve acdl-evidence via raw file URLs (unchanged from v1.0)
Environments API	None; act_runner ignores environment:	Model HITL gates via workflow_dispatch approval inputs (v1.0 D-013 pattern) — refined in Phase 07 for the real pre-execution gate model
repository_dispatch	Not supported	Cross-repo trigger via workflow_dispatch API (unchanged)
Reusable workflows	Supported	acdl/.gitea/workflows/pipeline.yml via uses: ...@<ref>
id-token: write / OIDC	Not supported (RESEARCH TARGET 1, conf 0.95). Gitea docs list id-token as an unsupported GitHub-only scope; open proposal go-gitea/gitea#33681; draft PR go-gitea/gitea#36988 unmerged. Even Gitea's own CI uses long-lived AWS keys (issue #37980).	Spike waiver D-039: per-run-rotated long-lived key (rotated after each run by scripts/rotate_spike_key.sh). Real OIDC deferred to v1.2, blocked on PR #36988.
actions/configure-aws-credentials	Unusable without OIDC	Spike uses static AWS creds from a (rotated) Gitea Actions secret via the aws-actions/configure-aws-credentials@v4 access-key-id/secret-access-key inputs, or plain AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY env vars. v1.2 switches to role-to-assume when OIDC lands.

Branch pinning rule (refined for W2.A)

• Dev/qa contracts reference the reusable workflow by tag (@v1.1-spike).
• Prod-bound workflows reference by SHA; the platform CLI (platform/cli/resolve-tag.ts, Phase 07) resolves the current tag to its SHA. (Spike scope: the CLI is a stub; the real CLI lands in v1.2.)

Verification toolchain

ACDL has no package.json. The verification gate substitutes:

• typecheck: terraform validate, python3 -m py_compile, JSON Schema validation (ajv or python -m jsonschema) against schemas/.
• test: per-phase scripts/verify_phaseNN.sh (Phase 06: archive integrity; Phase 07: schema validation + decision-resolution completeness; Phase 08: OIDC assume-role + state backend; Phase 09: IR + L1 + adapter terraform plan; Phase 10: end-to-end contract submission).
• build: terraform init (real build for the spike).
• See PERSONAS.md verification_toolchain.

Build order (v1.1)

1. Phase 06 — archive demo, reorient repo.
2. Phase 07 — finalize architecture v1.0; author schemas + designs.
3. Phase 08 — AWS OIDC bootstrap (use temp key once, rotate).
4. Phase 09 — IR + l1-s3 + Terraform adapter → terraform plan.
5. Phase 10 — l2-static-assets + contract→IR → end-to-end spike.
6. COMPLETE gate — review → ship v1.2.0 → audit. DONE.

v1.2 build-out scope

v1.2 takes the v1.1 spike (dev-only, plan-only, single S3 L1) to a real, simpler, better-documented platform that delivers a microservice to AWS ECS Fargate end-to-end. The locked architecture (§1–§12) is unchanged — v1.2 extends the implementation, not the design.

In scope (five axes, user-directed 2026-07-21)

1. Re-evaluate the current state. go-gitea/gitea#36988 (OIDC for Gitea Actions) re-checked 2026-07-21: still open (last updated 2026-05-27, not merged). Real OIDC remains deferred to v1.3+; v1.2 extends the D-039 per-run-rotated-key waiver as D-047. The waiver continues to satisfy §12.5's intent (no persistently long-lived key): the spike key is rotated after each run by scripts/rotate_spike_key.sh, and Phase 12 tightens the IAM scoping + rotation hygiene.
2. NFR improvements on the existing spike. Least-privilege IAM audit of spike_runner_policy.json; idempotent create_state_backend.py / create_iam_user.py; proper exit codes / error handling; P1-1 redaction (two AWS access key IDs in .ciagent/VERIFY.md Phase 09 narrative).
3. Streamline / simplify the current setup. Consolidate run_spike_plan.sh + run_spike_e2e.sh into one scripts/run_platform.sh; remove dead code and stale platform/ paths.
4. README.md fully up to date on how the platform works. Reflect v1.1 complete; document the actual spike flow, scripts/run_platform.sh, the real repo layout, and the v1.2 objective.
5. Bootstrap a consumer repo with a basic microservice deployed to ECS end-to-end. New Gitea repo acdl-consumer-microservice (org continuous-intelligence); new IR-typed L1s (l1-vpc, l1-ecs-cluster, l1-ecs-service, l1-iam-role, l1-alb, l1-ecr); new l2-microservice thin-composition; one contract submission → terraform apply (dev, autonomous per §10, confidence ≥ 0.50) → a live ECS Fargate service serving HTTP 200 → evidence event to the DynamoDB outbox → acdl-evidence timeline.

Substrate extension (ECS Fargate)

The Terraform adapter (§12) remains the only substrate-specific code. v1.2 expands the adapter TYPE_MAP to cover the six new ECS-shaped IR resource types. The L1 interface shape (IR-typed inputs/outputs/NFRs, registered in modules-ir/registry.json) is unchanged — only the set of registered L1s grows. The IR commitments (REQ-28) continue to hold: modules-ir/, schemas/, contracts/, core/confidence_signal.py, core/contract_resolver.py, core/outbox_writer.py remain substrate-agnostic.

terraform apply (dev only)

v1.2 lifts the substrate execution from plan to apply for the dev environment only. Dev is autonomous per §10 (confidence ≥ 0.50, no HITL). apply for qa/prod/dr remains HITL-gated and out of scope for v1.2. The apply result (resources created, plan diff) is captured in the evidence stream as a terraform.apply event.

Out of scope for v1.2 (deferred to v1.3+)

Feature	Reason
Real OIDC federation	go-gitea/gitea#36988 still open. v1.2 extends D-039 waiver (D-047); real OIDC is v1.3+.
Full HITL matrix wiring (qa/prod/dr)	v1.2 is dev-only autonomous apply; HITL wiring is v1.3.
Kyverno + OPA policy engines	v1.2 keeps Checkov only; Kyverno/OPA are v1.3.
MCP skill catalog + real L3B agent	v1.2 keeps the L3B stub; the 5-skill catalog is v1.3.
Audit ledger build-out (S3 Object Lock + JWS + async worker + DLQ + daily checkpoints)	v1.2 keeps the v1.1 outbox; the regulatory ledger is v1.3.
Multi-region state / outbox	Single-region in v1 (§9, §12.3); multi-region is v1.3+.
Prod/dr environments	v1.2 is dev-only; prod/dr are v1.3.
GitOps reconciler (ArgoCD/Flux)	v1.3+.

Build order (v1.2)

1. Phase 11 — re-eval #36988 + NFR audit + simplification findings + README rewrite.
2. Phase 12 — NFR harden + simplify (idempotent bootstrap, one run_platform.sh, IAM audit, redactions).
3. Phase 13 — six ECS L1s + adapter TYPE_MAP expansion.
4. Phase 14 — l2-microservice + contract schema extension.
5. Phase 15 — consumer repo + terraform apply (dev) → live ECS service.
6. Phase 16 — capstone e2e: consumer commit → live HTTP 200 → evidence → timeline.
7. COMPLETE gate — review → ship v1.3.0 → audit.

v1.8 Architecture Addendum

Milestone v1.8 (complete, tag v1.8.0). Adds encryption-by-default, deletion-protection-by-default, uptime monitoring, decommission alias, engineering standards, and path documentation.

New Primitives

• kms-key (aws:kms:key) — Per-stack customer-managed KMS key with enable_key_rotation = true. One key per L2 deployment (no shared keys). Wired into both L2 compositions as a child, with its kms_key_arn output connected to all children's kms_key_arn input. Adapter emits aws_kms_key + enable_key_rotation.
• uptime (aws:ecs:uptime-service) — Uptime-kuma on ECS Fargate with a feature flag (feature_flag_enabled), monitored endpoints (HTTP/DNS/TCP), alert channels (Teams/email/SMS/GitHub issues). Deployed by default after any L2 module with a separate terraform state. When the feature flag is false, the adapter emits no resources.

Encryption by Default

All 12 L1 primitives have encryption_enabled NFR (default true). Primitives with at-rest data (s3, rds, ecr, ecs-service, ecs-cluster) have an optional kms_key_arn input. The adapter emits encryption blocks (SSE-KMS for S3, storage_encrypted for RDS, encryption_configuration for ECR) referencing the per-stack CMK when provided. Managed KMS fallback with stderr warning for standalone L1 deployments.

Deletion Protection by Default

All 12 L1 primitives have deletion_protection NFR (default true). The adapter emits lifecycle { prevent_destroy = true } when true. L2 modules expose a features.deletion_protection flag (default true) propagated to all children via the resolver. Setting inputs.deletion_protection: false in the contract disables it for the whole stack.

Decommission Alias

A mode: decommission on the deploy pipeline implements a 2-step destroy:

1. Disable deletion protection (resolve with deletion_protection: false, terraform plan/apply, HITL SRE gate via GitHub environment).
2. Zero counts + destroy (decommission_transform zeroes all scalable counts, terraform plan/apply, second HITL SRE gate).

CMDB validation via DynamoDB acdl-change-requests table. The Lambda validate_change_request action queries the table and asserts status == "approved" + consumerRepo match.

Adapter Expansion

TYPE_MAP grew from 16 to 19 entries (+ aws:kms:key, aws:kms:alias, aws:ecs:uptime-service). Specialized emission branches added for KMS key rotation, S3 SSE-KMS configuration, uptime ECS Fargate task, and prevent_destroy lifecycle on all resources.

Pipeline Stages

The deploy pipeline grew from 8 to 9 stages (+ deploy-uptime after publish-outputs). The deploy-uptime stage constructs a synthetic uptime contract from the L2 stack outputs, resolves + adapts it to a separate terraform state directory, and publishes the uptime URL via PR comment.

Forge-Agnostic API URLs

The platform Lambda (contract_ingestor.py) reads GITHUB_API_BASE env for forge-agnostic API URLs. GitHub uses /search/issues; Gitea uses /repos/{owner}/{repo}/issues. Detection via /api/v1 in the base URL.