Jon Chery
930c24be6d
---ci---
project: acdl
phase: 0
milestone: v1.1
status: research
research_targets: 10
top_finding:
- TARGET 1 (conf 0.95): Gitea Actions does NOT support id-token: write / OIDC.
Real OIDC deferred to v1.2 (blocked on go-gitea/gitea#36988).
Spike adopts D-039: per-run-rotated long-lived key waiver.
decisions_adopted: [D-039, D-040, D-041, D-042, D-043, D-044, D-045, D-046]
risks:
- R-1 (highest): OIDC gap -> key-rotation waiver (mitigated by D-039)
- R-2: Checkov terraform_plan edge cases (mitigated by ACDL_TAG_NAMING skipped record)
- R-3: 6 confidence inputs are a recommendation (D-040 adopted in PROJECT.md)
---/ci---
RESEARCH.md (1330 lines) authored by ci-researcher. Key findings:
- TARGET 1: Gitea Actions does NOT support emitting an OIDC id-token
(Gitea docs list id-token as unsupported; go-gitea/gitea#33681 open;
draft PR #36988 unmerged; Gitea's own CI still uses long-lived keys
per #37980). Real-AWS OIDC is unachievable in this environment for v1.1.
- TARGETS 2-10: Terraform/Checkov install steps, IR prior art, Checkov
adapter, DynamoDB outbox, 6-input confidence signal, tiered ledger
(Object Lock deferred), HITL/SoD, contract schema, archive strategy
— all resolved with concrete sketches in RESEARCH.md.
Adopted D-039..D-046 in PROJECT.md. Re-scoped Phase 08 (ROADMAP) and
REQ-23 (REQUIREMENTS) to reflect the OIDC blocker: the spike uses a
per-run-rotated long-lived key (D-039) instead of OIDC; real OIDC is a
v1.2 deliverable. ARCHITECTURE.md Gitea API surface table updated with
the OIDC verdict.
228 lines
20 KiB
Markdown
228 lines
20 KiB
Markdown
# ACDL — Agentic Cloud Delivery Platform
|
|
|
|
## Vision / Core Value
|
|
|
|
Consumers declare intent; the platform delivers safe production
|
|
deployment through an agentic stack. The platform absorbs two frictions:
|
|
the cognitive load of getting the infrastructure right, and the
|
|
operational work of getting the change to production safely.
|
|
|
|
Source of truth for **why**: `docs/vision.md`.
|
|
Source of truth for **how**: `docs/architecture.md` + `.ciagent/ARCHITECTURE.md`.
|
|
Where the two conflict, the vision wins.
|
|
|
|
## North Star
|
|
|
|
A merged change progresses through lower environments end-to-end without a
|
|
platform engineer joining a thread, approving a ticket, or manually
|
|
triggering a stage gate. A non-technical consumer ships a production
|
|
deployment by declaring intent — without authoring a workflow, a
|
|
configuration file, or a Terraform module. Every production change is
|
|
traceable to a human attestation and an immutable evidence stream.
|
|
|
|
## Core Tenets (from `docs/vision.md`)
|
|
|
|
1. **Operations are Declared, Not Executed.** Consumers define what they
|
|
need; the platform reconciles, provisions, and progresses.
|
|
2. **The Delivery Lifecycle is a Sovereign Boundary.** The platform
|
|
governs infra and delivery; it does not penetrate upstream product/SDLC.
|
|
Integration is only through validated, published contracts.
|
|
3. **Lower Environments are Autonomous; Higher Environments are Attested.**
|
|
Dev = zero-touch agentic. QA/prod/dr = deliberate human attestation, not
|
|
rubber stamps.
|
|
4. **Safety is Computed, Not Assumed.** Every action produces a measurable,
|
|
explainable confidence signal. The signal is the platform's certified
|
|
answer to "is this safe to proceed?"
|
|
5. **Infrastructure is Consumed, Not Maintained.** Compute is abstract,
|
|
containerized, or serverless. No node/OS/bare-metal lifecycle.
|
|
6. **Two Consumer Surfaces, One Platform.** Technical developers (L3A) and
|
|
non-technical consumers (L3B) converge on the same contract schema, the
|
|
same policy envelope, and the same evidence stream.
|
|
|
|
## Domain Boundaries
|
|
|
|
- **In scope:** environment progression; cloud resource lifecycle; operational
|
|
security and observability NFRs; policy enforcement; immutable audit
|
|
lineage; confidence frameworks; two consumer surfaces (developer + agentic).
|
|
- **Out of scope:** application business logic; IDE workflows; product
|
|
backlog / sprint planning; compute requiring node-level or OS-level management.
|
|
- **Interface:** upstream systems integrate through a strict contract
|
|
boundary. The platform validates, enriches with operational standards,
|
|
and reconciles the target state.
|
|
|
|
## Objective for Milestone v1.1
|
|
|
|
Finalize the architecture to v1.0 (resolve all 11 open design decisions in
|
|
`docs/architecture.md` §13) and prove the locked commitments with one
|
|
end-to-end v1 implementation spike:
|
|
|
|
- **One L1 module** (`l1-s3`) — substrate-agnostic, IR-typed interface.
|
|
- **One L2 thin-composition** (`l2-static-asset`) — references the L1.
|
|
- **Terraform adapter** — compiles the IR to a real `terraform plan`
|
|
against AWS via OIDC (no long-lived credentials, per §12.5).
|
|
- **One contract submission** → contract→IR resolution →
|
|
`terraform plan` → PolicyCheckResult (Checkov) → confidence signal →
|
|
evidence event to the DynamoDB outbox.
|
|
|
|
The spike validates the architecture's claim that the IR-shaped commitments
|
|
do not require a polyglot mess (`docs/architecture.md` §14, step 2).
|
|
|
|
## Milestone v1.1 Phases
|
|
|
|
| Phase | Name | Goal |
|
|
|-------|------|------|
|
|
| 06 | archive-demo-and-reorient | Move the v1.0 demo (`modules/`, `scripts/`, `evidence-ui/`, `contracts/`, demo workflows) to `demo/`; establish the new repo layout (`platform/`, `schemas/`, `adapters/`, `terraform/`, `modules-ir/`); rewrite README. |
|
|
| 07 | architecture-v1-finalization | Resolve the 11 open decisions → architecture v1.0. Author IR JSON Schema, PolicyCheckResult schema, contract schema, confidence-signal spec, HITL matrix, outbox/ledger design under `schemas/` + `platform/`. |
|
|
| 08 | aws-oidc-bootstrap | One-shot use of a temporary long-lived key (waiver D-034) to create an IAM role + OIDC trust policy for the act_runner, an S3 state bucket, and a DynamoDB lock table. Rotate the key. Verify the runner assumes the role via OIDC with no long-lived secret. |
|
|
| 09 | v1-spike-ir-and-l1-and-adapter | Target Stack IR; one real L1 (`l1-s3`) with IR-typed interface; L1 registry; Terraform adapter (IR → Terraform var/output + `terraform plan`) running against AWS via OIDC. |
|
|
| 10 | v1-spike-l2-and-contract-e2e | One L2 thin-composition (`l2-static-asset`) referencing `l1-s3`; contract schema + contract→IR resolution; one end-to-end contract submission → `terraform plan` → Checkov → confidence signal → evidence event to outbox. Verify the IR commitments hold. |
|
|
|
|
Milestone COMPLETE gate: review → ship `v1.2.0` (feature milestone, next
|
|
minor per ship.md) → audit.
|
|
|
|
## Requirements
|
|
|
|
### v1.0 (Prior milestone — the demo)
|
|
|
|
Status: complete. Tag `v1.1.0`. All REQ-01..15 satisfied by the stub-driven
|
|
executive demo. See `REQUIREMENTS.md` §v1 and the prior decisions table
|
|
appendix below. The demo is **archived** to `demo/` in Phase 06.
|
|
|
|
### v1.1 (Active milestone — architecture finalization + v1 spike)
|
|
|
|
New requirements REQ-16..REQ-2x — see `REQUIREMENTS.md` §v1.1. Summary:
|
|
|
|
- **REQ-16:** Architecture finalized to v1.0 (11 open decisions resolved).
|
|
- **REQ-17:** Target Stack IR defined as JSON Schema; substrate-agnostic.
|
|
- **REQ-18:** PolicyCheckResult normalized schema defined; Checkov adapter.
|
|
- **REQ-19:** Six-input confidence signal specified with per-env thresholds
|
|
(dev 0.50 / qa 0.75 / prod 0.90 / dr 0.95) and severity→penalty mapping.
|
|
- **REQ-20:** Tiered audit ledger design (S3 Object Lock 7-yr + DynamoDB
|
|
outbox, RPO=0, JWS detached signatures, `prev_event_hash` chain).
|
|
- **REQ-21:** Full 8-concern HITL matrix + separation-of-duties design
|
|
(CODEOWNERS + DynamoDB identity-distinctness).
|
|
- **REQ-22:** Contract schema (JSON Schema draft 2020-12) with per-env
|
|
mandatory/optional inputs and `profile: agentic` marker for L3B.
|
|
- **REQ-23:** AWS OIDC bootstrap (IAM role + trust policy for act_runner);
|
|
the long-lived key is used once then rotated (waiver D-034).
|
|
- **REQ-24:** One real L1 module (`l1-s3`) with an IR-typed interface.
|
|
- **REQ-25:** One real L2 thin-composition (`l2-static-asset`) referencing
|
|
`l1-s3`.
|
|
- **REQ-26:** Terraform adapter compiles the IR to a real `terraform plan`
|
|
against AWS via OIDC; state in S3 + DynamoDB.
|
|
- **REQ-27:** One end-to-end contract submission → contract→IR resolution →
|
|
`terraform plan` → Checkov → confidence signal → evidence event to outbox.
|
|
- **REQ-28:** Spike verification proves the IR-shaped commitments hold (no
|
|
polyglot mess; the adapter is the only substrate-specific code).
|
|
|
|
## Constraints
|
|
|
|
- **Forge:** Gitea at `https://git.cloudinit.dev`, org `continuous-intelligence`.
|
|
- **CI runtime:** act_runner / Gitea Actions (reuses GitHub Actions workflow YAML).
|
|
- **Cloud:** AWS via OIDC federation. **Long-lived credentials are forbidden**
|
|
(§12.5). The v1.1 spike uses a temporary long-lived key **once** to bootstrap
|
|
OIDC (waiver D-034), then rotates it.
|
|
- **Substrate:** Terraform adapter in v1 (the only adapter). L1/L2 are
|
|
substrate-agnostic in shape; the adapter is the only substrate-specific code.
|
|
- **State:** S3 (state files) + DynamoDB (locking), single-region in v1.
|
|
- **Environments:** dev (autonomous) → qa (QA HITL) → prod (SRE HITL) → dr
|
|
(SRE HITL). **Staging does not exist** (Path A locked).
|
|
- **Compute:** abstract / containerized / serverless. No VMs, bare metal, OS
|
|
lifecycle.
|
|
- **Autonomy:** Full. Escalation hooks: deploy, delete_data, merge_to_main.
|
|
|
|
## Anti-Goals (from `docs/vision.md` §7)
|
|
|
|
- Not an upstream development platform (no product backlogs, IDE, code authorship).
|
|
- Not a general-purpose AI (autonomy is narrow, bounded by policy envelopes).
|
|
- Not a legacy infrastructure bridge (no VMs/bare metal/OS).
|
|
- Not a permissive delivery highway (no escape hatches past confidence or HITL).
|
|
- Not a mutable audit log (VCS history ≠ regulatory evidence).
|
|
|
|
## Context
|
|
|
|
- The `acdl` repo exists at the org root. `acdl-contracts` and
|
|
`acdl-evidence` exist from the v1.0 demo and continue as the developer
|
|
surface and the audit-timeline host respectively.
|
|
- `docs/vision.md` and `docs/architecture.md` (v0.2) are the upstream
|
|
vision/architecture sources, pulled from `origin/main` at the start of v1.1.
|
|
- The v1.0 demo (tag `v1.1.0`) is the reference of intent — it proved the
|
|
shape (L1/L2/contract/confidence/evidence/HITL) on stubs. v1.1 replaces the
|
|
stubs with the real platform substrate.
|
|
|
|
## Key Decisions (v1.1)
|
|
|
|
Carries forward the still-valid v1.0 decisions (see appendix). New v1.1
|
|
decisions:
|
|
|
|
| ID | Decision | Rationale | Outcome |
|
|
|----|----------|-----------|---------|
|
|
| D-034 | Temporary long-lived AWS key (waiver) used once in Phase 08 to bootstrap OIDC trust; rotated immediately after | §12.5 forbids long-lived creds; OIDC needs one bootstrapping `aws iam` call before the runner can assume a role | Spike achieves real `terraform plan` against AWS without violating the locked target after bootstrap |
|
|
| D-035 | Milestone version = `v1.1` (feature), ship tag `v1.2.0` | Real platform is a breaking reframing of the demo, but treated as the next incremental milestone per user choice; ship.md: feature milestone → next minor | Tag `v1.2.0` on milestone COMPLETE |
|
|
| D-036 | Spike picks `l1-s3` + `l2-static-asset` | Simplest real AWS resource (no IAM/network deps); smallest real `terraform plan`; proves the IR + adapter end-to-end | Spike scope fixed |
|
|
| D-037 | Demo archived to `demo/` (not deleted) | Preserves the working v1.0 demo as intent reference; new platform layout under `platform/`, `schemas/`, `adapters/`, `terraform/`, `modules-ir/` | No churn on demo code; clean separation |
|
|
| D-038 | Open decisions resolved in "accept recommendations + decide rest" mode | User-locked mode: accept architecture's stated recommendations (W1.A, W1.B, W2.A, BA.A); lead-developer decides the remaining 8 (W3.D, W3.E, BA.B, BA.C, BA.D, BA.E, BA.F, OpenTofu timing) with rationale | Architecture reaches v1.0 in Phase 07 |
|
|
| D-039 | Spike-only waiver: per-run-rotated long-lived AWS key. OIDC federation deferred to v1.2, blocked on go-gitea/gitea#36988. | **RESEARCH TARGET 1 verdict (conf 0.95):** Gitea Actions does NOT support `id-token: write` / OIDC token issuance as of Gitea 1.27.x / gitea-runner v2.1.0. GitHub's OIDC pattern is not portable. The waiver satisfies §12.5's *intent* (no persistent long-lived key) for the spike: the key is rotated after each run by `scripts/rotate_spike_key.sh`. v1.2 implements real OIDC when the Gitea PR merges. | Spike achieves real `terraform plan` against AWS without a *persistently* long-lived key; real OIDC is a v1.2 deliverable |
|
|
| D-040 | The 6 confidence-signal inputs are: policy (0.30), validation (0.25), freshness (0.10), source (0.15), history (0.10), nfrs (0.10). Weights frozen for v1, tuned in v1.2 alongside thresholds (BA.B). | Architecture §8 locks "six canonical inputs" but does not enumerate them; RESEARCH TARGET 6 chose the platform-computable subset present in every environment (incl. dev). | Confidence signal (Phase 10) has a concrete input enumeration |
|
|
| D-041 | Spike audit ledger = v1.0 hash chain + DynamoDB outbox + `acdl-evidence` mirror. S3 Object Lock (compliance mode, 7-yr) + JWS (platform KMS key, quarterly rotation) + daily checkpoints are v1.2 build-out, authored as design in Phase 07. | REQ-20 is "design authored," not "implemented." The spike proves the outbox write path; the regulatory ledger is v1.2. | Spike scope stays bounded; REQ-20 satisfied by the Phase 07 design doc |
|
|
| D-042 | HITL approver identity in Gitea = `gitea.actor` of the `workflow_dispatch` run that sets `approve_qa=true`/`approve_prod=true`/`approve_dr=true`. Separation-of-duties reads `approver_qa` from the DynamoDB outbox and compares to the prod-dispatch `gitea.actor`. | Gitea has no Environments API (re-confirmed in RESEARCH); `gitea.actor` is the only approval-identity signal. | SoD design (Phase 07) is concrete for the Gitea forge |
|
|
| D-043 | Tag/naming compliance deferred for the spike: the Checkov adapter emits a single `SKIPPED` PolicyCheckResult (`ruleId: ACDL_TAG_NAMING`, `severity: info`) so the confidence policy input is non-empty. Custom Checkov YAML rule lands in v1.2. | Checkov has no built-in tag-presence check; a custom rule in the spike is scope creep. | Spike's policy input is non-empty without a custom-rule dependency |
|
|
| D-044 | DynamoDB outbox = `PAY_PER_REQUEST`; PK `contractId`, SK `eventType#eventTs`, TTL `expire_at` = now + 365d. No separate async worker/DLQ in the spike (RTO = workflow re-run); v1.2 outbox worker + DLQ is a Phase 07 design artifact. | On-demand is zero-cost-at-idle for the spike's single dev submission. | Spike outbox is minimal; v1.2 worker design authored in Phase 07 |
|
|
| D-045 | Runner tooling: `runs-on: ubuntu-latest`; install `terraform` via HashiCorp apt repo (pin `1.9.*`), `checkov` via pip (pin `>=3.2,<4`, `--break-system-packages`). Neither is pre-installed on the default runner image. | RESEARCH TARGET 2; pinning avoids mid-spike version drift. | Phase 09/10 workflows have a concrete setup step |
|
|
| D-046 | `act_runner` → `gitea-runner` rename: Phase 07 updates docs to use the current name `gitea-runner` (renamed 2026-04 in gitea/runner#850). | RESEARCH TARGET 1 + R-4: naming drift between v1.0 docs and the current runner. | Docs reflect the current binary name |
|
|
|
|
### Open-decision resolutions (Phase 07 deliverable — recorded here for traceability)
|
|
|
|
| ID | Question | Resolution |
|
|
|---|---|---|
|
|
| W1.A | AI-refinement trigger | **Accept recommendation.** Joint condition: N ≥ 50 consecutive changes with zero rollbacks AND no L1/L2 incident in last 6 months AND Infra & Ops unilateral override. |
|
|
| W1.B | Multi-stack edge case rule | **Accept recommendation.** Permitted only for (a) DR-region mirror, (b) time-boxed experimental stack with TTL ≤ 30d, (c) explicit Infra & Ops approval with `multiStack.justification`. |
|
|
| W2.A | Tag mutability for prod | **Accept recommendation (Path B).** Tag for dev/qa, SHA for prod. Platform CLI resolves tag→SHA for prod-bound workflows. Justified by the "Audit truth lives outside the repository" bet. |
|
|
| BA.A | Initial L3B skill catalog | **Accept recommendation.** 5 skills: web API, worker, scheduled job, static asset, basic observability bootstrap. Addition criteria: (a) reviewable for sensitive data, (b) expressible as a single contract submission, (c) documented use case. |
|
|
| W3.D | L1/L2 standard versioning | **Decided.** Semver: interface → MAJOR, behavior → MINOR, lifecycle → PATCH (same as the v1.0 demo D-rule, lifted to the real platform). Pin model: L2 contracts pin L1 by `name@semver`; the resolver picks the highest compatible. Evolution: MAJOR bumps require a new registry entry (immutable publication); old entry enters a 12-month deprecation window. |
|
|
| W3.E | Schema mandatory vs optional inputs | **Decided.** Per-env mandatory table: dev requires `stack` + `environment`; qa adds `validation.e2eSuite` + `validation.loadTest`; prod adds `runbook` + `dashboard` + `oncall`; dr adds `drDrillRef`. `inputs` map is always optional. `profile: agentic` fields (`naturalLanguageIntent`, `confidenceAtSubmission`, `agentTrace`) optional everywhere. |
|
|
| BA.B | Confidence threshold tuning | **Decided.** Starting thresholds frozen for v1. Tuning begins in v1.2: track FP/FN per environment quarterly; override authority = Infra & Ops + SRE joint sign-off; any override is itself a confidence-event in the audit stream. |
|
|
| BA.C | On-call / operational ownership | **Decided.** Platform on-call = Infra & Ops rotation. Escalation: L3A/L3B halt → platform on-call pager (Sev2); consumer-visible outage → consumer on-call (Sev1) with platform on-call support. Consumer on-call relationship is contractual, defined at onboarding (BA.E). |
|
|
| BA.D | Cost / capacity governance | **Decided.** Cloud cost owner = Infra & Ops FinOps. Per-contract consumption reported monthly. Runaway spend: hard halt at 120% of contract-declared budget envelope via the confidence signal (cost is one of the 6 inputs); override = FinOps + SRE joint sign-off. |
|
|
| BA.E | Consumer onboarding | **Decided.** Two paths: developer (L3A) — `getting-started` walks through contract schema + central pipeline template; citizen developer (L3B) — onboarding grants a scoped agent + skill catalog, no workflow authoring. Both end in a sandbox dev submission that must pass the confidence gate before the consumer is promoted. |
|
|
| BA.F | Cross-platform evolution | **Decided.** The contract schema, IR, PolicyCheckResult, confidence signal, and audit stream are portable (substrate- and forge-agnostic). Forge-specific code: workflow YAML, OIDC trust, CODEOWNERS, Environments. A second forge (e.g., GitLab) requires a forge adapter + a workflow-template translator; no change to L1/L2/IR/confidence/audit. |
|
|
| Q1.3 | OpenTofu timing | **Decided (deferred).** Not in v1 or v1.1. The substrate abstraction (§12) makes OpenTofu a future adapter, not an architecture change. Revisit when an OpenTofu adapter is requested; no version committed. |
|
|
|
|
## Appendix — Prior milestone (v1.0 demo) decisions
|
|
|
|
The v1.0 demo (tag `v1.1.0`) carried decisions D-001..D-033. They governed
|
|
the stub-driven executive demo and remain valid **for the archived demo
|
|
under `demo/`**. They are **superseded** by the v1.1 decisions above for the
|
|
real platform. Full text preserved in git history at tag `v1.1.0`.
|
|
|
|
## Operational parameters (CLARIFY auto-resolution, full autonomy)
|
|
|
|
Resolved at the CLARIFY stage to unblock planning. None require user
|
|
sign-off (autonomy = full; all within locked constraints).
|
|
|
|
| Parameter | Value | Rationale |
|
|
|---|---|---|
|
|
| AWS region | `us-east-1` | Default; matches v1.0 demo references; single-region in v1 (§12.3) |
|
|
| Terraform state bucket | `acdl-tfstate-<account-id>-us-east-1` | Namespaced by account id to avoid collision; region-suffixed |
|
|
| Terraform lock table | `acdl-tflock` | DynamoDB; single-region v1 |
|
|
| OIDC IAM role | `acdl-act-runner-role` | Assumed by the act_runner via web-identity |
|
|
| OIDC trust subject | `repo:continuous-intelligence/acdl:ref:refs/heads/main` (+ phase branches) | Least-privilege; refined in Phase 08 |
|
|
| Spike L1 (`l1-s3`) inputs | `bucket_name: string`, `region: string` | Minimal S3 interface per §2 |
|
|
| Spike L2 (`l2-static-asset`) | thin-composition referencing `l1-s3` only; depth 1 | Smallest real plan per D-036 |
|
|
| Spike contract | `contracts/spike.yaml`: `stack: l2-static-asset`, `environment: dev`, `inputs: { bucket_name: acdl-spike-bucket, region: us-east-1 }` | One end-to-end submission (REQ-27) |
|
|
| Spike `terraform` command | `plan` only | `apply` is out of scope (Out of Scope table); HITL-gated in v1.2 |
|
|
| Checkov ruleset (spike) | the 4 L2 checks (secrets-in-plaintext, public ingress, IAM wildcard, KMS key reference) + tag/naming | §3 + §12.4; Kyverno/OPA deferred |
|
|
| v1.0 tags preserved | `v1.0.1`..`v1.0.5`, `v1.1.0` retained | Immutability; demo archive does not rewrite history |
|
|
| Next ship tag | `v1.2.0` | Feature milestone → next minor per ship.md (D-035) |
|
|
|
|
### Items deferred to RESEARCH (not clarifications)
|
|
|
|
- **Gitea/act_runner OIDC support** — does act_runner emit an OIDC
|
|
`id-token`? Determines whether real-AWS plan is achievable in this
|
|
environment or whether a spike-only waiver is needed. Highest-priority
|
|
research target.
|
|
- **Terraform + Checkov availability on the runner image** — install in the
|
|
workflow if missing.
|
|
- **`actions/configure-aws-credentials` action on act_runner** — if
|
|
unavailable, fall back to `aws sts assume-role-with-web-identity` from a
|
|
step. |