feat(P39): refresh design docs + parameterize adapter (P1-1)
---ci--- project: acdl phase: 39 milestone: v1.9 status: execute ---/ci--- Phase 39 — design-doc-refresh-and-p1-1-parameterization: Design docs (REQ-100, REQ-101): - hitl_matrix_design.md: 'dev-only spike'/'v1.2 wires the gates' framing replaced with v1.9 wired-gates reality; 8-concern matrix marked implemented (offline-testable subset + signed evidence artifacts, D-084); v1.9 wiring section cross-references hitl_gates.py + attestation_matrix.py; approver_dr noted. - audit_ledger_design.md: outbox marked shipped+production since v1.8; S3 Object Lock + JWS + async worker + DLQ + daily checkpoints clearly labeled 'Deferred to a future milestone (D-083)'; RPO/RTO table updated; approver fields note v1.9 hitl_gates.attest. P1-1 adapter parameterization (REQ-102, D-085): - ecs-service interface.json: desired_count (default 1), launch_type (FARGATE), family (app) inputs added. - alb interface.json: load_balancer_type (application), target_type (ip). - adapter.py: hardcoded defaults replaced with inputs.get(<name>, <default>); hardcoded 'acdl-microservice-rt'/'acdl-microservice-igw' Name tags derive from the VPC name input. - contract_resolver.py: child_input_map routes wires to the sub-resource that declares the input (desired_count → aws:ecs:service, family → aws:ecs:task_definition, target_type → targetgroup, etc.). - microservice composition.json: wires added for the new inputs. Tests: +21 (test_p1_1_adapter_parameterization.py, test_design_docs_current.py). 371 passed; run_ci.sh green; run_platform.sh --check-only green; v1.1 S3 regression preserved.
This commit is contained in:
+46
-30
@@ -1,9 +1,11 @@
|
||||
# ACDL Tiered Audit Ledger Design (REQ-20)
|
||||
|
||||
> **Status:** design authored in Phase 07 (milestone v1.1); the spike
|
||||
> (Phases 08-10) implements the **v1.0 hash chain + DynamoDB outbox write**
|
||||
> (D-041); the v1.2 build-out implements S3 Object Lock + JWS + async
|
||||
> worker + DLQ + daily checkpoints.
|
||||
> **Status:** design authored in Phase 07 (milestone v1.1); the
|
||||
> hash-chain + DynamoDB-outbox path is **shipped + production since
|
||||
> v1.8**. The S3 Object Lock + JWS + async worker + DLQ + daily
|
||||
> checkpoints build-out is **deferred to a future milestone (D-083)** —
|
||||
> it requires non-offline-testable AWS infrastructure (Object Lock
|
||||
> bucket, KMS signing key, SQS DLQ, Lambda worker) and is not in v1.9.
|
||||
|
||||
The audit stream is the platform's tamper-evident record of every delivery
|
||||
action. The vision's "Audit truth lives outside the repository" bet [1]
|
||||
@@ -16,15 +18,16 @@ is the source of truth.
|
||||
- **Cold tier (source of truth):** S3 with **Object Lock in compliance
|
||||
mode**, **7-year retention** (ARCHITECTURE.md §9). No one — including
|
||||
root — can delete or overwrite until retention expires. The regulatory
|
||||
record.
|
||||
record. **Deferred to a future milestone (D-083).**
|
||||
- **Hot tier (query index):** the `acdl-evidence` audit repo (unchanged
|
||||
from the v1.0 demo). Not part of the chain; a queryable mirror the
|
||||
evidence UI (`evidence-ui/index.html`) reads. Lightweight attestation
|
||||
linkage lives in the repo; the regulatory event body lives in S3.
|
||||
- **Outbox (write path):** DynamoDB, **RPO = 0** (synchronous write before
|
||||
contract submission ack). Single-region in v1 (`us-east-1`).
|
||||
**Shipped + production since v1.8.**
|
||||
|
||||
## Spike scope (D-041) — what Phases 08-10 implement
|
||||
## Shipped scope (D-041) — production since v1.8
|
||||
|
||||
- **DynamoDB outbox:** table `acdl-outbox`, `PAY_PER_REQUEST` (D-044),
|
||||
PK `contractId`, SK `eventType#eventTs`, TTL `expire_at` = now + 365d
|
||||
@@ -34,14 +37,20 @@ is the source of truth.
|
||||
from the v1.0 demo's `evidence_writer.py`. Auto-genesis: first event
|
||||
has `prev_hash="GENESIS"`.
|
||||
- **Synchronous write** via boto3 `put_item` (strong-consistent by
|
||||
default). No separate async worker / DLQ in the spike (RTO = workflow
|
||||
default). No separate async worker / DLQ in v1.9 (RTO = workflow
|
||||
re-run).
|
||||
- **Mirror to `acdl-evidence`:** unchanged from v1.0 — the finalize step
|
||||
commits `audit.json` to the evidence repo (the hot tier).
|
||||
- **Spike evidence event shape:**
|
||||
- **Evidence event shape:**
|
||||
`{seq, ts, stage, event, prev_hash, hash, contractId, environment, stack, score, band}`.
|
||||
|
||||
## v1.2 build-out — what Phase 07 designs but the spike defers
|
||||
## Deferred to a future milestone (D-083)
|
||||
|
||||
The following build-out was authored as design in Phase 07 and is **not
|
||||
in v1.9**. It requires AWS infrastructure that cannot be exercised
|
||||
offline (Object Lock bucket, KMS signing key, SQS DLQ, Lambda worker)
|
||||
and is deferred to a future milestone. The hash-chain + DynamoDB-outbox
|
||||
path above remains the v1.9 production audit record.
|
||||
|
||||
- **S3 Object Lock:** bucket `acdl-evidence-lock-<account-id>`, Object
|
||||
Lock enabled at creation, compliance mode, 7-yr retention
|
||||
@@ -52,52 +61,59 @@ is the source of truth.
|
||||
the signature is stored *detached* alongside the payload. Signing key =
|
||||
**platform-level KMS key** (not per-contract — a per-contract key would
|
||||
explode the key-management surface), rotated **quarterly**. The `jws`
|
||||
field is added to the event shape in v1.2.
|
||||
field is added to the event shape when this ships.
|
||||
- **Async worker + DLQ:** a Lambda (or a Gitea Actions scheduled workflow)
|
||||
reads the outbox, writes to S3 Object Lock, signs with KMS. DLQ = an
|
||||
SQS dead-letter queue for failed writes. RTO = DLQ replay.
|
||||
- **Daily checkpoints (§9):** a daily job reads the last event hash and
|
||||
writes a "checkpoint" event to the ledger (+ optionally to a public
|
||||
notarization service). The spike runs in minutes, not days — no
|
||||
checkpoint in spike.
|
||||
notarization service).
|
||||
|
||||
## JWS vs chain — orthogonality note
|
||||
|
||||
The `prev_event_hash` chain gives ordering/tamper-evidence *within* the
|
||||
log (a deleted event breaks the chain visibly); JWS gives authenticity
|
||||
*per event* (a forged event is detectable without re-reading the whole
|
||||
chain). The chain is spike-scope; JWS is v1.2. Together they cover both
|
||||
integrity properties the vision's "Not a mutable audit log" anti-goal
|
||||
requires.
|
||||
chain). The chain is shipped (v1.8+); JWS is deferred (D-083). Together
|
||||
they cover both integrity properties the vision's "Not a mutable audit
|
||||
log" anti-goal requires.
|
||||
|
||||
## Outbox item shape (full, spike + v1.2)
|
||||
## Outbox item shape (shipped + deferred fields marked)
|
||||
|
||||
- PK `contractId` (UUID).
|
||||
- SK `eventType#eventTs` (e.g. `POLICY_CHECKED#2026-07-21T12:00:00Z`).
|
||||
- `payload` (the event body — hash-chained in spike, JWS-signed in v1.2).
|
||||
- `payload` (the event body — hash-chained in v1.8+; JWS-signed when
|
||||
D-083 ships).
|
||||
- `prev_event_hash` (chain link; `GENESIS` for the first event).
|
||||
- `hash` (this event's SHA-256 over canonical JSON).
|
||||
- `approver_qa` (Gitea username of the QA approver; empty in dev-only
|
||||
spike; populated on qa-promotion — D-042).
|
||||
- `approver_prod` (SRE username; empty in spike).
|
||||
- `approver_qa` (Gitea/GitHub username of the QA approver; populated on
|
||||
qa-promotion by v1.9's `hitl_gates.attest` — D-042).
|
||||
- `approver_prod` (SRE username; populated on prod-promotion by v1.9's
|
||||
`hitl_gates.attest`).
|
||||
- `approver_dr` (SRE username; populated on dr-promotion by v1.9's
|
||||
`hitl_gates.attest`).
|
||||
- `environment`, `stack`, `score`, `band`.
|
||||
- `expire_at` (TTL = now + 365d).
|
||||
- **v1.2 only:** `jws` (detached signature), `checkpoint_ref`.
|
||||
- **Deferred (D-083):** `jws` (detached signature), `checkpoint_ref`.
|
||||
|
||||
## RPO / RTO table
|
||||
|
||||
| Phase | RPO | RTO |
|
||||
|-------|-----|-----|
|
||||
| Spike (D-041) | 0 (sync outbox write) | workflow re-run |
|
||||
| v1.2 | 0 (sync outbox) | async worker DLQ replay |
|
||||
| v1.8+ (production, shipped) | 0 (sync outbox write) | workflow re-run |
|
||||
| Future milestone (D-083) | 0 (sync outbox) | async worker DLQ replay |
|
||||
|
||||
## Decision trail
|
||||
|
||||
- **D-041** — spike scope = hash chain + outbox write; Object Lock + JWS
|
||||
+ worker + DLQ are v1.2.
|
||||
- **D-041** — shipped scope = hash chain + outbox write; Object Lock +
|
||||
JWS + worker + DLQ are deferred (D-083).
|
||||
- **D-044** — outbox mode `PAY_PER_REQUEST`; PK/SK; TTL `expire_at` =
|
||||
now + 365d; no separate async worker in spike.
|
||||
- **D-042** — approver identities (`approver_qa`, `approver_prod`) live
|
||||
in the outbox; the separation-of-duties check
|
||||
(`platform/separation_of_duties.py`) reads `approver_qa` and compares
|
||||
to the prod-dispatch `gitea.actor`.
|
||||
now + 365d; no separate async worker in v1.9.
|
||||
- **D-042** — approver identities (`approver_qa`, `approver_prod`,
|
||||
`approver_dr`) live in the outbox; the separation-of-duties check
|
||||
(`core/separation_of_duties.py`) reads `approver_qa` and compares
|
||||
to the prod-dispatch `gitea.actor` / `github.actor`. v1.9's
|
||||
`hitl_gates.attest` populates these attributes.
|
||||
- **D-083** (v1.9) — S3 Object Lock + JWS + async worker + DLQ + daily
|
||||
checkpoints deferred to a future milestone. Requires non-offline-
|
||||
testable AWS infra.
|
||||
@@ -141,6 +141,11 @@ def resolve_l2(contract, registry, repo_root):
|
||||
# For single-resource L1s, resourceId == childId
|
||||
# For multi-resource L1s, resourceId is the expanded sub-resource id
|
||||
child_outputs = {}
|
||||
# child_input_map[childId] = {inputName: sub_resource_id} for multi-resource L1s
|
||||
# so a wire targeting <childId>.inputs.<name> routes to the sub-resource
|
||||
# that actually declares that input (P1-1 — desired_count → aws:ecs:service,
|
||||
# family → aws:ecs:task_definition).
|
||||
child_input_map = {}
|
||||
resources = []
|
||||
|
||||
# Expand children to resources
|
||||
@@ -156,6 +161,7 @@ def resolve_l2(contract, registry, repo_root):
|
||||
|
||||
# Build the output->resourceId map for this child
|
||||
child_out_map = {}
|
||||
child_in_map = {}
|
||||
|
||||
# For multi-resource L1s (like vpc), the first resource type is the
|
||||
# primary; the adapter handles expansion. Use the interface's type
|
||||
@@ -178,6 +184,9 @@ def resolve_l2(contract, registry, repo_root):
|
||||
# Map each output to this sub-resource's id
|
||||
for out_name in sub_res.get("outputs", []):
|
||||
child_out_map[out_name] = res_id
|
||||
# Map each declared input to this sub-resource's id (P1-1)
|
||||
for in_name in sub_res.get("inputs", []):
|
||||
child_in_map[in_name] = res_id
|
||||
else:
|
||||
# Single-resource L1
|
||||
resource = {
|
||||
@@ -202,6 +211,7 @@ def resolve_l2(contract, registry, repo_root):
|
||||
child_out_map[out_name] = child_id
|
||||
|
||||
child_outputs[child_id] = child_out_map
|
||||
child_input_map[child_id] = child_in_map
|
||||
|
||||
# Resolve wires to populate inputs
|
||||
for wire in composition.get("wires", []):
|
||||
@@ -215,11 +225,22 @@ def resolve_l2(contract, registry, repo_root):
|
||||
|
||||
value = _resolve_wire_value(wire, inputs, child_outputs)
|
||||
if value is not None:
|
||||
# Find the target resource and set the input
|
||||
for res in resources:
|
||||
if res["id"] == target_child or res["id"].startswith(f"{target_child}-"):
|
||||
res["inputs"][input_name] = value
|
||||
break
|
||||
# Route to the sub-resource that declares this input (P1-1).
|
||||
# child_input_map maps <childId> -> {inputName -> sub_resource_id}.
|
||||
# If the input is declared on a specific sub-resource, route there;
|
||||
# otherwise fall back to the first matching resource (legacy).
|
||||
in_map = child_input_map.get(target_child, {})
|
||||
target_res_id = in_map.get(input_name)
|
||||
if target_res_id is not None:
|
||||
for res in resources:
|
||||
if res["id"] == target_res_id:
|
||||
res["inputs"][input_name] = value
|
||||
break
|
||||
else:
|
||||
for res in resources:
|
||||
if res["id"] == target_child or res["id"].startswith(f"{target_child}-"):
|
||||
res["inputs"][input_name] = value
|
||||
break
|
||||
|
||||
# Build the stack instance
|
||||
stack_instance = {
|
||||
|
||||
+78
-18
@@ -1,8 +1,10 @@
|
||||
# ACDL Human-in-the-Loop Matrix + Separation-of-Duties Design (REQ-21)
|
||||
|
||||
> **Status:** design authored in Phase 07 (milestone v1.1); v1.2 wires the
|
||||
> gates. The spike (Phases 08-10) is **dev-only**; HITL is not exercised
|
||||
> (the spike contract has `environment: dev`).
|
||||
> **Status:** design authored in Phase 07 (milestone v1.1); **v1.9 wires
|
||||
> the gates** (Phase 42). The spike (Phases 08-10) was dev-only; HITL was
|
||||
> not exercised then. v1.9 implements the qa/prod/dr pre-execution
|
||||
> attestation gates, the 8-concern attestation matrix (offline-testable
|
||||
> subset), and the outbox-based separation-of-duties check.
|
||||
|
||||
The vision's "Lower Environments are Autonomous; Higher Environments are
|
||||
Attested" tenet [1] and the "deliberate human attestation — not as a
|
||||
@@ -29,11 +31,16 @@ is modeled as a `workflow_dispatch` with approval inputs:
|
||||
- **dr gate:** `workflow_dispatch` with `approve_dr: true`; same.
|
||||
|
||||
The approver identity of record = `gitea.actor` of the dispatch run
|
||||
(D-042). There is no other approval-identity signal in Gitea. The v1.2
|
||||
real-OIDC path (blocked on go-gitea/gitea#36988) does not change this —
|
||||
(D-042). There is no other approval-identity signal in Gitea. The real
|
||||
OIDC path (blocked on go-gitea/gitea#36988) does not change this —
|
||||
OIDC authorizes the *runner* to AWS, it does not change how the platform
|
||||
records the *human* approver.
|
||||
|
||||
On GitHub, the equivalent is `github.actor` of the `workflow_dispatch`
|
||||
run; GitHub Environments with required reviewers are the native gate,
|
||||
but the `workflow_dispatch` approval-input fallback is used for
|
||||
byte-identical Gitea + GitHub workflows.
|
||||
|
||||
## Reviewer routing (ARCHITECTURE.md §10.2)
|
||||
|
||||
Gitea CODEOWNERS routes the right reviewer to the right gate:
|
||||
@@ -44,9 +51,32 @@ Gitea CODEOWNERS routes the right reviewer to the right gate:
|
||||
|
||||
CODEOWNERS **routes**; it does **not** enforce identity distinctness (that
|
||||
is the platform-internal outbox check in
|
||||
`platform/separation_of_duties.py`).
|
||||
`core/separation_of_duties.py`).
|
||||
|
||||
## Full 8-concern attestation matrix (§10.4, lifted verbatim)
|
||||
## Full 8-concern attestation matrix (§10.4)
|
||||
|
||||
The matrix is implemented in v1.9 as `core/attestation_matrix.py`
|
||||
(REQ-109, D-084). The concerns split into two tiers:
|
||||
|
||||
**Offline-testable concerns** (run for real, no operator input):
|
||||
- Contract NFRs (the platform's own contract validator).
|
||||
- Schema validity (jsonschema).
|
||||
- Policy pass (Checkov/Wiz/Kyverno `PolicyCheckResult` records).
|
||||
|
||||
**Operator-supplied concerns** (require an uploaded signed evidence
|
||||
artifact, validated for freshness + schema per D-084):
|
||||
- Functional correctness (e2e suite report).
|
||||
- Performance baseline (k6 / Gatling / Locust load test report).
|
||||
- Security posture (Trivy / Snyk / contract-declared scan + Security
|
||||
on-call signature).
|
||||
- Operational readiness (runbook published, dashboard exists, on-call
|
||||
rotation assigned, alerts configured).
|
||||
- Incident response (Sev-1 runbook tabletop or live drill completed).
|
||||
- Capacity / cost (FinOps forecast for next 30d within budget envelope).
|
||||
- Resilience (DR drill, chaos engineering report, backup verified).
|
||||
- dr-region deploy (most recent prod-bound dr drill as canary evidence).
|
||||
|
||||
The full table (lifted verbatim from §10.4):
|
||||
|
||||
| Env | Concern | Evidence artifact | Freshness | Source | Attester |
|
||||
|---|---|---|---|---|---|
|
||||
@@ -60,6 +90,13 @@ is the platform-internal outbox check in
|
||||
| prod | Resilience | DR drill, chaos engineering report, backup verified | DR: 180d; chaos: 90d; backup: 30d | SRE + Platform | SRE |
|
||||
| dr | dr-region deploy with the most recent prod-bound dr drill as canary evidence | dr drill report | Last 180d | SRE | SRE |
|
||||
|
||||
The operator-supplied evidence artifact is a JSON blob with `timestamp`,
|
||||
`type`, `payload`, and an optional `signature` (JWS detached). Freshness
|
||||
is validated against the window above. Signature verification runs when
|
||||
`ACDL_ATTESTATION_SIGNING_KEY_ID` is set; it is skipped + logged when
|
||||
unset (dev/CI — D-089). The matrix fails loud if an operator-supplied
|
||||
concern is missing or expired for prod/dr.
|
||||
|
||||
## Timeout behavior (§10.5)
|
||||
|
||||
| Time | State | Action |
|
||||
@@ -71,7 +108,8 @@ is the platform-internal outbox check in
|
||||
**Implementation:** a Gitea `on: schedule` workflow (runs hourly) that
|
||||
scans the DynamoDB outbox for `PENDING_ATTESTATION` events with `ts`
|
||||
older than 1/2 business days and emits the warn/freeze events. Not
|
||||
implemented in the spike (dev-only).
|
||||
implemented in v1.9 (roadmap item; the attestation gates themselves are
|
||||
wired, the timeout scanner is future work).
|
||||
|
||||
## Rejection and rollback (§10.6)
|
||||
|
||||
@@ -88,28 +126,50 @@ The identity-distinctness check is platform-internal, not GitHub-native,
|
||||
not Kyverno (in v1). Sequence:
|
||||
|
||||
1. On promotion dev → qa, the platform reads the QA approver's identity
|
||||
from the `workflow_dispatch` run's `gitea.actor` and writes it to the
|
||||
DynamoDB outbox keyed by `contractId` (attribute `approver_qa`).
|
||||
from the `workflow_dispatch` run's `gitea.actor` (or `github.actor`)
|
||||
and writes it to the DynamoDB outbox keyed by `contractId` (attribute
|
||||
`approver_qa`).
|
||||
2. On promotion qa → prod, the platform reads the stored `approver_qa`
|
||||
from the outbox and the new SRE approver's `gitea.actor` from the
|
||||
prod-dispatch run.
|
||||
3. If `approver_qa == approver_prod`, the platform blocks the prod
|
||||
promotion, writes a `SEPARATION_OF_DUTIES_VIOLATION` event to the
|
||||
evidence stream, and routes a halt artifact to the SRE on-call.
|
||||
4. The check is implemented in `platform/separation_of_duties.py`
|
||||
4. The check is implemented in `core/separation_of_duties.py`
|
||||
(T-7.8). The platform is the only writer to the outbox; the check is
|
||||
in the same process that has authority to block the promotion.
|
||||
|
||||
## Spike scope note
|
||||
v1.9 implements `route_halt_artifact` as a real SNS publish (topic
|
||||
`acdl-sod-halt`, ARN from `ACDL_SOD_HALT_TOPIC_ARN`) with an outbox-event
|
||||
fallback when the topic ARN is unset (REQ-107). The attestation gate
|
||||
itself is `core/hitl_gates.py` (`attest(contract_id, env, approver,
|
||||
evidence)`), which records the approver to the outbox, runs the SoD
|
||||
check on prod, invokes the attestation matrix, and returns `(ok, reason)`.
|
||||
|
||||
The spike is dev-only (REQ-27 contract has `environment: dev`), so HITL
|
||||
is not exercised. Phase 07 authors the design; Phase 10's
|
||||
`verify_phase10.sh` does not assert HITL behavior. v1.2 wires the gates
|
||||
against this design.
|
||||
## v1.9 wiring
|
||||
|
||||
v1.9 (Phase 41 + Phase 42) wires the gates end-to-end:
|
||||
|
||||
- **Phase 41** ships the per-environment CI job structure: one job per
|
||||
environment (dev/qa/prod/dr), each pointing at its respective contract
|
||||
(or the same contract + the `environment` workflow_call input). The
|
||||
qa/prod/dr caller workflows use `workflow_dispatch` with the approval
|
||||
inputs above; dev is autonomous (no gate). Promotion = running the
|
||||
matching job; no `environment:` field editing (D-082).
|
||||
- **Phase 42** implements `core/hitl_gates.py` (the attestation gate),
|
||||
`core/attestation_matrix.py` (the 8-concern matrix), and the real
|
||||
`route_halt_artifact` (SNS + outbox fallback). `scripts/run_platform.sh`
|
||||
calls `hitl_gates.attest` before apply for qa/prod/dr (dev skips).
|
||||
|
||||
## Decision trail
|
||||
|
||||
- **D-042** — approver identity = `gitea.actor` of the `workflow_dispatch`
|
||||
run; no Environments API in Gitea.
|
||||
run; no Environments API in Gitea. On GitHub, `github.actor`.
|
||||
- **D-013** (v1.0) — the `workflow_dispatch` approval-input fallback,
|
||||
re-used for the real platform's pre-execution gate model.
|
||||
re-used for the real platform's pre-execution gate model.
|
||||
- **D-084** (v1.9) — 8-concern attestation matrix: offline-testable
|
||||
concerns run for real; operator-supplied concerns accept signed
|
||||
evidence artifacts validated for freshness + schema.
|
||||
- **D-089** (v1.9) — attestation artifact signature verification is
|
||||
skipped when `ACDL_ATTESTATION_SIGNING_KEY_ID` is unset (dev/CI);
|
||||
required for prod/dr.
|
||||
Reference in New Issue
Block a user