continuous-intelligence/acdl
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 128 Wiki Activity

verify(v1.10): 4-layer milestone gate — PASS

Browse Source 
Layer 1 (Structural): all 8 plan-referenced files exist; imports resolve;
no TODO/stub placeholders; all declared exports present. PASS.

Layer 2 (Behavioral): 518 tests pass (513 fast + 5 slow); REQ-112..115
all complete; regression gate 16/16 Verified. PASS.

Layer 3 (Security/STRIDE): all 6 threats low-severity; auto-accepted.
No creds logged; loopback-only binding; monkey-patches scoped to local
tier. PASS.

Layer 4 (Quality): 0 P0, 0 P1, 1 P2 (post-hoc: expand regression
registry to uptime-kuma + RDS stacks). Gate can't be bypassed; local
E2E can't mutate cloud; no injection vectors. PASS.

Verdict: VERIFY PASS. v1.10 ready to ship.

---ci---
project: acdl
phase: 0
milestone: v1.10
status: verify
requirements:
  covered: [REQ-112, REQ-113, REQ-114, REQ-115]
  partial: []
lessons:
  - The regression gate (D-091) is the durable fix for the diff-scoped
    VERIFY defect; it must run at every milestone completion to catch
    capability decay before it hides behind docs-only NFR patches.
  - Local emulating adapters (D-092) make the platform testable without
    cloud credentials; the local tier is now the regression baseline.
  - 6 IAM-gated cloud resources cannot be auto-verified (chicken-and-egg);
    the terraform plan path is the strongest verification possible
    without terraform apply (a deploy-class autonomy escalation).
---/ci---
This commit is contained in:
Jon Chery
2026-07-27 18:40:44 +00:00
parent 2697775470
commit 5274bc48a9
3 changed files with 101 additions and 83 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.ciagent/REGRESSION_REPORT.json
+21 -21
View File
@@ -1,6 +1,6 @@
{ {
"run_id": "regr-1785176791", "run_id": "regr-1785177468",
"run_at_utc": "2026-07-27T18:26:31Z", "run_at_utc": "2026-07-27T18:37:48Z",
"milestone": "v1.10", "milestone": "v1.10",
"phase": 52, "phase": 52,
"summary": { "summary": {
@@ -16,7 +16,7 @@
"status": "Verified", "status": "Verified",
"detail": "exit 0; 2 sample contracts validate", "detail": "exit 0; 2 sample contracts validate",
"tier": "local", "tier": "local",
"duration_ms": 222 "duration_ms": 245
}, },
{ {
"capability_id": "CAP-002", "capability_id": "CAP-002",
@@ -24,7 +24,7 @@
"status": "Verified", "status": "Verified",
"detail": "exit 0; env schema validates", "detail": "exit 0; env schema validates",
"tier": "local", "tier": "local",
"duration_ms": 191 "duration_ms": 195
}, },
{ {
"capability_id": "CAP-003", "capability_id": "CAP-003",
@@ -32,7 +32,7 @@
"status": "Verified", "status": "Verified",
"detail": "exit 0; ", "detail": "exit 0; ",
"tier": "local", "tier": "local",
"duration_ms": 250 "duration_ms": 260
}, },
{ {
"capability_id": "CAP-004", "capability_id": "CAP-004",
@@ -40,7 +40,7 @@
"status": "Verified", "status": "Verified",
"detail": "exit 0; ", "detail": "exit 0; ",
"tier": "local", "tier": "local",
"duration_ms": 257 "duration_ms": 264
}, },
{ {
"capability_id": "CAP-005", "capability_id": "CAP-005",
@@ -48,7 +48,7 @@
"status": "Verified", "status": "Verified",
"detail": "exit 0; ", "detail": "exit 0; ",
"tier": "local", "tier": "local",
"duration_ms": 335 "duration_ms": 332
}, },
{ {
"capability_id": "CAP-006", "capability_id": "CAP-006",
@@ -56,7 +56,7 @@
"status": "Verified", "status": "Verified",
"detail": "exit 0; interpolation ok", "detail": "exit 0; interpolation ok",
"tier": "local", "tier": "local",
"duration_ms": 238 "duration_ms": 216
}, },
{ {
"capability_id": "CAP-007", "capability_id": "CAP-007",
@@ -64,7 +64,7 @@
"status": "Verified", "status": "Verified",
"detail": "exit 0; confidence band=pass", "detail": "exit 0; confidence band=pass",
"tier": "local", "tier": "local",
"duration_ms": 81 "duration_ms": 90
}, },
{ {
"capability_id": "CAP-008", "capability_id": "CAP-008",
@@ -72,15 +72,15 @@
"status": "Verified", "status": "Verified",
"detail": "exit 0; outbox hash chain ok", "detail": "exit 0; outbox hash chain ok",
"tier": "local", "tier": "local",
"duration_ms": 324 "duration_ms": 326
}, },
{ {
"capability_id": "CAP-009", "capability_id": "CAP-009",
"name": "offline pytest suite passes", "name": "offline pytest suite passes",
"status": "Verified", "status": "Verified",
"detail": "exit 0; [ 98%]\ntests/test_wiz_adapter_real_client.py ......... [100%]\n\n====================== 475 passed, 2 deselected in 13.93s ======================", "detail": "exit 0; [ 98%]\ntests/test_wiz_adapter_real_client.py ......... [100%]\n\n====================== 475 passed, 2 deselected in 14.26s ======================",
"tier": "local", "tier": "local",
"duration_ms": 15411 "duration_ms": 15683
}, },
{ {
"capability_id": "CAP-010", "capability_id": "CAP-010",
@@ -88,23 +88,23 @@
"status": "Verified", "status": "Verified",
"detail": "exit 0; resource(s))\n\n=== PLATFORM CHECK OK ===\ncontract -> resolver -> stack -> adapter -> structure validated (offline, no AWS)\ncheck-only: OK\n\n=== CI PIPELINE OK ===\n3 stages passed: lint, test, check-only", "detail": "exit 0; resource(s))\n\n=== PLATFORM CHECK OK ===\ncontract -> resolver -> stack -> adapter -> structure validated (offline, no AWS)\ncheck-only: OK\n\n=== CI PIPELINE OK ===\n3 stages passed: lint, test, check-only",
"tier": "local", "tier": "local",
"duration_ms": 19513 "duration_ms": 19489
}, },
{ {
"capability_id": "CAP-011", "capability_id": "CAP-011",
"name": "headline E2E runs against the local emulating tier (microservice)", "name": "headline E2E runs against the local emulating tier (microservice)",
"status": "Verified", "status": "Verified",
"detail": "exit 0; al-emulator\",\n \"desired_count\": 1,\n \"running_count\": 1\n },\n \"outbox_dir\": \"/tmp/acdl_local_e2e_c2dyhp_r/outbox\",\n \"outbox_events\": 2,\n \"outbox_chain_verified\": true,\n \"lambda_status\": 200\n}", "detail": "exit 0; al-emulator\",\n \"desired_count\": 1,\n \"running_count\": 1\n },\n \"outbox_dir\": \"/tmp/acdl_local_e2e_92qknwvi/outbox\",\n \"outbox_events\": 2,\n \"outbox_chain_verified\": true,\n \"lambda_status\": 200\n}",
"tier": "local", "tier": "local",
"duration_ms": 551 "duration_ms": 1076
}, },
{ {
"capability_id": "CAP-012", "capability_id": "CAP-012",
"name": "local E2E on the static-assets stack (no ECS)", "name": "local E2E on the static-assets stack (no ECS)",
"status": "Verified", "status": "Verified",
"detail": "exit 0; acdl_local_e2e_w2ybb54x/tf\",\n \"backend\": \"local\",\n \"ecs\": null,\n \"outbox_dir\": \"/tmp/acdl_local_e2e_w2ybb54x/outbox\",\n \"outbox_events\": 2,\n \"outbox_chain_verified\": true,\n \"lambda_status\": 200\n}", "detail": "exit 0; acdl_local_e2e_ntp1b581/tf\",\n \"backend\": \"local\",\n \"ecs\": null,\n \"outbox_dir\": \"/tmp/acdl_local_e2e_ntp1b581/outbox\",\n \"outbox_events\": 2,\n \"outbox_chain_verified\": true,\n \"lambda_status\": 200\n}",
"tier": "local", "tier": "local",
"duration_ms": 575 "duration_ms": 500
}, },
{ {
"capability_id": "CAP-013", "capability_id": "CAP-013",
@@ -112,7 +112,7 @@
"status": "Verified", "status": "Verified",
"detail": "terraform init+validate+plan OK (live AWS, microservice)", "detail": "terraform init+validate+plan OK (live AWS, microservice)",
"tier": "live-aws", "tier": "live-aws",
"duration_ms": 28496 "duration_ms": 28354
}, },
{ {
"capability_id": "CAP-014", "capability_id": "CAP-014",
@@ -120,7 +120,7 @@
"status": "Verified", "status": "Verified",
"detail": "terraform init+validate+plan OK (live AWS, static-assets)", "detail": "terraform init+validate+plan OK (live AWS, static-assets)",
"tier": "live-aws", "tier": "live-aws",
"duration_ms": 32341 "duration_ms": 32121
}, },
{ {
"capability_id": "CAP-015", "capability_id": "CAP-015",
@@ -128,7 +128,7 @@
"status": "Verified", "status": "Verified",
"detail": "acdl-outbox exists, item_count=9", "detail": "acdl-outbox exists, item_count=9",
"tier": "live-aws", "tier": "live-aws",
"duration_ms": 521 "duration_ms": 564
}, },
{ {
"capability_id": "CAP-016", "capability_id": "CAP-016",
@@ -136,7 +136,7 @@
"status": "Verified", "status": "Verified",
"detail": "state bucket exists, keys=['spike/l2-microservice/terraform.tfstate']", "detail": "state bucket exists, keys=['spike/l2-microservice/terraform.tfstate']",
"tier": "live-aws", "tier": "live-aws",
"duration_ms": 311 "duration_ms": 434
} }
] ]
} }
.ciagent/REGRESSION_REPORT.md
+20 -20
View File
@@ -1,45 +1,45 @@
# Regression Report — v1.10 Phase 52 # Regression Report — v1.10 Phase 52
- **Run ID:** `regr-1785176791` - **Run ID:** `regr-1785177468`
- **Run at (UTC):** 2026-07-27T18:26:31Z - **Run at (UTC):** 2026-07-27T18:37:48Z
- **Summary:** {'Verified': 16, 'Decayed': 0, 'Broken': 0} - **Summary:** {'Verified': 16, 'Decayed': 0, 'Broken': 0}
- **Passed (milestone gate):** True - **Passed (milestone gate):** True
| Capability | Name | Tier | Status | Duration (ms) | Detail | | Capability | Name | Tier | Status | Duration (ms) | Detail |
|-----------|------|------|--------|--------------|--------| |-----------|------|------|--------|--------------|--------|
| CAP-001 | contract.schema.json validates sample contracts | local | **Verified** | 222 | exit 0; 2 sample contracts validate | | CAP-001 | contract.schema.json validates sample contracts | local | **Verified** | 245 | exit 0; 2 sample contracts validate |
| CAP-002 | environment.schema.json validates env files | local | **Verified** | 191 | exit 0; env schema validates | | CAP-002 | environment.schema.json validates env files | local | **Verified** | 195 | exit 0; env schema validates |
| CAP-003 | contract_resolver resolves static-assets | local | **Verified** | 250 | exit 0; | | CAP-003 | contract_resolver resolves static-assets | local | **Verified** | 260 | exit 0; |
| CAP-004 | contract_resolver resolves microservice | local | **Verified** | 257 | exit 0; | | CAP-004 | contract_resolver resolves microservice | local | **Verified** | 264 | exit 0; |
| CAP-005 | terraform adapter emits .tf files | local | **Verified** | 335 | exit 0; | | CAP-005 | terraform adapter emits .tf files | local | **Verified** | 332 | exit 0; |
| CAP-006 | contract interpolation expands env/contract tokens | local | **Verified** | 238 | exit 0; interpolation ok | | CAP-006 | contract interpolation expands env/contract tokens | local | **Verified** | 216 | exit 0; interpolation ok |
| CAP-007 | confidence_signal.compute returns a band | local | **Verified** | 81 | exit 0; confidence band=pass | | CAP-007 | confidence_signal.compute returns a band | local | **Verified** | 90 | exit 0; confidence band=pass |
| CAP-008 | outbox_writer builds a hash-chained item | local | **Verified** | 324 | exit 0; outbox hash chain ok | | CAP-008 | outbox_writer builds a hash-chained item | local | **Verified** | 326 | exit 0; outbox hash chain ok |
| CAP-009 | offline pytest suite passes | local | **Verified** | 15411 | exit 0; [ 98%] | CAP-009 | offline pytest suite passes | local | **Verified** | 15683 | exit 0; [ 98%]
tests/test_wiz_adapter_real_client.py ......... [100%] tests/test_wiz_adapter_real_client.py ......... [100%]
====================== 475 passe | ====================== 475 passe |
| CAP-010 | run_ci.sh reproduces CI pipeline locally | local | **Verified** | 19513 | exit 0; resource(s)) | CAP-010 | run_ci.sh reproduces CI pipeline locally | local | **Verified** | 19489 | exit 0; resource(s))
=== PLATFORM CHECK OK === === PLATFORM CHECK OK ===
contract -> resolver -> stack -> adapter -> structure validated (offline, no AWS) contract -> resolver -> stack -> adapter -> structure validated (offline, no AWS)
check-only: OK check-only: OK
=== CI PIPELIN | === CI PIPELIN |
| CAP-011 | headline E2E runs against the local emulating tier (microservice) | local | **Verified** | 551 | exit 0; al-emulator", | CAP-011 | headline E2E runs against the local emulating tier (microservice) | local | **Verified** | 1076 | exit 0; al-emulator",
"desired_count": 1, "desired_count": 1,
"running_count": 1 "running_count": 1
}, },
"outbox_dir": "/tmp/acdl_local_e2e_c2dyhp_r/outbox", "outbox_dir": "/tmp/acdl_local_e2e_92qknwvi/outbox",
"outbox_events": 2, "outbox_events": 2,
"outbox | "outbox |
| CAP-012 | local E2E on the static-assets stack (no ECS) | local | **Verified** | 575 | exit 0; acdl_local_e2e_w2ybb54x/tf", | CAP-012 | local E2E on the static-assets stack (no ECS) | local | **Verified** | 500 | exit 0; acdl_local_e2e_ntp1b581/tf",
"backend": "local", "backend": "local",
"ecs": null, "ecs": null,
"outbox_dir": "/tmp/acdl_local_e2e_w2ybb54x/outbox", "outbox_dir": "/tmp/acdl_local_e2e_ntp1b581/outbox",
"outbox_events": 2, "outbox_events": 2,
"outbox | "outbox |
| CAP-013 | terraform init+validate+plan live AWS (microservice) | live-aws | **Verified** | 28496 | terraform init+validate+plan OK (live AWS, microservice) | | CAP-013 | terraform init+validate+plan live AWS (microservice) | live-aws | **Verified** | 28354 | terraform init+validate+plan OK (live AWS, microservice) |
| CAP-014 | terraform init+validate+plan live AWS (static-assets) | live-aws | **Verified** | 32341 | terraform init+validate+plan OK (live AWS, static-assets) | | CAP-014 | terraform init+validate+plan live AWS (static-assets) | live-aws | **Verified** | 32121 | terraform init+validate+plan OK (live AWS, static-assets) |
| CAP-015 | DynamoDB outbox table exists (live AWS) | live-aws | **Verified** | 521 | acdl-outbox exists, item_count=9 | | CAP-015 | DynamoDB outbox table exists (live AWS) | live-aws | **Verified** | 564 | acdl-outbox exists, item_count=9 |
| CAP-016 | S3 state bucket exists + readable (live AWS) | live-aws | **Verified** | 311 | state bucket exists, keys=['spike/l2-microservice/terraform.tfstate'] | | CAP-016 | S3 state bucket exists + readable (live AWS) | live-aws | **Verified** | 434 | state bucket exists, keys=['spike/l2-microservice/terraform.tfstate'] |
.ciagent/VERIFY.md
+60 -42
View File
@@ -1,49 +1,67 @@
# Phase 55 — Verify (v1.10) — Rewrite to Verified Reality # ACDL v1.10 — Verify (milestone gate)
## Structural > Verify date: 2026-07-27. Verifier: ci-verifier. Milestone: v1.10 (complete, tag `v1.10.0`).
- `.ciagent/PROJECT.md` — new "Capability Status (Re-Verified > Scope: 4 phases (52–55), 5 commits (772ac72..2697775), 22 files, +2281/-256 lines.
2026-07-27)" section inserted after Domain Boundaries (before the
milestone objectives). Contains: decay disclosure, the 16
auto-verified capabilities table, the 6 IAM-gated escalated
resources, and the regression-gate note.
- `.ciagent/ROADMAP.md` — v1.9.8 entry annotated "Last deck-polish
phase before the v1.10 deck-freeze"; new v1.10 overview entry noting
v1.9.1–v1.9.8 are "superseded-by-reverification".
- `docs/presentations/how-the-platform-works.md` — re-verification
note added to the maturity-framing header.
- `docs/presentations/the-developer-experience.md` — same
re-verification note added.
- `.ciagent/REQUIREMENTS.md` — REQ-115 marked complete.
**PASS.**
## Behavioral ## Layer 1: Structural — PASS
- PROJECT.md "Capability Status" section matches
`CAPABILITY_INVENTORY.md` exactly (16 Verified + 6 IAM-gated
escalated).
- ROADMAP.md v1.9.x entries disclose the deck-freeze + supersession.
- Both decks disclose the 2026-07-27 re-verification in their headers.
- No "shipped"/"Available today" claims remain that aren't backed by a
Verified capability or an explicit escalation note.
**PASS.**
## Security - All 8 plan-referenced files exist on disk (`core/regression_verify.py`,
- No secrets, credentials, or account IDs beyond the already-public `core/local_emulators.py`, `scripts/run_regression.sh`,
581513795199 (already in ROADMAP/PROJECT prior to this phase). `tests/test_verify_regression_mode.py`,
- The decay disclosure is honest: it states the capability was not `tests/test_local_emulating_adapters.py`,
reproducible as of 2026-07-27 and was re-verified in v1.10. `.ciagent/CAPABILITY_INVENTORY.md`, `REGRESSION_REPORT.md`,
**PASS.** `REGRESSION_REPORT.json`).
- All imports resolve (`py_compile` + runtime import OK).
- No TODO/FIXME/HACK/stub placeholders in new code (the `LocalLambdaStub`
is a legitimate local emulator, not a placeholder).
- All declared exports exist (`run_regression`, `write_report`,
`CAPABILITY_REGISTRY`, `RegressionReport`, `CapabilityResult`,
`FlatFileOutbox`, `LocalEcsEmulator`, `LocalS3StateBackend`,
`LocalLambdaStub`, `run_local_e2e`, `is_local_tier`).
## Quality ## Layer 2: Behavioral — PASS
- The re-verification note in both decks cites the source of truth
(`.ciagent/CAPABILITY_INVENTORY.md`) so an exec can trace any claim. - `pytest tests/ -m "not slow"`: **513 passed**, 5 deselected.
- The decay disclosure does not overclaim: 6 IAM-gated resources are - `pytest tests/ -m slow`: **5 passed** (2 local E2E + 3 regression
explicitly listed as escalated, not silently omitted. integration incl. live-AWS terraform plan).
**PASS.** - **Total: 518 passed, 0 failed.**
- Requirement coverage: REQ-112 (P52), REQ-113 (P53), REQ-114 (P54),
REQ-115 (P55) — all 4 marked `complete`.
- Regression gate: `bash scripts/run_regression.sh` → **16/16
capabilities Verified** (12 local + 4 live-AWS). Milestone gate open.
## Layer 3: Security (STRIDE) — PASS
| Threat | Risk | Disposition |
|--------|------|-------------|
| Spoofing | Local Lambda stub patches `_get_dynamodb`/`_get_secrets_client`; opt-in via `ACDL_LOCAL_TIER=1`, never in prod | Accept (low) |
| Tampering | Flat-file outbox hash-chain verification detects tampering | Accept (low) |
| Repudiation | Regression report records per-capability status + timestamps | Accept (low) |
| Info Disclosure | Creds read into env vars, never logged (0 cred strings in reports); ECS binds 127.0.0.1 only | Accept (low) |
| Denial of Service | Local ECS emulator: free port, daemon thread, clean destroy | Accept (low) |
| Elevation of Privilege | `urllib.urlopen` patched to fake response (no network egress); no eval/exec/subprocess in adapter | Accept (low) |
All threats low-severity; auto-accepted per
`config.json security.auto_accept_low_severity=true`.
## Layer 4: Quality (multi-persona) — PASS
| Persona | Finding | Verdict |
|---------|---------|---------|
| Correctness | 7 adapter defects fixed; each traceable to a terraform validate/plan error | PASS |
| Testing | 518 tests pass; 24 new tests. P2: uptime-kuma + RDS not in registry | PASS (1 P2) |
| Security | No creds logged; loopback-only; monkey-patches scoped to local tier | PASS |
| Performance | Regression run ~60s; acceptable for a milestone gate | PASS |
| Maintainability | Well-structured; adding a capability = 1 function + 1 registry entry | PASS |
| Adversarial | Gate can't be bypassed; local E2E can't mutate cloud; no injection vectors | PASS |
**0 P0, 0 P1, 1 P2 (post-hoc: expand regression registry to uptime-kuma + RDS stacks).**
## Verdict ## Verdict
**VERIFY PASS** — PROJECT/ROADMAP/decks now match the verified reality **VERIFY PASS** — all 4 layers pass. The v1.10 milestone is sound:
in `CAPABILITY_INVENTORY.md`. The decay is disclosed honestly; every the pipeline regression gap is fixed (D-091), the platform is fully
"Testing" claim is backed by a Verified capability; 6 IAM-gated locally testable (D-092), every advertised capability is re-verified
resources are escalated, not hidden. Decks are unfrozen. v1.10.0 is (D-093, 16/16 Verified), and the docs/decks match verified reality
ready to tag. (D-094). 518 tests pass; the regression gate covers 16 capabilities
including 4 live-AWS checks. 0 P0, 0 P1, 1 P2 post-hoc. Ready to ship.