Compare commits
5 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| e050e65158 | |||
| 6e23c168f1 | |||
| c816493e7e | |||
| 1598c54a8b | |||
| 2c6464afd4 |
+30
-112
@@ -1,125 +1,43 @@
|
||||
# ACDL v1.2 Milestone — Audit
|
||||
# Phase 18 — Audit (v1.3.2)
|
||||
|
||||
**Auditor:** ci-audit-verifier (model: glm-5.2)
|
||||
**Scope:** v1.2 milestone — Phases 11–16 (tags v1.2.1..v1.2.6), milestone ship tag `v1.3.0`, diff `v1.2.0..HEAD` (24 commits)
|
||||
**Date:** 2026-07-21
|
||||
**Verdict:** **CLEAN** — 0 P0 (no critical code issues; the 1 P0 is an operator action, not a code defect), 1 P1 post-hoc, 0 P2.
|
||||
**Date:** 2026-07-22
|
||||
**Phase:** 18 — testing-and-cicd-pipelines
|
||||
**Milestone:** v1.3 (active, NFR)
|
||||
**Tag:** v1.3.2
|
||||
|
||||
---
|
||||
## 1. Reconstruction Test
|
||||
|
||||
## 1. Reconstruction test
|
||||
Git log (2 commits for phase 18) matches `.ciagent/` files:
|
||||
|
||||
**PASS.** The project state can be reconstructed from the git log `---ci---` blocks alone, and it matches the `.ciagent/` file contents.
|
||||
| Commit | Status | .ciagent match |
|
||||
|--------|--------|----------------|
|
||||
| 1598c54 | verify | VERIFY.md updated, ROADMAP/REQUIREMENTS marked complete |
|
||||
| (specify was done in prior commit ae86a29 for phase 17) | | |
|
||||
|
||||
### Phase progression (walk-back through ci blocks)
|
||||
ROADMAP.md has Phase 18 with `Status: complete (v1.3.2)`.
|
||||
REQUIREMENTS.md has REQ-39, REQ-40, REQ-41, REQ-42 marked `complete (v1.3.2)`.
|
||||
VERIFY.md has `VERIFY PASS` verdict.
|
||||
Tag `v1.3.2` exists. **PASS.**
|
||||
|
||||
Each phase (11–16) shows the documented plan-as-execute → shipped → verify progression:
|
||||
## 2. File Discipline
|
||||
|
||||
| Phase | plan-as-execute commit | ship commit (release.tag) | post-ship traceability |
|
||||
|-------|--------------------------|----------------------------|------------------------|
|
||||
| 11 | 1ad9c35 + 81c6e39 | 87febc7 (`v1.2.1`) | 7ee57aa |
|
||||
| 12 | 0fea29c | 599db2e (`v1.2.2`) | 4c8de8e |
|
||||
| 13 | 4ed2542 | 5a3ab5e (`v1.2.3`) | 7c6b8c8 |
|
||||
| 14 | d103a37 | a3c7330 (`v1.2.4`) | d5cc01e |
|
||||
| 15 | 699aa54 | b993c15 (`v1.2.5`, PARTIAL) | 3cca5bb |
|
||||
| 16 | 64d35c7 | 3bb44d9 (`v1.2.6`) | faea213 |
|
||||
Working tree clean. All new files present (pyproject.toml,
|
||||
requirements-test.txt, 7 test files, 2 workflow YAMLs). Modified files
|
||||
(run_platform.sh, README.md, terraform/spike/terraform.tf) are expected.
|
||||
**PASS.**
|
||||
|
||||
Then the milestone tail: 18875cd (`status: review`, `verdict: READY TO SHIP`). ✅
|
||||
## 3. Branch Hygiene
|
||||
|
||||
### Tags
|
||||
On `main`, no stale phase branches. `milestone/v1.0-initial` is
|
||||
historical. **PASS.**
|
||||
|
||||
`git tag --list` returns the expected set:
|
||||
- `v1.2.0` (v1.1 milestone ship, preserved)
|
||||
- `v1.2.1`..`v1.2.6` (v1.2 phase patches 11–16)
|
||||
- `v1.3.0` (v1.2 milestone ship)
|
||||
## 4. Commit Discipline
|
||||
|
||||
All present; no missing; no extra. ✅
|
||||
All phase-18 commits have `---ci---` blocks with correct closing
|
||||
`---/ci---` tag. Tag `v1.3.2` follows NFR patch versioning (v1.3.1 →
|
||||
v1.3.2). **PASS.**
|
||||
|
||||
### ROADMAP.md ↔ tags
|
||||
## Verdict
|
||||
|
||||
- Phase 11 → `complete (v1.2.1)` ✅
|
||||
- Phase 12 → `complete (v1.2.2)` ✅
|
||||
- Phase 13 → `complete (v1.2.3)` ✅
|
||||
- Phase 14 → `complete (v1.2.4)` ✅
|
||||
- Phase 15 → `complete (v1.2.5, PARTIAL — terraform apply blocked by IAM P0)` ✅
|
||||
- Phase 16 → `complete (v1.2.6, capstone — terraform apply blocked by IAM P0, verified up to plan)` ✅
|
||||
|
||||
### REQUIREMENTS.md ↔ tags
|
||||
|
||||
| REQ | Phase | Status (file) | Tag (git) | Match |
|
||||
|-----|-------|---------------|-----------|-------|
|
||||
| REQ-29 | 11 | complete (v1.2.1) | v1.2.1 | ✅ |
|
||||
| REQ-30 | 12 | complete (v1.2.2) | v1.2.2 | ✅ |
|
||||
| REQ-31 | 13 | complete (v1.2.3) | v1.2.3 | ✅ |
|
||||
| REQ-32 | 14 | complete (v1.2.4) | v1.2.4 | ✅ |
|
||||
| REQ-33 | 15 | partial (v1.2.5, IAM-blocked) | v1.2.5 | ✅ |
|
||||
| REQ-34 | 15 | complete (v1.2.5) | v1.2.5 | ✅ |
|
||||
| REQ-35 | 16 | partial (v1.2.6, IAM-blocked) | v1.2.6 | ✅ |
|
||||
|
||||
**Reconstruction conclusion:** No drift. ✅
|
||||
|
||||
---
|
||||
|
||||
## 2. .ciagent/ file discipline
|
||||
|
||||
**PASS.** All 10 required files present; latest-phase PLAN/VERIFY in place; no orphans; no stale v1.1 framing.
|
||||
|
||||
| File | Exists | Notes |
|
||||
|------|--------|-------|
|
||||
| `config.json` | ✅ | mode=single, active_project=acdl, milestone=v1.2 |
|
||||
| `PROJECT.md` | ✅ | v1.2 objective + 6-phase table + D-047..D-049 |
|
||||
| `ARCHITECTURE.md` | ✅ | v1.2 build-out scope section |
|
||||
| `REQUIREMENTS.md` | ✅ | REQ-29..35 traceability |
|
||||
| `ROADMAP.md` | ✅ | v1.2 section, phases 11–16 complete |
|
||||
| `PERSONAS.md` | ✅ | P1-B fixed (platform/registry -> modules-ir/registry.json) |
|
||||
| `PLAN.md` | ✅ | Phase 16 (the last phase) |
|
||||
| `RESEARCH.md` | ✅ | v1.2 addendum (Targets 9–13, D-047/D-048/D-049) |
|
||||
| `VERIFY.md` | ✅ | Phase 16 capstone |
|
||||
| `REVIEW.md` | ✅ | v1.2 review — READY TO SHIP |
|
||||
| `AUDIT.md` | ✅ | this file |
|
||||
|
||||
No stale v1.1 framing in v1.2 files. ✅
|
||||
|
||||
---
|
||||
|
||||
## 3. Branch hygiene
|
||||
|
||||
**PASS.** Clean branch topology, clean working tree.
|
||||
|
||||
- `main` (current)
|
||||
- `milestone/v1.0-initial` (v1.0 milestone branch, retained)
|
||||
- `remotes/origin/main`
|
||||
- `remotes/origin/milestone/v1.0-initial`
|
||||
|
||||
**No leftover `phase/NN-*` branches** (all 6 phase branches deleted post-merge). Working tree clean. ✅
|
||||
|
||||
---
|
||||
|
||||
## 4. Commit discipline
|
||||
|
||||
**PASS.** 24 commits in `v1.2.0..HEAD`, all carry a well-formed `---ci---` block with `project/phase/milestone/status` from the documented set.
|
||||
|
||||
- `release.tag` appears only on the 6 ship commits (v1.2.1..v1.2.6). ✅
|
||||
- `verdict` appears only on the 6 verify commits + 1 review commit. ✅
|
||||
- `requirements.covered` on plan-as-execute + complete commits. ✅
|
||||
- `blocker` field on Phase 15/16 commits (P0-IAM documented). ✅
|
||||
- Merges: exactly the 6 documented `--no-ff` squash-merge ship commits. ✅
|
||||
- All ci blocks close with `---/ci---`. ✅
|
||||
|
||||
---
|
||||
|
||||
## P0 / P1
|
||||
|
||||
- **P0: 1 (operator action, NOT a code defect).** The `terraform apply` is blocked by the live IAM policy. This is not a code fix — the plan is valid (13 to add). Unblock: operator pushes `spike_runner_policy.json` via `create_iam_user.py`. Documented in REVIEW.md, Phase 15/16 VERIFY.md, the ship commit ci blocks. Non-blocking for the milestone ship (the code is complete + verified up to the apply).
|
||||
- **P1: 1 (adapter hardening, deferred to v1.3).** The adapter's ECS/ALB/VPC defaults (`desired_count`, `launch_type`, `target_type`, `tags`, `family`) should be parameterized via the L1 interfaces in v1.3.
|
||||
|
||||
---
|
||||
|
||||
## Final verdict
|
||||
|
||||
**v1.2 milestone audit: CLEAN.**
|
||||
|
||||
- 0 P0 code issues (the 1 P0 is an operator action, not a code defect).
|
||||
- 1 P1 post-hoc (adapter hardening, deferred to v1.3).
|
||||
- The milestone is shippable. The `v1.3.0` tag on main HEAD is valid.
|
||||
- The Gitea release for v1.3.0 is not yet created (blocked by the missing `ACDL_GITEA_TOKEN` — a documented manual step; the tag is pushed).
|
||||
**AUDIT CLEAN** — reconstruction, file discipline, branch hygiene, and
|
||||
commit discipline all pass. No critical issues.
|
||||
+48
-2
@@ -84,7 +84,7 @@ id 202 published. D-034 closed (root key deactivated by user).**
|
||||
Milestone COMPLETE gate: review → ship `v1.2.0` (feature milestone, next
|
||||
minor per ship.md) → audit. **DONE.**
|
||||
|
||||
## Objective for Milestone v1.2 (active)
|
||||
## Objective for Milestone v1.2 (prior — complete)
|
||||
|
||||
Platform hardening + first real consumer deployment. The v1.1 spike proved
|
||||
the IR commitments hold on a single dev-only `terraform plan` for one S3
|
||||
@@ -133,6 +133,40 @@ microservice), not just a plan.
|
||||
Milestone COMPLETE gate: review → ship `v1.3.0` (feature milestone, next
|
||||
minor per ship.md — v1.1 shipped `v1.2.0`) → audit.
|
||||
|
||||
## Objective for Milestone v1.4 (active)
|
||||
|
||||
Central pipeline contract + shell reproducibility + output streaming. The
|
||||
v1.3 milestone (Phases 17–18) created identical CI/CD pipelines for Gitea
|
||||
and GitHub but they were duplicated copies with no single source of truth.
|
||||
v1.4 makes the pipeline a declarative contract, enables full shell
|
||||
reproducibility, and streams terraform/checkov output so users can see
|
||||
what the platform is doing.
|
||||
|
||||
Three scope axes:
|
||||
|
||||
1. **Central pipeline contract.** A JSON Schema
|
||||
(`schemas/pipeline.schema.json`) + YAML instance (`pipelines/ci.yaml`)
|
||||
declares the pipeline stages, commands, triggers, and runner. Both
|
||||
`.gitea/workflows/ci.yml` (Gitea Actions, dev) and
|
||||
`.github/workflows/ci.yml` (GitHub Actions, production) implement the
|
||||
contract. A test validates conformance.
|
||||
2. **Shell reproducibility.** `scripts/run_ci.sh` mirrors the CI pipeline
|
||||
locally — runs the same 3 stages (lint, test, check-only) in sequence.
|
||||
The pipeline is fully reproducible from the shell, not just in CI.
|
||||
3. **Output streaming.** `scripts/run_platform.sh` streams terraform
|
||||
init/validate/plan output, Checkov compliance results, and
|
||||
PolicyCheckResult records to stdout by default, so the user sees what
|
||||
is happening. A `--quiet` flag suppresses streaming for log-only mode.
|
||||
|
||||
## Milestone v1.4 Phases
|
||||
|
||||
| Phase | Name | Goal |
|
||||
|-------|------|------|
|
||||
| 19 | central-pipeline-contract-and-shell-reproducibility | Create the central pipeline contract (JSON Schema + YAML instance). Create `scripts/run_ci.sh` for shell reproducibility. Update `run_platform.sh` to stream terraform/checkov output. Update both workflow YAMLs with contract references (staying byte-identical). Add tests for contract validation, workflow conformance, and streaming. |
|
||||
|
||||
Milestone COMPLETE gate: review → ship `v1.4.1` (feature milestone, next
|
||||
minor per ship.md — v1.3 shipped `v1.3.2`) → audit.
|
||||
|
||||
## Requirements
|
||||
|
||||
### v1.0 (Prior milestone — the demo)
|
||||
@@ -168,7 +202,7 @@ New requirements REQ-16..REQ-28 — see `REQUIREMENTS.md` §v1.1. Summary:
|
||||
- **REQ-28:** Spike verification proves the IR-shaped commitments hold (no
|
||||
polyglot mess; the adapter is the only substrate-specific code).
|
||||
|
||||
### v1.2 (Active milestone — platform hardening + first real consumer deployment)
|
||||
### v1.2 (Prior milestone — platform hardening + first real consumer deployment, complete)
|
||||
|
||||
New requirements REQ-29..REQ-35 — see `REQUIREMENTS.md` §v1.2. Summary:
|
||||
|
||||
@@ -189,6 +223,18 @@ New requirements REQ-29..REQ-35 — see `REQUIREMENTS.md` §v1.2. Summary:
|
||||
- **REQ-35:** End-to-end verification — consumer commit → live ECS service
|
||||
(HTTP 200) → evidence event → timeline.
|
||||
|
||||
### v1.4 (Active milestone — central pipeline contract + shell reproducibility + streaming)
|
||||
|
||||
New requirements REQ-43..REQ-45 — see `REQUIREMENTS.md` §v1.4. Summary:
|
||||
|
||||
- **REQ-43:** Central pipeline contract — `schemas/pipeline.schema.json` +
|
||||
`pipelines/ci.yaml`. Both Gitea and GitHub workflows implement the
|
||||
contract; a test validates conformance.
|
||||
- **REQ-44:** `scripts/run_ci.sh` mirrors the CI pipeline locally (lint →
|
||||
test → check-only), exiting 0 with "CI PIPELINE OK".
|
||||
- **REQ-45:** `scripts/run_platform.sh` streams terraform/checkov output by
|
||||
default (with `--quiet` for log-only mode). Both workflows byte-identical.
|
||||
|
||||
## Constraints
|
||||
|
||||
- **Forge:** Gitea at `https://git.cloudinit.dev`, org `continuous-intelligence`.
|
||||
|
||||
@@ -91,13 +91,35 @@
|
||||
### Category: End-to-End Verification
|
||||
- **REQ-35:** One end-to-end flow: consumer commit to `acdl-consumer-microservice` → pipeline triggered → contract→IR resolution → `terraform plan` → `terraform apply` (dev) → a live ECS Fargate service serving HTTP 200 on its ALB → evidence event written to the DynamoDB outbox → the event renders on the `acdl-evidence` timeline. `scripts/verify_phase16.sh` proves the full flow green.
|
||||
|
||||
## v1.3 (Active — module documentation + thin-composition removal)
|
||||
## v1.3 (Prior — module documentation + thin-composition removal, complete)
|
||||
|
||||
### Category: Thin-Composition Removal
|
||||
- **REQ-36:** The L2 thin-composition layer is removed completely: `composition.json` files, `acdl_platform/contract_resolver.py`, `schemas/contract.schema.json`, `contracts/spike.yaml`, `contracts/microservice.yaml`, and L2 entries in `modules-ir/registry.json` are deleted. The L2 directories are kept as placeholders with READMEs. The downstream pipeline (adapter → checkov → confidence → outbox) is patched to load a pre-existing IR instance instead of resolving a contract.
|
||||
- **REQ-37:** A `modules-ir/README-TEMPLATE.md` exists that works for both L1 and L2 modules, written in plain language (no jargon), with sections for Overview, Resources, Inputs, Outputs, Usage, Compliance extension points, and Versioning.
|
||||
- **REQ-38:** Every module has a `README.md`: the 7 L1 modules have full READMEs with Resources/Inputs/Outputs/Usage/Compliance-extension-points/Versioning sections derived from their `interface.json`; the 2 L2 modules have placeholder READMEs noting the composition is under redesign. A `modules-ir/README.md` catalog index lists all modules with one-line descriptions and links.
|
||||
|
||||
### Category: Testing
|
||||
- **REQ-39:** A pytest test suite exists under `tests/` covering the platform components offline (no AWS, no Checkov, no DynamoDB): the Terraform adapter (`adapters/terraform/adapter.py`), the confidence signal (`acdl_platform/confidence_signal.py`), the Checkov adapter (`adapters/terraform/policy/checkov_adapter.py`), and the outbox writer (`acdl_platform/outbox_writer.py`). The suite validates the IR schema, registry, spike_instance, and adapter output structure. `pyproject.toml` + `requirements-test.txt` pin test dependencies (pytest, jsonschema, pyyaml, boto3-stubs or moto for outbox mocking).
|
||||
|
||||
### Category: Shell Reproducibility
|
||||
- **REQ-40:** `scripts/run_platform.sh` has a `--check-only` mode that runs offline: loads the pre-existing IR instance, runs the adapter to emit Terraform, validates the JSON structure — without AWS credentials, Checkov, or DynamoDB. The existing `--plan-only` and full modes continue to require AWS. The `--check-only` mode is what CI pipelines run.
|
||||
|
||||
### Category: CI/CD Pipelines
|
||||
- **REQ-41:** Identical CI/CD pipelines exist for both Gitea Actions (`.gitea/workflows/ci.yml`, dev environment) and GitHub Actions (`.github/workflows/ci.yml`, production). Both run the same three stages: (1) lint — `py_compile` all Python files, (2) test — `pytest`, (3) check-only — `bash scripts/run_platform.sh --check-only`. Both trigger on push to main + pull request. Both use `ubuntu-latest`. Identical outcomes — the only difference is the runner environment.
|
||||
|
||||
- **REQ-42:** `pyproject.toml` exists at the repo root with pytest configuration (testpaths, markers) and the project metadata. `requirements-test.txt` pins test-only dependencies separate from runtime dependencies.
|
||||
|
||||
## v1.4 (Active — central pipeline contract + shell reproducibility + streaming)
|
||||
|
||||
### Category: Central Pipeline Contract
|
||||
- **REQ-43:** A central pipeline contract exists as `schemas/pipeline.schema.json` (JSON Schema draft 2020-12) + `pipelines/ci.yaml` (YAML instance). The contract declares the pipeline name, triggers (push/PR branches), runner, Python version, and stages (name + command + required + install + description). Both `.gitea/workflows/ci.yml` (Gitea Actions, dev) and `.github/workflows/ci.yml` (GitHub Actions, production) implement the same stages, commands, triggers, and runner as declared in the contract. A test (`tests/test_pipeline_contract.py`) validates the contract against the schema and asserts both workflows conform (same jobs, same commands, same triggers, same runner, byte-identical).
|
||||
|
||||
### Category: Shell Reproducibility
|
||||
- **REQ-44:** `scripts/run_ci.sh` reproduces the CI pipeline locally — runs the same 3 stages (lint, test, check-only) in sequence with proper exit codes, failing on first error. The script exits 0 with "CI PIPELINE OK" on success. A `--quiet` flag suppresses per-stage banners. The script mirrors the central pipeline contract (`pipelines/ci.yaml`) so the shell and CI environments produce identical outcomes.
|
||||
|
||||
### Category: Pipeline Streaming
|
||||
- **REQ-45:** `scripts/run_platform.sh` streams output by default: terraform init/validate/plan output is piped to stdout via `tee` (visible to the user and logged), Checkov results are printed in human-readable form, and PolicyCheckResult records are displayed with severity, rule ID, and pass/fail status per record. The `--check-only` mode streams the emitted Terraform file content. A `--quiet` flag suppresses streaming (output to log files only) for backwards compatibility. Both gitea and github workflows are byte-identical (identical outcomes — the only difference is the forge runtime).
|
||||
|
||||
## Out of Scope (v1.2)
|
||||
|
||||
| REQ | Original criterion | Clarified criterion (effective) | Decision |
|
||||
@@ -168,10 +190,22 @@
|
||||
| REQ-34 | 15 | complete (v1.2.5) |
|
||||
| REQ-35 | 16 | partial (v1.2.6, IAM-blocked) |
|
||||
|
||||
### v1.3 (active — module documentation + thin-composition removal)
|
||||
### v1.3 (prior — module documentation + thin-composition removal, complete)
|
||||
|
||||
| Requirement | Phase | Status |
|
||||
|-------------|-------|--------|
|
||||
| REQ-36 | 17 | complete (v1.3.1) |
|
||||
| REQ-37 | 17 | complete (v1.3.1) |
|
||||
| REQ-38 | 17 | complete (v1.3.1) |
|
||||
| REQ-38 | 17 | complete (v1.3.1) |
|
||||
| REQ-39 | 18 | complete (v1.3.2) |
|
||||
| REQ-40 | 18 | complete (v1.3.2) |
|
||||
| REQ-41 | 18 | complete (v1.3.2) |
|
||||
| REQ-42 | 18 | complete (v1.3.2) |
|
||||
|
||||
### v1.4 (active — central pipeline contract + shell reproducibility + streaming)
|
||||
|
||||
| Requirement | Phase | Status |
|
||||
|-------------|-------|--------|
|
||||
| REQ-43 | 19 | complete (v1.4.1) |
|
||||
| REQ-44 | 19 | complete (v1.4.1) |
|
||||
| REQ-45 | 19 | complete (v1.4.1) |
|
||||
+40
-3
@@ -5,7 +5,8 @@
|
||||
- **v1.0 (demo):** complete — tag `v1.1.0`, 2026-07-21. All 5 phases shipped + audited PASS.
|
||||
- **v1.1 (complete):** architecture finalization + v1 spike. 5 phases (06–10). Tag `v1.2.0`, 2026-07-21. All 5 phases shipped + verified; review READY TO SHIP (0 P0); audit CLEAN. Gitea release id 202.
|
||||
- **v1.2 (complete):** platform hardening + first real consumer deployment. 6 phases (11–16). Tag `v1.3.0`, 2026-07-21. All 6 phases shipped + verified; review READY TO SHIP (1 P0 operator action, 1 P1 deferred); audit CLEAN.
|
||||
- **v1.3 (active):** module documentation + thin-composition removal. The L2 composition layer is removed; module READMEs are built out.
|
||||
- **v1.3 (complete):** module documentation + thin-composition removal. The L2 composition layer is removed; module READMEs are built out. Tag `v1.3.2`.
|
||||
- **v1.4 (active):** central pipeline contract + shell reproducibility + output streaming. A declarative pipeline contract (`schemas/pipeline.schema.json` + `pipelines/ci.yaml`) binds the Gitea and GitHub workflows to a single source of truth. `scripts/run_ci.sh` mirrors the CI pipeline locally. `scripts/run_platform.sh` streams terraform/checkov output by default.
|
||||
- **v1.0 demo URL:** https://git.cloudinit.dev/continuous-intelligence/acdl-evidence/raw/branch/main/index.html
|
||||
|
||||
---
|
||||
@@ -221,7 +222,7 @@ After Phase 16: COMPLETE gate — review → ship `v1.3.0` → audit.
|
||||
|
||||
---
|
||||
|
||||
## v1.3 (Active — module documentation + thin-composition removal)
|
||||
## v1.3 (Complete — module documentation + thin-composition removal)
|
||||
|
||||
The v1.3 milestone starts with simplification: removing the unsatisfactory
|
||||
thin-composition layer and building out proper module documentation. The
|
||||
@@ -238,4 +239,40 @@ L2 composition mechanism will be redesigned in a later phase.
|
||||
- A README-TEMPLATE.md exists for both L1 and L2 modules.
|
||||
- Every L1 module has a README.md with Resources/Inputs/Outputs/Usage/Compliance-extension-points/Versioning.
|
||||
- Every L2 module has a placeholder README.md noting the composition is under redesign.
|
||||
- A modules-ir/README.md catalog index exists.
|
||||
- A modules-ir/README.md catalog index exists.
|
||||
|
||||
### Phase 18 — testing-and-cicd-pipelines
|
||||
- **Description:** Create a pytest test suite that reproduces the platform pipeline offline (adapter, confidence_signal, checkov_adapter, outbox_writer). Add an offline `--check-only` mode to `run_platform.sh` that runs the pipeline up to adapter emission without AWS/Checkov/outbox. Create identical CI/CD pipelines for both Gitea Actions (`.gitea/workflows/ci.yml`, dev environment) and GitHub Actions (`.github/workflows/ci.yml`, production) that run: lint, pytest, `run_platform.sh --check-only`. Add `pyproject.toml` + `requirements-test.txt` for dependency pinning.
|
||||
- **Status:** complete (v1.3.2)
|
||||
- **Depends on:** [17]
|
||||
- **Requirements:** REQ-39, REQ-40, REQ-41, REQ-42
|
||||
- **Success Criteria:**
|
||||
- `pytest` runs and passes offline (no AWS, no Checkov, no DynamoDB).
|
||||
- `run_platform.sh --check-only` runs offline and exits 0.
|
||||
- `.gitea/workflows/ci.yml` and `.github/workflows/ci.yml` exist with identical job stages (lint, test, check-only).
|
||||
- `pyproject.toml` + `requirements-test.txt` pin test dependencies.
|
||||
|
||||
After Phase 18: COMPLETE gate — review → ship `v1.3.2` → audit.
|
||||
|
||||
---
|
||||
|
||||
## v1.4 (Active — central pipeline contract + shell reproducibility + streaming)
|
||||
|
||||
The v1.4 milestone makes the CI/CD pipeline a declarative contract rather
|
||||
than duplicated workflow copies, enables full shell reproducibility of the
|
||||
CI pipeline, and streams terraform/checkov output so users can see what
|
||||
the platform is doing.
|
||||
|
||||
### Phase 19 — central-pipeline-contract-and-shell-reproducibility
|
||||
- **Description:** Create a central pipeline contract (`schemas/pipeline.schema.json` JSON Schema + `pipelines/ci.yaml` YAML instance) that both `.gitea/workflows/ci.yml` (Gitea Actions, dev) and `.github/workflows/ci.yml` (GitHub Actions, production) implement. Create `scripts/run_ci.sh` that mirrors the CI pipeline locally (lint → test → check-only). Update `scripts/run_platform.sh` to stream terraform init/validate/plan output, Checkov compliance results, and PolicyCheckResult records to stdout by default (with `--quiet` for log-only mode). Add `tests/test_pipeline_contract.py` validating the contract schema, workflow conformance, and run_ci.sh. Update both workflow YAMLs with contract reference headers (staying byte-identical).
|
||||
- **Status:** complete (v1.4.1)
|
||||
- **Depends on:** [18]
|
||||
- **Requirements:** REQ-43, REQ-44, REQ-45
|
||||
- **Success Criteria:**
|
||||
- `pipelines/ci.yaml` validates against `schemas/pipeline.schema.json`.
|
||||
- Both `.gitea/workflows/ci.yml` and `.github/workflows/ci.yml` are byte-identical.
|
||||
- A test parses both workflows and asserts their stages/commands match the contract.
|
||||
- `scripts/run_ci.sh` exits 0 and outputs "CI PIPELINE OK".
|
||||
- `scripts/run_platform.sh --check-only` streams the emitted Terraform to stdout.
|
||||
- `scripts/run_platform.sh --check-only --quiet` suppresses the Terraform stream.
|
||||
- `pytest` total count increases from 90 to 122 (32 new contract/streaming tests).
|
||||
+29
-31
@@ -1,47 +1,45 @@
|
||||
# Phase 17 — Verify (v1.3.1)
|
||||
# Phase 18 — Verify (v1.3.2)
|
||||
|
||||
## Structural
|
||||
|
||||
All 6 deleted files confirmed gone. All 7 kept files confirmed present.
|
||||
All 11 new files confirmed present. Registry has 7 L1 entries, 0 L2
|
||||
entries. `contracts/` directory deleted. L2 directories kept as
|
||||
placeholders with READMEs. **PASS.**
|
||||
All 11 new files confirmed present: pyproject.toml, requirements-test.txt,
|
||||
tests/__init__.py, tests/conftest.py, tests/test_adapter.py,
|
||||
tests/test_confidence_signal.py, tests/test_checkov_adapter.py,
|
||||
tests/test_outbox_writer.py, tests/test_pipeline.py,
|
||||
.gitea/workflows/ci.yml, .github/workflows/ci.yml. **PASS.**
|
||||
|
||||
## Behavioral
|
||||
|
||||
- All remaining Python files compile (`py_compile` OK).
|
||||
- `scripts/run_platform.sh` syntax valid; loads pre-existing IR instance
|
||||
(`modules-ir/l1/l1-s3/spike_instance.json`); no code calls to
|
||||
`contract_resolver.py` (only a documentation comment noting removal).
|
||||
- Adapter successfully compiles the pre-existing IR instance to
|
||||
Terraform (`main.tf` + `terraform.tf` + `providers.tf` emitted).
|
||||
- `registry.json` is valid JSON with only L1 entries.
|
||||
- `schemas/ir.schema.json` is valid JSON.
|
||||
- `spike_instance.json` validates against `ir.schema.json`.
|
||||
- No dangling references to deleted files in active code (only the
|
||||
documentation comment in `run_platform.sh` and historical
|
||||
`.ciagent/` + `verify_phaseNN.sh` files, which are expected).
|
||||
- `py_compile` passes on all Python files. **PASS.**
|
||||
- `pytest` — 90 tests, all passing, all offline (moto for DynamoDB
|
||||
mocking). **PASS.**
|
||||
- `run_platform.sh --check-only` — exits 0, outputs
|
||||
"PLATFORM CHECK OK", requires no AWS credentials. **PASS.**
|
||||
- `run_platform.sh --plan-only` — syntax valid (unchanged from phase 17).
|
||||
**PASS.**
|
||||
- Both workflow YAMLs are valid YAML, parseable. **PASS.**
|
||||
- Workflows are byte-identical (diff confirms). **PASS.**
|
||||
|
||||
## Security
|
||||
|
||||
- No secrets in new files. **PASS.**
|
||||
- P1 (deferred): `modules-ir/l1/l1-ecs-service/README.md` usage example
|
||||
contains the AWS account ID `581513795199` (same as the existing
|
||||
`terraform/microservice/main.tf`). Not a new leak — the account ID was
|
||||
already in the repo. Recommend replacing with a placeholder in a
|
||||
future docs pass.
|
||||
- No secrets in any new file (tests, workflows, pyproject, requirements).
|
||||
**PASS.**
|
||||
- CI pipelines do not use any AWS credentials — `--check-only` is fully
|
||||
offline. **PASS.**
|
||||
|
||||
## Quality
|
||||
|
||||
- README template has all required sections (Overview, Resources,
|
||||
Inputs, Outputs, Usage, Compliance extension points, Versioning)
|
||||
after adding the `## Overview` header. **PASS.**
|
||||
- All 7 L1 READMEs have all required sections. **PASS.**
|
||||
- Both L2 READMEs have placeholder notes mentioning redesign. **PASS.**
|
||||
- Catalog index lists all 7 L1s + 2 L2s. **PASS.**
|
||||
- pyproject.toml has pytest config (testpaths, markers, addopts).
|
||||
**PASS.**
|
||||
- requirements-test.txt pins all test deps. **PASS.**
|
||||
- Test suite covers all 4 platform components (adapter, confidence
|
||||
signal, checkov adapter, outbox writer) + pipeline integration.
|
||||
**PASS.**
|
||||
- Both workflows run 3 stages: lint, test, check-only. **PASS.**
|
||||
- README updated with "Test the platform" section + CI/CD documentation.
|
||||
**PASS.**
|
||||
|
||||
## Verdict
|
||||
|
||||
**VERIFY PASS** — all four layers pass. One P1 (account ID in usage
|
||||
example) deferred to post-hoc review.
|
||||
**VERIFY PASS** — all four layers pass. 90 offline tests, no AWS
|
||||
required for CI.
|
||||
@@ -4,7 +4,7 @@
|
||||
{
|
||||
"slug": "acdl",
|
||||
"name": "Agentic Cloud Delivery Platform",
|
||||
"milestone": "v1.3",
|
||||
"milestone": "v1.4",
|
||||
"status": "active"
|
||||
}
|
||||
],
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
# ACDL CI Pipeline — Gitea Actions (dev environment)
|
||||
#
|
||||
# This workflow implements the central pipeline contract:
|
||||
# pipelines/ci.yaml (validated against schemas/pipeline.schema.json)
|
||||
#
|
||||
# The same contract is implemented by .github/workflows/ci.yml (GitHub
|
||||
# Actions, production). Both files must be byte-identical — the only
|
||||
# declared difference is the forge/runtime, not the stages or commands.
|
||||
#
|
||||
# Shell reproducibility: scripts/run_ci.sh runs the same 3 stages locally.
|
||||
#
|
||||
# Stages (from the contract):
|
||||
# 1. lint — py_compile all Python files
|
||||
# 2. test — pytest test suite (offline, no AWS)
|
||||
# 3. check-only — run_platform.sh --check-only (offline, no AWS)
|
||||
name: acdl-ci
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
|
||||
jobs:
|
||||
lint:
|
||||
name: Lint
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Compile all Python files
|
||||
run: |
|
||||
python3 -m py_compile \
|
||||
acdl_platform/confidence_signal.py \
|
||||
acdl_platform/outbox_writer.py \
|
||||
adapters/terraform/adapter.py \
|
||||
adapters/terraform/policy/checkov_adapter.py \
|
||||
scripts/push_consumer_image.py
|
||||
|
||||
test:
|
||||
name: Test
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install test dependencies
|
||||
run: pip install -r requirements-test.txt
|
||||
|
||||
- name: Run pytest
|
||||
run: python3 -m pytest tests/ -v --tb=short
|
||||
|
||||
check-only:
|
||||
name: Platform check-only (offline)
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install runtime dependencies
|
||||
run: pip install jsonschema pyyaml boto3
|
||||
|
||||
- name: Run platform check-only
|
||||
run: bash scripts/run_platform.sh --check-only
|
||||
@@ -0,0 +1,74 @@
|
||||
# ACDL CI Pipeline — Gitea Actions (dev environment)
|
||||
#
|
||||
# This workflow implements the central pipeline contract:
|
||||
# pipelines/ci.yaml (validated against schemas/pipeline.schema.json)
|
||||
#
|
||||
# The same contract is implemented by .github/workflows/ci.yml (GitHub
|
||||
# Actions, production). Both files must be byte-identical — the only
|
||||
# declared difference is the forge/runtime, not the stages or commands.
|
||||
#
|
||||
# Shell reproducibility: scripts/run_ci.sh runs the same 3 stages locally.
|
||||
#
|
||||
# Stages (from the contract):
|
||||
# 1. lint — py_compile all Python files
|
||||
# 2. test — pytest test suite (offline, no AWS)
|
||||
# 3. check-only — run_platform.sh --check-only (offline, no AWS)
|
||||
name: acdl-ci
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
|
||||
jobs:
|
||||
lint:
|
||||
name: Lint
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Compile all Python files
|
||||
run: |
|
||||
python3 -m py_compile \
|
||||
acdl_platform/confidence_signal.py \
|
||||
acdl_platform/outbox_writer.py \
|
||||
adapters/terraform/adapter.py \
|
||||
adapters/terraform/policy/checkov_adapter.py \
|
||||
scripts/push_consumer_image.py
|
||||
|
||||
test:
|
||||
name: Test
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install test dependencies
|
||||
run: pip install -r requirements-test.txt
|
||||
|
||||
- name: Run pytest
|
||||
run: python3 -m pytest tests/ -v --tb=short
|
||||
|
||||
check-only:
|
||||
name: Platform check-only (offline)
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install runtime dependencies
|
||||
run: pip install jsonschema pyyaml boto3
|
||||
|
||||
- name: Run platform check-only
|
||||
run: bash scripts/run_platform.sh --check-only
|
||||
@@ -14,11 +14,21 @@ a configuration file, or a Terraform module.
|
||||
|
||||
## Status
|
||||
|
||||
- **v1.2 (active):** platform hardening + first real consumer deployment.
|
||||
Harden the v1.1 spike's NFRs, simplify the setup, rewrite the docs, and
|
||||
prove the platform delivers real value by deploying a basic microservice
|
||||
to AWS ECS Fargate end-to-end (`terraform apply`, dev autonomous). Ship
|
||||
tag `v1.3.0`.
|
||||
- **v1.4 (active):** central pipeline contract + shell reproducibility +
|
||||
output streaming. A declarative pipeline contract
|
||||
(`schemas/pipeline.schema.json` + `pipelines/ci.yaml`) binds the Gitea
|
||||
and GitHub workflows to a single source of truth. `scripts/run_ci.sh`
|
||||
mirrors the CI pipeline locally. `scripts/run_platform.sh` streams
|
||||
terraform/checkov output by default. Ship tag `v1.4.1`.
|
||||
- **v1.3 (complete, tag `v1.3.2`):** module documentation + thin-composition
|
||||
removal. The L2 composition layer is removed; module READMEs are built
|
||||
out. Testing + CI/CD pipelines (pytest, `--check-only`, Gitea + GitHub
|
||||
workflows).
|
||||
- **v1.2 (complete, tag `v1.3.0`):** platform hardening + first real
|
||||
consumer deployment. Harden the v1.1 spike's NFRs, simplify the setup,
|
||||
rewrite the docs, and prove the platform delivers real value by
|
||||
deploying a basic microservice to AWS ECS Fargate end-to-end (`terraform
|
||||
apply`, dev autonomous).
|
||||
- **v1.1 (complete, tag `v1.2.0`):** architecture finalization + v1 spike.
|
||||
Finalized the architecture to v1.0 (resolved all 11 open design
|
||||
decisions) and proved the IR commitments hold with one end-to-end spike
|
||||
@@ -110,15 +120,77 @@ ACDL_BOOTSTRAP_AWS_ACCESS_KEY_ID=... ACDL_BOOTSTRAP_AWS_SECRET_ACCESS_KEY=... \
|
||||
ACDL_BOOTSTRAP_AWS_ACCESS_KEY_ID=... ACDL_BOOTSTRAP_AWS_SECRET_ACCESS_KEY=... \
|
||||
bash scripts/rotate_spike_key.sh
|
||||
|
||||
# 3. Run the full platform pipeline (contract -> IR -> plan -> Checkov ->
|
||||
# confidence -> outbox)
|
||||
# 3. Run the full platform pipeline (IR -> adapter -> plan -> Checkov ->
|
||||
# confidence -> outbox). Output is streamed to stdout by default.
|
||||
bash scripts/run_platform.sh
|
||||
# Expected: "=== PLATFORM E2E OK ==="
|
||||
|
||||
# Or plan-only (contract -> IR -> terraform plan; no Checkov/outbox):
|
||||
# Or plan-only (IR -> adapter -> terraform plan; no Checkov/outbox):
|
||||
bash scripts/run_platform.sh --plan-only
|
||||
|
||||
# Add --quiet to suppress streaming (output to log files only):
|
||||
bash scripts/run_platform.sh --quiet
|
||||
```
|
||||
|
||||
### Test the platform (offline, no AWS required)
|
||||
|
||||
```bash
|
||||
# Install test dependencies
|
||||
pip install -r requirements-test.txt
|
||||
|
||||
# Run the test suite (122 tests, all offline — uses moto for DynamoDB mocking)
|
||||
python3 -m pytest tests/ -v
|
||||
|
||||
# Run the platform in check-only mode (offline — no AWS, no Checkov, no outbox)
|
||||
# Streams the emitted Terraform to stdout by default; --quiet suppresses it
|
||||
bash scripts/run_platform.sh --check-only
|
||||
# Expected: "=== PLATFORM CHECK OK ==="
|
||||
|
||||
# Reproduce the full CI pipeline locally (lint → test → check-only)
|
||||
bash scripts/run_ci.sh
|
||||
# Expected: "=== CI PIPELINE OK ==="
|
||||
```
|
||||
|
||||
### CI/CD pipelines
|
||||
|
||||
The CI/CD pipeline is defined by a **central pipeline contract** — a
|
||||
declarative YAML instance (`pipelines/ci.yaml`) validated against a JSON
|
||||
Schema (`schemas/pipeline.schema.json`). Both forge workflows implement
|
||||
the same contract:
|
||||
|
||||
- `.gitea/workflows/ci.yml` — Gitea Actions (dev environment)
|
||||
- `.github/workflows/ci.yml` — GitHub Actions (production)
|
||||
|
||||
Both workflow files are **byte-identical** — the only difference is the
|
||||
forge runtime. Both run three stages: **lint** (py_compile), **test**
|
||||
(pytest), and **check-only** (`run_platform.sh --check-only`). Both
|
||||
trigger on push to `main` and on pull requests. A test
|
||||
(`tests/test_pipeline_contract.py`) validates that both workflows conform
|
||||
to the contract.
|
||||
|
||||
`scripts/run_ci.sh` mirrors the CI pipeline locally — running the same
|
||||
three stages in sequence. This makes the pipeline fully reproducible from
|
||||
the shell, not just in CI:
|
||||
|
||||
```bash
|
||||
bash scripts/run_ci.sh # run all 3 stages (lint, test, check-only)
|
||||
bash scripts/run_ci.sh --quiet # suppress per-stage banners
|
||||
```
|
||||
|
||||
### Output streaming (run_platform.sh)
|
||||
|
||||
`scripts/run_platform.sh` streams output by default so the user can see
|
||||
what the platform is doing:
|
||||
|
||||
- **`--check-only`**: streams the emitted Terraform file content to stdout
|
||||
- **`--plan-only`** and **full mode**: streams `terraform init`, `terraform
|
||||
validate`, and `terraform plan` output via `tee` (visible and logged)
|
||||
- **Full mode**: prints Checkov compliance results and each
|
||||
PolicyCheckResult record with severity, rule ID, and pass/fail status
|
||||
|
||||
A `--quiet` flag suppresses streaming (output to log files only) for
|
||||
backwards-compatible log-only mode.
|
||||
|
||||
### Re-run the archived v1.0 demo (stubs only, no AWS)
|
||||
|
||||
```bash
|
||||
@@ -133,13 +205,14 @@ behavior rather than provisioning real cloud resources.
|
||||
|
||||
| Path | Purpose | Status |
|
||||
|------|---------|--------|
|
||||
| `acdl_platform/` | Platform code: confidence signal, contract resolver, outbox writer, HITL/ledger/SoD designs (renamed from `platform/` in Phase 08 to avoid shadowing the stdlib `platform` module) | v1.1 complete; v1.2 extends |
|
||||
| `schemas/` | JSON Schemas: IR, PolicyCheckResult, contract (draft 2020-12) | v1.1 complete; v1.2 extends contract schema |
|
||||
| `acdl_platform/` | Platform code: confidence signal, outbox writer, separation of duties, HITL/ledger designs | v1.1 complete; v1.3 removes contract_resolver |
|
||||
| `schemas/` | JSON Schemas: IR, PolicyCheckResult, pipeline contract (draft 2020-12) | v1.1 complete; v1.4 adds pipeline schema |
|
||||
| `pipelines/` | Central pipeline contract: `ci.yaml` (YAML instance validated against `schemas/pipeline.schema.json`) | v1.4 |
|
||||
| `adapters/` | Substrate adapters — Terraform adapter (the only substrate-specific code per §12) + Checkov policy adapter | v1.1 complete; v1.2 expands `TYPE_MAP` |
|
||||
| `terraform/` | State backend (S3 + DynamoDB) + spike TF (`terraform/spike/`) + bootstrap scripts (`terraform/bootstrap/`) | v1.1 complete; v1.2 adds ECS apply |
|
||||
| `modules-ir/` | IR-typed L1/L2 modules + `registry.json`. v1.1: `l1-s3`, `l2-static-asset`. v1.2: + 6 ECS L1s, `l2-microservice` | v1.1 complete; v1.2 expands |
|
||||
| `contracts/` | Sample contracts (`spike.yaml` for `l2-static-asset`) | v1.1 complete; v1.2 adds `microservice.yaml` |
|
||||
| `scripts/` | Verify scripts (`verify_phaseNN.sh`), platform run script (`run_platform.sh`; `--plan-only` for plan subset), key rotation | v1.1 complete; v1.2 consolidates |
|
||||
| `modules-ir/` | IR-typed L1/L2 modules + `registry.json`. v1.1: `l1-s3`. v1.2: + 6 ECS L1s. v1.3: L2 removed (placeholders) | v1.3 |
|
||||
| `scripts/` | Platform run script (`run_platform.sh` with `--check-only`/`--plan-only`/`--quiet`), CI pipeline script (`run_ci.sh`), verify scripts, key rotation | v1.4 |
|
||||
| `tests/` | Pytest suite (122 tests, all offline — adapter, confidence signal, checkov adapter, outbox writer, pipeline contract, streaming) | v1.4 |
|
||||
| `demo/` | Archived v1.0 executive demo (tag `v1.1.0`); runs locally via `demo/scripts/run_demo.sh --no-upload` | complete (archived) |
|
||||
| `.ciagent/` | CIAgent metadata (config, project, architecture, requirements, roadmap, personas, plans, research, verify, review, audit) | active |
|
||||
| `docs/` | Upstream vision + architecture sources (`vision.md`, `architecture.md`) | active |
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
# ACDL Central Pipeline Contract (v1.4)
|
||||
#
|
||||
# This is the single source of truth for the CI/CD pipeline. Both
|
||||
# .gitea/workflows/ci.yml (Gitea Actions, dev) and
|
||||
# .github/workflows/ci.yml (GitHub Actions, production) implement the
|
||||
# stages, commands, triggers, and runner declared here.
|
||||
# scripts/run_ci.sh mirrors the same stages for shell reproducibility.
|
||||
#
|
||||
# A test (tests/test_pipeline_contract.py) validates that both workflow
|
||||
# YAMLs conform to this contract and that run_ci.sh runs the same commands.
|
||||
#
|
||||
# The contract does NOT replace workflow YAML syntax — it declares the
|
||||
# *intent* that the forge-specific workflows implement. The workflow files
|
||||
# use Gitea/GitHub Actions syntax (checkout, setup-python, run blocks);
|
||||
# this contract declares what those blocks must contain.
|
||||
#
|
||||
# Validated against schemas/pipeline.schema.json.
|
||||
|
||||
name: acdl-ci
|
||||
environment: dev
|
||||
triggers:
|
||||
push: [main]
|
||||
pull_request: [main]
|
||||
runner: ubuntu-latest
|
||||
python_version: "3.12"
|
||||
|
||||
stages:
|
||||
- name: lint
|
||||
description: Compile all Python files (py_compile)
|
||||
command: |
|
||||
python3 -m py_compile \
|
||||
acdl_platform/confidence_signal.py \
|
||||
acdl_platform/outbox_writer.py \
|
||||
adapters/terraform/adapter.py \
|
||||
adapters/terraform/policy/checkov_adapter.py \
|
||||
scripts/push_consumer_image.py
|
||||
required: true
|
||||
|
||||
- name: test
|
||||
description: Run the pytest test suite offline
|
||||
command: python3 -m pytest tests/ -v --tb=short
|
||||
install: pip install -r requirements-test.txt
|
||||
required: true
|
||||
|
||||
- name: check-only
|
||||
description: Run the platform pipeline offline (no AWS/Checkov/DynamoDB)
|
||||
command: bash scripts/run_platform.sh --check-only
|
||||
install: pip install jsonschema pyyaml boto3
|
||||
required: true
|
||||
@@ -0,0 +1,34 @@
|
||||
[project]
|
||||
name = "acdl"
|
||||
version = "1.3.0"
|
||||
description = "Agentic Cloud Delivery Platform — consumers declare intent; the platform delivers safe production deployment."
|
||||
requires-python = ">=3.10"
|
||||
dependencies = [
|
||||
"boto3>=1.34",
|
||||
"jsonschema>=4.20",
|
||||
"pyyaml>=6.0",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
test = [
|
||||
"pytest>=8.0",
|
||||
"pytest-cov>=4.0",
|
||||
"moto[dynamodb]>=5.0",
|
||||
]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
markers = [
|
||||
"offline: tests that run without AWS/Checkov/DynamoDB",
|
||||
]
|
||||
addopts = "-v --tb=short"
|
||||
filterwarnings = [
|
||||
"ignore::DeprecationWarning:botocore.*",
|
||||
]
|
||||
|
||||
[tool.coverage]
|
||||
run.source = ["acdl_platform", "adapters"]
|
||||
|
||||
[build-system]
|
||||
requires = ["setuptools>=68"]
|
||||
build-backend = "setuptools.backends._legacy:_Backend"
|
||||
@@ -0,0 +1,6 @@
|
||||
pytest>=8.0
|
||||
pytest-cov>=4.0
|
||||
moto[dynamodb]>=5.0
|
||||
jsonschema>=4.20
|
||||
pyyaml>=6.0
|
||||
boto3>=1.34
|
||||
@@ -0,0 +1,77 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://acdl.cloudinit.dev/schemas/pipeline.schema.json",
|
||||
"title": "ACDL Central Pipeline Contract",
|
||||
"description": "Declarative contract for a CI/CD pipeline. Both Gitea Actions (.gitea/workflows/ci.yml, dev) and GitHub Actions (.github/workflows/ci.yml, production) implement the stages, commands, triggers, and runner declared here. The shell script scripts/run_ci.sh mirrors the same stages for local reproducibility. The contract is the single source of truth; the workflow YAMLs and run_ci.sh are generated/validated against it.",
|
||||
"$comment": "The pipeline contract does not replace workflow YAML syntax — it declares the *intent* (stages, commands, triggers, runner) that both Gitea and GitHub workflows implement. A test (tests/test_pipeline_contract.py) validates conformance: the workflow YAMLs must declare the same jobs/stages/commands as the contract, and run_ci.sh must run the same commands in the same order.",
|
||||
"type": "object",
|
||||
"required": ["name", "triggers", "runner", "stages"],
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": "string",
|
||||
"description": "Pipeline name (matches the workflow 'name:' field)."
|
||||
},
|
||||
"environment": {
|
||||
"type": "string",
|
||||
"enum": ["dev", "production"],
|
||||
"description": "Declared environment. dev = Gitea Actions; production = GitHub Actions. Does not change job commands — only documents which forge runs this instance."
|
||||
},
|
||||
"triggers": {
|
||||
"type": "object",
|
||||
"required": ["push", "pull_request"],
|
||||
"properties": {
|
||||
"push": {
|
||||
"type": "array",
|
||||
"items": {"type": "string"},
|
||||
"description": "Branches that trigger the pipeline on push."
|
||||
},
|
||||
"pull_request": {
|
||||
"type": "array",
|
||||
"items": {"type": "string"},
|
||||
"description": "Branches that trigger the pipeline on PR."
|
||||
}
|
||||
}
|
||||
},
|
||||
"runner": {
|
||||
"type": "string",
|
||||
"description": "Runner image (e.g. 'ubuntu-latest'). Both Gitea and GitHub use the same runner label."
|
||||
},
|
||||
"python_version": {
|
||||
"type": "string",
|
||||
"description": "Python version for setup-python action."
|
||||
},
|
||||
"stages": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": {"$ref": "#/$defs/stage"}
|
||||
}
|
||||
},
|
||||
"$defs": {
|
||||
"stage": {
|
||||
"type": "object",
|
||||
"required": ["name", "command", "required"],
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": "string",
|
||||
"description": "Stage name (maps to the workflow job name)."
|
||||
},
|
||||
"command": {
|
||||
"type": "string",
|
||||
"description": "The shell command to run for this stage. Must be identical in the workflow YAML 'run:' block and in scripts/run_ci.sh."
|
||||
},
|
||||
"required": {
|
||||
"type": "boolean",
|
||||
"description": "If true, a non-zero exit code fails the pipeline."
|
||||
},
|
||||
"install": {
|
||||
"type": "string",
|
||||
"description": "Optional: pip install command to run before the stage command."
|
||||
},
|
||||
"description": {
|
||||
"type": "string",
|
||||
"description": "Optional: human-readable description of what this stage does."
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Executable
+64
@@ -0,0 +1,64 @@
|
||||
#!/usr/bin/env bash
|
||||
# scripts/run_ci.sh - reproduce the CI pipeline locally.
|
||||
#
|
||||
# Mirrors the central pipeline contract (pipelines/ci.yaml) which both
|
||||
# .gitea/workflows/ci.yml (Gitea Actions, dev) and
|
||||
# .github/workflows/ci.yml (GitHub Actions, production) implement.
|
||||
#
|
||||
# Runs the same three stages in the same order:
|
||||
# 1. lint — py_compile all Python files
|
||||
# 2. test — pytest test suite (offline)
|
||||
# 3. check-only — run_platform.sh --check-only (offline, no AWS)
|
||||
#
|
||||
# Fails on the first stage that errors. Exits 0 with "CI PIPELINE OK"
|
||||
# when all stages pass.
|
||||
#
|
||||
# Usage:
|
||||
# bash scripts/run_ci.sh # run all stages
|
||||
# bash scripts/run_ci.sh --quiet # suppress per-stage banners
|
||||
set -euo pipefail
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
cd "$ROOT"
|
||||
|
||||
QUIET=0
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--quiet) QUIET=1 ;;
|
||||
*) echo "FAIL: unknown argument: $arg" >&2; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
banner() {
|
||||
[ "$QUIET" = "1" ] || echo ""
|
||||
echo "── $1 ──"
|
||||
[ "$QUIET" = "1" ] || echo ""
|
||||
}
|
||||
|
||||
fail() { echo "FAIL: $*" >&2; exit 1; }
|
||||
|
||||
echo "=== ACDL CI Pipeline (local reproduction) ==="
|
||||
echo "contract: pipelines/ci.yaml (3 stages)"
|
||||
echo ""
|
||||
|
||||
banner "Stage 1/3: lint (py_compile)"
|
||||
python3 -m py_compile \
|
||||
acdl_platform/confidence_signal.py \
|
||||
acdl_platform/outbox_writer.py \
|
||||
adapters/terraform/adapter.py \
|
||||
adapters/terraform/policy/checkov_adapter.py \
|
||||
scripts/push_consumer_image.py \
|
||||
|| fail "lint: py_compile failed"
|
||||
echo "lint: OK"
|
||||
|
||||
banner "Stage 2/3: test (pytest)"
|
||||
python3 -m pytest tests/ -v --tb=short || fail "test: pytest failed"
|
||||
echo "test: OK"
|
||||
|
||||
banner "Stage 3/3: check-only (run_platform.sh --check-only)"
|
||||
bash scripts/run_platform.sh --check-only || fail "check-only: run_platform.sh failed"
|
||||
echo "check-only: OK"
|
||||
|
||||
echo ""
|
||||
echo "=== CI PIPELINE OK ==="
|
||||
echo "3 stages passed: lint, test, check-only"
|
||||
exit 0
|
||||
+110
-21
@@ -1,11 +1,18 @@
|
||||
#!/usr/bin/env bash
|
||||
# scripts/run_platform.sh - the ACDL platform pipeline.
|
||||
#
|
||||
# Default: full end-to-end pipeline (load pre-existing IR instance ->
|
||||
# adapter -> terraform plan (real AWS) -> Checkov -> PolicyCheckResult ->
|
||||
# confidence signal -> evidence event to DynamoDB outbox).
|
||||
# --plan-only: load IR + adapter + terraform init/validate/plan (steps
|
||||
# 1-4), then exit.
|
||||
# Modes:
|
||||
# --check-only (offline, no AWS/Checkov/DynamoDB — for CI)
|
||||
# load IR -> adapter -> stream emitted TF -> validate structure -> exit 0
|
||||
# --plan-only (requires AWS creds, no Checkov/outbox)
|
||||
# load IR -> adapter -> terraform init/validate/plan (streamed) -> exit 0
|
||||
# (default) (requires AWS creds + Checkov + DynamoDB)
|
||||
# load IR -> adapter -> terraform plan (streamed) -> Checkov (streamed) ->
|
||||
# confidence -> outbox
|
||||
#
|
||||
# Flags:
|
||||
# --quiet suppress terraform/checkov streaming (output to log only)
|
||||
# default: stream to stdout so the user sees what is happening
|
||||
#
|
||||
# NOTE: contract resolution (contract_resolver.py) was removed when the
|
||||
# thin-composition layer was taken out. The pipeline now starts from a
|
||||
@@ -18,24 +25,30 @@ set -euo pipefail
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
cd "$ROOT"
|
||||
|
||||
CHECK_ONLY=0
|
||||
PLAN_ONLY=0
|
||||
QUIET=0
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--check-only) CHECK_ONLY=1 ;;
|
||||
--plan-only) PLAN_ONLY=1 ;;
|
||||
--quiet) QUIET=1 ;;
|
||||
*) echo "FAIL: unknown argument: $arg" >&2; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
fail() { echo "FAIL: $*" >&2; exit 1; }
|
||||
|
||||
ENV_FILE="$ROOT/.env.secrets"
|
||||
[ -f "$ENV_FILE" ] || fail ".env.secrets missing (run scripts/rotate_spike_key.sh)"
|
||||
set -a
|
||||
. "$ENV_FILE"
|
||||
set +a
|
||||
export AWS_ACCESS_KEY_ID="$ACDL_AWS_ACCESS_KEY_ID"
|
||||
export AWS_SECRET_ACCESS_KEY="$ACDL_AWS_SECRET_ACCESS_KEY"
|
||||
export AWS_DEFAULT_REGION="$AWS_DEFAULT_REGION"
|
||||
# stream: pipe a command's stdout+stderr to both a log file and the
|
||||
# terminal (unless --quiet). Usage: stream <logfile> -- <command...>
|
||||
stream() {
|
||||
local log="$1"; shift
|
||||
if [ "$QUIET" = "1" ]; then
|
||||
"$@" > "$log" 2>&1
|
||||
else
|
||||
"$@" 2>&1 | tee "$log"
|
||||
fi
|
||||
}
|
||||
|
||||
CONTRACT_ID="11111111-1111-1111-1111-111111111111" # spike fixed UUID
|
||||
WORK="/tmp/spike_e2e"
|
||||
@@ -51,11 +64,67 @@ echo "=== Step 3: adapter compiles IR -> terraform/spike/*.tf (regenerate) ==="
|
||||
python3 adapters/terraform/adapter.py "$WORK/spike_ir.json" terraform/spike || fail "adapter failed"
|
||||
echo "adapter: emitted terraform/spike/{main.tf,terraform.tf,providers.tf}"
|
||||
|
||||
if [ "$QUIET" = "0" ]; then
|
||||
echo ""
|
||||
echo "--- emitted terraform/spike/main.tf ---"
|
||||
cat terraform/spike/main.tf
|
||||
echo "--- end main.tf ---"
|
||||
fi
|
||||
|
||||
if [ "$CHECK_ONLY" = "1" ]; then
|
||||
echo ""
|
||||
echo "=== Step 3b: validate adapter output structure (offline) ==="
|
||||
python3 -c "
|
||||
import json, os
|
||||
d = json.load(open('$WORK/spike_ir.json'))
|
||||
assert d['stack']['name'] == 'l1-s3'
|
||||
assert len(d['resources']) == 1
|
||||
tf_dir = 'terraform/spike'
|
||||
for f in ('main.tf', 'terraform.tf', 'providers.tf'):
|
||||
assert os.path.isfile(os.path.join(tf_dir, f)), f'{f} missing'
|
||||
main = open(os.path.join(tf_dir, 'main.tf')).read()
|
||||
assert 'aws_s3_bucket' in main
|
||||
assert 'acdl-spike-bucket' in main
|
||||
assert 'versioning' in main
|
||||
tf = open(os.path.join(tf_dir, 'terraform.tf')).read()
|
||||
assert 'backend' in tf
|
||||
assert 'required_version' in tf
|
||||
prov = open(os.path.join(tf_dir, 'providers.tf')).read()
|
||||
assert 'provider \"aws\"' in prov
|
||||
print('adapter output: OK')
|
||||
"
|
||||
echo ""
|
||||
echo "=== PLATFORM CHECK OK ==="
|
||||
echo "IR instance -> adapter -> structure validated (offline, no AWS)"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "=== Loading AWS credentials (not needed for --check-only) ==="
|
||||
ENV_FILE="$ROOT/.env.secrets"
|
||||
[ -f "$ENV_FILE" ] || fail ".env.secrets missing (run scripts/rotate_spike_key.sh)"
|
||||
set -a
|
||||
. "$ENV_FILE"
|
||||
set +a
|
||||
export AWS_ACCESS_KEY_ID="$ACDL_AWS_ACCESS_KEY_ID"
|
||||
export AWS_SECRET_ACCESS_KEY="$ACDL_AWS_SECRET_ACCESS_KEY"
|
||||
export AWS_DEFAULT_REGION="$AWS_DEFAULT_REGION"
|
||||
|
||||
echo "=== Step 4: terraform init + validate + plan -lock=false (real AWS) ==="
|
||||
cd terraform/spike
|
||||
terraform init -reconfigure -lock=false -input=false >> "$WORK/tf.log" 2>&1 || fail "terraform init failed"
|
||||
terraform validate >> "$WORK/tf.log" 2>&1 || fail "terraform validate failed"
|
||||
terraform plan -lock=false -input=false -out=tfplan >> "$WORK/tf.log" 2>&1 || fail "terraform plan failed"
|
||||
|
||||
echo ""
|
||||
echo "--- terraform init ---"
|
||||
stream "$WORK/tf-init.log" terraform init -reconfigure -lock=false -input=false || fail "terraform init failed"
|
||||
|
||||
echo ""
|
||||
echo "--- terraform validate ---"
|
||||
stream "$WORK/tf-validate.log" terraform validate || fail "terraform validate failed"
|
||||
|
||||
echo ""
|
||||
echo "--- terraform plan ---"
|
||||
stream "$WORK/tf-plan.log" terraform plan -lock=false -input=false -out=tfplan || fail "terraform plan failed"
|
||||
|
||||
echo ""
|
||||
echo "terraform plan OK (1 to add, 0 to change, 0 to destroy expected)"
|
||||
cd "$ROOT"
|
||||
|
||||
@@ -65,16 +134,35 @@ if [ "$PLAN_ONLY" = "1" ]; then
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "=== Step 5: run Checkov on terraform/spike/main.tf ==="
|
||||
checkov -f terraform/spike/main.tf --framework terraform -o json --soft-fail > "$WORK/checkov.json" 2> "$WORK/checkov.err"
|
||||
if [ "$QUIET" = "0" ]; then
|
||||
checkov -f terraform/spike/main.tf --framework terraform -o json --soft-fail 2>&1 | tee "$WORK/checkov.json"
|
||||
else
|
||||
checkov -f terraform/spike/main.tf --framework terraform -o json --soft-fail > "$WORK/checkov.json" 2> "$WORK/checkov.err"
|
||||
fi
|
||||
[ -s "$WORK/checkov.json" ] || fail "checkov produced no output"
|
||||
echo "checkov: $(python3 -c "import json; d=json.load(open('$WORK/checkov.json')); print(len(d.get('results',{}).get('failed_checks',[])), 'failed,', len(d.get('results',{}).get('passed_checks',[])), 'passed')")"
|
||||
echo ""
|
||||
echo "checkov summary: $(python3 -c "import json; d=json.load(open('$WORK/checkov.json')); print(len(d.get('results',{}).get('failed_checks',[])), 'failed,', len(d.get('results',{}).get('passed_checks',[])), 'passed')")"
|
||||
|
||||
echo "=== Step 6: Checkov adapter -> PolicyCheckResult list ==="
|
||||
echo ""
|
||||
echo "=== Step 6: Checkov adapter -> PolicyCheckResult (compliance details) ==="
|
||||
python3 adapters/terraform/policy/checkov_adapter.py "$WORK/checkov.json" "$CONTRACT_ID" > "$WORK/pcr.json" || fail "checkov adapter failed"
|
||||
PCR_COUNT=$(python3 -c "import json; print(len(json.load(open('$WORK/pcr.json'))))")
|
||||
echo "PolicyCheckResult: $PCR_COUNT record(s)"
|
||||
python3 -c "
|
||||
import json
|
||||
pcrs = json.load(open('$WORK/pcr.json'))
|
||||
print(f'PolicyCheckResult: {len(pcrs)} record(s)')
|
||||
print()
|
||||
for pcr in pcrs:
|
||||
sev = pcr.get('severity', 'info')
|
||||
res = pcr.get('result', 'unknown')
|
||||
rule = pcr.get('ruleId', 'unknown')
|
||||
msg = pcr.get('message', '')
|
||||
marker = 'PASS' if res == 'pass' else 'FAIL' if res == 'fail' else 'SKIP' if res == 'skipped' else res.upper()
|
||||
print(f' [{marker}] {sev:8s} {rule:30s} {msg}')
|
||||
"
|
||||
|
||||
echo ""
|
||||
echo "=== Step 7: confidence signal compute ==="
|
||||
python3 <<PY > "$WORK/signal.json" || fail "confidence signal failed"
|
||||
import json
|
||||
@@ -96,6 +184,7 @@ SCORE=$(python3 -c "import json; print(round(json.load(open('$WORK/signal.json')
|
||||
echo "confidence: score=$SCORE band=$BAND"
|
||||
[ "$BAND" = "pass" ] || fail "confidence band is $BAND, expected pass for dev"
|
||||
|
||||
echo ""
|
||||
echo "=== Step 8: write evidence event to DynamoDB outbox ==="
|
||||
python3 <<PY > "$WORK/event.json" || fail "event build failed"
|
||||
import json, datetime
|
||||
|
||||
@@ -8,7 +8,7 @@ terraform {
|
||||
}
|
||||
backend "s3" {
|
||||
bucket = "acdl-tfstate-581513795199-us-east-1"
|
||||
key = "spike/l2-static-asset/terraform.tfstate"
|
||||
key = "spike/l1-s3/terraform.tfstate"
|
||||
region = "us-east-1"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def repo_root():
|
||||
return str(ROOT)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def spike_ir():
|
||||
return json.load(open(ROOT / "modules-ir/l1/l1-s3/spike_instance.json"))
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def ir_schema():
|
||||
return json.load(open(ROOT / "schemas/ir.schema.json"))
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def registry():
|
||||
return json.load(open(ROOT / "modules-ir/registry.json"))
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def policy_check_result_schema():
|
||||
return json.load(open(ROOT / "schemas/policy_check_result.schema.json"))
|
||||
@@ -0,0 +1,174 @@
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import jsonschema
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from adapters.terraform.adapter import (
|
||||
TYPE_MAP, INPUT_MAP, OUTPUT_MAP, adapt, _tf_value, _ref_expr,
|
||||
)
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
class TestSpikeInstance:
|
||||
def test_spike_instance_validates_against_ir_schema(self, spike_ir, ir_schema):
|
||||
jsonschema.validate(spike_ir, ir_schema)
|
||||
|
||||
def test_spike_instance_has_one_resource(self, spike_ir):
|
||||
assert len(spike_ir["resources"]) == 1
|
||||
r = spike_ir["resources"][0]
|
||||
assert r["id"] == "s3"
|
||||
assert r["type"] == "aws:s3:bucket"
|
||||
|
||||
def test_spike_instance_stack_is_l1_s3(self, spike_ir):
|
||||
assert spike_ir["stack"]["name"] == "l1-s3"
|
||||
assert spike_ir["stack"]["kind"] == "l1"
|
||||
|
||||
|
||||
class TestRegistry:
|
||||
def test_registry_has_7_l1_entries(self, registry):
|
||||
assert len(registry) == 7
|
||||
for key in registry:
|
||||
assert key.startswith("l1-")
|
||||
|
||||
def test_registry_has_no_l2_entries(self, registry):
|
||||
l2 = [k for k in registry if k.startswith("l2")]
|
||||
assert l2 == []
|
||||
|
||||
def test_all_l1_interfaces_exist(self, registry, repo_root):
|
||||
for name, versions in registry.items():
|
||||
for ver, entry in versions.items():
|
||||
iface_path = os.path.join(repo_root, entry["interface"])
|
||||
assert os.path.isfile(iface_path), f"{iface_path} missing"
|
||||
iface = json.load(open(iface_path))
|
||||
assert iface["name"] == name
|
||||
|
||||
|
||||
class TestTypeMap:
|
||||
def test_s3_in_type_map(self):
|
||||
assert TYPE_MAP["aws:s3:bucket"] == "aws_s3_bucket"
|
||||
|
||||
def test_vpc_types_in_type_map(self):
|
||||
assert TYPE_MAP["aws:ec2:vpc"] == "aws_vpc"
|
||||
assert TYPE_MAP["aws:ec2:subnet"] == "aws_subnet"
|
||||
assert TYPE_MAP["aws:ec2:routetable"] == "aws_route_table"
|
||||
|
||||
def test_ecs_types_in_type_map(self):
|
||||
assert TYPE_MAP["aws:ecs:cluster"] == "aws_ecs_cluster"
|
||||
assert TYPE_MAP["aws:ecs:task_definition"] == "aws_ecs_task_definition"
|
||||
assert TYPE_MAP["aws:ecs:service"] == "aws_ecs_service"
|
||||
|
||||
def test_alb_types_in_type_map(self):
|
||||
assert TYPE_MAP["aws:elbv2:loadbalancer"] == "aws_lb"
|
||||
assert TYPE_MAP["aws:elbv2:listener"] == "aws_lb_listener"
|
||||
assert TYPE_MAP["aws:elbv2:targetgroup"] == "aws_lb_target_group"
|
||||
|
||||
def test_iam_and_ecr_in_type_map(self):
|
||||
assert TYPE_MAP["aws:iam:role"] == "aws_iam_role"
|
||||
assert TYPE_MAP["aws:ecr:repository"] == "aws_ecr_repository"
|
||||
|
||||
|
||||
class TestTfValue:
|
||||
def test_string_quoted(self):
|
||||
assert _tf_value("hello") == '"hello"'
|
||||
|
||||
def test_bool_true(self):
|
||||
assert _tf_value(True) == "true"
|
||||
|
||||
def test_bool_false(self):
|
||||
assert _tf_value(False) == "false"
|
||||
|
||||
def test_int(self):
|
||||
assert _tf_value(42) == "42"
|
||||
|
||||
def test_float(self):
|
||||
assert _tf_value(3.14) == "3.14"
|
||||
|
||||
def test_dict_jsonencoded(self):
|
||||
result = _tf_value({"key": "val"})
|
||||
assert "jsonencode" in result
|
||||
assert '"key"' in result
|
||||
|
||||
def test_list_jsonencoded(self):
|
||||
result = _tf_value([1, 2])
|
||||
assert "jsonencode" in result
|
||||
|
||||
def test_json_string_jsonencoded(self):
|
||||
result = _tf_value('{"k":"v"}')
|
||||
assert "jsonencode" in result
|
||||
|
||||
def test_ref_raises(self):
|
||||
with pytest.raises(ValueError, match="ref: values"):
|
||||
_tf_value("ref:s3.bucket_arn")
|
||||
|
||||
|
||||
class TestRefExpr:
|
||||
def test_basic_ref(self):
|
||||
type_by_id = {"s3": "aws:s3:bucket"}
|
||||
result = _ref_expr("ref:s3.bucket_arn", type_by_id)
|
||||
assert result == "aws_s3_bucket.s3.arn"
|
||||
|
||||
def test_vpc_ref(self):
|
||||
type_by_id = {"vpc": "aws:ec2:vpc"}
|
||||
result = _ref_expr("ref:vpc.vpc_id", type_by_id)
|
||||
assert result == "aws_vpc.vpc.id"
|
||||
|
||||
def test_unknown_id_raises(self):
|
||||
with pytest.raises(ValueError, match="unknown IR resource id"):
|
||||
_ref_expr("ref:nonexistent.output", {"s3": "aws:s3:bucket"})
|
||||
|
||||
|
||||
class TestAdapt:
|
||||
def test_adapt_emits_three_files(self, spike_ir, tmp_path):
|
||||
out_dir = str(tmp_path / "tf_out")
|
||||
adapt(spike_ir, out_dir)
|
||||
assert os.path.isfile(os.path.join(out_dir, "main.tf"))
|
||||
assert os.path.isfile(os.path.join(out_dir, "terraform.tf"))
|
||||
assert os.path.isfile(os.path.join(out_dir, "providers.tf"))
|
||||
|
||||
def test_main_tf_has_s3_bucket(self, spike_ir, tmp_path):
|
||||
out_dir = str(tmp_path / "tf_out")
|
||||
adapt(spike_ir, out_dir)
|
||||
main_tf = open(os.path.join(out_dir, "main.tf")).read()
|
||||
assert 'resource "aws_s3_bucket" "s3"' in main_tf
|
||||
assert 'bucket = "acdl-spike-bucket"' in main_tf
|
||||
|
||||
def test_main_tf_has_versioning(self, spike_ir, tmp_path):
|
||||
out_dir = str(tmp_path / "tf_out")
|
||||
adapt(spike_ir, out_dir)
|
||||
main_tf = open(os.path.join(out_dir, "main.tf")).read()
|
||||
assert "versioning" in main_tf
|
||||
assert "enabled = true" in main_tf
|
||||
|
||||
def test_main_tf_has_outputs(self, spike_ir, tmp_path):
|
||||
out_dir = str(tmp_path / "tf_out")
|
||||
adapt(spike_ir, out_dir)
|
||||
main_tf = open(os.path.join(out_dir, "main.tf")).read()
|
||||
assert 'output "bucket_arn"' in main_tf
|
||||
assert 'output "bucket_name"' in main_tf
|
||||
|
||||
def test_terraform_tf_has_backend(self, spike_ir, tmp_path):
|
||||
out_dir = str(tmp_path / "tf_out")
|
||||
adapt(spike_ir, out_dir)
|
||||
terraform_tf = open(os.path.join(out_dir, "terraform.tf")).read()
|
||||
assert 'backend "s3"' in terraform_tf
|
||||
assert 'required_version' in terraform_tf
|
||||
assert ">= 1.9" in terraform_tf
|
||||
|
||||
def test_providers_tf_has_aws(self, spike_ir, tmp_path):
|
||||
out_dir = str(tmp_path / "tf_out")
|
||||
adapt(spike_ir, out_dir)
|
||||
providers_tf = open(os.path.join(out_dir, "providers.tf")).read()
|
||||
assert 'provider "aws"' in providers_tf
|
||||
assert "us-east-1" in providers_tf
|
||||
|
||||
def test_backend_key_uses_stack_name(self, spike_ir, tmp_path):
|
||||
out_dir = str(tmp_path / "tf_out")
|
||||
adapt(spike_ir, out_dir)
|
||||
terraform_tf = open(os.path.join(out_dir, "terraform.tf")).read()
|
||||
assert "spike/l1-s3/terraform.tfstate" in terraform_tf
|
||||
@@ -0,0 +1,126 @@
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import jsonschema
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from adapters.terraform.policy.checkov_adapter import (
|
||||
RULE_MAP, _to_pcr, _emit_tag_naming_skipped, adapt,
|
||||
)
|
||||
|
||||
|
||||
class TestRuleMap:
|
||||
def test_secrets_rules(self):
|
||||
assert RULE_MAP["CKV_AWS_41"][0] == "secrets-in-plaintext"
|
||||
assert RULE_MAP["CKV_AWS_45"][0] == "secrets-in-plaintext"
|
||||
|
||||
def test_public_ingress_rules(self):
|
||||
assert RULE_MAP["CKV_AWS_20"][0] == "public-ingress"
|
||||
assert RULE_MAP["CKV_AWS_57"][0] == "public-ingress"
|
||||
|
||||
def test_iam_wildcard(self):
|
||||
assert RULE_MAP["CKV_AWS_1"][0] == "iam-wildcard"
|
||||
|
||||
def test_kms(self):
|
||||
assert RULE_MAP["CKV_AWS_7"][0] == "kms-key-reference"
|
||||
|
||||
def test_all_have_severities(self):
|
||||
for rule_id, (cat, sev) in RULE_MAP.items():
|
||||
assert sev in ("high", "medium", "low", "info"), f"{rule_id} has bad severity {sev}"
|
||||
|
||||
|
||||
class TestToPcr:
|
||||
def test_passed_result(self):
|
||||
rec = {"check_id": "CKV_AWS_20", "check_name": "No public ingress", "file_path": "main.tf"}
|
||||
pcr = _to_pcr(rec, "contract-123", "PASSED")
|
||||
assert pcr["result"] == "pass"
|
||||
assert pcr["contractId"] == "contract-123"
|
||||
assert pcr["engine"] == "checkov"
|
||||
assert pcr["ruleId"] == "CKV_AWS_20"
|
||||
assert pcr["severity"] == "high"
|
||||
|
||||
def test_failed_result(self):
|
||||
rec = {"check_id": "CKV_AWS_1", "check_name": "No wildcard IAM"}
|
||||
pcr = _to_pcr(rec, "c-1", "FAILED")
|
||||
assert pcr["result"] == "fail"
|
||||
assert pcr["severity"] == "high"
|
||||
|
||||
def test_skipped_result(self):
|
||||
rec = {"check_id": "UNKNOWN_RULE", "check_name": "some check"}
|
||||
pcr = _to_pcr(rec, "c-1", "SKIPPED")
|
||||
assert pcr["result"] == "skipped"
|
||||
assert pcr["severity"] == "info"
|
||||
|
||||
def test_unknown_rule_defaults_to_info(self):
|
||||
rec = {"check_id": "UNKNOWN_RULE", "check_name": "unknown"}
|
||||
pcr = _to_pcr(rec, "c-1", "FAILED")
|
||||
assert pcr["severity"] == "info"
|
||||
|
||||
def test_pcr_validates_against_schema(self, policy_check_result_schema):
|
||||
rec = {"check_id": "CKV_AWS_20", "check_name": "test", "file_path": "main.tf",
|
||||
"resource": "aws_s3_bucket.s3", "resource_address": "aws_s3_bucket.s3"}
|
||||
pcr = _to_pcr(rec, "c-1", "FAILED")
|
||||
jsonschema.validate(pcr, policy_check_result_schema)
|
||||
|
||||
|
||||
class TestTagNamingSkipped:
|
||||
def test_skipped_pcr(self):
|
||||
pcr = _emit_tag_naming_skipped("c-1")
|
||||
assert pcr["result"] == "skipped"
|
||||
assert pcr["ruleId"] == "ACDL_TAG_NAMING"
|
||||
assert pcr["severity"] == "info"
|
||||
|
||||
|
||||
class TestAdapt:
|
||||
def _sample_checkov_json(self):
|
||||
return {
|
||||
"terraform_plan": {
|
||||
"results": {
|
||||
"passed_checks": [
|
||||
{"check_id": "CKV_AWS_20", "check_name": "no public ingress",
|
||||
"file_path": "main.tf", "resource": "aws_vpc.vpc"}
|
||||
],
|
||||
"failed_checks": [
|
||||
{"check_id": "CKV_AWS_1", "check_name": "no wildcard iam",
|
||||
"file_path": "main.tf", "resource": "aws_iam_role.r"}
|
||||
],
|
||||
"skipped_checks": []
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def test_adapt_returns_list(self, tmp_path):
|
||||
data = self._sample_checkov_json()
|
||||
f = tmp_path / "checkov.json"
|
||||
f.write_text(json.dumps(data))
|
||||
results = adapt(str(f), "c-1")
|
||||
assert isinstance(results, list)
|
||||
|
||||
def test_adapt_includes_tag_naming(self, tmp_path):
|
||||
data = self._sample_checkov_json()
|
||||
f = tmp_path / "checkov.json"
|
||||
f.write_text(json.dumps(data))
|
||||
results = adapt(str(f), "c-1")
|
||||
tag = [r for r in results if r["ruleId"] == "ACDL_TAG_NAMING"]
|
||||
assert len(tag) == 1
|
||||
assert tag[0]["result"] == "skipped"
|
||||
|
||||
def test_adapt_has_passed_and_failed(self, tmp_path):
|
||||
data = self._sample_checkov_json()
|
||||
f = tmp_path / "checkov.json"
|
||||
f.write_text(json.dumps(data))
|
||||
results = adapt(str(f), "c-1")
|
||||
passed = [r for r in results if r["result"] == "pass"]
|
||||
failed = [r for r in results if r["result"] == "fail"]
|
||||
assert len(passed) >= 1
|
||||
assert len(failed) >= 1
|
||||
|
||||
def test_adapt_empty_input(self, tmp_path):
|
||||
data = {"terraform_plan": {"results": {"passed_checks": [], "failed_checks": [], "skipped_checks": []}}}
|
||||
f = tmp_path / "checkov.json"
|
||||
f.write_text(json.dumps(data))
|
||||
results = adapt(str(f), "c-1")
|
||||
assert len(results) == 1 # just the tag naming skipped
|
||||
@@ -0,0 +1,181 @@
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from acdl_platform.confidence_signal import (
|
||||
WEIGHTS, PENALTY, THRESHOLDS, compute, Signal, _per_input_score,
|
||||
)
|
||||
|
||||
|
||||
class TestWeights:
|
||||
def test_weights_sum_to_one(self):
|
||||
assert sum(WEIGHTS.values()) == pytest.approx(1.0)
|
||||
|
||||
def test_policy_weight_highest(self):
|
||||
assert WEIGHTS["policy"] == 0.30
|
||||
|
||||
def test_validation_weight(self):
|
||||
assert WEIGHTS["validation"] == 0.25
|
||||
|
||||
|
||||
class TestThresholds:
|
||||
def test_dev_threshold(self):
|
||||
assert THRESHOLDS["dev"] == 0.50
|
||||
|
||||
def test_qa_threshold(self):
|
||||
assert THRESHOLDS["qa"] == 0.75
|
||||
|
||||
def test_prod_threshold(self):
|
||||
assert THRESHOLDS["prod"] == 0.90
|
||||
|
||||
def test_dr_threshold(self):
|
||||
assert THRESHOLDS["dr"] == 0.95
|
||||
|
||||
|
||||
class TestPenalty:
|
||||
def test_critical_is_none(self):
|
||||
assert PENALTY["critical"] is None
|
||||
|
||||
def test_high_penalty(self):
|
||||
assert PENALTY["high"] == 0.20
|
||||
|
||||
def test_medium_penalty(self):
|
||||
assert PENALTY["medium"] == 0.05
|
||||
|
||||
def test_low_penalty(self):
|
||||
assert PENALTY["low"] == 0.01
|
||||
|
||||
def test_info_no_penalty(self):
|
||||
assert PENALTY["info"] == 0.0
|
||||
|
||||
|
||||
class TestPerInputScore:
|
||||
def test_missing_input_returns_half(self):
|
||||
score, reasons = _per_input_score("policy", None)
|
||||
assert score == 0.5
|
||||
assert "INPUT_MISSING:policy" in reasons
|
||||
|
||||
def test_empty_policy_list(self):
|
||||
score, reasons = _per_input_score("policy", [])
|
||||
assert score == 0.5
|
||||
assert reasons == []
|
||||
|
||||
def test_all_pass_policy(self):
|
||||
pcrs = [{"result": "pass"}, {"result": "pass"}]
|
||||
score, reasons = _per_input_score("policy", pcrs)
|
||||
assert score == 1.0
|
||||
assert reasons == []
|
||||
|
||||
def test_mixed_policy(self):
|
||||
pcrs = [{"result": "pass"}, {"result": "fail"}]
|
||||
score, reasons = _per_input_score("policy", pcrs)
|
||||
assert score == 0.5
|
||||
|
||||
def test_skipped_counts_as_pass(self):
|
||||
pcrs = [{"result": "skipped"}]
|
||||
score, reasons = _per_input_score("policy", pcrs)
|
||||
assert score == 1.0
|
||||
|
||||
def test_validation_all_true(self):
|
||||
score, reasons = _per_input_score("validation", {
|
||||
"schema": True, "ir_resolved": True,
|
||||
"tf_validated": True, "tf_planned": True
|
||||
})
|
||||
assert score == 1.0
|
||||
|
||||
def test_validation_partial(self):
|
||||
score, reasons = _per_input_score("validation", {
|
||||
"schema": True, "ir_resolved": True,
|
||||
"tf_validated": False, "tf_planned": False
|
||||
})
|
||||
assert score == 0.5
|
||||
|
||||
def test_freshness_fresh(self):
|
||||
score, _ = _per_input_score("freshness", {"age_days": 0, "max_age_days": 7})
|
||||
assert score == 1.0
|
||||
|
||||
def test_freshness_stale(self):
|
||||
score, _ = _per_input_score("freshness", {"age_days": 7, "max_age_days": 7})
|
||||
assert score == pytest.approx(0.0)
|
||||
|
||||
def test_source_complete(self):
|
||||
score, _ = _per_input_score("source", {"submitter": "dev", "commit_sha": "abc"})
|
||||
assert score == 1.0
|
||||
|
||||
def test_source_partial(self):
|
||||
score, _ = _per_input_score("source", {"submitter": "dev"})
|
||||
assert score == 0.5
|
||||
|
||||
def test_history_clean(self):
|
||||
score, _ = _per_input_score("history", {"prior_rollbacks": 0, "prior_policy_fails": 0})
|
||||
assert score == 1.0
|
||||
|
||||
def test_history_with_failures(self):
|
||||
score, _ = _per_input_score("history", {"prior_rollbacks": 2, "prior_policy_fails": 3})
|
||||
assert score == pytest.approx(0.3)
|
||||
|
||||
def test_nfrs_none(self):
|
||||
score, _ = _per_input_score("nfrs", {"conformance": None})
|
||||
assert score == 0.5
|
||||
|
||||
def test_nfrs_full(self):
|
||||
score, _ = _per_input_score("nfrs", {"conformance": 0.95})
|
||||
assert score == 0.95
|
||||
|
||||
|
||||
class TestCompute:
|
||||
def _base_inputs(self):
|
||||
return {
|
||||
"policy": [{"result": "pass"}],
|
||||
"validation": {"schema": True, "ir_resolved": True,
|
||||
"tf_validated": True, "tf_planned": True},
|
||||
"freshness": {"age_days": 0, "max_age_days": 7},
|
||||
"source": {"submitter": "dev", "commit_sha": "abc"},
|
||||
"history": {"prior_rollbacks": 0, "prior_policy_fails": 0},
|
||||
"nfrs": {"conformance": None},
|
||||
}
|
||||
|
||||
def test_dev_pass(self):
|
||||
sig = compute("test-001", "dev", self._base_inputs())
|
||||
assert sig.band == "pass"
|
||||
assert sig.score >= 0.50
|
||||
|
||||
def test_missing_input_blocks(self):
|
||||
inputs = self._base_inputs()
|
||||
del inputs["policy"]
|
||||
sig = compute("test-002", "dev", inputs)
|
||||
assert sig.band == "block"
|
||||
assert sig.score == 0.0
|
||||
assert any("INPUT_MISSING" in r for r in sig.reasonCodes)
|
||||
|
||||
def test_critical_policy_blocks(self):
|
||||
inputs = self._base_inputs()
|
||||
inputs["policy"] = [{"result": "fail", "severity": "critical", "ruleId": "CKV_X"}]
|
||||
sig = compute("test-003", "dev", inputs)
|
||||
assert sig.band == "block"
|
||||
assert sig.score == 0.0
|
||||
assert any("CRITICAL_OVERRIDE" in r for r in sig.reasonCodes)
|
||||
|
||||
def test_high_policy_lowers_score(self):
|
||||
inputs = self._base_inputs()
|
||||
inputs["policy"] = [{"result": "fail", "severity": "high", "ruleId": "CKV_Y"}]
|
||||
sig = compute("test-004", "dev", inputs)
|
||||
assert sig.score < 1.0
|
||||
|
||||
def test_dev_warn_becomes_block(self):
|
||||
sig = compute("test-005", "dev", self._base_inputs())
|
||||
assert sig.band != "warn"
|
||||
|
||||
def test_signal_has_per_input(self):
|
||||
sig = compute("test-006", "dev", self._base_inputs())
|
||||
assert "policy" in sig.perInput
|
||||
assert "validation" in sig.perInput
|
||||
assert "nfrs" in sig.perInput
|
||||
|
||||
def test_all_six_inputs_present(self):
|
||||
sig = compute("test-007", "dev", self._base_inputs())
|
||||
assert len(sig.perInput) == 6
|
||||
@@ -0,0 +1,135 @@
|
||||
import datetime
|
||||
import hashlib
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from acdl_platform.outbox_writer import _canonical_hash, write_event
|
||||
|
||||
|
||||
class TestCanonicalHash:
|
||||
def test_deterministic(self):
|
||||
event = {"b": 2, "a": 1}
|
||||
h1 = _canonical_hash(event)
|
||||
h2 = _canonical_hash(event)
|
||||
assert h1 == h2
|
||||
|
||||
def test_order_independent(self):
|
||||
h1 = _canonical_hash({"a": 1, "b": 2})
|
||||
h2 = _canonical_hash({"b": 2, "a": 1})
|
||||
assert h1 == h2
|
||||
|
||||
def test_is_sha256_hex(self):
|
||||
h = _canonical_hash({"key": "val"})
|
||||
assert len(h) == 64
|
||||
assert all(c in "0123456789abcdef" for c in h)
|
||||
|
||||
def test_different_events_different_hash(self):
|
||||
h1 = _canonical_hash({"a": 1})
|
||||
h2 = _canonical_hash({"a": 2})
|
||||
assert h1 != h2
|
||||
|
||||
|
||||
class TestWriteEvent:
|
||||
def _sample_event(self):
|
||||
return {
|
||||
"contractId": "test-contract-001",
|
||||
"eventType": "CONFIDENCE_COMPUTED",
|
||||
"ts": "2026-07-22T00:00:00Z",
|
||||
"environment": "dev",
|
||||
"stack": "l1-s3",
|
||||
"score": 0.85,
|
||||
"band": "pass",
|
||||
"prev_event_hash": "GENESIS",
|
||||
}
|
||||
|
||||
def test_write_event_with_mock_dynamodb(self):
|
||||
from moto import mock_aws
|
||||
|
||||
import boto3
|
||||
|
||||
with mock_aws():
|
||||
dyn = boto3.client("dynamodb", region_name="us-east-1")
|
||||
dyn.create_table(
|
||||
TableName="acdl-outbox",
|
||||
KeySchema=[
|
||||
{"AttributeName": "contractId", "KeyType": "HASH"},
|
||||
{"AttributeName": "eventType#eventTs", "KeyType": "RANGE"},
|
||||
],
|
||||
AttributeDefinitions=[
|
||||
{"AttributeName": "contractId", "AttributeType": "S"},
|
||||
{"AttributeName": "eventType#eventTs", "AttributeType": "S"},
|
||||
],
|
||||
BillingMode="PAY_PER_REQUEST",
|
||||
)
|
||||
|
||||
event = self._sample_event()
|
||||
item = write_event(event, outbox_table="acdl-outbox", region="us-east-1")
|
||||
|
||||
assert item["contractId"]["S"] == "test-contract-001"
|
||||
assert item["prev_event_hash"]["S"] == "GENESIS"
|
||||
assert "hash" in item
|
||||
assert len(item["hash"]["S"]) == 64
|
||||
assert "expire_at" in item
|
||||
|
||||
def test_write_event_hash_matches_canonical(self):
|
||||
from moto import mock_aws
|
||||
|
||||
import boto3
|
||||
|
||||
with mock_aws():
|
||||
dyn = boto3.client("dynamodb", region_name="us-east-1")
|
||||
dyn.create_table(
|
||||
TableName="acdl-outbox",
|
||||
KeySchema=[
|
||||
{"AttributeName": "contractId", "KeyType": "HASH"},
|
||||
{"AttributeName": "eventType#eventTs", "KeyType": "RANGE"},
|
||||
],
|
||||
AttributeDefinitions=[
|
||||
{"AttributeName": "contractId", "AttributeType": "S"},
|
||||
{"AttributeName": "eventType#eventTs", "AttributeType": "S"},
|
||||
],
|
||||
BillingMode="PAY_PER_REQUEST",
|
||||
)
|
||||
|
||||
event = self._sample_event()
|
||||
item = write_event(event, outbox_table="acdl-outbox", region="us-east-1")
|
||||
expected_hash = _canonical_hash(event)
|
||||
assert item["hash"]["S"] == expected_hash
|
||||
|
||||
def test_write_event_persists_to_dynamodb(self):
|
||||
from moto import mock_aws
|
||||
|
||||
import boto3
|
||||
|
||||
with mock_aws():
|
||||
dyn = boto3.client("dynamodb", region_name="us-east-1")
|
||||
dyn.create_table(
|
||||
TableName="acdl-outbox",
|
||||
KeySchema=[
|
||||
{"AttributeName": "contractId", "KeyType": "HASH"},
|
||||
{"AttributeName": "eventType#eventTs", "KeyType": "RANGE"},
|
||||
],
|
||||
AttributeDefinitions=[
|
||||
{"AttributeName": "contractId", "AttributeType": "S"},
|
||||
{"AttributeName": "eventType#eventTs", "AttributeType": "S"},
|
||||
],
|
||||
BillingMode="PAY_PER_REQUEST",
|
||||
)
|
||||
|
||||
event = self._sample_event()
|
||||
write_event(event, outbox_table="acdl-outbox", region="us-east-1")
|
||||
|
||||
resp = dyn.get_item(
|
||||
TableName="acdl-outbox",
|
||||
Key={
|
||||
"contractId": {"S": "test-contract-001"},
|
||||
"eventType#eventTs": {"S": "CONFIDENCE_COMPUTED#2026-07-22T00:00:00Z"},
|
||||
},
|
||||
)
|
||||
assert "Item" in resp
|
||||
assert resp["Item"]["band"]["S"] == "pass"
|
||||
@@ -0,0 +1,67 @@
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
class TestPipelineIntegration:
|
||||
def test_load_ir_and_adapt_offline(self, tmp_path):
|
||||
ir = json.load(open(ROOT / "modules-ir/l1/l1-s3/spike_instance.json"))
|
||||
assert ir["stack"]["name"] == "l1-s3"
|
||||
|
||||
sys.path.insert(0, str(ROOT))
|
||||
from adapters.terraform.adapter import adapt
|
||||
out_dir = str(tmp_path / "tf")
|
||||
adapt(ir, out_dir)
|
||||
|
||||
assert os.path.isfile(os.path.join(out_dir, "main.tf"))
|
||||
assert os.path.isfile(os.path.join(out_dir, "terraform.tf"))
|
||||
assert os.path.isfile(os.path.join(out_dir, "providers.tf"))
|
||||
|
||||
main_tf = open(os.path.join(out_dir, "main.tf")).read()
|
||||
assert "aws_s3_bucket" in main_tf
|
||||
assert "acdl-spike-bucket" in main_tf
|
||||
|
||||
def test_confidence_signal_with_adapted_tf(self):
|
||||
sys.path.insert(0, str(ROOT))
|
||||
from acdl_platform.confidence_signal import compute
|
||||
|
||||
inputs = {
|
||||
"policy": [{"result": "pass"}],
|
||||
"validation": {"schema": True, "ir_resolved": True,
|
||||
"tf_validated": True, "tf_planned": True},
|
||||
"freshness": {"age_days": 0, "max_age_days": 7},
|
||||
"source": {"submitter": "test", "commit_sha": "test-sha"},
|
||||
"history": {"prior_rollbacks": 0, "prior_policy_fails": 0},
|
||||
"nfrs": {"conformance": None},
|
||||
}
|
||||
sig = compute("integration-test", "dev", inputs)
|
||||
assert sig.band == "pass"
|
||||
assert sig.score >= 0.50
|
||||
|
||||
def test_run_platform_check_only(self):
|
||||
result = subprocess.run(
|
||||
["bash", str(ROOT / "scripts/run_platform.sh"), "--check-only"],
|
||||
capture_output=True, text=True, cwd=str(ROOT),
|
||||
timeout=30,
|
||||
)
|
||||
assert result.returncode == 0, f"stdout: {result.stdout}\nstderr: {result.stderr}"
|
||||
assert "PLATFORM CHECK OK" in result.stdout
|
||||
|
||||
def test_run_platform_check_only_no_aws_creds(self):
|
||||
env = os.environ.copy()
|
||||
env.pop("AWS_ACCESS_KEY_ID", None)
|
||||
env.pop("AWS_SECRET_ACCESS_KEY", None)
|
||||
env.pop("AWS_DEFAULT_REGION", None)
|
||||
result = subprocess.run(
|
||||
["bash", str(ROOT / "scripts/run_platform.sh"), "--check-only"],
|
||||
capture_output=True, text=True, cwd=str(ROOT), env=env,
|
||||
timeout=30,
|
||||
)
|
||||
assert result.returncode == 0
|
||||
assert "PLATFORM CHECK OK" in result.stdout
|
||||
@@ -0,0 +1,236 @@
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import jsonschema
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
def _load_yaml(path):
|
||||
with open(ROOT / path) as f:
|
||||
return yaml.safe_load(f)
|
||||
|
||||
|
||||
def _load_workflow(path):
|
||||
wf = _load_yaml(path)
|
||||
if True in wf:
|
||||
wf["on"] = wf[True]
|
||||
return wf
|
||||
|
||||
|
||||
class TestPipelineSchema:
|
||||
def test_schema_is_valid_json_schema(self):
|
||||
schema = json.load(open(ROOT / "schemas/pipeline.schema.json"))
|
||||
jsonschema.Draft202012Validator.check_schema(schema)
|
||||
|
||||
def test_schema_has_required_fields(self):
|
||||
schema = json.load(open(ROOT / "schemas/pipeline.schema.json"))
|
||||
assert "name" in schema["required"]
|
||||
assert "triggers" in schema["required"]
|
||||
assert "runner" in schema["required"]
|
||||
assert "stages" in schema["required"]
|
||||
|
||||
def test_schema_stage_def_has_command_and_required(self):
|
||||
schema = json.load(open(ROOT / "schemas/pipeline.schema.json"))
|
||||
stage_def = schema["$defs"]["stage"]
|
||||
assert "command" in stage_def["required"]
|
||||
assert "required" in stage_def["required"]
|
||||
|
||||
|
||||
class TestPipelineContract:
|
||||
def test_contract_validates_against_schema(self):
|
||||
schema = json.load(open(ROOT / "schemas/pipeline.schema.json"))
|
||||
contract = _load_yaml("pipelines/ci.yaml")
|
||||
jsonschema.validate(contract, schema)
|
||||
|
||||
def test_contract_has_three_stages(self):
|
||||
contract = _load_yaml("pipelines/ci.yaml")
|
||||
stage_names = [s["name"] for s in contract["stages"]]
|
||||
assert stage_names == ["lint", "test", "check-only"]
|
||||
|
||||
def test_contract_runner_is_ubuntu_latest(self):
|
||||
contract = _load_yaml("pipelines/ci.yaml")
|
||||
assert contract["runner"] == "ubuntu-latest"
|
||||
|
||||
def test_contract_python_version(self):
|
||||
contract = _load_yaml("pipelines/ci.yaml")
|
||||
assert contract["python_version"] == "3.12"
|
||||
|
||||
def test_contract_triggers_push_main(self):
|
||||
contract = _load_yaml("pipelines/ci.yaml")
|
||||
assert "main" in contract["triggers"]["push"]
|
||||
|
||||
def test_contract_triggers_pr_main(self):
|
||||
contract = _load_yaml("pipelines/ci.yaml")
|
||||
assert "main" in contract["triggers"]["pull_request"]
|
||||
|
||||
def test_contract_all_stages_required(self):
|
||||
contract = _load_yaml("pipelines/ci.yaml")
|
||||
for stage in contract["stages"]:
|
||||
assert stage["required"] is True
|
||||
|
||||
def test_contract_lint_command_compiles_python(self):
|
||||
contract = _load_yaml("pipelines/ci.yaml")
|
||||
lint = next(s for s in contract["stages"] if s["name"] == "lint")
|
||||
assert "py_compile" in lint["command"]
|
||||
assert "acdl_platform/confidence_signal.py" in lint["command"]
|
||||
assert "adapters/terraform/adapter.py" in lint["command"]
|
||||
|
||||
def test_contract_test_command_runs_pytest(self):
|
||||
contract = _load_yaml("pipelines/ci.yaml")
|
||||
test_stage = next(s for s in contract["stages"] if s["name"] == "test")
|
||||
assert "pytest" in test_stage["command"]
|
||||
|
||||
def test_contract_check_only_runs_platform(self):
|
||||
contract = _load_yaml("pipelines/ci.yaml")
|
||||
check = next(s for s in contract["stages"] if s["name"] == "check-only")
|
||||
assert "run_platform.sh" in check["command"]
|
||||
assert "--check-only" in check["command"]
|
||||
|
||||
|
||||
class TestWorkflowConformance:
|
||||
def test_gitea_workflow_exists(self):
|
||||
assert (ROOT / ".gitea/workflows/ci.yml").is_file()
|
||||
|
||||
def test_github_workflow_exists(self):
|
||||
assert (ROOT / ".github/workflows/ci.yml").is_file()
|
||||
|
||||
def test_workflows_are_byte_identical(self):
|
||||
gitea = open(ROOT / ".gitea/workflows/ci.yml", "rb").read()
|
||||
github = open(ROOT / ".github/workflows/ci.yml", "rb").read()
|
||||
assert gitea == github, "Gitea and GitHub workflows must be byte-identical"
|
||||
|
||||
def test_gitea_workflow_name_matches_contract(self):
|
||||
wf = _load_workflow(".gitea/workflows/ci.yml")
|
||||
contract = _load_yaml("pipelines/ci.yaml")
|
||||
assert wf["name"] == contract["name"]
|
||||
|
||||
def test_gitea_workflow_has_three_jobs(self):
|
||||
wf = _load_workflow(".gitea/workflows/ci.yml")
|
||||
assert set(wf["jobs"].keys()) == {"lint", "test", "check-only"}
|
||||
|
||||
def test_gitea_workflow_triggers_match_contract(self):
|
||||
wf = _load_workflow(".gitea/workflows/ci.yml")
|
||||
contract = _load_yaml("pipelines/ci.yaml")
|
||||
assert wf["on"]["push"]["branches"] == contract["triggers"]["push"]
|
||||
assert wf["on"]["pull_request"]["branches"] == contract["triggers"]["pull_request"]
|
||||
|
||||
def test_gitea_workflow_runner_matches_contract(self):
|
||||
wf = _load_workflow(".gitea/workflows/ci.yml")
|
||||
contract = _load_yaml("pipelines/ci.yaml")
|
||||
for job in wf["jobs"].values():
|
||||
assert job["runs-on"] == contract["runner"]
|
||||
|
||||
def test_gitea_workflow_python_version_matches_contract(self):
|
||||
wf = _load_workflow(".gitea/workflows/ci.yml")
|
||||
contract = _load_yaml("pipelines/ci.yaml")
|
||||
for job in wf["jobs"].values():
|
||||
setup_step = next(
|
||||
s for s in job["steps"] if "setup-python" in s.get("uses", "")
|
||||
)
|
||||
assert setup_step["with"]["python-version"] == contract["python_version"]
|
||||
|
||||
def test_gitea_lint_command_matches_contract(self):
|
||||
wf = _load_workflow(".gitea/workflows/ci.yml")
|
||||
lint_job = wf["jobs"]["lint"]
|
||||
run_step = next(s for s in lint_job["steps"] if "run" in s)
|
||||
assert "py_compile" in run_step["run"]
|
||||
for py_file in [
|
||||
"acdl_platform/confidence_signal.py",
|
||||
"acdl_platform/outbox_writer.py",
|
||||
"adapters/terraform/adapter.py",
|
||||
"adapters/terraform/policy/checkov_adapter.py",
|
||||
"scripts/push_consumer_image.py",
|
||||
]:
|
||||
assert py_file in run_step["run"], f"{py_file} missing from lint command"
|
||||
|
||||
def test_gitea_test_command_matches_contract(self):
|
||||
wf = _load_workflow(".gitea/workflows/ci.yml")
|
||||
test_job = wf["jobs"]["test"]
|
||||
run_step = next(s for s in test_job["steps"] if "run" in s and "pytest" in s["run"])
|
||||
assert "pytest" in run_step["run"]
|
||||
|
||||
def test_gitea_check_only_command_matches_contract(self):
|
||||
wf = _load_workflow(".gitea/workflows/ci.yml")
|
||||
check_job = wf["jobs"]["check-only"]
|
||||
run_step = next(
|
||||
s for s in check_job["steps"] if "run" in s and "run_platform" in s["run"]
|
||||
)
|
||||
assert "run_platform.sh" in run_step["run"]
|
||||
assert "--check-only" in run_step["run"]
|
||||
|
||||
|
||||
class TestRunCiScript:
|
||||
def test_run_ci_script_exists_and_executable(self):
|
||||
path = ROOT / "scripts/run_ci.sh"
|
||||
assert path.is_file()
|
||||
assert os.access(path, os.X_OK)
|
||||
|
||||
def test_run_ci_script_contains_lint_stage(self):
|
||||
content = open(ROOT / "scripts/run_ci.sh").read()
|
||||
assert "py_compile" in content
|
||||
assert "acdl_platform/confidence_signal.py" in content
|
||||
assert "adapters/terraform/adapter.py" in content
|
||||
|
||||
def test_run_ci_script_contains_test_stage(self):
|
||||
content = open(ROOT / "scripts/run_ci.sh").read()
|
||||
assert "pytest" in content
|
||||
assert "tests/" in content
|
||||
|
||||
def test_run_ci_script_contains_check_only_stage(self):
|
||||
content = open(ROOT / "scripts/run_ci.sh").read()
|
||||
assert "run_platform.sh" in content
|
||||
assert "--check-only" in content
|
||||
|
||||
def test_run_ci_script_has_success_message(self):
|
||||
content = open(ROOT / "scripts/run_ci.sh").read()
|
||||
assert "CI PIPELINE OK" in content
|
||||
|
||||
def test_run_ci_lint_and_check_only_pass(self):
|
||||
result = subprocess.run(
|
||||
["bash", "-c",
|
||||
f"cd {ROOT} && "
|
||||
"python3 -m py_compile "
|
||||
"acdl_platform/confidence_signal.py "
|
||||
"acdl_platform/outbox_writer.py "
|
||||
"adapters/terraform/adapter.py "
|
||||
"adapters/terraform/policy/checkov_adapter.py "
|
||||
"scripts/push_consumer_image.py && "
|
||||
"echo 'lint: OK' && "
|
||||
"bash scripts/run_platform.sh --check-only && "
|
||||
"echo 'check-only: OK'"],
|
||||
capture_output=True, text=True, cwd=str(ROOT),
|
||||
timeout=30,
|
||||
)
|
||||
assert result.returncode == 0, f"stdout: {result.stdout}\nstderr: {result.stderr}"
|
||||
assert "lint: OK" in result.stdout
|
||||
assert "check-only: OK" in result.stdout
|
||||
assert "PLATFORM CHECK OK" in result.stdout
|
||||
|
||||
|
||||
class TestRunPlatformStreaming:
|
||||
def test_check_only_streams_emitted_terraform(self):
|
||||
result = subprocess.run(
|
||||
["bash", str(ROOT / "scripts/run_platform.sh"), "--check-only"],
|
||||
capture_output=True, text=True, cwd=str(ROOT),
|
||||
timeout=30,
|
||||
)
|
||||
assert result.returncode == 0
|
||||
assert "PLATFORM CHECK OK" in result.stdout
|
||||
assert "--- emitted terraform/spike/main.tf ---" in result.stdout
|
||||
assert "aws_s3_bucket" in result.stdout
|
||||
|
||||
def test_check_only_quiet_suppresses_terraform(self):
|
||||
result = subprocess.run(
|
||||
["bash", str(ROOT / "scripts/run_platform.sh"), "--check-only", "--quiet"],
|
||||
capture_output=True, text=True, cwd=str(ROOT),
|
||||
timeout=30,
|
||||
)
|
||||
assert result.returncode == 0
|
||||
assert "PLATFORM CHECK OK" in result.stdout
|
||||
assert "--- emitted terraform/spike/main.tf ---" not in result.stdout
|
||||
Reference in New Issue
Block a user