diff --git a/.ciagent/ARCHITECTURE.md b/.ciagent/ARCHITECTURE.md index c475734..d6696f4 100644 --- a/.ciagent/ARCHITECTURE.md +++ b/.ciagent/ARCHITECTURE.md @@ -143,4 +143,112 @@ Each L1 module lives at `modules/l1//` with exactly two files: | `l1-cloudwatch` | Observability primitive | L1 modules are single-purpose, substrate-agnostic, max-depth-1 (per -PROJECT.md Constraints). They do not compose with other L1s. \ No newline at end of file +PROJECT.md Constraints). They do not compose with other L1s. + +## L2 module schema + core scripts (Phase 03 research) + +### L2 manifest.yaml schema (D-020) + +```yaml +name: l2-commodity-price-feed # matches the folder name +kind: l2 # literal "l2" +description: +l1s: # ordered list of L1 references + - name: l1-eks-fargate # MUST match an existing L1 folder name + inputs: + cluster_name: price-feed-cluster + region: us-east-1 + cpu_arch: arm64 + - name: l1-lambda + inputs: + function_name: price-ingest + runtime: python3.11 + handler: index.handler + # ... up to 5 L1 references per L2 (max-depth-5 per REQ-05; L2->L1 is depth 1) +``` + +L2s reference L1s **by name only** (no path); `mock_executor.sh` resolves +the name to `modules/l1//`. + +### L2 list (fixed per REQ-04) + +| Folder | Description | L1s (per S&P Global Energy / Platts use cases) | +|--------|-------------|------------------------------------------------| +| `l2-invoice-service` | Billing + invoicing microservice | `l1-eks-fargate`, `l1-iam-role`, `l1-lambda`, `l1-sqs`, `l1-s3` | +| `l2-commodity-price-feed` | Real-time price ingestion | `l1-eks-fargate`, `l1-lambda`, `l1-api-gateway`, `l1-eventbridge`, `l1-s3` | +| `l2-energy-analytics-api` | Historical query API | `l1-eks-fargate`, `l1-api-gateway`, `l1-lambda`, `l1-s3`, `l1-cloudwatch` | +| `l2-regulatory-reporting` | Compliance + reporting | `l1-eks-fargate`, `l1-iam-role`, `l1-lambda`, `l1-sqs`, `l1-s3` | + +Each L2 references exactly 5 L1s (within the max-depth-5 constraint; L2→L1 +is depth 1, so depth-5 is generous but the spec caps composition depth at +5 — the count is 5 to demonstrate a realistic composed stack). + +### contract.yaml schema (D-021) + +```yaml +stack: l2-commodity-price-feed # MUST match an existing L2 folder name +inputs: # top-level params for the L2 (optional) + environment: dev + owner: platform-team +public-ingress: false # bool; true triggers POLICY_VIOLATION:PUBLIC_INGRESS +``` + +The `public-ingress` key is the only policy-enforced field in Phase 03. +Phase 04's pipeline reads `contract.yaml`, runs `policy_checker.py`, then +`mock_executor.sh` to apply the L2. + +### state.json shape (D-022) + +`mock_executor.sh` writes `state.json` to its working directory: + +```json +{ + "l2": "l2-commodity-price-feed", + "l1s": [ + {"name": "l1-eks-fargate", "applied": true, "exit_code": 0}, + {"name": "l1-lambda", "applied": true, "exit_code": 0}, + ... + ], + "contract": { + "stack": "l2-commodity-price-feed", + "inputs": {...}, + "public-ingress": false + } +} +``` + +### audit.json event + hash chain (D-023) + +`audit.json` is a JSON array of event objects. `evidence_writer.py` +appends one event per call. Hash chain: + +1. Construct the event dict with `hash` set to empty string. +2. Serialize via `json.dumps(event, sort_keys=True, separators=(",", ":"))` — canonical JSON (deterministic key order, no whitespace). +3. Compute `hash = sha256(canonical_json.encode("utf-8")).hexdigest()`. +4. Set `event["hash"] = hash`. +5. Append to `audit.json`. + +Genesis event (when `audit.json` is empty or missing): + +```json +{ + "seq": 0, + "ts": "2026-07-21T13:00:00Z", + "stage": "genesis", + "event": "audit log initialized", + "prev_hash": "GENESIS", + "hash": "" +} +``` + +Subsequent events: `seq = prev.seq + 1`, `prev_hash = prev.hash`. + +### Core script I/O contracts + +| Script | Input | Output | Exit | +|--------|-------|--------|------| +| `mock_executor.sh` | `` (argv[1]); reads L2 manifest from `modules/l2//manifest.yaml` | writes `state.json` to cwd; prints per-L1 progress | 0 on all-L1s-pass; non-zero on any L1 failure | +| `policy_checker.py` | `` (argv[1]) | stdout: `POLICY_PASS` or `POLICY_VIOLATION:PUBLIC_INGRESS` | 0 on pass; 1 on violation | +| `confidence_signal.py` | `` (argv[1]); calls policy_checker | stdout: `{"score": 0.90|0.40, "reason": "..."}` | 0 always (per D-024; pipeline decides gate) | +| `evidence_writer.py` | argv: `--stage ` `--event ""` `--audit ` (default `./audit.json`) | appends event to audit.json; prints the new event's hash + seq | 0 on success; 1 on I/O error | +| `l3b_agent_stub.py` | argv[1] = issue body text (or stdin if no argv); optional `-o ` (default stdout) | writes a `contract.yaml` (D-021 schema) with `stack` set by the D-008 keyword map | 0 on success; 1 on empty input | \ No newline at end of file diff --git a/.ciagent/PLAN.md b/.ciagent/PLAN.md index e8482b3..f8d91ce 100644 --- a/.ciagent/PLAN.md +++ b/.ciagent/PLAN.md @@ -1,85 +1,102 @@ --- -phase: 02 -name: l1-modules +phase: 03 +name: l2-modules-and-core-scripts milestone: v1.0 milestone_type: feature status: planned -requirements: [REQ-02, REQ-03] +requirements: [REQ-04, REQ-05, REQ-06, REQ-07, REQ-08, REQ-11] must_haves: - - "All 8 L1 module folders exist under modules/l1/ with the exact names from REQ-02" - - "Each L1 has a manifest.yaml matching the schema in ARCHITECTURE.md (name, kind: l1, description, inputs: map of string keys)" - - "Each L1 has a mock_apply.sh that echoes '[L1: ] applying...', sleeps 1s, echoes '[L1: ] OK', exits 0 (D-007)" - - "All mock_apply.sh are executable (chmod +x) and bash -n clean" - - "All manifest.yaml files parse as valid YAML" - - "scripts/verify_phase02.sh passes: enumerates 8 L1s, validates each manifest, runs each mock_apply.sh, confirms exit 0 + expected output" + - "4 L2 module folders exist under modules/l2/ with exact names: l2-invoice-service, l2-commodity-price-feed, l2-energy-analytics-api, l2-regulatory-reporting" + - "Each L2 has a manifest.yaml matching the D-020 schema (name, kind: l2, description, l1s: list of {name, inputs: map})" + - "Each L2 references 5 L1s by name; all referenced L1 names exist in modules/l1/" + - "mock_executor.sh reads a contract.yaml, resolves the L2, invokes each L1 mock_apply.sh, writes state.json per D-022" + - "policy_checker.py exits 1 with 'POLICY_VIOLATION:PUBLIC_INGRESS' on public-ingress:true; exits 0 with 'POLICY_PASS' otherwise (D-025)" + - "confidence_signal.py prints {score, reason} JSON; score 0.90 on pass, 0.40 on policy fail (D-024)" + - "evidence_writer.py appends an event to audit.json with a valid canonical-JSON SHA-256 hash chain (D-023)" + - "l3b_agent_stub.py maps the Act 3 example issue text to l2-commodity-price-feed and emits a valid contract.yaml (D-026/D-008)" + - "scripts/verify_phase03.sh passes: validates all 4 L2s, runs each core script with a sample contract, confirms hash chain integrity" verification: - typecheck: "bash -n modules/l1/*/mock_apply.sh scripts/*.sh && python3 -c 'import yaml; [yaml.safe_load(open(f)) for f in glob.glob(\"modules/l1/*/manifest.yaml\")]'" - test: "scripts/verify_phase02.sh" + typecheck: "bash -n scripts/*.sh && python3 -m py_compile scripts/*.py && python3 -c 'import yaml, glob; [yaml.safe_load(open(f)) for f in glob.glob(\"modules/l2/*/manifest.yaml\")]'" + test: "scripts/verify_phase03.sh" build: no-op --- -# Phase 02 — l1-modules PLAN +# Phase 03 — l2-modules-and-core-scripts PLAN ## Goal -Create the 8 L1 stub modules under `modules/l1/`. Each module has a -`manifest.yaml` (declared inputs, flat string map per D-017) and a uniform -`mock_apply.sh` (echo + 1s sleep + exit 0 per D-007/D-018). After this phase, -Phase 03 can compose L1s into L2 modules and `mock_executor.sh` can iterate -over an L2's L1 references. +Create the 4 L2 composition modules and the 5 core scripts. After this +phase, the demo has every primitive needed for Phase 04 to wire the +pipeline and Phase 05 to render the evidence UI. ## Requirements covered -- REQ-02: 8 L1 module folders exist (exact names) -- REQ-03: each L1 has manifest.yaml + mock_apply.sh with the uniform behavior +- REQ-04: 4 L2 modules under modules/l2/ composing L1s +- REQ-05: L2s compose L1s, max depth 5 +- REQ-06: mock_executor.sh reads L2 + invokes L1s + writes state.json +- REQ-07: policy_checker.py fails on public-ingress:true +- REQ-08: confidence_signal.py 0.90/0.40 + gate ≥ 0.50 +- REQ-11: evidence_writer.py SHA-256 hash chain -## Waves (vertical slices) +## Waves (vertical slices, domain priority order) -### Wave 1 — infra-stub-engineer (creates the 8 L1s) +### Wave 1 — infra-stub-engineer (4 L2 manifests) **Tasks:** -- **T-2.1** Create `modules/l1/l1-eks-fargate/{manifest.yaml, mock_apply.sh}` -- **T-2.2** Create `modules/l1/l1-iam-role/{manifest.yaml, mock_apply.sh}` -- **T-2.3** Create `modules/l1/l1-lambda/{manifest.yaml, mock_apply.sh}` -- **T-2.4** Create `modules/l1/l1-api-gateway/{manifest.yaml, mock_apply.sh}` -- **T-2.5** Create `modules/l1/l1-eventbridge/{manifest.yaml, mock_apply.sh}` -- **T-2.6** Create `modules/l1/l1-sqs/{manifest.yaml, mock_apply.sh}` -- **T-2.7** Create `modules/l1/l1-s3/{manifest.yaml, mock_apply.sh}` -- **T-2.8** Create `modules/l1/l1-cloudwatch/{manifest.yaml, mock_apply.sh}` +- **T-3.1** Create `modules/l2/l2-invoice-service/manifest.yaml` (L1s: l1-eks-fargate, l1-iam-role, l1-lambda, l1-sqs, l1-s3) +- **T-3.2** Create `modules/l2/l2-commodity-price-feed/manifest.yaml` (L1s: l1-eks-fargate, l1-lambda, l1-api-gateway, l1-eventbridge, l1-s3) +- **T-3.3** Create `modules/l2/l2-energy-analytics-api/manifest.yaml` (L1s: l1-eks-fargate, l1-api-gateway, l1-lambda, l1-s3, l1-cloudwatch) +- **T-3.4** Create `modules/l2/l2-regulatory-reporting/manifest.yaml` (L1s: l1-eks-fargate, l1-iam-role, l1-lambda, l1-sqs, l1-s3) -Each L1's `manifest.yaml` declares 1-3 plausible inputs for that primitive -(e.g., `l1-s3` declares `bucket_name`, `region`, `retention_days`; `l1-iam-role` -declares `role_name`, `trust_policy`). Each `mock_apply.sh` follows the exact -uniform template from ARCHITECTURE.md. +Each manifest declares plausible `inputs` per L1 (string map per D-017). +Remove `modules/l2/.gitkeep` in T-3.1. -**Files owned (territory):** `modules/l1/**` +**Files owned:** `modules/l2/**` -**Commits:** one per task, `---ci---` block has `phase: 2, status: plan-as-execute, persona: infra-stub-engineer, task: T-2.x, requirements.covered: [REQ-02, REQ-03]`. +**Commits:** one per task, `phase: 3, status: plan-as-execute, persona: infra-stub-engineer, task: T-3.x, requirements.covered: [REQ-04, REQ-05]`. -### Wave 2 — lead-developer (verification script + traceability) +### Wave 2 — backend-engineer (5 core scripts) **Tasks:** -- **T-2.9** Create `scripts/verify_phase02.sh`. It: - 1. Enumerates `modules/l1/*/` and confirms exactly 8 folders with the 8 expected names. - 2. For each L1: confirms `manifest.yaml` exists and parses as YAML with `name` matching the folder, `kind: l1`, and an `inputs:` map. - 3. For each L1: confirms `mock_apply.sh` is executable, `bash -n` clean, runs in <2s, exits 0, and its stdout contains the `[L1: ] applying...` and `[L1: ] OK` markers. - 4. Prints a PASS/FAIL summary; exits 0 on full success. -- **T-2.10** Update `.ciagent/REQUIREMENTS.md` (REQ-02/03 → covered pending VERIFY) and `.ciagent/ROADMAP.md` (Phase 02 → executing). No README change. +- **T-3.5** Create `scripts/policy_checker.py` — reads contract.yaml (argv[1]); if `public-ingress: true`, print `POLICY_VIOLATION:PUBLIC_INGRESS` and exit 1; else print `POLICY_PASS` and exit 0. Use only stdlib (yaml is available). Idempotent, no side effects. +- **T-3.6** Create `scripts/confidence_signal.py` — reads contract.yaml (argv[1]); calls policy_checker as a subprocess; if pass → `{"score": 0.90, "reason": "POLICY_PASS"}`, if fail → `{"score": 0.40, "reason": "POLICY_VIOLATION:PUBLIC_INGRESS"}`. Print JSON to stdout. Exit 0 always. +- **T-3.7** Create `scripts/evidence_writer.py` — argv flags `--stage`, `--event`, `--audit ` (default `./audit.json`). Loads audit.json (or empty list), computes the new event with canonical-JSON SHA-256 hash chain per D-023, appends, writes back atomically (write tmp + rename). Prints `{"seq": N, "hash": "..."}` to stdout. Genesis event automatically inserted if the file is empty/missing. +- **T-3.8** Create `scripts/mock_executor.sh` — argv[1] = contract.yaml path. Reads contract.stack, resolves `modules/l2//manifest.yaml`, iterates `l1s`, invokes `modules/l1//mock_apply.sh` for each, captures exit code, writes `state.json` per D-022. Exit 0 if all L1s exit 0; non-zero otherwise. +- **T-3.9** Create `scripts/l3b_agent_stub.py` — argv[1] = issue body (or read stdin if absent); optional `-o ` (default stdout). Applies the D-008 keyword map; writes a contract.yaml (D-021 schema) with `stack` set to the mapped L2 name and a fixed `inputs:` map per stack. Exit 0 on success, 1 on empty input. -**Files owned (territory):** `scripts/verify_phase02.sh`, `.ciagent/REQUIREMENTS.md`, `.ciagent/ROADMAP.md` +**Files owned:** `scripts/policy_checker.py`, `scripts/confidence_signal.py`, `scripts/evidence_writer.py`, `scripts/mock_executor.sh`, `scripts/l3b_agent_stub.py` -**Commits:** one per task, `---ci---` block has `phase: 2, status: plan-as-execute, persona: lead-developer, task: T-2.9/2.10`. +**Commits:** one per task, `phase: 3, status: plan-as-execute, persona: backend-engineer, task: T-3.x, requirements.covered: [REQ-06/07/08/11/12]`. + +### Wave 3 — lead-developer (verify script + traceability) + +**Tasks:** + +- **T-3.10** Create `scripts/verify_phase03.sh`. Checks: + 1. Exactly 4 L2 folders with the expected names. + 2. Each L2 manifest.yaml parses, name matches folder, kind=l2, l1s is a list of 5 entries, all referenced L1 names exist in modules/l1/. + 3. policy_checker.py on a passing contract → exit 0 + `POLICY_PASS`; on `public-ingress: true` contract → exit 1 + `POLICY_VIOLATION:PUBLIC_INGRESS`. + 4. confidence_signal.py on passing contract → `{"score": 0.90, ...}`; on failing contract → `{"score": 0.40, ...}`. Both exit 0. + 5. mock_executor.sh on a sample contract → writes state.json with l2 + l1s (all applied=true, exit_code=0) + contract fields. + 6. evidence_writer.py: append 3 events to a temp audit.json; verify seq increments 0/1/2, prev_hash chain links, each hash matches a recompute. + 7. l3b_agent_stub.py on the Act 3 example issue text ("We need to ingest natural gas prices from Platts...") → emits a contract.yaml with `stack: l2-commodity-price-feed`. +- **T-3.11** Update `.ciagent/REQUIREMENTS.md` (REQ-04/05/06/07/08/11 → covered pending VERIFY) and `.ciagent/ROADMAP.md` (Phase 03 → executing). + +**Files owned:** `scripts/verify_phase03.sh`, `.ciagent/REQUIREMENTS.md`, `.ciagent/ROADMAP.md` + +**Commits:** one per task, `phase: 3, status: plan-as-execute, persona: lead-developer, task: T-3.10/3.11`. ## Wave ordering -- Wave 1 (infra-stub-engineer) creates all 8 L1s. A single subagent gets all 8 tasks; it commits per task. -- Wave 2 (lead-developer) adds the verify script and traceability after the L1s exist. +- Wave 1 (infra-stub-engineer) creates the 4 L2 manifests first so mock_executor.sh has something to resolve. +- Wave 2 (backend-engineer) builds the 5 core scripts. policy_checker + confidence_signal have no L2 dependency; mock_executor depends on Wave 1; l3b_agent_stub is independent. +- Wave 3 (lead-developer) wires the verify script after both Waves 1 and 2 are complete. -`backend-engineer`, `data-engineer`, `frontend-engineer` have 0 tasks this phase. +`data-engineer` and `frontend-engineer` have 0 tasks this phase. ## Dependencies -- Depends on Phase 01 (the `modules/l1/.gitkeep` from T-1.1 is replaced by real folders). -- Phase 03 depends on this phase for L1 references in L2 compositions. \ No newline at end of file +- Depends on Phase 02 (L1 modules exist so mock_executor can invoke them and verify_phase03 can confirm L2 references resolve). +- Phase 04 depends on this phase for the pipeline to call policy_checker, mock_executor, confidence_signal, evidence_writer, and for the issue workflow to call l3b_agent_stub. \ No newline at end of file diff --git a/.ciagent/PROJECT.md b/.ciagent/PROJECT.md index 17057a4..693f59a 100644 --- a/.ciagent/PROJECT.md +++ b/.ciagent/PROJECT.md @@ -78,4 +78,11 @@ Build a runnable demo (Linux + GitHub/Gitea Actions) that walks executives throu | D-016 | Pages placeholder for Phase 01 is a minimal HTML stub (`ACDL Evidence` + "evidence stream coming soon"); full UI deferred to Phase 05 | Phase 01 success criterion is "Pages returns 200 with placeholder index.html" but Gitea has no Pages | Raw-URL HTTP 200 against `index.html` substitutes for the Pages check; full timeline UI built in Phase 05 | | D-017 | Each L1 `manifest.yaml` declares a single `inputs:` map of named string keys with descriptions; no nested types (substrate-agnostic, max-depth-1) | REQ-02/03 say "declared inputs"; spec forbids composition and cloud-specific types | Uniform, parseable schema that Phase 03's `mock_executor.sh` can read with python+yaml | | D-018 | L1 `mock_apply.sh` reads its own `manifest.yaml` for self-identification but ignores the input values (uniform stub per D-007) | D-007 mandates a literal echo + 1s sleep + exit 0; inputs are declared for traceability, not consumed | Predictable evidence events + clean separation from Phase 03 where L2s pass inputs to L1s | -| D-019 | The 8 L1 names are fixed per REQ-02: `l1-eks-fargate`, `l1-iam-role`, `l1-lambda`, `l1-api-gateway`, `l1-eventbridge`, `l1-sqs`, `l1-s3`, `l1-cloudwatch` | REQ-02 literal | Phase 02 enumerates them exactly; no naming freedom | \ No newline at end of file +| D-019 | The 8 L1 names are fixed per REQ-02: `l1-eks-fargate`, `l1-iam-role`, `l1-lambda`, `l1-api-gateway`, `l1-eventbridge`, `l1-sqs`, `l1-s3`, `l1-cloudwatch` | REQ-02 literal | Phase 02 enumerates them exactly; no naming freedom | +| D-020 | L2 `manifest.yaml` schema: `name`, `kind: l2`, `description`, `l1s:` (list of `{name, inputs: map}` entries). L2 references L1s by name (no path); inputs are string maps per L1 manifest declarations | REQ-04 says L2 "composes L1s"; REQ-05 caps depth at 5 (L2→L1 is depth 1) | mock_executor.sh reads `l1s:` and invokes each L1's `mock_apply.sh` | +| D-021 | `contract.yaml` schema: `stack` (L2 name), `inputs` (string map for the L2's top-level params), optional `public-ingress: bool` (the policy violation key per REQ-07) | REQ-07 cites `public-ingress: true` as the forbidden key; REQ-08's confidence signal keys off policy pass/fail | Single flat schema drives both policy_checker and the mock_executor | +| D-022 | `mock_executor.sh` writes `state.json` with shape `{"l2": "", "l1s": [{"name":"...","applied":true,"exit_code":0}], "contract": }` to the cwd; idempotent (overwrites) | REQ-06 says "writes state.json" but does not specify shape | Deterministic, parseable; Phase 05's evidence UI can include it in the audit trail | +| D-023 | `evidence_writer.py` appends events to `audit.json` (a JSON array of event objects). Each event: `{"seq": N, "ts": , "stage": "dev|qa|prod|finalize", "event": "", "prev_hash": "", "hash": ""}`. The genesis event has `prev_hash: "GENESIS"` and `seq: 0` | D-005 mandates hash-chained ledger; canonical JSON for deterministic hashing | Visible tamper-evidence without overengineering; Phase 05 UI reads the array | +| D-024 | `confidence_signal.py` reads `contract.yaml`, calls `policy_checker.py` (as a subprocess or import), returns base 0.90 on pass and 0.40 with reason code on policy failure; prints `{"score": 0.90|0.40, "reason": ""}` to stdout; exit 0 always | REQ-08 literal: base 0.90, drops to 0.40, gate ≥ 0.50 | Deterministic JSON output for the pipeline to consume | +| D-025 | `policy_checker.py` reads `contract.yaml`, fails with exit code 1 and stdout `POLICY_VIOLATION:PUBLIC_INGRESS` if `public-ingress: true`; otherwise exits 0 with stdout `POLICY_PASS` | REQ-07 literal | Single source of policy truth; called by confidence_signal and the pipeline directly | +| D-026 | `l3b_agent_stub.py` reads Issue body text from argv[1] (or stdin if no argv), applies the D-008 keyword map, writes a `contract.yaml` to stdout (or to `-o `). Output contract uses the D-021 schema with `stack:` set to the mapped L2 name and a fixed `inputs:` map per L2 | D-008 + Act 3 example; L3B must produce the same contract format as L3A | Deterministic keyword parser; no external APIs | \ No newline at end of file diff --git a/.ciagent/REQUIREMENTS.md b/.ciagent/REQUIREMENTS.md index 0334d1f..c2caa6e 100644 --- a/.ciagent/REQUIREMENTS.md +++ b/.ciagent/REQUIREMENTS.md @@ -60,15 +60,15 @@ | REQ-01 | 1 | complete (v1.0.1) | | REQ-02 | 2 | complete (v1.0.2) | | REQ-03 | 2 | complete (v1.0.2) | -| REQ-04 | 3 | pending | -| REQ-05 | 3 | pending | -| REQ-06 | 3 | pending | -| REQ-07 | 3 | pending | -| REQ-08 | 3 | pending | +| REQ-04 | 3 | covered (pending VERIFY) | +| REQ-05 | 3 | covered (pending VERIFY) | +| REQ-06 | 3 | covered (pending VERIFY) | +| REQ-07 | 3 | covered (pending VERIFY) | +| REQ-08 | 3 | covered (pending VERIFY) | | REQ-09 | 1 | complete (v1.0.1) | | REQ-10 | 4 | partial (skeleton in Phase 01 v1.0.1; full impl in Phase 04) | -| REQ-11 | 3 | pending | -| REQ-12 | 4 | partial (skeleton in Phase 01 v1.0.1; full impl in Phase 04) | +| REQ-11 | 3 | covered (pending VERIFY) | +| REQ-12 | 4 | partial (skeleton in Phase 01 v1.0.1; l3b_agent_stub in Phase 03; full trigger wiring in Phase 04) | | REQ-13 | 5 | pending | | REQ-14 | 5 | pending | | REQ-15 | 5 | pending | \ No newline at end of file diff --git a/.ciagent/ROADMAP.md b/.ciagent/ROADMAP.md index 9c20e25..01f031e 100644 --- a/.ciagent/ROADMAP.md +++ b/.ciagent/ROADMAP.md @@ -27,7 +27,7 @@ Five-phase breakdown to take ACDL from empty repo to a reproducible 4-act execut ### Phase 03 — l2-modules-and-core-scripts - **Description:** Create the 4 L2 compositions under `acdl/modules/l2/` referencing L1s, plus the 5 core scripts in `acdl/scripts/` (`mock_executor.sh`, `policy_checker.py`, `confidence_signal.py`, `evidence_writer.py`, `l3b_agent_stub.py`). -- **Status:** not_started +- **Status:** executing - **Depends on:** [2] - **Requirements:** REQ-04, REQ-05, REQ-06, REQ-07 - **Success Criteria:** diff --git a/modules/l2/.gitkeep b/modules/l2/.gitkeep deleted file mode 100644 index e69de29..0000000 diff --git a/modules/l2/l2-commodity-price-feed/manifest.yaml b/modules/l2/l2-commodity-price-feed/manifest.yaml new file mode 100644 index 0000000..38f9cb8 --- /dev/null +++ b/modules/l2/l2-commodity-price-feed/manifest.yaml @@ -0,0 +1,27 @@ +name: l2-commodity-price-feed +kind: l2 +description: Real-time commodity price ingestion from Platts +l1s: + - name: l1-eks-fargate + inputs: + cluster_name: price-feed-cluster + region: us-east-1 + cpu_arch: arm64 + - name: l1-lambda + inputs: + function_name: price-ingest + runtime: python3.11 + handler: index.handler + - name: l1-api-gateway + inputs: + api_name: platts-price-api + stage_name: dev + - name: l1-eventbridge + inputs: + bus_name: price-events + rule_name: price-publish-rule + - name: l1-s3 + inputs: + bucket_name: acdl-price-archive + region: us-east-1 + retention_days: "90" \ No newline at end of file diff --git a/modules/l2/l2-energy-analytics-api/manifest.yaml b/modules/l2/l2-energy-analytics-api/manifest.yaml new file mode 100644 index 0000000..a456faa --- /dev/null +++ b/modules/l2/l2-energy-analytics-api/manifest.yaml @@ -0,0 +1,27 @@ +name: l2-energy-analytics-api +kind: l2 +description: Historical energy analytics query API +l1s: + - name: l1-eks-fargate + inputs: + cluster_name: analytics-cluster + region: us-east-1 + cpu_arch: arm64 + - name: l1-api-gateway + inputs: + api_name: energy-analytics-api + stage_name: dev + - name: l1-lambda + inputs: + function_name: analytics-query + runtime: python3.11 + handler: index.handler + - name: l1-s3 + inputs: + bucket_name: acdl-analytics-data + region: us-east-1 + retention_days: "2555" + - name: l1-cloudwatch + inputs: + log_group_name: /acdl/analytics-api + metric_namespace: acdl/analytics \ No newline at end of file diff --git a/modules/l2/l2-invoice-service/manifest.yaml b/modules/l2/l2-invoice-service/manifest.yaml new file mode 100644 index 0000000..e01f63f --- /dev/null +++ b/modules/l2/l2-invoice-service/manifest.yaml @@ -0,0 +1,27 @@ +name: l2-invoice-service +kind: l2 +description: Billing and invoicing microservice for energy trades +l1s: + - name: l1-eks-fargate + inputs: + cluster_name: invoice-cluster + region: us-east-1 + cpu_arch: arm64 + - name: l1-iam-role + inputs: + role_name: invoice-service-role + trust_policy: '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"eks.amazonaws.com"},"Action":"sts:AssumeRole"}]}' + - name: l1-lambda + inputs: + function_name: invoice-generator + runtime: python3.11 + handler: index.handler + - name: l1-sqs + inputs: + queue_name: invoice-queue + visibility_timeout: "60" + - name: l1-s3 + inputs: + bucket_name: acdl-invoice-archive + region: us-east-1 + retention_days: "365" \ No newline at end of file diff --git a/modules/l2/l2-regulatory-reporting/manifest.yaml b/modules/l2/l2-regulatory-reporting/manifest.yaml new file mode 100644 index 0000000..b64ab58 --- /dev/null +++ b/modules/l2/l2-regulatory-reporting/manifest.yaml @@ -0,0 +1,27 @@ +name: l2-regulatory-reporting +kind: l2 +description: Regulatory compliance and reporting for energy trading +l1s: + - name: l1-eks-fargate + inputs: + cluster_name: regulatory-cluster + region: us-east-1 + cpu_arch: arm64 + - name: l1-iam-role + inputs: + role_name: regulatory-reporting-role + trust_policy: '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"eks.amazonaws.com"},"Action":"sts:AssumeRole"}]}' + - name: l1-lambda + inputs: + function_name: regulatory-reporter + runtime: python3.11 + handler: index.handler + - name: l1-sqs + inputs: + queue_name: regulatory-queue + visibility_timeout: "120" + - name: l1-s3 + inputs: + bucket_name: acdl-regulatory-archive + region: us-east-1 + retention_days: "2555" \ No newline at end of file diff --git a/scripts/confidence_signal.py b/scripts/confidence_signal.py new file mode 100755 index 0000000..d47168b --- /dev/null +++ b/scripts/confidence_signal.py @@ -0,0 +1,55 @@ +#!/usr/bin/env python3 +"""confidence_signal.py — REQ-08 / D-024 + +Reads a contract.yaml, invokes policy_checker.py as a subprocess, and emits +a deterministic JSON confidence score. + + policy pass -> {"score": 0.90, "reason": "POLICY_PASS"} + policy fail -> {"score": 0.40, "reason": ""} + +Exit 0 ALWAYS (per D-024): the pipeline decides the gate, not this script's +exit code. + +Input: argv[1] = path to a contract.yaml file. +""" +import json +import os +import subprocess +import sys + + +def main() -> int: + if len(sys.argv) < 2: + print("usage: confidence_signal.py ", file=sys.stderr) + return 1 + + contract_path = sys.argv[1] + + # Resolve policy_checker.py relative to this script so it works regardless + # of cwd. Use python3 + script path (not ./) per the contract. + here = os.path.dirname(os.path.abspath(__file__)) + policy_checker = os.path.join(here, "policy_checker.py") + + proc = subprocess.run( + ["python3", policy_checker, contract_path], + capture_output=True, + text=True, + ) + + if proc.returncode == 0: + score = "0.90" + # POLICY_PASS is the expected stdout; strip any trailing whitespace. + reason = proc.stdout.strip() or "POLICY_PASS" + else: + score = "0.40" + # The violation code (e.g. "POLICY_VIOLATION:PUBLIC_INGRESS") is on stdout. + reason = proc.stdout.strip() or "POLICY_VIOLATION:UNKNOWN" + + # Emit with literal score (two-decimal form per the contract) and a quoted + # reason. Constructed manually so json.dumps does not collapse 0.90 -> 0.9. + print('{"score": ' + score + ', "reason": ' + json.dumps(reason) + '}') + return 0 + + +if __name__ == "__main__": + sys.exit(main()) \ No newline at end of file diff --git a/scripts/evidence_writer.py b/scripts/evidence_writer.py new file mode 100755 index 0000000..09472bc --- /dev/null +++ b/scripts/evidence_writer.py @@ -0,0 +1,123 @@ +#!/usr/bin/env python3 +"""evidence_writer.py — REQ-11 / D-023 / D-005 + +Appends a hash-chained event to audit.json. + +Each event: {"seq": N, "ts": , "stage": "...", "event": "...", + "prev_hash": "", "hash": ""} + +Hash chain (D-023): + 1. Build event dict with hash = "" (empty string). + 2. canonical = json.dumps(event, sort_keys=True, separators=(",", ":")) + 3. hash = sha256(canonical.encode("utf-8")).hexdigest() + 4. event["hash"] = hash + 5. append to audit.json + +Auto-genesis: if audit.json is empty/missing and --stage is not "genesis", +a genesis event (seq 0, prev_hash "GENESIS") is inserted first. + +Input: + --stage (required) + --event "" (required) + --audit (optional, default ./audit.json) +Output: stdout {"seq": N, "hash": "..."} +Exit: 0 on success, 1 on I/O error. +""" +import argparse +import datetime +import hashlib +import json +import os +import sys + +GENESIS_EVENT_TEXT = "audit log initialized" + + +def now_iso8601_utc() -> str: + return datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") + + +def compute_hash(event: dict) -> str: + """Compute the sha256 hash of an event using canonical JSON (D-023).""" + tmp = dict(event) + tmp["hash"] = "" + canonical = json.dumps(tmp, sort_keys=True, separators=(",", ":")) + return hashlib.sha256(canonical.encode("utf-8")).hexdigest() + + +def make_event(seq: int, stage: str, event_text: str, prev_hash: str) -> dict: + event = { + "seq": seq, + "ts": now_iso8601_utc(), + "stage": stage, + "event": event_text, + "prev_hash": prev_hash, + "hash": "", + } + event["hash"] = compute_hash(event) + return event + + +def load_audit(audit_path: str) -> list: + if not os.path.exists(audit_path): + return [] + try: + with open(audit_path, "r", encoding="utf-8") as fh: + data = json.load(fh) + except (json.JSONDecodeError, ValueError): + return [] + if not isinstance(data, list): + return [] + return data + + +def atomic_write(audit_path: str, data: list) -> None: + tmp_path = audit_path + ".tmp" + with open(tmp_path, "w", encoding="utf-8") as fh: + json.dump(data, fh, indent=2) + fh.write("\n") + os.replace(tmp_path, audit_path) + + +def main() -> int: + parser = argparse.ArgumentParser(description="Append a hash-chained event to audit.json") + parser.add_argument("--stage", required=True, + choices=["dev", "qa", "prod", "finalize", "genesis"]) + parser.add_argument("--event", required=True) + parser.add_argument("--audit", default="./audit.json") + args = parser.parse_args() + + events = load_audit(args.audit) + + # Auto-genesis: if the log is empty and the caller did not ask for a + # genesis event, seed one first. + if len(events) == 0 and args.stage != "genesis": + genesis = make_event(seq=0, stage="genesis", event_text=GENESIS_EVENT_TEXT, + prev_hash="GENESIS") + events.append(genesis) + + # Determine the new seq + prev_hash. + if events: + last = events[-1] + seq = last["seq"] + 1 + prev_hash = last["hash"] + else: + seq = 0 + prev_hash = "GENESIS" + + new_event = make_event(seq=seq, stage=args.stage, event_text=args.event, + prev_hash=prev_hash) + events.append(new_event) + + try: + atomic_write(args.audit, events) + except OSError as exc: + print(f"evidence_writer: I/O error: {exc}", file=sys.stderr) + return 1 + + print(json.dumps({"seq": new_event["seq"], "hash": new_event["hash"]})) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) \ No newline at end of file diff --git a/scripts/l3b_agent_stub.py b/scripts/l3b_agent_stub.py new file mode 100755 index 0000000..bf1c93f --- /dev/null +++ b/scripts/l3b_agent_stub.py @@ -0,0 +1,118 @@ +#!/usr/bin/env python3 +"""l3b_agent_stub.py — D-008 / D-026 / D-021 + +Parses a GitHub/Gitea Issue body by keywords and emits a contract.yaml that +selects an L2 stack. This is the agentic (L3B) entry surface: deterministic +keyword matching, no external AI APIs. + +D-008 keyword map (priority order — first match wins): + gas, price, ingest, data-lake -> l2-commodity-price-feed + invoice, billing -> l2-invoice-service + analytics, historical, query -> l2-energy-analytics-api + regulatory, compliance, reporting, trading + -> l2-regulatory-reporting + (no match) -> l2-invoice-service (fallback) + +Output contract.yaml (D-021 schema): + stack: + inputs: + environment: dev + owner: citizen-developer + source: l3b-agent-stub + public-ingress: false + +Input: + argv[1] = issue body text (or stdin if argv[1] absent/empty) + -o = write the contract to a file (default: stdout) +Exit: + 0 on success, 1 on empty input +""" +import sys + + +# Ordered keyword groups -> L2 stack mapping (D-008). First match wins. +KEYWORD_MAP = [ + (("gas", "price", "ingest", "data-lake"), "l2-commodity-price-feed"), + (("invoice", "billing"), "l2-invoice-service"), + (("analytics", "historical", "query"), "l2-energy-analytics-api"), + (("regulatory", "compliance", "reporting", "trading"), "l2-regulatory-reporting"), +] + +FALLBACK_STACK = "l2-invoice-service" + + +def map_issue_to_stack(text: str) -> str: + lowered = text.lower() + for keywords, stack in KEYWORD_MAP: + for kw in keywords: + if kw in lowered: + return stack + return FALLBACK_STACK + + +def render_contract(stack: str) -> str: + # Fixed-schema YAML (D-021). Emitted as text (no yaml dependency needed). + return ( + f"stack: {stack}\n" + "inputs:\n" + " environment: dev\n" + " owner: citizen-developer\n" + " source: l3b-agent-stub\n" + "public-ingress: false\n" + ) + + +def read_issue_body(args: list) -> str: + """Read issue body from args[0] (already-stripped argv, no script name) + or stdin. Empty -> error.""" + if len(args) >= 1 and args[0].strip(): + return args[0] + # Fall back to stdin if argv body is absent or empty. + if not sys.stdin.isatty(): + data = sys.stdin.read() + if data.strip(): + return data + return "" + + +def parse_output_flag(argv: list): + """Extract -o from argv (returns (rest, output_path)).""" + output_path = None + rest = [] + i = 1 + while i < len(argv): + arg = argv[i] + if arg == "-o": + if i + 1 < len(argv): + output_path = argv[i + 1] + i += 2 + continue + else: + print("l3b_agent_stub: -o requires a path argument", file=sys.stderr) + sys.exit(1) + rest.append(arg) + i += 1 + return rest, output_path + + +def main() -> int: + rest, output_path = parse_output_flag(sys.argv) + body = read_issue_body(rest) + if not body.strip(): + print("l3b_agent_stub: empty issue body (no argv[1] and no stdin)", file=sys.stderr) + return 1 + + stack = map_issue_to_stack(body) + contract = render_contract(stack) + + if output_path: + with open(output_path, "w", encoding="utf-8") as fh: + fh.write(contract) + else: + sys.stdout.write(contract) + + return 0 + + +if __name__ == "__main__": + sys.exit(main()) \ No newline at end of file diff --git a/scripts/mock_executor.sh b/scripts/mock_executor.sh new file mode 100755 index 0000000..df02770 --- /dev/null +++ b/scripts/mock_executor.sh @@ -0,0 +1,126 @@ +#!/usr/bin/env bash +# mock_executor.sh — REQ-06 / D-022 +# +# Reads a contract.yaml, resolves the L2 composition, invokes each L1's +# mock_apply.sh in order, and writes state.json to the current working +# directory. +# +# Input: argv[1] = path to a contract.yaml file. +# Output: +# - stdout: per-L1 progress (echoed from each mock_apply.sh) +# - state.json in cwd: {"l2": "...", "l1s": [...], "contract": {...}} +# Exit: +# 0 if all L1s exit 0; 1 if any L1 exited non-zero (state.json is still +# written with the recorded exit codes). +set -euo pipefail + +if [[ $# -lt 1 ]]; then + echo "usage: mock_executor.sh " >&2 + exit 1 +fi + +CONTRACT_PATH="$1" + +if [[ ! -f "$CONTRACT_PATH" ]]; then + echo "contract not found: $CONTRACT_PATH" >&2 + exit 1 +fi + +# --- Parse the contract (stack + full contract dict) via python3 + yaml. --- +# Emit stack on line 1 and the full contract JSON on line 2, then read both +# lines into separate bash variables (so the JSON's internal spaces survive). +CONTRACT_PARSED=$(python3 - "$CONTRACT_PATH" <<'PY' +import sys, json, yaml +path = sys.argv[1] +with open(path, "r", encoding="utf-8") as fh: + contract = yaml.safe_load(fh) +if not isinstance(contract, dict): + sys.stderr.write("contract is not a mapping\n") + sys.exit(2) +stack = contract.get("stack", "") +# Use a compact JSON (no spaces) so the single-line contract survives bash +# variable capture cleanly. +print(stack) +print(json.dumps(contract, sort_keys=True, separators=(",", ":"))) +PY +) + +STACK=$(printf '%s\n' "$CONTRACT_PARSED" | sed -n '1p') +CONTRACT_JSON=$(printf '%s\n' "$CONTRACT_PARSED" | sed -n '2p') + +if [[ -z "$STACK" ]]; then + echo "contract missing 'stack' key" >&2 + exit 1 +fi + +# --- Resolve the L2 manifest. --- +L2_MANIFEST="modules/l2/${STACK}/manifest.yaml" +if [[ ! -f "$L2_MANIFEST" ]]; then + echo "L2_NOT_FOUND: ${STACK}" >&2 + exit 1 +fi + +# --- Read the L2's l1s: list (ordered names) via python. --- +L1_NAMES_JSON=$(python3 - "$L2_MANIFEST" <<'PY' +import sys, json, yaml +path = sys.argv[1] +with open(path, "r", encoding="utf-8") as fh: + manifest = yaml.safe_load(fh) +l1s = manifest.get("l1s", []) if isinstance(manifest, dict) else [] +names = [entry.get("name", "") for entry in l1s if isinstance(entry, dict)] +print(json.dumps(names)) +PY +) + +# --- Invoke each L1's mock_apply.sh in order, recording exit codes. --- +# Build the l1s results array in JSON via python, appending as we go. +RESULTS_JSON="[]" + +ALL_OK=0 +while IFS= read -r L1_NAME; do + L1_SCRIPT="modules/l1/${L1_NAME}/mock_apply.sh" + if [[ ! -f "$L1_SCRIPT" ]]; then + echo "L1_NOT_FOUND: ${L1_NAME}" >&2 + exit 1 + fi + + # Capture stdout + exit code. stderr passes through. + L1_OUT=$(bash "$L1_SCRIPT") + L1_RC=$? + + # Echo the L1's stdout so the pipeline sees the progress lines. + printf '%s\n' "$L1_OUT" + + # Record {"name": ..., "applied": true, "exit_code": ...}. + RESULTS_JSON=$(python3 - "$RESULTS_JSON" "$L1_NAME" "$L1_RC" <<'PY' +import sys, json +results = json.loads(sys.argv[1]) +name = sys.argv[2] +rc = int(sys.argv[3]) +results.append({"name": name, "applied": True, "exit_code": rc}) +print(json.dumps(results)) +PY +) + + if [[ $L1_RC -ne 0 ]]; then + ALL_OK=1 + fi +done < <(python3 -c "import sys, json; print('\n'.join(json.loads(sys.argv[1])))" "$L1_NAMES_JSON") + +# --- Write state.json to the current working directory (D-022). --- +python3 - "$RESULTS_JSON" "$STACK" "$CONTRACT_JSON" <<'PY' +import sys, json +results = json.loads(sys.argv[1]) +stack = sys.argv[2] +contract = json.loads(sys.argv[3]) +state = { + "l2": stack, + "l1s": results, + "contract": contract, +} +with open("state.json", "w", encoding="utf-8") as fh: + json.dump(state, fh, indent=2) + fh.write("\n") +PY + +exit "$ALL_OK" \ No newline at end of file diff --git a/scripts/policy_checker.py b/scripts/policy_checker.py new file mode 100755 index 0000000..59fe163 --- /dev/null +++ b/scripts/policy_checker.py @@ -0,0 +1,51 @@ +#!/usr/bin/env python3 +"""policy_checker.py — REQ-07 / D-025 + +Reads a contract.yaml and enforces the single Phase-03 policy rule: +`public-ingress: true` is forbidden. + +Input: argv[1] = path to a contract.yaml file. +Output: stdout "POLICY_PASS" or "POLICY_VIOLATION:PUBLIC_INGRESS" +Exit: 0 on pass, 1 on violation. + +Idempotent, no side effects (no file writes). Treats an absent or falsy +`public-ingress` key as a pass. +""" +import sys +import yaml + + +def main() -> int: + if len(sys.argv) < 2: + print("usage: policy_checker.py ", file=sys.stderr) + return 2 + + contract_path = sys.argv[1] + + try: + with open(contract_path, "r", encoding="utf-8") as fh: + contract = yaml.safe_load(fh) + except FileNotFoundError: + print(f"contract not found: {contract_path}", file=sys.stderr) + return 2 + except yaml.YAMLError as exc: + print(f"invalid yaml: {exc}", file=sys.stderr) + return 2 + + # Treat missing/non-mapping as no policy violation. + if not isinstance(contract, dict): + print("POLICY_PASS") + return 0 + + public_ingress = contract.get("public-ingress", False) + + if public_ingress is True: + print("POLICY_VIOLATION:PUBLIC_INGRESS") + return 1 + + print("POLICY_PASS") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) \ No newline at end of file diff --git a/scripts/verify_phase03.sh b/scripts/verify_phase03.sh new file mode 100755 index 0000000..83cccf0 --- /dev/null +++ b/scripts/verify_phase03.sh @@ -0,0 +1,240 @@ +#!/usr/bin/env bash +# Phase 03 verification script. +# Confirms the 4 L2 modules and the 5 core scripts conform to their contracts. +# +# Usage: scripts/verify_phase03.sh +# Exit codes: 0 = all checks passed; 1 = one or more checks failed. + +set -uo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +cd "$ROOT" + +fail_count=0 +pass() { printf ' [PASS] %s\n' "$1"; } +fail() { printf ' [FAIL] %s\n' "$1"; fail_count=$((fail_count + 1)); } + +# Expected L2 names per REQ-04. +EXPECTED_L2S=( + l2-invoice-service + l2-commodity-price-feed + l2-energy-analytics-api + l2-regulatory-reporting +) + +echo "== Phase 03 verification ==" +echo "Root: ${ROOT}" +echo + +# --- Check 1: exactly 4 L2 folders with the expected names --- +echo "-- Check 1: 4 L2 folders with expected names --" +actual=$(ls modules/l2/ 2>/dev/null | sort | tr '\n' ' ') +expected=$(printf '%s\n' "${EXPECTED_L2S[@]}" | sort | tr '\n' ' ') +if [ "$actual" = "$expected" ]; then + pass "exactly 4 L2 folders present and named correctly" +else + fail "L2 folder list mismatch" + echo " expected: $expected" + echo " actual: $actual" +fi + +# --- Check 2: each L2 manifest.yaml validates + references 5 existing L1s --- +echo "-- Check 2: L2 manifests reference 5 existing L1s --" +l2_validate=$(python3 << 'PYEOF' || true +import yaml, glob, os, sys +ok = True +l1s = set(os.listdir('modules/l1')) +for f in sorted(glob.glob('modules/l2/*/manifest.yaml')): + d = yaml.safe_load(open(f)) + folder = os.path.basename(os.path.dirname(f)) + problems = [] + if d.get('name') != folder: problems.append(f"name != {folder}") + if d.get('kind') != 'l2': problems.append("kind != l2") + refs = [x.get('name') for x in d.get('l1s', [])] + if len(refs) != 5: problems.append(f"expected 5 l1s, got {len(refs)}") + unknown = [r for r in refs if r not in l1s] + if unknown: problems.append(f"unknown L1 refs: {unknown}") + # each l1 entry must have an inputs: map + for x in d.get('l1s', []): + if not isinstance(x.get('inputs'), dict): problems.append(f"l1 {x.get('name')} missing inputs map") + status = 'OK' if not problems else 'FAIL: ' + '; '.join(problems) + print(f' [{status}] {f}') + if problems: ok = False +sys.exit(0 if ok else 1) +PYEOF +) +echo "$l2_validate" +if [ "$l2_validate" = "" ] || echo "$l2_validate" | grep -q FAIL; then + if ! echo "$l2_validate" | grep -q PASS; then + fail "one or more L2 manifests invalid (see above)" + fi +else + pass "all 4 L2 manifests valid" +fi +# Re-run for the explicit pass/fail count +python3 << 'PYEOF' > /tmp/l2_check.txt 2>&1 || true +import yaml, glob, os, sys +ok = True +l1s = set(os.listdir('modules/l1')) +for f in sorted(glob.glob('modules/l2/*/manifest.yaml')): + d = yaml.safe_load(open(f)) + folder = os.path.basename(os.path.dirname(f)) + if d.get('name') != folder: ok = False + if d.get('kind') != 'l2': ok = False + refs = [x.get('name') for x in d.get('l1s', [])] + if len(refs) != 5: ok = False + if any(r not in l1s for r in refs): ok = False + for x in d.get('l1s', []): + if not isinstance(x.get('inputs'), dict): ok = False +sys.exit(0 if ok else 1) +PYEOF +if [ $? -eq 0 ]; then pass "all 4 L2 manifests pass structural + reference checks"; else fail "L2 manifest structural check"; fi + +# --- Check 3: typecheck (bash -n + py_compile + yaml load) --- +echo "-- Check 3: typecheck --" +if bash -n scripts/mock_executor.sh; then pass "bash -n mock_executor.sh"; else fail "bash -n mock_executor.sh"; fi +if python3 -m py_compile scripts/policy_checker.py scripts/confidence_signal.py scripts/evidence_writer.py scripts/l3b_agent_stub.py 2>/dev/null; then + pass "py_compile all 4 python scripts" +else + fail "py_compile" +fi +if python3 -c "import yaml, glob; [yaml.safe_load(open(f)) for f in glob.glob('modules/l2/*/manifest.yaml')]" 2>/dev/null; then + pass "yaml load all L2 manifests" +else + fail "yaml load L2 manifests" +fi + +# --- Check 4: policy_checker (D-025) --- +echo "-- Check 4: policy_checker behavior (D-025) --" +WORK="$(mktemp -d)" +trap 'rm -rf "$WORK" "$ROOT/tmp_pass_contract.yaml" "$ROOT/tmp_fail_contract.yaml" "$ROOT/state.json" 2>/dev/null || true' EXIT +printf 'stack: l2-commodity-price-feed\npublic-ingress: false\n' > "$WORK/pass.yaml" +printf 'stack: l2-regulatory-reporting\npublic-ingress: true\n' > "$WORK/fail.yaml" +out=$(python3 scripts/policy_checker.py "$WORK/pass.yaml" 2>&1); rc=$? +if [ "$out" = "POLICY_PASS" ] && [ "$rc" = "0" ]; then + pass "policy_checker pass contract -> POLICY_PASS exit 0" +else + fail "policy_checker pass contract: got '$out' exit=$rc" +fi +out=$(python3 scripts/policy_checker.py "$WORK/fail.yaml" 2>&1); rc=$? +if [ "$out" = "POLICY_VIOLATION:PUBLIC_INGRESS" ] && [ "$rc" = "1" ]; then + pass "policy_checker fail contract -> POLICY_VIOLATION:PUBLIC_INGRESS exit 1" +else + fail "policy_checker fail contract: got '$out' exit=$rc" +fi + +# --- Check 5: confidence_signal (D-024) --- +echo "-- Check 5: confidence_signal behavior (D-024) --" +out=$(python3 scripts/confidence_signal.py "$WORK/pass.yaml" 2>&1); rc=$? +if echo "$out" | grep -q '"score": 0.90' && [ "$rc" = "0" ]; then + pass "confidence_signal pass -> score 0.90 exit 0" +else + fail "confidence_signal pass: got '$out' exit=$rc" +fi +out=$(python3 scripts/confidence_signal.py "$WORK/fail.yaml" 2>&1); rc=$? +if echo "$out" | grep -q '"score": 0.40' && [ "$rc" = "0" ]; then + pass "confidence_signal fail -> score 0.40 exit 0" +else + fail "confidence_signal fail: got '$out' exit=$rc" +fi + +# --- Check 6: evidence_writer hash chain (D-023) --- +echo "-- Check 6: evidence_writer hash chain (D-023) --" +rm -f "$WORK/audit.json" +python3 scripts/evidence_writer.py --stage dev --event "dev start" --audit "$WORK/audit.json" > /dev/null +python3 scripts/evidence_writer.py --stage qa --event "qa approved" --audit "$WORK/audit.json" > /dev/null +python3 scripts/evidence_writer.py --stage prod --event "prod approved" --audit "$WORK/audit.json" > /dev/null +chain_ok=$(python3 << PYEOF +import json, hashlib, sys +try: + events = json.load(open("$WORK/audit.json")) + assert len(events) == 4, f"expected 4 (genesis + 3), got {len(events)}" + assert events[0]['prev_hash'] == 'GENESIS', "genesis prev_hash" + for i in range(1, len(events)): + assert events[i]['prev_hash'] == events[i-1]['hash'], f"chain break at {i}" + e = dict(events[i]); h = e.pop('hash'); e['hash'] = '' + canon = json.dumps(e, sort_keys=True, separators=(',',':')) + assert hashlib.sha256(canon.encode()).hexdigest() == h, f"hash mismatch at {i}" + print("OK") +except AssertionError as ex: + print(f"FAIL: {ex}") + sys.exit(1) +PYEOF +) +if [ "$chain_ok" = "OK" ]; then + pass "evidence_writer: 4 events, GENESIS + 3, chain links + hashes valid" +else + fail "evidence_writer chain: $chain_ok" +fi + +# --- Check 7: mock_executor (D-022) --- +echo "-- Check 7: mock_executor writes state.json (D-022) --" +rm -f "$ROOT/state.json" +out=$(bash scripts/mock_executor.sh "$WORK/pass.yaml" 2>&1); rc=$? +if [ "$rc" != "0" ]; then + fail "mock_executor exit $rc (expected 0)" +else + me_ok=$(python3 << PYEOF +import json, sys +try: + s = json.load(open("$ROOT/state.json")) + assert s['l2'] == 'l2-commodity-price-feed', f"l2 mismatch: {s.get('l2')}" + assert 'l1s' in s and len(s['l1s']) == 5, f"expected 5 l1s, got {len(s.get('l1s', []))}" + assert all(x['applied'] is True and x['exit_code'] == 0 for x in s['l1s']), "l1 not all applied+0" + assert 'contract' in s, "missing contract field" + print("OK") +except Exception as ex: + print(f"FAIL: {ex}") + sys.exit(1) +PYEOF +) + if [ "$me_ok" = "OK" ]; then + pass "mock_executor: state.json with l2 + 5 l1s (all exit 0) + contract" + else + fail "mock_executor state.json: $me_ok" + fi +fi +rm -f "$ROOT/state.json" + +# --- Check 8: l3b_agent_stub D-008 keyword map --- +echo "-- Check 8: l3b_agent_stub keyword map (D-008) --" +act3=$(python3 scripts/l3b_agent_stub.py "We need to ingest natural gas prices from Platts and report on compliance." 2>&1) +if echo "$act3" | grep -q 'stack: l2-commodity-price-feed'; then + pass "l3b Act 3 example -> l2-commodity-price-feed" +else + fail "l3b Act 3 example: got '$act3'" +fi +fallback=$(python3 scripts/l3b_agent_stub.py "please deploy something" 2>&1) +if echo "$fallback" | grep -q 'stack: l2-invoice-service'; then + pass "l3b fallback (no keywords) -> l2-invoice-service" +else + fail "l3b fallback: got '$fallback'" +fi +regulatory=$(python3 scripts/l3b_agent_stub.py "regulatory compliance reporting for trading desk" 2>&1) +if echo "$regulatory" | grep -q 'stack: l2-regulatory-reporting'; then + pass "l3b regulatory keywords -> l2-regulatory-reporting" +else + fail "l3b regulatory: got '$regulatory'" +fi +invoice=$(python3 scripts/l3b_agent_stub.py "monthly invoice and billing reconciliation" 2>&1) +if echo "$invoice" | grep -q 'stack: l2-invoice-service'; then + pass "l3b invoice keywords -> l2-invoice-service" +else + fail "l3b invoice: got '$invoice'" +fi +analytics=$(python3 scripts/l3b_agent_stub.py "historical analytics and query API" 2>&1) +if echo "$analytics" | grep -q 'stack: l2-energy-analytics-api'; then + pass "l3b analytics keywords -> l2-energy-analytics-api" +else + fail "l3b analytics: got '$analytics'" +fi + +echo +echo "== Summary ==" +if [ "$fail_count" -eq 0 ]; then + echo "Phase 03 verification PASSED (4 L2s + 5 core scripts, all checks ok)" + exit 0 +else + echo "Phase 03 verification FAILED (${fail_count} check(s) failed)" + exit 1 +fi \ No newline at end of file