Commit Graph

12 Commits

Author SHA1 Message Date
Jon Chery 07c0349131 feat(P24): platform Lambda + DynamoDB contract ingestion + cross-account IAM
Phase 24 — platform-lambda-and-contract-ingestion.

- core/lambda/contract_ingestor.py: AWS Lambda handler invoked via Function
  URL (IAM auth). Parses JSON body, validates required fields, writes the
  contract to DynamoDB table acdl-contracts (PK consumerRepo, SK
  contractId#submittedAt, status submitted, ISO-8601 submittedAt). report_error
  action is a stub returning "error_report_prepared"; GitHub issue creation is
  wired in Phase 25. Returns 400 on missing fields / unknown action, 500 on
  error. Table name + GitHub-token secret ID come from env (set by Terraform).
- core/lambda/__init__.py: empty package marker.
- terraform/platform/main.tf: DynamoDB acdl-contracts (PITR, SSE via CMK),
  KMS customer-managed key with alias/acdl-platform, Secrets Manager secret
  acdl/github-token, IAM execution role (DynamoDB write + Secrets Manager read +
  KMS decrypt + CloudWatch logs), Lambda acdl-contract-ingestor (Python 3.12,
  handler contract_ingestor.lambda_handler), Function URL with AWS_IAM auth.
  State key platform/terraform.tfstate (distinct from spike/microservice).
- terraform/platform/README.md: documents what it deploys, the state key, how
  to apply, and the cross-account invocation model.
- terraform/platform/consumer_invoke_policy.json: ABAC-scoped policy template
  applied to consumer deploy roles during onboarding; grants
  lambda:InvokeFunctionUrl conditioned on aws:PrincipalTag/acdl:owner ==
  consumerRepo.
- tests/test_contract_ingestor.py: 11 tests (moto-backed DynamoDB mock) covering
  submit_contract put_item shape, report_error stub, missing-field 400, unknown
  action 400, the lambda_handler wrapper with a Function-URL-style event, dict
  body, default action, and internal-error 500.
- docs/environments/index.md: new section documenting the cross-account
  contract-ingestion grant (one-way consumer→platform, D-051) and that
  onboarding now also grants the consumer deploy role InvokeFunctionUrl.
- scripts/run_ci.sh, pipelines/ci.yaml, .gitea/workflows/ci.yml,
  .github/workflows/ci.yml: add core/lambda/contract_ingestor.py to the lint
  py_compile list. The two workflow YAMLs remain byte-identical.

Verification: scripts/run_ci.sh passes all 3 stages (lint/test/check-only);
python3 -m pytest tests/ -v passes all 213 tests (11 new + 202 existing).

---ci---
project: acdl
phase: 24
milestone: v1.7
status: execute
---/ci---
2026-07-22 20:04:10 +00:00
Jon Chery dca35c78ec feat(P22): rename static-asset→static-assets + cloudfront/waf primitives + production stack + @v1.6 bump
---ci---
phase: 22
title: rename-and-production-static-assets-stack
status: complete
verification:
  - scripts/run_ci.sh: PASS (CI PIPELINE OK)
  - python3 -m pytest tests/ -v: 175 passed
  - scripts/run_platform.sh --check-only: PASS (PLATFORM CHECK OK)
  - grep -R "static-asset[^s]" . (excl .git/): 0 hits
  - grep -R "static-asset$" . (excl .git/): 0 hits
  - floating git tags v1.6 + v1 point at v1.6.0 (a90a756)
changed_files:
  - Task 1 (rename): contracts/static-asset.yaml→static-assets.yaml (git mv); modules/l2/static-asset→static-assets (git mv); sed replaceAll static-asset→static-assets in 22 files (README, docs, scripts/run_platform.sh, pipelines/deploy.yaml, modules/registry.json, tests/*, .ciagent/* historical narrative)
  - Task 2 (cloudfront primitive): modules/l1/cloudfront/interface.json + README.md
  - Task 3 (waf primitive): modules/l1/waf/interface.json + README.md
  - Task 4 (registry): modules/registry.json (+cloudfront, +waf, static-assets renamed)
  - Task 5 (augment static-assets): modules/l2/static-assets/composition.json (s3+cloudfront+waf, depth 1); modules/l1/s3/interface.json +instance.json (+bucket_regional_domain_name output); modules/l2/static-assets/README.md (production stack docs)
  - Task 6 (adapter): adapters/terraform/adapter.py (+TYPE_MAP/INPUT_MAP/OUTPUT_MAP for cloudfront distribution+OAC+wafv2 webacl; special handling in _emit_resource for OAC defaults, distribution origin/cache_behavior/restrictions/viewer_certificate/web_acl_id, waf scope/default_action/visibility_config/managed rules)
  - Task 7 (contract schema): no change needed (generic inputs object; new module names match ^[a-z][a-z0-9-]*$)
  - Task 8 (@v1.6 bump): contracts/static-assets.yaml, .gitea/.github/workflows/deploy.yml (ref: v1.6 + header comments), docs/consumer-guide.md, docs/contracts/index.md, docs/pipeline/versioning.md, docs/pipeline/index.md, docs/architecture.md, README.md, modules/l2/microservice/README.md, tests/test_environment_check.py, tests/test_pipeline_contract.py
  - Task 9 (floating tags): git tag -f v1.6 v1.6.0; git tag -f v1 v1.6.0
  - Task 10 (tests): tests/test_adapter.py (registry 11 entries/9 L1/2 L2; cloudfront+waf type map tests; TestS3Output bucket_regional_domain_name; TestStaticAssetsStack 4 tests); tests/test_contract_resolver.py (+s3/cloudfront/waf resource assertions)
generated:
  - terraform/spike/main.tf + terraform.tf (regenerated by run_platform.sh --check-only; reflect static-assets production stack + backend key spike/static-assets/)
notes:
  - D-048 full rewrite of .ciagent/ historical narrative (verbatim phase descriptions, REQ-25/27/50, D-036) — produces intentional tautologies (e.g. "Rename static-assets → static-assets") per the decision to override the v1.6 preservation precedent.
  - cloudfront interface.json resources array ordered distribution-first so the resolver (first-match wire resolution) routes bucket_regional_domain_name/waf_web_acl_arn/region to the distribution; the OAC gets adapter-provided defaults (name=acdl-oac, origin_type=s3, signing_behavior=always).
  - .ciagent/ @v1.4 references left as historical record (D-048 scope was static-asset rename only; @v1.4 is historical narrative of Phase 20).
  - s3 OUTPUT_MAP bucket_regional_domain_name not added (identity fallback in adapt() already handles it; OUTPUT_MAP documents non-identity mappings only).
---ci---
2026-07-22 19:56:52 +00:00
Jon Chery f68f85c9fd review(v1.5): READY TO SHIP — multi-persona code review
acdl-ci / Lint (push) Successful in 7s
acdl-ci / Test (push) Successful in 15s
acdl-ci / Platform check-only (offline) (push) Successful in 9s
---ci---
project: acdl
phase: 20
milestone: v1.5
status: review
verdict: READY TO SHIP
p0: 1 (fixed — contract path resolution in deploy workflow)
p1: 6 (flagged post-hoc)
---/ci---

Multi-persona review of v1.5 phase 20 (docs + reusable deploy workflow).

P0 (blocking) — AUTO-FIXED:
- C1: scripts/run_platform.sh contract path resolution broken in deploy
  workflow. The reusable workflow invokes run_platform.sh from the consumer
  workspace root with a relative contract path (.acdl/contract.yaml), but
  run_platform.sh does `cd "$ROOT"` (platform repo) early, so the relative
  path resolved against the platform repo and the pipeline could never run.
  Fix (commit 75c2274): capture CALLER_CWD before cd "$ROOT"; resolve
  caller-supplied relative paths against CALLER_CWD; default no-arg contract
  stays relative to ROOT (preserves platform-local CI). Reproduced pre-fix;
  verified post-fix.

P1 (important) — FLAGGED FOR POST-HOC REVIEW (do not block ship):
- C2: ref: v1.4 in the deploy workflow platform checkout — no v1.4 tag exists
  (only v1.4.0 / v1.4.1). Operator must create a floating v1.4 tag or change
  the ref to v1.4.1.
- C3: modules/l2/{static-asset,microservice}/README.md still use @v1 in their
  Usage examples; missed by the v1.4 bump.
- S1: static-key override is not wired. ACDL_AWS_* env vars on the OIDC step
  are not read by aws-actions/configure-aws-credentials@v4 (it reads AWS_*
  or its own access-key/secret-key inputs). The README/CONSUMER_GUIDE claim
  a working override that doesn't function as written. Needs a conditional
  step or renamed env vars + input wiring.
- S2: README overstates ABAC repo:org/repo:ref:... scoping. The workflow
  constructs a numeric role name (github.repository_id); the actual claim
  enforcement lives in the IAM trust policy, not in this workflow.
- T1: no deploy-workflow triggers conformance test (CI workflow has one;
  deploy doesn't). Minor — reusable workflows use workflow_call, not push
  triggers, but the contract's triggers field is then unenforced.
- A1: terraform/spike/terraform.tf uploaded as artifact leaks the AWS account
  ID via the state-backend bucket name. Recommend excluding terraform.tf or
  gating artifact upload to non-public repos.

P2 (nits) — listed for awareness: floating-tag terminology imprecision (M1),
  header comment "Gitea Actions" in the GitHub copy (M2, intentional byte-
  identical), pip install split (P1-perf), comment drift in pipelines/deploy.yaml
  header (C4), module README internal inconsistency (C5).

Verdict: READY TO SHIP. The one P0 is fixed. The 6 P1s are post-hoc items —
the deploy workflow is a scaffold whose first real consumer run requires
operator setup (tag, IAM role, secrets) that gates go-live. The P1s should
be addressed before any consumer invokes uses: acdl/.gitea/workflows/
deploy.yml@v1.4 in earnest.

Tests: 154 pass (19 new). run_ci.sh green.
2026-07-22 17:24:28 +00:00
Jon Chery 1598c54a8b feat(P18): testing + CI/CD pipelines - pytest suite, check-only mode, Gitea + GitHub workflows (v1.3.2)
90 offline tests covering adapter, confidence_signal, checkov_adapter,
outbox_writer, and pipeline integration. Identical CI/CD workflows for
Gitea Actions (dev) and GitHub Actions (production). New --check-only
mode for run_platform.sh (offline, no AWS).

---ci---
project: acdl
phase: 18
milestone: v1.3
status: verify
---/ci---
2026-07-22 14:26:11 +00:00
Jon Chery f874879973 fix: compress spike_runner_policy.json to fit AWS 2048-char inline limit
---ci---
project: acdl
phase: 0
milestone: v1.2
status: fix
---/ci---

The expanded policy (4727 chars pretty / 3464 compact) exceeded the AWS
2048-char inline policy limit (total across all inline policies on a user).
Compressed to 1667 chars by: (1) removing DenyEverythingElse (redundant —
IAM is default-deny; the user has no other inline policies), (2) using
action-prefix wildcards (ecs:Create*, ecr:Get*, etc.) instead of listing
every action, (3) removing SIDs.

The compressed policy grants the same effective permissions. The repo
file now matches what should be applied in the AWS Console.
2026-07-22 13:42:44 +00:00
Jon Chery 699aa542df docs(P15): plan-as-execute + verify (v1.2.5, PARTIAL — terraform apply blocked by IAM)
---ci---
project: acdl
phase: 15
milestone: v1.2
status: verify
verdict: PARTIAL
requirements:
  covered: [REQ-34]
  partial: [REQ-33]
blocker:
  - id: P0-IAM
    description: terraform apply fails with AccessDenied on ECS/ECR/IAM/EC2 — live spike_runner_policy.json not pushed (root key deactivated per D-034)
    unblock: operator runs create_iam_user.py with root/admin creds to push the expanded policy, then terraform apply succeeds (plan valid, 13 to add)
---/ci---

Phase 15 plan-as-execute + verify. PARTIAL: terraform apply blocked by IAM.
- Consumer microservice content authored (app.py + Dockerfile + README.md).
- Docker image acdl-microservice:latest built.
- Adapter fixed: ref emission (bare), JSON-string jsonencode, ECS service
  network_configuration/load_balancer/desired_count/launch_type/task_definition,
  listener default_action/load_balancer_arn, target group target_type/vpc_id/protocol,
  VPC tags (not name), IGW + route table association, managed_policy_arns list.
- L1 fixes: l1-ecs-service (removed port from service sub-resource),
  l1-vpc (added intra_refs, removed igw_id output).
- Resolver: intra_refs resolution (refs between sub-resources of same L1).
- terraform validate + plan succeed (13 to add).
- terraform apply BLOCKED (AccessDenied — live IAM policy not updated).
- Evidence event TERRAFORM_APPLY_BLOCKED written to DynamoDB outbox.
- v1.1 S3 regression: byte-identical.
Ready to ship v1.2.5 (partial).
2026-07-21 22:21:36 +00:00
Jon Chery 0fea29cdbb docs(P12): plan-as-execute + verify (v1.2.2)
---ci---
project: acdl
phase: 12
milestone: v1.2
status: verify
verdict: VERIFIED
requirements:
  covered: [REQ-30]
---/ci---

Phase 12 plan-as-execute + verify. scripts/verify_phase12.sh green (22
assertions). All Wave 1 + Wave 2 tasks complete:
- T-12.1: run_spike_*.sh -> run_platform.sh (D-048, --plan-only flag)
- T-12.2: spike_runner_policy.json expanded (ECS + ECR + ELB + IAM + EC2)
- T-12.3: idempotency documented in bootstrap scripts
- T-12.4: P1-1 redacted (no live AWS key IDs in .ciagent/)
- T-12.5: P1-B fixed (PERSONAS.md platform/registry -> modules-ir/registry.json)
Subagent confirmed run_platform.sh --plan-only runs against real AWS, exit 0.
Ready to ship v1.2.2.
2026-07-21 21:01:51 +00:00
Jon Chery 622abe015b phase: 10, status: plan-as-execute, persona: backend-engineer, task: T-10.4+T-10.5+T-10.7
---ci---
project: acdl
phase: 10
milestone: v1.1
status: plan-as-execute
persona: backend-engineer
task: [T-10.4, T-10.5, T-10.7]
requirements.covered: [REQ-27]
---/ci---

Wave 2: contract + resolver + outbox writer.

- T-10.4: contracts/spike.yaml - the spike contract (stack:
  l2-static-asset, environment: dev, inputs bucket_name + region). D-P10-2:
  YAML consumer surface; the resolver parses YAML -> validates against the
  JSON contract schema.
- T-10.5: acdl_platform/contract_resolver.py - resolve(contract_path) ->
  IR instance. 7 steps: load YAML, validate contract schema, look up L2 in
  registry, load composition.json, map inputs through wires, emit IR
  instance, validate IR against ir.schema.json. Verified end-to-end:
  spike.yaml -> IR instance with kind=l2, one l1-s3 resource, validates
  against ir.schema.json.
- T-10.7: acdl_platform/outbox_writer.py - write_event(event) ->
  DynamoDB put_item. SHA-256 over canonical JSON, prev_event_hash=GENESIS
  for the first event (D-P10-3), PK contractId, SK eventType#eventTs, TTL
  expire_at = now + 365d (D-044). stdlib + boto3.

Also regenerated terraform/spike/{main.tf,terraform.tf} by running the
adapter against the resolved L2 IR (the backend key is now
spike/l2-static-asset/terraform.tfstate, derived from the stack name per
D-P10-1).
2026-07-21 19:35:10 +00:00
Jon Chery 3070a68e1d phase: 9, status: plan-as-execute, persona: platform-engineer, task: T-9.5..T-9.7+T-9.9
---ci---
project: acdl
phase: 9
milestone: v1.1
status: plan-as-execute
persona: platform-engineer
task: [T-9.5, T-9.6, T-9.7, T-9.9]
requirements.covered: [REQ-26]
---/ci---

Waves 2+3: Terraform adapter + generated spike TF + run script.

- T-9.5: adapters/terraform/adapter.py - compiles an IR instance to a
  Terraform root module. TYPE_MAP {aws:s3:bucket -> aws_s3_bucket}. Thin
  layer; does not own L1 content. Emits main.tf (resource + outputs) +
  terraform.tf (required_version/providers + S3 backend, NO
  dynamodb_table per D-P09-1) + providers.tf (aws provider region from
  the IR). CLI: adapter.py <ir_instance.json> <out_dir>.

- T-9.6: terraform/spike/{main.tf,terraform.tf,providers.tf} - generated
  by running the adapter against modules-ir/l1/l1-s3/spike_instance.json.
  Committed so verify_phase09.sh can validate/plan without regenerating
  (D-P09-4); the verify script will regenerate + diff to prove
  reproducibility.

- T-9.7: scripts/run_spike_plan.sh - loads rotated spike key from
  gitignored .env.secrets, exports AWS env vars, cd terraform/spike,
  terraform init -lock=false, terraform validate, terraform plan
  -lock=false -out=tfplan. Plan-only; no apply.

- T-9.9: .gitignore - add terraform/spike/.terraform/ + tfplan +
  *.tfstate*.

EXECUTE: ran scripts/run_spike_plan.sh against real AWS via the rotated
spike key (D-039). terraform plan succeeded: 1 to add (the S3 bucket),
outputs computed. One non-blocking deprecation warning (aws_s3_bucket
versioning block -> use aws_s3_bucket_versioning in v1.2). No long-lived
credential in the workflow (key loaded from .env.secrets at runtime).
2026-07-21 19:15:10 +00:00
Jon Chery 1d5c4d2ae7 phase: 8, status: plan-as-execute, persona: platform-engineer+lead-developer, task: T-8.5..T-8.7
---ci---
project: acdl
phase: 8
milestone: v1.1
status: plan-as-execute
persona: platform-engineer+lead-developer
task: [T-8.5, T-8.6, T-8.7]
requirements.covered: [REQ-23]
---/ci---

Waves 3+4: rotation script + verify script + README + .gitignore.

- T-8.5 (platform): scripts/rotate_spike_key.sh - boto3 with bootstrap
  root key from env (ACDL_BOOTSTRAP_AWS_*); creates new key for
  acdl-spike-runner, deactivates+deletes old, writes new to gitignored
  .env.secrets (chmod 600); idempotent (re-run ends with exactly 1
  active key); optional Gitea secret upload if ACDL_GITEA_TOKEN set;
  does NOT rotate the root key (D-034 closure = manual user step).

- T-8.6 (lead): scripts/verify_phase08.sh - loads rotated key from
  .env.secrets, asserts caller identity is acdl-spike-runner (not root),
  S3 bucket + DynamoDB table + IAM user + scoped policy with
  DenyEverythingElse all present, .env.secrets + .bootstrap_state.json
  gitignored. Uses heredoc python to avoid bash quoting issues.

- T-8.7 (lead): terraform/bootstrap/README.md runbook (6 steps incl.
  manual D-034 root-key rotation) + .gitignore (.env.secrets +
  .bootstrap_state.json). Spike vs v1.2 boundary table.

bash -n + gitignore checks pass.
2026-07-21 19:00:08 +00:00
Jon Chery f8ddd8b182 phase: 8, status: plan-as-execute, persona: security-engineer+platform-engineer, task: T-8.1..T-8.4
---ci---
project: acdl
phase: 8
milestone: v1.1
status: plan-as-execute
persona: security-engineer+platform-engineer
task: [T-8.1, T-8.2, T-8.3, T-8.4]
requirements.covered: [REQ-23]
---/ci---

Waves 1+2: IAM policy + state backend + IAM user creation scripts.

- T-8.1 (security): terraform/bootstrap/spike_runner_policy.json —
  least-privilege IAM policy: Allow S3 r/w on the state bucket, DynamoDB
  r/w on the outbox table, sts:GetCallerIdentity; final Deny statement
  (Action *, NotResource = the above ARNs) enforcing least privilege. No
  terraform apply permission (plan-only spike).

- T-8.2/T-8.3 (platform): terraform/bootstrap/create_state_backend.py —
  boto3, idempotent: creates S3 bucket acdl-tfstate-581513795199-us-east-1
  (versioning enabled) + DynamoDB table acdl-outbox (PAY_PER_REQUEST, PK
  contractId, SK eventType#eventTs per D-P08-1 one table for both lock
  + outbox). Writes .bootstrap_state.json marker.

- T-8.4 (platform + security review): terraform/bootstrap/create_iam_user.py
  — boto3, idempotent: creates IAM user acdl-spike-runner, attaches the
  inline policy from spike_runner_policy.json, creates an initial access
  key if none active exists (prints to stdout for the orchestrator to
  capture; NEVER committed).

py_compile + policy JSON valid.
2026-07-21 18:58:29 +00:00
Jon Chery e044a2de0d phase: 6, status: plan-as-execute, persona: lead-developer, task: T-6.1..T-6.4
---ci---
project: acdl
phase: 6
milestone: v1.1
status: plan-as-execute
persona: lead-developer
tasks: [T-6.1, T-6.2, T-6.3, T-6.4]
---/ci---

Archive the v1.0 demo under demo/ (D-037) and reorient the repo to the
real platform. Wave 1 of the Phase 06 plan.

- T-6.1: git mv modules/, scripts/, evidence-ui/, contracts/,
  contracts-repo/, .gitea/ -> demo/; mv ACDL_DEMO.md + runner-data/ -> demo/
- T-6.2: scaffold new v1.1 top-level dirs (platform/, schemas/, adapters/,
  terraform/, modules-ir/) with .gitkeep
- T-6.3: create top-level scripts/verify_phase06.sh (v1.1 verify scripts
  live at top-level, NOT demo/scripts/ which holds the v1.0 demo verify
  scripts)
- T-6.4: rewrite README.md to reflect the real platform (vision +
  architecture links, new layout, status v1.1 active); add runner-data/
  to .gitignore

All moves via git mv (history preserved). Repo root now contains only
README.md, demo/, docs/, .ciagent/, and the new empty v1.1 dirs.
2026-07-21 18:27:21 +00:00