Commit Graph

10 Commits

Author SHA1 Message Date
Jon Chery 0e6ecae26d feat(P4): Nova rebrand — AWS resource migration (REQ-163)
Rename all acdl-* AWS resources → nova-* across terraform (DynamoDB,
Secrets Manager, Lambda, SNS, SG, KMS alias, ECS, ECR, IAM user/policy,
state bucket, ALB, VPC/subnet names). Lambda default table names → nova-*
(D-111). State bucket backend → nova-tfstate (-migrate-state documented).
New docs/NOVA_AWS_MIGRATION.md runbook (staged migration + rollback).
New scripts/migrate_dynamodb_data.py (scan+copy, dry-run default).
acdl-deploy- → nova-deploy- role ARN in deploy workflows. Test fixtures
updated; terraform validate + pytest + run_ci.sh PASS.

---ci---
project: acdl
phase: 4
milestone: v1.15
status: execute
---/ci---
2026-07-30 01:54:26 +00:00
Jon Chery 51c3edf458 feat(P3): Nova rebrand — SSM path + tag keys (REQ-161/162)
SSM path /acdl/{env}/{contractId}/{output} → /nova/... across
core/output_publisher + contract resolver + consumer docs. New
scripts/migrate_ssm_paths.py (copy/verify/delete, dry-run default).
AWS tag keys acdl:owner|environment|contract|cost-center|ref → nova:*
across terraform tagging + ABAC session policies (iam:ResourceTag/acdl:*
→ iam:ResourceTag/nova:*). nova_tagging.py hard mode (D-109 warn→hard).
tagging-standard.json tag-key values → nova:*. New
scripts/untag_acdl_keys.py (remove old acdl:* tags, dry-run default).
Test fixtures updated; pytest + run_ci.sh PASS.

---ci---
project: acdl
phase: 3
milestone: v1.15
status: execute
---/ci---
2026-07-30 01:38:30 +00:00
Jon Chery 3b1181f39b Merge milestone/v1.14-refinement — v1.14 complete (NFR Refinement: bug fixes, security, stubs, tests, docs; 20 phases + final; tag v1.13.24)
acdl-ci / Lint (push) Successful in 10s
acdl-ci / Platform check-only (offline) (push) Successful in 25s
acdl-ci / Test (push) Successful in 6m34s
v1.14 NFR Refinement milestone complete. 20 execution phases (P1-P20) +
1 final (P21). All P1/P2 backlog from v1.11 review resolved. Security
posture hardened (swallowed errors, account ID externalized, IAM scoped,
schema validation, credential hygiene). Stubs resolved (kyverno --kube-
version removed). 7 untested scripts gained coverage. Documentation
synced (ARCHITECTURE v1.11-v1.14 addenda, stale @v1.6-1.9 -> @v1.13,
GRILL G-005/G-008 resolved, COST.md window extended, D-083 deferral
recorded). Platform VPC parameterized.

561 tests pass (was 528 at v1.13.2; +33). 22/22 capabilities Verified.
6 grill binding decisions (G-101..G-106) applied. 1 escalation (E-001)
auto-resolved at full autonomy (D-101).

---ci---
project: acdl
phase: 21
milestone: v1.14
status: complete
---/ci---
2026-07-29 21:36:37 +00:00
Jon Chery c4e94cf171 fix(terraform/platform): make Lambda conditional on zip existing
acdl-ci / Lint (pull_request) Successful in 7s
acdl-ci / Test (pull_request) Successful in 4m5s
acdl-ci / Platform check-only (offline) (pull_request) Successful in 20s
acdl-modules-lifecycle / Platform VPC apply (pull_request) Failing after 2m44s
acdl-modules-lifecycle / L1 lifecycle (alb) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (cloudfront) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (ecr) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (ecs-cluster) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (ecs-service) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (iam-role) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (kms-key) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (rds) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (s3) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (uptime) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (vpc) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (waf) (pull_request) Has been skipped
acdl-modules-lifecycle / Platform VPC destroy (pull_request) Successful in 50s
The Lambda function's filename attribute (contract_ingestor.zip) fails
during terraform apply when the zip doesn't exist (the lifecycle pipeline
only needs the VPC, not the Lambda). Made the Lambda + Function URL
conditional with count = fileexists('contract_ingestor.zip') ? 1 : 0.
The source_code_hash also uses the fileexists guard.

This lets the lifecycle pipeline apply only the VPC resources without
requiring the Lambda zip build artifact.

---ci---
project: acdl
phase: P59
milestone: v1.11
status: execute
---/ci---
2026-07-28 16:55:15 +00:00
Jon Chery 2f8c0203be fix(terraform/platform): quote acdl: tags + fix Lambda + replace interpolation
acdl-ci / Lint (pull_request) Successful in 7s
acdl-ci / Test (pull_request) Successful in 4m5s
acdl-ci / Platform check-only (offline) (pull_request) Successful in 21s
acdl-modules-lifecycle / Platform VPC apply (pull_request) Failing after 42s
acdl-modules-lifecycle / L1 lifecycle (alb) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (cloudfront) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (ecr) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (ecs-cluster) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (ecs-service) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (iam-role) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (kms-key) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (rds) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (s3) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (uptime) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (vpc) (pull_request) Has been skipped
acdl-modules-lifecycle / L1 lifecycle (waf) (pull_request) Has been skipped
acdl-modules-lifecycle / Platform VPC destroy (pull_request) Successful in 48s
3 fixes in terraform/platform/main.tf that prevented terraform validate
from passing in CI:

1. All 40 acdl:owner/contract/environment/cost-center tag keys were
   unquoted (acdl:owner = ...). HCL requires quoting keys with colons.
   Fixed to "acdl:owner" = ...

2. filebase64sha256("contract_ingestor.zip") failed when the zip didn't
   exist (it's a build artifact). Wrapped with fileexists() guard.

3. ${account_id} and ${region} in the replace() call were interpreted
   as Terraform interpolation, not literal strings. Escaped as
   $${account_id} and $${region}.

Platform terraform now passes terraform validate.

---ci---
project: acdl
phase: P59
milestone: v1.11
status: execute
---/ci---
2026-07-28 16:42:17 +00:00
Jon Chery fda4564a7f feat(P58): single platform VPC + deterministic env-aware state keys
EXECUTE stage. Fixes the 4-VPC bug: adds a single shared VPC to
terraform/platform, drops the vpc child from the microservice composition
(references the platform VPC via data source), and makes state keys
env-aware (spike/{id}/{env}/terraform.tfstate — stable across lifecycle).

Platform VPC (terraform/platform/main.tf):
- aws_vpc.acdl_shared (10.0.0.0/16) + 2 subnets + IGW + route table + SG
- Outputs: vpc_id, subnet_ids, ecs_security_group_id

Microservice composition (modules/l2/microservice/composition.json):
- Dropped the vpc child (no per-contract VPC ever again).
- Added data_sources block: platform_vpc → terraform_remote_state (platform).
- Wires: vpc.outputs.subnet_ids → platform_vpc.outputs.subnet_ids.
- Wires: platform_vpc.outputs.vpc_id → alb.inputs.vpc_id.
- Wires: platform_vpc.outputs.ecs_security_group_id → service.inputs.security_group.

Contract resolver (core/contract_resolver.py):
- Added environment to the stack instance (stack.environment).
- Added data_sources handling: pseudo-children with outputs but no resources.
- data_sources propagated through fragment merge to the final stack instance.

Adapter (adapters/terraform/adapter.py):
- State key: spike/{stack_name}/{environment}/terraform.tfstate (env-aware).
- Emits data "terraform_remote_state" "platform" block when data_sources present.
- ref:platform_vpc.<output> → data.terraform_remote_state.platform.outputs.<output>.

Tests (tests/test_adapter.py):
- test_adapt_env_aware_state_key: spike/msvc/prod/terraform.tfstate.
- test_adapt_emits_data_source_block: data.terraform_remote_state.platform.
- test_adapt_no_vpc_for_microservice: no resource "aws_vpc" in microservice output.
- Updated existing state key assertion (spike/s3/dev/terraform.tfstate).

Regression: 467 passed, 0 skipped, 5 deselected. run_platform.sh --check-only
passes for both microservice (9 resources, no VPC) and static-assets (5 resources).

---ci---
project: acdl
phase: P58
milestone: v1.11
status: execute
---/ci---
2026-07-28 16:07:57 +00:00
Jon Chery e74a8c2f5d feat(P42): stub implementation — SoD, HITL gates, attestation matrix, Wiz, Kyverno
---ci---
project: acdl
phase: 42
milestone: v1.9
status: execute
---/ci---

Phase 42 — stub-implementation (REQ-107..111, D-084):

route_halt_artifact (REQ-107):
- core/separation_of_duties.py: real SNS publish (ACDL_SOD_HALT_TOPIC_ARN)
  + outbox fallback (SEPARATION_OF_DUTIES_VIOLATION event via
  outbox_writer) + stderr emission. No silent print-only stub.
- terraform/platform/main.tf: aws_sns_topic.acdl-sod-halt + output.

HITL attestation gates (REQ-108):
- core/hitl_gates.py: attest(contract_id, env, approver, evidence,
  outbox_client) records approver_qa/approver_prod/approver_dr to
  outbox, runs SoD check on prod, invokes attestation matrix, returns
  (ok, reason). Dev skips (autonomous). approver_from_env() reads
  GITHUB_ACTOR/GITEA_ACTOR.
- scripts/run_platform.sh: Step 7b HITL gate before apply for qa/prod/dr.

8-concern attestation matrix (REQ-109, D-084):
- core/attestation_matrix.py: check(env, evidence) runs the 8 concerns
  from hitl_matrix_design.md §10.4. Offline-testable (contract_nfrs,
  schema_validity, policy_pass) run for real. Operator-supplied accept
  signed artifacts validated for freshness (FRESHNESS_DAYS table) +
  schema. Signature skip when ACDL_ATTESTATION_SIGNING_KEY_ID unset
  (D-089). Fail loud if missing/expired for prod/dr.

Wiz real client (REQ-110):
- adapters/wiz/wiz_adapter.py: WizClient (GraphQL API, Bearer auth,
  pagination via pageInfo.hasNextPage + endCursor). fetch_and_adapt
  translates issues → PolicyCheckResult; graceful degrade when
  WIZ_API_TOKEN/WIZ_API_URL unset.

Kyverno fleshed out (REQ-111):
- adapters/kyverno/kyverno_adapter.py: full PolicyReport →
  PolicyCheckResult mapping (pass/fail/skip/warn + severity + skip-with-
  reason + resource ref construction from kind/name/namespace).
  adapt_inactive() emits KYVERNO_INACTIVE_TF_STACK guard. --kube-version
  stub parsed for future GitOps.

Tests: +47 (test_route_halt_artifact.py, test_hitl_gates.py,
test_attestation_matrix.py, test_wiz_adapter_real_client.py, expanded
test_kyverno_adapter.py). Existing wiz_adapter tests updated for the
real client's control.name ruleId. 493 passed; run_ci.sh green;
run_platform.sh --check-only green.
2026-07-23 04:40:44 +00:00
Jon Chery 134f85d2df feat(P34): decommission alias + CMDB validation (REQ-92, REQ-93, REQ-94)
---ci---
project: acdl
phase: 34
milestone: v1.8
status: execute
---/ci---

- DynamoDB acdl-change-requests table added to terraform/platform/main.tf
  (PK changeRequestId, SK submittedAt, SSE via CMK, PITR).
- validate_change_request Lambda action added to contract_ingestor.py:
  queries CMDB, asserts status=approved + consumerRepo match.
- decommission_transform() added to contract_resolver.py: zeroes all
  counts (desired_count, min/max_capacity) + sets deletion_protection=false.
- Decommission mode added to deploy pipeline + both deploy workflows
  (mode: decommission + changeRequestId input). Byte-identical.
- run_platform.sh --decommission flag: validates CR, resolves with
  deletion_protection=false (step 1), then decommission_transform
  (step 2). HITL SRE gates documented.
- docs/consumer-guide.md: new "Decommissioning a stack" section with
  CR request, trigger, 2-step HITL SRE gates, CMK deletion window, uptime.

Tests: +14 (318 -> 332). All pass.
2026-07-22 22:18:28 +00:00
Jon Chery 843cd17b97 fix(P29): SSM fail-loud without CMK + Terraform-rendered invoke policy (P1-3, P1-6)
---ci---
project: acdl
phase: 29
milestone: v1.8
status: execute
---/ci---

P1-3: SSM publisher now raises RuntimeError when ACDL_KMS_KEY_ID is
unset. ACDL_ALLOW_DEFAULT_KMS=1 escape hatch for local testing.
P1-6: consumer_invoke_policy.json now uses ${account_id} and ${region}
placeholders. Terraform renders them via data.aws_caller_identity +
data.aws_region + replace() at apply time. No more hardcoded 000000000000.

Tests: +7 (285 -> 292). All pass.
2026-07-22 22:05:40 +00:00
Jon Chery 07c0349131 feat(P24): platform Lambda + DynamoDB contract ingestion + cross-account IAM
Phase 24 — platform-lambda-and-contract-ingestion.

- core/lambda/contract_ingestor.py: AWS Lambda handler invoked via Function
  URL (IAM auth). Parses JSON body, validates required fields, writes the
  contract to DynamoDB table acdl-contracts (PK consumerRepo, SK
  contractId#submittedAt, status submitted, ISO-8601 submittedAt). report_error
  action is a stub returning "error_report_prepared"; GitHub issue creation is
  wired in Phase 25. Returns 400 on missing fields / unknown action, 500 on
  error. Table name + GitHub-token secret ID come from env (set by Terraform).
- core/lambda/__init__.py: empty package marker.
- terraform/platform/main.tf: DynamoDB acdl-contracts (PITR, SSE via CMK),
  KMS customer-managed key with alias/acdl-platform, Secrets Manager secret
  acdl/github-token, IAM execution role (DynamoDB write + Secrets Manager read +
  KMS decrypt + CloudWatch logs), Lambda acdl-contract-ingestor (Python 3.12,
  handler contract_ingestor.lambda_handler), Function URL with AWS_IAM auth.
  State key platform/terraform.tfstate (distinct from spike/microservice).
- terraform/platform/README.md: documents what it deploys, the state key, how
  to apply, and the cross-account invocation model.
- terraform/platform/consumer_invoke_policy.json: ABAC-scoped policy template
  applied to consumer deploy roles during onboarding; grants
  lambda:InvokeFunctionUrl conditioned on aws:PrincipalTag/acdl:owner ==
  consumerRepo.
- tests/test_contract_ingestor.py: 11 tests (moto-backed DynamoDB mock) covering
  submit_contract put_item shape, report_error stub, missing-field 400, unknown
  action 400, the lambda_handler wrapper with a Function-URL-style event, dict
  body, default action, and internal-error 500.
- docs/environments/index.md: new section documenting the cross-account
  contract-ingestion grant (one-way consumer→platform, D-051) and that
  onboarding now also grants the consumer deploy role InvokeFunctionUrl.
- scripts/run_ci.sh, pipelines/ci.yaml, .gitea/workflows/ci.yml,
  .github/workflows/ci.yml: add core/lambda/contract_ingestor.py to the lint
  py_compile list. The two workflow YAMLs remain byte-identical.

Verification: scripts/run_ci.sh passes all 3 stages (lint/test/check-only);
python3 -m pytest tests/ -v passes all 213 tests (11 new + 202 existing).

---ci---
project: acdl
phase: 24
milestone: v1.7
status: execute
---/ci---
2026-07-22 20:04:10 +00:00