dca35c78ec
---ci---
phase: 22
title: rename-and-production-static-assets-stack
status: complete
verification:
- scripts/run_ci.sh: PASS (CI PIPELINE OK)
- python3 -m pytest tests/ -v: 175 passed
- scripts/run_platform.sh --check-only: PASS (PLATFORM CHECK OK)
- grep -R "static-asset[^s]" . (excl .git/): 0 hits
- grep -R "static-asset$" . (excl .git/): 0 hits
- floating git tags v1.6 + v1 point at v1.6.0 (a90a756)
changed_files:
- Task 1 (rename): contracts/static-asset.yaml→static-assets.yaml (git mv); modules/l2/static-asset→static-assets (git mv); sed replaceAll static-asset→static-assets in 22 files (README, docs, scripts/run_platform.sh, pipelines/deploy.yaml, modules/registry.json, tests/*, .ciagent/* historical narrative)
- Task 2 (cloudfront primitive): modules/l1/cloudfront/interface.json + README.md
- Task 3 (waf primitive): modules/l1/waf/interface.json + README.md
- Task 4 (registry): modules/registry.json (+cloudfront, +waf, static-assets renamed)
- Task 5 (augment static-assets): modules/l2/static-assets/composition.json (s3+cloudfront+waf, depth 1); modules/l1/s3/interface.json +instance.json (+bucket_regional_domain_name output); modules/l2/static-assets/README.md (production stack docs)
- Task 6 (adapter): adapters/terraform/adapter.py (+TYPE_MAP/INPUT_MAP/OUTPUT_MAP for cloudfront distribution+OAC+wafv2 webacl; special handling in _emit_resource for OAC defaults, distribution origin/cache_behavior/restrictions/viewer_certificate/web_acl_id, waf scope/default_action/visibility_config/managed rules)
- Task 7 (contract schema): no change needed (generic inputs object; new module names match ^[a-z][a-z0-9-]*$)
- Task 8 (@v1.6 bump): contracts/static-assets.yaml, .gitea/.github/workflows/deploy.yml (ref: v1.6 + header comments), docs/consumer-guide.md, docs/contracts/index.md, docs/pipeline/versioning.md, docs/pipeline/index.md, docs/architecture.md, README.md, modules/l2/microservice/README.md, tests/test_environment_check.py, tests/test_pipeline_contract.py
- Task 9 (floating tags): git tag -f v1.6 v1.6.0; git tag -f v1 v1.6.0
- Task 10 (tests): tests/test_adapter.py (registry 11 entries/9 L1/2 L2; cloudfront+waf type map tests; TestS3Output bucket_regional_domain_name; TestStaticAssetsStack 4 tests); tests/test_contract_resolver.py (+s3/cloudfront/waf resource assertions)
generated:
- terraform/spike/main.tf + terraform.tf (regenerated by run_platform.sh --check-only; reflect static-assets production stack + backend key spike/static-assets/)
notes:
- D-048 full rewrite of .ciagent/ historical narrative (verbatim phase descriptions, REQ-25/27/50, D-036) — produces intentional tautologies (e.g. "Rename static-assets → static-assets") per the decision to override the v1.6 preservation precedent.
- cloudfront interface.json resources array ordered distribution-first so the resolver (first-match wire resolution) routes bucket_regional_domain_name/waf_web_acl_arn/region to the distribution; the OAC gets adapter-provided defaults (name=acdl-oac, origin_type=s3, signing_behavior=always).
- .ciagent/ @v1.4 references left as historical record (D-048 scope was static-asset rename only; @v1.4 is historical narrative of Phase 20).
- s3 OUTPUT_MAP bucket_regional_domain_name not added (identity fallback in adapt() already handles it; OUTPUT_MAP documents non-identity mappings only).
---ci---
269 lines
10 KiB
Bash
Executable File
269 lines
10 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# scripts/run_platform.sh - the ACDL platform pipeline.
|
|
#
|
|
# Usage:
|
|
# run_platform.sh <contract.yaml> (full e2e with AWS)
|
|
# run_platform.sh --check-only [contract.yaml] (offline, no AWS/Checkov/DynamoDB)
|
|
# run_platform.sh --plan-only <contract.yaml> (AWS plan only, no Checkov/outbox)
|
|
#
|
|
# Modes:
|
|
# --check-only (offline, no AWS/Checkov/DynamoDB — for CI)
|
|
# contract -> resolver -> stack -> adapter -> stream TF -> validate -> exit 0
|
|
# --plan-only (requires AWS creds, no Checkov/outbox)
|
|
# contract -> resolver -> stack -> adapter -> terraform init/validate/plan -> exit 0
|
|
# (default) (requires AWS creds + Checkov + DynamoDB)
|
|
# contract -> resolver -> stack -> adapter -> terraform plan -> Checkov ->
|
|
# confidence -> outbox
|
|
#
|
|
# Flags:
|
|
# --quiet suppress terraform/checkov streaming (output to log only)
|
|
#
|
|
# The contract file is a YAML file validated against schemas/contract.schema.json.
|
|
# The resolver (core/contract_resolver.py) resolves it to a Target Stack
|
|
# instance, which the adapter (adapters/terraform/adapter.py) compiles to Terraform.
|
|
#
|
|
# Uses the rotated spike key (D-039/D-047) from gitignored .env.secrets.
|
|
# Plan-only (no apply); -lock=false per D-P09-1.
|
|
set -euo pipefail
|
|
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
|
|
|
# Capture the caller's CWD before we cd to ROOT. The reusable deploy workflow
|
|
# invokes this script from the CONSUMER repo's workspace root with a relative
|
|
# contract path (e.g. .acdl/contract.yaml); the contract must resolve against
|
|
# the consumer repo, not the platform repo (platform/). Without this, the
|
|
# `[ -f "$CONTRACT" ]` check below looks for the contract inside the platform
|
|
# repo and fails (P0 fix — see docs/CONSUMER_GUIDE.md Step 4).
|
|
CALLER_CWD="$(pwd)"
|
|
cd "$ROOT"
|
|
|
|
CHECK_ONLY=0
|
|
PLAN_ONLY=0
|
|
QUIET=0
|
|
CONTRACT=""
|
|
|
|
for arg in "$@"; do
|
|
case "$arg" in
|
|
--check-only) CHECK_ONLY=1 ;;
|
|
--plan-only) PLAN_ONLY=1 ;;
|
|
--quiet) QUIET=1 ;;
|
|
--*) echo "FAIL: unknown flag: $arg" >&2; exit 1 ;;
|
|
*) CONTRACT="$arg" ;;
|
|
esac
|
|
done
|
|
|
|
# Resolve a caller-supplied contract path to an absolute path against the
|
|
# caller's CWD (captured before `cd "$ROOT"`). The default contract below is
|
|
# intentionally left relative to ROOT (it is set only when no contract was
|
|
# supplied and resolves against ROOT, which is correct for platform-local CI).
|
|
if [ -n "$CONTRACT" ]; then
|
|
case "$CONTRACT" in
|
|
/*) ;;
|
|
*) CONTRACT="$CALLER_CWD/$CONTRACT" ;;
|
|
esac
|
|
fi
|
|
|
|
# Default contract for --check-only (CI uses this)
|
|
if [ -z "$CONTRACT" ]; then
|
|
if [ "$CHECK_ONLY" = "1" ]; then
|
|
CONTRACT="contracts/static-assets.yaml"
|
|
else
|
|
echo "FAIL: contract file required (usage: run_platform.sh <contract.yaml>)" >&2
|
|
exit 1
|
|
fi
|
|
fi
|
|
|
|
fail() { echo "FAIL: $*" >&2; exit 1; }
|
|
|
|
# stream: pipe a command's stdout+stderr to both a log file and the
|
|
# terminal (unless --quiet). Usage: stream <logfile> -- <command...>
|
|
stream() {
|
|
local log="$1"; shift
|
|
if [ "$QUIET" = "1" ]; then
|
|
"$@" > "$log" 2>&1
|
|
else
|
|
"$@" 2>&1 | tee "$log"
|
|
fi
|
|
}
|
|
|
|
CONTRACT_ID="11111111-1111-1111-1111-111111111111" # spike fixed UUID
|
|
WORK="/tmp/acdl_platform_run"
|
|
rm -rf "$WORK"; mkdir -p "$WORK"
|
|
|
|
echo "=== Step 0: environment onboarding check ==="
|
|
if [ -f "$CONTRACT" ]; then
|
|
python3 core/environment_check.py "$CONTRACT" || {
|
|
echo "FAIL: environment not bound — see the onboarding prompt above" >&2
|
|
exit 1
|
|
}
|
|
else
|
|
python3 core/environment_check.py --env=dev || {
|
|
echo "FAIL: environment not bound — see the onboarding prompt above" >&2
|
|
exit 1
|
|
}
|
|
fi
|
|
|
|
echo "=== Step 1: validate contract against contract.schema.json ==="
|
|
[ -f "$CONTRACT" ] || fail "contract file $CONTRACT missing"
|
|
python3 -c "
|
|
import json, yaml, jsonschema
|
|
schema = json.load(open('schemas/contract.schema.json'))
|
|
contract = yaml.safe_load(open('$CONTRACT'))
|
|
jsonschema.validate(contract, schema)
|
|
print(f'contract: module={contract[\"module\"]} env={contract[\"environment\"]} inputs={list(contract.get(\"inputs\",{}).keys())}')
|
|
"
|
|
|
|
echo ""
|
|
echo "=== Step 2: resolve contract -> Target Stack instance ==="
|
|
python3 core/contract_resolver.py "$CONTRACT" "$WORK/stack.json" || fail "resolver failed"
|
|
python3 -c "import json; d=json.load(open('$WORK/stack.json')); print(f'stack: {d[\"stack\"][\"name\"]} {d[\"stack\"][\"kind\"]} {len(d[\"resources\"])} resource(s)')"
|
|
|
|
echo ""
|
|
echo "=== Step 3: adapter compiles stack -> terraform/spike/*.tf ==="
|
|
python3 adapters/terraform/adapter.py "$WORK/stack.json" terraform/spike || fail "adapter failed"
|
|
echo "adapter: emitted terraform/spike/{main.tf,terraform.tf,providers.tf}"
|
|
|
|
if [ "$QUIET" = "0" ]; then
|
|
echo ""
|
|
echo "--- emitted terraform/spike/main.tf ---"
|
|
cat terraform/spike/main.tf
|
|
echo "--- end main.tf ---"
|
|
fi
|
|
|
|
if [ "$CHECK_ONLY" = "1" ]; then
|
|
echo ""
|
|
echo "=== Step 3b: validate adapter output structure (offline) ==="
|
|
python3 -c "
|
|
import json, os
|
|
d = json.load(open('$WORK/stack.json'))
|
|
assert d['stack']['name'] == 'static-assets', f\"expected static-assets, got {d['stack']['name']}\"
|
|
assert len(d['resources']) >= 1
|
|
tf_dir = 'terraform/spike'
|
|
for f in ('main.tf', 'terraform.tf', 'providers.tf'):
|
|
assert os.path.isfile(os.path.join(tf_dir, f)), f'{f} missing'
|
|
main = open(os.path.join(tf_dir, 'main.tf')).read()
|
|
assert 'aws_s3_bucket' in main
|
|
assert 'acdl-spike-bucket' in main
|
|
assert 'versioning' in main
|
|
tf = open(os.path.join(tf_dir, 'terraform.tf')).read()
|
|
assert 'backend' in tf
|
|
assert 'required_version' in tf
|
|
prov = open(os.path.join(tf_dir, 'providers.tf')).read()
|
|
assert 'provider \"aws\"' in prov
|
|
print('adapter output: OK')
|
|
"
|
|
echo ""
|
|
echo "=== PLATFORM CHECK OK ==="
|
|
echo "contract -> resolver -> stack -> adapter -> structure validated (offline, no AWS)"
|
|
exit 0
|
|
fi
|
|
|
|
echo "=== Loading AWS credentials (not needed for --check-only) ==="
|
|
ENV_FILE="$ROOT/.env.secrets"
|
|
[ -f "$ENV_FILE" ] || fail ".env.secrets missing (run scripts/rotate_spike_key.sh)"
|
|
set -a
|
|
. "$ENV_FILE"
|
|
set +a
|
|
export AWS_ACCESS_KEY_ID="$ACDL_AWS_ACCESS_KEY_ID"
|
|
export AWS_SECRET_ACCESS_KEY="$ACDL_AWS_SECRET_ACCESS_KEY"
|
|
export AWS_DEFAULT_REGION="$AWS_DEFAULT_REGION"
|
|
|
|
echo "=== Step 4: terraform init + validate + plan -lock=false (real AWS) ==="
|
|
cd terraform/spike
|
|
|
|
echo ""
|
|
echo "--- terraform init ---"
|
|
stream "$WORK/tf-init.log" terraform init -reconfigure -lock=false -input=false || fail "terraform init failed"
|
|
|
|
echo ""
|
|
echo "--- terraform validate ---"
|
|
stream "$WORK/tf-validate.log" terraform validate || fail "terraform validate failed"
|
|
|
|
echo ""
|
|
echo "--- terraform plan ---"
|
|
stream "$WORK/tf-plan.log" terraform plan -lock=false -input=false -out=tfplan || fail "terraform plan failed"
|
|
|
|
echo ""
|
|
echo "terraform plan OK (1 to add, 0 to change, 0 to destroy expected)"
|
|
cd "$ROOT"
|
|
|
|
if [ "$PLAN_ONLY" = "1" ]; then
|
|
echo ""
|
|
echo "=== PLATFORM PLAN OK ==="
|
|
exit 0
|
|
fi
|
|
|
|
echo ""
|
|
echo "=== Step 5: run Checkov on terraform/spike/main.tf ==="
|
|
if [ "$QUIET" = "0" ]; then
|
|
checkov -f terraform/spike/main.tf --framework terraform -o json --soft-fail 2>&1 | tee "$WORK/checkov.json"
|
|
else
|
|
checkov -f terraform/spike/main.tf --framework terraform -o json --soft-fail > "$WORK/checkov.json" 2> "$WORK/checkov.err"
|
|
fi
|
|
[ -s "$WORK/checkov.json" ] || fail "checkov produced no output"
|
|
echo ""
|
|
echo "checkov summary: $(python3 -c "import json; d=json.load(open('$WORK/checkov.json')); print(len(d.get('results',{}).get('failed_checks',[])), 'failed,', len(d.get('results',{}).get('passed_checks',[])), 'passed')")"
|
|
|
|
echo ""
|
|
echo "=== Step 6: Checkov adapter -> PolicyCheckResult (compliance details) ==="
|
|
python3 adapters/terraform/policy/checkov_adapter.py "$WORK/checkov.json" "$CONTRACT_ID" > "$WORK/pcr.json" || fail "checkov adapter failed"
|
|
python3 -c "
|
|
import json
|
|
pcrs = json.load(open('$WORK/pcr.json'))
|
|
print(f'PolicyCheckResult: {len(pcrs)} record(s)')
|
|
print()
|
|
for pcr in pcrs:
|
|
sev = pcr.get('severity', 'info')
|
|
res = pcr.get('result', 'unknown')
|
|
rule = pcr.get('ruleId', 'unknown')
|
|
msg = pcr.get('message', '')
|
|
marker = 'PASS' if res == 'pass' else 'FAIL' if res == 'fail' else 'SKIP' if res == 'skipped' else res.upper()
|
|
print(f' [{marker}] {sev:8s} {rule:30s} {msg}')
|
|
"
|
|
|
|
echo ""
|
|
echo "=== Step 7: confidence signal compute ==="
|
|
python3 <<PY > "$WORK/signal.json" || fail "confidence signal failed"
|
|
import json
|
|
import core.confidence_signal as c
|
|
pcr = json.load(open("$WORK/pcr.json"))
|
|
inputs = {
|
|
"policy": pcr,
|
|
"validation": {"schema": True, "stack_resolved": True, "tf_validated": True, "tf_planned": True},
|
|
"freshness": {"age_days": 0, "max_age_days": 7},
|
|
"source": {"submitter": "consumer", "commit_sha": "consumer-sha", "signed": False},
|
|
"history": {"prior_rollbacks": 0, "prior_policy_fails": 0},
|
|
"nfrs": {"conformance": None},
|
|
}
|
|
sig = c.compute("$CONTRACT_ID", "dev", inputs)
|
|
print(json.dumps({"score": sig.score, "band": sig.band, "perInput": sig.perInput, "reasonCodes": sig.reasonCodes}, indent=2))
|
|
PY
|
|
BAND=$(python3 -c "import json; print(json.load(open('$WORK/signal.json'))['band'])")
|
|
SCORE=$(python3 -c "import json; print(round(json.load(open('$WORK/signal.json'))['score'],3))")
|
|
echo "confidence: score=$SCORE band=$BAND"
|
|
[ "$BAND" = "pass" ] || fail "confidence band is $BAND, expected pass for dev"
|
|
|
|
echo ""
|
|
echo "=== Step 8: write evidence event to DynamoDB outbox ==="
|
|
STACK_NAME=$(python3 -c "import json; print(json.load(open('$WORK/stack.json'))['stack']['name'])")
|
|
python3 <<PY > "$WORK/event.json" || fail "event build failed"
|
|
import json, datetime
|
|
sig = json.load(open("$WORK/signal.json"))
|
|
event = {
|
|
"contractId": "$CONTRACT_ID",
|
|
"eventType": "CONFIDENCE_COMPUTED",
|
|
"ts": datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
|
|
"environment": "dev",
|
|
"stack": "$STACK_NAME",
|
|
"score": sig["score"],
|
|
"band": sig["band"],
|
|
"prev_event_hash": "GENESIS",
|
|
}
|
|
print(json.dumps(event, indent=2))
|
|
PY
|
|
python3 core/outbox_writer.py "$WORK/event.json" > "$WORK/outbox_item.json" || fail "outbox write failed"
|
|
echo "outbox: $(python3 -c "import json; d=json.load(open('$WORK/outbox_item.json')); print('contractId=', d['contractId'], 'hash=', d['hash'][:16]+'...')")"
|
|
|
|
echo ""
|
|
echo "=== PLATFORM E2E OK ==="
|
|
echo "contract -> resolver -> stack -> terraform plan -> Checkov -> confidence ($BAND) -> outbox"
|
|
exit 0 |