docs(P15): plan-as-execute + verify (v1.2.5, PARTIAL — terraform apply blocked by IAM)
---ci---
project: acdl
phase: 15
milestone: v1.2
status: verify
verdict: PARTIAL
requirements:
covered: [REQ-34]
partial: [REQ-33]
blocker:
- id: P0-IAM
description: terraform apply fails with AccessDenied on ECS/ECR/IAM/EC2 — live spike_runner_policy.json not pushed (root key deactivated per D-034)
unblock: operator runs create_iam_user.py with root/admin creds to push the expanded policy, then terraform apply succeeds (plan valid, 13 to add)
---/ci---
Phase 15 plan-as-execute + verify. PARTIAL: terraform apply blocked by IAM.
- Consumer microservice content authored (app.py + Dockerfile + README.md).
- Docker image acdl-microservice:latest built.
- Adapter fixed: ref emission (bare), JSON-string jsonencode, ECS service
network_configuration/load_balancer/desired_count/launch_type/task_definition,
listener default_action/load_balancer_arn, target group target_type/vpc_id/protocol,
VPC tags (not name), IGW + route table association, managed_policy_arns list.
- L1 fixes: l1-ecs-service (removed port from service sub-resource),
l1-vpc (added intra_refs, removed igw_id output).
- Resolver: intra_refs resolution (refs between sub-resources of same L1).
- terraform validate + plan succeed (13 to add).
- terraform apply BLOCKED (AccessDenied — live IAM policy not updated).
- Evidence event TERRAFORM_APPLY_BLOCKED written to DynamoDB outbox.
- v1.1 S3 regression: byte-identical.
Ready to ship v1.2.5 (partial).
This commit is contained in:
Jon Chery
@@ -0,0 +1,131 @@
|
||||
#!/usr/bin/env python3
|
||||
"""ACDL Phase 15 — push the consumer microservice Docker image to ECR.
|
||||
|
||||
Steps performed by this script:
|
||||
1. Load AWS creds from /root/acdl/.env.secrets
|
||||
(ACDL_AWS_ACCESS_KEY_ID, ACDL_AWS_SECRET_ACCESS_KEY, AWS_DEFAULT_REGION).
|
||||
2. Create the ECR repo `acdl-microservice` if it doesn't exist
|
||||
(ecr:DescribeRepositories / ecr:CreateRepository). Region: us-east-1.
|
||||
3. Get the ECR login password (ecr:GetAuthorizationToken) and run
|
||||
`docker login` with it.
|
||||
|
||||
After this script runs, it prints the docker `tag` and `push` commands
|
||||
for the caller to run in the shell (steps 4-5 of T-15.1).
|
||||
|
||||
Usage:
|
||||
python3 scripts/push_consumer_image.py
|
||||
|
||||
Constraints (T-15.1): the `aws` CLI is NOT installed — boto3 is used for
|
||||
every AWS API call. `docker` is invoked via subprocess for the login
|
||||
(since docker is the only thing that can use the auth token meaningfully).
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
import subprocess
|
||||
import pathlib
|
||||
|
||||
import boto3
|
||||
|
||||
|
||||
REPO_ROOT = pathlib.Path(__file__).resolve().parent.parent
|
||||
ENV_FILE = REPO_ROOT / ".env.secrets"
|
||||
AWS_ACCOUNT_ID = "581513795199"
|
||||
AWS_REGION = "us-east-1"
|
||||
ECR_REPO_NAME = "acdl-microservice"
|
||||
IMAGE_TAG = "latest"
|
||||
|
||||
|
||||
def _load_env(path):
|
||||
"""Load ACDL_AWS_* + AWS_DEFAULT_REGION from a flat KEY=VALUE file."""
|
||||
creds = {}
|
||||
with open(path, "r") as fh:
|
||||
for line in fh:
|
||||
line = line.strip()
|
||||
if not line or line.startswith("#") or "=" not in line:
|
||||
continue
|
||||
k, v = line.split("=", 1)
|
||||
creds[k.strip()] = v.strip()
|
||||
return creds
|
||||
|
||||
|
||||
def main():
|
||||
if not ENV_FILE.exists():
|
||||
print(f"FAIL: {ENV_FILE} not found", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
creds = _load_env(ENV_FILE)
|
||||
access_key = creds.get("ACDL_AWS_ACCESS_KEY_ID")
|
||||
secret_key = creds.get("ACDL_AWS_SECRET_ACCESS_KEY")
|
||||
region = creds.get("AWS_DEFAULT_REGION", AWS_REGION)
|
||||
if not access_key or not secret_key:
|
||||
print("FAIL: ACDL_AWS_ACCESS_KEY_ID / ACDL_AWS_SECRET_ACCESS_KEY missing",
|
||||
file=sys.stderr)
|
||||
return 2
|
||||
|
||||
# Export the creds for the docker subprocess (it doesn't need them, but
|
||||
# keeps parity with the terraform step that runs after this).
|
||||
os.environ["AWS_ACCESS_KEY_ID"] = access_key
|
||||
os.environ["AWS_SECRET_ACCESS_KEY"] = secret_key
|
||||
os.environ["AWS_DEFAULT_REGION"] = region
|
||||
|
||||
session = boto3.Session(
|
||||
aws_access_key_id=access_key,
|
||||
aws_secret_access_key=secret_key,
|
||||
region_name=region,
|
||||
)
|
||||
ecr = session.client("ecr")
|
||||
|
||||
# Step 2: create the ECR repo if it doesn't exist.
|
||||
repo_uri = None
|
||||
try:
|
||||
resp = ecr.describe_repositories(repositoryNames=[ECR_REPO_NAME])
|
||||
repo = resp["repositories"][0]
|
||||
repo_uri = repo["repositoryUri"]
|
||||
print(f"ecr: repository {ECR_REPO_NAME!r} already exists -> {repo_uri}")
|
||||
except ecr.exceptions.RepositoryNotFoundException:
|
||||
print(f"ecr: repository {ECR_REPO_NAME!r} not found, creating...")
|
||||
resp = ecr.create_repository(repositoryName=ECR_REPO_NAME)
|
||||
repo = resp["repository"]
|
||||
repo_uri = repo["repositoryUri"]
|
||||
print(f"ecr: created repository {ECR_REPO_NAME!r} -> {repo_uri}")
|
||||
except Exception as exc:
|
||||
print(f"FAIL: ecr describe/create failed: {exc}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
# Step 3: get login password + run `docker login`.
|
||||
auth = ecr.get_authorization_token()
|
||||
token = auth["authorizationData"][0]["authorizationToken"]
|
||||
# The token is base64(USERNAME:PASSWORD); docker login wants them split.
|
||||
import base64
|
||||
user_pw = base64.b64decode(token).decode("utf-8")
|
||||
username, password = user_pw.split(":", 1)
|
||||
registry = f"{AWS_ACCOUNT_ID}.dkr.ecr.{region}.amazonaws.com"
|
||||
|
||||
print(f"docker: logging in to {registry} ...")
|
||||
login_cmd = [
|
||||
"docker", "login",
|
||||
"--username", username,
|
||||
"--password-stdin",
|
||||
registry,
|
||||
]
|
||||
proc = subprocess.run(login_cmd, input=password.encode("utf-8"),
|
||||
capture_output=True)
|
||||
if proc.returncode != 0:
|
||||
print("FAIL: docker login failed:", file=sys.stderr)
|
||||
sys.stderr.write(proc.stderr.decode("utf-8", "replace"))
|
||||
return 1
|
||||
print("docker: login OK")
|
||||
|
||||
# Steps 4-5: print the tag + push commands for the caller to run.
|
||||
full_tag = f"{repo_uri}:{IMAGE_TAG}"
|
||||
print("")
|
||||
print("=== NEXT: run these commands in the shell to tag + push ===")
|
||||
print(f"docker tag acdl-microservice:latest {full_tag}")
|
||||
print(f"docker push {full_tag}")
|
||||
print("")
|
||||
print(f"ECR_IMAGE={full_tag}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Executable
+80
@@ -0,0 +1,80 @@
|
||||
#!/usr/bin/env bash
|
||||
# scripts/verify_phase15.sh - verify Phase 15 (consumer-repo-and-terraform-apply).
|
||||
# NOTE: terraform apply is BLOCKED by IAM (live spike_runner policy not updated;
|
||||
# root key deactivated per D-034). This verify confirms everything UP TO the apply.
|
||||
set -euo pipefail
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
cd "$ROOT"
|
||||
|
||||
fail() { echo "FAIL: $*" >&2; exit 1; }
|
||||
|
||||
echo "=== Phase 15 verification (partial — terraform apply blocked by IAM) ==="
|
||||
|
||||
# 1. Consumer microservice content
|
||||
[ -f consumer-repos/acdl-consumer-microservice/app.py ] || fail "consumer app.py missing"
|
||||
[ -f consumer-repos/acdl-consumer-microservice/Dockerfile ] || fail "consumer Dockerfile missing"
|
||||
[ -f consumer-repos/acdl-consumer-microservice/README.md ] || fail "consumer README.md missing"
|
||||
grep -q "acdl-microservice" consumer-repos/acdl-consumer-microservice/app.py || fail "app.py: no service name"
|
||||
grep -q "EXPOSE 8080" consumer-repos/acdl-consumer-microservice/Dockerfile || fail "Dockerfile: no EXPOSE 8080"
|
||||
echo "Consumer microservice content: OK (app.py + Dockerfile + README.md)"
|
||||
|
||||
# 2. Docker image built
|
||||
docker images acdl-microservice:latest --format '{{.Repository}}:{{.Tag}}' | grep -q "acdl-microservice:latest" || fail "Docker image acdl-microservice:latest not built"
|
||||
echo "Docker image: OK (acdl-microservice:latest built)"
|
||||
|
||||
# 3. ECR push script
|
||||
[ -f scripts/push_consumer_image.py ] || fail "scripts/push_consumer_image.py missing"
|
||||
python3 -m py_compile scripts/push_consumer_image.py || fail "push_consumer_image.py: py_compile failed"
|
||||
echo "ECR push script: OK (present + compiles)"
|
||||
|
||||
# 4. Contract + resolver + adapter pipeline (up to terraform plan)
|
||||
set -a; . .env.secrets; set +a
|
||||
export AWS_ACCESS_KEY_ID=$ACDL_AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY=$ACDL_AWS_SECRET_ACCESS_KEY AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION:-us-east-1}
|
||||
WORK=/tmp/p15_verify
|
||||
rm -rf "$WORK" terraform/microservice; mkdir -p "$WORK"
|
||||
python3 acdl_platform/contract_resolver.py contracts/microservice.yaml "$WORK/ms_ir.json" 2>/dev/null || fail "resolver failed"
|
||||
python3 adapters/terraform/adapter.py "$WORK/ms_ir.json" terraform/microservice 2>/dev/null || fail "adapter failed"
|
||||
python3 -c "import json; ir=json.load(open('$WORK/ms_ir.json')); assert len(ir['resources'])>=11, f'expected >=11 resources, got {len(ir[\"resources\"])}'" || fail "IR: wrong resource count"
|
||||
echo "Contract -> IR -> adapter: OK (11 resources)"
|
||||
|
||||
# 5. terraform init + validate + plan (the plan succeeds; apply is the IAM-blocked step)
|
||||
cd terraform/microservice
|
||||
terraform init -reconfigure -lock=false -input=false 2>&1 | tail -1
|
||||
terraform validate 2>&1 | grep -q "Success" || fail "terraform validate failed"
|
||||
terraform plan -lock=false -input=false -out=tfplan > /tmp/p15_plan.txt 2>&1
|
||||
grep -q "Plan:" /tmp/p15_plan.txt || { echo "--- plan output ---"; cat /tmp/p15_plan.txt | tail -20; fail "terraform plan failed"; }
|
||||
PLAN_SUMMARY=$(grep "Plan:" /tmp/p15_plan.txt | head -1 | sed 's/\x1b\[[0-9;]*m//g')
|
||||
echo "terraform validate + plan: OK ($PLAN_SUMMARY)"
|
||||
cd "$ROOT"
|
||||
|
||||
# 6. Evidence event written to outbox (TERRAFORM_APPLY_BLOCKED)
|
||||
python3 -c "
|
||||
import boto3, os
|
||||
s = boto3.Session(aws_access_key_id=os.environ['AWS_ACCESS_KEY_ID'], aws_secret_access_key=os.environ['AWS_SECRET_ACCESS_KEY'], region_name=os.environ['AWS_DEFAULT_REGION'])
|
||||
d = s.client('dynamodb')
|
||||
r = d.query(TableName='acdl-outbox', KeyConditionExpression='contractId = :cid', ExpressionAttributeValues={':cid': {'S': '22222222-2222-2222-2222-222222222222'}})
|
||||
items = r.get('Items', [])
|
||||
assert len(items) >= 1, 'no events in outbox for contract 22222222...'
|
||||
assert any('TERRAFORM_APPLY_BLOCKED' in str(item) for item in items), 'no TERRAFORM_APPLY_BLOCKED event in outbox'
|
||||
print(f'outbox: OK ({len(items)} event(s) for contract 22222222...)')
|
||||
" || fail "outbox: no TERRAFORM_APPLY_BLOCKED event"
|
||||
echo "Evidence event: OK (TERRAFORM_APPLY_BLOCKED in DynamoDB outbox)"
|
||||
|
||||
# 7. Adapter fix regression: v1.1 spike still works
|
||||
python3 acdl_platform/contract_resolver.py contracts/spike.yaml "$WORK/spike_ir.json" 2>/dev/null || fail "v1.1 regression: resolver failed"
|
||||
python3 adapters/terraform/adapter.py "$WORK/spike_ir.json" "$WORK/spike_tf" 2>/dev/null || fail "v1.1 regression: adapter failed"
|
||||
grep -q 'resource "aws_s3_bucket" "s3"' "$WORK/spike_tf/main.tf" || fail "v1.1 regression: no aws_s3_bucket"
|
||||
echo "v1.1 regression: OK (spike.yaml -> l1-s3 -> aws_s3_bucket)"
|
||||
|
||||
# 8. .ciagent/ consistency
|
||||
grep -q '"milestone": "v1.2"' .ciagent/config.json || fail "config.json: milestone not v1.2"
|
||||
echo ".ciagent/ consistency: OK"
|
||||
|
||||
echo ""
|
||||
echo "=== Phase 15: PARTIALLY VERIFIED ==="
|
||||
echo "Consumer microservice + Docker image + adapter fixes: DONE."
|
||||
echo "terraform plan succeeds (13 to add)."
|
||||
echo "BLOCKER: terraform apply fails with AccessDenied — live IAM policy not updated."
|
||||
echo "UNBLOCK: operator runs create_iam_user.py with root/admin creds to push the expanded policy."
|
||||
echo "Then re-run terraform apply; Phase 16 will complete the e2e."
|
||||
exit 0
|
||||
Reference in New Issue
Block a user