docs(P59): create phase plan — L1 module lifecycle pipeline (author)
PLAN stage. P59 authors the modules-lifecycle pipeline that matrix-tests
every L1 module's examples/{simple,complex}.yml contracts through
apply→modify→destroy against live AWS. No per-module Python.
5 tasks: declarative contract, byte-identical Gitea+GitHub workflows,
schema, tests, README update.
---ci---
project: acdl
phase: P59
milestone: v1.11
status: plan
---/ci---
This commit is contained in:
+67
-60
@@ -1,84 +1,91 @@
|
||||
---
|
||||
phase: P58
|
||||
name: single-platform-vpc-deterministic-state
|
||||
phase: P59
|
||||
name: l1-lifecycle-pipeline-author
|
||||
milestone: v1.11
|
||||
requirements: [REQ-126]
|
||||
wave: 1
|
||||
depends_on: [P57]
|
||||
requirements: [REQ-127]
|
||||
wave: 2
|
||||
depends_on: [P58]
|
||||
---
|
||||
|
||||
# P58 — Single Platform VPC + Deterministic State Keys
|
||||
# P59 — L1 Module Lifecycle Pipeline (Author)
|
||||
|
||||
**Phase:** P58
|
||||
**Phase:** P59
|
||||
**Milestone:** v1.11 (RESTART)
|
||||
**Requirement:** REQ-126
|
||||
**Wave:** 1 (foundation — the 4-VPC bug fix)
|
||||
**Branch:** `milestone/v1.11-restart` → `phase/p58-platform-vpc-state`
|
||||
**Requirement:** REQ-127
|
||||
**Wave:** 2 (the testing milestone — author the pipeline, P60 runs it live)
|
||||
**Branch:** `milestone/v1.11-restart` → `phase/p59-l1-lifecycle-pipeline`
|
||||
|
||||
## Goal
|
||||
|
||||
1. Add a single shared VPC to `terraform/platform/main.tf` (VPC + 2 subnets +
|
||||
IGW + route table + ECS security group). Output vpc_id, subnet_ids,
|
||||
ecs_security_group_id.
|
||||
2. Drop the `vpc` child from the microservice composition; reference the
|
||||
platform VPC via a `data_sources` block. The alb + service children wire
|
||||
to the platform VPC, not an inline VPC.
|
||||
3. Fix the state key to be env-aware: `spike/{stack_name}/{environment}/terraform.tfstate`.
|
||||
Same contract+env always hits the same state key → terraform modifies,
|
||||
never duplicates.
|
||||
4. The adapter emits `data "terraform_remote_state" "platform"` + `data "aws_vpc"`
|
||||
blocks when a stack references platform-owned resources.
|
||||
Author the `modules-lifecycle` pipeline that matrix-tests every L1 module's
|
||||
lifecycle via its own `examples/{simple,complex}.yml` contracts through
|
||||
apply→modify→destroy against live AWS. No per-module Python. The "test" =
|
||||
the pipeline cell going green.
|
||||
|
||||
## Tasks (single wave)
|
||||
|
||||
### Task 1 — Platform VPC in terraform/platform/main.tf (data-engineer)
|
||||
### Task 1 — pipelines/modules-lifecycle.yml (declarative contract)
|
||||
|
||||
Add to `terraform/platform/main.tf`:
|
||||
- `aws_vpc.acdl_shared` (10.0.0.0/16)
|
||||
- 2 `aws_subnet` across 2 AZs (10.0.1.0/24, 10.0.2.0/24)
|
||||
- `aws_internet_gateway` + `aws_route_table` + associations
|
||||
- `aws_security_group.ecs` (ingress on port 80, egress all)
|
||||
- Outputs: `vpc_id`, `subnet_ids` (comma-separated), `ecs_security_group_id`
|
||||
Author the declarative pipeline contract:
|
||||
```yaml
|
||||
name: acdl-modules-lifecycle
|
||||
triggers:
|
||||
pull_request: [main]
|
||||
workflow_dispatch: []
|
||||
runner: ubuntu-latest
|
||||
python_version: "3.12"
|
||||
stages:
|
||||
- name: validate-contract
|
||||
command: python3 -c "import yaml,json; c=yaml.safe_load(open('$CONTRACT')); assert c['id']"
|
||||
required: true
|
||||
- name: resolve-stack
|
||||
command: python3 core/contract_resolver.py "$CONTRACT" /tmp/lifecycle-stack.json
|
||||
required: true
|
||||
- name: terraform-apply
|
||||
command: bash scripts/run_platform.sh --apply "$CONTRACT"
|
||||
required: true
|
||||
- name: terraform-modify
|
||||
command: bash scripts/run_platform.sh --apply "$CONTRACT_COMPLEX"
|
||||
required: true
|
||||
- name: terraform-destroy
|
||||
command: bash scripts/run_platform.sh --destroy "$CONTRACT_COMPLEX"
|
||||
required: true
|
||||
```
|
||||
|
||||
### Task 2 — Microservice composition: drop vpc, add data_sources (backend-engineer)
|
||||
### Task 2 — .gitea/workflows/modules-lifecycle.yml + .github/workflows/modules-lifecycle.yml
|
||||
|
||||
Update `modules/l2/microservice/composition.json`:
|
||||
- Remove `{"id": "vpc", "module": "vpc@1.0.0"}` from children.
|
||||
- Add a `data_sources` block: `{"name": "platform_vpc", "type": "terraform_remote_state", "source": "platform"}`.
|
||||
- Update wires: `vpc.outputs.subnet_ids` → `platform_vpc.outputs.subnet_ids`.
|
||||
- Add a wire for `platform_vpc.outputs.vpc_id` → `alb.inputs.vpc_id`.
|
||||
- Add a wire for `platform_vpc.outputs.ecs_security_group_id` → `service.inputs.security_group`.
|
||||
Byte-identical workflows. Matrix over 12 L1 modules × {simple, complex}.
|
||||
Each cell:
|
||||
- Checkout + Python 3.12 + Terraform 1.9.* + AWS creds (from secrets).
|
||||
- `run_platform.sh --apply modules/l1/<module>/examples/simple.yml`
|
||||
- `run_platform.sh --apply modules/l1/<module>/examples/complex.yml` (same state key → terraform modifies)
|
||||
- `run_platform.sh --destroy modules/l1/<module>/examples/complex.yml`
|
||||
|
||||
### Task 3 — Contract resolver: pass environment + handle data_sources (backend-engineer)
|
||||
VPC-dependent L1s (alb, ecs-service, rds, uptime) reference the platform VPC
|
||||
applied by a prerequisite job; standalone vpc L1 applies its own. Non-VPC
|
||||
L1s (s3, kms-key, ecr, iam-role, ecs-cluster, cloudfront, waf) run independent.
|
||||
|
||||
Update `core/contract_resolver.py`:
|
||||
- Add `"environment": env_name` to the stack instance's `stack` block (line 568-572).
|
||||
- Handle `data_sources` in the composition: resolve data source references to
|
||||
stack-level pseudo-resources that the adapter can emit as `data` blocks.
|
||||
### Task 3 — schemas/modules-lifecycle-pipeline.schema.json
|
||||
|
||||
### Task 4 — Adapter: env-aware state key + data block emission (backend-engineer)
|
||||
Schema for the new pipeline shape (extends pipeline.schema.json with
|
||||
workflow_dispatch trigger + matrix stages).
|
||||
|
||||
Update `adapters/terraform/adapter.py`:
|
||||
- State key: `spike/{stack_name}/{environment}/terraform.tfstate` (read
|
||||
environment from `stack_instance["stack"]["environment"]`).
|
||||
- When a resource input references a platform data source (e.g.
|
||||
`ref:platform_vpc.subnet_ids`), emit a `data "terraform_remote_state" "platform"`
|
||||
block + pass the data source output as the module argument.
|
||||
### Task 4 — Extend tests/test_pipeline_contract.py
|
||||
|
||||
### Task 5 — Tests + offline regression
|
||||
Add a TestModulesLifecyclePipeline class:
|
||||
- Schema validates.
|
||||
- Gitea + GitHub workflows are byte-identical.
|
||||
- Matrix lists all 12 L1 modules.
|
||||
- Pipeline contract validates against the new schema.
|
||||
|
||||
- Update `tests/test_adapter.py`: assert the state key is env-aware.
|
||||
- Update `tests/test_pipeline.py`: the microservice contract now resolves
|
||||
without a vpc resource.
|
||||
- `pytest tests/ -m "not slow"` — all pass.
|
||||
- `run_platform.sh --check-only contracts/microservice.yml` — exits 0.
|
||||
### Task 5 — Extend pipelines/README.md
|
||||
|
||||
Add the modules-lifecycle pipeline to the table.
|
||||
|
||||
## Success Criteria (phase gate)
|
||||
|
||||
1. `terraform/platform/main.tf` has a VPC + subnets + IGW + route table + SG.
|
||||
2. `contracts/microservice.yml` resolution produces NO `aws:ec2:vpc` resource.
|
||||
3. Two contract applies (dev + prod) → ONE VPC, two state keys
|
||||
(`spike/msvc/dev/`, `spike/msvc/prod/`), two ECS services.
|
||||
4. Same contract+env re-applied → same state key → terraform modifies, never duplicates.
|
||||
5. `pytest tests/ -m "not slow"` — all pass.
|
||||
6. `run_platform.sh --check-only contracts/microservice.yml` — exits 0.
|
||||
1. `pipelines/modules-lifecycle.yml` validates against its schema.
|
||||
2. `.gitea/workflows/modules-lifecycle.yml` and `.github/workflows/modules-lifecycle.yml` are byte-identical.
|
||||
3. Matrix lists all 12 L1 modules.
|
||||
4. `test_pipeline_contract.py` passes (offline).
|
||||
5. Full offline pytest suite green.
|
||||
Reference in New Issue
Block a user