diff --git a/.ciagent/PERSONAS.md b/.ciagent/PERSONAS.md index 6768169..7e766a9 100644 --- a/.ciagent/PERSONAS.md +++ b/.ciagent/PERSONAS.md @@ -1,7 +1,7 @@ --- project: acdl -milestone: v1.8 -generated_at: 2026-07-22 +milestone: v1.9 +generated_at: 2026-07-23 generator: lead-developer verification_toolchain: typecheck: "terraform validate && python3 -m py_compile core/**/*.py && python3 -m jsonschema schemas/*.schema.json" @@ -16,7 +16,7 @@ verification_toolchain: ci-* agents read PERSONAS.md before running verification commands. --- -# ACDL — Persona Roster (project-level, v1.8) +# ACDL — Persona Roster (project-level, v1.9) ## Active personas @@ -56,14 +56,14 @@ verification_toolchain: - **Territory:** `core/hitl_matrix_design.md`, `core/audit_ledger_design.md`, `adapters/terraform/policy/**` (Checkov adapter + custom rules), `adapters/wiz/**` (Wiz adapter), `adapters/kyverno/**` (Kyverno adapter + sample policies), `core/separation_of_duties.py`, `schemas/tagging-standard.json`, `schemas/policy_check_result.schema.json` (engine enum) - **Reason:** Owns the HITL matrix design, separation-of-duties, the audit ledger design, the Checkov→PolicyCheckResult adapter + the custom tagging rule (D-054, D-043 closure), the Wiz adapter (D-052), the Kyverno adapter (D-053), and the tagging standard. Enforces the "Safety is Computed, Not Assumed" + "Audit truth lives outside the repository" vision tenets. -### lambda-engineer (custom, v1.8) +### lambda-engineer (custom, v1.9) - **Domain:** serverless - **Active:** true -- **Phase-specific:** true (reactivated for v1.8; removed after milestone COMPLETE) -- **Frameworks:** python, aws-lambda, boto3, dynamodb, aws-secretsmanager, github-api, gitea-api -- **Constraints:** lambda-is-stateless, dynamodb-is-the-state-store, secrets-from-secrets-manager-never-logged, idempotent-actions, cross-account-iam-via-abac, forge-agnostic-api-urls -- **Territory:** `core/lambda/**` (contract_ingestor.py + handler), `terraform/platform/main.tf` (Lambda + Function URL + DynamoDB + KMS + Secrets Manager + IAM + acdl-change-requests table), `terraform/platform/consumer_invoke_policy.json`, `terraform/platform/variables.tf` -- **Reason:** Reactivated for v1.8 Phase 29 (Terraform-rendered invoke policy), Phase 30 (forge-agnostic API URLs in contract_ingestor.py), Phase 34 (validate_change_request Lambda action + acdl-change-requests DynamoDB table). The Lambda is stateless; all state is in DynamoDB. Forge-agnostic API URLs (GitHub + Gitea) via GITHUB_API_BASE env var. Removed from the roster after milestone COMPLETE (the code persists, but the persona is no longer active). +- **Phase-specific:** true (reactivated for v1.9; removed after milestone COMPLETE) +- **Frameworks:** python, aws-lambda, boto3, dynamodb, aws-secretsmanager, aws-sns, github-api, gitea-api +- **Constraints:** lambda-is-stateless, dynamodb-is-the-state-store, secrets-from-secrets-manager-never-logged, idempotent-actions, cross-account-iam-via-abac, forge-agnostic-api-urls, sns-topic-arn-from-env +- **Territory:** `core/lambda/**` (contract_ingestor.py + handler), `terraform/platform/main.tf` (Lambda + Function URL + DynamoDB + KMS + Secrets Manager + IAM + acdl-change-requests table + acdl-sod-halt SNS topic), `terraform/platform/consumer_invoke_policy.json`, `terraform/platform/variables.tf` +- **Reason:** Reactivated for v1.9 Phase 42 (acdl-sod-halt SNS topic for `route_halt_artifact`, defined in `terraform/platform/main.tf`). The Lambda is stateless; all state is in DynamoDB. Forge-agnostic API URLs (GitHub + Gitea) via GITHUB_API_BASE env var. Removed from the roster after milestone COMPLETE (the code persists, but the persona is no longer active). ### frontend-engineer - **Domain:** frontend @@ -107,18 +107,23 @@ verification_toolchain: | 36 schemas-adapters-pipelines-readmes | lead-developer (lead: 3 READMEs), backend-engineer (pipelines + schemas README content), platform-engineer (adapters README content) | security/lambda/frontend idle | | 37 verify | lead-developer (lead: 4-layer verification), all personas (review their territory) | — | | 38 review-audit-complete | lead-developer (lead: review + audit + milestone completion), all personas (review participation) | — | +| 39 design-doc-refresh-and-p1-1-parameterization | security-engineer (lead: hitl_matrix_design.md + audit_ledger_design.md refresh), platform-engineer (lead: P1-1 adapter defaults → L1 interface.json inputs), backend-engineer (contract_resolver.py + env schema adjacent review) | lambda/frontend idle | +| 40 contract-interpolation | backend-engineer (lead: _expand_vars in contract_resolver.py + environment.schema.json + sample contracts), platform-engineer (interface.json adjacent review) | security/lambda/frontend idle | +| 41 per-environment-ci-jobs | backend-engineer (lead: deploy.yml environment input + run_platform.sh --environment + per-env contracts + caller-workflow docs), security-engineer (HITL gate structure review) | platform/lambda/frontend idle | +| 42 stub-implementation | security-engineer (lead: route_halt_artifact SNS + hitl_gates.py + attestation_matrix.py + Wiz real client + Kyverno fleshed out), backend-engineer (run_platform.sh HITL gate wiring), lambda-engineer (acdl-sod-halt SNS topic in terraform/platform/main.tf) | platform/frontend idle | +| 43 verify-review-audit-complete | lead-developer (lead: 4-layer verify + review + audit + milestone completion), all personas (review participation) | — | ## Domain priority (used by TaskDecomposer) `coordination → security → platform → backend → lambda → frontend` -Rationale: in v1.8, the security commitments (encryption by default, -KMS rotation, deletion protection, CMDB validation, HITL SRE gates) -and the platform commitments (kms-key primitive, uptime primitive, -adapter expansion, prevent_destroy emission) are the binding -constraints; backend wires the pipeline + decommission mode + API -portability; lambda owns the CMDB validation + forge-agnostic APIs; -frontend is unchanged from v1.0 (evidence timeline). +Rationale: in v1.9, the security commitments (HITL gates, attestation +matrix, SoD halt artifact, Wiz/Kyverno adapters) and the design-doc +accuracy are the binding constraints; platform owns the P1-1 adapter +parameterization + L1 interface inputs; backend owns the contract +interpolation + per-env CI jobs + the deploy workflow env input; +lambda owns the SNS topic Terraform; frontend is unchanged from v1.0 +(evidence timeline). ## Conflict resolutions (lead-developer arbitration) diff --git a/.ciagent/RESEARCH.md b/.ciagent/RESEARCH.md index cf36d9b..a13c75b 100644 --- a/.ciagent/RESEARCH.md +++ b/.ciagent/RESEARCH.md @@ -1721,4 +1721,195 @@ be codified into standards. Key patterns identified:** --- -*End of RESEARCH.md. Path: `/root/acdl/.ciagent/RESEARCH.md`.* \ No newline at end of file +## v1.9 Research Addendum (Phase 0, 2026-07-23) + +> Milestone v1.9. Researcher: lead-developer. Autonomy: full. The v1.9 +> scope is well-grounded in the existing codebase; the research is a +> focused addendum covering the four new implementation domains +> (interpolation, per-env workflow inputs, Wiz GraphQL, attestation +> matrix freshness validation) + the design-doc drift audit. + +### RA-1 — Contract interpolation prior art + syntax choice (D-081) + +**Finding:** Variable expansion in declarative manifests is a solved +pattern. Terraform uses `${var.x}` / `${local.x}`; Helm uses `{{ .Values.x }}`; +GitHub Actions uses `${{ }}`; CloudFormation uses `!Ref` / `!Sub`. The +contract schema is YAML validated by `jsonschema` — the schema does not +inspect string *contents*, so any token syntax is schema-safe. + +**Choice:** `${env.}` + `${contract.}` (D-081). Rationale: +- Shell-style `${...}` is the most familiar to the platform's audience + (DevOps engineers comfortable with Terraform/HCL). +- Dotted paths (`${env.state_backend.bucket}`) mirror Python attribute + access and the existing `wire["from"]` syntax (`contract.inputs.x`, + `.outputs.y`). +- No conflict with YAML (`${}` inside a YAML string is a literal until + the resolver expands it) or with `jsonschema` (string content is not + schema-constrained). +- Jinja `{{ }}` was considered (supports future filters) but rejected — + the contract is a data file, not a template; filters would invite + logic-in-config anti-patterns. + +**Implementation shape:** a single `_expand_vars(value, context)` +recursive walker in `core/contract_resolver.py`. Context = +`{"env": , "contract": }`. Unknown +token → `ValueError` with the token text (fail loud, no silent +passthrough — consistent with the P1-3 SSM fail-loud precedent). + +**Confidence:** 0.92. Risk: none — the expansion is post-schema-validation +and pre-IR-resolution, so it cannot break the schema or the adapter. + +### RA-2 — GitHub Actions `workflow_call` `environment` input + per-env jobs (D-082) + +**Finding:** GitHub Actions `workflow_call` inputs support `type: string` +with no enum constraint at the workflow-call layer (enum constraints +exist only for `choice`-typed *workflow_dispatch* inputs). The deploy +workflow already uses `workflow_call` with `contract` + `mode` + +`changeRequestId` string inputs. Adding an `environment` string input +(default empty, validated by `run_platform.sh`) is a one-line addition. + +**Per-env job pattern:** the consumer repo's *caller* workflow +(`.github/workflows/deploy-.yml`) does: +```yaml +jobs: + deploy-qa: + uses: acdl/.github/workflows/deploy.yml@v1.9 + with: + environment: qa + contract: .acdl/static-assets.qa.yaml +``` +One caller workflow per environment = one CI job per environment. The +`environment:` field in the contract is not edited for promotion; +promotion = running the qa caller. The hybrid model (D-082) also lets a +single contract be promoted via the `environment` input alone. + +**Gitea caveat:** Gitea Actions supports `workflow_call` (reuses the +GitHub Actions workflow YAML). The `environment` input works identically. +Gitea has no Environments API (D-013/D-042) — the HITL gate is the +`workflow_dispatch` approval-input fallback (already documented in +`hitl_matrix_design.md`). For `workflow_call` (reusable), the caller +workflow's `workflow_dispatch` trigger carries the approval input. + +**Confidence:** 0.90. Risk: the Gitea `workflow_call` + approval-input +combination needs the caller to be `workflow_dispatch`-triggered (not +`workflow_call`-triggered) for the gate to fire — documented in Phase 41. + +### RA-3 — Wiz GraphQL API shape (D-0xx, REQ-110) + +**Finding:** Wiz exposes a GraphQL API at `/graphql`. Auth += `Authorization: Bearer `. The primary query for issues: +```graphql +query IssuesQuery($filterBy: IssueFilter) { + issues(filterBy: $filterBy) { + nodes { id severity title entity { name type } control { name } + createdAt } + pageInfo { hasNextPage endCursor } + } +} +``` +Wiz severity enum: `CRITICAL | HIGH | MEDIUM | LOW | INFORMATIONAL`. +Mapping to `PolicyCheckResult`: +- `engine: "wiz"` +- `ruleId: ` (or `WIZ_` fallback) +- `severity: ` +- `status: FAIL` (Wiz issues are findings; pass = no issues returned) +- `message: ` +- `resource: <entity.name>` + +**Graceful degrade:** when `WIZ_API_TOKEN` or `WIZ_API_URL` unset → emit +the existing single `SKIPPED` `WIZ_NOT_CONFIGURED` record (no network +call). Offline tests use a recorded JSON fixture (no live Wiz tenant). + +**Confidence:** 0.80. Risk: Wiz API version drift — the query shape is +stable as of Wiz API v2 (2026), but the fixture is the test's source of +truth, not the live API. + +### RA-4 — Attestation matrix freshness validation (D-084, REQ-109) + +**Finding:** The 8 concerns in `hitl_matrix_design.md` §10.4 have +declared freshness windows (24h, 7d, 30d, 90d, 180d). Operator-supplied +evidence (load test, DR drill, FinOps forecast, runbook) is uploaded as +a signed blob. The matrix validates: +1. **Presence** — the evidence artifact exists for the target env. +2. **Freshness** — `artifact.timestamp` is within the declared window. +3. **Schema** — the artifact matches a per-concern JSON schema (e.g. + load-test artifact has `p99_latency`, `throughput`, `pass_rate`). +4. **Signature** (when `ACDL_ATTESTATION_SIGNING_KEY_ID` set) — JWS + detached signature verification against a platform KMS key. When + unset (dev/CI), signature verification is skipped (offline-testable). + +**Offline-testable concerns** (run for real, no operator input): +- Contract NFRs (the platform's own contract validator). +- Schema validity (jsonschema). +- Policy pass (Checkov/Wiz/Kyverno `PolicyCheckResult` records). + +**Operator-supplied concerns** (require uploaded artifact): +- Functional correctness (e2e suite report). +- Performance baseline (k6/Gatling report). +- Security posture (Trivy/Snyk scan + Security signature). +- Operational readiness (runbook/dashboard/oncall/alerts). +- Incident response (Sev-1 drill record). +- Capacity/cost (FinOps forecast). +- Resilience (DR drill, chaos report, backup verification). +- dr-region deploy (dr drill report). + +**Confidence:** 0.88. Risk: the signature verification path is only +exercised when a signing key is configured (dev/CI skips it); production +deployment must set `ACDL_ATTESTATION_SIGNING_KEY_ID`. + +### RA-5 — Design doc drift audit (REQ-100, REQ-101) + +**`core/hitl_matrix_design.md` drift:** +- Status block says "v1.2 wires the gates" — stale (v1.9 wires them). +- "Spike scope note" says "the spike is dev-only; HITL is not exercised" + — stale (v1.9 exercises qa/prod/dr). +- §10.4 matrix is presented as design-only — v1.9 implements the + offline-testable subset (D-084). +- D-042 approver-identity mechanics are still accurate (Gitea has no + Environments API; `gitea.actor` is the approver of record). + +**`core/audit_ledger_design.md` drift:** +- "Spike scope (D-041)" says "Phases 08-10 implement" — stale (the + outbox is shipped + production since v1.8). +- "v1.2 build-out" (S3 Object Lock + JWS + worker + DLQ + checkpoints) + never shipped; v1.9 defers it explicitly (D-083). +- The outbox item shape is still accurate; the `approver_qa`/ + `approver_prod` attributes are populated by v1.9's `hitl_gates.attest`. + +**Confidence:** 0.95. Risk: none — doc-only. + +### RA-6 — P1-1 adapter defaults audit (D-085, REQ-102) + +**Hardcoded defaults found in `adapters/terraform/adapter.py`:** +- `desired_count = 1` (ECS service, 2 occurrences: line 238, 481). +- `launch_type = "FARGATE"` (ECS service, line 239, 482). +- `family = "app"` (task def, line 254 — reads `inputs.get("family", "app")` + so partially parameterized; the `"app"` default should move to the + interface). +- `target_type = "ip"` (ALB target group, line 274). +- `load_balancer_type = "application"` (ALB, line 272). +- `Name = "acdl-microservice-rt"` (route table, line 283) + `Name = ...` + tags on VPC/IGW (lines 515, 542 `name = "app"`). + +**Fix:** add `desired_count`, `launch_type`, `family`, `target_type`, +`load_balancer_type`, `name` (VPC/IGW/RT) to the corresponding L1 +`interface.json` `inputs` with defaults. The adapter reads +`inputs.get("<name>", <default>)` — but the resolver should populate +the default from the interface so the adapter reads `inputs["<name>"]` +with a fallback only for safety. Tests assert an override emits the +overridden value. + +**Confidence:** 0.90. Risk: low — the v1.1 S3 regression test must +still pass (S3 has none of these inputs). + +### Decisions surfaced (v1.9) + +| ID | Decision | Rationale | Confidence | Alternatives | +|----|----------|-----------|------------|--------------| +| **D-087** | Interpolation expansion is recursive over dicts + lists + strings (not just top-level inputs). | A nested input like `env: { DATABASE_URL: "acdl-${env.environment}-db" }` should expand too. | 0.90 | Top-level only (misses nested maps). | +| **D-088** | The `environment` workflow_call input overrides the contract's `environment` field *before* schema validation, so the schema sees the overridden value. | Interpolation context depends on the resolved environment; override must happen pre-validation so `${env.environment}` is consistent. | 0.92 | Override post-validation (inconsistent interpolation context). | +| **D-089** | Attestation artifact signature verification is skipped when `ACDL_ATTESTATION_SIGNING_KEY_ID` is unset (dev/CI); required for prod/dr. | Offline tests cannot sign with a real KMS key. The skip is explicit + logged. | 0.85 | Always require signature (breaks offline tests). | + +--- + +*End of RESEARCH.md v1.9 addendum.* \ No newline at end of file