acdl/.ciagent/RESEARCH.md

# Nova — v1.24 Research Findings

> Phase: research (pre-execution). Milestone: v1.24 (Consumer Guide Accuracy
> & Env-Promotion Lifecycle Enforcement). Status: research.
> Researcher: ci-researcher. Autonomy: full.

## 1. Problem domain

Two distinct problem spaces in one milestone:

### 1a. Consumer guide accuracy (docs)

The consumer guide (`docs/consumer-guide.md`, 477 lines) has 5 accuracy
defects identified in review:

1. **Step 3 contract fields table (lines 141-147)** lists `uses`, `module`,
   `environment`, `inputs`. The actual schema
   (`schemas/contract.schema.json:7`) requires `id`, `name`, `environment`,
   `infrastructure`. The `uses` field was dropped in v1.10.2 (REQ-50
   superseded) and `module` was replaced by the `infrastructure` map key.
   The worked examples (lines 111-137) use the correct fields.

2. **Step 4 caller (lines 173-183)** omits `environment:` in `with:`, while
   Step 2 (lines 94-101) shows `environment: dev`. The two canonical caller
   snippets disagree.

3. **Step 5 stage 8 (lines 232, 253)** says "(dev only)" for the apply
   stage. Higher environments *do* apply — they apply after HITL
   attestation per `docs/environments/index.md:44-54`.

4. **Step 8 (lines 290-306)** says "Change `environment` in your contract"
   to promote, which contradicts the same doc's "Per-environment
   deployment" section (lines 398-402): "you do not edit the `environment:`
   field… Promotion = running the matching job." The test
   `test_consumer_guide_states_no_field_editing` asserts the no-editing
   model.

5. **Reference table (lines 329-330)** says sample contracts "use `@v1.19`"
   but the sample contracts (`contracts/static-assets.yml`,
   `contracts/microservice.yml`) don't carry `uses:` — the version pin
   lives in the caller workflow.

### 1b. Environment-promotion lifecycle enforcement (feat)

**Root cause confirmed by code inspection:**

- `adapters/terraform/adapter.py:129` sets the Terraform state key to:
  `spike/{stack_name}/{environment}/terraform.tfstate`
- `stack_name` = `contract["id"]` (stable across env changes, per
  `core/contract_resolver.py:584`).
- When a consumer edits `environment:` from `dev` → `qa` on the same
  contract `id`, the state key changes from `spike/assets/dev/` to
  `spike/assets/qa/`. Terraform initializes a **fresh state file** in the
  new env's state path. The prior env's resources remain live in AWS with
  their state file untouched. **No destroy ever runs.** This orphans
  resources.

**The user's binding directive:** Editing `environment:` on a stable
`contract.id` is a valid promotion path (Shape A). The platform **must**
destroy the prior env's resources before building the new env. There must
be **no path that orphans resources** — fail closed if the destroy fails.

## 2. Existing codebase structure (integration points)

### DynamoDB `nova-contracts` table (the prior-env source of truth)

- **Table:** `nova-contracts` (env var `CONTRACTS_TABLE`, default
  `nova-contracts`). Defined in `core/lambda/contract_ingestor.py:25`.
- **Schema:** PK `consumerRepo` (S), SK `contractId#submittedAt` (S).
  Attributes: `contractId`, `contract`, `environment`, `status`,
  `submittedAt`.
- **Written by:** `_submit_contract()` at
  `core/lambda/contract_ingestor.py:135-176`. The consumer's deploy
  workflow submits the contract via the Lambda Function URL.
- **Read pattern for env-transition:** Query by PK `consumerRepo` + SK
  begins_with `contractId#` + FilterExpression `status = "submitted"` →
  sort by `submittedAt` desc → take the latest → read its `environment`.
  This is the last-submitted env. For the last-*applied* env, a new
  `#LAST_APPLIED` SK suffix is added (REQ-283).
- **Test pattern:** `tests/test_contract_ingestor.py:74-110`
  (`moto_contracts_table` fixture) uses moto `mock_aws` to create the
  table. The env-transition tests will mirror this pattern.

### Outbox writer (evidence events)

- `core/outbox_writer.py` writes hash-chained evidence events to
  `nova-outbox` table. PK `contractId`, SK `eventType#eventTs`.
- The env-transition destroy step emits a `nova.env.destroyed` event via
  this writer (REQ-284c). Pattern: build an event dict with `contractId`,
  `eventType: "ENV_DESTROYED"`, `environment: <prior_env>`, `ts`, then
  call `write_event()`.

### Contract resolver (environment override)

- `core/contract_resolver.py:460-483` `resolve()` accepts
  `environment_override` — when set, it overrides the contract's
  `environment` field **before** schema validation and interpolation
  (D-088). This is the mechanism the destroy step uses to re-resolve the
  contract against the prior env: `resolve(contract, env_override=prior_env)`.
- Already tested in `tests/test_deploy_workflow_env_input.py:35-52`.

### run_platform.sh (where Step 0b goes)

- `scripts/run_platform.sh` is the platform pipeline. Step 0 (lines 222-239)
  is the environment onboarding check. Step 1 (lines 241-249) is contract
  validation. **Step 0b goes between them** (after onboarding, before
  validation).
- The destroy step mirrors the existing `--destroy` mode (lines 376-394):
  `terraform init -reconfigure` + `terraform destroy -auto-approve`. The
  difference: it runs against the *prior* env's state key, not the current
  one.
- `CONTRACT_ID` is already set at line 217 (`NOVA_CONTRACT_ID` with a
  default). `CONSUMER_REPO` needs to be derived from `GITHUB_REPOSITORY`
  or a new `NOVA_CONSUMER_REPO` env var (REQ-286).

### deploy.yml (consumer repo → platform)

- `.github/workflows/deploy.yml:132` runs
  `bash platform/scripts/run_platform.sh $MODE_FLAG $ENV_FLAG "${{ inputs.contract }}"`.
- To pass `NOVA_CONSUMER_REPO`, add `NOVA_CONSUMER_REPO=${{ github.repository }}`
  as an env var on the "Run the platform pipeline" step (REQ-286).

### Test patterns

- **Shell script assertions:** `tests/test_pipeline.py:79-95` reads the
  script text and asserts substrings. The env-transition tests follow this
  pattern for `run_platform.sh`.
- **DynamoDB mocking:** `tests/test_contract_ingestor.py:74-110` uses moto
  `mock_aws` + `boto3.client` + `create_table`. The env-transition tests
  follow this pattern.
- **Consumer guide assertions:** `tests/test_consumer_guide_per_env_section.py`
  reads `docs/consumer-guide.md` text and asserts substrings. The updated
  tests follow this pattern.

## 3. Persona assessment (v1.24)

This milestone has two distinct work territories:

1. **Docs (P1):** `docs/consumer-guide.md` edits + test updates. This is
   lead-developer territory (narrative/docs + test assertions).
2. **Platform code (P2):** `core/env_transition.py` (new), `scripts/run_platform.sh`
   edits, `.github/workflows/deploy.yml` edit, `adapters/terraform/adapter.py`
   doc comment. This is backend-engineer territory (Python + bash + YAML).
3. **Tests (P3):** `tests/test_env_transition.py` (new), `tests/test_run_platform_env_transition.py`
   (new), `tests/test_consumer_guide_per_env_section.py` updates. Split:
   backend-engineer for the env_transition + pipeline tests; lead-developer
   for the consumer guide test updates.

**Roster:** lead-developer (docs + guide tests) + backend-engineer (Python +
bash + YAML + pipeline tests). frontend-engineer stays deactivated (no UI).
data-engineer not needed (no schema changes — the `nova-contracts` table
already exists with the right shape; we only add a new SK suffix). No new
personas.

## 4. Key decisions logged

| ID | Decision | Confidence | Source |
|----|----------|------------|--------|
| D-201 | Both promotion shapes supported (A: edit+destroy, B: per-env callers) | 0.95 | User directive |
| D-202 | Prior-env source of truth = `nova-contracts` DynamoDB table | 0.85 | User directive + code inspection |
| D-203 | Detect-and-destroy at pipeline start (Step 0b) | 0.85 | User directive |
| D-204 | Fail closed on destroy failure (no orphan path) | 0.95 | User directive |
| D-205 | Cross-account destroy out of scope (same-account only) | 0.80 | CLARIFY A3 |
| D-206 | Env-transition destroy is NOT the HITL decommission pipeline | 0.85 | CLARIFY A5 |
| D-207 | Last-applied env recorded via `#LAST_APPLIED` SK in `nova-contracts` | 0.85 | RESEARCH §2 |
| D-208 | State key `spike/{id}/{env}/` stays as-is (correct for both shapes) | 0.90 | RESEARCH §1b |

## 5. Pitfalls

1. **Destroy needs the prior env's Terraform config, not the new env's.**
   The destroy step must re-resolve the contract with
   `environment_override=prior_env` so the emitted TF matches the prior
   env's resources. If we resolve with the new env, the destroy plan won't
   match the prior state → terraform tries to create, not destroy.
2. **`terraform init -reconfigure` is required** when switching state
   backends between envs (if envs use different state buckets). The
   `-reconfigure` flag tells Terraform to forget the previous backend config.
3. **The `deletion_protection` NFR (REQ-86) blocks destroy.** The destroy
   step must resolve with `deletion_protection: false` injected (same as
   decommission Step 2 in `scripts/run_decommission.sh:34-37`). Without
   this, `terraform destroy` fails on `prevent_destroy` lifecycle blocks.
4. **DynamoDB may not be reachable in local/CI mode.** The detect step
   must handle `ClientError` / `EndpointNotFound` gracefully → log warning
   + return `None` (conservative). This is documented in REQ-282.
5. **The consumer guide test `test_consumer_guide_states_no_field_editing`
   will fail after the Step 8 rewrite.** It must be updated in the same
   phase as the guide edit (P1) or the test suite breaks.