diff --git a/.ciagent/CHECKPOINT.json b/.ciagent/CHECKPOINT.json index 55c1c6b..2611cb3 100644 --- a/.ciagent/CHECKPOINT.json +++ b/.ciagent/CHECKPOINT.json @@ -1,10 +1,10 @@ { - "phase": 0, - "stage": "plan", + "phase": 1, + "stage": "execute", "milestone": "v0.2", - "phase_role": "pre_execution", + "phase_role": "execution", "project": "atelier", "attempts": 0, - "updated_at": "2026-08-05T01:30:00Z", + "updated_at": "2026-08-05T01:40:00Z", "milestone_complete": false } \ No newline at end of file diff --git a/domains/infrastructure-as-code/first-principles.md b/domains/infrastructure-as-code/first-principles.md new file mode 100644 index 0000000..adac564 --- /dev/null +++ b/domains/infrastructure-as-code/first-principles.md @@ -0,0 +1,55 @@ +# Infrastructure as Code — First Principles + +## 1. The Principles + +### P1. Declarative Intent +Describe the desired state, not the steps to reach it. The tool +reconciles current → desired. Imperative scripts describe how; +declarative config describes what. + +### P2. Idempotence +Applying the same configuration twice yields the same result. A +second `apply` with no changes is a no-op, not an error. Idempotence +is what makes `plan` trustworthy. + +### P3. State is Truth +The state file is the authoritative record of what the tool believes +exists. Drift between state and reality is a bug to be reconciled, +not tolerated. Lose state, lose the ability to reason about +infrastructure. + +### P4. Plan Before Apply +Preview every change before mutating real infrastructure. `plan` is +the contract review; `apply` is the signature. No `apply` without a +read `plan`. The plan is the rollback rehearsal. + +### P5. Version Everything +Configuration, state, providers, and modules are all versioned and +reproducible. A commit pins a complete, rebuildable world. Pin +providers; pin module sources; never `latest`. + +### P6. Modules Compose +Encapsulate repeatable patterns as versioned modules. Compose +modules; do not copy them. A module is the unit of reuse, review, +and versioning — the IaC expression of composition. + +### P7. Least Privilege Providers +Provider credentials are scoped to the minimum needed for the +declared resources. No account-wide admin keys in CI. One credential +per environment, per boundary. + +### P8. Remote State with Locking +State is stored remotely with locking. Local state is for a single +developer on a throwaway sandbox. Concurrent `apply` without a lock +is data corruption waiting to happen. + +### P9. Drift is Recoverable +`plan` detects drift; `apply` reconciles it. Manual mutation of +managed infrastructure is an incident, not a shortcut. Drift is +expected; unreconciled drift is the bug. + +### P10. Secrets Never in Code +Secrets come from providers, external secret stores, or environment +variables — never hardcoded in HCL, never committed to the repo, +never written to state in plaintext. State is a secret-bearing +artifact; treat it accordingly. \ No newline at end of file diff --git a/domains/infrastructure-as-code/modules.md b/domains/infrastructure-as-code/modules.md new file mode 100644 index 0000000..aeb413f --- /dev/null +++ b/domains/infrastructure-as-code/modules.md @@ -0,0 +1,64 @@ +# Modules — Derived Rules + +> Derives from `domains/infrastructure-as-code/first-principles.md`. P6 (Modules Compose) lives here. Referenced by `terraform.md` and `opentofu.md`. + +## Why Modules (P6 Modules Compose) + +- A module is the unit of reuse, review, and versioning in IaC. It encapsulates a repeatable pattern behind a typed interface. +- Composition — building large from small — is the IaC expression of core C6 Composability. Without modules, every stack is a one-off; with modules, a stack is an assembly of reviewed parts. +- A good module has one job (a VPC, a database, a load balancer), a small typed surface, and no hidden side effects. + +## Module Structure (P1 Declarative Intent, C2 Clarity) + +- The conventional layout: `main.tf` (resources), `variables.tf` (inputs), `outputs.tf` (outputs), `versions.tf` (provider/version pins). A `README.md` is required for any published module. +- Inputs are typed and validated: `variable "name" { type = string, description = "...", validation { ... } }`. The description is the contract. +- Outputs are the module's interface to consumers. Mark sensitive outputs `sensitive = true`. Document non-obvious outputs in the description. +- A module does not declare a provider configuration unless it owns the provider. Most modules declare only `required_providers` (the constraint) and let the consumer configure the provider. + +## Versioning (P5 Version Everything) + +- Modules are versioned. The registry expects SemVer tags (`v1.0.0`). A consumer pins to a version or a range (`~> 1.0`). +- A breaking change bumps the major. An additive change bumps the minor. A fix bumps the patch. No silent breaking changes within a minor. +- Tag the module repo; the tag IS the version. Never `source = "git::...?ref=main"` in prod — unversioned modules drift. + +## Source Patterns (P5 Version Everything) + +| Source | When | Risk | +|--------|------|------| +| Registry (`//`) | Public, versioned, signed | Verify the publisher; pin the version | +| Git (`git::https://...?ref=v1.0.0`) | Private modules across repos | Pin to a tag, not a branch | +| Local (`./modules/networking`) | Monorepo, single repo | Re-reviewed on every change; no independent version | +| Inline (no module) | Trivial one-off | Becomes a copy-paste anti-pattern at scale | + +- Local modules in a monorepo are fine — they trade independent versioning for co-evolution. The boundary is the review unit: if the module and the consumer always change together, local is correct. +- Cross-repo modules must be versioned via git tags; unversioned cross-repo modules are the worst case (drift without a version to pin). + +## The Module-vs-Copy Boundary (P6 Modules Compose) + +- If a block is used more than once, it is a module. If it is used once and will never be reused, inline is acceptable. +- If two copies differ in one attribute, that is a module with a variable, not two copies. The variable is the difference; the shared body is the module. +- If you find yourself copy-pasting a block and editing it, stop. The edit is a variable. The copy is a module call. +- A module that has grown to do many jobs should be split. A module with 20 variables is two modules. + +## Composition (P6 Modules Compose, C6 Composability) + +- Compose by calling modules from a root configuration: `module "vpc" { source = "...", version = "..." }`. The root is the assembly; the modules are the parts. +- Outputs of one module feed inputs of another: `module "app" { vpc_id = module.vpc.vpc_id }`. This is the composition edge. +- Avoid hidden coupling: a module should not reach into another module's state. If two modules must share state, promote the shared concern to the root or a parent module. + +## Reviewing Modules (P4 Plan Before Apply) + +- A module is reviewed once, at its version. Consumers trust the version pin. A module change requires a new version and a review of the diff. +- When a module changes, every consumer that bumps the version gets the change. Treat a module version bump as a real change: review the module diff, run the consumer's `plan`. +- A module with a breaking change must not auto-bump in consumers. Pin consumers to the old major until they explicitly migrate. + +## What Violates Module Discipline + +| Violation | Principle | +|-----------|-----------| +| Copy-pasted block with a one-line difference | P6 Modules Compose | +| `source = "git::...?ref=main"` in prod | P5 Version Everything | +| Module with 20 variables | P6 Modules Compose (split it) | +| Silent breaking change within a minor | P5 Version Everything | +| Module reaching into another module's state | C6 Composability, P1 Declarative Intent | +| Unpublished module with no README | C2 Clarity | \ No newline at end of file diff --git a/domains/infrastructure-as-code/opentofu.md b/domains/infrastructure-as-code/opentofu.md new file mode 100644 index 0000000..f100fb3 --- /dev/null +++ b/domains/infrastructure-as-code/opentofu.md @@ -0,0 +1,50 @@ +# OpenTofu — Derived Rules + +> Derives from `domains/infrastructure-as-code/first-principles.md`. OpenTofu is the open-source fork of Terraform; this doc covers fork-specific governance, license, and migration. The shared HCL/state/module model is documented in `terraform.md`. See also `state.md` and `modules.md`. + +## Fork Lineage (P5 Version Everything) + +- OpenTofu is a 2023 fork of Terraform, created when HashiCorp switched Terraform from MPL-2.0 to the Business Source License (BUSL), which is not open source. +- OpenTofu is stewarded by the Linux Foundation under a genuinely open-source license. The fork's reason for existing is license neutrality. +- Both tools implement the same HCL configuration language, the same provider protocol, and the same state model. Configuration written for one runs on the other at the fork point; divergence accrues slowly over time. + +## When to Choose OpenTofu (P7 Least Privilege Providers, supply-chain) + +- **License neutrality matters:** if your organization cannot accept BUSL's "competitive use" ambiguity, OpenTofu removes it. +- **Supply-chain provenance:** Linux Foundation stewardship means no single vendor can relicense the tool out from under you. +- **Community governance:** features and fixes are accepted on merit, not vendor strategy. +- **When NOT to switch:** if you depend on HCP Terraform (HashiCorp's managed platform), BUSL-licensed providers, or provider features that have diverged since the fork, stay on Terraform. The decision is supply-chain, not syntax. + +## CLI Parity (P1 Declarative Intent) + +- `tofu init`, `tofu plan`, `tofu apply`, `tofu destroy` mirror `terraform init/plan/apply/destroy`. +- The lock file (`.terraform.lock.hcl` → `.tofu.lock.hcl`) is committed; it makes `init` reproducible. +- Workspaces, state backends, and module sources behave as in Terraform — see `terraform.md` and `state.md`. + +## Registry Parity (P6 Modules Compose) + +- OpenTofu can consume the Terraform Registry and the OpenTofu Registry. Module version pinning works identically. +- Some providers have BUSL licenses that OpenTofu cannot ship; verify a provider's license before adopting it. An MPL or Apache provider is portable; a BUSL provider is not. +- See `modules.md` for module structure, which is unchanged from Terraform. + +## Migration from Terraform (P5 Version Everything, P9 Drift is Recoverable) + +- `terraform state pull > state.json` → `tofu state push state.json` carries state across. Validate with `tofu plan` after the push — the plan should be empty. +- Rename the binary in CI: replace `terraform` with `tofu` in scripts. The lock file may need regeneration. +- Migrate one workspace at a time. Do not big-bang a migration; rehearse on a non-prod workspace first (P4 Plan Before Apply applies to the migration itself). +- Pin the OpenTofu version in CI. A migration is a versioned, reviewed change, not a quiet swap. + +## Governance and Community (cross-link `domains/security/supply-chain.md`) + +- OpenTofu's governance model — impartial, community-driven, layered, modular, backwards-compatible — is itself a supply-chain principle. A tool you cannot trust to remain open is a tool you cannot build on. +- This is the OpenTofu angle on `security/supply-chain.md`: license is a supply-chain property, not a legal footnote. + +## What Violates OpenTofu Discipline + +| Violation | Principle | +|-----------|-----------| +| Assuming OpenTofu == latest Terraform (unverified parity) | P5 Version Everything | +| Migrating prod state without a non-prod rehearsal | P4 Plan Before Apply | +| Adopting a BUSL-licensed provider into OpenTofu CI | P7 Least Privilege Providers, supply-chain | +| Quiet swap of `terraform` for `tofu` without a versioned change | P5 Version Everything | +| Losing state during migration | P3 State is Truth | \ No newline at end of file diff --git a/domains/infrastructure-as-code/state.md b/domains/infrastructure-as-code/state.md new file mode 100644 index 0000000..21af40f --- /dev/null +++ b/domains/infrastructure-as-code/state.md @@ -0,0 +1,77 @@ +# State — Derived Rules + +> Derives from `domains/infrastructure-as-code/first-principles.md`. State is the cross-cutting IaC concern: P3 (State is Truth) and P8 (Remote State with Locking) live here. Referenced by `terraform.md` and `opentofu.md`. + +## Why State Matters (P3 State is Truth) + +- The state file is the tool's memory. It records every resource it has claimed, every attribute it has set, and every dependency it has inferred. +- Without state, `plan` cannot compute a diff — it would have nothing to diff against. Lose state, lose the ability to reason about infrastructure safely. +- State can contain plaintext secrets (any sensitive resource attribute). Treat state as a secret-bearing artifact: encrypt at rest, restrict access, never commit it. + +## Remote State is Mandatory (P8 Remote State with Locking) + +- Local state (`terraform.tfstate` on disk) is acceptable only for a single developer on a throwaway sandbox. Any shared or production environment uses a remote backend. +- A remote backend provides: durability (state survives workstation loss), shared access (team members and CI read the same state), and locking (concurrent `apply` is serialized). +- No locking = data corruption. Two `apply` runs against the same unlocked state race; the loser's changes are silently overwritten. + +## Backend Comparison (P8, C4 Locality) + +| Backend | Locking | Encryption | Best for | Notes | +|---------|---------|------------|----------|-------| +| S3 + DynamoDB | DynamoDB | SSE-KMS | AWS-hosted | The canonical AWS backend; DynamoDB provides the lock | +| GCS | Built-in | CMEK | GCP-hosted | Native locking via GCS object versioning | +| Azure Blob | Lease | Customer key | Azure-hosted | Lease-based locking | +| HTTP (remote) | Server-side | Server-side | Self-hosted / on-prem | Requires a backend server (e.g., `terraform-backend`) | +| Local | None | None | Single-dev sandbox | Never for shared or prod | +| Consul | KV lock | — | Consul shops | Locking via Consul sessions | +| Postgres | TX | DB encryption | DBA-owned infra | Row-level locking | + +- Pick one backend per environment family. Mixing backends across environments fragments operational knowledge (C4 Locality). +- The backend config is part of the configuration, not a runtime secret. Credentials for the backend are runtime secrets. + +## State Isolation per Environment (P4 Plan Before Apply, C4 Locality) + +- One state per environment. Never share a single state file across dev, staging, and prod. A `plan` against a shared state crosses environment boundaries — a prod change could appear in a dev plan. +- Isolation patterns: separate workspaces, separate state keys in the same backend, or separate backends entirely. Stricter isolation = safer (separate backends for prod vs non-prod). +- Name state keys by environment and stack: `env:/prod/Networking`, not `prod` or `state`. + +## Locking Discipline (P8 Remote State with Locking) + +- `terraform force-unlock` is for a stuck lock after a crashed run, not for impatience. Verify the run is actually dead before forcing. +- A forced unlock without verifying the other run is dead causes the corruption the lock prevents. +- In CI, set a lock timeout so a wedged job fails rather than hanging. + +## Sensitive Values in State (P10 Secrets Never in Code) + +- Any `sensitive = true` attribute is hidden from plan output but stored in state in plaintext (unless the provider encrypts it). +- Backends with at-rest encryption (S3 SSE-KMS, GCS CMEK) protect state at rest. Access to the state file itself is the boundary. +- Never log, print, or commit state. Never pipe `terraform show` to a public channel. + +## State Commands (P3 State is Truth) + +- `terraform state list` — enumerate resources in state. First step of any state investigation. +- `terraform state show ` — inspect one resource's recorded attributes. +- `terraform state mv` — rename a resource's address without destroying and recreating it. Use when refactoring module structure. +- `terraform state rm` — stop managing a resource without destroying it. Use when handing a resource to another configuration. +- `terraform state pull` / `push` — export and import state. Used in migrations (see `opentofu.md`). +- `terraform import` — bring an existing resource under management by recording its state. The resource must already exist; `import` does not create. +- All `state` subcommands except `list` and `show` mutate state. Treat them as changes: review the intent, run in CI where possible, and commit the resulting config change that justifies the state move. + +## Drift and Reconciliation (P9 Drift is Recoverable) + +- `terraform plan` reports drift: resources that exist in state but were changed out-of-band, or resources in state that no longer exist in the provider. +- `terraform apply` reconciles drift by bringing reality back to the declared state. +- Manual changes to managed resources are the cause of drift. Treat a drift report as an incident: find who made the manual change and why, then close the access path or the gap that allowed it. +- `terraform plan -refresh=false` skips drift detection. Use only when you know state is current and you want a fast plan; never use it to hide drift. + +## What Violates State Discipline + +| Violation | Principle | +|-----------|-----------| +| Committed `terraform.tfstate` | P3 State is Truth, P10 Secrets | +| Local state in prod | P8 Remote State with Locking | +| `force-unlock` without verifying the dead run | P8 Remote State with Locking | +| Shared state across environments | P4 Plan Before Apply, C4 Locality | +| Unnamed state keys (`env:/prod`) | C4 Locality | +| Manual change to a managed resource | P9 Drift is Recoverable | +| `state rm` to "fix" a stuck resource | P3 State is Truth | \ No newline at end of file diff --git a/domains/infrastructure-as-code/terraform.md b/domains/infrastructure-as-code/terraform.md new file mode 100644 index 0000000..ea23a7f --- /dev/null +++ b/domains/infrastructure-as-code/terraform.md @@ -0,0 +1,58 @@ +# Terraform — Derived Rules + +> Derives from `domains/infrastructure-as-code/first-principles.md`. Applies P1–P10 to Terraform specifically. See also `opentofu.md` (the open-source fork), `state.md`, and `modules.md`. + +## HCL Structure (P1 Declarative Intent) + +- Resources are declared, not scripted. A resource block states what should exist; Terraform reconciles it. +- `resource "aws_s3_bucket" "logs" { ... }` — the type and name are the identity; the body is the desired state. +- Data sources read existing state without claiming ownership: `data "aws_caller_identity" "current" {}`. +- Variables are the input contract; outputs are the interface to consumers. Both are typed. + +## Providers (P5 Version Everything, P7 Least Privilege) + +- Pin the provider version: `required_providers { aws = { source = "hashicorp/aws", version = "~> 5.0" } }`. +- A provider block configures credentials and region. Credentials come from environment, files, or a secrets manager — never inline. +- One credential set per environment. Do not reuse a prod credential in a dev workspace. + +## The Core Workflow (P4 Plan Before Apply) + +- `terraform init` — resolve providers and modules. Reproducible from the lock file (`.terraform.lock.hcl`), which is committed. +- `terraform plan` — preview the diff. Read it. Every line. The plan is the contract review. +- `terraform apply` — execute the plan. Requires a reviewed plan in CI; in interactive use, requires typing `yes`. +- `terraform destroy` — tear down. Treat `destroy` as a first-class operation with its own plan review; prod destroys are a change event, not a keystroke. + +## Workspaces (P4 Locality of Environments) + +- Workspaces separate state for the same configuration across environments (dev, staging, prod). +- Do not use workspaces to separate unrelated stacks — use separate configurations. A workspace is an environment axis, not a project axis. +- State is isolated per workspace (see `state.md`). + +## State Backends (P3 State is Truth, P8 Remote State with Locking) + +- Remote state is mandatory for any shared or production environment. See `state.md` for backend selection and locking. +- Never commit `terraform.tfstate` to the repo. It is a secret-bearing artifact and a source of drift. +- `terraform state` subcommands inspect and manipulate state directly — use sparingly, only for recovery. + +## Registry and Modules (P6 Modules Compose) + +- The Terraform Registry hosts versioned, signed modules. Reference modules by version: `source = "terraform-aws-modules/vpc/aws"`, `version = "5.x"`. +- Compose modules rather than copy-pasting blocks. A module is reviewed once and reused many times. +- See `modules.md` for module structure, versioning, and the module-vs-copy boundary. + +## Secrets (P10 Secrets Never in Code) + +- Secrets via provider data sources (`aws_secretsmanager_secret_version`), environment variables, or a dedicated secrets provider. Never a literal string in a resource block. +- State may contain plaintext secrets if a resource attribute is sensitive. Mark attributes `sensitive = true` to keep them out of plan output; use a backend that encrypts state at rest (see `state.md`). + +## What Violates Terraform Discipline + +| Violation | Principle | +|-----------|-----------| +| Unpinned provider (`source` without `version`) | P5 Version Everything | +| `terraform apply` without a read `plan` | P4 Plan Before Apply | +| Local state in a shared environment | P8 Remote State with Locking | +| Hardcoded secret in HCL | P10 Secrets Never in Code | +| Copy-pasted resource blocks instead of a module | P6 Modules Compose | +| Manual change to a managed resource | P9 Drift is Recoverable | +| Admin credentials in CI | P7 Least Privilege Providers | \ No newline at end of file