atelier/domains/compliance/data-retention.md

Data Retention — Derived Rules

Derives from domains/compliance/first-principles.md. Covers P3 (Retention is Policy, Not Storage) and the data-shape angle on P8 (Subject Access is Honored). Referenced by audit-logs.md (retention applies to audit logs) and evidence.md (evidence has a retention lifecycle). Framework-agnostic per D-024 — no regulation-specific retention periods.

Retention is Policy, Not Storage (P3 Retention is Policy, Not Storage)

• Data lifetime is declared and enforced as policy, in code — not left to the storage layer's defaults. The policy names what data class is retained for how long, what action fires at end-of-life (delete, archive, anonymize), and what exception path exists (a legal hold suspends deletion).
• Deletion at end-of-life is a feature, not a failure. A system that cannot delete on schedule is a system that over-retains, which is the symmetric violation of a system that under-retains. Both are P3 violations; the policy is the arbiter.
• "We kept it because the bucket was cheap" is a violation. "We deleted it because the policy said to" is correct. Cost does not override policy; policy is the contract.

Retention Policy as Code

• Retention rules live as code: lifecycle rules on the storage layer, scheduled deletion jobs, tiered storage transitions, and anonymization transforms. The code is versioned, reviewed, and auditable. A retention rule in a spreadsheet is a wishlist; the same rule in a reviewed, deployable lifecycle policy is a control.

  // illustrative lifecycle policy (abstracted, no vendor DSL)
  // object-storage lifecycle
  {
    "rules": [
      {
        "name": "user-events-90d",
        "match": { "prefix": "events/" },
        "transitions": [
          { "after": "30d", "to": "tier-cold" },
          { "after": "90d", "action": "delete" }
        ]
      },
      {
        "name": "audit-log-7y",
        "match": { "prefix": "audit/" },
        "transitions": [
          { "after": "365d", "to": "tier-archive" },
          { "after": "2555d", "action": "delete" }
        ],
        "legal_hold": "suspends-action"
      }
    ]
  }
  

• The retention policy is itself auditable: which rule fired when, against which objects, with what result. The deletion events are logged (audit-logs.md) — deletion is a significant action.

Retention vs. Backup — The Distinction

• A backup is a recovery mechanism: it exists to restore data after loss. A retention rule is a deletion mechanism: it exists to remove data at end-of-life. Conflating them produces data that survives both the deletion policy and the disaster — which is the opposite of compliance.
• A backup is governed by a recovery-point / recovery-time objective; a retention rule is governed by a lifetime. They are independent contracts. A backup that is also the retention store is a store where nothing is ever deleted, which is a P3 violation.
• A legal hold suspends retention deletion for a defined data set (e.g. data under investigation). The hold is itself a policy action, auditable and time-bounded, not a manual override.

Retention is Distinct per Data Class

• Different data classes have different lifetimes. The retention policy enumerates the classes and their rules; it does not apply one number to everything. Typical classes (the names are abstract; the periods are policy decisions, not regulation- specific):
  • Audit logs — long, often multi-year, governed by accountability needs (audit-logs.md).
  • User-generated content — tied to the user's account lifetime; deletion follows account deletion (cross P8 Subject Access).
  • Telemetry / metrics — short, governed by observability need (domains/observability/metrics.md); high-resolution data ages to downsampled aggregates.
  • Evidence artifacts — tied to the audit cycle (evidence.md); the cycle ends, the evidence ages out.
• A single retention rule for "all data" is a C3 (Simplicity) violation of the wrong kind: it is simpler than the requirement allows.

Subject Access is Honored (P8 Subject Access is Honored)

• Data-subject rights — access (what do we have on this subject), export (in a portable form), deletion (and prove it), correction — are operations with defined contracts and audit trails, not ad-hoc tickets. The system implements them as first-class operations; a subject-access request that requires a forensics team is a correctness defect.
• Retention and subject access interact at deletion: a subject deletion request fires the deletion policy for that subject's data, the deletion is audited, and the proof of deletion is returned to the subject (and recorded). A subject deletion that skips the audit is a P8 violation dressed as a P3 success.
• Cross domains/data/schema-design.md: subject access is only computable if the schema tags which records belong to which subject. A schema with no subject linkage cannot honor a subject request — it cannot find the data to delete.

Retention Migration Discipline

• Retention rules change. When the policy changes (a class's lifetime shortens or lengthens), the change is a migration: the new rule applies to data ingested after the cutover, and a backfill applies the new rule to existing data where applicable. Cross domains/data/migrations.md for the schema-lifecycle discipline this mirrors.
• A retention rule change that is not versioned, not reviewed, and not backfilled is a P3 violation: the policy is not actually the policy if the storage layer does not reflect it.

What Violates Retention Discipline

Violation	Principle
Data kept indefinitely because "storage is cheap"	P3 Retention is Policy, Not Storage
A retention rule in a spreadsheet, not in code	P3 Retention is Policy, Not Storage
A backup bucket used as the retention store (nothing ever deletes)	P3 Retention is Policy, Not Storage
A single retention period applied to all data classes	P3 Retention is Policy, Not Storage
A subject deletion with no audit record of the deletion	P8 Subject Access is Honored
A subject-access request that requires a forensics team	P8 Subject Access is Honored
A schema with no subject linkage (cannot find data to delete)	P8 Subject Access is Honored
A legal hold applied ad hoc, not as a policy action	P3 Retention is Policy, Not Storage
A retention rule change with no backfill to existing data	P3 Retention is Policy, Not Storage

Relationship to Other Domains

• domains/data/schema-design.md — retention requires the schema to tag data class and subject linkage; subject access is only computable over a schema that supports it.
• domains/data/migrations.md — retention rule changes are migrations; the discipline (version, review, backfill) mirrors schema migrations.
• domains/compliance/audit-logs.md — audit logs have their own retention floor; deletion of an audit segment is meta-audited.
• domains/compliance/evidence.md — evidence artifacts have a retention lifecycle tied to the audit cycle.
• domains/observability/metrics.md — telemetry retention is governed by observability need; high-res data ages to aggregates.
• domains/security/secrets.md — secrets have a retention lifecycle tied to rotation; a secret past its rotation date is overdue, not retained.