Jon Chery
1.3 KiB
Observability — First Principles
1. The Principles
P1. Structured by Default
Logs, metrics, and traces are structured. Free-form text is for humans; machines need fields.
P2. Correlation
Every event is traceable to a request, a user, an action, a trace ID. Context flows through the system.
P3. Sufficient Context
The information needed to debug is in the event itself, not in tribal knowledge. "What was the user doing?" is answerable from logs.
P4. Cardinality Discipline
Labels and tags have bounded cardinality. Unbounded labels are an unbounded bill.
P5. Sampling with Intent
Sampling is deliberate, documented, and consistent. Head-based, tail-based, or none — chosen with reason.
P6. No Secrets in Observability
Observability data is not a secrets channel. Tokens, passwords, and PII do not enter logs, metrics, or traces.
P7. Actionable Alerts
Alerts are for things humans must act on. Every alert has a runbook. Alert fatigue is a defect.
P8. SLI/SLO Awareness
"Good enough" is defined. SLOs are targets, not aspirations. Error budgets are real.
P9. Cost Awareness
Observability has a cost — storage, compute, attention. Spend it on what earns it.
P10. Debuggability Over Coverage
A few high-cardinality traces beat millions of low-context logs. Signal beats volume.