atelier/domains/ai-ml/first-principles.md

AI / ML — First Principles

Scope per D-023: this domain covers ML engineering discipline — data versioning, evaluation methodology, serving patterns, and drift detection. It does not cover algorithm design, model architecture selection, hyperparameter tuning, or model-family comparison. Those are research choices, not engineering principles, and they have no derivation in the core C-rules.

1. The Principles

P1. Reproducibility is the First Class

Every training run is reproducible from pinned data + code + config + environment. An unreproducible run is an unreviewable run: you cannot decide whether a result is correct if you cannot recreate it. Reproducibility is the ML analogue of domains/devops/P1 Reproducibility and inherits its non-negotiable status. Lose any one of data, code, config, or environment pinning, and the run is anecdote, not evidence.

P2. Data is Versioned, Not Just Code

Datasets, features, and train/val/test splits are first-class versioned artifacts with content hashes and lineage. git alone is insufficient — datasets do not fit in git, and a dataset is not a commit hash. A model trained on "the data" is a model trained on an unknown input, which is a correctness violation. Version data the way you version code: pinned, named, and recoverable.

P3. Lineage is Traceable End-to-End

Any deployed prediction traces back through model → training run → dataset → source. No orphan models. A model in production with no lineage is a correctness defect: you cannot reason about its failure modes, you cannot roll it back to a known-good dataset, and you cannot tell whether drift is in the model or in the data that built it. Lineage is the audit trail of ML (domains/observability/logging.md).

P4. Evaluation is Defined Before Training

Metrics, splits, and acceptance thresholds are declared a priori, in code, before the model is trained. Cherry-picking metrics post-hoc is a correctness violation: the evaluation is no longer measuring the model, it is rationalizing it. The eval spec is a contract — it is reviewable, it is versioned, and it is the gate the model must pass before it leaves the experiment. This is the ML angle on C2 Clarity: the intent of the model is obvious to its reader because the eval declared it first.

P5. Models are Versioned Artifacts

A model is a pinned, immutable, registry-tracked artifact with a unique identifier. Never "the latest." A serving endpoint that pulls "latest" is serving an unknown model — its behavior is undefined, its rollback is impossible, and its lineage is broken. The model registry is to models what a container registry is to images (domains/devops/P7 Immutability): immutable, addressed by digest, promoted by stage.

P6. Serving is Observable

Inference latency, throughput, input distributions, and prediction confidence are first-class signals. Silent serving is a bug. A model in production that emits no metrics is a model you cannot operate: you cannot see latency regressions, you cannot see input drift, you cannot see a failing downstream consumer. Observability is designed in, not bolted on (domains/observability/metrics.md).

P7. Drift is Expected and Detected

Data drift, concept drift, and prediction drift are monitored as a matter of course. A drift signal is an incident, not a curiosity. ML systems decay without code changes — the world changes under the model — so "no code changed" is not a defense against a serving regression. Detecting drift is the ML-specific form of C7 Observability: you cannot fix a model you cannot see degrading.

P8. Inference Inputs are Validated

The model's input contract — schema, value ranges, types, and categorical domains — is enforced at the serving boundary. Out-of-contract inputs are rejected, not silently scored. Scoring an out-of-contract input is a correctness violation: the model's output is undefined for inputs outside its training distribution, and returning a number for it is lying to the caller. This is the ML angle on domains/security/input-validation.md and inherits C1's non-tradeable status.

P9. Pipelines Compose, Notebooks Don't

Training and serving flows are composable pipelines with explicit steps, named inputs, named outputs, and contracts between stages. Notebooks are for exploration, not production. A notebook in the serving path is a correctness defect: its state is implicit, its order is human-dependent, and its reproducibility is whatever the last operator remembered. Compose pipelines; keep notebooks in the lab.

P10. Rollback Includes the Model

A serving rollback restores the prior model artifact, not just the prior code. Promotion is reversible at the model layer. A rollback that redeploys old code but keeps the new model has not rolled back — the model was the thing that regressed. The rollback path must name the prior model digest, the prior dataset version, and the prior eval that cleared it. This is the ML angle on domains/devops/P4 Rollback First and domains/kubernetes/P10 Roll Forward, Roll Back.

2. Core Principle Trace

Each AI/ML P-rule derives from one or more core C-rules (C1–C8). The matrix extension lands in P4 of the v0.3 plan; the traces below are authoritative.

P-rule	Core	Why
P1 Reproducibility is the First Class	C1, C5	Correctness of results; reversibility of runs
P2 Data is Versioned, Not Just Code	C5, C7	Reversibility of datasets; observability of data lineage
P3 Lineage is Traceable End-to-End	C7, C1	Observability of provenance; correctness of attribution
P4 Evaluation is Defined Before Training	C1, C2	Correctness of the eval gate; clarity of a-priori intent
P5 Models are Versioned Artifacts	C5, C6	Reversibility of model identity; composability of registry stages
P6 Serving is Observable	C7	Observability of inference
P7 Drift is Expected and Detected	C7, C1	Observability of degradation; correctness of detection
P8 Inference Inputs are Validated	C1	Correctness of the serving boundary (security subset)
P9 Pipelines Compose, Notebooks Don't	C6, C2	Composability of stages; clarity of explicit contracts
P10 Rollback Includes the Model	C5	Reversibility at the model layer

3. What Violates These Principles

Violation	Principle Breached
A training run that cannot be replayed from pinned inputs	P1 Reproducibility is the First Class
A dataset referenced by a mutable path, not a hash	P2 Data is Versioned, Not Just Code
A production model with no record of its training data	P3 Lineage is Traceable End-to-End
Metrics chosen after seeing the results	P4 Evaluation is Defined Before Training
A serving endpoint that pulls latest from the registry	P5 Models are Versioned Artifacts
A model in production with no latency or throughput metrics	P6 Serving is Observable
A serving regression dismissed as "no code changed"	P7 Drift is Expected and Detected
An input with an out-of-range feature scored silently	P8 Inference Inputs are Validated
A notebook in the serving or training pipeline path	P9 Pipelines Compose, Notebooks Don't
A rollback that restores code but keeps the regressed model	P10 Rollback Includes the Model

4. Relationship to Other Domains

AI/ML is the engineering-discipline layer for model-bearing systems. It borrows the reproducibility, immutability, rollback, and observability disciplines of domains/devops/ and applies them to the data → model → serving lifecycle. Cross-links are one-directional (per D-026 extended):

• domains/devops/P1 Reproducibility ← P1
• domains/devops/P4 Rollback First ← P10
• domains/devops/P5 Progressive Delivery ← P10 (model canary)
• domains/devops/P7 Immutability ← P5 (model images)
• domains/data/migrations.md ← P2 (data versioning ↔ migration discipline)
• domains/data/schema-design.md ← P8 (inference input contract)
• domains/observability/metrics.md ← P6, P7
• domains/observability/logging.md ← P3 (lineage)
• domains/security/input-validation.md ← P8
• domains/security/secrets.md ← P8 (serving credentials)
• domains/performance/backend.md ← P6 (serving latency)
• domains/kubernetes/workloads.md ← P9 (serving on k8s)
• domains/testing/first-principles.md ← P4 (eval as a gate)
• domains/gitops-operators/first-principles.md ← P10 (model rollback in a GitOps loop)