Jon Chery
496303471d
---ci--- project: atelier phase: 7 milestone: v0.1 status: complete phase_role: final milestone_complete: true requirements: covered: [ATELIER-01, ATELIER-02, ATELIER-03, ATELIER-04, ATELIER-05, ATELIER-06, ATELIER-07, ATELIER-08, ATELIER-09, ATELIER-10, ATELIER-11, ATELIER-12, ATELIER-13, ATELIER-14, ATELIER-15, ATELIER-16, ATELIER-17, ATELIER-18, ATELIER-19, ATELIER-20, ATELIER-21, ATELIER-22, ATELIER-23, ATELIER-24, ATELIER-25, ATELIER-26, ATELIER-27, ATELIER-28, ATELIER-29, ATELIER-30, ATELIER-31, ATELIER-32, ATELIER-33, ATELIER-34, ATELIER-35] partial: [] ship: milestone: v0.1 type: NFR tag: v0.0.7 merge: milestone/v0.1-atelier -> main release: https://git.cloudinit.dev/cloudinit-bot/atelier/releases/tag/v0.0.7 ---/ci--- Milestone v0.1 — Initial Framework (NFR, complete). 8 core principles (C1-C8), 11 domains, 110 domain principles, 27 derived docs, 4 good + 3 bad examples, 4 language docs, full matrix, 3 review docs. All 35 requirements covered. 7 patches (v0.0.0 pre-execution through v0.0.7 final). v0.0.7 IS the v0.1.0 milestone release.
44 lines
1.3 KiB
Markdown
44 lines
1.3 KiB
Markdown
# Testing — First Principles
|
|
|
|
## 1. The Principles
|
|
|
|
### P1. Tests as Specification
|
|
Tests document what the code should do. Reading the tests is reading
|
|
the contract.
|
|
|
|
### P2. Independence
|
|
Tests do not depend on each other. Order does not matter. Parallelism
|
|
is the default.
|
|
|
|
### P3. Determinism
|
|
Same input, same output, every time. No time, randomness, network, or
|
|
filesystem in the test path unless explicitly modeled.
|
|
|
|
### P4. Fast Feedback
|
|
Tests run in seconds, not minutes. Slow tests are skipped, then
|
|
deleted.
|
|
|
|
### P5. Coverage of Behavior
|
|
Cover what the code does, not what it is. Lines covered is not the
|
|
goal. Behaviors exercised is the goal.
|
|
|
|
### P6. Failure Specificity
|
|
A failing test names the file, the function, the input, the
|
|
expectation, and the actual. A test that fails unhelpfully is
|
|
broken.
|
|
|
|
### P7. Realism
|
|
Test data resembles production data in shape, distribution, and
|
|
edge cases. Toy data hides bugs.
|
|
|
|
### P8. Maintainability
|
|
Tests are first-class code. They are read, reviewed, and refactored.
|
|
Test code is not throwaway.
|
|
|
|
### P9. Edge Case Coverage
|
|
Boundaries, nulls, empty sets, maximums, minimums, and invalid inputs
|
|
are tested. The middle of the range is the easy part.
|
|
|
|
### P10. No Test Theater
|
|
A test that cannot fail is not a test. A test that asserts nothing
|
|
is a lie. Tests earn their place by being able to catch real bugs. |