VIII · Statistics & inference
Confidence intervals
What it is
A range that would contain the true parameter in 95% of repeated samples. The dual of hypothesis testing — and usually more informative.
Where it lives
Latency reports ("p99 is 45ms ± 3ms"), benchmark variance, A/B-test effect estimation.
The key insight
A 95% CI is not "95% probability the true value is here". It is "95% of intervals computed this way contain the true value". Frequentist subtlety.