Skip to content

J1 · Estimate and errors

  • Author: Krishna Nandimandalam
  • Date: 2026-10-09
  • Card: #4 J1 · PR: TBD

What I built

The Estimate value object: a frozen dataclass that holds any reportable statistical result (point estimate, confidence interval, sample size, method, unknown rate). Also the EvalhawkError exception hierarchy with subclasses DataError, ConfigError, StoreError.

Main signatures: 

@dataclass(frozen=True, slots=True, kw_only=True)
class Estimate:
    point: float
    low: float
    high: float
    n: int
    method: str
    confidence: float = 0.95
    unknown_rate: float = 0.0

The idea, simply

An Estimate is like a receipt: the number you computed, the range you're confident in, how many items you measured, and how you got there. Every stats function returns one.

The key insight: Estimate doesn't enforce point containment (low ≤ point ≤ high). Bootstrap methods can legitimately produce an interval that excludes the observed point because they're resampling the data, not fitting a parametric distribution. The Wilson interval method guarantees containment; others document whether they do.

How I know it's right

  • 19 unit tests cover valid values (defaults, zero width, point outside, negatives, n=0, inclusive bounds), invalid values (19 parameter combinations of NaN/inf/out-of-range), immutability (frozen, slots, keyword-only), equality, and string format.
  • Test the exception hierarchy separately: all subclasses inherit from EvalhawkError, which inherits from Exception.
  • Spot-checked against existing test conventions (test_ids.py): parameter names in ids=, section dividers, behavior-describing names.

What surprised me / gotchas

  1. NaN comparisons are always False. math.isfinite(nan) is the right check, not nan != nan (which is also False). That's why tests use math.nan, not direct != nan comparisons.

  2. Dataclass inheritance detail. I discovered that the field order in the class definition matters for how the generated __init__ works. We put the required fields first (point, low, high, n, method), then optional fields with defaults (confidence, unknown_rate). With kw_only=True, positional order doesn't matter to the user, but it affects the dataclasses.replace examples.

  3. Slots prevent __dict__. When using slots=True, Python doesn't allocate a __dict__ on instances. Trying to add new attributes raises AttributeError or TypeError depending on what you try. The test catches both because the exact error varies by Python version.

  4. __str__ must match exactly. The format string uses .1% for unknown_rate, which formats 0.125 as "12.5%". I verified this against the spec's example.

How AI helped (and where it was wrong)

Left for the maintainer to fill in.

Learn more