Your evaluation dataset is lying to you. Here's how to catch it.

Three ways eval datasets mislead:

Data leakage between train and eval. More common than you think, especially when both come from the same source. A duplicate example in both sets inflates your metrics. I've found leakage in 4 out of the last 10 projects I audited.

Distribution mismatch with production. Your eval set was collected 6 months ago. The world has changed. Your users have changed. New patterns exist that your eval set doesn't cover.

Selection bias. Your eval set was curated by engineers who unconsciously picked "reasonable" examples. It doesn't include the weird, malformed, adversarial inputs that real users send.

How to build an eval set that's actually honest:

Sample from production logs (with proper anonymization). Include recent data, not just historical. Deliberately add edge cases and adversarial examples. Have domain experts validate the labels. Version it and update quarterly.

And the most important check: compare your eval set distribution against recent production traffic. If they look different, your eval metrics are unreliable.

A model that scores 95% on a bad eval set is more dangerous than one that scores 85% on a good eval set. At least the second one gives you honest information.