I once deployed a model with 99.2% accuracy. The client was thrilled.

Then we discovered it was a fraud detection model where only 0.5% of transactions were fraudulent. The model was predicting "not fraud" for everything and achieving 99.5% accuracy by being completely useless.

Accuracy is the most misleading metric in machine learning.

For classification with imbalanced data — which is most real-world problems — precision, recall, and F1-Score tell the real story. AUC-ROC shows ranking quality. Always, always visualize the confusion matrix.

For LLM evaluation, the metrics have shifted entirely. RAGAS for RAG systems. Hallucination rate for factuality. LLM-as-Judge for scalable assessment. And human evaluation remains the gold standard, however expensive.

The model that scores 99% accuracy on a 99% majority class is worse than a coin flip on the minority class you actually care about.

Choosing the right metric is a design decision, not a formality. Get it wrong and you'll confidently deploy a model that's useless in the exact way that matters most.