The best model I ever deployed was a logistic regression.

A client wanted a churn prediction system. I started with logistic regression as a baseline while the "real" model (a gradient boosted ensemble with 200 features) was being developed.

The logistic regression with 8 features hit 91% accuracy. The complex ensemble hit 93%. Two percentage points better, but 10x more complex, 5x more expensive to serve, and significantly harder to explain to stakeholders.

We shipped the logistic regression.

The 2% accuracy difference wasn't worth the operational complexity. The business impact was identical — the churn prevention team couldn't meaningfully act on the difference between 91% and 93% prediction accuracy.

This goes against everything the ML community celebrates. We worship complexity. Bigger models. More parameters. Novel architectures.

But in production, simplicity is a feature. Simple models are easier to debug, explain, monitor, maintain, and trust.

My rule: always start with the simplest model that could possibly work. Only add complexity when you can prove it delivers meaningful business value, not just incremental metric improvement.

The most senior thing you can do as an ML engineer is choose NOT to use a complex model when a simple one works.