“Do Not Optimize Prematurely”, but be careful!

“Do not optimize prematurely”. A sentence that is worthy of hanging all gilded and framed on any software professional’s wall. Living up to its hallowed words has made our software more robust and easier to understand, therefore better and cheaper to maintain and operate. However, the credo has a nefarious side-effect that needs to be addressed.

The law ostensibly appears to assume a dichotomous condition; either the software is optimized or it is not. This is plainly wrong. In reality, there are three conditions; the logic and algorithms are i) naive, ii) sensible or iii) optimized. No one likes a professional in their home who implements naive solutions; kitchen sink pipes that are taped together, paint that is mindlessly being applied over old paint and a cable mess that is near-impossible to sort out. Most of us don’t like that, and neither do organizations that hire software professionals.

Defenders of the naive implementations must be resisted tooth and nail. These ‘professionals’ will not shirk from employing the ‘Do not optimize prematurely’ credo as a shield to legitimize their choices. We are talking here for example about indexes on databases, the good practice of by default lazily fetching in an ORM framework, or using string buffers to concatenate strings. The costs of these measures are relatively low, they have a proven track record and there seldom is a good reason not to apply them to any project. These measure firmly move the project from the ‘naive’ to the ‘sensible’ group. The credo this post is about is being abused if it used to justify ‘naive’ choices.

If we accept the premiss that ‘naive’ choices are mostly plain wrong, the discussion must always be between a ‘sensible’ and an ‘optimized’ solution. This is where the credo shines. Optimized solution are, for example, having views in the database to fetch data quicker, overnight batch processing to pre-calculate data, caches for returning much-asked for data and asynchronous queues to process resource-intensive orders. These measures make the software intrinsically more complex, therefore requiring a trade-off decision between performance and complexity. Such a decision requires proper justification and this is exactly the red line where “Do not optimize prematurely” is meant to trigger parties to discuss and weigh their options.

At 42, we subscribe to the principle of sensible solutions by default and optimized solutions by exception. Our customers are better served in the long run by having a craftsman-attitude in the professionals they hire. It is our business to care.