It’s great to be coding again, after spending so long lost in thought and theory. This has been my longest break from programming since I started at 13. I now have a working virtual machine that supports the essential semantics of the language. The next step is a compiler. The good thing about having a long break from programming is that it made the experience of starting up again quite vivid. What was most vivid was the many ways I make programming needlessly harder for myself.
It was insane to try using Scala instead of Java. A language so complicated that few people can fully understand it, with spotty documentation, and immature buggy IDE support. How could that possibly be a win over a language like Java that I already know cold and which has state of the art tool support? I was bored to death with Java, and I was seduced by Scala’s cleverness; it’s ability to express things very succinctly, and it’s more powerful abstraction capabilities. When you think about programming in the abstract, such issues loom large. But when you actually have work to do, these idealisms pale in comparison to having trusty, well understood, well supported tools. We programmers love abstraction and elegance, but it is often a siren song leading us to disaster.
An even graver mistake I found myself making is compulsive performance optimization. I know well the evil of premature optimization. It is even central to my philosophy of programming language design — I believe that existing languages are far more complicated than they have to be because they are obsessed with letting programmers hand-optimize everything. I am building a prototype VM where performance is largely irrelevant. Yet still I repeatedly found myself optimizing: caching, interning, fast-pathing, etc. I just can’t stop myself. There is something particularly insiduous about performance. I think it is that it is so much easier to think about performance than subjective design issues. Performance is objective — you can actually quantify it, unlike most of the issues that really matter: subjective design issues with no clear cut answer, which can be almost impossible to evaluate while you are still developing. As engineers we crave the objective and quantifiable world of performance. So we focus on that even when it isn’t important. Most of the time it isn’t important. It reminds me of the joke where a drunk loses his car keys in an alley, but looks for them under the lampost because the light is better there.
Cleverness and performance are major delusions of programming. We let ourselves be seduced by them to the detriment of our code. Unfortunately these are exactly the issues driving much research and development these days. It seems everyone is working on fancy complicated techniques to exploit the performance of multicore systems. I guess that just shows how much I am out of step with the times. So be it.
Update. I apologize to all the Scala enthusiasts who were upset by this post. Scala is an awesome language. But it is new, and it is complicated, and it is way overkill for my needs. Building a VM is really more of a job for C, and Java kind of feels like C to me now: low-level and simple. The point I was trying to make was that I was seduced by the sexy advanced features of Scala even though I didn’t really need them and using it would end up slowing me down.
I also want to thank the people who have stepped in and tried to keep the discussion on a mature level. You don’t see that too often on the interwebs. It is gratifying that my last several posts have attracted some nice thoughtful discussions. Good work guys.