I suspect that many programmers and computer scientists are Vulcans. You see, on Vulcan, programming is a form of applied mathematics. All software is precisely specified using logic and mathematics. What’s more, these formal specifications rarely change. So programming is like proving a theorem, and Vulcan programming languages are like mathematics. Vulcans prize elegant programs, and because the specs don’t change they can refine and polish their programs to perfection. Adding a level of abstraction is always worthwhile if it shortens the program. Vulcans don’t use debuggers, because errors are a sign of a flaw in reasoning, and it is better to puzzle out the root mistake than to make ad hoc fixes. Dijkstra must have been a high priest or something back on Vulcan.

Vulcans are renowned throughout the galaxy for their compilers and algorithm libraries. You would be crazy to use a non-Vulcan regular expression engine. But they are equally as famous for their terrible user interfaces, where they lack objective metrics to optimize. And don’t ask them to build something without first having a precise and complete specification, unless you’re into that nerve pinch thing. Luckily for the galactic software industry, there is an even larger community of programmers who originated on the Planet of the Apes.

The Apes aren’t into mathematics and logic. Instead their passion is for socio-economic-grooming systems of unrivaled intricacy. Galactic anthropologists are baffled by the incredible complexity of Ape society. There are no written rules, and the rules are continually evolving, yet Apes can simultaneously fulfill a dozen different roles without breaking their rules. On top of this, the Apes have integrated software throughout their society. All sorts of automated systems play central roles: communication, entertainment, accounting, manufacturing, distribution, you name it. All these systems exist primarily to interact with Apes in their informal shifting social games. So there are no formal specifications, nor can there be any. As a result the Apes have developed a style of programming inconceivable to Vulcans. Their software is constantly being redesigned, so they make it as lightweight and clear-cut as possible. The Apes can’t prove their software is correct, and they seem puzzled by the very idea, as if they were being asked to prove correct a chair. However they design amazing user interfaces that harmoniously blend cognitive, artistic, and emotional significance. Much like the Apes themselves do. Their systems are imperfect and provisional, but they are forever fixing and improving and transforming them at a furious pace. For these qualities, Ape programmers are highly prized throughout the galaxy. If the purpose of your system is to interact with sentients in a social or economic context, you get an Ape to program it. Using Vulcan tools.

The Vulcans and the Apes despise each other. The Vulcans believe the Apes are too stupid to think abstractly, which is the essence of programming. The Apes believe the Vulcans are crippled at communication, which is is the essence of programming. My doom is to be torn between these irreconcilable opposites. For I am Vulcan, and was raised Vulcan, but I married an Ape, and made her planet my home. I will not know peace until I find a way to combine the best of both worlds.

It is an easy trap to fall into. Programming requires a certain kind of analytical intelligence. Being more intelligent in that way increases your programming ability exponentially. It is emotionally satisfying to think of yourself as a different species from the average programmer. Programming becomes a demonstration of your superior intellect. Surely such powers shouldn’t be wasted on mundane chores, but instead applied to timeless works of brilliant inspiration, to be admired by the common programmer only through protective eyewear. What a load of self-indulgent adolescent crap. In programming as in the rest of life, attitude trumps intelligence. I had to learn that the hard way.

My road to perdition was the seemingly noble cause of doing computer science research in the real world. It is in the crucible of practical problems that great new ideas can form. Unfortunately the result was a strange brew of triumph and disaster. I built some systems that were uniquely powerful and far ahead of their time. Yet they were also wildly idiosyncratic, and deeply flawed in some practical aspects. I inflicted a legacy of pain upon the programmers who had to work with my misbegotten creations for decades afterwards. I too was afflicted. I spent those decades debugging the Frankenstein. In the OS crash debugger. In hex. Because wasn’t it oh so clever and efficient to build a radically new kind of database with extreme reliability requirements as a highly multithreaded kernel extension to the OS?

What may have saved my soul is that I did not sell the company and walk away from my mess. I stuck with it for 20 years, and eventually cleaned up the mess as best I could with a massive migration to a new framework. That experience taught me a lot about what really matters in programming. It is not about solving puzzles and being the brightest kid in the class. It is about realizing that the complexity of software dwarfs even the most brilliant human; that cleverness cannot win. The only weapons we have are simplicity and convention. Tattoo that on your forehead in reverse so that you always see it reflected in the screen. What is truly decisive on the battlefield are attitudes: hard work, responsibility, and paying attention to reality instead of the voiceover in your head.

Programming is an embarrassment compared to other fields of engineering and design. Our mainstream culture is one of adolescent self-indulgence. It is like something from Gulliver’s Travels, with the curly-bracketeers vs. the indentationites vs. the parenthesesophiles. The only thing that everyone seems to agree upon is how stupid all the other programmers are. Try googling “stupid programmers”. We have met the enemy, and he is us.

Programming will not grow up until our culture grows up. We can only patiently and persistently do our part to elevate the level of discourse, and share what wisdom we have gained.

P.S. I deleted a bunch of the cursing, but I think I’ll let the rest stay as a testament to my point.

The objective evidence we have is that the differences in programming performance are almost entirely due to the individual cognitive differences of programmers. It doesn’t matter what language a great programmer uses – they will still be an order of magnitude more productive than an average programmer, also regardless of what that programmer uses.

The anecdotal benefits of esoteric languages are a selection effect. Here is a common scenario. Lots of really smart programmers think they are too good to waste their talents doing mere application programming. But they also love esoteric languages that show off how smart they are. So you can get them to do application programming by letting them use their beloved smarty-pants languages. Presto, amazing results. But the ubermensch aren’t about to stoop to maintenance programming. Once the fun development is done, they are gone. When you bring in professional programmers to take care of things, they are dumbfounded by the towering monument to mental masturbation. The system gets thrown out and rewritten in a normal programming language using normal techniques that normal people understand. The super programmers blame it on the stupidity of the new hires, further confirming their sense of superiority.

There are no super programming languages, only super programmers. And they tend to be super jerks. I should know – I used to be one. What would really make a programming language be super powered is the ability to be used by normal people.

P.S. I seem to have hit a nerve with this! Methinks they doth protest too much. I responded to some points in this comment. The final paragraph bears repeating:

These kinds of claims are just thinly veiled bragging about how smart Lisp/Haskell programmers are. I actually agree that they are smarter. But ascribing it to the language is a form of tribalism: our language (country/race/sports team) is better than yours, so we are better than you. Language tribalism is a pernicious affliction on our profession and our art. Grow up, everyone.

P.P.S. I guess I was a little intemperate, and got it back ten-fold. Publicly calling out cherished myths and immature behavior is not going to win me many converts. I also probably overstated the science, which is old and ambiguous. Still, my experience is that when people talk about how great their language is, they are really talking about how great it makes them feel. That’s fine, programmers should use the language that makes them happy, and being happy could very well make them more productive. But programmers who dislike the language will probably be equally disadvantaged. And in any case, the effects are small compared to the differences between programmers. But it does raise some interesting questions: what are the factors that make languages more amenable to programmers? Can we identify specific cognitive styles or personality types and map them to specific language design decisions?

I am re-opening comments, so long as the discussion remains reasonable.

Opinion is split as to whether this decision shows Steve Jobs to be a visionary hero seeking only to improve the quality of the user experience, or that he is a megalomaniac Fascist. But I think people are missing what this is really about. It’s about the money. The Ad money.

App Ads are potentially a Google-scale cash flow, and Jobs wants it. He wants to sell ads and inject them into apps on the iPhone/iPad/iDesk, just like Google injects ads into web pages. He can’t do that unless all the apps are using his libraries. If he lets Flash apps in, then Adobe would be able to sell the ads instead. That is why section 3.3.9 of the new developer agreement says “the use of third party software in Your Application to collect and send Device Data to a third party for processing or analysis is expressly prohibited”. Like, for example, Ad networks.

Help me, Google; you’re my only hope.

We cannot evolve beyond imperative programming until we solve the problem of coordinating changes to state. Functional programming and transactions segregate the problem, but don’t solve it. I am exploring a new kind of language, inspired by modern “Model-View-Binding” application frameworks. The language uses bidirectional data binding as its fundamental mechanism, generalized into arbitrary change-triggered actions. Bindings propagate changes coherently: execution order is determined automatically based on the interdependencies between bindings. The programmer does not have to worry about execution order. A side benefit is the automatic discovery of parallelism. Unfortunately coherence doesn’t work in a conventional heap of objects with arbitrary dynamic pointers to each other. I propose a data model in which objects and collections of objects are nested to form trees. Pointers are replaced with cursors constrained within a collection. This richer structure permits coherent and deterministic execution of bindings. Coherent binding offers the simplicity of declarative programming while maintaining complex dynamic state.

Comments? The idea is to use data binding as a metaphor that many programmers are already familiar with. Can I assume people will get what “Model-View-Binding” is, even though I just invented the term?

data Attack = Magic | Melee
hit surprise defense attack = damage
  where effectiveness = power * (if surprise then 3 else 2)
      (power, damage) = case attack of
                        Magic -> (5, max 0 (effectiveness - defense))
                        Melee -> (4, 2 * effectiveness / defense)

This passes pairs of values (power, damage) through conditionals. That allows the conditionals to multiplex more than one value, rather than being repeated. I suspect I could stymie that approach with more complex data flows, but I am happy to acknowledge that Haskell does OK on this example. Thanks, Per.

Why didn’t I see this before? Maybe I drank too much of the Self kool-aid. Subtext has always been a prototypical language like Self. That is an elegant idea, but static classes really are a handy way to impose structure on the chaos. That structure is needed to guarantee deterministic parallelism, and it also comes in handy in a lot of other places where you want to impose constraints.

My next step is a precise description of typed Subtext, focusing on the data model. I may start off with a normal imperative semantics and add coherence in the subsequent step. I am going to use a textual syntax, since that is what people need and expect. Subtextuality manifests in that the syntax will require an IDE to be easily editable, since it is actually just a pretty-printing of an underlying code meta-model.