Programming is deeply disappointing. We continually fail to live up to our own expectations, more so than any other form of engineering or craft. And it’s not getting much better. The lack of any substantial progress is the most disappointing thing of all. Worst of all: it is our own fault.
Most of the software and standards we use are deeply flawed. We all know it, but we accept it as a fact of life. I guess that at least half of programming effort is wasted by having to live with the accumulation of past mistakes that have become entrenched into dysfunctional standards and platforms. The problem is that it’s no one’s job to fix things. We are powerless as individuals, and we have no institutions whose job it is to fix mistakes. We need to build such institutions. But in the meantime, there is one thing we can do as individuals.
The fundamental reason that it always makes sense to live with flawed designs is that software reproduces without cost and never wears out. This is unique in human history, and we have not yet come to terms with its downside. Every other work of man has a natural lifespan that forces us to replace it regularly. Replacement is when things improve. You can incorporate new technologies and new designs. Without the steady demand for replacement, innovation languishes. This law holds true for life as well: evolution requires death.
I have therefore decided that in the future all of my software will have an expiration date. I will release it under a Darwinian License that says the software and any derivatives of it can not be used after a fixed date, except for educational and research purposes. By forcing an eventual complete rewrite, progress gets a chance. It is our duty as programmers to not let the dead hand of our own works hold back progress by those who come after.
The software itself is more like the genetic code of a species. If really Darwinian, it should die when it is no longer fit for survival in its environment: when no users use it any more. And that should be decided by its environment: its users.
It sounds like the GPL is the license you want.
The problem is that fitness in the software marketplace is destroying fitness for software development, and ultimately leading to stagnation for everyone. Sometimes evolution needs an asteroid or two to help things along.
Whilst I agree with the sentiment, I think you’re seriously underestimating the “pain point” that the market is soon going to be experiencing. In a manner not too dissimilar from the global economy, we have been accruing technical debt for a few decades now and it simply is not sustainable.
There are issues at all levels of the stack — from basic IP allocation and to the security of apps to mobile interfaces. If anything, I would say that this offers a wonderful opportunity to step in, solve a number of the low-hanging fruits and bring on a “soft reboot” of sorts. Such an opportunity simply hasn’t existed for a while…
Whilst the point about evolution is true enough, I do not believe the premise carries to software. This is because, as we can see from the long term success and ‘resurgence’ of Unix, the creation of assembly language, then C, the increased abstraction in programming, and the growth of mobile computing, or any major trend in software or computing: people would prefer to build on what we have to get something new done, or something old done faster and more efficiently, than to obsess over making things perfect. This is pragmatism. This is humanity. Perfection is a myth, and its pursuit without reference to resource utilisation or goal often (but not always) a pointless and wasteful endeavour.
I am not calling for perfectionism. Good-ism would be OK. Or even not-disgustingly-horrible-ism.
It’s like the “Carousel” from Logan’s Run but for software: http://www.youtube.com/watch?v=xSnLU9nyFSA
Sounds like somebody has read Henry Petroski.
I laughed at Darwinian license. But seriously, software does expire with time. DOS died, ed died, vim inherited vi, and no one plays Doom 1 anymore. In fact, innumerable softwares died in your lifetime. Your problem is with the “elders”, those that for some reasons, such as dependency or lack of a heir, are still alive after so many years. Who should decide when a software will die? Like with people, no one. They’ll die when it’s their time, and if you want something better, go and make it.
I agree that we should have institutions that fix things from the bottom. Why can’t we have them now? We can, we just need enough people to organize and make it happen. If you start anything, count me in.
True progress, as Petroski points out, needs to be married with studying engineering failures.
I can’t even begin to tell you how many poorly designed accounting systems I’ve seen. But to people at MIT — Jonathan’s colleagues — accounting systems are boring. But talk to a random accountant and he will tell you the system he uses is garbage.
At first when you said “the problem is that it’s no one’s job to fix things” I had thought you drank the “Technological Debt” kool-aid. It’s a bad metaphor for several reasons: (1) debt is a useful tool and can be managed; (2) it can be used to justify pointless “pay down” activities that aren’t even on the project’s critical path; and (3) it’s devoid of the context from which new features will be added, a project’s liability with respect to one new feature is the same project’s asset with respect to a different new feature.
I see these methodological movements as well-intentioned responses that fail to address the root problems. To solve our problems we need organized political action. The Darwinian License is just a small personal step to encourage progress.
I don’t feel the biggest problem were old software. Actually, software companies — who are most responsible for the overall quality of this industry — often force out old software to make space for the new ones. Often even against the users’ will.
Jonathan, this time, it’s a miss.
I can’t remember ever seeing that happen. Except for Steve Jobs. And when DEC dumped the VAX for Alpha (and killed the company).
Microsoft does it quite often. (E.g. Windows XP or the 2000 server line). Maybe I’m wrong with this.
But for old standards: you are absolutely right. Take SMTP and e-mail for example: it’s just ridiculously flawed. It is, because it’s 50 years old. FIFTY! (about)
Old standards are a huge anchor on the industry, agreed. How on earth you image we could get rid of them? Their existence seems so natural from the Darwinian point of view.
Forcing out old software without learning what was wrong with it is even worse than anything Jonathan covered here.
Here is a good example: Upgrading your transaction processing subsystem from version 5.5 of your software to version 6.0. Suddenly the vendor decides that reversed transactions in your general ledger are no longer important, and in version 6.0 they are marked as “comments”. I am not making this shit up.
Don’t worry, you always can have a marketing department to cover how dumb engineers can be.
Agreed. I call it “technological support fire”. Software companies try to create as many new technologies as they are able to. This is to prevent competitors to gain advantage, to force them to align to the “new technology”. This way competitors will spend their resources on getting compatible with all the new stuff and won’t have time for real development.
Also, it’s a financial pressure: a software is usually sold once. To make more money, you have to sell it again.
Hmm, this sheds a good light on software rental: software factories are less forced to create new versions, so they can put more focus on real development. Need to think on it more…
Germs don’t “die” to evolve.
Just don’t forget to license Darwinian License in Darwinian License itself. 🙂
We all have the power of choice. Individuals can change the direction of the industry – these are often called thought-leaders. Individuals can find like-minded peers and then use social pressure towards their ends.
If individuals cannot change anything in software development, who can? The great dictator of software development? If there was one, we could organise a coup d’état :).
As for the lifetime of softwares – they are short enough. They are always in need of upgrading to the latest platform or library api. Instead of Darwinian evolution, I would evoke Schumpeter’s “creative destruction” — the idea that entrepreneurs will drive out old ideas with their new ones.
Invent without fear! Use a liberal (perpetual) licence – BSD, MIT/X11, zlib, etc. – but rest assured that your softwares will, at best, last no more than a few decades and more likely, no more than a few short months or years :D.
For me this all comes down to what makes our jobs possible. Any good developer adds value by writing software. If you can add value by fixing mistakes in software, then go ahead and fix it.
In fact, I’ve initiated this process some time ago with the company I work for. You can read about it in my blog.
The point I’m trying to make is that even improved maintainability of code can be added value for a company (and without them noticing, for the users as well as development can speed up).
Reminds me of the rewrite question, which has recently been discussed by K. Scott Allan , Steve Blank, and many more. Software is for business, and if the business doesn’t deem the app too painful, or customers are using it because there is no alternative, well, that software is there to stay, and rewriting the whole of it doesn’t add any value. However, this does not mean software should remain unchanged – business changes, needs change, and so should the application(s) serving them. If rewriting some component of an app saves money for the business, then you can make a case for it and rewrite/replace that component. Meaning (some of) the old component “died”. That does sound like evolution to me.
I prefer to think about evolution in terms of the system, where the components die and new components come to life – like a forest with the trees, bushes, squirrels, etc.
Evolution requires reproduction, not death. Death may result as existing species are crowded out by the new, better adapted inhabitants but death is not required for evolution per se.
“Programming is deeply disappointing. We continually fail to live up to our own expectations, more so than any other form of engineering or craft. And it’s not getting much better. The lack of any substantial progress is the most disappointing thing of all. ”
Really? Really? Software seems pretty damn good to me. Progress has been a lot faster than in nearly any other field. Consider the pathetic software of 1971 next to the video games, mobile apps, websites, and jeopardy-paying software of 2011. Now consider the 747 of 1971 to the 747/777/A380 of 2011….
I saw your post and listened to this story at the same time:
http://www.npr.org/blogs/krulwich/2011/02/02/133188723/tools-never-die-waddaya-mean-never
Cool. The difference, of course, is that while there are still people in India using grain drills, if it were software it would be incorporated as a subassembly in every unit of modern farm equipment.
You realize that your software has a giant evolutionary disadvantage against software in just about any other license? All other things being equal, software that survives will be the kind not released under Darwinian licenses.
Just as Niko says. Your revolutionary software plain old won’t be used if you release under a non-free license, which one with reversion to default copyright would be. Copyright is the wrong lever for what you want to accomplish, unless you merely want to make a statement.
Isn’t the trouble you’re addressing chiefly caused by data formats rather than software itself? The original Windows/Word or C/Unix implementations are long dead, but the plentiful rewrites had to be mostly compatible with everyone’s existing .doc, .c, .exe, .sh, etc. files (and the skill to create more of those files). If one wanted to set the date for the death of C or Word, the license would have to prohibit the processing of .c/.doc files past that date – otherwise you’d only force the creation of a compatible clone, not?
Yossi – In the case of Word, I would argue that a rewrite could have allowed it to be rationalized and simplified – the problem is not inherent in the doc format. But I do agree in the case of C. It is not enough to rewrite the compiler. It is the design of the language that is flawed. Any self-respecting engineering discipline would have abandoned or fixed C decades ago. We need to form institutions to deal with such problems. I don’t yet see how to do that though.
I agree that most of the software around today is rather flawed, though I’m not sure that I would go so far as to put out a license forcing people to cease usage of my code. However, in place of that I think rewriting all the software that is relied upon, not necessarily keeping the same file formats, would be a better idea^1. So I’m writing an operating system. It’s a good thing I have so much free time on my hands, as it can be difficult to find the requirements of an operating system’s bootloader, never mind the rest of the code.
^1 Except, for compatibility purposes, enabling easy conversion between the formats for internal usage, and the formats readable by another computer. The host computer doesn’t really need to be able to read external file formats, it just has to know how to convert them to something useful, and convert its own files to something transferrable.
Gentlemen,
We are missing the point – how can matters be improved ?
1. The “new stuff”
New proggramming languages and APIs/SDKs are
being rushed out the door – prematurely and not self contained.
Having been implemented very naive and far from feature-complete
Quickly “being improved” and causing fragmentation of the work by the
developers and the various code bases.
2. OO, higher and higher degrees of abstration
We have a confusing tangled mess of layers, encapsulation and
information hiding in the code being implemented.
3. The Editor monopoly
All the new fancy stuff still depends on a simple text editor or
a too complex IDE/GUI. Indeed a very poor automation
4. 20/80 API/SDK/OS code ratio
80% of the base can not be used either due to side-effects or preventing
the application being upgraded or scaled in the future.
We need to concentrate on quality and stable performance.
The tools (Apps for the developers) of the trade has to be improved.
Something like Triz is needed (http://www.triz-journal.com/)
– to improve the way we document, analyze, design and implement
We need to look harder into parametric design, weak coupling,
more self-contained (complete) and better implemented
components that does not pull in tons of dependencies in the
name of portability and code reuse – two terms that has to be
considered harmful and obstructive.
Apparantly some of the programming languages are trying to
cope with this by using simplistic VM, placing limitation and
skewing the efficient use of the hardware.
The low-level being narrowed down to pointless abstracted
magic (nonsense) and dreamed-up features to protect us from
our self ?
Software Tools are 10+ years behind features and functionality
available in the hardware – the tools and apps could query and
adapt dynamically.
We need well designed query-interfaces, without going all the
way to put everything inside a sql database.
Certain things has to be fixed atomically and yet other issues
has to be dealt with at a overall level.
We have the work cut out for us 🙂