Developer inequality and the technical debt crisis

Recently there have been heated complaints that the culture of programming unfairly excludes some groups. They want to join the programming elite and get a spot at the startup trough. More power to them. I really have nothing to say about this issue because I think it is a distraction from a bigger issue with far greater importance to society. The bigger injustice is that programming has become an elite: a vocation requiring rare talents, grueling training, and total dedication. The way things are today if you want to be a programmer you had best be someone like me on the autism spectrum who has spent their entire life mastering vast realms of arcane knowledge — and enjoys it. Normal humans are effectively excluded from developing software. The real injustice of developer inequality is that it doesn’t have to be this way.

In the old days there was a respected profession of application programming. There was a minority of elite system programmers who built infrastructure and tools that empowered the majority of application programmers. Our goal was to allow regular people without extensive training to easily and quickly build useful software. This was the spirit of languages like COBOL, Visual Basic, and HyperCard. Elegant tools for a more civilized age. Before the dark times… before the web.

The civilized platforms controlled by large companies who invested in developer tools are all gone, strangled by the Darwinian jungle of the web. It is hard for programmers who have only known the web to realize how incredibly awful it is compared to past platforms. The web is just an enormous stack of kluges upon hacks upon misbegotten designs. This Archaeology of Errors is no place for the application programmers of old: it takes a skilled programmer with years of experience just to build simple applications on today’s web. What a waste. Twenty years of expediency has led the web into a technical debt crisis. To my shame, we are OK with that.

These are indeed dark times. The dominant programming platform is in technical debt bankruptcy. Programmer culture has been seduced by the dark side of startup success into a paroxysm of self-aggrandizement. I see only one glimmer of hope: the status quo is a complacent fragile system ripe for disruption.

Imagine an Internet HyperCard that allowed regular people to easily build web apps, as easily as using a spreadsheet. [Clarification: combining the programming simplicity of HyperCard with the ability to build networked systems for communication, collaboration, and commerce.] It would have to pave over and completely hide the Tower of Babel of HTML-CSS-JavaScript-HTTP-SQL-bash-etc. There are many app-in-a-can tools that generate highly stereotypical apps but to be truly disruptive we need to match the broad generality of frameworks like Rails. We really don’t know how to do this yet, which is why it hasn’t been done, and so it will involve research and technical risk. I believe it should abstain from the baroque complexity of OO and FP languages. Sadly, modern programmer culture fetishizes complexity as a status marker or tribal initiation ritual. Attempting to simplify and democratize programming will attract only scorn and derision (as did COBOL and Visual Basic). Programmer culture is broken so we have to route around it and appeal to the 99%.

The web triumphalists love to talk about changing the world. Well if you really want to change the world, empower regular people to build web apps. Disrupt web programming! Who’s with me?

This entry was posted in AgitProp, General. Bookmark the permalink. Both comments and trackbacks are currently closed.

77 Comments

  1. David
    Posted July 7, 2014 at 2:36 pm | Permalink

    Heck yes! Bert’s recent experiment to put Etoys on the web has had surprising success, and might lead the way:

    SqueakJS runs Etoys now

    (Etoys is object-oriented, but kids and teachers don’t seem to mind, but there is no reason why a Squeak-hosted language has to be OO. See Scratch, for example, which is pretty much event-driven imperative scripts. Next task: democratize the server-side tools.)

    • Posted July 7, 2014 at 8:56 pm | Permalink

      Look out for IndieBox OS!

  2. frog
    Posted July 7, 2014 at 3:33 pm | Permalink

    erm, web programming already is brain-dead simple. how do you want to simplify it? use something that is even simpler than python?

    • Michael
      Posted July 7, 2014 at 5:29 pm | Permalink

      Yes- simpler than Python.

      In the mid-90′s I was able to teach 8 years olds how to create usable Hypercard stacks in an hour. It was simple and powerful, visual and intuitive.

      Try to teach an 8 year old how to create a picture and make it clickable using Python. It’s certainly possible- kids are smart- but it’s going to take a lot more time than Hypercard, and the chance the kid will see it through is far less.

      The author is right. The barriers to entry have never been so high. I learned how to program with a book called “Basic Computer Games” by David Ahl. I turned my computer on, typed in the listings, and bam! I had games and I could edit them. I didn’t need anything but the book and a computer. All I had to do was turn it on and type.

      There’s no way for a child to do that today- unless they use the same book and the same old computers, and that’s a probably still a fantastic place to start learning programming.

      • Posted July 8, 2014 at 1:30 am | Permalink

        Is Impress / PowePoint basically the same thing as hypercard?

        • Posted July 9, 2014 at 7:29 am | Permalink

          PowerPoint is like HyperCard without the Power (i.e. any kind of programming).

  3. Posted July 7, 2014 at 3:50 pm | Permalink

    Hi Jonathan,

    You know, of course, that I agree with you deeply.

    In concert with building tools that allow beginners to build their own tools, we also need to grow a culture of creation. A culture where people with problems don’t, by default, go looking for an existing solution but instead, by default, look for a way to solve their problem through their own ingenuity, creatively recombining the resources they have at hand.

    Honestly the place where I’ve seen this attitude most strongly is at Burning Man, where by necessity people need to solve pressing problems under severe constraints. But probably any environment with constrained conditions promotes this attitude. Buckminster Fuller was extremely influenced by seafaring culture which presents similar design challenges.

    Much of the culture of the present web and mobile app economy promotes a passive “user” mentality. It is very difficult to encourage people to solve their own problems in situations where they only can see pre-existing solutions (“there’s an app for that”). I think it would be productive to think about and discuss niches of the web (or other parts of our lives) where we have pressing needs with severe constraints where we can nurture a creative culture.

    Toby

  4. John Baptist
    Posted July 7, 2014 at 4:03 pm | Permalink

    I agree that current web technology is a horrible pile of kludges. Let’s try to build something better. Some attempts are already being made. Please check out the amazing Ur/Web language [1] that lets you write front-end code, back-end code (compiled to native), and database code in a single, type-safe language that guards against errors that are rife in dynamically-typed web scripting languages. It is far from a finished product but it is an amazing glimpse and what could be with intelligent develop tools.

    [1] http://www.impredicative.com/ur/

    • Posted July 7, 2014 at 6:17 pm | Permalink

      Ur/Web is indeed impressive and highly principled. One of a whole spectrum of approaches we should be exploring. Adam says he is targeting experienced programmers, which is a different part of the spectrum from where I am working.

  5. Posted July 7, 2014 at 4:25 pm | Permalink

    Consider the haiku.

    The more ascetic the aesthetic the more elegant and powerful the result.

    When it comes to tools, it isn’t about what you add … but rather what you take away.

    • Posted July 7, 2014 at 10:35 pm | Permalink

      Agree totally. That is a hard lesson for techies like myself to learn. Just look at how bad we are at UI design. Turns out we are even worse at programming language design.

  6. Johan Stuyts
    Posted July 7, 2014 at 4:33 pm | Permalink

    Microsoft made an attempt with TouchDevelop (https://www.touchdevelop.com/). It needs some finishing touches, especially in the deployment section, but it shows that it is possible to guide application developers during development.

    Note that making good use of the platform still requires effort. I am a seasoned developer myself, but without reading the documentation, I would not have written a very simple application in the correct way.

    The platform also supports background tasks, which adds another level of complexity, which I doubt lots of application developers could come to grips with.

  7. Chris
    Posted July 7, 2014 at 4:36 pm | Permalink

    Problem solved already…seriously

    Maidsafe.net

    The SAFE network offers every developer the opportunity to create applications without any infrastructure costs and without any barrier to entry.

    • Scott
      Posted July 10, 2014 at 5:09 pm | Permalink

      MaidSafe is very interesting. Thanks for sharing.

      But it is attempting to solve an entirely different problem than this article is addressing: the complexity of development process for the web (html, css, javascript, server-side language, etc) and the fact that this complexity is a step backwards compared to the relative ease of thick client application development.

      • Posted July 10, 2014 at 6:19 pm | Permalink

        Hi Scott,

        “But it is attempting to solve an entirely different problem than this article is addressing”

        Sorry, that my comment is a little out of context to this post, but not by much. I’m simply going where the smart folk are and giving a heads up, in the hope that authors will give consideration.

        Once the penny drops for developers, that a global brain is being built then the scramble will be on and all this worry about the state of the web can be put aside…sounds crazy, I admit.

        My point being you won’t need to worry about servers, security, viruses, infrastructure costs etc etc if all goes to plan, the current internet/web will collapse onto the footprint of a fully distributed network within 10 years.

        It really is a question of, do you want to build the next YouTube, Facebook, Google with no upfront infrastructure costs? how big is first mover advantage in any area of business?

        2 monetary opportunities: (1) Form a ‘POD’ with other C++ developers and contribute to the core.
        (2) Become a builder on the API and as an example, build the next YouTube which has an inbuilt payment wallet…rewarded in the network currency SAFEcoin, automatically when anyone accesses a the video.

        The more people use your app, the faster it gets …now that is relevant to this post.

        How shocking is this NSA stuff….does that boggle one’s mind where this is headed? Well thankfully, one man had a vision 8 years ago, inspired with the simplicity of natures building blocks.

        To read the vision of the man who spent 8 years of his life creating a network, who’s vision is to free humanity from tyranny and include ALL humans on this planet, then maybe start with the following blog post:

        http://metaquestions.me/2014/02/10/a-compelling-business-strategy-for-a-fair-world/

        Thanks for taking interest, most folk have tunnel vision and will miss out on the opportunity of a lifetime.

        Cheers,

        Chris

  8. Posted July 7, 2014 at 4:47 pm | Permalink

    The web was a small but extremely significant step forward in its time. I believe that it has way outgrown its boots; it is beginning to facilitate some practices which we should be seeing as very threatening on a personal level. I recently wrote a blog article about this: http://perceptrobyte.blogspot.co.uk/2014/07/psych-research-facebook-and-fundamental.html

    Although my take is more on the backend and how it makes secrecy the path of least resistance, which has snowballed into this affair with Facebook and possibly other large companies carrying out unconsented research on unwitting subjects.

    That said, the basic premises of modern web languages are clearly dated. There has been some basic attempt to reconcile this in HTML 5 but frankly it’s just more of a mess than it was before. Slash and burn, that’s what I say. Use the current web to facilitate the adoption of a be:tter system before making it totally redundant. Better language, better fundamental back-end architecture – better everything. One day we’ll do it. :)

  9. Posted July 7, 2014 at 5:28 pm | Permalink

    I agree with you and I think that we need to disrupt the people first and then create tools that open the web up to everyone. In addition, I think we need to start teaching computer science and programming to everyone.

    The problem really is the elite attitude that the programming world has adopted over the last 20+ years. We need to kill that off. We need to open the doors to everyone and not be threatened by those that think differently. Basically, we need to grow up and not feel that we must prove we are the smartest person in the room every time someone else has an idea or implements something.

    Change this and the rest will follow. I think the idea of starting with a tool and hoping it will bring change tech is a little naive. If we change the people, the attitudes, then the tools and openness will follow.

    • Posted July 7, 2014 at 6:20 pm | Permalink

      But changing people and culture is the hardest thing to do! Change seems to happen more often by starting new communities from scratch.

      • Posted July 8, 2014 at 1:27 am | Permalink

        R. Buckminster Fuller certainly thought so: “You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.”

      • Posted July 8, 2014 at 12:15 pm | Permalink

        The only community I’ve found that has reasonably solid tech but still cares deeply about usability and teachability is Racket.

        I’m reading through How to Design Programs in order to prepare for teaching my own kids, and it’s been very promising so far. But I agree that it’s nearly impossible to retrofit caring about amateurs to a community that has evolved around professional programmers.

  10. Posted July 7, 2014 at 6:22 pm | Permalink

    Indeed. I think, or at least hope, at some point we will turn the economy of learning to use a computer for whatever arbitrary task on its head. The effort would all be on the computer’s side and we’d be able to apply things like Fitts’ law and quantified versions of usability heuristics to generate user interfaces and web pages based on the meaning of the content you want to provide. It could be a pipe dream.

    That said I think even a “better” HTML would be a big step forward. If it did what it said on the tin and wasn’t so limiting in its idioms the web wouldn’t be full of the nasty hacks that so many of us turn our noses up at. Again it’s this concept of least resistance: If you make writing good code the easiest thing to do, then that’s what will happen. It’s a challenge, no doubt. Lots of people would object to that because it overturns a lot of what they have invested in.

  11. Posted July 7, 2014 at 6:32 pm | Permalink

    I don’t know enough history to be sure. Perhaps it has to do with the demand for doing the sorts of calculations that spreadsheets allowed, combined with the fact that the “pre-existing” tool to do it didn’t exist or was too much trouble to acquire. As a kid, I learned hypercard because it was the easiest way for me and my friends to create our own entertainment. We’d buy computer/video games too, but those cost money and required a ride to the store.

  12. William D. Volk
    Posted July 7, 2014 at 6:34 pm | Permalink

    I’d be curious of your opinion of LiveCode, which appears to have taken on the HyperCard mantle.

    http://en.wikipedia.org/wiki/LiveCode

    • Posted July 9, 2014 at 11:18 am | Permalink

      The talk of Internet Hypercard is becoming a reality: LiveCode is currently running a crowdfunding campaign to add HTML5 as a deployment platform, in addition to existing mobile and desktop platforms.

      For those who don’t know, LiveCode is an open-source re-implementation of Hypercard and shares the goal of making programming as simple and easy-to-learn as possible.

      To help with this project, you can pledge at http://livecode.com/livecode-to-html5/ – any help is greatly appreciated!

  13. Ryan
    Posted July 7, 2014 at 7:10 pm | Permalink

    Many people want to go back to the good-ole-times, but sadly that time never existed. It’s a fallacy. All of those applications and systems were full of kludge and inconsistencies. The difference is that was the kludge you first learned with and loved at a time when you were implementing simple single-user single-platform applications. Not one of these platforms has ever reached the ubiquity that the web now offers.

    This article offers nothing but nostalgia for a time that never existed.

    • Gary Herrington
      Posted July 9, 2014 at 1:20 pm | Permalink

      It did exist for many many very large IT shops, and probably does for some still. Those developing there probably did not learn on single-user apps but apps used by thousands of folks (although only in one company).
      If you are interested see my (long and probably boring) history review I posted here elsewhere. The amount of IT history we have lost is remarkable, but really no one’s fault, just the circumstances.

  14. David Pokorny
    Posted July 7, 2014 at 7:17 pm | Permalink

    Imagine an Internet HyperCard…

    This is close to becoming a literal reality. There is a “PCE.js” that simulates a macintosh in your browser. So in theory, you should be able to get the original experience…now the only piece that is missing is the networking and storage, but in theory the social web should make that easy…right? Seems like it should be a piece of cake to hook that up to some kind of storage and sharing system. Of course the lawyers would just laugh…

    The icing on the cake would be the original HyperCard source code, so the users could tinker with the application itself, but I fear that is unavailable, at least for the moment.

    Link: http://jamesfriend.com.au/pce-js/

  15. Posted July 7, 2014 at 7:41 pm | Permalink

    So, I’ve been asked to cover a web programming course at very short notice, and one of the first things I had to explain to my students was that the web is how it is because of a network effect — that is, the technologies aren’t used because they are good or well designed for the purposes we’re putting them to, but because there’s such a strong economic push to build for the platform that so many people have clients for (browsers).

    Cheesy hurriedly-produced short teaching video here: https://www.youtube.com/watch?v=7DPIbtParq8

    This hurdle seems to be the trickiest to get past. There seems to be two solutions. The first is the mobile phone OS solution — brute force it by putting a new application framework on a device that you put into the hands of millions, creating an alternate ecosystem (mobile apps). The second is the Dart / Typescript / Scala.js solution where you build something that feels a little neater, but automatically compiles down to the old kludge.

    I’d be curious to know which way you’re looking at solving this (or if there’s a third way that could work better).

  16. Pete
    Posted July 7, 2014 at 9:20 pm | Permalink

    Actually, it’s only recently that the Web got so complex – original HTML was not complex to learn, as evidenced by the hordes of web developers that fueled the dot-com boom.

    I think that we went astray when the serer-side stuff started to grow – that is when it required something more than display, and so the expediency of “whatever works” snuck in

  17. Posted July 8, 2014 at 1:31 am | Permalink

    i agree to most of what you said , and would like to see a detailed discussion on this topic , one of the ways i see to fix it is with initiatives like https://webwewant.org/ , may be we can talk to larger set of programmers who understand this point of view .

  18. philip andrew
    Posted July 8, 2014 at 2:40 am | Permalink

    The problem and solution is larger than programming, the solution is to create a simulated shared universe which has rules which makes programming easy and is shared by many users. This is a digital “heaven” and requires integration of many ideas to build and revolutionary thinking, user interface, hypercard, time travel in data, semantic web, etc.
    It cannot be done easily by adding to the current system.

    • Posted July 9, 2014 at 1:40 pm | Permalink

      You should try OpenSim. It’s a ‘a simulated shared universe which has rules which makes programming easy and is shared by many users’ It’s not heaven, but certainly is one way out of the web. I spent four years professionally programming in it and Second Life, but uptake for serious uses has been slow. Why? Because ‘different’ means user training, and that’s a barrier that’s hard to cross.

  19. Danyil
    Posted July 8, 2014 at 4:59 am | Permalink

    This reminds me of http://tilestack.com/, the last company-backed attempt to bring HyperCard to the web that I know of. Unfortunately, it failed around 2011. TileStack let you build HyperCard-like stacks using a visual form designer and a code editor that ran in the browser. The stacks would then be hosted on their servers and could be embedded in web pages. I can’t comment on how well it worked for nontrivial stacks since I never got to try it out in a serious way before it shut down but people did seem to make things with it. Because it used JavaScript and not, e.g., Flash the stacks ran on the iPhone, which was presented as a major selling point.

    A co-founder of the company said on Quora that he couldn’t disclose the exact circumstances of why they had to shut down but If you wanted to speculate, however, one possible reason is that rather than sell to the business-minded customers this blog post talks about (the “COBOL programmers”) it aimed more at the creative users of HyperCard; it was an ad-supported service and IIRC didn’t provide server-side storage.

    I also wonder if the emphasis on backwards compatibility with HyperCard was a contributing factor on the technical side. Impressively, TileStack let you upload and convert existing HyperCard stacks and you could even mail in your old floppies. If a stack couldn’t be converted automatically the developers were happy to help you with manual conversion.

    Perhaps that combination was not where their potential customers could be found the time. These days Google Docs Spreadsheet is probably the closest there is to TileStack, although it’s more of an Excel than a HyperCard for the web.

  20. Posted July 8, 2014 at 4:59 am | Permalink

    If you want a solution to the web checkout applicative, functional, reactive, programming. ELM: http://elm-lang.org/ It’s the best idea I’ve seen yet.

  21. Pattern-chaser
    Posted July 8, 2014 at 11:16 am | Permalink

    “… it should abstain from the baroque complexity of OO and FP programming. Sadly, modern programmer culture fetishizes complexity as a status marker or tribal initiation ritual.”

    If that is so, then it shouldn’t be. However, I think you’re trivialising the skill required to do programming. These simplifying toys for initiates to play with are only simple because the person who wrote them made it so, with her hard work. … And skill.

  22. edmund ronald
    Posted July 9, 2014 at 10:10 am | Permalink

    Web? What about just local apps?

    Apple goes out of their way interdict any app that could turn into iPhone Hypercard.

    Software companies seem to hate any app that can spawn clean independent “stacks”.

    It’s a culture thing. Squeak never managed to make “stacks” either, you end up with something with a mediocre UI and code buried inside Squeak’s intestines.

    And the rest of the world has turned into endless ever-evolving APIs and script languages.

    Show me any clean way out and I’ll help. But no more permanently boiling pot of spaghetti.

    Edmund

  23. Joe Monco
    Posted July 9, 2014 at 10:20 am | Permalink

    The way things are today if you want to be a programmer you had best be someone like me on the autism spectrum who has spent their entire life mastering vast realms of arcane knowledge — and enjoys it.

    No you don’t. Some of the best programmers I’ve ever worked with have are not on the autism spectrum. And that’s before you get to the fact that most who will claim to be on the autism spectrum are self diagnosing themselves as a post hoc attempt at excusing why they’re socially-awkward losers.

    I hate to break it to you, but being a programmer doesn’t mean you have to be a socially awkward creep.

  24. Posted July 9, 2014 at 10:21 am | Permalink

    I think the main problem is this culture of the Übermensch which often starts in college. Most colleges only have one track for programmers and it’s computer science. Computer science doesn’t just let anyone in. You have to have top-level math SAT scores. Once you are in, you are barraged with a bunch of “weed out” math classes to further whittle the number of people down. The people left are the people you are describing. And they internalize this attitude that programming is only for people like them.

    I couldn’t get into computer science, I did economics and in economics there is this concept called comparative advantage, which posits that specialization is highly efficient. Many programming cultures just don’t have it. Programming is for the Übermensch, who does all and knows all. There is no room for anyone else.

    Other countries don’t do this as much. In some Eastern European countries for example, you can study programming if you want to study programming and don’t have to wade through 10 high-level mathematics classes. Hilariously, a lot of companies are hiring outsourcing from these countries to do programming. It’s not always great, but kind of undermines the Übermensch mentality.

    But it’s also a bad way to go about things. Have high-level senior programmers and have them delegate some work to lower level programmers. It’s more efficient and it leads to a less fragile and more diverse team.

    I am a self-taught programmer and by all means a “normal person.” I could not have gotten into a C.S. program. My math scores aren’t bad, but they are only OK and even if they weren’t I’m unwilling to tolerate the toll things like weed-out classes have on my health and social life. I have always been interested in computers, but it’s only really an accident I got into programming and it was after college. I was planning on going to law school at the time, a profession that siphons off a lot of women who have strong analytical and logical skills, but maybe not the greatest hard math skills.

    I have a niche where I am useful on a team like this. I am unlikely to ever be a senior programmer, though many OK programmers in these kind of more diverse environments move into management, which is what I’ve done. And that’s good too because then you have managers who know some programming rather than business administration graduates who don’t really understand the people who are working for them. But these environments are hard to build because the college pipeline as I described is so messed up.

    A lot of advocates of women in science go on and on about getting little girls interested in math and science. But they are unwilling to attack the elitism in the pipeline that prevents many women who are interested from moving into these careers.

  25. Posted July 9, 2014 at 10:31 am | Permalink

    I don’t think this is a problem of a lack of will on the part of developers to create easy to learn programming packages. I can think of many web-based attempts at various types of “help anyone get on the web” solutions over the years. From web publishing like WordPress to data mashing in the form of If This Then That or Yahoo Pipes, to more complex web site construction applications from Wix to Squarespace, all the way out to visual programming abortions like Blockly and the entire existence of ASP.NET and Visual Studio’s editor.

    There are always 4 huge mountains all of these applications need to scale: 1) Normal People just DON’T WANT to program, 2) With ease and flexibility you always sacrifice power and customization so eventually businesses want something the easy solution doesn’t support and we are back to scratch-building, 3) The toppling hack-stack of the web is constructed out of exceptions, the very bones of the internet are errors, so attempts to design around the damage are very difficult, 4) Security.

  26. Ben Murray
    Posted July 9, 2014 at 10:38 am | Permalink

    I’m not with you. This article is a great soap-box speech, but I don’t think it reflects the reality of developing software. On top of that, I don’t think the stereotype of software developers is a useful one either.

    Sure, most of us have joked at some point about being on the autism spectrum; I don’t know if you are saying it in that fashion or if you have actually been diagnosed with autism in some form. Even if the latter is true your position still doesn’t apply to the world of software development in general.

    One of the things you do in this article is put software development on a pedestal. Do you really think it is that different from any other sci/eng discipline? Really really? Are you suggesting that nothing else requires the unique dedication that software engineering does? How about outside of sci/eng, say medicine or law? Are these exacting disciplines that require countless hours of study and years of dedication any different in terms of the requirements they put on someone who wishes to master them?

    That said, writing production software is hard for many reasons that are nothing to do with the tools available to create it:
    1. The software is one small part the ecosystem which includes documentation and test code and defect tracking; something that most hobby programmers don’t even know about, let alone do properly
    2. Taking a hazy solution to a problem and defining it in a precise fashion is hard for even simple problems, regardless of the power of the tool-set in which you then phrase that definition. Think about user-rights for access to software features; the API for this can be as simple as you like but establishing the correct policy for your software will always be hard
    3. In order for most software to be useful, it must be developed in an extensible fashion. Given that, for the most part, the future requirements are not precisely defined, it takes a deal of experience to look at a problem and determine where your future extensibility/scalability concerns are likely to be lurking
    4. A computer has finite resources; not being careful about their use can make even simple problems fail to scale

    Now, I agree that the wrong tools can add to the burden of developing software effectively and this is particularly true of the web stack. Your article, however, seems to suggest that the right tool should make the development of software a piece of cake such that anyone who has picked up Windows for Dummies should be able to write and publish web apps. To come back to the medical analogy I’m sure that surgery has become easier with the advent of medical robots and laser scalpels but I don’t want a hobbyist taking my appendix out any time soon.

    Software development is hard because it requires precision, discipline and experience that transcend what tools are used to create the software. Sure, make the tools as good as they can be and solve the problems that people shouldn’t have to solve every time; you’ll still be left with a lot to learn in order to be a professional software developer.

    • Posted July 9, 2014 at 10:48 am | Permalink

      Yes programming is far worse than any field of engineering. I think we can quantify this, though I haven’t actually done the experiment yet. Stack up all the textbooks and reference manuals that a proficient web programmer needs to know. Include estimates for things like Rails that don’t even bother to have docs anymore. Compare the height to that for engineers in other fields like electrical or mechanical engineering. I believe our stack will be 10 times as high.

      • Joe Monco
        Posted July 9, 2014 at 11:12 am | Permalink

        What a joke. EEs need to know way more than a web programmer and far more advanced subjects. Web monkeys need to know some JS/CSS and HTML which pales in comparison to even medicore EE curriculums which require one to know advanced math, physics, circuit design, signal processing, digital systems design, microprocessor architecture, communication system design, RF engineering, etc.. And most EE degrees also involves programming in assembly, C and some language such as VHDL.

        An electrical engineer’s library and toolbox makes your web monkey pile of books look like kindergarten subjects.

        • Posted July 9, 2014 at 11:47 am | Permalink

          I disagree. You seem to be confusing theoretical depth with quantity of (stupid detailed) knowledge. A dozen textbooks that capture most of the knowledge an EE needs would be dwarfed by printing out just the docs for how a browser works. Except there are no complete docs because no one actually knows all the details. There are thousands of pages of docs that attempt to post-facto rationalize it but to actually figure anything out you’ll have to search a million stackoverflow questions.

          • Joe Monco
            Posted July 9, 2014 at 12:19 pm | Permalink

            I disagree. You seem to be confusing theoretical depth with quantity of (stupid detailed) knowledge.

            Nope, I’m not. You seem to be confusing what I said.

            A dozen textbooks that capture most of the knowledge an EE needs would be dwarfed by printing out just the docs for how a browser works.

            Because EE work never involves docs and manuals? Documentation for even simple 8-bit microprocessors is easily hundreds of pages each.

          • Joe Monco
            Posted July 9, 2014 at 12:27 pm | Permalink

            BTW to add to above, here is one such document. That is for a pretty basic 8-bit microcontroller such as used by the Arduino yet its documentation is nearly 450 pages long and far more complex than some browser document. Now when you start getting into more advanced microcontrollers, GPS devices, cellular modules, etc. the page length multiples many times over and is much, much more complex.

            Sorry, but as I said, the library of a working EE is far more complex is big than the kindergarten stuff you web monkeys deal with.

        • Posted July 9, 2014 at 3:42 pm | Permalink

          You seem to just be confirming my point. Microprocessors are among the most complicated electrical components. Their docs are comparable to a medium sized library. And that is not even adjusting for the fact that SW docs tend to be woefully ambiguous and would have to be many times larger to be as precise as hardware docs. Fully documenting how a browser works would probably be at least ten thousand pages. Unix is at least another order of magnitude. Software complexity is off the charts. It would be surprising if that were not the case, since the unique power of software is that it is soft and can be freely elaborated without limit.

          Since we disagree on observable facts this is not a productive conversation and I am terminating it here. And I don’t care to talk with someone who calls me an idiot as in your other post I deleted.

    • Gary Herrington
      Posted July 9, 2014 at 1:14 pm | Permalink

      It DOES reflect a huge reality of software development that was once the dominant way of development, and still exists in some very large shops. He is trying to point out what we have lost, something I expound on in another (probably overlong and boring) reply elsewhere.
      Your points 1 thru 4 are of course very correct, something a professional like yourself does well by pointing out.

  27. Derek P. Moore
    Posted July 9, 2014 at 10:39 am | Permalink

    Hypercard was not usable by normal people either.

    Automated speech recognition has the only hope of getting normal humans to command a computer through complex sequences of actions (or, ideally, undefined requests, “do this for me” vs. “here’s how to do this for me”), but not enough autistic people are working on writing complete grammars of the English language.

    • Matt
      Posted July 10, 2014 at 4:11 pm | Permalink

      This particular thought/idea/hope is not ever going to happen. The reason is something obvious that everyone keeps leaving out of the conversation: the simple fact that the people themselves do not actually know what they want the computer to do.

      This is highlighted over and over again during every software requirements gathering meeting and is also why Agile software development (when done correctly) is such a good idea.

      A well trained software developer or business analyst has to understand how to ask the correct questions to elicit the proper instructions from the non-programmer. The non-programmer will never be able to just talk out loud and have a computer build something for it (well, anything moderately complex anyway).

  28. greg
    Posted July 9, 2014 at 11:02 am | Permalink

    Problems that are to be solved with algorithms can be categorized in terms of computational complexity. The Chomsky hierarchy is one such example. Writing programs to solve problems at the regular expression level is easier that writing programs to solve problems at the context free level. The suggestion that programming can be easier and thus made accessible to people that are not skilled programmers is not accounting for complexity of the problem. Being a skilled programmer includes not only being able to write algorithms in a language but being able to fathom algorithms of greater complexity. There is no way to provide a language which allows people to write programs that are only of the regular expression complexity but the problems being solved are of greater complexity.

  29. John Johnson
    Posted July 9, 2014 at 11:05 am | Permalink

    <claps>What a great speech !!!</claps>

    Consider the following professions and answer yes or no how many of these positions a “normal” human being can get unless that person has “rare talents, grueling training, and total dedication”:

    – Architect that builds buildings/bridges
    – Pilot
    – Engineer that builds cars/complex machinery
    – Welder that welds ships/subs
    – Race-car driver
    – Chef
    – Animal breeder

    Sure any “normal human” can aim to be one and start working towards any of the above jobs and one day will reach them but its not going to happen on day one.

    Sorry to burst your bubble, but that’s how the society works.

    • Aaron
      Posted July 10, 2014 at 1:01 pm | Permalink

      Couldn’t agree more. Some things are hard. Harder than the average human has the capacity for (or desire to attain).

      I quit reading the article after the first paragraph.

  30. Emily
    Posted July 9, 2014 at 11:06 am | Permalink

    It isn’t that hard for people who really actually want to learn to make web apps to do it.

    And having enough creativity to actually think of something interesting to do on the computer, either personally or for business? Not to sound elitist, but I just don’t think that a lot of normal people even consider thinking about these things. They are satisfied being consumers.

    Normal people are excluded because they don’t want to make the effort.

  31. edmund ronald
    Posted July 9, 2014 at 11:31 am | Permalink

    The spreadsheet shows most people can program simple solutions for simple problems, and find a use for that.

    Now why is there no equivalent framework to construct and distribute simple apps?

    E.

  32. Gary Herrington
    Posted July 9, 2014 at 1:07 pm | Permalink

    Mr. Edwards is correct. His second and third paragraphs are something that few will ever be aware of, and most cannot have the background to comprehend.

    Few developers today have ever had a glimpse into truly massive application systems, such as exist in the largest banks. These systems can have hundreds of subsystems, EACH one of which may be the result of person-centuries of work.

    These banking systems were generally composed of IBM mainframe technologies such as COBOL, CICS, DB2, JCL, etc. It really did take extensive training to be able to use all of these together – “regular folks” probably would never be able to use – one did have to have a baseline of knowledge in each to be able to do anything really sophisticated.

    HOWEVER – the incredible complexity of many of these applications – was in the applications themselves, and not in endless layers of ever-changing technologies. The effort shifted from the techie to implementing business solutions.

    The traditional IBM mainframe was created with a then-revolutionary idea in mind: that business would NOT have to continually rewrite big costly applications, but basically use them forever, with little if any rewriting just to be able to scale them up, and also be able to hook future needs / applications into them without endless rewriting.

    So one had: stability of platform, code clarity (if the developer took or was allowed the effort) and therefore maintainability, reusability almost beyond belief, ease of application integration (across languages if needed), centralized application control, real data security (that required LITTLE effort on the developer’s part!), and really nice powerful development tools. Bright folks were definitely needed – but they did not have to be freaking geniuses, just bright and dedicated. AND, aside from the inevitable IT emergency situation, they could have lives with relatively normal hours.

    The amount of “stuff” the developer needed to know was quite significant, but once learned quite stable. There was always more that could, or sometimes had, to be learned, but it was an extension to the base knowledge, and could be learned as needed. Also, the learning curve could be more gradual – one could learn COBOL all by itself, then extend its power by learning those other application tools (online, batch, and database technologies). And ALL of these application tools themselves got continual enhancements – but again these could be learned and implemented gradually, making them more fun than burden.

    The biggest limitation to the IBM mainframe world? Management. In large organizations managers wanted to be “real managers” and not seen as “techies”. So even direct developer management was often very non-techie: good in that much of the detail app development was left up to the developer, but horrendous for at least two reasons:

    (1) Efforts to make best use of the reusability and modularity of the total tech environment would be impaired in order to “get something out in a hurry” (sound familiar?), and thus sacrifice huge potential productivity gains.
    (2) Short-term costs would rule above all – for example – a client needed a solution NOW – so buy a little Windows/Unix server packaged with some software package. Short-term happiness – but multiplied over time one wound up with a forest of little systems that resulted in horrendous integration and security costs that only showed up later.

    So why did the web / distributed technologies become the universal computing environment?

    First, isolation of the mainframe developer and technical staffs. Given the complete and mature technological environment they worked in, there was no reason for them to go out and form social techie groups discussing their work. There were a few professional societies, but these societies just were not essential to the profession. ALSO, given that mainframe technologies (or even emulators for them) were breathtakingly expensive, there could be no base of hobbyists to play with these technologies. Therefore: if one didn’t work in a mainframe shop, there was really no way to know what had already been created.

    Secondly, the advent of really cheap hardware and software enabled a whole new separate world of computing to grow up, knowing nothing of the mainframe world of computing.

    Therefore – equivalent of thousands of years of person-effort has effectively disappeared. And this effort included not only the technologies themselves, but knowledge gained from studies about actual projects, practices, and productivity.

    In hindsight, I did not see the opportunities in this situation: take some old concept, relabel it, turn it into a movement, become a guru and make one’s fortune selling books and lectures and seminars and maybe software to support it . . Fad-Driven Computing at its finest!

    • DDD
      Posted July 9, 2014 at 1:35 pm | Permalink

      I’ll guess that your mainframes ran Unix – that never went away (it’s called Linux and MacOS X now). I’ll also take a guess that these “massive application systems” all fitted into 4Mb somehow or other.

      What has changed is that we expect computers to do somewhat more than add and subtract bank balances…

  33. Jerry
    Posted July 9, 2014 at 3:17 pm | Permalink

    Hey, when we first started developing web apps in the 90′s they were done in Hypercard. Some of my current PHP libraries derive from scripts first written in Hypercard.

  34. Austin
    Posted July 9, 2014 at 3:48 pm | Permalink

    So instead of putting in the work to learn the tools of today you want other people to work again to program you something easier for you to program in. Sounds like laziness to me, sorry I worked hard to learn the skills of my trade and I put in the effort to know how to write complex web applications, you want to write applications that run on the computer for one person and crunch some numbers and maybe print some stuff to the screen fine any tool can make you something like that. You want complex web APIs that do serious server side processing to give you useful information with a modern looking responsive UI, too damn bad you have to work for that

  35. Ko
    Posted July 10, 2014 at 12:46 pm | Permalink

    I think Web/App programming is already disrupted. It’s not difficult at all any more and almost everyone can code if he/she is young to learn something new.

  36. Kelly Davis
    Posted July 10, 2014 at 12:53 pm | Permalink

    I kind of completely disagree. I am 53. I started when I was probably 14 or 15 (can’t remember). My dad bought himself a personal computer that predates everything you have ever heard about. He had to build it himself. All of his software came on cassettes and while it loaded the screen would half fill with 0′s and switch to 1′s then to 2′s all the way to 7′s and then to 0′s and then start over again at 0. I think the computer was from a company called “Denver Digital Group.”

    Anyway, the first language was in machine code (octal). Which I, at 14 or 15, had no hope of understanding. The second was Basic. My first introduction was from a magazine. I copied my first program from it, don’t ask me the name. It took eight hours. It never worked. I replaced the “$” sign, which meant string of course, with “%” because I liked it better. I got stuck on the fifth line because I kept making a mistake. I didn’t know that I only had to retype the same line to fix the mistake (no editor) at the time. I thought I had to retype the program perfectly. It only had nine lines. Eight hours, nine lines. It never worked.

    Why, you might ask, did I choose this as my chosen profession? It is easier than farming.

    On the other hand, the barrier to entry is always going to be the ‘switch’ statement. The ‘if’ statement. The ‘for’ statement. The ‘function’, and the ‘structure’. Most of all, the ‘logic.’ The barrier to entry is always going to be complexity of anything longer than ten or fifteen lines of anything.

    Software isn’t about the language or the web site. or the tools. It is about the willingness to sit for eight hours and type the same line fifty or sixty time over and over and over until you finally get it right. This is why there is a barrier to entry to normal people.

    I had a neighbor’s kid over a couple of years ago. He is a smart kid, good in school and in science. He has a really good brain. He wanted to go into Software Engineering as a career and his dad thought it would be a really great idea if he brought him over to work on an assignment with me. Now, he didn’t ask me before hand if I was okay with this. I was though, I love to talk, as can be gleaned from my typing.

    We worked together on his first Java program. I hadn’t seen Java in about five years so it took me a bit to understand what he was trying to do. He didn’t really understand it either, and what he was trying to do really didn’t work at all anyway. So, we worked for about an hour. I gave him my Java reference manual. We got his program working and he went on his merry way. I talked to his dad a week later and this is what his dad said: “Thanks for spending time with me son. He really appreciated it. However he found that hour to be the most miserable of his life and he changed his major because of it.”

    The hour I spent with his son was a calm, reflective hour in which I used the Socratic method to draw out answers for problem. If I had been even the slightest bit abusive toward this man’s son my wife, who was in the room would have killed me on the spot (she is good about that). The fact is that Software Engineer, when trouble shooting, is hard. It is very hard. It requires you to think. It requires you to try new things to understand the problem, to guess about things you don’t know, to accept that you don’t know things, and then to invent new things to try to understand the problems you don’t understand in the first place.

    There are no tools that can make you smarter than you already are.

    Which is why the author, while well meaning, is wrong. It has not been easier in the past, it will not be easier in the future. It will never be easier. More importantly, as the machines get smarter and the problems get bigger, I suspect, the problems will only get harder.

    • Posted July 10, 2014 at 3:22 pm | Permalink

      Amen. As Fred Brooks said in 1986, “There is No Silver Bullet”.

      http://en.m.wikipedia.org/wiki/No_Silver_Bullet

    • Posted July 10, 2014 at 3:35 pm | Permalink

      As Fred Brooks said, there is essential and accidental complexity. Experts become blind to complexity. A novice, like that boy, can see more clearly how much of the complexity is just stupid unnecessary puzzle solving irrelevant to the actual problem. The vast majority of programming is to solve simple problems for which the Tower of Babel we have constructed is massively overcomplicated. Learning on Python rather than Java is a dramatic simplification for novices. If all the complexity were essential as you believe this would not be the case.

  37. Elisabeth Robson
    Posted July 10, 2014 at 2:40 pm | Permalink

    I’m with you ;-)

  38. Mark N.
    Posted July 10, 2014 at 3:27 pm | Permalink

    It’s been since 2002 that my job title has had the word “programmer” in it; now I’m a glorified data scrubber (they call me a “data and program services analyst” where I work). I found the docs online at amazon for setting up my desktop at home so I could learn how to write Android apps, having recently decided that I might possibly be able to recreate an application I wrote years ago on an early Windows PC. I think that once upon a time I was a reasonably intelligent person, capable of learning the occasional new trick on my own, but I’m not so sure any more. I tried to set up my Eclipse development environment, hoping to run the requisite “Hello, world” downloadable Android app, only to discover (I think) that the video card in my desktop isn’t smart enough to be able to do the necessary emulation of a phone or other display device so that I can even get it running. I guess I’m quickly going down the path to becoming one of Rod Serling’s obsolete men.

  39. Doug
    Posted July 10, 2014 at 3:28 pm | Permalink

    What? The web is hard and arcane??
    I have worked on firmware for 15 years. I would be one of the “elite” systems programmers you refer to in paragraph 2.
    Yes, I have programming experience, and have worked at the coal face between the hardware and software. The web is a magical candy land with dancing leprechauns by comparison.
    Currently I am learning Python Django and building tiny toy example web sites. The thing that fools me is I think there must be a hard way to do things. No, it is all simple and straight forward. There is complete documentation on the Internet, tutorials, how to videos, and Stack Overflow if all else fails.

    There are tools out there that are easy to use. Multiple sites let me build my own site using drag and drop. If you are doing something amazing and different, you may be beyond the point of super easy to use tools. Because of that the entire industry is somehow broken and exclusive?

    The door stands wide open. Walk through. No, it is not trivial. Adding value may require some effort on the part of the learner. Don’t blame the tools and environment for your failings.

    The Internet is imperfect and hard because it evolved from open collaboration. It works and scales for the same reason. The same thing can be said for humans. (Star Trek moment.) Here is a proposal. Let’s get one company, say a large government contractor and have them rebuild the Internet using one language for all the bits. That will make life easy for you. All we ask is you don’t shut off the imperfect, exclusive, working internet until it is done.

  40. Walt Karas
    Posted July 10, 2014 at 3:49 pm | Permalink

    I read this, and it seems analogous to saying “people should be listening to recordings from a larger, more diverse group of musicians”. Tools matter, pianos and guitars with frets mean there are more competent musicians than if we only had violins. But tools don’t eliminate the limitation imposed by how much inherent aptitude is out there in the population. That’s why SW development outsourcing should be a far-flung global search for aptitude, not low hourly rates.

  41. James Moody
    Posted July 10, 2014 at 5:00 pm | Permalink

    Thanks for publishing my thoughts better than I’ve been able to express them. My preferred web dev eye poke is “America’s think HTML is an STD; I’m one of them” from http://time.com/#12410/11-of-americans-think-html-is-an-std/ It wasn’t until HTML5 that web tech could produce apps of the complexity of Wolfenstein. That puts web tech on par with desktop tech of the 80s or early 90s. LOL

    My hope for the future is the app store. Specifically app stores could solve the ONLY chronic problem web tech actually does better than desktop tech: deployment. IMHO if Apple/Microsoft can figure out how I can setup my own isolated “corporate app store” and/or Google can create a completive Android laptop/desktop then the web’s golden age is over. It’ll be relegated to what it does best… relaying information. Here’s hoping for the future!

  42. Graeme Martin
    Posted July 10, 2014 at 6:46 pm | Permalink

    Tools are just things you use to get a job done – which applies to software development tools the same as anything else. Pick the wrong tool and it will be more difficult, less efficient and maybe impossible. We tend to learn a particular tool or set of tools when we start in an occupation and subsequently tryt o continue to do things with the ones we know because it takes time and effort to learn new ones. There are so many tools available now it is hard to know what to try and use even if here may be better ones available.

    I started programming in 1971 on IBM 360′s using IBM Assembler and Fortran. Subsequently I’ve worked with many languages and software development tools. Many of the things that are now in APIs and various libraries we just had to build ourselves. The principles I learned more than 40 years ago still apply. Perhaps amazingly, I still program professionally at an age most younger professionals would consider to be three quarters brain dead.

    Programming is not so much a matter of what tool you may use but how you can apply it to the task at hand. Many tools available now have a lot of ‘black box’ stuff available that means you are liberated from many of the mundane tasks that apply in creating a system. This now requires a lot more knowledge than in previous systems but there is no change in the capability required to use the tools competently.

    In any sort of complicated system you still need the kind of mind and dedication that has been described by many of the previous posters. This is not for everyone. There will always be that elite that can really do these kind of things well, those who are also rans and those who shouldn’t be doing it at all because they are just not smart enough. “Programmers” don’t need to be the autistic kind but they do need to be able to apply a high degree of logic, concentration and dedication if they are going to be efficient. This does breed a kind of arrogance, especially in the younger group just as it did 30 and 40 years ago, though I think it is worse now.

    There are many useable systems that people can pick up to create things that may be useful but often they will be tools which are misapplied. This particularly may apply to web apps because “that’s the current thing”. Even the good old spread sheet is so often used (misused?) by untrained people who may be very proud of what they have done but which end up being a costly hodge podge which an efficient professional “programmer” could have done much more effectively and securely using a more appropriate tool.

    It will always be thus. Each generation produces it’s own systems building on what was done previously, often without any detailed knowledge of what went on before. New tools will come and go. There will be good ones and bad ones.

    The programmer is dead! Long live the programmer!

  43. Mark
    Posted July 10, 2014 at 11:18 pm | Permalink

    While I have seen this in interviews, where I work, if you are a monkey with a typewriter you are over qualified.

  44. Posted July 11, 2014 at 4:34 am | Permalink

    Fractalide: A HyperCard for the Internet of Things. HyperTalk is swapped out for a Flow-based Programming environment. Fractalide aims to be the main development environment for the IOIO-OTG board and is built on Mozart Oz. Consider this project ready for language enthusiasts and early adopters.

    http://github.com/fractalide/fractalide

  45. Nardev
    Posted July 11, 2014 at 9:24 am | Permalink

    I see your point, it’s been lurking in the back of my mind – the answer lies in organization, yet organization slows innovation – solve that puzzle and we will have the right way.

  46. Posted July 11, 2014 at 9:30 am | Permalink

    I agree with Jonathan.

    But netherless , there are some tool that attempt to facilitate web development.

    Have a look, to the WinDev suite. http://www.windev.com/

    It is a development tool existing since the 90′s. First it was for oriented for Client/server application and then moved to the web development.

    It is not perfect. But we can do lot of thing very easly.

  47. LaughingSkeptic
    Posted July 11, 2014 at 1:05 pm | Permalink

    I am sure you are one of those people who dismisses Microsoft as the evil empire since pretty much all of your arguments ignore the application development tools that Microsoft provides. You seriously need to take a look at the Microsoft world once decade and stop being so proud of your collection of arcana that is obviated by their toolsets. I know a number of soldiers, who with no training, set up SharePoint servers for group collaboration. I know secretaries who have created business line applications using Access, Excel and Word. I know artists who have become adept at Expression. Your religious approach to your career has blinded you to just how wrong this article is.