The problem with programming and how to fix it

Programming could be much much easier most of the time. This unnecessary difficulty wastes the effort of professional programmers, but worse, prevents many people from programming at all. The problem is a nasty hairball of culture and technology and money that is not easily untangled. I’ll try to address this essay to a general audience.

Programming today is exactly what you’d expect to get by paying an isolated subculture of nerdy young men to entertain themselves for fifty years. You get a cross between Dungeons & Dragons and Rubik’s Cube, elaborated a thousand-fold. I’m serious: Dungeons & Dragons requires mastery of an encyclopedia of minutiae; Rubik’s Cube is a puzzle requiring abstract mathematical thinking. That is programming in a nutshell. As a bonus you also get a toxic culture of intellectual one-upmanship.

There is something unique about software that exacerbates these problems. Unlike every other technology, software doesn’t wear out. The traditional cycle of replacement and renewal is halted; there is no evolution. Once a software technology gets adopted it lives forever, enshrining every mistake and obsolete design decision. It is always more expedient to layer more software on top of the obsolete than to redesign and replace it, and so our Tower of Babel grows ever higher. At this point to be a “full stack” developer you must master the content of something like a hundred thousand pages of documentation. Can we keep adding more layers forever? When do we start throwing obsolete things away? Whose responsibility is this?

I don’t blame the kids. Ultimately the problem is a failure of our institutions. We need senior leaders to band together to invest in the long term health of the field. We lack such institutions to a shocking extent. Why is there no American Programmers Association, like what all the other professions have? Instead we have only the Association for Computing Machinery, which is run by and for academic computer scientists, paying only lip service to practitioners. Compounding this bias, academic computer science has become estranged from practice. The stereotypical attitude is that professional programming is such a mess because programmers are stupid. There is some truth to this — most people are stupid compared to computer science professors. But apart from eugenics, isn’t the solution to design technology that doesn’t require a PhD? And maybe it would help if those computer scientists had more actual experience of professional programming. Unfortunately there is no path for practitioners to enter computer science more than a couple of years post graduation. Why is there no Professor of Practice as in other fields? The studied irrelevancy of computer science is a big obstacle to progress, because it absorbs and dissipates the funding from policy makers and philanthropists trying to help.

What of the #CSForAll movement? I have mixed feelings. The name itself betrays confusion — want we really want is #ProgrammingForAll. Computer science is not a prerequisite for most programming, and may in fact be more of a barrier to many. The confusion of computer science with programming is actually part of the problem, which seems invisible to this movement. Instead they are focused on education and social engineering, which has a bad track record for solving knotty cultural problems. I hear talk of equity and fighting toxic culture and offering nurturing learning environments. These are all good things, but I don’t think they are enough, because the cultural problems have become baked into the technology itself. No amount of coaching and cheerleading is going to make most people enjoy spending their lives playing Dungeons & Dragons and Rubik’s Cube. Why is making the technology itself more humane off the agenda? Because that would be too hard or too disruptive? This movement wants to believe we can quickly fix the problem without upsetting any apple carts by just throwing more money at the entrenched institutions that should have prevented the problem in the first place. Good luck with that.

What reason is there to believe we can improve programming technology to be vastly easier and more accessible? To start with, spreadsheets. Spreadsheets are by far the most popular programming tool, used by far more people than “real” programming. Surely it must be possible to carve out other domains like that of spreadsheets without taking on the full complexity of programming. In fact HyperCard was just that, enabling millions of novices to build simple applications. We can also look to the many incarnations of Basic, designed for ease of learning, and particularly Visual Basic, which was a serious business programming language. Visual Basic was an example of “application programming”: using languages and tools designed for simplicity of learning and use in solving relatively pedestrian problems. I recall that application programmers had a more welcoming and supportive culture than the “systems programmers” that looked down upon them. Unfortunately application programming got trampled in the internet gold rush. Suddenly all that mattered was building large-scale systems as fast as possible, and money was no object, so the focus shifted to “rock star” programmers and the sophisticated high-powered tools they preferred. As a result the internet age has seen an exponential increase in the complexity of programming, as well as its exclusivity. It is long past time to return to designing tools not just for rock stars at Google but the vast majority of programmers and laypeople with simple small-scale problems. We’ve done it before so we can do it again, even better this time. The problem is funding. Academic computer science doesn’t reward the kind of work required to build high quality tools (it’s “just” design and engineering). The big internet companies aren’t interested because it doesn’t give them lots more eyeballs. The venture capitalists aren’t interested because there is no hyper-growth play. Open source development has succeeded mostly at solving the problems of expert programmers. This funding barrier is a big reason we need new institutions whose interests are aligned with that of all programmers, expert and casual, present and future.

In conclusion, to bring programming to everyone we need to fix both our culture and our technology. This is best done in fresh institutions outside of computer science, institutions dedicated to improving the practice of programming, but even more dedicated to bringing the power of programming to everyone. At long last society at large is awakening to the inequities caused by the problems of programming. The time is now ripe to fix programming.

See also

Alan Kay. The inventor of object-oriented programming and grandfather of software contrarians.

Richard Gabriel. A founding father of LISP; author of the influential “Worse is Better” essay; has proposed a Master of Fine Arts in Software.

Ellen Ullman, “Close to the Machine”

Nathan Ensmenger, “The Computer Boys Take Over”

Bret Victor, Dynamicland. A new human interface to software.

Chris Granger, Eve. A moonshot to make programming accessible to everyone.

Evan Czaplicki, Elm. A humane programming language and community.

Brad Myers, CMU. Natural Programming Project

Adam Chilapala, MIT. Another point of view: math for the win.

Education and equity: Margaret Burnett, Oregon State; Mark Guzdial, U Mich; Andy Ko, UW; Philip Guo, UCSD.

My pitiful research.

TODO

Agenda of a Programming Institute

The myth of essential complexity.

The limits of “no code” and “low code” application generators.

The siren call of dataflow.

Tear down this wall: the cost of specialized software components/domains.

The great schism: do specs exist?

Making computer science weird again.

Thanks to Tomas Petricek for helpful comments.

 

49 Replies to “The problem with programming and how to fix it”

  1. Always a pleasure reading your thoughts!
    I hope there is a part 2 of this post, with more information on how people interested in changing the status quo can help.

    Best.

  2. There are many tools to teach programming to novices, stuff like scratch, Blockly or GameMaker. Even languages like python/visual basic still exist and are more approachable to people. However Computer Science as a field is much bigger than just programming….It involves understanding stuff like data structures/algorithms/databases and even stuff like networks. How do you simplify such stuff for novices because to get a holistic view of Software Development…you need to know Computer Science. Otherwise you just end with Quack developers who get stuff to work without understanding how it actually works and why it works like that

  3. Good article, but I missed some ideas how to fix it. #ProgrammingForAll, but I’m sure you don’t want to resurrect Visual Basic.

    1. Probably HTML since has to be “inclusive” to all IQs and attention spans, or lack of passion forced diversity students

    1. Identified, but not well-defined.

      Which, in my experience, is the majority of the problem with typical people trying to define what they need from IT – let alone attempting programming of their own.

      If he was serious about this problem and effort, the author would need to enumerate all apparent problems, their contexts, and how they interrelate with technology, humans, and the current state of the industry.

  4. Talking(or dreaming) is cheap. Writing in a blog about what the world should be is 6, 7, or 8 orders of magnitude easier, faster and cheaper than actually doing something about it.

    We could agree that nuclear fusion should power our cars,phone batteries should last for years, cancer or AIDS cured with a pill, aging replaced with eternal youth.

    To actually get to solve those issues takes a radically different skillset, work, and over all, (painful)failures.

    There is a lot of people devoting their life, and risking a lot (like relationships,time, status and personal money) trying to improve programming. Some of them will do. Others will fail.

    Because of their limited success(for now) they look as they do not exit. If they success they become the “new normal” as people get used to them, like a person taking a plane or powering on the light without thinking about all the effort it took to get there.

    It works like an electronic amplifier in reverse. Our dreams come as an input to the system, they come out as real things as output diminished a million times or so as they face reality.

    It seems to me that if we want to actually do something about that we should focus less on what the ideal system should be and more on how to make the system diminish “just” three orders of magnitude or so. Then do the tremendous work to make it happen.

    PS: I am working on improving programming too.

  5. Like many already said, it’s easy to whine about something, it’s harder to think about real solutions. I don’t think programmers like the current state of programming, that’s why many chase the new shiny thing.

    Spreadsheet are easy because they only use a VERY limited subset of programming constructs. If programming was all about that subset, students would be able to learn it in less than 10 hours.

    I’m a college teacher. To fill my teaching task, I often have to teach Excel to people not in the computer science field. As easy to use as Excel can be, some students just don’t get it and can’t use it correctly. The conditional function is often confusing to them. So I’m not sure Spreadsheets are a good example.

  6. I’ve long taken issue with the CS curriculum at most colleges. The sad truth is that such a program simply doesn’t prepare a person to become a programmer. If I had an employee who spent company time creating their own implementation of QuickSort, for example, that employee would probably be terminated, yet that’s exactly the kind of thing that CS seems to prepare people to do. Further, most CS academics I’ve known and read are terrible at the art of crafting “good” (readable, maintainable) code. Most of the best programmers I’ve ever known didn’t have a CS education. Focusing only on theoretical performance bounds and not paying attention to basic learnability and usability is a huge detriment to everybody involved.

    The idea of a professional society for programming practitioners has some serious merit. I’d join such an organization

    1. > … creating their own implementation of QuickSort…

      But that is apparently the only way to become employed.

      Oh: besides being an intricate part of the curriculum talking about big O estimates and complexities, comparing algorithms, and more aspects of logic or even the implementation language(s). Which are important to programmers – even if those parts may not be important to developer or code monkeys.

  7. I like the article, thank you for making us think.

    How large and how interconnected (eg talk to database, other programs etc) should the tasks be for which you envisage that programming should be made easier?

    I think that programming should be made harder, in order to be correct. Take the spreadsheet example: it is easy to write spreadsheet programs, but they are often wrong.

    But I admit that I am not a practitioner; I am a programming language researcher.

    1. Hi Sophia. I see multiple dimensions for making things simpler. We are carrying around a ton of historical baggage that is straightforwardly fixable if we only had the will, like the insane complexity of the unix commands, and the usability disaster of git. Then there is more speculative research into unifying our Balkanized technologies, for example a PL-DB unification that would avoid the massive complexity of impedance matching. There is also promising work on DSLs.

      Your point about correctness cuts straight to a deep disagreement between theory and practice. Many practitioners think correctness is a theoretical ideal inapplicable to most real world problems. My slightly more nuanced opinion is that if we can dramatically simplify the expression of a solution we will also be eliminating many opportunities for error. In your spreadsheet example, if we were to replace those spreadsheets with custom written C code I would expect far more errors, just because there would be more moving parts to break.

      1. “…If we were to replace those spreadsheets with custom written C code I would expect far”
        fewer analyses and reports.
        Spreadsheet programs lower the resources and skill levels required to achieve the analysis. They allow for self service, reducing time and people communications costs between a person with a reqiurement and a person who can write a program . Spreadsheets enable the person with the need to iterate through many alternatives, and additional analysis, until they eventually ask the right question.
        Yes, this facilitates analysis paralysis.
        Yes, it allows producing erroneous and bogus results.

  8. The gist seems to be, “no love for applications programmers.” We do have languages that are “simplified,” but the problem is that to do anything complex, the language also must become complex. I learned with interpreted BASIC, and then learned COBOL, and had a part-time job in high school writing COBOL.

    COBOL, in its original form, was “a language where a clerk could write the program, and could be understood by the manager.” However, real needs required concepts more complex than simple addition and subtraction, so the language became a monster. Now we have object-oriented COBOL.

    There have been, rather, are, many languages created for learning to program. Take Logo as an example. Created in 1967, it has a penetration in the classroom of, what, next to nothing?

    The problem really isn’t about programming is only for toxic nerds. The problem is that even “simple” applications are not simple, and it really does take abstract thinking and extreme attention to detail for things to work properly. Yes, there are many simple programs that can be written. But then the features aggregate, and it does take someone who can hold both the universe and the grain of sand to provide a good solution.

    Thems be the breaks of that thar game.

    1. Don’t worry about the academic sounding terminology. #CSForAll is not teaching Computer Science, but it isn’t teaching “Computer Literacy” either. I think the goal of the name is to make it clear to all audiences that it is not about wasting time and money on worthless “Computer Literacy.”

      When I was in junior high school, boys took shop and girls took cooking and sewing. This wasn’t vocational school. The goal was not to make us carpenters, electricians, chefs, and seamstresses. It was so we could do some things on our own, and have a clue when to call and how to speak with a professional tradesman.

      What we are talking about here is enabling more people to do self-service personal solution creation or assembly. This will lead to people being able to program their DVR, thermostat, crock pot, online banking autopay, and If This Then That on their phone. They are not expecting to be doing this as professional programmers, but as graphic artists, musicians, homeowners, and professionals in any domain other than programming. The results are not expected to have the Qualities of Service of commercial products, and certainly not of life and health critical systems.

      It is terrific when self-service amateurs and professionals can exploit end-user tools, domain specific frameworks, and libraries that are created by experts in a domain, maybe with a higher level of skill, and with professional programmer involvement. That’s what happened with spreadsheets. That’s what is happening now with python and to a narrower degree with R. Learn python once. For every domain you step into, find the appropriate library or framework so you can focus on the problem with the aid of higher level functions to use rather than write. It is also the goal of the much younger Red Language effort.

      Some of the problems in the original post and its replies come from terminology, labels, definitions, and turf: Programming, Computer Science, Coding, … Automation, Scripting, Recipes.

      Turf? Yes. Management Information Systems (MIS) and Information Technology (IT) are in the Business School’s turf. Computer Science is either in the Engineering School, if it includes hardware concerns and grew out of the EE department, or maybe the School of Arts and Sciences if it doesn’t, and grew out of the Math Department. Designing a multiuser database server is Computer Science. Designing a manufacturing system database schema using it is MIS or IT.

      Nowadays there is much more than Systems vs Applications: Utilities, middleware, developer tools, operations/deployment/orchestration, management, cross industry, industry vertical, usage accounting (especially for cloud), embedded & IoT, etc.

      Another big change in the enterprise is the transition to running customized commercial software products rather than home-grown applications. The Independent Software Vendors have efficiencies of scale. These permit them to hire more technically skilled developers for programming and business domain experts for requirements and usage.

  9. Agree strongly with a lot of this.

    > Suddenly all that mattered was building large-scale systems as fast as possible, and money was no object, so the focus shifted to “rock star” programmers and the sophisticated high-powered tools they preferred. As a result the internet age has seen an exponential increase in the complexity of programming, as well as its exclusivity.

    Yes, and large scale systems are now fast becoming unwieldy, even with the high-powered (but ‘mircoscopic’) tools. An incredible amount of effort is wasted by the bureaucracy in the systems we’ve created. The sub-subgoals have become the goals so even the rock stars doing sophisticated work only work with a tiny window view of the system – peephole optimizers of a kind.

    > the cultural problems have become baked into the technology itself.

    > I recall that application programmers had a more welcoming and supportive culture than the “systems programmers” that looked down upon them.

    Sadly this sentiment is still present today and is the key point, I think.

    Can we have pedestrian, everyday programming that integrates well with the more advanced, professional programming? I was thinking about home improvement projects I do as a non-professional – they are small in scale but integrate well with the professionally built stuff. I learn a little more with every project and may graduate to do more advanced things too. Can we create a programming substrate that provides a similar learning slope? I hope so. But yes, we may first need to create a culture from which such a thing could emerge.

    BTW, really looking forward to “The myth of essential complexity.”

  10. Upon reading some of the other blog posts, I see a recurring theme: programming has become a closed society, open only to a select few.

    Have you ever worked with Mr. Peepers? That character from Saturday Night Live. There’s some videos out there, so go take a moment and have some fun.

    Ready? Good.

    In one of my prior jobs, a company (now out of business) had a number of Mr. Peepers. Is Mr Peepers really supposed to write code, or manage a software project? Mr. Peepers cannot write software. Mr. Peepers cannot manage a software project. Mr. Peepers is very, very active, jumps around, makes noises, and reacts when his “personal space” is “invaded.”

    Mr. Peepers thinks he can write code. Mr. Peepers thinks he can manage a software project. Mr. Peepers doesn’t fail, because he doesn’t understand failure. He doesn’t understand success, either.

    What to do about Mr. Peepers? Is there a way to keep away from Mr. Peepers?

    When Mr. Peepers is managed by another Mr. Peepers, what can be done in that situation?

    There is no solution to changing Mr. Peepers. What you see is what you get. You cannot apply any stimulation that might possibly change Mr. Peepers’ actions. Mr. Peepers will not understand any words you say to him.

    Now, how do you keep Mr. Peepers out in the first place? Um, would it have to something to do with building a castle, with a moat, with a drawbridge, a maze after the entrance, and finally a clique of devotees? Can you say, “defensive mechanism?” There, I knew you could do it!

    Go ahead, devise a programming language for Mr. Peepers. But please, keep him with you, OK?

  11. I think you are wrong overall.

    True, there exists this toxic “rock star” programmer attitude, but its incidence isn’t as big as it seems if you’re looking around the bloggosphere. I consider that more a symptom of the real underlying problem.

    And yes, there are lot’s of similar problems like the “bad management” problem which is often the reason for horrible code bases. But that’s again a symptom and not the disease.

    The real problem is that it’s hard to write big applications. That’s it. To tackle this problems programmer invented all kinds of solutions: Lots and lots of programming languages, huge numbers of frameworks and libraries and to manage all this tools, tools and more tools. The result is the situation we’re in today and which you’re criticizing here.

    But you’re barking up the wrong tree, because the situation exists exactly because programmers want so solve the problem. They use different approaches as you to (for example creating a new framework instead of trying to find new programming paradigms), but in the end most of them try to solve the same problem as you do and try to make programming more easy. And they even had some success, if you are honest, you have to admit, that it’s really easier than ever to build complex systems. But yes, you need to learn lots of stuff first and jump thru lots of hoops to get there.

    I’m following your experiments for some time now and you had some good ideas. But there was one think I’ve always missed from your presentations: Showing that your ideas scale. And I suspect that’s the reason why you haven’t created something really useful until now, because you noticed that too: Creating nice but specific solutions of a small problem domain is not to hard, but scaling them up to be able to create big systems is. And so you abandoned your experiments, trying something else.

    I’m working an similar ideas as you do and I’ve also created lots of prototypes. And the main thing I’ve learned is that it’s extremely hard to scale such prototypes and make them general enough for general purpose application development.

    So it’s not some big conspiracy of rock star programmers who want control the world by creating tools only they can understand, it’s simply that the best modern tools really work. Yes, not perfect, not easy enough, but at least they work and they do scale.

  12. Great article, I think you’re addressing important topics here.
    Personally, I think there are two powers in play here:

    1) developers tend to get addicted to complexity
    2) customers take invisible complexity for granted. They think in buying X ‘screens’ for X dollars 🙂

    Developer(companies) could try to mature their businessmodel.
    Customers should realize that software development is a service, not a product.
    Selling fixed-price software, is selling the illusion of 24/7 bugfree software.
    I’d ??? to see companies which put expiration dates on certain software-components, obligatory periodical database-maintenance etc.

  13. I agree with a lot of this. Programmers have a long way to go with our training and tooling before we can really call ourselves ‘professionals’, as doctors, mechanical engineers, and architects do. But I don’t quite see this ‘application programming for all’ dreamland the author seems to want becoming a reality. What analogous transformation has there been in other professions? I don’t know much about it but I imagine a lot has changed in the training and regulation of doctors, architects, and mechanical engineers, etc, over the last few hundred years to make them much more systematic and less error-prone. You don’t see many buildings spontaneously fall down through poor design and if a doctor really screws up they tend to get fired. However, you don’t often hear about people architecting their own buildings or performing surgery on their family members.

  14. Cool post, Jonathan. Though I suspect some well-aimed trolling here. I’ll bite.

    If we are serious about democratizing programming, and obviously you are, we should be following the patterns employed by software that end users can actually approach and use without reams of complex documentation. Namely software built on top of a database.

    The most glaring distinction between what developers do and what “normal” users do is that normal users edit data in a database through a UI, but developers do… a bunch of other stuff, usually involving a command line or a text file and a blinking cursor. I’m not saying syntax-based UI is bad, often times it might be the very best interface for an experienced developer, but it is generally not very approachable for the beginner. Editing data through a UI clearly can be.

    The relational model has proven itself to be an apt abstraction for modeling countless domains it in the real world. There is nothing special about programming, it is made up of logic and models and constraints just like every other industry. We can model it just the same as some company’s business logic. The one-two punch of database and frontend has proven itself over decades. We are really good at making UI against a database. Why not turn all this expertise on the problem of simplifying programming?

    I sure understand why developers don’t want to spend their days on the psql prompt. Databases, for all their utility, have historically pretty much been awful in terms of developer experience. Yet the inconvenient fact remains, they dominate the world of user-facing apps, no matter what UI framework you use.

    My project, Aquameta, is trying to model the web stack as relational data, and then build a user interface for it. Though it’s far from finished, the model we’ve developed is pretty sound and the user interface is starting to take shape. Under the hood, doing programming is always (ALWAYS) either a call to a stored procedure, or some form of data manipulation, either SELECT, INSERT, UPDATE or DELETE against a clear normalized relational model that roughly mirrors the web stack we’re used to.

    Strict adherence to this first principle has led to some really challenging design problems, most of which I believe we have solved, and also some remarkable and unexpected benefits. To mention just one, Aqaumeta’s version control system operates not in a file-centric world, but in a row-centric world — it version controls rows in the database. Because all the layers of the stack share the same information model, we have a concept of “meta-modularity”, where most layers in a typical web stack have their own plugin system that is limited with that layer’s specific domain of concern. You can mix and match schema, data, functions, widgets, security, into a single version-controlled “bundle”, making vertical modularity a reality. Also building UI against this data model is like falling off a log. Allegedly very easy.

    I guess my question is, if you’re trying to make programming easier, why not follow in the footsteps of all the apps that mere mortals can actually use?

    1. Hi Eric, it’s great to see you’re still working on aquameta. Have you recorded any demos of the programming experience? I did make a stab at meta-modeling code in my Chorus work, and it was kind of a disaster. Very confusing, utterly unreadable. I’d love to see how you’ve done better.

  15. Substitute “genetic engineering” for “software engineering” and you’ll see how vacuous the claim is that amateurs should be able to write software without training. The knowledge gap (and dangers inherent) in building Excel macros vs operating system kernels are of similar scope and size to breeding dogs vs gene editing. Some things require substantial expertise.

    1. This common reaction makes me think we need to revive the concept of “application programming” to refer to the 95% of programming which isn’t OS kernels and other infrastructure. Most programming really is more like dog breeding than genetic engineering. Even better would be to coin a whole new term other than “programming” but I haven’t discovered one.

  16. I’ll say, I’m having a real tough time of getting back into programming because not only have the languages changed over time, I just don’t have the time to grasp the seemingly forty different things I need to learn to create a simple app.

    When I was learning to code as a kid, I had a Cyrix 6×86 and SAMS Teach Yourself C++ in 24 hours book, with Borland C++ 4.0, no internet / nothing, and my computer was located down in the furnace room. I only needed to learn 3 or 4 things to make an application, and it was easy enough to grasp programming concepts.

    Today I’m learning Golang, it’s a great language and what really draws me towards it is it’s standard library that does almost anything in a very concise and idiomatic way, it reminds me a lot of sitting in the furnace room as a kid, enjoying coding again. Then I decide, hey, I’d like to make an app.. so I’m researching it and now it’s learning jQuery, HTML5 — maybe Angular or some other framework; and now I’m told that WebAssembly is coming down the pipe and it’s supposed to solve all of this.

    I’m not a professional developer, a hobbyist and tinkerer moreso; but I do remember enjoying coding a lot more as a kid. I find now I have to spend a lot of time creating my development environment, tinkering, figuring out library versions and stuff.. way more complex.

    It’s a total barrier to getting people into coding, and I have been exposed to code before, so what of people that haven’t?

    1. The web is an insult to professional programmers. CSS lacks variables because fuck you. Why is there no institution speaking for the needs of programmers who have to eat this shit every day?

      1. Actually, modern CSS does not lack variables. They were added in a few years back.
        Most modern browsers (not Internet Explorer) have supported them for at least a couple of years.

        Please see https://developer.mozilla.org/en-US/docs/Web/CSS/Using_CSS_variables

        Not saying that CSS is perfect (because it’s definitely not), but it is much better now than it was in the past. For example CSS grids – see https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Grid_Layout

        Modern browsers also support ECMAScript 2015 (a.k.a. es6) which IMHO was a vast improvement over previous versions of JavaScript.

        And as for an institution speaking for the needs of programmers, I would recommend you take a look at WHATWG – https://whatwg.org/faq

        The web today is not what it was even a couple of years ago. Currently is is quietly but inexorably undergoing a rapid evolution.

        It is just unfortunate that there are still a lot of sites living in the proverbial stone age.
        But that’s OK. There were also a lot of winners and losers back at the end of the real stone age.

        As to whether web programming is “much more accessible”, well no, not as such. But I believe it is now at least possible to build a framework on top of it that has a real shot at it.

        I believe it enough that I am currently working on something along those very lines.

  17. Computers are tools. Tools are for making things. most available programs and apps are prefab houses someone built and will rent to me for my use. i don’t own it, and I can’t modify it. But what if I want to build my own house — or even my own table?
    Thirty-five years ago I built a document assembly system using WordPerfect macros. I built a complex docket program using DBase III. I built, using Basic, a flashcard program for learning a language.
    I controlled the product, I could modify the product, and not least, I had the pleasure of living in a house I had built.
    When I learn a new language — Italian, or Chinese, or Turkish – I discover a new world. Today’s programming languages don’t open doors for me — they bar the gates.

  18. First off, where does the need “to bring programming to everyone” come from? Democratize computing? Especially on a professional level. I can’t think of a single other subject where that is so. Math? No. Writing? No. Chemistry? Art? Music? Accounting? All no. Also, each is a common subject taught in schools, but not for everyone and not everyone takes it to a professional level, requiring membership in a professional society.

    Second, it sounds like you are confusing programming and design. Obviously related topics, but different.

    Third, there ARE programming associations. Several. There’s even one named the National Association of Programmers. How is that different from “American”. And why the heck is this a US issue?

    Fourth, lots of programming has nothing to do with the web. How did that creep in? Anyone have any metrics on that? Also, web programming is different from network programming in case you are lumping the two together.

    Fifth, not all programmers embrace game culture. Personally, I’ve never found any game as interesting as “Build an app or a service or control a device, etc.”. Now that’s a great game. You make it sound so much harder than it really is. Some days things are “easy”, some days it can drive you nuts; like most things in life. Balance is the key. If it’s always too hard, something else may be amiss other what you are imagining.

    Sixth, anyone got any accurate and recent numbers of how many “programmers” come from a CS background? My guess is that the percentage is shrinking over time. No cheating and lumping in other scientific disciplines like Math or Engineering, as those departments are not all pushing the agendas you ascribe to the academic stereotypes you mention. If the professors you work with or learn from are devoid of real experience, then I recommend you take a serious look elsewhere.

    Lastly, it sounds to me like you just want money to “build higher quality tools”, whatever the heck that means. I know it’s old-fashioned CS dogma, but good requirements often makes programming much easier.

  19. I agree with much of what you write and, despite being an “estranged from practice” academic, I’m working on fixing it. Like you, we’ve looked at the broad use of spreadsheets and thought about how to bring that to web application development. What’s missing from spreadsheets is the ability to craft the kind of user interfaces people want in their applications. So we’ve created Mavo ( http://mavo.io/ ), where you build your application by writing its html skeleton, marking the data items on the page, and then adding spreadsheet formulas that refer to the data items. Mavo automatically provides editing interfaces for the data, updates the formulas whenever the underlying data changes, and stores the data for you in dropbox or github.

    We’ve got various demos at http://mavo.io/demos which I believe show that you can build some nice applications with this paradigm. For example, a typical todo list app can be authored with 25 lines of html to which you add about 5 data attributes and 5 formulas.

    I think it’s important to recognize that computer programming can be used to build extremely complex artifacts, and that doing so will be complicated no matter how good the tools are, because just thinking about them is complex. What amazes me is how complicated we make it to build simple things; I don’t think that is necessary and I hope Mavo demonstrates a way to keep simple things simple.

    1. Thanks David, Mavo is great! I also like the work of your students Ted Benson and Eirik Bakke. So let me ask you, do you feel it has been difficult to publish such research? Do you think we should have a venue where such work is more welcome, where we don’t have to force it into a HCI or PL frame?

  20. It’s been pretty easy to publish. UIST, the symposium on User Interface Systems and Technology, is a natural venue for research on systems that have an important HCI angle. Both Benson’s work and Mavo have appeared there (with a new Mavo paper coming next time), with most of our work getting accepted on first submission. UIST is home to an active “End User Programming” community (and more recently “End User Software Engineering”), that thinks about questions like those you’ve asked. I don’t think such work needs to be “forced” into an HCI or PL frame; it’s naturally there as an interdisciplinary form that draws from both. I sense that HCI is more welcoming of that interdisciplinary work than PL, but admit I haven’t tried much with the PL community. It’s true that these tools might be a harder sell at CHI, which is less focused on systems and more on users, but that’s why UIST was founded.

    1. Can I ask for some advice, David? Let’s say that Mavo, instead of building on HTML/CSS, replaced it with a new language that was much simpler and easier to learn – supposedly. How would you go about evaluating that claim, as is required for publication? It still might take days to learn this new language, and the control group would have to be complete beginners learning HTML/CSS, which would take even longer, so it gets to be like a clinical trial in medicine, which is impractical. Have you seen any clever ways to do such evaluations easily?

      1. Jonathan, I think that trying out a whole new and different programming language *is* a lot like a clinical trial. It’s something that is going to take a significant investment in time, and that you therefore do not want to do until you have quite a lot of confidence that it’s going to work. You can build that confidence by trying out the different ideas that make up the language in isolation. So for example our latest Mavo paper did a 90 minute user study focused entirely on data mutation, and our next one may focus entirely on e.g. access control. These are like doing the biology lab experiments that precede a big clinical trial.

  21. Also coding a long time. I went the c programming route to do major projects. I preferred things like Hypercard, though.

    Today, I use Wolfram Language, which has most of the advantages of Hypercard, plus it won’t be orphaned any time soon.

    This does not solve the toxic atmosphere, though.

  22. Hi Jonathan,

    Interesting article. I agree with the general sentiment. My own answer to this, which might interest you as well, was something called “concept programming”. The idea is to focus on the translation of ideas into code. The following presentation gives an outline of what this means and a few early results: http://xlr.sourceforge.net/Concept%20Programming%20Presentation.pdf

    From that I derived a concept programming language called XL, https://github.com/c3d/XL. Which keeps evolving too fast to ever stabilize (I’m still trying to make it “as fast as C”). Two semi-stable variants emerged, however, one called Tao3D for interactive real-time animations (http://tao3d.sourceforge.net), one for distributed programming and the Internet of Things (https://github.com/c3d/elfe).

    Both variants demonstrate, technically, how well the concept programming approach works, and how it addresses some of the questions you ask in your post. Unfortunately, nobody cares. None of these languages ever reached a “good enough” status, i.e. a status where you can really make a living out of programming them.

    I’m still working on this, though, and I still believe that the original idea is sound. It just needs more focus on execution, ironing out all the details (e.g. having a complete runtime support library), building a community, etc, things I never really had enough resources to do well enough.

  23. Programming today reminds me of the ornamental clutter pervading so many homes in the 70s to the 90s. No surface was free from some curio, knick-knack or souvenir – some useful, some symbolic, some just for entertainment.

    Programming needs its own minimalism movement to regain clean mental work surfaces — free of clutter, with necessary sophistication baked into the tools that are visible. But the approach that takes us there may not be the most obvious one.

    One of the better ideas I’ve seen is ‘webMethods’ – a Java-based graphical programming IDE created back in the 90s. A ‘statement’ in webMethods Flow is not a line of text – instead, it’s a 2-D canvas graphically showing all the in-scope stack variables. “Hold on”, you say — “this way be clutter-dragons”. But wait: there is a method to the madness. The graphics mean variable types (objects, strings, arrays, etc) are instantly recognisable. The canvas lets you explore object variables and their members (collapse, expand, set values) and transformed them directly in the canvas itself. So a single ‘statement’ has become an implicit group of graphical operations under one comment. In addition, the webMethods runtime provides some sweet builtins — including operations to snapshot or restore the entire runtime stack-frame.

    Unfortunately, the webMethods language didn’t progress much in the last two decades. Nor is it open-source. It’s pretty much a specialised middleware language that sells to the big end of town. But it has the right idea — increasing communication ‘bandwidth’ between users and program (‘line’ becomes ‘canvas’), and providing nifty builtins (‘save stack-frame to file’) increases productivity. A good programmer becomes a great one. A mediocre programmer (like me) becomes somewhat decent.

    A graphical language that takes these ideas further will have even more success. So a programming language environment that supports rich media (a la Sikuli), color, 2.5 or 3D canvas dimensions, mathematical notation, rich text documentation; that accepts multi-touch, voice and embodied input, offers support async and parallel programming, and back-in-time debugging (so, for instance, a runtime exception generates a reproducible testcase in seconds)… that would be programming for the masses.

  24. The world has changed. Programming tools, hardware, GUIs and peoples’ minds too. Programming today is mostly focused in Internet and related applications. Not more in “unleashing the ASM power” of some hardware.
    One can accept that. Or just say there should be some fixing it.
    Cheers.

  25. The old adage is that a good programming language should make the easy things easy and the hard things possible. Learning C++ in the late 90s, I experienced the opposite: the hard things were easy, and the easy things were nearly impossible. Many of the new languages that have either been developed or risen to prominence since then have moved to correct this, allowing developers to do sophisticated things when we need to, while also doing things like taking care of memory management and simplifying string operations.

    I think there’s real potential for extending this to making programming—even in existing languages—accessible to general users. Languages like Python, whose beauty lies in its readability—what it says truly is what it does—show the most promise. An emphasis on creating libraries or tools with the goal of usability by non-programmers, taking care of the standard tasks that we have to write anew for each project or that run the bulk of an application, allowing a user to plug in just the bits of code that they need will go a long way.

    As someone who regularly has to fix other people’s VBA code (Visual Basic for Applications) in Microsoft Access, I have a hard time getting behind Visual Basic, but I think the principles on which the idea is founded are sound: make it easy to build an interface, handle stuff internally when it’s something you almost never need to modify, and use a language that’s clear and simple, while also allowing the user to go further when they learn more of it.

Leave a Reply

Your email address will not be published. Required fields are marked *