How will we escape the current Dark Age of Programming? What will trigger a Software Renaissance? History tells us that major intellectual shifts usually arise not from lone geniuses but from subcultures that suddenly coalesce in a burst of invention and discovery. I have found an interesting book that looks at the group dynamics of such subcultures: Collaborative Circles: Friendship Dynamics & Creative Work by Michael P. Farrell (suggested by Patrick Collison). From the blurb:
Many artists, writers, and other creative people do their best work when collaborating within a circle of like-minded friends. In a unique study, Michael P. Farrell looks at the group dynamics in six collaborative circles, and gives vivid narrative accounts of each: the French Impressionists; Sigmund Freud and his friends; C. S. Lewis, J. R. R. Tolkien, and the Inklings; social reformers Elizabeth Cady Stanton and Susan B. Anthony; the Fugitive poets; and the writers Joseph Conrad and Ford Maddox Ford.
Farrell presents a systematic description of how such collaborative circles arise and develop. I’ve been trying for years to organize various sorts of groups to advance progress in programming, following the established topography of academic organizations like workshops and working groups. Farrell’s book presents a roadmap to an entirely different terrain: the informal underground rebellions where breaking changes actually happen. Much to consider…
Continue reading “Roadmap to the Renaissance”
Steve Krouse interviewed me on his podcast: De-Nerding Programming. There are some other interesting interviews, and he also hosts a Slack discussion.
We’re sorry to inform you that your submission #12
Title: Direct Programming
was rejected, and will not be presented at the LIVE 2018 workshop. Your reviews are included below; all papers were reviewed by 3 people. We had a record 22 submissions this year, and were only able to accept 10, so making the decision was hard.
We would like to take this opportunity to thank you for your submission and we hope you find the reviewers’ comments helpful.
– Roly, Ravi and Jonathan
Continue reading “TFW you reject your own submission”
A short peek at what I’ve been working on, as a postscript to my Direct Programming video.
Programming could be much much easier most of the time. This unnecessary difficulty wastes the effort of professional programmers, but worse, prevents many people from programming at all. The problem is a nasty hairball of culture and technology and money that is not easily untangled. I’ll try to address this essay to a general audience. Continue reading “The problem with programming and how to fix it”
Here is a brief description of my latest work in progress, which I am submitting to PPIG.
To make programming more accessible to regular people it may help to make it less abstract and more concrete. Towards that end I am taking another run at the old ideas of macro recording and Programming by Example. Many such efforts abstracted user actions into conventional code. Instead I co-design the PL and UI to be analogous, so that programs and users have much the same capabilities, and programs look much like recorded transcripts of user actions. Conscientiously following this design principle leads in the opposite direction of current PL trends, towards a deeply imperative semantics and a de-emphasis of functional abstraction.
Any attempt to make programming more concrete faces two challenges: loops and conditionals, for they necessarily deal with hypothetical situations. My approach is to extend the direct manipulation metaphor onto template states that prototype iteration, and example states that witness alternative cases.
This submission is a demo of early-stage work lacking evaluation, and indeed it is submitted in the hope of receiving guidance on an appropriate evaluation methodology.
As always, comments and criticisms welcome.
Here is the abstract for my next talk:
The need to think abstractly makes programming difficult, especially for normal people. To make programming more concrete I propose to represent programs as data structures reifying their execution. Programming shifts from editing textual abstractions to direct inspection and manipulation of concrete executions. Reified execution also helps ground abstraction through a new testing mechanism: interventions. An intervention tests and overrides behavior at a specific point within a reified execution, replacing hand-coded tests and mocks. Intervention points are stable because programs are edited structurally, not textually. Interventions also provide scaffolding that assists program development through refactoring-like steps of gradual abstraction.
Update Here is the video I submitted to LIVE’17.
Sean McDirmid and I are organizing the LIVE Programming Workshop at SPLASH. Please consider submitting your work – deadline is Aug 1.
I’ve assembled a gallery of notable/interesting user interfaces for programming, as inspiration for the next Subtext. [Google Slides]
Interesting code-essay from Joe Edelman: Chatterbase. I see this as a reply to my Transcript proposal (now Chorus). It is gratifying to get this kind of feedback. This is how we make progress – it is what academics mean when they talk about how research is a “conversation” – people’s work triggers others to contradict/extend it in a virtuous cycle of progress. While I have many issues with how academic computer science works, this is one piece of wisdom we shouldn’t ignore.
Joe’s declarative social appears to be essentially what I call social datatypes, but his soft automation attempts to go beyond the fairly rigid workflow envisioned in Chorus. As with Joe’s previous work he is deeply concerned with the humanism of social media – making it suit the way people naturally function rather than the other way around. I take his point, and would like to make Chorus work as informally as possible.
Joe makes the intriguing suggestion that out-of-band communication like email, chat, or even voice should suffice to satisfy obligations in the workflow. It is interesting to think about how that could be done. Here is a speculation: many current attempts at AI chatbots will fail to work beyond the most generic conversations because they have insufficient knowledge of the subject at hand. Perhaps the declarative scripts that Joe and I envision could supply the needed formal description.
I’m hoping this is the start of a little mini-field of people working on Socially-facilitating Software. Maybe someday we can hold a workshop. But in the meantime, in the spirit of encouraging the sort of conversation I spoke of above, I’ve add a “See also” section to the Chorus website.
I’m afraid the only way to realize the Transcript vision of end-user programming is to start a company. I did that once. After many years of toil and tears I was incredibly lucky to exit successfully and I swore then I would never put myself through that again. I would much prefer to have an open-source project delivering an excellent end-user experience while also continuing to research and innovate, working with interesting people on a humane schedule. As far as I can tell that has never happened.
Building a high-quality end-user product requires a lot of hard work that is not appealing to open-source contributors. They want to work on what they know and use – programming tools and libraries. Academic researchers want to play with cool new ideas. You need to pay professionals to do the hard design, engineering and support work required to deliver a product. You need to have revenue to pay those professionals. Unless I can find some benevolent sugar daddy I am afraid I may eventually need to start a company.
Does anyone know of open-source or academic projects that have built an innovative end-user product? Note: for end-users, not for programmers. Also note: innovative, not a clone of an existing commercial project. I’d love to see a successful example.
A lot has happened in the last year. I left MIT and joined CDG (Alan Kay’s lab), working with Alex Warth. I’ve been going to LA a lot. We are working on an end-user programming tool called Transcript.
My Holy Grail has been to radically simplify professional programming. I now realize that simplification is not fundamentally a technical problem but rather a cultural one. Our nerd culture embraces inhuman levels of complexity. Mastering mind-boggling complexity is our mutant superpower. It is our tribal marker. Complexity is the air we breath, and so it is invisible to us. Simplification will only come from outside this culture. To disrupt programming I first have to reinvent it for a fresh audience of non-programmers.
Here’s a video of our first (very rough and preliminary) talk about Transcript.
A colleague asked this question and here is my answer:
Programming gives us the power of the Gods to create things out of pure thought. Programming languages are the incantations and gestures we use to perform this magic.
Unfortunately we got only the power of the Gods, not their wisdom, and so we have created things of vast ugliness and secreted our powers within a cognitive priesthood. Perhaps programming languages could also play a part in solving these problems.
For programming to start over we need to target the fresh minds of non-programmers, and find a new application domain that isn’t already completely built up.
I’ll be talking about my next initiative at YOW!: Transcript: End-User Programming Of Social Apps.
I’m teaming up with Alex Warth.
The reaction to my latest work [Two-way Dataflow] has been distinctly underwhelming. My diagnosis: I’ve solved a problem most people aren’t aware they have, and it comes at the steep price of replacing the conventional tech stack. Facebook has a cheaper solution: Flux. I have to give them credit for taking the problem seriously (unlike many of the reactive programming enthusiasts) and coming up with a pragmatic solution that just adds another layer to the stack. That’s what programmers want, not a fundamental solution in a new programming paradigm. It is obvious that we can’t keep growing the stack forever but no one wants to deal with it now. Especially computer scientists (don’t get me started).
I think I need to shift focus to end-users and non-programmers. I’ve avoided that because I want to prove that we can dramatically simplify real large-scale programming — simplifying simple problems is not so convincing. But replacing the whole stack is just too hard to do all at once and all by myself, and only doing it on small examples is not convincing either. End-user programming could be a way to get started and get users and (especially) have collaborators. Industrial strength programming would come later, but at least I now have a sketch of how it should look. Building something real would be such a refreshing change from the last decade of thought experiments. I’m excited by the prospect of once again shipping buggy code and having irate users and getting into fights with my colleagues. That’s life.
The Future Programming Workshop will return this year to SPLASH and Strange Loop. See http://www.future-programming.org/call.html. This year we are taking any kind of media, not just videos. Web pages and papers are welcome too. By request of the academic members of our community we will publish proceedings containing the paper-format submissions. We are applying for permission to publish the proceedings in the ACM Digital Library but will go somewhere else if necessary.
Like last year we will offer the option to present at the Strange Loop pre-conference FPWxELC event. Unlike last year there will also be a chance to present publicly at SPLASH. The SPLASH event will be two days: the first for public presentations and the second for the private writers’ workshop.
Thanks to Richard Gabriel and Alex Payne for teaming up with me to make this happen.
Submission deadline is August 7 (pre-submit July 19 for Strange Loop). So get to work and show us what you’ve got!
We’ve published the final videos from the Future Programming Workshop. We will also be publishing a final report about our experiences and lessons from the workshop.
Included in the videos is my latest screencast about Subtext: Two-way Dataflow. The abstract:
Subtext is an experiment to radically simplify application programming. The goal is to combine the power of frameworks like Rails and iOS with the simplicity of a spreadsheet. The standard MVC architecture of such frameworks makes execution order hard to understand, a problem colloquially called callback hell. I propose a new approach called two-way dataflow, which breaks the program into cyclic output and input phases. Output is handled with traditional one-way dataflow, which is realized here as a form of pure lazy functional programming. Input is governed by a new semantics called one-way action which is a highly restricted form of event-driven imperative programming. These restrictions statically order event execution to avoid callback hell. Two-way dataflow has been designed not only to simplify the semantics of application programming but also to support a presentation that, like a spreadsheet, provides a fully WYSIWYG programming experience.
It’s time to reformulate the principles guiding my work.
[Revised definition of complexity in response to misunderstandings]
Continue reading “The Future Programming Manifesto”
Final version of the paper is up, and an essay with embedded videos is here. Sean graciously invited me to coauthor but the ideas are really his – I just helped spin them.
We think there is great promise in abstracting away from the computer model of time. There is a large design space that is still largely unexplored. I will be presenting my own new approach for the first time in public at the FPW workshop at Strange Loop. We are hoping to excite other researchers to take up this challenge and develop their own approaches. Come talk with us at SPLASH or drop us a line.
I’ll be demoing my latest work at the Future Programming Workshop at both Strange Loop and SPLASH. My talk is called “Two-way Dataflow”. Here is the abstract:
Subtext is an experiment to radically simplify application programming. The goal is to combine the power of frameworks like Rails and iOS with the simplicity of a spreadsheet. Mutable state is a notorious source of complexity in application programming and indeed has long been a major dilemma in programming language design. I propose a new approach called two-way dataflow, which breaks the program into cyclic output and input phases. Output is handled with traditional one-way dataflow, which is realized here as a form of pure lazy functional programming. Input is governed by a new semantics called one-way action which is a highly restricted form of event-driven imperative programming. These restrictions statically order event execution to avoid callback hell. Two-way dataflow has been designed not only to simplify the semantics of application programming but also to support a representation that, like a spreadsheet, provides a fully WYSIWYG programming experience.
This work is still very experimental and preliminary but I hope it is complete enough to convey several key new ideas that are relevant to the current trends of “reactive” and “live” programming.