What should I build Subtext 9 on?

This will still be just a research prototype of a programming language and environment, so I don’t care about deployment issues. Want to have:

  1. Discriminated unions and pattern matching
  2. IDE with debugging
  3. Rich GUI framework with functional style (Elm/React)
  4. Healthy ecosystem

Options:

  • Elm — nice and simple, but better suited to apps than an entire language environment. I really need mutation. Bit of a walled garden. Needs a real debugger (yes reproducibility improves printf debugging but it’s still just printf).
  • Dart/Flutter — could work. Refreshing simplicity compare to the chaos of web programming. Flutter is very appealing. But I’ve used Dart before. It is an old-fashioned OO language and I’ve gotten tired of that. Wake up and smell the pattern matching! Working in Dart felt like living in a remote province.
  • F#/Fable/React — nice, but a big language, with a lot of .NET baggage. Initial experiments ran into difficulties with the .NET tooling.
  • ReasonML/Bucklescript/OCaml/React — technically powerful, but no one seems to be in charge and it’s a big hot mess. Bucklescript is one guy in China. No source level debugger.
  • Swift/SwiftUI — very enticing: a modern language with a modern UI from people who really know their UI. Stealing the best parts of Flutter. Full tech stack fully documented and supported from one company with unbounded resources. Sadly, SwiftUI looks to be just too new. Still very mobile-centric. There isn’t even a table layout.
  • Rust/? — Rust looks a bit scary, but some people I respect love it. There is no good GUI story yet. In a couple of years I expect there will be a SwiftUI clone, compiling to WASM and native. Could be my future production platform.
  • TypeScript/React — utterly mainstream, and I already know it. Has a weak form of discriminated unions. Good enough.

The User Liberation Front

From the beginning this blog had the subtitle “Dispatches from the Programmer Liberation Front”. I have changed it to the User Liberation Front. Really this change has been building for years now. I started out wanting to fix programming, to help realize its full potential, and to uplift our tribe of nerdy misfits. I slowly realized that the heart of the problem is not our technology but our culture itself. Programming sucks because we like it that way. It entertains us with puzzles; it affirms our differences from the outgroup; it rewards us with power and wealth. Enough. I am now an anti-programmer.

The User Liberation Front works to put the power of computers into the hands of users, freeing them from the domination of technologists and corporations. Spreadsheets do this today. We will spread the freedom of spreadsheets to other domains of software, so that, users no more, all can create the software they need.

I am not alone in this. Are you with us?

Roadmap to the Renaissance

How will we escape the current Dark Age of Programming? What will trigger a Software Renaissance? History tells us that major intellectual shifts usually arise not from lone geniuses but from subcultures that suddenly coalesce in a burst of invention and discovery. I have found an interesting book that looks at the group dynamics of such subcultures:  Collaborative Circles: Friendship Dynamics & Creative Work by Michael P. Farrell (suggested by Patrick Collison). From the blurb:

Many artists, writers, and other creative people do their best work when collaborating within a circle of like-minded friends. In a unique study, Michael P. Farrell looks at the group dynamics in six collaborative circles, and gives vivid narrative accounts of each: the French Impressionists; Sigmund Freud and his friends; C. S. Lewis, J. R. R. Tolkien, and the Inklings; social reformers Elizabeth Cady Stanton and Susan B. Anthony; the Fugitive poets; and the writers Joseph Conrad and Ford Maddox Ford.

Farrell presents a systematic description of how such collaborative circles arise and develop. I’ve been trying for years to organize various sorts of groups to advance progress in programming, following the established topography of academic organizations like workshops and working groups. Farrell’s book presents a roadmap to an entirely different terrain: the informal underground rebellions where breaking changes actually happen. Much to consider…
Continue reading “Roadmap to the Renaissance”

TFW you reject your own submission

Dear Jonathan,

We’re sorry to inform you that your submission #12

Title: Direct Programming

was rejected, and will not be presented at the LIVE 2018 workshop. Your reviews are included below; all papers were reviewed by 3 people. We had a record 22 submissions this year, and were only able to accept 10, so making the decision was hard.

We would like to take this opportunity to thank you for your submission and we hope you find the reviewers’ comments helpful.

– Roly, Ravi and Jonathan

Continue reading “TFW you reject your own submission”

Direct Programming

Here is a brief description of my latest work in progress, which I am submitting to PPIG.

Direct Programming

To make programming more accessible to regular people it may help to make it less abstract and more concrete. Towards that end I am taking another run at the old ideas of macro recording and Programming by Example. Many such efforts abstracted user actions into conventional code. Instead I co-design the PL and UI to be analogous, so that programs and users have much the same capabilities, and programs look much like recorded transcripts of user actions. Conscientiously following this design principle leads in the opposite direction of current PL trends, towards a deeply imperative semantics and a de-emphasis of functional abstraction.

Any attempt to make programming more concrete faces two challenges: loops and conditionals, for they necessarily deal with hypothetical situations. My approach is to extend the direct manipulation metaphor onto template states that prototype iteration, and example states that witness alternative cases.

This submission is a demo of early-stage work lacking evaluation, and indeed it is submitted in the hope of receiving guidance on an appropriate evaluation methodology.

[Video]

As always, comments and criticisms welcome.

Reifying Programming

Here is the abstract for my next talk:

The need to think abstractly makes programming difficult, especially for normal people. To make programming more concrete I propose to represent programs as data structures reifying their execution. Programming shifts from editing textual abstractions to direct inspection and manipulation of concrete executions. Reified execution also helps ground abstraction through a new testing mechanism: interventions. An intervention tests and overrides behavior at a specific point within a reified execution, replacing hand-coded tests and mocks. Intervention points are stable because programs are edited structurally, not textually. Interventions also provide scaffolding that assists program development through refactoring-like steps of gradual abstraction.

Update Here is the video I submitted to LIVE’17.

Chatterbase

Interesting code-essay from Joe Edelman: Chatterbase. I see this as a reply to my Transcript proposal (now Chorus). It is gratifying to get this kind of feedback. This is how we make progress – it is what academics mean when they talk about how research is a “conversation” – people’s work triggers others to contradict/extend it in a virtuous cycle of progress. While I have many issues with how academic computer science works, this is one piece of wisdom we shouldn’t ignore.

Joe’s declarative social appears to be essentially what I call social datatypes, but his soft automation attempts to go beyond the fairly rigid workflow envisioned in Chorus. As with Joe’s previous work he is deeply concerned with the humanism of social media – making it suit the way people naturally function rather than the other way around. I take his point, and would like to make Chorus work as informally as possible.

Joe makes the intriguing suggestion that out-of-band communication like email, chat, or even voice should suffice to satisfy obligations in the workflow. It is interesting to think about how that could be done. Here is a speculation: many current attempts at AI chatbots will fail to work beyond the most generic conversations because they have insufficient knowledge of the subject at hand. Perhaps the declarative scripts that Joe and I envision could supply the needed formal description.

I’m hoping this is the start of a little mini-field of people working on Socially-facilitating Software. Maybe someday we can hold a workshop. But in the meantime, in the spirit of encouraging the sort of conversation I spoke of above, I’ve add a “See also” section to the Chorus website.

 

I scare myself

I’m afraid the only way to realize the Transcript vision of end-user programming is to start a company. I did that once. After many years of toil and tears I was incredibly lucky to exit successfully and I swore then I would never put myself through that again. I would much prefer to have an open-source project delivering an excellent end-user experience while also continuing to research and innovate, working with interesting people on a humane schedule. As far as I can tell that has never happened.

Building a high-quality end-user product requires a lot of hard work that is not appealing to open-source contributors. They want to work on what they know and use – programming tools and libraries. Academic researchers want to play with cool new ideas. You need to pay professionals to do the hard design, engineering and support work required to deliver a product. You need to have revenue to pay those professionals. Unless I can find some benevolent sugar daddy I am afraid I may eventually need to start a company.

Does anyone know of open-source or academic projects that have built an innovative end-user product? Note: for end-users, not for programmers. Also note: innovative, not a clone of an existing commercial project. I’d love to see a successful example.

 

ch-ch-changes

A lot has happened in the last year. I left MIT and joined CDG (Alan Kay’s lab), working with Alex Warth. I’ve been going to LA a lot. We are working on an end-user programming tool called Transcript.

My Holy Grail has been to radically simplify professional programming. I now realize that simplification is not fundamentally a technical problem but rather a cultural one. Our nerd culture embraces inhuman levels of complexity. Mastering mind-boggling complexity is our mutant superpower. It is our tribal marker. Complexity is the air we breath, and so it is invisible to us. Simplification will only come from outside this culture. To disrupt programming I first have to reinvent it for a fresh audience of non-programmers.

Here’s a video of our first (very rough and preliminary) talk about Transcript.

 

 

Why programming languages matter

A colleague asked this question and here is my answer:

Programming gives us the power of the Gods to create things out of pure thought. Programming languages are the incantations and gestures we use to perform this magic.

Unfortunately we got only the power of the Gods, not their wisdom, and so we have created things of vast ugliness and secreted our powers within a cognitive priesthood. Perhaps programming languages could also play a part in solving these problems.

Flux is good enough

The reaction to my latest work [Two-way Dataflow] has been distinctly underwhelming. My diagnosis: I’ve solved a problem most people aren’t aware they have, and it comes at the steep price of replacing the conventional tech stack. Facebook has a cheaper solution: Flux. I have to give them credit for taking the problem seriously (unlike many of the reactive programming enthusiasts) and coming up with a pragmatic solution that just adds another layer to the stack. That’s what programmers want, not a fundamental solution in a new programming paradigm. It is obvious that we can’t keep growing the stack forever but no one wants to deal with it now. Especially computer scientists (don’t get me started).

I think I need to shift focus to end-users and non-programmers. I’ve avoided that because I want to prove that we can dramatically simplify real large-scale programming — simplifying simple problems is not so convincing. But replacing the whole stack is just too hard to do all at once and all by myself, and only doing it on small examples is not convincing either. End-user programming could be a way to get started and get users and (especially) have collaborators. Industrial strength programming would come later, but at least I now have a sketch of how it should look. Building something real would be such a refreshing change from the last decade of thought experiments. I’m excited by the prospect of once again shipping buggy code and having irate users and getting into fights with my colleagues. That’s life.

Future Programming Workshop 2015

The Future Programming Workshop will return this year to SPLASH and Strange Loop. See http://www.future-programming.org/call.html. This year we are taking any kind of media, not just videos. Web pages and papers are welcome too. By request of the academic members of our community we will publish proceedings containing the paper-format submissions. We are applying for permission to publish the proceedings in the ACM Digital Library but will go somewhere else if necessary.

Like last year we will offer the option to present at the Strange Loop pre-conference FPWxELC event. Unlike last year there will also be a chance to present publicly at SPLASH. The SPLASH event will be two days: the first for public presentations and the second for the private writers’ workshop.

Thanks to Richard Gabriel and Alex Payne for teaming up with me to make this happen.

Submission deadline is August 7 (pre-submit July 19 for Strange Loop). So get to work and show us what you’ve got!

New Subtext screencast

We’ve published the final videos from the Future Programming Workshop. We will also be publishing a final report about our experiences and lessons from the workshop.

Included in the videos is my latest screencast about Subtext: Two-way Dataflow. The abstract:

Subtext is an experiment to radically simplify application programming. The goal is to combine the power of frameworks like Rails and iOS with the simplicity of a spreadsheet. The standard MVC architecture of such frameworks makes execution order hard to understand, a problem colloquially called callback hell. I propose a new approach called two-way dataflow, which breaks the program into cyclic output and input phases. Output is handled with traditional one-way dataflow, which is realized here as a form of pure lazy functional programming. Input is governed by a new semantics called one-way action which is a highly restricted form of event-driven imperative programming. These restrictions statically order event execution to avoid callback hell. Two-way dataflow has been designed not only to simplify the semantics of application programming but also to support a presentation that, like a spreadsheet, provides a fully WYSIWYG programming experience.

Comments welcome.