Alarming Development

Abstract of a talk I just gave:

It would be useful to have version control like git but for data structures, particularly the data structures we call spreadsheets, databases, and ASTs. Operational Version Control observes changes to typed data structures by recording high-level operations performed in a GUI or API. These operations can change both values and types, with type changes inducing corresponding value changes (so-called schema migration). Version control capabilities such as differencing, merging, and reverting are constructed out of transformations on operation histories. This theory requires as input a set of rules formalizing the intuitive sense that two operations “do the same thing” in different states. Our prototype implementation presents the user a simplified conceptual model: branches with linear append-only histories, forking by copying branches, and cherry-picking as the fundamental merge operation.

Related work: Diffing and patching tabular data

To: the crazy ones, the misfits, the rebels, the troublemakers, the ones who see things differently
Subject: you should write an Onward! paper

I and many people I know who are doing interesting work have been repelled by the bullshit parts of Academics: the gatekeeping, the compartmentalization, the performative rigor. Many of us have chosen to not play those games and instead make our own way as independent researchers. I’ve been in this fight for 20+ years now and I’m going to give you some advice you probably neither expect nor want to hear: publish.

Publishing a paper is the only proven way to engage with other researchers over the long term. Many people post demo videos and “web essays”, and if those meet your needs then by all means use them, for they are a lot easier and less risky than publishing papers. But they reach a different audience, mainly professional developers looking for something inspiring beyond the stagnant state of our art. They are not so effective in engaging with other researchers working on deep hard problems. Only papers have the time and space to explain our work in sufficient detail for others to learn from it and criticize it. Only papers painstakingly unravel the connections with other people’s work that we are building upon or improving upon. Only papers contain these burdensome elements because the peer-review process demands it, and for good reason: they are what carry a conversation between researchers iterating and refining new ideas. That is how progress actually happens, “by standing on the shoulders of giants”.

We need to ask ourselves a tough question: are we trying to add to human knowledge, or are we providing entertainment for jaded developers? The latter can be valuable, like science fiction, setting aspirations for the future. But to get to that future someone needs to solve hard problems in ways currently unimagined, and then communicate those discoveries to others who can appreciate them and improve upon them. Peer-reviewed papers are the best way we know to make progress happen.

I’m the program chair for Onward! Papers 2024. Onward! is expressly designed for people like us working on the fringe. You will not find a more welcoming place to publish computer science research. Onward explicitly reduces the need for rigorous evaluation and encourages interdisciplinary early-stage work with big aspirations. That said, you will still need to explain your ideas in concrete detail and convincingly argue their benefits over alternative approaches. Check out some of the recenty accepted papers. See the conference website for more details. Onward! gave me a crucial boost when I was getting started as an independent researcher. You should write an Onward! paper. Get in the arena.

The submission deadline is April 25, so you need to get started now! It is hard. In the worst case you are rejected, but at least you will get constructive feedback on how to convey your ideas more convincingly. As chair I will do my best to see that the reviewers provide helpful feedback. In any case just having put in the effort to write a paper will inevitably refine your ideas and your ability to explain them. In the best case you get an archival publication of your work, and you connect with a community of other researchers. My DMs are open.

Onward! and upward,
Jonathan Edwards

P.S. Thanks for helpful comments from Tomas Petricek, Stephen Kell, Geoffrey Litt, Mariano Guerra, and Richard Gabriel.

Too much Twitter has distracted me from this blog. I even forgot to mention my last paper: Live & Local Schema Change: Challenge Problems, with Tomas Petricek and Tijs van den Storm.

Schema change is an unsolved problem in both live programming and local-first software. We include in schema change any change to the expected shape of data, whether that is expressed explicitly in a database schema or type system, or whether those expectations are implicit in the behavior of the code. Schema changes during live programming can create a mismatch between the code and data in the running environment. Similarly, schema changes in local-first programming can create mismatches between data in different replicas, and between data in a replica and the code colocated with it. In all of these situations the problem of schema change is to migrate or translate existing data in coordination with changes to the code.
This paper contributes a set of concrete scenarios involving schema change that are offered as challenge problems to the live programming and local-first communities. We hope that these problems will spur progress by providing concrete objectives and a basis for comparing alternative solutions.

If you have any other good examples of schema change problems please let us know.