My OOPSLA submission (to the Research Paper track) was rejected. Not a big surprise. The paper was ill-conceived, more of a brain-dump of my latest research than a focused story. Still, I thought it contained some real contributions, like a new approach to the problems of mutable state and concurrency.
I think the moral is that I should be working on demos instead of papers. The ideas in Subtext will succeed only by convincing real programmers that they are worth a disruptive change. I need to show cool features on practical programs. Theoretical papers can come later.
For the record, here are the reviews:
>>> Summary of the submission < << Defines "inclusions" as a formalization of copy & paste. Shows how this can do things like functional programming. Then the paper goes on to talk about reactive programming, hypotheticals, and IO a bit. Oh, and some concurrency. >>> Evaluation < << The introduction and semantics of inclusion are a salami-slice contribution; you need to do it at some point these days. The new material that talks about reactive programming and hypotheticals and concurrency etc seem to be a tack-on, to make the salami-slice paper a little more substantive. I really don't know what to recommend to you wrt this paper. If it were just the semantics, I'd recommend to the committee that it be accepted and given a couple of minutes of presentation to just say, yup, I did a semantics, go read about it. But with the other material included, it seems like the paper would be a grab bag. What is really happening, it seems to me, is that you're putting together a long paper or monograph about this work and packaging it into a series of conference papers. It would be better if the paper could be kept to just one topic - semantics, rather than introducing new ideas also. The other advice I have is that the presentation of how Subtext works is cortex-withering. At least to me. I think you need to work on how to present these ideas. The videos work ok, but still there are mysteries to it as well. I suspect that more detailed, longer explanations of the operations and notation and walking through the examples more thoroughly are what's needed. This argues that conference papers are not the right genre (too short) and you need to work on a monograph instead.
>>> Summary of the submission < << This paper provides a building block of the Subtext project who was previously described at OOPSLA 2005. That is, a formal model (operational semantics) complete with systematic illustrations of integral trees is provided. The approach is also used for the encoding of functional and reactive programming. The emphasis is on formal details that complement previously motivated and illustrated facilities of Subtext. >>> Evaluation < << The paper provides deep insight into the integral tree semantics and its use a foundation of "programming with copy and paste". The paper is ingeniously written and deserves publication, but he level of theoretical study strikes me to fit much better with a TCS-like forum (such as MFCS or SAS).
>>> Summary of the submission < << This paper presents a formalism of the essentials of the Subtext visual programming language, and also describes some advanced features of that language. >>> Evaluation < << The design of the (evolving) subtext programming language has been presented in two papers at Onward! over the last couple of years. Those papers were primarily about the design of the language and its user interface: this paper seeks to complement them by presenting a formalism describing the underlying computational model. As such, it is quite different in both tone and content to the previous papers on Subtext, and the formalism presented seems novel. (Disclaimer: I have not checked the formalism in detail) I have three concerns with this paper however. The first is about the purpose of the formalism. I assume the relationship between the formalism and the Subtext system is the same as between say Java and FGJ. With textual languages, however, there is generally a good notion of why one model captures the important parts of the language for some purpose. The model presented here differs from Subtext in some important respects: syntax, ordering, constraints, error handling, concurrency - and the paper needs to make clear *why* particular aspect were included in the model, and which are not. This then means it is possible to demonstrate (or at least to argue) that the formal model in some sense successfully models the language. Furthermore, a formalism - in this case effectively the operational semantics of a subtext subset - is not useful of itself: the point of modelling is to prove some property about a model, and therefore of the whole system. The paper states a number of properties about Subtext and Integral Trees, and although some are discussed I could find no attempt to prove any of them formally - e.g. that integration is confluent; that concurrency is in fact transparent; or that the translation into the lambda calculus is sound. My second concern is about the presentation of the paper. While the bulk and (most of) the main narrative of the paper is presenting the formal model, signifcant asides contain design discussions, advocacy, moral arguments ("decriminalisation") or promises about subtext, or the directions within which subtext will evolve. I like these arguments and think they are worthwhile. In comparison to the rest of the paper, these discussions are generally strictly informal, and so feel very much out of place. Finally, the paper does not critically situate Subtext into the wider context of realted work in (visual) programming language formalisms, rather than just visual languages themselves: e.g. the point of this paper is not (just) to compare with Pictorial Janus, but also with Janus itself. How does Subtext related to other languages with similar notions of history, such as Kaleidoscope with its pelucid variables, or Liebermann's reversible interpreters? The system also seems to be more than peripherially related to various graph reduction machines for functional language evaluation, for example, but the paper does not discuss other formal presentations. Given these issues, I am concerned that publication of this paper would be premature in its current state. I encourage the author to keep working both on the system and the formal models, and look forwards to hearing more about Subtext in future.
I think the moral you take from this is exactly correct. Speaking from personal experience, it is much more productive to just go ahead and build it, prove it out, and then worry about persuading the academic community (proving something in the wild forces them to pay attention).
In my experience, academics know very little about building practically useful programming languages – just look at the real-world usage of “uninteresting” languages like Java and C++ versus more “interesting” languages like Scheme, ML, and Prolog.
I don’t think there were any morals in this story. It may be refused because of it’s illnes, as you said, or for any technical reason (out of main focus, late submission, whatever).
I also don’t think, that working on demos is so much more important than writing papers. This is also a form of communication or cooperation. Sharing ideas is an equally important part of the work.
However, Ian is right: nothing more convincing than working stuff.
Why I write this is that I’m impressed by Subtext and I don’t want you to be discouraged by such a rejection. The more you share from your ideas, the greater the chances are that your dream written in the Manifesto will come true. And that’s what you work for, don’t you?
And one more ‘pro’ for papers: whenever you try to write your ideas in exact words for others to understand, you realize how solid and stable your ideas are. Rendering thoughts into words and sentences help you to get them more organized, more “finished”. That’s my experience, at least.