Dear Jonathan,
We’re sorry to inform you that your submission #12
Title: Direct Programming
was rejected, and will not be presented at the LIVE 2018 workshop. Your reviews are included below; all papers were reviewed by 3 people. We had a record 22 submissions this year, and were only able to accept 10, so making the decision was hard.
We would like to take this opportunity to thank you for your submission and we hope you find the reviewers’ comments helpful.
– Roly, Ravi and Jonathan
Review #12A
=================================================================
Overall merit
————-
3. Weak accept
Reviewer expertise
——————
3. Knowledgeable
Paper summary
————-
This submission presented a prototype of a direct manipulation programming environment that records the actions that the programmer performs on a tree structured “document” in an editable history. It also records intermediate states when evaluating procedures and allows the programmer to view and perform some modifications to the program by selecting from menus.
Comments for author
——————-
There were a number of interesting technical ideas in this presentation. I was particularly intrigued by the idea of an editable history of edit actions, and in the operation that abstracts portions of the history into a formula. This sort of direct manipulation of history seems to represent a fertile avenue for further exploration, so I would be happy to see this work presented at the workshop. The presentation was quite well-organized and clear, with appropriately simple examples and careful explanations. I’d love to play around with the presented tool (it would be nice if it had a name).
However, I gave the submission a “weak accept” because the qualitative claims seem to be unnecessarily grandiose — why is it necessary to claim that the interface is for “normal/regular people” (what does that mean?) or that the “notional machine is not a robot — it is yourself”? In fact, the system is fundamentally oriented around a rather complex programming language (it seems to have some sort of stack-based imperative semantics, combined with some reactive features, and an idiosyncratic transactional semantics for conditionals). The programmer will need to develop an understanding of its rather subtle semantics to be able to use the system effectively, just as in any other programming environment.
The direct manipulation features (which mostly seem to be simple context menus that insert low-level operations into the program) and live programming features seem like valuable cognitive aids and time-saving conveniences, but to claim that they eliminate the need to understand the programming model (“notional machine”) is shockingly bold, especially when the author has neither evidence, nor even an idea of how to operationalize these kinds of claims such that evidence could be gathered.
Why not just describe the interesting features of the programming environment being designed, explain precisely the benefits these features _might_ provide in various settings, and leave out the grand claims about reinventing or democratizing programming? (I hope that the LIVE community as a whole starts having these discussions about humility in framing our work!)
Finally, it would have been nice to see more related work mentioned. In particular, the Hazelnut edit action calculus explores the idea of a formalized language of edit actions (see the paper by Omar et al at POPL 2017). Another related project is “Expressions of Change”: http://www.expressionsofchange.org/. Many aspects of the system also reminded me of Smalltalk, but no direct comparison was made.
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
Review #12B
=================================================================
Overall merit
————-
3. Weak accept
Reviewer expertise
——————
3. Knowledgeable
Paper summary
————-
This video proposes a programming environment that combines Programming-by-Demonstration (PbD) and live programming (LP) for beginners. A programming step starts manipulating the current set of states, which contains a list of variable-and-value pairs. The manipulation, such as applying some built-in operation, is recorded as executable code in a neighboring pane. The recorded code can be edited, whose effect is immediately reflected on the states pane—where LP comes in. The video then discusses a bit more advanced constructs such as functions, conditional branches, and looping.
Comments for author
——————-
As the video claims, a combination of PbD and LP would be a novel idea. In this sense, this work is worth presenting at the workshop.
The motivation (or a part of the motivations) of the work, which is for beginners’ learning, should have been clarified at the beginning of the talk. Otherwise, the audience would start thinking about more complicated and generalized programming situations.
It is better to distinguish the terms programming-by-example and programming-by-demonstration. The former usually means systems that deduce a program from pairs of input and output, while the latter means systems that converts the user’s operations into code. This work falls into the latter.
While the idea of combining PbD and LP could be useful to learners, the mechanisms for programming constructs beyond assignments (i.e., functions, looping and conditional branches) are rather complicated. This is maybe I am too biased to read traditional programming languages. But I am not sure if the efforts on learning those mechanisms (in addition to the concepts of those constructs) can be reduced from a situation where the programmer uses the traditional syntax with this kind of liveness support.
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
Review #12C
=================================================================
Overall merit
————-
2. Weak reject
Reviewer expertise
——————
3. Knowledgeable
Paper summary
————-
This presentation describes direct programming, a novel approach which
incorporates the programming in the live environment.
Comments for author
——————-
I find this paper highly relevant for the workshop. Nevertheless I am
not sure what is the intended usage of the presented tool. The authors
agree that this is not a programming platform. The other potential use
is for educational purposes, but again I cannot see how it can help
students since one cannot even demonstrate a simple sorting algorithm.
Furthermore, there is a constant misuse of the “programming by
example” (PBE) expression. In PBE, there is a list of inputs and
outputs, and the PBE engine has to synthesize a program P which
returns given outputs for given inputs. Note that program P is not
known and has to be synthesized. In this work, the authors apply a
known program to a single example and then their tool propagates this
known program to other given inputs.
On a positive note, I really like the GUI for presented work. I
believe that this GUI and user interactions reflect the real spirit of
live coding.