My GPCE paper was rejected, and rather harshly. You can read the reviews below. My analysis is that I was rejected for not being incremental to prior work in the field. <\/p>\n
——————– review 1 ——————–<\/p>\n
PAPER: 41
\nTITLE: Modular Generation and Customization<\/p>\n
OVERALL RATING: 1 (weak accept)
\nREVIEWER’S CONFIDENCE: 2 (medium)
\n———————– REVIEW ——————–<\/p>\n
This paper addresses an interesting problem: generating a variety of
\nrepresentations of some data, even when that data evolves over time.
\nIt distills the problem into an interesting challenge problem in order
\nto raise awareness of this problem and provide a common problem for
\nuse in evaluating different solutions. It also proposes a solution to
\nthis problem in the form of “differential trees” that can be used to
\ndescribe data and transformations on that data.<\/p>\n
I think that the problem and the proposed solution are quite
\ninteresting and would be right for the GPCE audience. The field trees
\nand differential trees overcome many of the problems in dealing with
\nunique names and positions in sequences that are found in program,
\ndata, and model transformations.<\/p>\n
There are, however, some problems with the presentation of the paper.
\nAlthough the paper claims to solve this problem of dealing with the
\nevolution of data, and the transformations that will still be
\napplicable, without change, to the modified data, not enough space is
\ndevoted to explaining this in the solution. In Section 4.2, on page
\n8, column 1, we have the sentence “As we saw in the previous section,
\nit is this positional correspondence between Data and Form that allows
\nCustomize to survive the evolution of Data.” But this is no further
\nexplanation of why this is. The reader can figure it on his or her
\nown but I would expect some explanation of why this all works out.<\/p>\n
The preceding description of transformations in differential trees is
\nalso not always so clear. The semantics of this language are
\nrelegated to a PDF file on the Subtext web site. This might be OK if
\nwe had a better description of the example in 4.2. For example, it
\nseems that “body” is the instantiated body of the function given to
\nmap but this isn’t made clear.<\/p>\n
I think that there is room for some discussion of semantics since the
\nauthor is using a larger font size that used in most of the other
\npapers.<\/p>\n
The discussion of transformations also suffers in section 3. The
\n“Field tree summary” claims “Field trees solve the challenge problem
\nby providing stable identities and positions so that transformations
\nexpressed in those terms can be composed modularly.” There is no way
\nto evaluate the validity of this claim since we do not, at this point,
\nsee any of these such transformations – only the results of them. On
\npage 5, column 2, we read “Fig 9 shows who Customize transforms Form
\ninto CustomForm”, but we only see the result of this transformation.
\nIn order to see that the field tree is a good data structure over
\nwhich transformations can be applied we really need to see more
\nclearly how one expresses transformations over them. A little
\npseudo-code or something would be helpful here. Granted, the
\ndifferential trees provide this solution, but on the first reading of
\nthe paper one is left wondering how transformations on field trees
\nwould behave.<\/p>\n
Overall I really like the ideas presented in this paper, but the
\npresentation is sometimes rather hard to follow.<\/p>\n
Minor concerns:<\/p>\n
abstract, item 2 in column 2 of page 1: It isn’t clear what you mean
\nby “unshifting positions” at this point.<\/p>\n
page 4, column 2: Is the discussion of trees to represent positions is
\nnot really needed. There are other more important things that should
\nbe said and this discussion could be removed.<\/p>\n
Other parts are also note so clear in the presentation.
\nI found the description of field trees in column 1 of page 5
\nconfusing. A little more description about how field trees are
\ndisplayed in the figures would help before getting into the contents
\nof the trees. Even just adding a statement about fields with no
\nvalues but only positions are shown with no colon would be helpful.
\nAlso, the fact that positions are reused – as described in the third
\nparagraph of 3.3 could be moved up. After we understand the
\nstructure and visualization of the trees, then get into the encoding
\nof XML in field trees. <\/p>\n
——————– review 2 ——————–<\/p>\n
PAPER: 41
\nTITLE: Modular Generation and Customization<\/p>\n
OVERALL RATING: 3 (strong accept)
\nREVIEWER’S CONFIDENCE: 2 (medium)
\n———————– REVIEW ——————–<\/p>\n
This well-written paper presents a challenge problem that captures the
\ntradeoff between modularity and flexibility that often occurs when
\ntransforming programs in a source language (such as a database schema)
\nto programs in a target language (such as an HTML form): how to preserve
\nmanual customizations made to the result of automatic generation. In
\na well-motivated progression towards a satisfying solution of this
\nchallenge problem, the author presents the concepts of positions (which
\nstably identify keys in a sequence), field trees (which organize
\nexpressions in a hierarchy of positions), and differential trees (which
\ngeneralize spreadsheets to declarative specifications of field trees and
\ncomputations on them). These concepts are situated within the author’s
\nlarger project of so-called transformative programming.<\/p>\n
I find these concepts compelling as a programming model and convincing
\nas a solution to the challenge problem. The relation between the paper
\nand the general tradeoff between modularity and flexibility would be
\nstrengthened if the author could mention (without details, as space
\nis limited) the same machinery solving more instances of the tradeoff
\nbesides the single concrete challenge problem. I also wonder how this
\nwork relates to Benjamin Pierce et al.’s Harmony\/Boomerang project.<\/p>\n
Minor comments:<\/p>\n
On page 4, “The current implementation … this notional tree”, in
\nparticular “ordered before it, by ascending serial number”, is unclear
\nbecause two orderings have been mentioned: one on positions defined
\nby the sequence abstraction and one on serial numbers defined by this
\nparticular implementation.<\/p>\n
Also on page 4, “objects ID’s” -> “object ID’s”<\/p>\n
On page 6, the top of the left column discusses storing positions in a
\nfield tree whereas the bottom discusses storing paths of positions in a
\nfield tree, if I understand correctly. This distinction is a bit subtle
\nso I would appreciate the text confirming or denying it.<\/p>\n
On page 7, the discussion of the := mode (the third paragraph) is
\nunclear. I can’t figure out: what goes wrong if I replace := by :: in
\nFigure 10(e)? For that matter, why distinguish between : and :: at all? <\/p>\n
——————– review 3 ——————–<\/p>\n
PAPER: 41
\nTITLE: Modular Generation and Customization<\/p>\n
OVERALL RATING: -3 (strong reject)
\nREVIEWER’S CONFIDENCE: 3 (high)
\n———————– REVIEW ——————–<\/p>\n
The author presents a solution in the context of subtext for the
\nproblem of modular customization of generators. The example problem is
\na system where a database model changes and the application code has
\nto be updated to take new, removed, or reordered fields into
\naccount. The authors propose an example challenge, and builds towards
\na solution by introducing a sequential data structure where positions
\ndon’t change on inserts or deletions, such that positions are a stable
\nidentity for an element of the data structure. Next, the sequences are
\nincluded in trees to allows structured objects \/ records \/
\nfields. Finally, the complete solution in subtext is presented, called
\ndifferential trees. Differential trees unify data with the
\ncustomization\/transformation. The tree describes the customization of
\nreferences to other definitions using overriding differences.<\/p>\n
+ the underlying proposed model of subtext seems simple and elegant<\/p>\n
– however the paper is rather confusing and vague, perhaps due to
\n being premature<\/p>\n
– the argument that the proposed challenge reflects the actual problem
\n of generation and customization in practice is missing<\/p>\n
– a good story about how this works with multi-language programming is
\n missing<\/p>\n
– insufficient discussion of related work in customization (in
\n particular virtual classes).<\/p>\n
The writing is unnecessarily vague. Maybe this is due to prematurity
\nof the work. The structure of the paper, from positional sequences to
\nfield trees to differential trees is not helpful to me. Actually, the
\npart I found most comprehensible are the differential trees. Maybe
\nreversing the paper would help: first explain the complete solution of
\ndifferential trees, then dissect its ingredients.<\/p>\n
I advice to be much more explicit about your definitions of positional
\nsequences, field trees, and differential trees. Including some
\nformalization or pseudo code for some of the operations might be
\nhelpful. More concrete examples would help as well.<\/p>\n
The proposed challenge is too limited. I’m not convinced that the
\nchallenge you introduce reflects the complexity of the problem.
\nPlease justify the change from practice (structured HTML generation,
\nintegration of the database in the language\/editor). You simplify the
\nproblem enormously by introducing a universal representation and
\ncomplete integration. However, complete integration frequently solves
\nall problems. Yet, in many cases complete integration is not
\ndesirable. I admit that it might actually be the only complete
\nsolution, but this should still be very clearly motivated. Also,
\nplease clearly identify that this is the key to the solution.<\/p>\n
Large parts of the paper revolve around positioning of
\nfields. However, I would argue that the order of fields in the
\ndatabase and data model should be insignificant altogether (according
\nto good design principles). Therefore, I don’t get the significance of
\nthe position sequences to the generation and customization
\nproblem. After that, two main themes remain: 1) avoiding textual
\nidentifiers for stable identity and 2) customization by describing
\nonly overriding differences.<\/p>\n
For 1), you employ a new development environment with a new,
\nstructured\/visual form of editing code. This solves the issue, but
\nit’s hardly a contribution to explain that in this paper: it’s
\ncompletely obvious that this solves the unstable nature of textual
\nidentifiers.<\/p>\n
For 2), you employ differential trees. That is definitely an
\nattractive solution, but similar solutions exist that are not even
\nreferred to. In particular, this kind of customization is highly
\nrelated to virtual classes and in particular the simplicity of your
\napproach reminds me of the essential operators of the DEEP calculus
\n(“Eliminating Distinctions of Class: Using prototypes to Model Virtual
\nClasses”). This paper is largely about typing, but the kind of
\ncustomization that is possible is very similar to yours. I suggest to
\nat least compare your work to prototypes and virtual classes.<\/p>\n
The complete story about how this will be the core of a multi-language
\nsystem is missing. I get the idea, but there is no discussion at all
\nabout the mapping of multiple languages to the proposed differential
\ntree core of subtext. How are customizations going to be specified?<\/p>\n
The reader is currently a bit overwhelmed by the introduction of a
\nseries of unusual names, which obscure the actual underlying
\ntechniques. I would suggest to stick to more conventional
\nterminology. Too much significance is attached to the introduction of
\nnames, which are also used to list the contributions.<\/p>\n
The discussion of position sequences in 3.1 is not clear enough. The
\npaper describes the solution of using serial numbers only very
\nbriefly.<\/p>\n
Inline examples would make 4.1 more clear. In particular, on the first
\nread I had problems understanding:<\/p>\n
* “The assignments of values to fields are seen as a set of
\n definitions whose implications are worked out, a process called
\n integration”<\/p>\n
* “The arrows on the right indicate references between fields, and
\n serve as non-textual identifiers for anonymous fields.” ~ This is
\n confusing because the fields actually appears to have names in the
\n example.<\/p>\n
* “”live” execution” ~ please provide a bit more context of how the
\n trees will be used in subtext to understand this. Currently, you
\n assume that the reader is familiar with what subtext is.<\/p>\n
* “Every definition expresses an overriding difference between its
\n containers and their definitions” ~ It’s completely clear later, but
\n an inline example would clear this up completely.<\/p>\n
In general, it seems to be that the part on differential trees does
\nnot describe the solution to the challenge in full detail. You don’t
\ngo back to the challenge to show step by step how the introduction of
\na company and the removal of a phone number works. Figure 11: for
\ndifferential trees, it’s unclear what determines the ordering of the
\nelements when the contents of the CustomForm gets evaluated. How is a
\nposition assigned to a new definition, for example what controls the
\nposition of the new id2 definition? What mechanism makes phone come
\nlast in the result, as required?<\/p>\n
Some more minor details:<\/p>\n
* page 5, at the top of the right column please refer to Figure 3 when
\n you are discussing Figure 9.<\/p>\n
* page 5: “Note that these new positions are allocated once when the
\n transformations are written, not every time they are executed” ~
\n This is unclear. I don’t get this note at this point.<\/p>\n
* page 5: “the name of the position can be changed”. I assume a name
\n is a purely symbolic name for presentation purpose, but could you be
\n a bit more specific about the role of the name.<\/p>\n
* page 6: “But we also don’t want to hard-code hex strings for
\n internal position IDs” ~ I can see what you want to say here, but
\n readers will wonder where hex strings suddenly come in. Please
\n rephrase.<\/p>\n
* page 6, second paragraph (“The problem could be solved”). After
\n reading the rest of the paper I understand this part, but at this
\n point it is rather unclear. It would help a lot of you would give
\n the reader more context information earlier in the paper.<\/p>\n
* page 8: “These approaches handle customization through roundtrip
\n engineering”. That statement is a bit too general. <\/p>\n
——————– review 4 ——————–<\/p>\n
PAPER: 41
\nTITLE: Modular Generation and Customization<\/p>\n
OVERALL RATING: -3 (strong reject)
\nREVIEWER’S CONFIDENCE: 4 (expert)
\n———————– REVIEW ——————–<\/p>\n
Summary of the Paper:<\/p>\n
A method, “field trees” for labelling places in code is described.
\nA generalization called “diffential trees” is built on
\nthe field tree foundation. These schemes are used to
\n“solve” the problem of propagating informatoin to
\nmultiple targets in generated software artifacts.<\/p>\n
Comments for the Authors<\/p>\n
I found your paper impossible to read, because you did not use terms
\nthat I understood, and your key terms were never defined crisply.
\nConsequently, I did not understand except in the vaguest sense what
\nyou were trying to tell me.<\/p>\n
You define “positional sequence to be a sorted map from a domain of
\nidentifiers”. Usually when one defines a map (you mean this in
\nmathematical sense, right?), one defines a domain and a range. What’s
\nthe range of a “positional sequence”? What does “sorted map” mean?
\nYou don’t provide decent definition, and you don’t provide an example.
\nOK, I’m already lost, I have no intuition and no examples.<\/p>\n
“Field tress are nested positional sequences”. OK, I already don’t
\nknow what positional sequences are. So now I don’t know what field
\ntrees are. If I believe that positional sequences are a mathematical
\n“map”, I don’t know what “nested maps” are. You show an example
\n(Figure 8) and claim some kind of connection between id for Data
\ncontaining 1234 and value 1234 but I don’t see how this relationship
\nis justified. Offering me an implementation in an appendix which
\nisn’t provided in the paper is pointless.<\/p>\n
“Positions are encoded as a path … from the root of this tree”.
\nLike, child1.child3.child9.child2? And what if a subtree gets
\ndeleted? Gets moved?<\/p>\n
“Differential tres declaratively specify field tree transformations”.
\nOK, how? THe answer, “…by example” isn’t adequate for a technical
\npaper in which this is the supposed to be the primary result.<\/p>\n
Regarding the problem definition: I would have characterized
\nit as:
\n Data —Gen1–> Form1
\n | |
\n | –Gen2–> Form2
\n evolve
\n |
\n |
\n V
\n Data’ —Gen1–> Form1
\n |
\n –Gen2–> Form2<\/p>\n
which gives you a completely different view of the world.
\nNow what I need is given that I’ve computed Gen1(Data) (==>form1),
\nhow do I compute Gen1(Data+delta) where Data+Delta==Data’?
\nIdeally, it is some function
\n Update(Data,delta)=Gen1(Data)+Mods(Data,delta)
\nso I can take the original result Gen1(Data) and patch (“+”)
\nit. This gives you a completely different view of the problem.
\nCheck out finite differencing.<\/p>\n
Now, most of your argument seems to be how to label a place in source
\ncode. You don’t like comments containing generated symbols “because
\nthey are invisible to many tools”. So your solution is propose some
\n*one* tool that stores a database of “persistent internal IDs”. I
\ndon’t see how this is different than the generated symbols, and I
\ndon’t see how this makes getting tools any easier, now I have to have
\n*your* specific tool.<\/p>\n
You claim to somehow use differential trees to “generate” the code?
\nHow? Where does a differential tree encode all the crud that really
\nmakes up an HTML page?<\/p>\n
Finally, you whole scheme seems focused around some simply reshuffling
\nof a few fields. Changes in real programs are often more interesting
\nthan that. What if the problem is fields move from one page to
\nanother? If they are added together to get the new result?
\nUltimately the problem is one of traceability: one has to be able to
\nsay how one element of code was derived from underlying structures
\n(e.g., this HTML output field was derived from the database). So a
\ntrace that somehow links the database slot to the HTML code that
\ndepends on it, along with an indication of the dependency type seems
\nfundamental. I don’t how a position numbering scheme can provide that
\ntraceability.<\/p>\n
That’s the major stuff.<\/p>\n
Minor stuff:<\/p>\n
What on earth does PHP have to do with this paper? All of
\nyour points could have been made with just the HTML.<\/p>\n
But if the HTML form had been embedded in a PHP script,
\nI don’t see how you would have generated whatever PHP code
\nfrom whatever the examples you had in the paper.<\/p>\n
Points in favor or against (sent to authors)<\/p>\n
– Incoherent definitions of key concepts and results <\/p>\n","protected":false},"excerpt":{"rendered":"
My GPCE paper was rejected, and rather harshly. You can read the reviews below. My analysis is that I was rejected for not being incremental to prior work in the field.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[1],"tags":[],"class_list":["post-92","post","type-post","status-publish","format-standard","hentry","category-general"],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/pfEnU-1u","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/alarmingdevelopment.org\/index.php?rest_route=\/wp\/v2\/posts\/92","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/alarmingdevelopment.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/alarmingdevelopment.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/alarmingdevelopment.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/alarmingdevelopment.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=92"}],"version-history":[{"count":0,"href":"https:\/\/alarmingdevelopment.org\/index.php?rest_route=\/wp\/v2\/posts\/92\/revisions"}],"wp:attachment":[{"href":"https:\/\/alarmingdevelopment.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=92"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/alarmingdevelopment.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=92"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/alarmingdevelopment.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=92"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}