I got some good feedback here on my last paper. I want to see if I can get some equally good advice about what my next paper should be. My past papers have been about making programming easier. I would like to move on to making programming more powerful. Specifically, by making transformations and views a fundamental language feature. I have thought of a simple challenge problem that I might use to motivate and evaluate this idea. Please let me know whether you agree that a) this is actually an unsolved problem, and b) whether it is a worthwhile challenge. If there is interest, I will post this onto some collaborative editing surface. Thanks!\n<\/p>\n
<\/p>\n
It is often suggested that software designs can profitably be seen as the transformation of information from one structure into another. Such transformations are also called mappings or views. The appeal of this idea is the potential to construct complex systems by modularly combining simple transformations. But our programming languages do not express transformations very well, forfeiting their potential.\n<\/p>\n
I want to present a simple challenge problem to motivate and evaluate techniques of programming with transformations. This challenge is the archetypal problem of presenting data fields on a screen form, and the input of user changes to that data. Conceptually, this is merely a matter of mapping the internal data structure to a screen-form structure, and then inversely mapping user inputs back again. But actual practice typically involves complex frameworks and multiple languages that are not structured in terms of transformations.\n<\/p>\n
We start with a simple record of data fields. Expressed in Java, it is:\n<\/p>\n
\nclass Customer {\r\n int id;\r\n String name;\r\n String phone;\r\n String address;\r\n}<\/code><\/pre>\n<\/blockquote>\nFor this challenge, the data can alternatively be expressed in any preferred format, such as XML:\n<\/p>\n
<\/p>\n\r\n<Customer>\r\n <id>1234<\/id>\r\n <name>John Smith<\/name>\r\n <phone>555-1212<\/phone>\r\n <address>32 Vassar St\r\n Cambridge, MA 02139\r\n <\/address>\r\n<\/Customer><\/pre>\n<\/code><\/p><\/blockquote>\n
The problem is to transform an instance of this data into a screen form, using any desired presentation language. In HTML, the result would be:\n<\/p>\n
<\/p>\n<form method=\"post\" action=\"\">\r\n id: <input name=\"id\" type=\"text\" value=\"1234\"\/><p>\r\n name: <input name=\"name\" type=\"text\" value=\"John Smith\"\/><p>\r\n phone: <input name=\"phone\" type=\"text\" value=\"555-1212\" \/><p>\r\n address: <textarea name=\"address\" rows=\"3\">32 Vassar St\r\n Cambridge, MA 02139<\/textarea><p>\r\n<\/form><\/pre>\n<\/code><\/p><\/blockquote>\n
The challenge comes from the fact that both the data definition and the form layout can change. The data definition will evolve due to changing requirements. The form layout will be customized to add features and improve the usability of the presentation. These changes pose a dilemma. If we generate the form from the data, it is easy to accommodate change to the data, but hard to allow change to the form. On the other hand, we can sever the relationship between the data and the form, maintaining them as independent artifacts, and so easily customize the form, but changing the data will then require manual adaptation of the form.\n<\/p>\n
The latter option is the standard approach, treating the screen form as an independently maintained source artifact. To deal with online changes to data, some kind of “binding” occurs in the form to pull the data fields in and push user inputs back out again. Some UI frameworks offer bidirectional binding as a service between data and presentation classes. Template languages, common in use on the web, use “escapes” containing interpreted code that pull in data values in the right places at the right time.\n<\/p>\n
The problem with the standard approach is that it leaves the forms internally hard-wired into the data definitions. If the data definitions evolve, the forms must be manually adjusted. To make the problem precise, we will specify that the following data evolutions are to be handled:\n<\/p>\n
\n
- Inserting new fields, and\n<\/li>\n
- Renaming existing fields.\n<\/li>\n<\/ol>\n
One approach to solving this problem would be to treat these evolutions as refactorings that automatically make the appropriate changes to the form. To be an acceptable solution to this challenge, such refactorings must be decidable and sound. Good luck with that, as bindings and escapes are typically expressed in general purpose Turing-complete languages. The form language would have to be a limited special-purpose language of some sort. That is the approach taken in so-called Model-Driven Development (MDD), where domain specific models are defined declaratively. MDD raises the “round-trip engineering” problem, which is the need to map changes in one model into the others to keep them consistent. MDD is based on transformations, but ones that map changes between independent artifacts. I challenge MDD proponents to show how they would solve this simple problem. To make the problem more precise, I will list the customizations that the form language must support:\n<\/p>\n
\n
- Introducing a hierarchy of form layout containers to control where fields are displayed. For example, splitting the form into two side-by-side panels.\n<\/li>\n
- Distributing fields across different layout containers, and changing the order of fields within a layout container. For example, moving the address field into the right-hand panel created above, and moving the id field to be after the name field.\n<\/li>\n
- Renaming field labels, for example “id” to “customer id”.\n<\/li>\n<\/ol>\n
These customizations must be invariant in the presence of the data evolutions stipulated earlier: introducing new data fields (which should pop up within the form, preserving relative order with other fields that have not been explicitly moved), and renaming data fields (which should have no effect other than to replace displayed labels that haven’t been explicitly renamed).\n<\/p>\n
The purpose of this challenge is to evaluate an alternative resolution of the dilemma of coordinating change in software designs. I want to treat the form as being directly generated from the data definition, not as an independent artifact. The generator will be specified in a special language designed to express transformations between structures. Customizations to the form will also be transformations, which are composed together to convert the generated generic form into the desired customized form. The key technical hurdle is that these customizing transformations must be invariant under the stipulated evolutions of the input data. I challenge existing transformation languages to handle this short list of customizations and evolutions. I think they will be hard-pressed, because those customizations and evolutions, while commonplace and natural, play havoc with the ways we typically express structure in languages.\n<\/p>\n
Renaming is particularly nasty, for the na\u00c3\u00afve way to specify most of these transformations is by keying off of the names. Any reasonably expressive transformation language is unlikely to support decidable and sound refactoring of its matching semantics. Order is another stumbling block, for insertion is often specified in terms of integer ordinals, which are unstable in the presence of other insertions, violating the data evolution requirement. So for example moving the id field after the name field, if encoded in terms of ordinal positions, would be incorrect after inserting a new field before the id field.\n<\/p>\n
These technical problems reveal a surprising, even scandalous, situation: that in this day and age we still don’t understand sequential and hierarchical structures well enough to build modular transformations on them. It is not my purpose here to present a solution, but those familiar with my past work will not be surprised to hear me claim that the root cause of the problem is that we have adopted the wrong way to encode such structure: text strings. Strings (or the morally equivalent ASTs\/DOMs parsing them) are not a useful way to represent complex information artifacts. I claim that once we get over our textual fixation, the transformation challenge becomes tractable. By an odd coincidence, it turns out that the internal data model of Subtext is close to a solution.\n<\/p>\n
Meeting the challenge of transformations would yield a pleasing result: that instead of building a system as a set of artifacts with subtle internal interdependencies that must be manually maintained, we can instead plug together a set of modular transformations. Note that the data definition artifact itself can be replaced by a history of evolutions, seen as transformations, starting from the empty definition. It’s transformations all the way down. We win because the transformations are modular, unlike the structures they generate. I call this approach Transformative Programming<\/em>.<\/p>\n","protected":false},"excerpt":{"rendered":"
I got some good feedback here on my last paper. I want to see if I can get some equally good advice about what my next paper should be. My past papers have been about making programming easier. I would like to move on to making programming more powerful. Specifically, by making transformations and views … <\/p>\n