Dynamic Aspects has published their upcoming OOPSLA presentation of domain/object, the language underlying their Java IDE. There is a remarkable correspondence between their ideas and my own. This makes reviewing their paper a difficult proposition: it would be easy to nitpick over differences; but it would be equally easy to falsely read my own ideas between the lines of theirs.
Their primary objective is a fancy Java IDE called domain/j that supports what I call semantic editing: direct manipulation of an abstract, non-syntactic model of a program. The IDE is implemented as an incremental bidirectional transformation between the syntactic and semantic structure of a Java program. The incrementality provides the same abilities as Eclipse’s incremental compilation: being able to get live semantic feedback while editing syntax. It is the bidirectional nature that is novel, allowing semantic level edits (which they call micro-refactorings) to be transformed back into syntax changes. This reverse transformation allows direct manipulation of the program at the semantic level.
Domain/object is the language that implements this bidirectional transformation. It provides facilities such as persistence, versioned execution, transactions, and bidirectional execution. At base it is a LISP-ish data-flow language embedded syntactically within Java. Eclipse internally implements some of the same capabilities, albeit in a more restricted form. For example, bidirectional transformations on the internal semantic model are supported through the mechanism of "live lists". Eclipse has internal persistence and versioning models as well. The point of domain/j is to provide these capabilities in a simpler and more general fashion by building them into a language, avoiding the massive complexity of the Eclipse API’s. Domain/j is essentially a Domain Specific Language for building IDE’s.
Domain/object is supposed to have a "live" execution model like Subtext, in which all code is always executing, eliminating the distinction between compile-time and run-time. But I do not understand how that is achieved, since it is embedded within Java and thus must be compiled prior to execution. Perhaps this is a meta-circular thing, whereby their Java IDE is lively interpreting the domain/object code. This would imply their IDE makes Java fully live, a significantly larger accomplishment than they seem to be claiming.
I wholeheartedly agree with the grand strategy they call "semantic computing", which (reading between the lines now) is what I call Structure Oriented Programming. The idea is to build systems through bidirectional transformations, combinations, and queries between different sorts of structures. It would be nice to construct typical business applications as a bidirectional transformation between a UI and a database. This is the kind of technology needed by DSL workbenches and Model Driven Architecture.
Where we disagree (nitpicking now) is on how conceptually elegant the language needs to be. I have been exploring along the same lines for a long time, but have so far found it to contain nasty semantic black holes that require ugly ad-hocery to avoid. I detect hints of the same problems in their paper, for example in handling the indeterminacy of reverse execution, and in handling the impact of asynchronous side-effects on the data-flow model of execution. I imagine they can come up with workarounds in the context of their IDE implementation, but I am less sure that the result is a clean and simple language that can be widely applied elsewhere. Perhaps they are clever enough to solve these problems – I look forward to seeing the details. Or perhaps I am just too finicky, and these are not actually problems in practice.
My approach to the problems of mutable state and side-effects is to develop a model of computation that treats time and change in a more tractable way. My intuition about bidirectional execution is that it should not be a universal property of the language, but of a select set of higher-level constructions. Benjamin Pierce has done some work that may be relevant.
In summary, Dynamic Aspects is doing some interesting and potentially important work. I am happy to find other people of like mind, and I wish them the best of luck. The one piece of advice I would give them is to tone down their claims, which are so dramatic as to invite skepticism. I look forward to attending their presentation at OOPSLA.
Jonathan,
(I hope you don’t mind my cross-posting this rather long reply to your blog and mailing list. Please let me know if there’s a different protocol for responding to your messages!)
The similarity between Subtext and domain/object is indeed striking, and rather exciting. I think it bodes well for both projects: it lends further support to our belief that declarative dataflow languages will be the next big revolution in software. This in turn meshes well with the idea that massive parallelism is likely to be the next big revolution in hardware, since dataflow computing and concurrency were made for each other.
I think you’re right about structure-oriented programming being what we call “semantic computing”. (Your rather satisfying coinage was such a natural fit for our thinking that within hours of first hearing it we were all using it here.) The only difference with “semantic computing” is perhaps one of emphasis: while you’ve focused on the programming activity, we emphasise the applications of the paradigm to many areas of computing. But in the end the difference is probably smaller than it seems, as fundamental to the perspective seems to be the ultimate loss of any real distinction between “programming” software and “using” it. (And this itself is hardly a new idea, of course, as people – usually Lisp programmers, strangely enough – are fond of pointing out.)
Feel free to nitpick to your heart’s content when it comes to conceptual elegance, by the way. It’s a deep and important topic, which is far too often neglected when it comes to software. I guess it’s kind of where the distinction between aesthetics and form and function blurs away, when you get down to the very simple.
Bidirectionality is an “extreme experiment” at the moment. To give you further background, domain/j currently runs on an earlier version of domain/object, which provides a “framework” for providing inverse semantic mappings for individual relations – a considerably more structured approach than Eclipse, but still an almost entirely manual process within that framework.
The version of domain/object under development offers more than a framework for bidirectionality but at the cost of having to confront the non-determinism problem almost immediately. Nonetheless, I think it’s a problem that our industry will have to start solving fairly soon, and some degree of ad hocery will perhaps be inevitable and indeed acceptable, as long as it doesn’t pollute the language itself. (Talking of extreme experiments: aren’t your “adaptive conditionals” related to modality in the same kind of way as bidirectional execution is, or could be?)
I guess I don’t fully understand the issue with mutable state and side-effects. More precisely: we don’t seem to have an issue here, which probably suggests we’re just a bit behind you in our thinking. (I’d love to see an example, if you can be bothered.) We have had to introduce some notion of time, or rather of a time interval during which everthing is effectively simultaneous. The time intervals can be sequenced and nested. The intervals correspond roughly to nested actions or transactions. I think they are effectively a projection of the history of the dependency closure of the initially modified object. We’re at a fairly early stage in our thinking on this subject, so there are probably some bugbears lurking in there that we simply haven’t hit yet.
To be absolutely clear about the “liveness” thing: domain/object does not make *Java* live – we couldn’t do that without changing the semantics of Java, and moreover implementing our own JVM on top of domain/object. In fact, domain/j only has a static semantic model at the moment, and as I mentioned, isn’t yet running on the latest version of domain/object. (The migration will be a significant activity, but will deliver domain/j’s versioning and persistence capability.)
But a domain/object application is fully live (modulo any currently disconnected program fragments, which will only be synchronised on reconnection) in the Subtext sense: execution is by dataflow only; assertions, tests and type invariants and so forth must hold generally not just when control flows over them. The only qualification is that (Java) “control” must have flowed through a domain/object embedded expression at least once in order for it to become “active” in the first place, otherwise domain/object simply doesn’t know that the expression exists.
Oh, and thanks for the advice about toning our claims down. (You’re not the first to offer thoughts along those lines, by the way.) It’s a difficult and somewhat annoying balance we’re trying to strike as a commercial outfit. Generally we’d prefer to make more humble claims and let the software do the talking. I guess we’re trying to locate ourselves in the same commercial space as Intentional Software, which is fairly grandiose in its claims too (“the death of programming languages”, etc). But I’ll see what I can do to twiddle the dial in the direction of increased plausibility.
On that note, I’m looking forward to the OOPSLA presentation, as we’ll be in the land of concepts and ideas which are interesting in their own right rather than the world of marketing and commercial yukkiness where I feel decidedly uncomfortable. Certainly we won’t be trying to pitch domain/object as anything important; just something interesting, hopefully.
Finally, by way of friendly counter-suggestion, I have a thought of my own to offer about your positioning of Subtext. Feel free to disregard entirely.
One thing that strikes me (especially having read some of the rather unfair, and at points silly, flaming on LtU) is that you could adopt a slightly more accommodating position to good old fashioned text without compromising any of your principles. I fully agree that text, at best, must be demoted to “just another model”…but one way of looking at this is that to do so is actually to *elevate* text to its true potential as a rich, structured, semantic artefact. “Text” doesn’t have to be synonymous with uninterpreted ASCII characters.
To try a bit of nitpicking myself: what’s really the deep difference between one of the 2-dimensional projections of a Subtext program in your Swing GUI, where fields have textual labels and so forth, and a bit of text rendered in a well-implemented editor with an integrated incremental parser? (I’m not suggesting *we* have such a thing yet. But it’s been done several times in the past, and seems rather compatible in fact with the structure-oriented perspective.)
What if you allowed the user to add “nodes” by hitting carriage return and typing the name of the new field? Isn’t that kind of like interacting with a piece of structured text? Aren’t anaphoric names a powerful concept? You’re already familiar with all of these arguments, of course; all I’m suggesting here is that you could quite easily adopt a more pragmatic take on text without giving up anything of your overall vision, and perhaps avoid responses like the over-reaction on LtU.
Well, we’re looking forward to meeting you at OOPSLA, and I’m personally hoping to make a better impression than I did with my bumbling, mumbling, half-hearted (and ultimately unsuccessful) attempt to make you come and see our demo last year. Onward! and upwards.
Roly
Roly,
Thanks for the thoughtful comments. This is blogging at its best.
I agree that bidirectional computation is a deep matter, presenting both great challenges and rewards. My early attempts were unfruitful, so I am focusing first on the simpler case of unidirectional computation. I look forward to learning more details about your approach.
Here is an example of the classic problem of side-effects in functional languages. Say you are modifying some global data at one point in a program, which at another point also depends on reading the global state before the modification. Therefore you need to establish an execution order within the program so that the read happens before the write. But one of the major advantages of data-flow and functional languages is that they do not specify execution order, leaving open change-driven, demand-driven, memoized, and parallel execution schemes. Side-effects are closely related to input/output operations, which need to be both strictly ordered and execute only once. Functional languages have struggled with these problems for decades. The current favorite solution is Monads. How does domain/object deal with this?
On the “liveness” issue: you seem to be saying that a running domain/object program functions as a big data-flow network which automatically propagates changes throughout itself. That is great, but full liveness means additionally that changes to the program code itself are reflected immediately and automatically in its execution. Tanimoto calls this level-5 liveness. I don’t want to be pedantic, but since you cite him yourself I assume you are using the standard definition of the term.
You are probably right about the proper attitude to take towards text. I have taken an extreme position as a guiding principle. The standard techniques for encoding programs in text files are insidious. I continually find textual influences infiltrating my thinking. An absolutist rejection of text helps me defend against such reactionary tendencies. It also sharpens my message. Creating consternation and outrage on Lambda the Ultimate is not necessarily a bad thing. In the end, once I see how far the rejection of text takes me, I will need to come full circle and embrace text in its right and proper place.
Jonathan