Subtext faces enormous obstacles to becoming an accepted programming language. It is still just a sketch, missing crucial pieces. But even as I slowly fill it out, I am encountering unyielding resistance. There is a great deal of skepticism about the whole idea of non-textual programming, due to the past failures of Visual Programming Languages. Overcoming that bad reputation will require proof that the UI and VM can scale to industrial-strength, and that there are unequivocal productivity benefits. I can’t do that alone, and it seems that I won’t get much help (particularly from the academic community) until I can first overcome the skepticism. Chicken and egg. Modern programming languages have dug a deep rut over the last 50 years, and dislodging them will not be easy. It will require finesse.
I am thinking of repositioning Subtext as a modeling language. There are a number of advantages to this approach:
- The dominant modeling language, UML, sucks. It is an ugly stew of informal diagrams with no coherent design or precise meaning.
- Non-textual representations are accepted in modeling, as are fancy CASE tools. Modelers are not in thrall to Emacs.
- Example-centrism should be quite valuable in modeling, because it helps communication with users. That is the whole point of use cases.
- Flexibility is paramount during design. A key goal of Subtext, admittedly not yet realized, is to increase the flexibility of program design, even at the expense of efficiency and elegance.
- Performance is not a requirement. The ability to execute only small-scale examples would be perfectly fine during modeling. In fact the ability to execute models at all would be a big advance over the state of the art.
- Programmers could still use conventional languages to implement the model. Code could be generated from the model in the usual (typically ugly) ways.
- Example-centrism would allow the model to serve as a test suite for a separate implementation.
- Small-scale systems could be directly implemented by the model, without the need for a "real" programming language. This benefit would justify incremental R&D investment to scale up the modeling language, finessing the chicken-and-egg problem. Conventional languages would slowly retreat into the role played by Assembly Language today: optimization of performance-critical functions.
This possible marketing strategy suggests a specific technical direction to take Subtext. Most modeling methodologies center on data modeling, as in UML’s class diagrams, and the earlier Entity-Relationship diagrams. Accordingly, Subtext would need to become a database, with schemas and queries.
This requirement raises a deep and old problem: how can the semantics of a database be cleanly unified with that of a programming language? It is sometimes referred to as the "impedance mismatch" problem. Prolog is the only good solution so far. I have been trying to solve this problem with a normal functional semantics for a long time. Maybe it is time for another try.
I think you should use Subtext to describe what the program really does, and why it does it so.
Main stream programming languages (at least Algol-like, imperative procedural) instead describes the code a compiler needs to build a program that does it.
The code includes procedures, each with a memory of variables that can be used anywhere and anyhow in the procedure, and the procedure organized by key-words into nested segments, that can be chosen, repeated and included in the program thread by control structures (which really are nested control statements with an explicit inline-definition and an implicit component-interface).
The use of the variables (dataflow, program control, state, reference or whatever) are being described one statement at a time, with all the statements in a multitasking procedure, ordered in the order that does all the tasks and dataflows in the intended (but unspecified) ways.
And here comes the kicker:
All the statements in the procedure can do a lot more than what the procedure shows. There is another side hidden in every statement, with all the flows to a place somewhere in the system where the state and the action is, and where events happens in some order, together with the events from other statements or procedures.
And there is code in the statements that does the communication and synchronization (or messaging) with the system, with synchronized or asynchron execution of the action in the statement.
I am sure you could improve the semantics in that program a few notches, maybe even make it easy to understand and edit, and expressive enough to make it possible for a higher intelligence than the compiler to create the source with the right order in the procedure (or orders, if multithreaded) without further help (when you first have specified the syntaxes for the necessary orders and the designed orders in the programs behavior).
Subtext has some strong properties that makes it a better fit for describing the semantics in a program:
– It is an information-structure instead of a syntax-imitating text, so it can contain more information, without the information comes in the way.
– It is a structure, so it can describe the structure directly instead of invent a structure-describing syntax.
– It uses nameless links instead of variable names to specify the flows, and can show all the flows through the same value (or to and from the same state in a variable).
– It shows a dynamic presentation of the information, so it can show relevant information and hide irrelevant information. This makes it possible to show functional structures instead of the structures in the text or the syntax, so the useer can see the program one task at a time, in all the used places and structures (which can be presented in different windows or panels).
– It can follow programflows and dataflows during the execution of a task in the program, and go through interfaces from one task to another task in the same dataflow, instead of following the multitasking procedure.
– It can use labels to show the function in the program and reduce the complexity, by creating abstractions.
– It can show the structure in the logic and programflow in the procedure, and create synchronized components from control statements, with specified semantics, state, interfaces and dataflows.
It was a lot, but it is a lot in what a programmer must do that never leaves a recognizable trace in the code (the so called “source”) to the dumb compiler.
Wow. I feel for the difficulty of breaking through skepticism, but I think repositioning it as a modeling language is _exactly_ the wrong idea. You’ll still face the same resistance from academics and “experts” to whom your ideas are a challenge to their preconceptions, but as a modeling language you’ll be that much further from the pragmatic power of “but it works.” If you position it as a modeling language, _every_ conversation you have will ask if it’s UML and when you say “no,” there will be a slight frown and deaf ears.
On the other hand, if what you’re saying is “position it as a graphical domain-specific language” then I think you might be able to finesse your way past the UML forces and into the realm where people look at interesting new approaches.
Please no. Most experienced developers I’ve talked to really get it. They work with programs at the conceptual level with tools like IntelliJ, ReSharper or Eclipse, and understand that the source text is just a representation of the program, not the program itself. They want more than that, and Subtext looks like the way to go.
However, those same developers see UML-type modelling as not very useful. Sure, they sketch some UML on a whiteboard or paper while discussing their program designs but they don’t use modelling tools or MDA. If you turn Subtext into a modelling language you’ll lose the people who currently find it interesting.
I think it’s too early for Subtext to reach critical mass. And who cares about the reaction of academics? Why not invite more developers into the project? Run it as an open-source project of the Beneficient Grand Dictator model and wait for momentum to build.
I think this could be a very good use for Subtext, although a large part of me would still love to see something like this work for large scale application development. (Maybe I should just use Self…)
Subtext seems to fit the modeling language are pretty well because, from what I can tell, it is very similar to standard data-centric languages like the whole SGML family. Except that Subtext’s data is actually code, and it is a running simulation. So I guess I’m trying to say, Subtext is like an application simulation and the data it is simulating is “written” in Subtext.
If it means that Subtext would be available to people sooner, I’m all for marketing Subtext as a sort of unification of application modeling and application prototyping.
—
As a clarification of my previous message…
From my point of view, Subtext is basically a combination of a data-flow language and the “always-on” concept from Smalltalk and Self. (Please correct me if I’m incorrect about that.) For this reason I think that “marketing” Subtext should at the very least make apparent those two aspects of it.
If it were advertised as a straight modeling language like UML, it would very much confuse people because modeling languages are not running, most times not even runnable, and thus do not have any actual data in them normally. Obviously you need to incorporate the ‘modeling’ side of things because Subtext is very clearly not an imperative programming language.
It also might be worthwhile to see if there is any interest in the simulation crowd. By its nature as an always-running language, and by the fact that the user-interface of it can (well should) be customizable to fit the domain, Subtext is essentially a pre-built environment for running simulations.
In any case, I would still really love to see some code available so that people can interact with it, build on it, and generally help test out ideas, as well as be able to point to Subtext and say “Check this thing out!” (Without having to add “….when it’s done.” On the other hand, I had a terrible time understanding your latest paper (I don’t really know anything of lambda calculus), so perhaps the code wouldn’t help me any.
—
Charles Ellis
Jonathan,
Here’s a brain dump in response to Plan B:
If Subtext is to be a modelling language then I believe it should
model systems that are not being modelled and certainly not being
programmed. In particular, Subtext seems to be uniquely designed to
model/implement web-based applications. This decidedly has nothing to
do with SOA (yuck!); it has a great deal to do with semi-structured
data and computation.
Most semi-structured work centers on developing a query language for
web information. That’s good, but Subtext could and can offer much
more. We are seeing a blurring (disappearance) of the boundaries
surrounding computational entities out on the Internet. And these
entities are being almost arbitrarily being coupled together.
Pi calculus, mobile ambient and, more recently, bigraphs are being
used to model the semantics of mobile computation. Mobile doesn’t
necessarily mean a change of physical location; it can also mean a
change in the configuration of the network – a very common event on
the Internet.
Luca Cardelli (of ambient calculus fame) has a paper that shows
different ways to represent edge-labeled trees. Also this paper,
http://research.microsoft.com/Users/luca/Papers/Semistructured%20Computation.A4.pdf
The Ambient Calculus uses these tree structures to describe mobile
computation, which is seen as the evolution of tree structures over
time. Therefore, the full Ambient Calculus has both a spatial and a
temporal component. The spatial component consists of information
trees, that is, semistructured data. The temporal component includes
operations that locally modify the spatial component.
XML-centric solutions are being proposed for web computation but most
of the proposals simply use XML as a way of avoiding syntax and
enhancing interoperability. What’s missing is the deep semantic
underpinnings of Subtext.
There is no modelling system that allows one to create and run
networked applications; Subtext is well on the way towards that end.
I would also suggest not getting too bogged down with the user
interface. That is, don’t let the user interface drive the design.
My 5c.
Peter
—
Peter C. Marks
“Eschew State”
IMHO, SubText is a prototyping language being a direct descendant of spreadsheets.
The only fine point of difference being in spreadsheets, cell values are first class; In subtext cell formulae are first class macros. Having first class cell values are problematic since it goes against the principle of having the least number of formulae on a program.
Perhaps subtext is an extended spreadsheet environment capable of being a code generator, one that combines specification with test results? I recall in As-Easy-As … as spreadsheet clone from the 90s, where a group of cells could be defined as a formulae, with some being inputs and one being the output value. It turned out to be of immense value. I believe Dan Bricklin hit a sweet spot with spreadsheets, because it offered some structure, but not too much structure for applications to be quickly developed.
Perhaps subtext is an embeddable grid / spreadsheet component, which can be used in running programs? To this day, programming environments are notoriously weak in displaying /specifying spreadsheet-like data.
Perhaps subtext is a scripting language add-on? One that handles the functional parts of a program and at the same time provides in-built Fitnesse-style test harness.
I’ve prototyped many programs using a spreadsheet, and often wished the source could stay in spreadsheet form because it is easy to test and verify.
On Apr 25, 2006, at 7:41 PM, Jonathan Edwards wrote:
> Subtext faces enormous obstacles to becoming an accepted programming language. It is still just a sketch, missing crucial pieces. But even as I slowly fill it out, I am encountering unyielding resistance. There is a great deal of skepticism about the whole idea of non-textual programming, due to the past failures of Visual Programming Languages. Overcoming that bad reputation will require proof that the UI and VM can scale to industrial-strength, and that there are unequivocal productivity benefits. I can’t do that alone, and it seems that I won’t get much help (particularly from the academic community) until I can first overcome the skepticism. Chicken and egg. Modern programming languages have dug a deep rut over the last 50 years, and dislodging them will not be easy. It will require finesse.
This argument makes sense to me, and I think the idea of finding a niche market where subtext is well suited to that particular application and can get established is an excellent idea (though I can’t evaluate it’s use as a modeling language; I don’t know enough about how UML is used). It strikes me as taking a strategy parallel to the “disruptive technologies” described in Clayton Christensen’s book; the new technology gets established and evolves within a small market niche, and evolves in overall capability faster than the mainstream technology, so moving out of that small niche.
Still, I’m a little surprised your thinking about positioning of the language before taking a couple of (IMO) obvious steps to increase mindshare around subtext. Using myself as an example (15 years in programming, eclectic academic background): my initial reactions to subtext were:
* “Neat concepts, but if the key is usability, the proof is in the pudding. I want to play with it.”, and
* “How is it going to scale to any large software project?”
So when I think about how to get subtext adopted more widely, I immediately want to see two things: A general release that people can try out, and an example large application written in subtext to prove that it’s more than a toy. (Historically the latter has often been the application itself; Dogfood that puppy! :-}). And the first might help you lead to the second; giving other people the ability to (e.g.) integrate libraries into subtext might get a lot of the work of building a platform for full applications done for you. So how does your reasoning differ from mine, that you give focussing the language on a particular market niche higher priority?
(Please don’t take this as negative feedback; I do like the language and would like to see it succeed. I’m even excited about you taking a swing at the impedance mismatch problem in subtext. But it’s not clear to me it’s the right pathway for market adoption).
— Randy Smith
It’s really great to get all this feedback! I have posted all the email comments onto the blog.
As several people point out, the main goal should be to make Subtext real. There are only two problems with this. First, Subtext is still incomplete: there are several more major ideas I want to include before I release it into the world. The database unification mentioned before is one of these ideas.
Secondly, getting real is a LOT of work, especially because it will require new implementation techniques, which means a lot of risky R&D has to be done. This is way more work than I can handle. I need to figure out how to seduce other people into doing all this work. One way is by getting the academic community interested. So far I have failed at that, but it may just take persistence. Another approach is to find some niche where an immature and unscalable language technology can still flourish because it has some unique benefits. That is the idea behind modeling.
Several people pointed out some disadvantages to this plan. Basically, the popularity of UML calls into question the judgment and values of modelers. Maybe I need to keep looking for a niche. Nevertheless, adding database features to Subtext may be a good direction to head now. It will surely require a redesign of the language semantics, so I want to do it before worrying about optimization techniques. And DB features ought to make demos more impressive. Giving good demo seems to be the main strength of Subtext at the moment. I should stick with what’s working.
I think I might have posted this link once before but it seems relevent to Plan B. http://groups.inf.ed.ac.uk/links/ is a link to the Links project which is led by Philip Wadler. “Links will solve the impedance mismatch problem by providing a single language for all three tiers.” The three tiers correspond to the browser, server and database. I personally think that starting with the idea of collapsing three or more languages into one is the wrong approach (if they are not careful they might end up with another PL1!). I’m sure they will spend a great deal of time developing a syntax. Links seems to be accepted (at least academically) probably because the people involved are not taking any big chances; there is nothing very disruptive in its ideas.
Instead,I would rather see a complete re-examination of what lies underneath the World Wide Web. And then take the Subtext approach of banishing syntax, treating program as data, history, etc. This could be a parallel effort with the same goals as Links but with the Subtext approach. This would mean initially abandoning the UI demo and focusing more on the development of a system/infrastructure that works with and within the Internet.
Peter
Interesting about the Links link. I’ve been a Flash/Actionscript developer for quite a while now, and the guy that wrote the open source AS2.0 compiler is now working on a project called Haxe (http://haxe.org/). A single language which targets 3 different “platforms” – the Flash VM, Javascript and bytecode runnable via an apache mod.
To try to stay on topic, i’m happy to see some reasonable arguments against the ‘subtext as a model language’ – it didn’t sit right with me, but i was unable to express why.
While it may be an unfinished tool, and may yet radically change, i think its obvious that there is a core of developers, if not academia, who want to both play with the app itself, but also, possibly, work on the code base. I don’t know if sourceforge is the place for it, but starting to draw in the existing community seems like a plus. Other then the issues of dealing with support and management of a quasi-open source (or fully) project.
I think you should add a more “traditional” mode to Subtext:
Instead of constructing the same code many times and execute it one time it would construct the same code one time and execute it many times, with different values (and different codes, if it contains conditionals).
Instead of showing the whole thread in one plane you could show every use of the code in the thread in one plane, and navigate through the planes in one presentation of the code, with two values (current and next, or previous if you goes that way).
Then you would have something that is more like the code that would implement the program, but still keep the automatic construction of the code during the execution, so you can see all the paths you must support, confirm that all the code beeing used and declare asserts that can function as instant unit-tests.
In addition to the planes in the execution of the code you would have the planes in the code, with different alternatives in certain places.
Therefore you should have a way to choose between all the variants of the code within a loop or stack (and the asserts) without seek out all the places where the code in the loop can have alternatives.
Besides Links, there’s Microsoft LINQ which try to solve the impedance mismatch problem. Good people from the Haskell community are behind it.
As for Subtext, I concur that it would be interesting to 1) experiment with it and 2) have examples of real systems built with it to look for. Couldn’t you get students to work on it?
Regarding repositioning it as a modeling language, I don’t know. It may work, but the modeling community is trying to transform UML into a full-blown programming language. Wretched, I know, but I don’t think they would be open to a new modeling language.
The main problem with getting students to work on it would be finding the right students. 🙂
Sorry to say… but I think you’re a bit on the wrong track with this whole project… I think you need to work WITH what we already have, and not try to replace everything with something completely new….