The Subtext UI sucks. One of the biggest problems is linking: the welter of compasses and vectors is bewildering. We need something that is much easier to read. I have been thinking about an approach that borrows from Alice.
Here is a screenshot of an Alice method:
Alice is a conventional OO language with a visual syntax based on nested “tiles”, which are just small forms with labels and fields. Squeak Etoys is similar but less slick. Alice uses a visual syntax that directly maps to standard language syntax. Nested tiles map to nested expressions. Sequences of tiles (as in the “Do in order” tile above) map to statements in begin-end blocks. In both of these cases, what are explicit links in Subtext are implicitly specified by physical adjacency. In other words, syntax.
Let’s face it: syntax is a simple and intuitive way to represent structure. But a fundamental principle of Subtext is to avoid syntactically encoding programs so that their semantics are explicit and immediate. It should be possible to use syntactic techniques as a visualization in the UI instead of as a representation of the source.
Here is factorial in Alice:
Here is factorial in Subtext:
Here is a new view of factorial:
Every box (tile) corresponds to a non-leaf tree node. Labels are black, and links are blue. Values have a gray background. Every function has a primary input argument with a blank label, and a primary output argument with the label “=”. There are three techniques used to represent links. The first is to represent a link to an input argument by preceding its name with a “$”. A link to the unnamed primary input argument thus becomes just “$”. This is a familiar convention from scripting languages.
The second technique for representing links is to vertically stack functions which are chained from output to input. The chain from Subtract->Factorial->Multiply is an example of this. Thus what would normally be written as “H(G(F))” becomes the stack F;G;H. One could argue that this is more readable than the parenthesized notation. Note that these vertical stacks will work nicely for histories of actions too.
The If function is an example of the third technique. It has a secondary input argument (else) that is linked to another function. The If box is layed out vertically, with the linked functions stacked to its right, and enough vertical white space added to avoid overlap. Note that this layout is automatic. It is based on determining a spanning tree within the graph of all intra-function links.
This layout provides a natural reading order. Normally we read from left to right on a horizontal line, then continue from the left side of the next line. Reading the Factorial function in this way gives you a sort of linear narrative of executing the function. Instead of using parentheses to stack up computations, the text is moved to the right of a vertical tile. Function tiles may be layed out horizontally or vertically, but maintain the internal reading order of their contents.
Here is a variant, in which the variable references are replaced with explicit arrows. Perhaps the vertical stacking reduces the clutter enough to make these arrows acceptable.
In fact, maybe the vertical stacking is all we need, and horizontal layout to replace parenthesized nesting is not needed:
Here the pink regions indicate consecutively linked components. This layout is simpler than the previous ones, at the cost of making the data flow harder to comprehend.
What do you think?
I like the idea in general, but I\’m worried about too many arrows all over the place. For example, in the subtract/factorial/multiply stack there\’s a little arrow from the end of each line to the beginning of the next. This seems like a more complicated version of a calculator paper tape. For example, on a calculator tape you might see
3
x4
-1
+2
__
(That doesn\’t look so good in this non-monospaced font, but you get the idea)
In that case, the convention is that the result of the operation above is fed into the operation below. Perhaps there is some way to do something similar to avoid the arrow problem (assuming it is a problem. Maybe it\’s just me).
So, perhaps
$
-1
factorial
x$
=
or:
$
subtract 1
factorial
multiply $
=
The idea would be that it\’s implicit that the output of the previous step is an input to the next step. It might not be bad to have little arrows (I\’m bringing them back already) simply pointing to the next step in the center of each item. Hm. It kind of sounds like I\’m describing Apple\’s Automator workflows.
Anyway, just my $.02.
[David – I tried a number of approaches like what you describe. The problem is to allow for the full range of possibilities: arguments can not only link to a previous function but also to literal values or arbitrary variables. I tried using icons like an up-pointing arrow to indicate a data flow from the previous function. But this seemed like a cryptic symbolism, while the explicit arrow is clear.
Thanks for pointing out Apple Automator. It looks nice, but seems very limited. You can only parameterize actions over a single input, typically a list. All the other arguments seem to be hard-wired to constants.
— Jonathan]
I guess what I’m getting at in general, is that if you’re getting away from text based programming, don’t feel that things need to be little left-to-right sentences.
Been looking forward to an update on what you’re doing with Subtext.
The other day I formalized (more or less) a diagram method for program flow and structure, and I started to look at if it could be adapted for a Subtextual-like usage. I’m currently planning to build a tool that can make it really easy to map out the code you need to create with this diagramming, and then generate Python code (and maybe other languages, too) from this diagram. Not templates, but the actual working code. Of course, with a lanuage like python or a target like .Net, I could do that transparently at runtime, and this is part of what made me think it might be adaptable for Subtext.
I have some work to do on it and to have the time to do a proper write-up.
I think it ends up harder to understand with arrows everywhere than traditional code. Just because expressions don’t appear in their place of evaluation:
if (x==0), 1, (* x, factorial(- x, 1))
Hello Jonathan,
You state in “First Class Copy & Paste” that “… a weak form of reactivity is already present: data-flow computation.” I’m wondering if, rather than have the UI focus on the logical flow of the program, instead the UI should focus on the data flow of the Subtext program. Then, of course, the arrows become important.
There are arrows – data pipes – in the present and proposed Subtext UI’s. But, I believe, there is a tension created between the organizing principle of the current UI, which seems to be syntactical, and the importance of showing how the output of one function feeds the input of another function.
I haven’t thought this through much further. I probably should’ve included some diagrams, although there are plenty of graphical data-flow tools with examples.
Regards,
Peter
I recently came across this project. It is graphical way
Let me try to summarize the options for visualizing “reference” I’ve seen:
– lines on the screen (the ariade thread – drawing a line for you to follow)
– compass (pointing – giving the direction, but not the precise distance)
– hyperlinks (invisble, jump when clicked; possibly with additional interaction, such as pop-up menus)
– name equality (as in textual programming languages and natural language)
– spatial adjacency (also useful to make the 2d representations more readable)
– order (refers to predecessor/successor) and indentation level (refers to parent and siblings)
– equality of presentation attributes such as color, shape, texture
Whose choice – the computer’s, or mine? For example, a clever layout algorithm might switch from line to compass or to name to avoid intersection, or if the distance gets too large. (when a program gets too large for the screen, I cannot recognize the indentation relationships any more, so I start factoring out sub-functions, essentially replacing spatial layout by naming).
Then there’s also the temporal aspect:
– switch to a more precise representation when link is about to be activated (for user feedback)
(e.g. compass to line, hyperlink to text)
– folding and unfolding of subtrees based on current user interest
(e.g. unfold a target node before jumping to it; hiding irrelevant detail)
Does there have to be one “master copy” “source” diagram, or is a diagram just a view? Could there be several different diagrams for the same program at the same time? Just like naming, spatial arrangement has no semantics to the programming language’s execution, but can carry information for the reader.
If you want to go 2D: I once saw an interesting demonstration of a diagramming tool that could zoom into and out of hierarchical components (fish-eye style), see http://www.ifi.unizh.ch/req/ftp/papers/ICSE2003_tool_demo_paper.pdf
> Thus what would normally be written as “H(G(F))†becomes the stack F;G;H.
I believe this is what monads do, too: masking the state parameter.
Regards,
Axel
Whoops! I truncated myself. My mistake.
I recently came across a project (I think it is defunct) that has a dataflow UI interface; that’s not the interesting bit, though. What’s interesting is the application: a visual way of creating XSL Transformations. XSLT is a functional language and one that, I think, would create an effective demo.
Regards,
Peter
OK. It’s not my day…
Here’s the pointer: http://jamper.sourceforge.net/
Please take a look at the pictures at the bottom of this page.
Peter
Hi Jonathan,
I firstly came across some weeks ago and I like your aim to change the way of programming, but you are right: “The Subtext UI sucks.” Maybe you want to take a look at my damn draft*… I wouldn’t 😉 you will understand when you take a look. It’s a work of some minutes infront of my TV.
Some notes on the UI:
– The arrows should by light, especially the arrows which go straight over the body of such a function, cause they harm the readability. But when you move the cursor over the beginning or end of such an arrow, it should be drawn solid.
– I always treated the arguments of a function as part of the identity(name) of the function, so you could put the input fields of a function into the label of the function(and also the result):
Factorial 3 = 6
(…body…)
Multiply ? times ? = ?
(…body…)
so you could click the body of the function away but you would keep the example-values and the ” = ” would be part of the label-string. The “?”s are linkable with the “code“ in the body.
– At the end of each arrow the field could be filled with the label of the referenced-value, so their would be not so many empty fields like between „Multiply ? times ? = 1“.
– How would you handle arrows like this? (Same letters (ABCDE) have to be linked)
Foo A with B = C
( Bar B on A = D )
( Crap D at B = E )
( Joke E to A = C )
many crossings between the arrows…
*) this draft doesn’t contain everything I want, but I had time and still want to publish it here
244 KB
http://www.uploadsky.com/files/5/a%20draft%20of%20a%20subtext%20editor.jpg
When you wants to show nesting as a vertical, linked flow (new UI 2), i think it would be better to link the expression to the result of the expression where the result beeing used (and where the expression should be nested).
This should give a more direct and clear flow.
Besides, if the definition of a function only is a single operator then there is not much use to give the expression a name.
If you wants to name it you should name the result, depending on what the result represents in the program, rather then name the way the result beeing computed.
Looking at the above examples, I wonder if perhaps you are attacking the wrong level of representation. Are the above graphics really easier to grok than the text form?
int factorial(int v) {
return ((0 == v) ? 1 : (v * factorial(v – 1)));
}
Now if I had a live evaluation gone wrong, a drill-down from the text form to decorated graph could be wonderful.
I think what we are all looking for is a better representation for program structure – textual, graphic, or whatever combination works. Both are representations for the underlying prorgam structure. Which representation is better when … is the question in need of answer.
BTW, the over busy diagrams above could be cleaned up considerably by presenting the diagonal arrows only on mouse-over. No point telling the user (all the time) about aspects not of immediate interest.
I think the best solution is to have a dynamic view system where you can rearrange, hide, show, zoom, group and move things even automatically as you navigate. The GUI have to answer and shape under our hands (whatever this means 😉 )
It is also important to switch between different views through animation because it is easier to follow for the mind.
Hi… I remember seeing SubTextual a while back, thinking the UI is a bit awkward (ugly), but also one of the better attempts I\’ve seen. I definately gave it a good look. I happened upon your new idea and I think it\’s a step in the right direction.. but, since I\’m here, might as well through my two cents in. I intuit a user-friendly \”abstract syntax tree\” represented as an actual graphical tree-list would be perfect:
(PRETEND THIS WAS A GRAPHICAL TREE-LIST VIEW (FULL EXPANSION))
-DECLARE_FUNCTION
_-FUNC_NAME
__-Factorial
_-ARGUMENT
__-INT
___-INT_NAME
____-num
_-IF_ELSE
__-IF
___-IS_EQUAL
____-INT
_____-0
____-INT
_____-num
___-RETURN
____-INT
_____-1
__-ELSE
___-RETURN
____-MULTIPLY
_____-INT
______-num
_____-INT
______-FUNCTION
_______-FUNC_NAME
________-Factorial
_______-ARGUMENT
________-INT
_________-MINUS
__________-INT
___________-num
__________-INT
___________-1
_-RETURN
__-INT
___-1
Long? Sure.. Ugly? Maybe at first, but it\’s easy for me to understand. But what\’s interesting is the things you could do with the context menu on the above tree. What if we right-clicked \”DECLARE_FUNCTION\” and clicked \”hide types\” and then right-clicked \”IF\” and clicked \”hide body\” and did the same for the last \”RETURN\” statement. Then our tree would look like:
+DECLARE_FUNCTION
_-Factorial
_+ARGUMENT
__-num
_-IF_ELSE
__+IF
__-ELSE
___-RETURN
____+MULTIPLY
_____-num
______+FUNCTION
_______-Factorial
_______+ARGUMENT
________+MINUS
_________-num
__________-1
_+RETURN
And who knows what other contextual commands one could use to operate on the tree to invoke whatever view or perspective the programmer would want on the representation.
I wrote this up in an email to Jonathan, but I thought I’d post it here, for anyone who wants to see.
It’s a different representation of what’s above.
Imagine we turn some knobs in the context menu (like “collaps types”, “collapse declarations”, “stack expressions”, and “indent”) to make it look like a prefix scripting language.
(It was with backslashes, but wordpress doesn’t like those, so just pretend the forward slashes are back slashes.)
DEF/Factorial, num
IF/==/0, num
1
ELSE/*/num, Factorial/-/num, 1
1
Here we’d be letting the machine interpret what the types, declarations, and full syntax tree should be. One could type this code out by hand, while the machine in the backround constructs the full tree; and when there is a type error, or any other error in the full syntax tree, the machine can imediately alert the programmer of the error.
I like how there’s no precedence confusion either. All operators operate on two values, explicitly.
But we could click on “unstack expressions” and “show collapsed nodes” and we could see this:
DEF/~/Factorial
~/num
~/IF/==/~/0
~/IF/==/~/num
~/1
~/ELSE/~/*/~/num
~/ELSE/~/*/~/Factorial/~/-/~/num
~/ELSE/~/*/~/Factorial/~/-/~/1
~/1
Then we could expand each node if we wanted to. So how dynamic the language is just depends on the UI.
Then we could click “expland all nodes” and get this:
DEF/NAME/Factorial
DEF/ARG/NAME/INT/num
DEF/IF_ELSE/IF/IS_EQUAL/INT/0
DEF/IF_ELSE/IF/IS_EQUAL/INT/num
DEF/IF_ELSE/IF/IS_EQUAL/RETURN/INT/1
DEF/IF_ELSE/ELSE/RETURN/MULTIPLY/INT/num
DEF/IF_ELSE/ELSE/RETURN/MULTIPLY/INT/FUNC/Factorial/ARG/MINUSINT/num
DEF/IF_ELSE/ELSE/RETURN/MULTIPLY/INT/FUNC/Factorial/ARG/MINUS/INT/1
DEF/RETURN/INT/1
So the most expanded view would be the one in my above post.
The beauty of this representation, is that it shows how similar this looks to a directory tree. One could easily imagine this actually being a directory tree, where one edits and builds a program out of files and folders (properly named) within they’re OS’ native file browser. In fact, it lends itself to component based programming interface.
From what I see, Subtext should be a text based language, that is under the hood a database that links states and so on….
I do not see why
f(x): if(x==0) 1 else (x*f(x-1))
should be wrote in 8 lines !
Of course, the code would be not text ! Text would be (part of) the UI. You still having that superb features, and you have a much more permeable interface.
Programmers know text. For sure there will be “whow” and “ahhh” when they see that this text becomes alife, edit time == run time as you said.
The basic idea is: keep things as they are, but allow some “syntactic sugar”.
I don’t agree that the current GUI sucks; I prefer it to any of the replacements that have been suggested in this blog. As it currently exists, the GUI constitutes a fairly straightforward reflection of the underlying datatype and the allowable operations on that datatype. It may even represent a locally optimal solution. At any rate, I think it is a more useful prototype than the suggested alternatives.
In fairness, I have only given Subtext about 30 minutes of my life so far, and I’m sure my impressions come from having spent the last 25 years of my life doing it “the other way”. I think the cure in this case is worse than the disease. But this is a subject I have thought about on occasion, and I fundamentally agree with the core of your philosophical position; creating software by writing cryptic things in text editors (even ones that make pretty colors) seems archaic.
It seems to me that there’s an atomic truth to programming; everything software does breaks down to actions and decisions. For some reason I have yet to comprehend, every time a new language is created, the creator feels the need to re-invent the semantics of these basic operations.
I think we can simplify software development by creating better IDE’s. Imagine an infinite 2D space where objects can exist as geometric shapes (circles, rectangles…whatever you like.) Inheritance is done by simply copying and modifying an object. You can “rubberstamp” an object to instantiate it. Magnifying an object sets the “scope” to that object and the infinite 2D space now shows only items within the scope of that object. Flow is directed by creating object method references and connecting them with lines.
Ultimately, though, when you devolve the interface to atomic actions and decisions, they are implemented with our well-established and familiar methodologies.
This is one rough idea, not incredibly well thought out, but conceptually you can see how it gets us away from editing text files. Ultimately, we should deal with a project as a single file. We can include other external libraries by dragging them onto our space. It is insane to me that in 2006 we are still setting include and lib paths.
In summary, I applaud subtext for its outside-the-box approach…but we might need to put at least a toe or 2 back inside the box. For now at least.
-John
This reminds me of Interlisp-D SEdit
http://www.mahalito.net/mirrors/LispMachineVideos/InterlispSEdit.mov
Here’s a video of it in action.
I know this is an old topic, but you might wanna look at Dassault’s Virtools solution. When we started this tool, we had the same goals as you, but without being as formal maybe. In other words we didn’t really know exactly where we were heading to, but in the end the visual scripting with building blocks worked quite well. That is the Schematic View in Virtools – it provides a bit what you’re doing now with the tables, but without the table structure. And I realize now that without that structure it proved to be pretty hard sometimes to see what was going on. I don’t know if the issue was adressed better in the new version (3dvia MP). Anyway take a look: http://www.gamedev.net/features/reviews/productreview.asp?categoryid=35&productid=518