Per Vognsen suggests a Haskell version of the damage function from my Schematic Tables talk:
data Attack = Magic | Melee hit surprise defense attack = damage where effectiveness = power * (if surprise then 3 else 2) (power, damage) = case attack of Magic -> (5, max 0 (effectiveness - defense)) Melee -> (4, 2 * effectiveness / defense)
This passes pairs of values (power, damage) through conditionals. That allows the conditionals to multiplex more than one value, rather than being repeated. I suspect I could stymie that approach with more complex data flows, but I am happy to acknowledge that Haskell does OK on this example. Thanks, Per.
Ouch, the indentation looks funky. It must have suffered as I copied it from Emacs to GMail. [My fault – fixed. J]
As I hinted in my email, it’s really tupling plus lazy evaluation that does the trick here. Were you tempted to express the same code structure in Standard ML, you would be forced into ugly explicit thunking.
If you diagram the data flow of value interdependencies in the Haskell code, you’ll find it looks identical to your diagram for the control flow between functions in your polymorphic OO example in the talk. In particular, it goes alternately downwards and upwards in the linear program text.
-Per
I think there are lots of interesting parallels between Haskell and both Subtext 1 and Subtext 2.
I think Jonathan has been single-handedly reinventing a lot of the Haskell ideas, which on the one hand, is very impressive. But on the other hand, it’s a bit of a shame that he’s spending his energy rediscovering the Haskell concepts, rather than building upon those concepts to get to new undiscovered lands.
You may be right. Thanks, J.
Eh, this is always going to be true, and you can’t worry about it too much.
Lately I’ve been noticing Conal Elliott appears to be reinventing some declarative reactive programming that, as far as I am aware, was pioneered by Udo Montanari in graph grammars and networks of (concurrent) constraints. FRP started off as a way to do deterministic animation, but things like modeling distributed and federated systems as event sources has led him to develop abstractions like amb (ambiguous choice) and unamb (unambiguous choice) to handle commited-choice nondeterminism. It is only a matter of time before Conal addresses how to handle backtracking non-deterministic processes, probably using delimited continuations or continuations with markers, and it remains to be seen how that will jive with garbage collection yet again. So each time Conal adds some new feature to FRP, making it less FRP, “killing FRP” (as one person on the pdxfunc mailing list joked), he has to re-address space and time complexity issues and how all this “clean” denotational semantics actually works in the real world, with a garbage collector and other bounded resource concerns. It’s these sort of engineering concerns, as well as notational issues, that represent barriers.
So if Jonathan is developing different notations for the same ultimate semantics, then so be it – it’s evolutionary progress.
John Gossman made a similar excellent point on the WPF Disciples mailing list not long ago, about the history of inventions and how things go through weird revisions before arriving at a rock solid design [1]; separately, I pointed to Henry Petroski’s books on evolutionary design [2] [3] [4] to WPF Disciple Paul Stovall.
[1] The Strangest Man: The Hidden Life of Paul Dirac, Quantum Genius, ISBN-13 978-0571222780
[2] The Evolution of Useful Things: How Everyday Artefacts – from Forks and Pins to Paperclips and Zippers – Came to be as They are, ISBN-13 978-0679740391
[3] The Pencil: A History, ISBN-13 978-0571217632
[4] To Engineer is Human: The Role of Failure in Successful Design, ISBN-13 978-0679734161
For what it’s worth, I haven’t noticed any striking resemblances. Certainly Haskell has many excellent ideas worth careful study. The subsumption of control flow under data flow by means of lazy evaluation, the subtle interplay between values, types and type classes that is mediated by type inference, etc. That is part of what makes Haskell Haskell; I see few immediate affinities with Jonathan’s work on Subtext.
The similarities that I find:
Subtext 1: Laziness, pure functionality, sort-of monadic IO
Subtext 2: Separating conditionals (horizontal) from computations/actions (vertical) — similar to separating patterns/conditionals to the LHS and computations/actions on the RHS of Haskell’s assignment (=)
The various ideas Edwards is working with right now sound like they are steps towards the re-discovery of the various forms of FRP in the Haskell world…
Subtext 1&2 were essays on the UI of programming, seeing what could be done without the limitations of textual syntax. The semantics borrowed a lot from lazy FP. I am now working on the semantics of change, but taking an approach that is in some ways the opposite of FRP.
In what ways is it opposite?
Well, that will be the subject of my next paper. Basically, FRP inverts the common sense notion of change. It forces the changed object to define all the sources from which it changes, and all the conditions under which it changes. Most Earthlings instead think of change as performed by active agents on passive oblivious objects.
I think “Most Earthlings” think using whatever framework they have been accustomed to…
I think defining time-varying values as functions of other time-varying values makes a whole lot of sense.
In a non-textual editor, doing the reverse look-up (What time-varying values are affected by the one I’m looking at?) is also pretty trivial and still lets you follow the “effects” of change pretty directly.
FRP makes perfect sense if you are doing signal processing. I don’t understand how to apply it to a database or a document.
To me it has always seemed obviously wrong-headed to base FRP on continuous time. The proper foundation is discrete ordinal time: sequencing of events relative to an observer. This is what event-based systems have always been doing if only implicitly.
By the way, Jonathan, if you are going to be addressing UI system building, you might want to consider a problem that has vexed me recently:
Composable visualizations are not very difficult; composable interactions are the real problem. It’s easy enough when events can be transparently distributed by a top-level multiplexer to individual interactors (e.g. buttons), but that does not work smoothly in all cases.
The classic example is how buttons deal with mouse clicks: when you click and hold a button, move the cursor away from the button’s screen area, and then release the click, the button’s action isn’t activated. Drag and drop provides another, even trickier example. In both cases the interactor temporarily “takes control” of the mouse cursor and its interactions.
The question is how you model this in a composable way. The usual hack is to have the interactor set a flag on the mouse that says “I currently own this” and have other interactors (and possibly the top-level multiplexer) respect this flag. This feels a bit like a mutex.
Per,
Thanks for the examples.
I agree that drag-n-drop is an ugly hack in the GUI frameworks I have used, using “mouse capture” as you describe. I don’t have a linguistic solution, except a vague intention to make it easy to have localized state machines as in actor languages. I think a cleaner alternative design for mouse capture would be to create a transparent layer on top of the window that intercepts all mouse events.
I have more to say about coordinating cascades of events. For example, when the mouse moves from one control to another, the former receives a MouseLeave event, and the latter gets a MouseEnter. I have seen these events fired in seemingly random order. In my synchronous semantics those events can be guaranteed to happen simultaneously.
Actually, Per’s two examples are fundamentally the same. The difference depends in whether the button externally exposes its capabilities. In other words, if the button has nothing meaningful to drag, then when you hold down the the left button (associated with dragging) and then move the mouse outside the application window and then release the left button, then nothing happens. When there is some capability worth dragging, then interpolated between the first and last events is a visual cue. The reason the button gives no visual cue is because it has no capability it wishes to pass on to the rest of the application, or give to another application.
You are right that some events should fire simultaneously, or at least not mislead developers into thinking they fire simultaneously. Is the guilty platform WPF? If so, using VisualStateManager probably masks the issue.
Tangent: I have not fully covered the wide expanse of Joe Goguen’s work, but before he died he was very interested in algebraic semiotics and he made comparisons between “conceptual blending” of linguistic philosophers and algebraic semiotics. Goguen’s classic example was the meaning of a stop sign, and preserve the meaning of the stop sign when, say, cars interact with it. In Goguen’s model, the stop sign is a *very* complex entity: you literally must describe the full meaning of a stop sign. The frightening thought in my mind is that there may be some problems where such an algebraic semiotics is necessary (but I hope not), since my intuition says it is rather monolithic. On the other hand, there may be some unmined elegance in Goguen’s ideas. Probably, true agents (in the artificially intelligent sense), with independent existence, would benefit from modeling observers in this way. The reason it is monolithic is that the true agent would in effect need to be a human, and we are very complex, monolithic designs.
For a neat way to handle drag and drop with FRP read section 2.4 of the Flapjax paper http://www.cs.brown.edu/~sk/Publications/Papers/Published/mgbcgbk-flapjax/paper.pdf
Running example with code here: http://www.flapjax-lang.org/try/index.html?edit=drag.html
Jules, thanks for pointing that out. But at first glance I don’t think it addresses the problem Per raised, which is to “capture” further mouse events after the drag is initiated, so that no one else sees them.
I have spent a lot of time thinking about the drag mouse capture problem in Bling. I think the trick is not to compose event streams, but to compose state machines instead. With a state machine, its more obvious where to implicitly capture and uncapture the mouse. State machines are also fairly composable.
Perhaps there is a relationship with and opportunity for statecharts here.
Ah, yes I see. What will go wrong if you don’t capture the mouse? Other widgets will get mousemove events, is this a problem in practice?
Yes, the failure to mouse capture is a show stopper that makes these abstractions unusable. For example, the Rx-based Silverlight drag and drop adapter is not very useful because it doesn’t capture the mouse: it leads to a flaky drag and drop experience that most users will observe pretty quickly.
Mouse capture isn’t the only reason you’d want to use state machines for expressing a drag adapter. Many drag adapters have non-trivial requirements, you have to distinguish between click, hold, drag, hold drag, flick…expressing this behavior is fairly difficult if you just have event streams, much easier if you have state machines that are driven by event streams.
Right, Sean. It also makes testing way easier, since modeling a state machine really amounts to mocking an object’s event queue.
I am not sure I followed the discussion entirely, but “state machines that are driven by event streams” are pretty nice to express with conal’s FRP (accumE):
accumE :: a -> Event (a -> a) -> Event a
accumE initStateMachine (fmap advanceStateMachine eventStream)
The point of my argument is not whether state machines can be expressed with event streams, but whether state machines themselves can be composed without decomposing them back into event streams.
This doesn’t solve the problem of not propagating mouse events to other places.
If you aren’t dealing with a pure environment, mouse capture solves the problem of turning off propagation, or more generally, routing all mouse events to one component even if they would otherwise not go there. capture/uncapture are imperative operations, but you can have them called implicitly on arcs in the drag state machine.
Related to but different from mouse capture is the more general problem of how to deal with event bubbling and tunneling. I suspect that state machines may also present a solution here, but I haven’t come up with anything yet.
I am thinking of using multi-method predicate dispatch to replace pub/sub of events.
With the predicate “mouse in my bounding box”, you would dispatch to the lowest level control containing the mouse that defined a method to listen to the mouse. Calling next-method would bubble the event upwards.
One thing I don’t like about any of the approaches in that style is the inversion of control. Combining visualization and interaction in a direct-style program means you don’t have to create a lot of explicit state to communicate between the two parts. It’s still a state machine of course, but it has the usual advantages (and some disadvantages) of direct style over continuation passing style.
A guy I know, Casey Muratori, has been writing his UIs for several years in what he calls an immediate mode style. It deals with a lot of these issues, though it isn’t a panacea; in particular, some things that normal UIs do well are harder to do with IMUI if you want to retain its unique advantages.
Anyway, he made a video a few years ago with an overview of how it works:
http://mollyrocket.com/861
Per, thanks for the link. I hadn’t heard of immediate mode GUI’s. It makes a lot of sense. It has a nice old-school feel with nary a mention of objects. I certainly agree that the state of the art in GUI’s is a toxic brew of declarative and imperative constructs. IMGUI solves this by making it entirely imperative. Conversely, I want to make it entirely declarative. Maybe that should be called “retained mode application logic”
Yeah, Jonathan, me too. And in my case not only declarative but algebraic. The problem with algebraic approaches to the problem we’re discussing is that there seems to be something cross-cutting and hierarchy-subverting at play.
You can restore some order by having the processing function for controls accept the current context (e.g. mouse and key state) and return an action. The event distributor, itself a control, could take these actions that bubble up from its delegates and either act on them (e.g. setting a certain control as hot until further notice, which will affect event distribution) or bubble them further up.
Per,
Bubbling events through the UI is in general a bad idea, and non-modular; it creates a recursive path through the system that makes debugging difficult, especially when event handlers can be dynamically plugged into the system. What is your *actual* intention? Put another way, I would like to see code that proves me wrong.
If you have been following the thread, you should know the intention. The goal is composability of interactions; an example is how to handle apparently non-modular features such as mouse capture without resorting to monolithic imperative kludges.
In my very loosely sketched proposal, you seem to have latched on to the metaphor of bubbling and made some fanciful extrapolations. There is no dynamic plugging of event handlers into the system in anything like the usual sense; that whole premise implies control state fragmentation and inversion of control, which is what I want to avoid.
Speaking of Haskell, Z-Bo linked to an amusing rant.