What’s next?

Since my last paper was rejected I have been thinking about what to do next. To tell the truth, by the time I had finished that paper I had lost interest in the topic. It seems that for the last few years I have been lurching around in different directions. Here are some of my thoughts about a new strategy. Please let me know what you think.

Semantics instead of syntax

I have been focusing on non-textual programming. The name Subtext itself headlines that issue. I have demonstrated small-scale examples of how a non-textual language can make programming fundamentally easier. This just isn’t selling, for a number of reasons.

First of all, I get tarred with the past failures of visual programming. Small-scale examples are not convincing. More fundamentally, people just don’t want to admit that they aren’t smart enough. They are willing to admit that non-textual programming might help novices, but they don’t need it. And no one wants to write off their years of training in textual programming.

I need to change focus from syntax to semantics. I have to show how a new kind of programming language can offer higher-level concepts that make building certain kinds of systems dramatically simpler. Non-textuality might be acceptable as a requirement of those semantic benefits. Non-textuality can only win by hiding in a Trojan Horse.

One starting point is the model of hypothetical functional state that I demo’d at OOPSLA05. To make the idea more approachable, I should use a normal textual language if possible, even if only as a pedagogical aid. I may have to drop the name Subtext.

Top-down instead of bottom-up

The proper scientific method is to study special situations in which a new idea can be evaluated in isolation and detail. I tried that in my last paper, and it didn’t go over at all. Radical new ideas must first be strongly justified in order to be taken seriously. I need to take on a really big practical problem using a mix of new ideas. If the result demonstrates clear benefits to practicing programmers, that might provide sufficient motivation for the scientific process of isolating, evaluating, and refining the ideas. I strongly believe that such a communal refinement process is crucial, and that its lack is why so many of our languages, tools, and standards are half-baked.

The obvious domain to take on is web application frameworks. Basically what Macneil suggested some time ago: Subtext on Rails. Such web apps are really just the latest incarnation of the archetypal UI-to-a-DB application. I have repeatedly tried working on this domain, and repeatedly backed off because it is just so big, and so complicated by half-baked standards. But I have been making progress in little bits, and maybe have now gotten within striking distance.

Design instead of implementation

Subtext, or whatever it is called, will not have competitive performance any time soon. I need to position it as a modeling/specification/prototyping language. The big win comes from allowing a web app to be fully specified in a high-level declarative fashion without imperative semantics: no methods or events; and WYSIWYG live execution. A fully working prototype would be of great value even if it can’t scale into production. It could be used as as FIT-like specification tool, as an oracle for testing a production implementation, and as the source for transformations into production implementations.

Presentation; prototype; paper

My work flow has been to write a paper about an idea, prototype an implementation of it, and then present it. It might be easier to do it in the reverse order.

Comments?

This entry was posted in General. Bookmark the permalink. Both comments and trackbacks are currently closed.

28 Comments

  1. Madoc
    Posted August 21, 2008 at 11:51 am | Permalink

    It is fascinating how your views seem to be compatible with mine. I am also thinking of a non-textual approach to programming. When you drop the constraint that the algorithm must be formulated in terms of symbols arranged in lines of text, this opens up a whole range of new possibilites. However, you then have to drop many of the common “wisdom” and good practices of contemporary textual programming as well. I think that many of our “code conventions” and “good practices” would become irrelevant when the source code is not a one-dimensional text file any more.

    When I am programming, then there are times when I just have to take a break and ask myself questions like: “How does it come that this idea seems so simple in my mind, yet I had to write some long and complicated-looking code to formulate it as a program?”

    Many times, the reason is that the computer does not “know” everything, has no intuition whatsoever and needs to be told everything in great detail. But I find that in many other cases, the computer could very well be “smarter”. One just has to look at some common coding conventions to see that many of them have to do with formatting the source, choosing certain prefixes for member variables and so on. I have come to think that is the computer’s job.

    Well, and there is much more to non-textual programming. However, I find it extremely hard to even make a realistic visualization of what I have in mind. It would be good if I at least would have some static images of non-textual code. Then I could show it to people and say: “Look, that’s what I mean.”

    I want to be able to generate such examples fast and easily (I don’t have much time to spend on that), and they should be expressive and impressive. I have tried PowerPoint (Keynote, actually), SVG (through Inkscape), HTML, Java Swing and maybe something else, but nothing was even remotely feasible. Currently, I am working with XUL, and I find it really pleasing.

    Maybe I am putting too much focus on making some visual examples fist, but I do not think so. For having a vision that can inspire oneself and others, one needs to have some mock-up prototype in advance. Also, there are so many decisions to make on non-textual programming that I just feel I have to see in advance what the result would look and feel like. After all, non-visual programming MUST be visually and intuitively appealing if it shall become successful. People will only love what looks and feels nice, and gives them fast results. And if there is a new approach to programming, it is required that people love it, otherwise no one will use it. When I read what people write about Ruby and Python, I read a lot of emotion and affection in that. I think this is a new aspect to the programming culture. Programmers always used to have some feelings towards programming and their code, but only today does this get valued and discussed openly. For some programming language design decisions, cognitive adequancy and emotion have become an argument. Any new non-textual way of programming must take that into account.

    That’s why I sometimes sit at home and try to make a mock-up, a visualization, of what future programming could look like.

  2. Nat
    Posted August 21, 2008 at 12:34 pm | Permalink

    Sorry, but that doesn’t grab me. It doesn’t sound particularly exciting and will probably just get lost among the myriad web app frameworks out there.

    You’re also going to be playing constant catch-up with web technologies. If your framework doesn’t do the latest AJAX stuff, or Flash or Silverlight or whatever is the cool thing by then, programmers will drop down to the underlying technologies and only see Subtext getting in the way unless it delivers something significantly more productive.

    I think web apps are too “of the now” for Subtext. How about leaping forward and attacking what is currently a tough area for programmers and will be a big deal in the future, such as massively parallel programming on GPUs or distributed stream processing?

    [Good points Nat. Especially since Rails, it seems that everyone is trying to become famous by building a new cool web framework. But that also shows the domain is still in ferment, not locked down with a dominant standard. I do wish I could think of something less obvious and competitive though. The whole multi-core parallel programming frenzy is not too appealing, as it focuses on performance at the expense of programmability - a mistake in my opinion. - Jonathan]

  3. Posted August 21, 2008 at 1:11 pm | Permalink

    A convincing implementation is always stronger than a sound theory. I think it’s the right way to go.

    Also, web developements seems like a great ground to gather actual usage. The web has a strong UI dependency and millions of novice-to-intermediate developers who just want their site to look good enough. They will embrace any tool that makes it easier for them, and if they can avoid all that cryptic code – even better. Of course, it must be at least as functional as rails in the hands of a novice.
    If it becomes popular, you won’t have to try to convince anyone.

  4. Nat
    Posted August 21, 2008 at 1:28 pm | Permalink

    “The whole multi-core parallel programming frenzy is not too appealing, as it focuses on performance at the expense of programmability – a mistake in my opinion.”

    Sounds like a perfect opportunity to demonstrate the power of the Subtext approach!

    Array processing has long been the domain of innovative programming approaches and strange, niche languages like APL, J and K, so you’ve already got an audience who can cope with different viewpoints of what programming is fundamentally about.

  5. John "Z-Bo" Zabroski
    Posted August 21, 2008 at 1:35 pm | Permalink

    Framework design is all about peer-to-peer communication. Circuits need to talk to each other, but not be too chatty. They also need to be independently programmable, and that means they can’t depend on an external context. The reason Rails isn’t perfect is that its model of p2p is wrong, because all the dynamism to it is faked.

    I consider myself an expert in the area of web frameworks and GUI architecture, and what I see popular out in the market doesn’t impress me. The best ideas are in old academic journals. Those won’t be read by David Heinemeier-Hanson or some product manager at Microsoft building ASP.NET MVC. On occasion I read a blog post where someone seems close to getting the right picture, but they need to go two levels more meta into the problem domain’s underlying nature to see what they are missing. The problem is, from their stand point, they are addressing the problem at the most meta-level possible. See for instance Martin Fowler’s Patterns of Enterprise Application Architecture, and his blog posts about GUI architectural patterns.

    @More fundamentally, people just don’t want to admit that they aren’t smart enough. They are willing to admit that non-textual programming might help novices, but they don’t need it. And no one wants to write off their years of training in textual programming.

    People do get upset when they realize their way of doing things is stupid, because from their perspective they are working with the state-of-the-art. When you demonstrate you’ve got something way better, they basically feel empty inside, because now they are maintaining “what everybody knows how to do” and it becomes clear to them that they are not an innovator but a code monkey who churns out text blocks to be compiled.

    @To tell the truth, by the time I had finished that paper I had lost interest in the topic. It seems that for the last few years I have been lurching around in different directions.

    You should read PJ Brown’s essay on bringing a research idea into a consumer product: ” Building Novel Software: The Researcher and the Marketplace.”

    Also, get in a rut, now. Start pounding away at some problem, mercilessly. Accumulate lessons you’ve learned, post them here to share with others. A manifesto is cool, but a list of idioms suggests you have wisdom about problems that most people can’t collect into a book.

  6. Justin Bailey
    Posted August 21, 2008 at 1:41 pm | Permalink

    Jonathan,

    A very interesting post. Don’t be too discouraged. I really enjoyed the paper you wrote “No Ifs, Ands or Buts” – a great demonstration of what Subtext could do. Maybe you are just speaking to the wrong audience? Have you considered presenting these ideas to functional programmers? Much of what you discussed would play well to an audience sold on pure, functional, and maybe even lazy, languages.

    Semantics instead of syntax – Subtext is a cool idea. The “visual programming” thing may turn people off. I know when I read the paper above I wondered “but does it scale?” – i.e. would the UI become unweildy when writing a large program? How could the ideas be integrated into existing languages?

    Top-down instead of bottom-up – LIke others have said, web apps are the “right-now” thing. Be bold – look for something thats impossible and show its easy. YAWF would just be a big yawn. Also, if you are making a new language, you lack the library support that really drives adoption.

    Design instead of implementation – Maybe consider code-generation. Target a platform with large libraries and support. make sure your interop story is solid and you’ll have a lot more mileage.

    Presentation; prototype; paper – I guess this depends on your goal. I lvoe to see the ideas but it sucks when you can’t play with them. Yet, when the code is written, many times the ideas are obscured. Again, maybe your audience is the problem. The Haskell community has a great tradition of high-quality papers that turned into great libraries. Contrary to the previous poster, give me an implementation built on solid theory any day. A prototype with principles would just be too easy for you!

  7. John
    Posted August 21, 2008 at 2:22 pm | Permalink

    “The whole multi-core parallel programming frenzy is not too appealing, as it focuses on performance at the expense of programmability – a mistake in my opinion.”

    But, could the Subtext approach make this type of programming relatively easy and accurate? I’m not asking this question pedantically, btw.

    Rather than array programming, I really think you should try to solve the general problem of concurrent programming for ordinary application programmers–which is presently a total cluster of misery. If you do, Subtext’s only real competitor in the domain will be Rich Hickey’s Clojure language (which will, at the very least, make for interesting company; what Rich has done with the implementation might possibly give you some ideas about dealing with performance issues, as well).

  8. Posted August 21, 2008 at 2:39 pm | Permalink

    I like subtext.

    1. Have you considered a system which generates code in text-based languages so we can gain (some of) the benefits of subtext at design-time and then use it in our existing projects without having to drop our entire workflow and runtime? Several times since seeing your demos I’ve been working on a Python project and have complained that I couldn’t just use subtext for a problem. If I could generate Python code from a “Subtext IDE”, then I’d be able to use some part of Subtext. (actually, code generation is generally bad. Perhaps an embeddable runtime would be better).

    2. How will subtext fit into our revision control systems? How will automatic merging be done? How will I review the changes that someone else makes (like I view diffs now)?

    If you want to make something people are even remotely likely to use, these issues need to be dealt with.

  9. william pearson
    Posted August 21, 2008 at 6:07 pm | Permalink

    I have been following subtext with great interest for a while.

    Just a thought – something Bill Gates thinks will be absolutely huge is “personal robotics” http://www.sciam.com/article.cfm?id=a-robot-in-every-home. Lego Nxt has a visual language already, but I think Subtext would be very different and very superior.

  10. Posted August 21, 2008 at 9:17 pm | Permalink

    Hi Jonathan,

    I think you’re approach of design/present, prototype, and then document is a good one. I’ve been doing some reading on software testing recently, and the axiom of only building what you need is a good one. This also implies that you need a real world problem to base the concepts on.

    I’ve been thinking about the idea of building a new browser for quite a while now. As you’ve said, it’s a big task. However, I also think that to get big shifts in thinking you need to show people the way with actual product that you can discuss.

    As you’ve said, Web Frameworks are a hot area of development. The danger of them is that you need to be tied down to HTML/CSS/XML and various other technologies. These technologies put a lot of restraints around what can be achieved and requires that non-textual language ties back to a textual language. To me it just sounds dangerous.

    The direction I prefer is throwing most of it away; cherry picking the concepts that can work. Replace HTML/CSS/Javascript with a single non-textual language. Build that language on a modern VM such as JVM or CLR; this can provide the speed and solid basis for the system. Build on the structured rendering engine from a browser like Mozilla.

    I’ve been looking at building a binary object REst Distributed (BORED) protocol which could provide the communications layer for such a system. I’ve already built Argot as the data encoding system.

    I would be interested in some collaborative efforts to reduce the workload. I’d also be interested in getting some insights from people like John Zabroski that responded on your blog. Inspiration can also be gained from the Mozilla Labs conept series (eg. Adaptive Path’s Aurora).

    Regards,
    David.

  11. Cristian Lotito
    Posted August 21, 2008 at 11:39 pm | Permalink

    Hi,

    For me the value of subtext is the ability to work with very complex logic or algorithms. I think what Subtext will be able to do is to allow programmers to handle more complexity with ease. With the power of Subtext as a base we should be able to see a new breed of applications that handle complexity that is not even attempted at the moment.

    I like the idea of using Subtext to handle complicated problems and then embedding the solution into another language. For example SubText could be part of the .NET CLR. Just like now you can use Fortran in .NET you could use SubText.

    What I think you can do is grab a classic computer science problem that requires a complicated algorithm. For example you could grab some compression algorithms, or the traveling salesmen problem. Then write the solution using Subtext. At the end compare the difference between the SubText code vs the traditional code. By looking at both the value of Subtext will become apparent. You might even find that you can improve on an algorithm because it is easier to manipulate it in SubText.

    Another idea might be pattern recognition algorithms. These are popular in Web Search engines and Business Intelligence.

    Thanks for all your work on SubText,

    Cristian.

  12. Posted August 22, 2008 at 2:59 am | Permalink

    The following question popped into my head as an interesting area of practical exploration: What would a constrained-by-touch interface, programming language look like?
    When I think of visual programming languages moving mainstream I think of touch screens, multi-touch, table-top, wiimote, the wall, the globe that Microsoft showed off recently and other constrained interfaces of interaction. Is that something that would interest you? Those kinds of constrained environments where text is awkward to input that require another approach… the Subtext approach? :)

  13. Posted August 23, 2008 at 12:36 pm | Permalink

    I would suggest javascript, is a nice language, pretty functional too. It may gain support from the web developing community. It can be run both client side and server side, etc.
    On the client site (browser) can be easily prototyped and it would be simple to add visual programming elements (widgets, etc.)

  14. Posted August 23, 2008 at 2:07 pm | Permalink

    Have you ever thought about writing a structured editor for a general-purpose programming language (say C#)? I’ve worked in that field for 3 years and I barely scratched the surface of what can be done there. People have learned how to write compilers, but there is no science about how to write editors, IDE and tooling. Properly architecting an IDE and sharing data-structures between compiler and the editor is an ad-hoc area without solid research behind it – there’s no Dragon Book for building IDEs. I like the fact that subtext is a MVC, why don’t you turn a GPPL editor into a view on the program model?

    http://www.guilabs.net/
    http://kirillosenkov.blogspot.com/2007/08/structured-editors.html
    http://delicious.com/KirillOsenkov/StructuredEditors

    P.S. Since Silverlight 2, I don’t care about the web anymore. The web is now just a desktop with a bunch of webservices.

  15. John "Z-Bo" Zabroski
    Posted August 23, 2008 at 5:31 pm | Permalink

    @Chris Armstrong
    @2. How will subtext fit into our revision control systems?

    The short answer is that it shouldn’t. I also am very careful about using the word “revision” as opposed to “version”. Revision literally means to re-see, whereas version means to turn. Very little has been written about rethinking version control systems in order to make them true revision control systems. In part, this is because the bit level is the wrong level of abstraction to track high-level changes in design direction. See Linus Torvalds’ funny rant about Subversion for an example of someone who designed explicitly a version control system (git), and did it well. See Greg Wilson’s 2004 ACM Queue article on improving development tools for a totally different problem. http://www.acmqueue.com/modules.php?name=Content&pa=showpage&pid=247&page=1

    Also, the attempt to shoehorn in the concept of “revision” into the meaning of “version” is precisely what has led us to researchers proposing feature-oriented programming on top of object-oriented programming. The argument goes like this: “We’re really bad at using tools for their intended purpose, because useful tools end up being applied everywhere, including where they shouldn’t, so lets continue to shoehorn, since that’s what we’ve always done”. Once researchers have stressed the boundaries the tool was intended to live within, they insist on a paradigm shift. See for instance mixing together feature-oriented programming and aspect-oriented programming. http://www.cs.lth.se/home/Lars_Bendix/Publications/BE08a/cvsm-final.pdf To a large extent, language and tool research has become “How to program between the lines”, but no one knows how to do best do this, so they just propose hacks around premature abstractions, such as “features” and “aspects” and “subjects”, or they propose abstractions without encapsulation (e.g. Gregor Kizcales’ “Beyond the Black Box: Open Implementations”)!

    I do think there is something that might be easy to add to subtext, which would certainly aid in revision control: rationale. The logic flaws pointed out by subtext 2.0 already implicitly identify traces of rationale in programs. In my experience, keeping the functional spec. up to date with the implementation is far more important than having the ability to go through a complete history of program changes stored in version control, for me to “turn” through. This goes with along with the generally understood idea that functional specs are a good thing, because they force you to design the program. Model-based languages like subtext are good, because they force you to uncover flaws in the spec you can’t see by looking at text (you need “sub-text”).

  16. John "Z-Bo" Zabroski
    Posted August 23, 2008 at 5:48 pm | Permalink

    @Jonathan
    @The whole multi-core parallel programming frenzy is not too appealing, as it focuses on performance at the expense of programmability – a mistake in my opinion. – Jonathan

    Could you elaborate on this some more? For instance, “Iterators: Signs of weakness in object-oriented languages” by Baker basically argues the foreach construct is a code smell. It does in fact prematurely encourage thinking about programs in terms of many iterative steps, rather than one step (a comprehension). With nested foreachs, it seems the compiler writer is not given a fertile dependence-based approach to optimization. The compiler writer has to figure out what can be executed independently. Furthermore, the program implementor can’t easily visualize the transformation they’ve just written. This strikes me as a minus for compiler writers and program implementors. And, within the .NET community, there is a push to obsolete it with monad comprehensions and lambda expressions (PLINQ).

  17. Yair
    Posted August 25, 2008 at 7:53 pm | Permalink

    I disagree with Cristian. You shouldn’t program a sophisticated algorithm. Do that and someone will reply with a sick 10 character equivalent solution in K or something.

    Instead if you want to use an example, I suggest to use an ordinary boring real-life problem, and a real one if possible. The gain should be that with subtext it would be less of a pain than with alternatives.

  18. Kazagistar
    Posted August 26, 2008 at 12:37 am | Permalink

    I really do like the idea of implementing subtext as an IDE overlay to exiting programming languages. One possible way to get a relatively large audience for your project might be to create a sort of visual shell-scripter for Linux, like Automator is to Mac OSX except far more powerful. There have been numerous calls for such a program by many Linux sub-communities such as Ubuntu, and so a Free, relatively bug free and innovative system such as this one would likely be quickly adopted and would be likely to reach an audience of thousands of people (and a higher percentage of Linux users know how to program then on Mac or Windows, by far).

  19. Posted September 1, 2008 at 4:56 am | Permalink

    Interesting video on subtext. Graphical decision-making logic tree analyis for business decision applications hold some potential.

    If the objective is to create a new visual method of programming to take hold amongst independently thinking, overtly sensitive, programmers via “it’s a superior method” concept, to er, eventually make money, then you may wish to consult business professionals to gain a foothold on how to sell/understand professional programmers, how they think and how they influence buying decisions amongst those that purchase programming software.

    Developing a better mousetrap prototype has never resulted in sales to anyone and it has also not resulted in adoption of any new concept by the mainstream users. However, marketing inferior but readily apparent utility (to the actual buyer) of products is unprecedented in its degree of success. Leave product development concepts to professionals.

    This appears to be a highly marketable concept outside the realm of programmers, who would only kick and scream and argue, rather than adopt a clearly superior system for developing/manipulating logic trees of all types. I have seen thousands of these type products disappear due to the “builders monkeying around outside there expertise with business decisions”

    Anyway, good luck, I would buy/use a graphical decision making logic tree application based on your easy to follow concepts and GUI immediatly if geared towards multivariable/intraactive excel based inputs across spreadsheets at a price lower than the million bucks SAS charges. Consult a business profesional and steer your hard work towards a more receptive customer.

  20. John Nilsson
    Posted September 21, 2008 at 10:11 am | Permalink

    As the previous poster suggested. There is much to do in the area of back office programming.

    I am developing the back end of a mail order system (together with all the internal front ends needed by the workers of such a company). And one of the most annoying part of my day to day work is the heaps of boiler plate, redendancy and maintainance burden resulting from how the bussines logic is mapped onto the DBMS.

    Maybe a fruitfull focus for your future work is to implement a programming model for better integrating data persistance and busines logic.

    On my spare time I’ve started to play around with implementing a language integrated version C.J. Dates Tutorial D. Dates main argument is that SQL is a total failure of how a DB languge should be implemented and argues that a more faithfil implementation of the Relational Model is required as a foundation for future development in this area. As an example of such an implementation he has developed Tutorial D, but I guess that with your focus you might come up with some interesting and new ways to implement a true and usable Relational Model.

  21. Tom Van Cutsem
    Posted September 22, 2008 at 9:09 am | Permalink

    Jonathan,

    First of all, don’t be too disappointed that Subtext is not appraised enthousiastically by everyone. I am in a similar situation and my research focuses on traditional, textual languages. In my research domain (object-oriented programming), I can’t help but notice that almost all research presented at top conferences such as ECOOP and OOPSLA are extremely small incremental changes to existing things. This is a problem for all of us who cannot formulate our research in this way (because we’re not incrementally extending mainstream concepts).

    However, your key achievement with Subtext is that you’ve made a lot of people (including me) realize that there exist radically different ways to program a computer, and that a lot of the ‘visual processing power’ in our brains is left unexploited by today’s tools and languages.

    That being said, here’s some feedback about possible problem domains to tackle with Subtext:
    - Maybe a good community to target is one which is already using visual languages, and to show how Subtext improves upon existing such languages. For example, System Description Language (SDL) is very imperative in nature, and I can imagine most other visual languages are as well. The declarativeness of Subtext may be a good asset here.

    - Another possibility: one of the more well-known success stories of ‘dynamic’ languages is that of Ericsson’s use of the Erlang language to program complex IT hardware such as routers and telephone switches. Such a system may really be a good use case for Subtext, because I can imagine that a telephone switch must react to a lot of events differently depending on the state it is in. Hence, there’s a lot of decision-making (which is not challenging per-se, but it becomes challenging if you consider the possibility of failing components and the non-determinism, i.e. the fact that events may arrive at any point in time)

  22. Vinod
    Posted October 4, 2008 at 3:31 am | Permalink

    First off, keep the Subtext name. In fact copyright it, if possible :).

    Second: I’m currently struggling to understand a large database quickly and without actually reading every line of code. I’m using code comprehension tools such as structure 101 and Lattix, and most of them eventually settle on the reality that code (static or dynamic) is actually a graph, and applying graph algorithms to process, analyze and display code makes the most sense. In fact they claim they’re fast and capable of handling large codebases *because* they implement advance graph algorithms.So I cannot tell you how much on the mark you are with Subtext. In fact, code comprehension could be one dimension of use for Subtext:

    If you could convert from text (ie, existing code) to subtext, and show the gaps and overlaps (like you already do), that along with focusing together are worth its weight in gold in dealing with legacy code. Thats your in – maybe not with OOPSLA, but with the real world. Savvy business folks and testers should love just looking gaps and overlaps, even more so fixing them with assertions, not raising a bug.

    Developers should love the focusing feature – it blew my mind when I saw that because we’ve been struggling with this “if(foo) Problem” where foo is a flag that represents a major business execution path accross the application, eg, anonymous user vs signed in vs, preferred, or bid for book vs buy it now. Code is “reused” by sticking an if after the common parts to separate out the path-specific ones (very similar to your Attack and Melee example), and the codebase ends up having tons of tiny snippets of path-specific logic duct-taped together using these flags. With focusing we could just “see” the different paths. Even more telling is that the obvious solution to this problem is to refactor out the common code into an ancestor, and put the path-specific code into sub classes – which you’ve shown to be equivalent to the schematic table representation.

    One last thought on the “Subtext as a code comprehension tool” topic: Dealing with real codebases requires scaling up past the method/class level. I dont think that should present a particularly difficult problem for the subtext concept, however, as beyond the method, the graph relationships are much simpler (if larger in number), and even then their complexity is derived due to the complexity of the methods themselves. So it should be able to scale. This btw, is something existing code comprehension tools dont do very well yet, I also imagine that the actual problem of visualizing large codebases in the schematic table view could be helped by the advances that the visualization field have made.

    Third: could you not piggyback on the vm/clr, and consume the types available from the runtime to just “inherit” all the messy bits – already in cooked form similar to JRuby, Jython or Scala? Sorry if there’s a fundamental reason this cannot be done, but it seems to me that Subtext should be able to abstract out some “platform” methods out as unexpandable, and therefore treat it as a black box – much similar to how code comprehension has exclusion lists. This would free you from either writing or having to comprehend all those base classes that you need to use the web framework, and you can work on the true programming logic of the application itself.

    Finally, are you planning on making a copy of the software itself available? Or if/when you expect to open source this? This is probably reckless, but I’d sign up to write the text to subtext converter!

    PS: I trust subtext programs can be persisted accross multiple sessions of the Subtext enviornment. How are they stored? Would that be text ;)?

    ————————-
    Thanks for your comments. I like your practical perspective.

    I face a difficult choice with Subtext: should I take the ideas and apply them to conventional languages, or try to develop the ideas “purely” in an entirely new language? A lot of the stuff you see in the Schematic Tables demo is based on restricting what you can test in a conditional to a sub-language with some special logical properties. I also assume no side-effects. I haven’t actually tried to handle a standard programming language – there would be many difficulties – but it is often the case that with a lot of work you can get surprisingly good results. My preference is to instead push forward and make Subtext a complete language that demonstrates how far you can go in the best case. There are still major unsolved problems in the language that I need to figure out, if only for my own peace of mind. At that point I hope that people like you will tell me whether to productize the language or retrofit the ideas into existing languages, or better yet go and do it themselves!

    Subtext currently serializes to a binary file. I haven’t considered export formats yet, and I actually hope to avoid them by allowing separate Subtext macines to directly communicate like a distributed database. I don’t plan to release the current code, as it is too fragmentary.

    – Jonathan

  23. Raoul Duke
    Posted October 10, 2008 at 4:36 pm | Permalink

    there are too many comments for me to be caught up just yet, so this might be a repetition of what has been said, but:

    stop pontificating and just open source it all and use it to build itself or other neat things. don’t keep trying to *tell* people about it. i know that isn’t fair, i know that you have been showing as well, but by *show* i would mean more than (freaking cool!) videos, i would mean being able to say “subtext is written in subtext. and it is open source. and Person X who reads my blog is working on a nifty new tool with it. and Person Y is writing game AI with it.” don’t let all this work die on the vine.

  24. Raoul Duke
    Posted October 10, 2008 at 4:39 pm | Permalink

    P.S.: if you open source it, your dilemma of “should i adapt it for stinky-poo Java, or should i boldly continue to forge ahead to show what the future *could* be?” could be converted into, “hey, there are more people involved having fun and trading ideas and heck somebody is hacking up something into Eclipse so they can see how foobar’d their Groovy game script logic are, and at the same time i’m getting to push the group ever further forward.” be an optimist. :-)

  25. Raoul Duke
    Posted October 10, 2008 at 4:41 pm | Permalink

    apologies. but, to keep kicking the horse… you say “I don’t plan to release the current code, as it is too fragmentary” to which i have to yell “b.s!” and suggest that you are maybe more worried about people dogging your code or something like that which is really holding you back, more than you really care about advancing the cause. get it out! get it open! let it grow. be selfless.

    [easy for me to say.]


    Thanks for your comments. I may open source the core logic engine from Subtext 2, as there is another research project here that has expressed some interest in using it. I do not want to publish any software that I feel is not in a usable or even understandable state. – Jonathan

  26. Gary Helmstetter
    Posted November 15, 2008 at 10:35 am | Permalink

    Since my last paper was rejected I have been thinking about what to do next. To tell the truth, by the time I had finished that paper I had lost interest in the topic. It seems that for the last few years I have been lurching around in different directions. Here are some of my thoughts about a new strategy. Please let me know what you think.

    I think any effort outside the traditional language design box is of value, and potentially revolutionary. If you run low on motivation to continue the struggle against complexity, go review the Java class taxonomy. I recently decided it was time to really learn Java, so I started reading about nested, inner, member, etc. classes. Geez. I figured upon abandoning c++ I’d escaped this swamp.

    All language designers should be forced to represent and serialize a tree in something ancient like “Ten Statement Fortran” to get it through their thick heads that simple tools can handle arbitrary complexity.

    I love the following sentence fromJoseph Darcy’s blog about Java Class Taxonomy:

    “While it would be technically possible to add a getNestingKind method to Class, that would create an undesired dependency of a java.lang.* class on a javax.* package.”

    IMHO that admits a significant flaw in the package design or the class taxonomy, maybe both. But more important, it leaks Java’s dirty little secret: it didn’t really cure fragile superclass disease, it just treated some symptoms.

    I believe copy-and-paste with persistent history, aka transformative programming, does cure it.

  27. Gary Helmstetter
    Posted November 25, 2008 at 6:37 pm | Permalink

    Update: Java’s class taxonomy is a day in the park. Generics, now that is motivation!

  28. Jason Livesay
    Posted November 26, 2008 at 8:14 pm | Permalink

    Your reviews are available on this web site. I don’t think that papers are nearly as useful as functioning demos or videos and the fact that you cannot submit videos or demos to these organizations is an indication that they outdated and not really relevant. The fact that you went to the trouble to write a paper and it was rejected on technicalities or (for being too original?) is another indication.

    You just need to figure out how to get people to use your software.

    I think the secrets to that are largely a) instant gratification and b) engaging social networks

    My suggestion is that the next demo use ActionScript 3 (Flash) (95% browser penetration) and if possible allow real-time collaboration. That will give you a & b together.

2 Trackbacks

  • [...] number of comments on my last post suggested I work on multi-core parallelism. There are a number of reasons why I am steering clear [...]

  • By Dark Views on March 2, 2009 at 11:07 am

    Subtext: Visual Programming, New Angle…

    If you have no idea what subtext is, lean back and watch this presentation.
    It’s nice to finally find another person concerned about the state of programming languages. I started with C, toyed a bit with some other languages, moved to Java and to…