31 Replies to “Subtext 5 – Teaser Trailer”

  1. Is part of the idea that there are two classes of subtext programmers? I’ll call them “library programmers” and “normal programmers” where library programmers are the ones that generally have to worry about writing the code that tends to execute in the dynamic phase? I suspect that if normal programmers are expected to deal w/ figuring out which code can be put in some places but not others it could get out of hand pretty quickly. The example is where there’s a mutable variable just above the line which makes it look trivial, but I imagine things would have to cascade to where some functions can and can’t be used in certain places, and perhaps even more tricky if you’re writing a function for general consumption (which I suppose by definition makes you a “library programmer”) you might not even get an error when you make the mistake until it’s used in the “wrong” context.

    1. Hi Jake,

      There aren’t places where different kinds of code are allowed. There are no special declarations for dynamic constructs, nor does the programmer write a procedure to perform initialization. It is a global analysis of the entire program that picks out the dynamic constructs wherever they occur.

      Also you can’t write a definition without having it both type-checked and executed with default values at the point of definition.

      That said, there is surely a difference between library writers and users in all languages. I am definitely favoring the users over the writers. Languages like C++ and Scala seem to do the opposite.

      1. Hrm, well you demonstrated in the video that you can’t reference a mutable variable in a metaprogramming context which of course makes perfect sense to someone that reads your blog, but I think I would have trouble explaining it to a spreadsheet programmer.

  2. A small disagreement with the example in the beginning where you say statically typed languages cant provide dynamically derived types.
    Isn’t type providers in F# exactly this feature?

    1. I was also taken aback a bit by this comment but for a different reason. I feel like the solution to the specific problem doesn’t itself require a new language (though there are plenty of other reasons for it), but IDEs that deal w/ generated code better which I think you might agree with since you mentioned that current IDEs don’t work very hard to support it in the beginning of the talk.

      Of course it still doesn’t allow you to generate types at runtime, but that seems like we’re not talking about types anymore to me if you’re truly generating them at runtime of the final program.

  4. I recently made a similar design choice: typechecking as staged programming with a ‘dynamic’ compile-time behavior constructing a ‘static’ run-time behavior. I think it has nice properties. I really love the ability to work with heterogeneous lists, heterogeneous zippers, text-as-types, etc. in the compile-time. It provides powerful metaprogramming and visualization features.

    OTOH, one feature I miss is the ability to guarantee that only well-typed behaviors are constructed at runtime. I can ensure that no ill-typed behaviors are installed, but it’d be even better to avoid constructing them.

    1. While my language has since gone in a different direction, I also had something similar working a few years ago.

      Magpie basically ran your top level code as a dynamic language. At that point, you could imperatively modify types, build classes from DB schemas, or whatever other crazy metaprogramming you wanted to do.

      Then it would look at all of the class and function definitions that remained at the top level and statically type check them.

      If everything was OK, it would invoke main(), which was sort of the boundary line between the dynamic and static worlds.

      1. Hi Bob, somehow I overlooked your comment before. I can’t believe we were just hanging out at StrangeLoop together and didn’t talk about this! You clearly had the idea of distinct dynamic and static phases during execution. What made you change direction? Did it not work out?

    2. David, it sounds like you may be thinking along similar lines, though I have to admit I didn’t really understand the post you linked to. As I think I’ve said before, it would really help if you used some examples to explain your ideas.

  5. Exciting progress! I just re-watched the Subtext 1 video too. So glad to see things coming together.

    You mentioned above that you’re doing a global analysis to identify the dynamic constructs. I was expecting that there’d need to be annotations so I’m curious how it works and how it interacts with modules, but happy to wait while you polish off more pressing things.

  6. Nice! A couple of comments that come to mind quickly:

    * Why not make this a web essay with maybe embedded videos? Webcasts are hard to consume efficiently, and web essays are more likely to go viral.

    * One of the best original ideas in the original SubText was that “names were too valuable to waste on compilers.” The obvious alternative to witness expressions in that context is to have “+” be selected directly by the programmer (if + is overloaded, they select one explicitly).

    1. Hi Sean

      Yes a web essay is a nice compromise between a paper and a talk. I wish we could submit papers in that form. I recall you had some difficulty with that though. I should do that when I get past demos.

      As to “names are too valuable to waste on compilers”, you have no idea how much pain that idea has caused me! It turns out that lexically scoped binding is a really simple and useful mechanism. I struggled so much in previous versions trying to avoid that but now I have finally given in and it is such a relief! I now see lexical naming as a low-level addressing mechanism. Programmer-level references are actually paths made out of names. The IDE can google your code and find the path to any name you like, but it resolves to a relative path, much like a URL.

      1. I think as long as you are including an IDE with your language, you can always leverage it to handle name ambiguity. Sometimes, you can just look at already inferred types and resolve the name right away. Sometimes, the resolution is ambiguous and you just keep it in a multi-option state, either for later resolution by further inference, or by explicit programmer intervention in the IDE. I mean, its possible, but it has to be designed in early. The names are still relevant to the programmer of course, but in the compiler…it can treat them with more care; this has to be designed in early of course.

    2. Where would you select the right + operator? In SummedPair it has to be generic, so selecting there makes no sense. This leaves only selecting the right + operator when instantiating the generic variable. I’m not sure how that would look/work, but could become “not simple”.

      1. It is possible to leave the type variable open, just constrained by the fact that it must support some + operator selected explicitly. Type classes in Haskell work like this, a bit. Alternatively, a specific binding can be chosen when the + operator is selected (so + is overloaded directly on all types that support it). See my typeless paper for some ideas on how to do that.

  7. Please, clarify: have you abandoned the idea of “no static source code” forever or just put is aside to simplify the demo? In case of the first one, would you please share the reasoning? Thanks

    1. Good question Peter, I owe everyone an update. Currently and for the immediate future I will have a human-editable textual syntax for Subtext. It is necessary to bootstrap and explain the language. The IDE will allow both structural and textual editing, though I am hoping to avoid fancy incremental parsing and partial analysis ala Eclipse. As I add smarter features to the IDE they will be based on structural editing. Eventually I will cut the cord with text files, but that may not need to happen till I add distributed version control (which can’t be done right with text). The IDE will probably always offer an optional editable text view even though that is not how the code is stored.

      1. Wow, this makes it sounds like you really have a plan to put together a useful tool and not just an academic series of demos. Very exciting! 🙂 I bet you could find volunteers to help out if you asked :).

        1. Thanks Jake. I know I can’t do this alone but I keep hitting dead ends that force me to rethink and start over from scratch. I hope this time I can finally build a solid base and open it up. Sure feels a lot more solid this time so I am hopeful.

      3. What will the structural editor look like? Is it anything like Subtext2 with the schematic tables?

        1. Not initially. My top priority is to get Subtext fully working ASAP. The quickest way seems to be to use conventional conditionals for now. I do plan to reincorporate the ideas from Subtext 2 as soon as I can.

  8. Yes if you opened this up, you could make the website like jsbin (http://jsbin.com/) where people can write code in your language and compile it to JS and play with it, share it, discuss it with each other, it will help as you can have a community around it.

    Community really builds up a good body of knowledge around a language and tooling.

    Even if you just open it up at a prototype stage with a list of unfinished items, thats better as people can use it and discuss it and give opinions, your still the creator, the benevolent dictator.

    1. I agree totally Phillip. My whole goal in life is to get Subtext to the point where other people can play/help/steal/mess with it. I feel terribly isolated. But I need to get a basic foundation in place first, which keeps getting washed away to my great frustration. I am still doing basic conceptual design of the language and it is very hard to collaborate at that deep level. And the few people working at that level tend not to be into collaboration. We’re all a bunch of wannabe Picassos. Still it is good that you keep pressuring me to open it up.

      1. Hi Jonathan!

        I’ve been following your blog for years, and highly respect the work you’re doing. Though you may feel alone, I have the same goals as you do, and I know many many very smart programmers do as well. The thing is, there are more approaches to making programming easier and more intuitive than there are people working on it, so I’ve found collaboration to be rare.

        Although I’ve been thinking deeply on these problems for about 10 years, none of my ideas have even made it to the prototype implementation stage, so again, I have highly admired your Subtext projects.

        I’m especially excited about this latest development, because it’s the first time where your work on Subtext actually closely parallels my ideas and intuitions on what the next step should be. I’m really looking forward to poking around with the code, and adding to it.

        I also agree with Phillip that an implementation that targets JS would likely be very beneficial to this project. Also, I’d love to hear / talk about your ideas for the future development environment interface.

        1. Hi Abram,

          Nice to meet a fellow seeker. Thanks for the encouragement.

          My current plan is to start working on the IDE as soon as I finish with collections. I am thinking that the killer feature is not the new language semantics but the programming experience it enables. Witness Bret Victor.

          You should try to write up some of your ideas, or storyboard some scenarios. I’d be happy to take a look.

  9. You should check out D, it does something very similar. There is a template system like C++, but they also have `static if`/`static assert` which works like a normal if/assert but at compile time so you can put constraints on template arguments.

  10. This is startling progress, Jonathan.

    It seems to me to replace a ubiquitous technology one cannot simply leap to a new paradigm leaving the current universe behind; doing so would also leave behind the journeymen one hopes to benefit. The new reality must be created as an accessible transformation of the existing one. Therefore the first stepping stone to a new programming paradigm must be textual.

    I think your latest effort has accomplished that. Having read a bit about F#, I think you’ve made better choices regarding simplicity. You also seem to have a good eye for avoiding the creeping tendency for languages to become “too meta”, thus less readable. Be careful that opaqueness doesn’t sneak in through the IDE, though.

    Bob Nystrom’s Magpie seemed to be heading in the same excellent direction. I hope he looks back here and responds to your question.

Comments are closed.