Programming is not just engineering — it is design. All my hopes and beliefs about programming boil down to that one assertion, which has become my main cause. The misunderstanding of programming as engineering has many pernicious effects, infecting our tools, technologies, and practices. Computer Science embodies this fallacy in its very name. Therefore I have banded together with some other programming language researchers and developers to form the IFIP Working Group 2.16 on Programming Language Design.
Design is the rallying cry of our mission statement and proposal (though some in the group are not as passionate about the issue as I, nor even completely in agreement with my take on it). The group is in some ways a reaction to the way that the top programming language conferences have become stuffy and stilted, because of their fixation on rigorous evaluation (which would make perfect sense if programming was just engineering). I am amazed that I set out to form this group just a year ago, starting with a dinner conversation with Andrew Black while I was at the Emerging Languages Camp in Portland. Much of the credit goes to Andrew — I am just the pebble that started the landslide. It appears there was a lot of pent-up demand for such a group, because we got an enthusiastic reception to the idea, and quickly gathered an impressive set of people.
We are following the proven model of IFIP working groups, which means that this is an invitation-only group, with permanent members and provisional visitors. Some of us are a little uncomfortable with having an exclusive club, and we may experiment with ways to open up participation, or maybe run a workshop. I hope that this group will in some small way enable further progress in programming languages, and also help foster a better appreciation of what programming is really about.
I think disagreement on this issue is simply due to overloaded terminology. The term “Interaction Design” was coined in 1990 to separate itself from other design professions. Programming Language Interaction Design, then, would be a subculture of Interaction Designers who are formally trained in programming language theory.
Programming Language Design, on the other hand, does not necessarily need to care how humans will interact with its results. I hope that makes sense.
I am mostly interested in Theory of Organization stuff as it applies to language design. I got hooked after reading enough Joe Goguen papers and Carl Hewitt papers. Seriously smart guys both with similar ideas and perspectives on computing. And Joe was really one of the first researchers to be heavily formal yet also take into account interaction design, with projects like TATAMI.
Bill Moggridge. That’s who coined the term Interaction Design. I couldn’t think of it yesterday.
Language design IS interaction design. Your users just happen to be programmers and usually its in the domain-specific tool category. As a result, user testing is difficult since programmers, as a profession, have a lot of learning invested in their tools.
You should probably read Dan Saffer’s book on Interaction Design.
Language design is not isomorphic to interaction design.
The IFIP Working Group document quotes Steve Jobs. The Jobs quote is referencing interaction design. Another Steve Jobs quote:
With mixed fealings I read this post. At first I was thinking “Finally some people start stepping up to the plate.” Then I read on and thought “A closed group? That’s not very good.” Then I read the IFIP proposal home page and though “Oh, good. It is still about making something useful.”
But when I saw the members list I just had to comment on this post. I’m convinced these are all smart people with a lot knowledge and understanding about language design. However, I do get the feeling they are mostly on either a theoratical background, or they are on the side of already providing language design.
Why is there no one in the group, who just made a living out of actually producing software with the tools provided? That experience, with all the issues they entail, will potentially provide more insight than any of these members can bring to the table.
I hope this will change in some way so there will be input from the people actually just using what has been designed so far.
Those ivory tower types put men on the moon.
Laymen on programming language committees have tended not to do well. Bruce Eckel on the C++ committee is a good example. Filling committee or working group slots with practitioners ultimately denies slots from people who have areas of expertise. In the case of C++, the committee’s biggest problem was that it was woefully unprepared for the multi-core hardware revolution, having no concurrency or parallelism experts.
In terms of areas of expertise that seem to be missing, end-user programming language researchers. I don’t know what people like Martin Erwig or Margaret M. Burnett think of this committee, but they have both studied issues relating to spreadsheets as programming languages. Likewise, there are probably members for the Institute of End-User Computing who might add something current members lack.
For example, one of IEUC’s missions is to improve the security of software where domain experts have a lot of control over what they can accomplish. Such alchemical goals might be best achieved using object capability security and Mark S. Miller’s Authority-Driven Design ideas for Secure Interaction. From reading a lot of end-user computing research, researchers in that domain don’t appear to be aware of the security needs for applications involving more than two principals. Adding somebody from end-user computing would perhaps allow greater cross-fertilization of ideas.
I don’t see many theoretical people on that list. Some of them are trained in theory yes, but they are primarily focused on language design.
Hmm. Put like that, it sounds mainly like a win for the ivory tower, but let’s see what happens.
Yes, yes, yes. A programming language is a user interface to the computer for programmers, and it should be treated as such, instead of as a mathematical artifact. It is not a theoretical subject, like most type system/formal verification/functional programming/etc researchers treat it; at best it is an experimental science. A paper shouldn’t be structured around proofs that your type system is sound or something like that, it should contain evidence that your type system is actually useful for humans, evidence that it is a good addition to the programmer’s interface to the computer.
Therefore I very much agree with Jonathan van de Veen: you need to involve the humans that are actually going to use the interface into your research. In fact the whole research should be built around them.
Jules, I disagree with your position. A programming language is both a user interface and a mathematical artefact. It is a formal system that we use. We should prove that a type system is actually useful, and that it is theoretically safe. Both are needed.
Jonathan’s working group is meant to emphasize a part of programming language research that is underdeveloped, in comparison to other valuable fields of research that are well developed — but still have large progress to do. It is not meant to replace some kind of research with a different kind, seen as superior as you suggest, but rather to complement the scientific and engineering considerations on programming languages.
I agree, theory is still important. What I said was too strong. But I do believe that the motivation should come from the practical side, not from the theoretical side. Proving a type system sound is useful precisely because sound type systems are good for humans to use.
Currently research treats the theorems as the central piece, almost a necessary and sufficient condition for a paper to be considered good. In reality for language construct paper to be useful theorems are neither necessary nor sufficient. You cannot mathematically prove that a type system is useful, the only way to do it involves actual humans using the language.
Usability testing and user studies loose effectiveness as the tool’s learning curve increases. So, for example, CAD tools that architects depend on are very difficult to evaluation unless the tool is just an incremental variation of what the testing architect is already familiar with. You see very few CAD products because architects literally learn their tools when they are completing their degree. Programming is very similar. This is why the design process is more defined for consumer products, which by definition cannot support a large learning curve (they have to be intuitive, natural).
Given that observation, how would you evaluate a programming language design? You can’t do it very easily empirically. I’m not a big believer in Greene’s Cognitive Dimension although they are decent heuristics to follow when designing your language. What else is there?
Lutz Prechelt is an empirical software engineering researcher who has demonstrated ways to measure and evaluate large software systems like web application platforms.
Just because we’ve yet to open Skinner’s Box in programming language interaction design, does not mean it can’t be opened and the gift be beautiful. Programming language interaction design needs a Stanley Milgrim (as do all fields).
We don’t need Stanley Milgrim – we already know that programmers are quite willing to torture code.
Someone is being too witty.
Milgrim had way more famous experiments than the peer administered shock experiment:
(1) The Shock Experiment was just one of many that measured how people show obedience to authority
(2) The Lost Letter Experiment
(3) Small World Phenomenon, which later gave birth to Duncan Watts’ Small Worlds Theory
(4) Anti-social behavior experiment
He is broadly recognized as one of the best experiment designers in the history of social science, especially psychology.
It is not very useful to compare PL design to forms of design where social science and usability testing is effective.
Usability testing is limited on tools that require substantial learning investments. You instead have to do something called an “expert review” on these tools that isn’t very scientific, the results from such a review are biased on unreliable. All of the PL HCI work has been done on end-user programming languages for this reason. Kodu was extensively user tested; Scratch; Hands; etc… But the process completely falls down when you have a real programming language with a real learning curve.
Social science requires observing people “in a natural environment.” A programming environment is not natural nor is it meant to be. We can examine programmers in their current environment, but we can’t break that down to fundamental observations. See also: design research.
So what do we do as language designers? How do we evaluate our language designs if we can’t rely on A/B testing? It is really the dirty secret of our field: if we write a technical paper, we have to have some empirical or theoretical results to get beyond the PC…so we throw something in that isn’t really related to design, perhaps only to satisfy the PC. This is probably the main reason why Onward is very appealing to language designers.
Currently, we can only really judge the success of a PL overtime: does it get adopted, does it change our thinking, are programmers using PL A more effective than programmers using PL B over a period of 10 years? That is clearly way too long to inform the design ‘in the now,” so instead we rely more on the experience of the designer (who aggregates this knowledge over time).
My imagination may be running wild, because we probably are approaching the idea of formal studies very differently.
Empirical software engineering has a harried past when it comes to good research. For example, Fagin’s seminal study on code review was highly flawed, as shown by Victor Basili and others. But refined experiments educate a lot about tool design without testing whole languages.
Z-Bo,
I also think that “divide and conquer” might be a way forward. But it is easier said than done: we still even lack micro studies showing the virtue of functional programming over say..imperative programming.
We really need to study other fields of design that have the same problem. I mentioned architecture before: architects are married to their tools which makes evolution and revolution very difficult. They spend much of their schooling mastering their tools and so if you do user testing on a trained architect of a new approach to architecture tooling, the results will definitely be biased toward what exists.
I had this very discussion with an HCI candidate a few weeks ago. It is nice being in a lab where I get to interact with HCI researchers and Italian designers 🙂
Sean,
I come from a practical background, and mostly engage academics because I strive to be better. That said, I am stunned by how poorly thought out most of my tools are. For example, SQL Server Management Studio 2008 was the first TSQL editor by Microsoft with any sort of Intellisense support, and the quality of the Intellisense is quite poor. Even so, it raises my productivity substantially when I have to write TSQL.
Yet, I think the entire IDE for SQL Server Management Studio is completely broken by design. I think it demonstrates an entire problem domain where both academics and practitioners fail to understand secure interaction design. Oracle IDEs suck, too. There are no good IDEs for talking to databases. Even the APIs suck. I don’t need Gucci the Italian Designer to tell me these tools are hell in a handbasket.
I’ve slowly made inroads, through talking to some academics, that the problem is actually pretty bad. The only way I get any credibility is through demonstrating a lot of academic knowledge about the problem, and then I am allowed to explain how neither theory nor practice is up to par.
On the other hand, there is a lot of merit to the idea that experts use advanced tools. The SQLPSX Codeplex project I am a member of is striving to one day replace SSMS 2008 and its future releases by building an IDE out of PowerShell scripts. It’s not even the language here that is the difference maker. It’s interacting with the language(s) with tools that is the difference. That’s another thing. Interaction Design shouldn’t just be about one language, but many, perhaps different versions. The software factory community had a very good idea there, still quite untapped. It’s completely unknown to the HCI community.
Anyway, end rant.
Z-Bo,
Designers aren’t the flakes that we engineers often mistake them to be. They are often very logical and pragmatic and getting to know the good ones can be very insightful. One of the most mindblowing experience I had was working with Surface designers, the output just made so much sense but I knew I could never come up with that. I’ve also worked with designers for healthcare software, where design mistakes will literally kill people, and so they had very strong principles to abide by, almost like what we would call a theory…
The HCI field is also more diverse than we can imagine. There are HCI researchers who specialize in software development (e.g., Andrew Begel at MSR), who are coming up with some great stuff. HCI is actually a bit different from design, it is more about discovering and evaluating principles.
We can and should learn a lot from design and HCI. My only point is that even they have challenges when dealing with professional tools, challenges which we will surely share in.
Surely *some* kind of experimental evaluation is better than nothing at all (the current situation).
First, language creators should clearly state their goals with their language. What kind of applications are they targeting? What are they trying to optimize? Programmer productivity, runtime performance, etc. Then they should create a couple of benchmark tasks that are done
with the language.
For example if you are creating a language or language construct for web programming one of your benchmark tasks could include a blog like this one.
Then you let a couple of programmers implement the benchmark task. You compare the result with the same thing implemented in other languages (for web programming you compare with PHP, Ruby+Rails, etc.). There might be something interesting to say about how the people who were alreayd experts in the new language did, and how the newbies did.
You also ask the programmers their opinion on the new language. They probably have some very interesting things to say.
Sure, this is not perfect. It is better than the current method where sometimes *no programs get written in the new language at all beyond the few lines of example code in the papers*! And sometimes the language or language feature *isn’t even implemented*, let alone used.
Am I the only one who thinks this group is redundant? Programming IS engineering. I’m not entirely sure what you mean when you say “design”, but if you mean the aesthetics of the typed language, how verbose it is, how intuitive the keywords are, etc…then these are also engineering problems.
Separating engineering from design strikes me as dangerous, because you’re removing tasks from the well quantified approach of the engineer and giving them to the vague semi-religious approach of the artistic designer.
Engineering and design aren’t all that different from each other. Designers help ensure that systems work well in human contexts, while engineers might focus on ensuring systems that work well in more physical contexts.
I’m really amazed at how many people don’t get design at all. If you ever get a chance to work with a designer, you’ll realize they can be as rigorous as any engineer.
The definition I have always used for engineering is “design under constraint”. I’m guessing that came from noted engineering author Henry Petroski’s “The Evolution of Useful Things”. In most engineering disciplines these would be physical constraints but extending it to encompass logical constraints never bothered me. So I guess I never thought engineering (and programming) was anything other than design. To me, it sounds more like you have a misunderstanding of what engineering is rather that a deep insight into programming based on the distinction between engineering and design.
You seem to think there is no difference between engineering and design. Petroski argues that at its boundaries, engineering verges into design. But that is not the same thing as equivalence. I define engineering as solving quantifiable constraints, and design as solving unquantifiable constraints. Witness the incompetence of many software engineers at UI design.
I consider engineering to be a way of designing rather than something separate from design. So I would say that all engineering is design but not all design is engineering. Or to put it another way, if something isn’t design then it isn’t really engineering either. So I don’t consider them to be equivalent by any means.
The fundamental difference between software engineering and other engineering disciplines can be seen in the difference between projects. For example with a bridge the engineering is essentially coming up with the design but there is also a construction phase which dominates the project in terms of cost and time. In contrast the production phase of a software project (essentially compilation and replication) is automated to the point where it can practically be ignored in the overall cost.
This means that managing a typical “engineering” project is mostly about managing a production activity while managing a software project is mostly about managing a design activity. (this may be similar to your insight?) Now the design activity in a project like a bridge is the engineering phase of the project. This is why the sentence “Programming is not just engineering — it is design.” doesn’t make sense to me. I don’t know that we are all that far apart in our views but you seem to be using engineering in a different sense that I am.
Engineering is about an existential quantification (can we build one example of Foo) where Design is about universal quantification (can we ensure that all examples are Foo).
An example Foo would be: deadlock freedom. There is a difference between building one application that is deadlock free vs. ensuring that all applications are deadlock free.
That you want human factors is not unique to ‘design’. Engineering can also handle human factor requirements… just, in a per-project manner. If you want properties to hold when composing between projects (including use of libraries), that’s where language design would be relevant.
My own pursuit of language design focuses on the notion of ‘compositional properties‘ – i.e. not Foo directly, but rather the ability to reason inductively about the Foo property when composing two subprograms – i.e. I want the ability to say: ‘X*Y is deadlock-free because X and Y are individually deadlock free’ without peeking inside X or Y.