How to make a new livecoding language

I was just reading an article on language and communication and I’d thought I’d share some provocations on livecoding language design & evolution. Here, I’ve laid out a couple of nice principles that I think I’ll try and live by when (if??) I’m making new languages, and more immediately, when I’m refining LivePrinter’s syntax.

First, an article on communication. Gärdenfors works with a pretty straightforward definition of “communication”:
“At its basics, communication is for solving problems of actions, i.e., coordination”

She then proposes a “division of communication into a hierarchical set of levels: (1) Instruction; (2) Coordination of common ground; and, (3) Coordination of meanings”

I’m really struck by this, because it’s what I’ve seen emerge from my project in both my personal experience and user workshops.

On the one hand, I need to often tell the computer (or printer, or other system) exactly what to do, e.g. send it precise instructions. In my code, this translates to sending 3D printer states that translate to movements of the printer’s motors. In audio, I’d argue that this is where you actually play the note.

Other times, I need to work with the audience and other participants to coordinate where we are in the process of making something and need to coordinate where we are in that process. I’m mapping this to “common ground,” which is declaring variables and setting the program state (speed, etc.) for things that don’t have immediate effect, e.g. not instructions for action but preparation for action that need to be agreed on prior to action.

The syntax itself needs to be negotiated through functions and other higher-level structures (during livecoding), up to changing the syntax itself (likely done in when coding the re/developing itself). I take this as “coordination of meanings.”

It’s interesting (or maybe obvious, in retrospect) that this kind of process comes up in design. I recently read a PhD thesis from Tabor Balint, a NASA engineer who studied at the RCA and was interested in changing the culture of NASA’s project planning and execution using ideas from Cybernetics and design research. He came up with this circular diagram which I’d suggest is a good summary of design stages for livecoding language design:

Lastly, Blackwell, Britton, Cox, Green et. al wrote about the “cognitive dimensions of notation” which Alan Blackwell linked to livecoding in some of his past papers. The CDNs were mentioned by @yaxu in a previous post and were to

to assist the designers of notational systems and information artifacts to evaluate their designs with respect to the impact that they will have on the users of those designs

“Notational systems” can be user interfaces, meaning the “cognitive ergonomics” of screen-based representations. In our case, they definitely apply to the hybrid text/editor systems for livecoding.

It’s interesting to look at their “activities in constructing notations” and how similar they are to the previous examples: I gave:

  • transcription
  • incrementation
  • modification
  • search
  • (added later:) exploratory understanding

Examples they gave of these activities:

searching for a single piece of information (such as looking up a name in a telephone
book) and incrementally understanding the content of the information structure expressed by a notation (such as reading a textbook).

incrementing an existing structure by adding new information, transcribing information from one notational form to another, modifying the structure, or exploring possible new information structures in exploratory design.

I’m curious to hear what other people think of these examples? Is it something people are aware of and use?

1 Like

Great topic, thanks Evan. For now, just sharing a sci-hub link for the Gärdenfors paper, for those (like me) who don’t have institutional access or £35 to burn :slight_smile:
https://sci-hub.se/https://link.springer.com/article/10.1007/s11229-014-0493-3

1 Like

Right… non-open access is a PITA. If you do need access, give me a shout, there might be a way I can help.

They can get the article by clicking the scihub link, Evan.
Thanks again for posting this, I’ve been heavily influenced by Gärdenfors and his theory of conceptual spaces when I read it eight years ago, but not read any of his more recent work.

I’m not sure how this applies to live coding. I think Gärdenfors isn’t too interested in syntax, but semantic systems of metaphor and so on. He’s talking about negotiation of meaning between humans. That seems different from live coding, where I suppose language is primarily used to explore rather than negotiate. It’s an interesting idea though, after all one of the historical synonyms for live coding is ‘conversational programming.’

I hadn’t heard of Gärdenfors before today, I’ll have to read through his back catalogue.

Is livecoding primarily about exploration? That’s an interesting take. I suppose it makes sense to think more about the context for livecoding if we’re talking about its purpose, because that might change based on what we want to do with it. I’ve been kicking around this idea that we can think of livecoding in the classic music/threatre/performance modes of:

  1. composition - writing new scores and exploring relationships, likely but not always in private
  2. rehearsal - preparing and practicing for a live performance; consolidating/refining tools and sets; more constrained exploration
  3. performance - playing live; improvisational but prepared, or tightly scripted

Since livecoding is before an audience, we can think of it as a communication of sorts (mostly one-way) and of performances as establishing a syntax of a meaningful relationship between code and action that the audience can comprehend on some level. Meaning, you type in patterns in tidal and the audience starts to understand that they represent musical events, over time. You name variables and functions in a way that both programmer and audience gets some meaning from them (even if they are abstract and poetic and more about metaphor than strict instruction, like an “ADSR” function vs “crunch”).

Also, maybe I’m the only one doing this, but I think of coding as a conversation between my present and future self. What do I need to know in the future, when I perform this piece? Or later on, when I rediscover it in an old hard drive? How do I talk to myself from the past? I do spend a lot of time writing annotated code to teach others, which feeds into that. I think everyone should be forced to do that…

All that aside, my original post was a bit more concerned with language design, which I see as more of a legal/syntactical wrangling done with interested groups. In that sense, it’s more about establishing and negotiating a working system of shared meanings between performer-developers and less about being understandable to a general audience. There inevitably has to be some politics involved, and that means a lot of human coordination.

I’ve not seen many studies that look into the evolution of livecoding languages and study the changes in syntax and the discussions that led to them, although this is something that could easily be done using the conversations from, say, tidal. A person could look at how ‘slow’ and ‘fast’ evolved and changes names and behaviours, and what discussions led to that. It might be interesting to see what sort of words a live language coalesces around, and where they are applied, like function names vs. variables and control structures, etc.

Just to add more to that point - I think a mistake we often make is to think of langauges statically, when of course they change. Livecoding languages are very experimental, and change very fast! Is it reasonable to think that they will stop changing so fast at some point? Or will we look back on them in 10 years and laugh at what weird-sounding functions “the kids” are writing these days? “In my day, a person could walk into a club and read any line of code. These days, it all looks like random characters to me. What happened to good old Gabba?”

2 Likes

I am loving this conversation, mostly because it relates to my current project. I need to read the articles before I comment too much but I wanted to mention this idea of designing the language.

Right now I keep saying I am making a language. I think of the language as a piece right now. I don’t really see anyone else using it (although they are welcome). It’s not for a community. If it was for others I think I would opt for the term design and would have a much different process of how to consider communication.

Also since my end output is human movement there is a definite need to find a way of agreeing on movement making… this is a specific concern to my work but I think there is a connection to Gärdenfors but again I need to go and read.

2 Likes

Sorry to do this, but I wrote quite a bit about Gädenfors and cognitive semantics in the context of live coding in my thesis, might be worth a look, although predates this work on communication.

Thanks @EvanR or the amazing recapitulation on the topic! I hope I’ll find some time to read the articles soon.

It kinda makes me realize how naive (?) my approach to designing a language is. So far I never really thought about it from the point of view of communication, negotiation, and all that.

My approach was basically, analyze my preferred paradigm of music organization, find the most appropriate data structure to represent it, and find the most concise syntax to enter the data.

Then I just kept working on it in a somewhat exploratory manner, while slowly sorting out inconveniences.

In that sense I recall an article i read a while ago, I think by Sam Aaron and/or Alan Blackwell, which compared the evolution of live coding environments with the knowledge accumulation process in toolmaking … but i can’t find it right now, maybe s.b. else remembers this ?

Totally second that on the one hand … I guess my own code from 2 months ago might not run anymore on today’s version of my language.

On the other hand, I’d say there’s also some convergence ? In the sense that certain paradigms seem to occur frequently, in slightly different flavors, in many of the systems I had the chance to see in action since I first encountered live coding …

EDIT found the article and it actually is by Sam Aaron and Alan Blackwell, and it’s called Craft Practices of Live Coding Language Design. It was published in the proceedings of the ICLC2015, but i can’t find it online right now. It reviews the design of live coding languages from a very practice-led point of view …

You can download it here.

1 Like

This was a fascinating read by the way, I guess I’m on a similar wavelength as @parkellipsen.

For me, designing a language for livecoding was more focused on UX, my thinking was that it was going to be performative, it needed a few things:

  • Constantly operating, try/catch everything by design.
  • Shortest possible function names.
  • Visual queues instead of error logs.
  • Nothing obfuscated, everything always visible.
  • No scrolling.

There inevitably has to be some politics involved, and that means a lot of human coordination.

That aspect is especially fascinating. Your language is bound to find itself in the hands of like-minded people. These people will often share your flaws.

1 Like

but also celebrate your quirks… that’s one reason I decided to try tidal ages ago, and also loved aspects of MaxMSPJitter. You can really feel the personality of the individual developers/artists in it. There’s no strict logical reason for the way things work, it’s something deeper and more poetic.

In my experience, making is one aspect of designing but not really something separate. At the RCA, there’s a pathway in Design Products called Design Through Making (DTM) which celebrates the individual craft and process of design, less strategically, and more about playing with the raw materials and tool making. Donald Schön wrote about “thinking-through-making” that goes on inside the design process, where there’s a sense of knowing that isn’t explicit and representational, it’s more of an intuitive knowing that comes from doing something.

It’s all about practice-based research, isn’t it? What Pickering called “The Mangle of Practice” and what I’ve been trying to unpick for my PhD writing. So far, I’ve liked Rosan Chow’s review of design research methods which I found through Wolfgang Jonas’s writings in a chapter on A Cybernetic Model of Design Research.

According to Jonas, you can start by making things for yourself, as someone who is a practicing professional embedded in a community, and it’s a valid mode of design research:

Research AS design : An embodied/situated/intentional observer inside a design/inquiring system, concentrating on the production of ‘variations’ AS raw material for the design/inquiring process. Research in action, performed in the medium of design. → Design as the inaccessible medium of knowledge production: a learning process. Probably the essential mental and social ‘mechanism’ of generating new ideas, the location of abductive reasoning. Research AS design may denote ‘Design Thinking’ as a cognitive and social process, which, in turn, can be the subject of inquiry ABOUT or THROUGH design.

It’s taken me awhile to get my head around that… but I think he’s onto something. (For more, this will be a chapter in my future thesis, eventually)

I should also add that the legendary Bill Gaver works and writes quite a bit in this area. I’m not so familiar with people outside this design/architecture sphere but I’d be interested in others, I know that there’s the “performative ontology” thread in social sciences too.

Thanks @parkellipsen, how did I miss that article?? It’s the missing link in many ways, you’ve got Sam & Alan referencing Frayling and Gaver about software/art/craft and practice-based research. Good find!