What is this live coding system that you have made?

Making Liveness
21

Long post ahead!* Most interesting part at the bottom :stuck_out_tongue:

Iā€™ve been researching different interaction models and uses for livecoding. First I built a roughly block-based UI dataflow engine for controlling shader uniforms from a MIDI control surface and VJing to live MIDI streams from drum machines etc.
(there was a link to clips of the results and some hints of the UI here, but Iā€™m only allowed two links that I need further down :smile:)

After a many fruitless attempts at building a node-based language with keyboard controls, I stumbled upon a different idea and context, an ā€œimmediate mode livecoding environmentā€ for scripting direct-manipulation tools in CAD and Vector tools. I built a rough proof of concept in love2d and showed it around a bit:

  • 5 minute presentation: https://youtu.be/JXgZJosmme4?t=34m43s
  • PoC video (text heavy): https://youtu.be/3_gDRfFtPEQ
  • PoC video (voiceover but very early): https://youtu.be/zlG01j462A4

The main idea here was to write a single piece of code that is continuously executed, with state management being abstracted away using Immediate-Mode UI techniques. It worked pretty well and I moved on from the PoC (https://github.com/s-ol/watch-cad) to implementing the concept as am InkScape tool. Itā€™s now half working but I put the project on hold because I canā€™t find a good purpose for this tool, and developing a tool without practice and an aim is hard (and pointless). I still think that the IM-scripting idea is powerful, but I havenā€™t found the right place for it. I would be very interested to know if anyone can think of something here.

Finally, some weeks ago I revisited my shader-livecoding aspirations. I split off a hot-reloading shader viewer with OSC controls for the uniforms, and then decided to focus on a new tool purely for the logic/control layer. After some conversations with a friend and a SuperCollider workshop I found a direction: a (new?) text-based interaction mode between the user and the livecoding system.

Iā€™m a heavy keyboard user and always appreciated the expressivity of text-based livecoding. I also use an obscure editor (kakoune), so Iā€™m often unhappy with the ā€œstandardā€ text editing capabilities in integrated systems. On the other hand, I often prefer dataflow logic and some other apsects usually found on the visual side of the spectrum. A particular thing I noticed about the Tidal, SuperCollider etc. interaction is that the code left in the editor is completely separate from what is being executed, and the state of the program and performance are completely invisible to the user. Evaluating lines generally causes long-running processes in the background, that are often completely inaccessible afterwards unless extra care is taken to label them initially, or by killing everything.

I came up with a different approach, taking inspiration from tools like ORCA, where the program and itā€™s execution environment are coalesced into one. The idea is pretty simple: Instead of evaluating strings of code that are edited in a file, a complete file or buffer is edited and evaluated. All the expressions in the buffer execute continuously while they are present in the source code, and when an expression is removed, it stops executing. The syntax tree can be freely changed (expressions added, moved, removed) in any way and all expressions that are unchanged will continue executing, while changed expression have the chance of updating gracefully. This is enabled by tagging every expression with a number representing its identity in order to correlate it across evaluations. Whenever the file is evaluated, all new expressions are instantiated and tagged, and the source code is modified accordingly and written back to disk. This way the system is compatible with any text editor (that can reload files).

You can see a demo from two days ago here:

Source: https://github.com/s-ol/alivecoding

My implementation is in really early stages, but everything I mentioned above works perfectly and it feels like a really promising approach to me. Iā€™d love to hear what you think of this and whether anyone has been doing or seen something similar :smile:

1 Like