Can we talk about REPL interaction?

Making Liveness
1

There’s a lot of chatter about live-coding systems, but not much regarding the middleware we use to send code to them. For some systems, there are already nice REPL-plugins available, but I’ve noticed that many live-coding libraries/language/frameworks/systems have their own plugins, or sometimes even their own specialized editors.

In most “traditional” programming scenarios, you can use whatever editor you want. Just compile and run your program, or execute the entire file. But live-coding often necessitates the ability to execute segments of code in isolation, which requires specialized tools that are often bound to particular editors and platforms.

The need for these tools gives rise to a bunch of (I think) interesting questions:

  • How do you send code to your language/interpreter?
  • Does this require a plugin?
  • Is this a generic plugin for the language, or specific to your live-coding system?
  • How do you go about making a plugin/editor targeted at live-coding?
  • What medium do you use to communicate between live-coding system and editor? (network? pipes? files?)
  • What editors do you support? How do you choose?
  • Have other people made plugins for your live-coding system?
  • What features are unique to your editor/plugin? What features does it have specific to your live-coding system?
  • Is your editor or plugin inextricably tied to the system? Is it the system itself? (Orca comes to mind)

I’ve been thinking about this specifically because I cannibalized an existing Lua REPL plugin for Emacs while building Cybin, but I’m beginning to yearn for something more specialized, robust, and modern. There are, as far as I can tell, no resources out there to guide someone if they want to create a live-coding-centric REPL plugin or editor. I think it would be valuable to hear from folks who have done this.

1 Like
2

personally I’m a big fan of building languages, so my preference for the live-coding things I’ve built has mostly been to eschew a REPL. Either by having an interpreter that accepts code via some other transport (Improviz gets sent code via HTTP requests) or by having it built into the editor (Atomiix runs in Atom, with the interpreter just being javascript so running within the Atom plugin).

In the case of Improviz, editors need a plugin, but the hope was that it’s pretty simple to create one. I guess it’s specific to that system as it needs to send to specific URLs, but in theory it could be fairly generic.

I guess that thinking about Atomiix in terms of your questions, the editor is the language system itself (It’s still throwing OSC to supercollider, but then what isn’t heh)
In theory the parser/interpreter is a standalone JS library, so could pretty easily be built into VSCode, a web app or a standalone Node server that accepts HTTP, but for the time being at least it’s intrinsicly linked to the editor.
The plus is that it’s pretty easy to install. The minus is that if you don’t like Atom then you’re out of luck.

It’s worth pointing out though, that as much fun as it is building languages from the ground up, and having more control over how interpreters accept code to run, it’s a significantly larger amount of work, and probably not worth it if you’re more interested in actually making music than building parsers/interpreters :smiley:

re-reading this I’m not sure how much I’ve actually answered your questions heh, but it’s certainly something I want to think/talk more about. I’m wondering if, as live-coders, it’s worth us contributing more to the languages we use so that we can get better support for the slightly odd ways we want to interact with these language runtimes?

3

This is an interesting topic for me, and tangentially relates to parts of a blog post I’m thinking of extending for an ICLC paper - something about the benefits of transactional code changes, or No Right of REPL. :smile:

PraxisLIVE obviously fits in your definition of “own specialized editors”, but the underlying PraxisCORE actor system really doesn’t require it - PraxisLIVE is just a graphical and code representation of that system. Every actor in that system accepts (and can respond with) its own code as a String. So how you deliver the code to that actor is up to you - you can have a node that listens on a file, or map the code property to OSC, use the command line, or whatever - you can even have meta-programming nodes that can do templating, etc. There are no simple answers to your questions, at least from an editor perspective.

Secondly, Java has very limited support for hotswapping out existing code. I looked at it initially, but swiftly realised that not using it was an asset not a liability. Despite the IDE editor flashing select lines when you update (actually those that have changed) the entire code is replaced. The system and API is carefully built to make a transactional change between one code state and the next - creating, copying over, resetting and/or disposing of resources as required. This simplifies external editing, because there is no need to execute segments in isolation. It also ensures that the code at any time is a complete picture - you can serialize down and restart the actor graph later and ensure you’re back where you were, without requiring a complete replay of REPL history.

As far as I know there’s a few system that do whole code replacement for similar underlying technical reasons - CLive comes to mind.

4

This is a good topic - I really, really, really didn’t want to write another REPL for my 3D printing system so I used web browers, which already have them built in. The odd side-effect of this is that I found it very complicated to work in existing REPLs after making that choice (I’m not exactly a browser developer and there were too many projects with too many dependencies for my liking). What I got was a hybrid of the built-in ones and a CodeMirror editor which simply attaches new JavaScript scripts, which are then compiled, and then deletes them from the DOM. This sounds a bit ridiculous but I borrowed the idea from Mozilla’s online MDN web editor, and it seems to be the only way that JavaScript compiles properly (eval doesn’t use all the optimisations available, apparently?) So, in a way the entire program is also hot-swapped, but, this being JavaScript, there are myriad ways to have side-effects persist.

I hope that makes sense?

TL;DR: I tried to swap in a JavaScript REPL and it was more complicated than I’d like, but the idea of a more general REPL repo sounds fascinating.

5

I’m working on a new approach that is not a REPL!
In REPL-based environments, your editor shows a scratch buffer that you use to stage commands, that you can then send whenever you like. The commands are incremental changes (deltas) that get sent to the server, who keeps an entirely separate and invisible model of the project. This has many consequences that I don’t like:

  • what is visible in the editor window and what is currently happening are not correlated (if the performer wants to, care can be taken to keep them as correlated as possible, but the only way to actually keep a correlation is to only add commands at the bottom and never edit anything)
  • sessions cannot be reopened because the code doesn’t reflect the session
  • things that are not assigned names/to variables can be ‘lost’ on the editor side but keep running on the server, where it may be hard to stop them

So on the other hand what I wanted was to keep the representation in the editor and the representation on the execution side identical. That means that all expressions in the code are constantly “live”, and when an expression is deleted and the file is saved it disappears. Anything should be changeable and affect only the expression that was changed, while everything else should keep existing happily and without interrupting execution and local state (e.g. LFO phase, current sequence step, …).

I figured out that all of this can ba made to work by using a language that has a syntactically-clear model of expressions (Lisp), tagging every expression with an identity, and writing the tag back to the file. As long as the tag is not removed from the expression, the expression will keep existing and running, and any changes to its parameters are applied. When an identity-tag is not found anywhere in the new file, the corresponding expression is destroyed on the other hand.

To run through the OP-quiz quickly:

How do you send code to your language/interpreter?

By saving the file. The whole editor’s contents have to be flushed each time, so this is convenient, but it could happen any other way (TCP, UDP, OSC…)

Does this require a plugin?

no! One of the benefits is that the approach is compatible with any editor / environment that can save files.

  • Is this a generic plugin for the language, or specific to your live-coding system?
  • How do you go about making a plugin/editor targeted at live-coding?
  • What medium do you use to communicate between live-coding system and editor? (network? pipes? files?)
  • What editors do you support? How do you choose?
  • Have other people made plugins for your live-coding system?

(don’t apply)

What features are unique to your editor/plugin? What features does it have specific to your live-coding system?
Is your editor or plugin inextricably tied to the system? Is it the system itself? (Orca comes to mind)

these are also technically a “doesn’t apply” but since Orca was a partial inspiration, I want to mention here that like in Orca, what you see in the editor is the system itself (or rather ‘appears to be’, but then again that is the same for Orca): there is no hidden execution of code and when you remove something from the editor representation is is gone from the ‘live’ representation.
[/quote]

I recorded a new demo of the system here:

It doesn’t specifically show off any of the features mentioned above, but maybe you can see how working is a bit more directly related to “what is happening” and less to “how to make things happen” - there are no .play() or .stop(), create, destroy verbs anywhere for example.