Designing a Multi-Language Live Programming Tool With Phoenix and Genstage
Nextjournal enables living documents for researchers. Our goal is to make technical writing and scientific research easily reproducible and shareable through beautiful typography and live coding tools.
In this talk I'll show how Phoenix channels, GenStage and Erlang ports elegantly solved the complex challenges of building a multi-language, browser-based live programming environment.
Starting with a demo of our editor, this talk will cover:
- Scheduling of code evaluation using a directed acyclic graph of GenStage nodes: despite a non-standard configuration of stages (mostly of constant demand = 1) I’ll show how to take advantage of GenStage subscriptions to mimic dependencies between code cells.
- Execution workflows shaped around reactive/functional concepts like observables and signal mapping.
- Interfacing with foreign languages through erlang ports and having such languages “talk together” with suitable serialisation protocols. Code is executed in user-customisable isolated environments to guarantee reproducibility of the methods used.
- Using the transit format on phoenix channels and Ecto custom types, for efficiently transferring values between applications
Extra topics, time permitting:
- How we integrated a ClojureScript frontend with Phoenix, building our assets live with figwheel
- How we implemented real-time collaboration by broadcasting editing operations over Phoenix channels
- Although no advanced Erlang/Elixir knowledge is required, we’ll assume some notion of Phoenix channels and a rough idea about GenStage behaviour
Andrea left academia 6 years ago after a PhD in Algebra and embraced software craftsmanship, inheriting a sense for coherent systems and amalgamation of structures.
Elixorian of the first hour, he is very grateful to this community for letting him discover the elegance of the Erlang virtual machine. Andrea contributed with some more or less useful Elixir open source projects including exotic plug adapters, a phoenix OSC controller and some perceptron simulation with OTP. He uses Elixir at work, full-time.