Orpheus: A networked functional layer

Whenever I try to explain how a piece of software works, I hit a library whose insides are off-limits. But to understand programming, looking at the libraries is sometimes the most important thing.

Let’s say you’re reading Martin Kleppmann’s Designing Data-Intensive Applications. It discusses B-trees and log-structured storage, Bloom filters, consistent hashing, snapshot isolation, vector clocks and CRDTs. What if you want to look at a database application where such features are implemented in a transparently visible library?

This is the kind of problem Orpheus is meant to solve. It runs on Ubuntu (and probably other distros). First install Rust and glibc. Open a graphical terminal and run:

Orpheus is a small but complete computing environment that lets you understand it all the way down, making it convenient to teach from.

Orpheus has a persistent state. All actions get written down as an event in an append-only log. To find the current state, you replay the log from the start, folding each event into the one before. Because the replay is deterministic, the same events always rebuild the same state.

This state can persist across machines. Each machine keeps its own copy of the log. They keep each other current over a network. A machine tells the others which events it has, asks for the ones it lacks, and passes along new ones as they happen. Every machine replays the events in one fixed order. Two machines that have seen the same set of events end up in exactly the same state regardless of the order those events arrived in.

Inside the environment, high level apps are written in a simple language resembling a functional fragment of Rust combined with Lisp. Expressions are like Lisp. E.g. (add 2 3) is 5. Underneath, there are only numbers and pairs, written [a b]. Everything else is made of pairs. A list is a chain of pairs that ends in a 0, so [1 2 3 0] has three elements. Every piece of a program is an expression that produces a value.

You write a function with fn. A definition is a name, the arguments in brackets, and the expression it returns:

sq = fn [x] -> (mul x x)

So (sq 3) is 9.

Libraries are lists of such definitions. You can select a function name and click the Def button on the top bar to display the function definition in the system log.

I plan to use this system to create a free course where every algorithm can be traced down to first principles.

I hope you think this is as cool as I do.