So, there is another Erlang VM on the way 
9 Likes
Feels like this is a problem:.
A process is simply a long running future, scheduled on top of tokio-threadpool work-stealing queue
A process should be exactly what it is in the normal BEAM, otherwise you’re gonna run into problems.
7 Likes
Yeah I see more future for ErlangRT because Enigma VM does shortcuts like that.
1 Like
This is describing how this VM implements a process, it isn’t saying that the process will be something different from the perspective of an erlang programmer.
2 Likes
Tokia does IO based scheduling AFAIK, so the machine will probably behave differently under CPU bound load than the original BEAM.
Programmers on the BEAM usually rely on the fact, that regardless the load of the system, they will eventuelly get some time slot to do their thing.
3 Likes
So, I’m not disagreeing that the new VM may perform differently than the current one. I’m simply making the sort of ontological point that a Future is not fundamentally a different kind of thing than an erlang process as viewed from the VM. A process in the BEAM is a struct with various properties that runs on a threaded work stealing scheduler. That’s exactly what a Future is too.
2 Likes
Well, but the underlying schedulers work differently. While the BEAM is pre-emptive, Tokios futures are not.
Of course, from a (erlang/elixir/whatever) programmer things won’t change. They just spawn their way through, but programms will behave differently under load.
Of course there might be interest for the changed behaviour, or perhaps its the price some people are willing to pay for other benefits of this re-implementation, but currently I have not yet seen any benefits…
3 Likes
I think it’s just like @NobbZ said and you’ll lose one of the biggest benefits of BEAM. Preemptive scheduling was the biggest reason I got interested in Elixir to create web apps…
2 Likes
Elixir / Erlang is preemptive, but the C code that runs a process’s BEAM byte code is not pre-emptive. Languages built on this byte code are effectively preemptive because the underlying C can choose to yield at frequently occurring points in that byte code. Presumably the Rust code inside the Future could work the same way.
It’s entirely possible that I missed something here and their plan is to have each future churn on bytecode till the bytecode does IO, but there’s nothing about a future that requires that that’s the case any more than the fact that the BEAM is written in C.
3 Likes
I actually think that the new BEAM implementations should look into exploring other design trade-offs - other memory management schemes, different kinds of schedulers, different data representation, interpreters vs compilers, etc. We already have BEAM working like BEAM, so I don’t see much value in just replicating that exactly, on the other hand exploring new ideas might bring back some valuable insight into the main implementation.
23 Likes
@benwilson512 you are right that interpreter could yield at any point, not just IO. Unlike languages like C# and its async/await implementation that only yields on IO. Not sure how that preemptive scheduler is done in BEAM but it’s really impressive and I bet it’s not that easy to copy it’s functionality.
In my earlier post I said that I see more future for ErlangRT. What I actually wanted to express, is that I see it as more as direct replacement for BEAM. Nothing wrong with having multiple VM implementations. One that would work with Webassembly, would be great, so you could do full stack Elixir.
3 Likes
I agree that it might be worthwhile to try new things, but honestly I’m pessimistic. The advantages to a preemptive fair scheduler should be pretty obvious. I’m super-optimistic about a rust-based VM in general though.
Also I should caution that I don’t mean that any rust-based VM should implement its multitasking in exactly the way that BEAM does (via goto statements? apparently?) but that it should implement a reduction-based fair scheduler first then elaborate other threading models. Maybe even do a rust-safe fair scheduler, and then carefully abstract out a goto-based scheduler in an unsafe block? Otherwise, we’re comparing apples to oranges. Rust code is much more modular than C, so while difficult, I don’t imagine it to be insurmountably difficult.
I’d be curious if a rust-based BEAM could implement maps differently for better performance, with some sort of global heap, and instead of copy-on-write, (or I don’t know how BEAM does maps behind the scenes, TBH) some sort of linked-list overlay-on-write/overlay-on-pass-with-tombstoning model. It might also not be terribly hard to mock this out on the BEAM itself using references.
2 Likes
As with any improvement or substantial change, one first must have a prototype, a baseline to discuss. It is already hard to just repeat existing BEAM with a good degree of similarity, and I believe it is prohibitively hard to change something WHILE reimplementing BEAM. So all things come in their time. Rust makes this refactoring step simple and reliable.
8 Likes
So far as I know is the existing code not very well understood by many, so less hack on it. So, it is also for educational reasons, code cleanup (due to rewrite :p) and such kinds of things. I honestly believe that alone is worth a rewrite since sooner or later, the people with a lot of understanding about the current implementation disappear, so its development stucks.
2 Likes
