Start cheap and fast using elixir until you forced to extract one compute expensive component which will be implemented in go.
2 Likes
Unlikely, I don’t think go even has a NIF library built for it yet so you would end up having to do a Port or network request, compared to Rust, C++, or others where you can get the same or better speed and all can directly interface with the VM in every way the VM can be interfaced with, thus I don’t see go being used as such a thing unless latency is not a factor and the place already uses go both.
4 Likes
BEAM itself has too much startup overhead to be very useful in a generic serverless runtime.
Running serverless functions within the BEAM, on the other hand, would be pretty awesome; your task would be to use hot code loading to pull in F, spawn a process to do all the work, kill F, then replace F with the next function to run. It’d probably have a lot less cold start overhead than what Lambda does. I’ve heard of people doing application-specific “serverless functions” with erlua already. The only part that’s actually hard is sandboxing.
The serverless runtimes I know of spin up containers to house and handle requests. This solves the sandboxing problem. It also mitigates the startup overhead problem. As others have shown in this thread, you can have an Elixir app startup in a few hundred ms.
Now if your 99% latency reqs don’t allow for a few hundred ms cold start times, then move on. Otherwise, you’re starting fast enough, and then running for as long as your container stays active. And that would be the trick to running Elixir within serverless runtimes. Starting up the BEAM runtime along with the container, and then use the serverless’ native function support to dispatch to your BEAM service.
Cold start = few hundred ms
Per request = native function dispatch + xmit to BEAM + BEAM processing + xmit response (hopefully this round trip is fast)
I’m still dabbling with Elixir, most of the time i’m programming in Go. This is my 2 cents.
Go is pretty fast, the deploy is amazing because it’s static compiled and the concurrency is awesome. The problem for me is, the type system for a compiled language is too limited. So, i got to use interface{} more then i like it (yep, no generics). The error handling could be better, but that don’t bothers me much.
Elixir in the other hand play a nice role between performance and expressiveness. It’s like if Go and Ruby had a child 
The thing is, i still find Go a lot easier then Elixir in places Elixir was supose to be better, like concurrency. Maybe, and to be very honest, it’s because i really still don’t fully understand Elixir patterns. But in Go channels, select and the context are amazing. It’s trivially easy to write concurrent code and because of this i’m still struggling to adopt Elixir in more projects.
1 Like
You might want to start a “Elixir for the Go guy” thread. I’m sure you’ll get a bunch of feedback. Feedback that might pollute this thread. If you’re interested in higher level solutions and to improve your Go vs Elixir concurrency understanding.
4 Likes
Preemptive scheduling in the BEAM VM makes a big difference that people regularly underestimate.
Go mostly follows the POSIX’s pthreads model (as does Java): mutexes, semaphores, conditional variables that unlock the mutex, select / poll on multiple I/O operations. It is however mostly helpless if you overload it with a lot of concurrent processing without I/O. It relies on programmers knowing the lower level details and using its API responsibly. Which is fine with many former C/C++ devs but not fine for almost everybody else. Go goes a little bit the extra mile to stop people from shooting themselves in the foot but even the best compiler cannot do much when it exposes a raw access to OS threads.
Not to go too off-course here, the point is that the BEAM VM hides the raw OS threads access and orchestrates the actors (processes) transparently. And it gives them time slices via preemptive scheduling. Which is much better.
I think the original problem was with the high-level language aspects of concurrency and Elixir. Not the runtime aspects.
About the preemptive schedule, yes, Go is not fully preemptive, the checks are at function calls. So a for {} may hang the thread forever, but this is a work in progress for the next release. Most of the time this won’t be a issue.
Also the tooling to trace this kind of problems is amazing. Using the language constructors, this problems are less likely to happen (the ones i was talking about earlier).
Doesn’t the fact that one panic can cause the entire process to crash bother you? Any one of the go functions can do this and it’ll stop all the rest with no recovery option. The if err construct fills up methods bloating them massively and causing you to lose context on where the error actually occurred since they can literally happen on almost every line in most go programs. I hate getting an obscure unintelligible error message from go, and the hours following tracking down which line it came from is horrible as well.
I think if you take a bit more time you’ll find Elixir’s is far simpler. Sure you can use channels in go, but you don’t have to and sometimes mutexes and locks are still used mixing to two concurrency concepts which muddies the water quite a bit.
Elixir is simple, everything is a green thread and they all run independently in terms of CPU and memory. There are no locks, each process just has a mailbox that it can check whenever its ready, same as this forum or basically any other form of communication we do today; its all asynchronous, you check when you’re ready. At this point everything is simpler in Elixir until you start to want to resurrect your processes from crashes (something go can’t do), thats where supervisors come in. They just reboot each individual green thread when they crash, but setting them up takes a bit of thought and time to learn the API and how it works, I’ll give you that. If you really wanted to you could run things all “wild west” like go does and just not use supervisors, but I wouldn’t recommend it.
In the end, for every person you find who truly understands mutexes and locks, you’ll find 10s or 100s (or more) of people who can’t and even the ones who do frequently get it wrong because theres just too many variables and possibilities to think of. Remove that and switching to an async actor model, thats something everyone can understand because its what we already do every day.
4 Likes
I would even go as far as to say that every person I know that truly understands locks, mutexes, semaphores (Pop quiz: What is the difference between a mutex and a semaphore? 
, atomics, spurious wakes, deadly embraces, how to not get bitten by the ABA problem, et cetera is very happy to have a reason to not touch them.
Getting them right is very difficult because their lifetimes and working is orthogonal to normal code flow. Making sure concurrency flow and normal code flow match up is exactly what the actor model provides.
3 Likes
Yes, i agree with you. Everything in Go is happening in green threads too, but, if one of then panic without a recover, the whole system goes down. Elixir/Erlang is one of a kind that handle this perfectly.
When i was talking about the error handling does not bothers me, was in the sense of typing if err != nil. Coming from exception based languages, i can get much better errors and understanding of what is happening the way Go does it. It’s very easy to contextualize the errors like this: errors.Wrap(err, "error during query"). And if you stop to think about it. it’s like Elixir does it too, the difference is the pattern matching.
This talk GOTO 2018: The Robustness of Go: Francesc Campoy talk about the robustness of Go and later compare it to Erlang. In the end, Go need something like Kubernetes to have the features Erlang has built in.
About the mutex and locks, yes, Erlang does a better job abstracting this from the programmer, and it’s very good. The thing is, i still find it difficult to use concurrent code with Elixir, maybe it didn’t clicked yet for me. Take a look at this example in Go:
g, gCtx := errgroup.WithContext(ctx)
var count int
g.Go(func() error {
c, err := countRecords()
count = c
return err
})
var results []Products
g.Go(func() error {
pp, err := fetchProducts()
products = pp
return err
})
if err := g.Wait(); err != nil {
return errors.Wrap(err, "yep, something got wrong")
}
It starts two goroutines, one to count the records and one to fetch the results, in case of any error, it stops and goes on. I don’t need to wait the both to be finished in case of error, just a single one. The ctx can be canceled by the user when it closes the browser or if a timeout happens, and this is propagated all the way down.
And this one:
chanDone := make(chan result)
chanErr := make(chan err)
for i := 0; i < 100; i++ {
go func() {}()
}
select {
case result := <-chanDone:
// finished the job
case err := <-chanErr:
// got a error during execution
case <-ctx.Done():
// contxt canceled
}
I don’t know how idiomatic do this with Elixir. But again, this maybe is my limited understanding about the language. This is why i’m still here trying 
2 Likes
That might be because your examples are about quick one-off concurrency. When concurrency comes up in Erlang/Elixir people often jump onto genserver and supervision topics, which are more important when you have long running processes doing work concurrently. For those quick “make that concurrent” tasks you might want to to take a look at the Task module in Elixir, which handles usecases like you posted in a similar fashion.
4 Likes
Indeed this is the problem, for long running tasks, i don’t have any doubt, it’s way better to work with Elixir. The cancelation problem still a problem, but we talk about this later.
But how the pattern i just did with Go can be made with Elixir? How to spawn 10 tasks, in case of any error, stop all the others and report the error. Or, wait the response from the first task and return?
I did this come somethimes, but, well, it didin’t looked very idiomatic.
Task.async
Really. Start them, wait on them. Really, that’s it.
They are linked to your spawning process, so if one goes down it takes down the parent process which in turn takes down all its children, including all the other Tasks.
If you’re doing this in Phoenix, without any more effort on your part, this will result in an error report in the logs, and a 500 back to the client (in production, in dev it will return a ridiculously detailed error page). If you want to trap the error and provide some other handling, then you have to learn a bit more about OTP and set that up.
3 Likes
@sribe is quite on point. Additionally, you can also use Task.Supervisor. Again, if one fails, everything else stops. This is exactly what the :one_for_all supervising strategy does.
Ah, so you are doing these with simple interfaces instead of frameworks, well in elixir:
# Since the count and products return different types, no need to identify them differently
# You return the values via the yield, not a channel, hence why these are more simple but the yield is more complex
ac = Task.async(&countRecords/0)
ap = Task.async(&fetechProducts/0)
results =
[ac, ap]
|> Task.yield_many()
|> Enum.map(&elem(&1, 1)
if Enum.any(results, &(elem(&1, 0) != :ok)) do
{:error, "yep, something got wrong"}
else
# Do whatever else...
end
In Elixir:
for _ <- 0..99 do
spawn &func/0 # I'm not actually sure what the go version is doing here to be honest, just an empty function?
end
receive do
{:done, result} -> # finished the job
{:error, err} -> # got an error during execution
:done -> # context canceled
after 5000 ->
# Lets add a timeout too so we can handle what happens when we don't get a message within a set time. :-)
end
The whole channel stuff is very much not elixir, but the elixir way is very simple, basic message passing, timeout handling, and a whole lot more even before we start getting in to the built in OTP framework!
9 Likes
About FaaS, there is this interesting thread on twitter: https://mobile.twitter.com/coryodaniel/status/1029414668681469952
You’re thinking very small-scale but perhaps more importantly you’re also thinking about how to manually handle these kinds of things. Elixir and Erlang are built for setting up structures wherein you remove the need to think about how to manage these things specifically and instead use supervision trees and other processes to automatically handle these in a reasonable way.
While it’s certainly entirely possible to handle small-scale starting and managing of a few processes because you want to spread some tasks out on processors, where Elixir and Erlang really shine is where you’ve set up a server to automatically spawn these tasks and have them fit into a supervision scheme that provides guarantees with regards to “Does this have to finish? What happens if it crashes, etc.?”. Your above example, having all tasks fail when one fails is, as @dimitarvp said, encompassed in a supervision strategy so that you can not write this yourself.
Concurrency on the BEAM with OTP is meant to abstract away the tedious primitives so that you can focus on your actual problem. The primitives are still there but unless you have something extremely specific in mind it’s likely that a network of GenServers and Supervisors will do exactly what you want them to and all you’ll have to specify is “What happens when the server gets this signal?” along with “What happens when a server fails?”, because that’s what the actual problem is. Spawning processes and handling them manually is not.
7 Likes
It is kind of low-level, but contrary to the impression that everybody else is giving, it’s actually perfectly easy to write equivalent code in Elixir.
The only difference is that you don’t use shared variables for the results and count. Honestly, that’s bad Go code; you should be using channels for count and results.
parent = self()
{count_pid, count_monitor} = spawn_monitor(fn ->
case count_records() do
{:ok, c} -> send(parent, {:count, c}
{:error, e} -> raise e
end
end)
{products_pid, count_products_monitor} = spawn_monitor(fn ->
case count_records() do
{:ok, r} -> send(parent, {:records, r}
{:error, e} -> raise e
end
end)
# This receive will handle whichever message gets sent first
# so it should probably be in a loop
# (then again, it should probably be in a GenServer, which already does the looping for you)
receive do
{:count, c} -> use_count(c)
{:records, r} -> use_records(r)
{:DOWN, ^count_monitor, :process, object, reason} -> handle_count_crash(object, reason)
{:DOWN, ^records_monitor, :process, object, reason} -> handle_records_crash(object, reason)
end
Sixteen non-empty lines in Go, nineteen non-comment lines in Elixir, and the Elixir code has lines to handle the count and records variables while the Go code doesn’t.
4 Likes
