Nothing to do with Elixir in this case. I haven’t measured the beam release startup for a typical phoenix app in a while, but it’s not a concern in the context of this original question. The three seconds is for Fly to pull the image, place it, connect it to our proxy and the child to Node.connect back to the parent. The release start is in there somewhere, but complete unmeasured guess is less than 0.5s for a phoenix app to be up serving traffic. So I don’t think anything will move the needle much here with any beam/elixir optimizations.

Having said all that, the current 3s baseline is not a concern for me. Faster is always better of course, and it’s possible the fly team can work some optimizations on our side. For example, waking up idle (already created) machines is usually 500ms. But there is some nuance to comparing cold starts to typical FaaS. I know AWS also keeps their functions “hot” underneath but it’s much less 1:1 for us. Runners in the pool will almost always be serving many concurrent operations (the pools :max_concurrency) vs thinking of them as cold starting to serve 1 user. Also, the way elixir supervision trees start up also guarantee that :min runners will be hot before the app starts serving traffic, so you aren’t bouncing servers to then hit thundering herd of cold starts. You can also use a min: 0 , but still warm up a min number in your sup tree to get scale to zero, but warm startup. The runners would idle down after boot if no traffic is active.

The missing piece currently is our pool growth strategy waits for the pool to be at capacity before growing, so cold starts can be hit as the pool grows. Next steps are to mitigate these cases with more sophisticated strategies like observing growth rate / percentage based early starts / etc

So I hope that helps give some insight. Faster is always better, but current baseline is not a concern for me, and I don’t think beam/elixir startup has much to move the needle on the FLAME startup side.

Better to say that it meets your tolerances - and those of many developers and organizations. But let’s not discuss it as if there’s no room for improvement, or for criticism, or that “slow startup” is not a matter of perspective. Dismissing the OP’s concern like that isn’t cool IMO.

Elixir’s a VM-based language with a runtime that takes measurable time to start, which is why we’re still suboptimal for things like adhoc scripting and creating CLI applications and wouldn’t have been a natural choice for actual Lambdas either. The BEAM is probably competitive with something like Python or Node for cold-start time but it is probably not quite the same level as something like Golang or Rust + Tokio is capable of. It’s a little better than non-Graal JVM IIRC.

Anecdata on my 2023 16" M2 Max:

hyperfine --warmup 2 -S none --export-markdown results.md -- 'elixir -e \'System.halt(0)\'' 'elixir -e \'IO.puts("Hello world")\''

That’s not loading an application or starting an Ecto pool or anything a lot of projects would need to be actually useful, just the most trivial things the runtime is capable of. So IMO it’s not a great floor to start from.

This Rust example starts a Tokio runtime (with a WAY smaller scope than OTP, so not particularly apples-to-apples in that regard), then does a TCP dial and writes to the conn before shutting down:

gh repo clone tokio-rs/tokio
cd tokio/examples
cargo build --release --example hello_world
cd ..
# https://nmap.org/ncat/
ncat -kl 6142
# new shell
hyperfine --warmup 3 -S none --export-markdown rust-results.md -- './target/release/examples/hello_world'

On a platform like Fly or EC2 any of that is dwarfed by provisioning time, of course. And for “traditional” Elixir applications, no one much cares because the ratio between “time spent serving a purpose” and “time spent starting/restarting” is extreme.

I accept that there is a different setup process in how workloads get spun up on fly vs AWS.

The Firecracker VM (a KVM based VM tech that AWS developed and what fly use to run our workloads and “transmogrify” containers into).

Firecracker is awesome as it provides VM level isolation that containers can’t , but the VM abstraction is also different to allow incredibly fast boot times (eg no ACPI).

Fundamentally a Linux VM takes about 75ms to start on Firecracker.

In contrast FreeBSD boots in 25ms so a savings of 50ms in start time.

Phoenix server startup time for releases is quite fast but I haven’t timed it precisely either. It is may be as high as 50-100ms and I am interested if anyone has a precise metric to time taken to start phoenix and listen on the socket.

So accounting for VM start, OS boot and BEAM/phoenix start should be achievable in well under 100ms. I remember the Erlang Ling VM could boot on Xen and serve a request in about 50ms.

A 50ms target is certainly a possibility on FreeBSD but not on Linux as its boot time on Firecracker is 75ms, not including starting the BEAM. Fly doesn’t support FreeBSD currently so right now it’s more likely that integating FLAME on AWS offers the path to much lower startup latency.

The 3+ second start time is almost all provisioning overhead and further opportunity for fly to optimise. Certainly scheduling work, and copying images to the target instance adds some unavoidable overhead due to that approach.

I guess it’s a fly business decision for approaching this differently from a warm start service tier. I can imagine low latency startup for serving dynamic workloads is something people would pay for, but getting the balance right between a dynamic on demand service pricing vs commitment tier pricing is something that needs consideration also.