Bandit - a pure Elixir HTTP server for Plug & WebSock applications

All,

After several years of effort, I just published version 1.0.0 of both the Bandit and Thousand Island libraries. Folks that are depending on versions in the 0.x.y or 1.0.0-pre series of either library should update your dependencies to be ~> 1.0.

This has been a ton of work, and has been made possible in large part due to the help of tons of contributors. In particular, @moogle19, @ryanwinchester and @alisinabh have gone above and beyond on all fronts. The project wouldn’t be the success it is without help from folks like them. Thanks all!

I put together a bit of a retrospective blog post about the whole journey here, if anyone cares to learn more!

I have tried bandit only once. There was some simple bug, but it was hard to debug. For some reason my instinct suddenly directed me to this library. What’s funny I read about it only once before and I simply forgot its name, but somehow I have managed to find it on forum.

Anyway, bandit’s error handling was an amazing help which saved me a lot of energy and time. Been waiting for a stable 1.x release, so I could use it by default. Thanks!

Well done guys and congratz on 1.0! Been using ThousandIsland for a small hobby project of mine and it works great

Awesome effort! And I‘ll definitely upgrade to it right away! Thank you so much!

But one question I have still: what are the typical performance improvements one can expect with an ordinary liveview app? I have no good estimate how much time per route is spent in the http layer vs other parts of the pipeline. Anyone ideas?

Excellent question and one I get a lot. The short answer is that in most cases you probably won’t see much of a difference between Bandit and Cowboy from a performance perspective; your plug’s implementation is going to be the dominant factor in overall performance, and switching out the underlying server won’t magically make that work go away.

That having been said, there are many workloads in which you could expect to see a benefit to Bandit. The ideal case would be large numbers of HTTP/1 clients doing lots of IO on very short lived connections. In that case you could see some substantial benefits (see my latest benchmark for more).

Some workloads are going to be worse. In particular, HTTP/2 performance in Bandit is pretty awful at the moment, but is going to be getting a lot of attention as part of the work to add WebSockets over HTTP/2 (RFC 8441) support. This will be one of the next things I’m working on.

In terms of LiveView, Bandit’s WebSocket implementation is generally a little bit faster than Cowboy’s (around 10-20%). You might see some real-world benefit there; it really depends on your particular usage patterns.

@mtrudel Hi
I was digging into Absinthe today and noticed it has a code path that results in a process just exiting when a certain internal timeout is exceeded. From Bandit perspective, this results in this kind of error:

15:08:27.012 [error] GenServer #PID<0.1581.0> terminating
** (stop) :foobar
    bandit.exs:13: anonymous fn/2 in Router.do_match/4
    /Users/amartsinovich/Library/Caches/mix/installs/elixir-1.17.3-erts-15.1.3/5a1480805a98346fc77508013de370cc/deps/plug/lib/plug/router.ex:246: anonymous fn/4 in Router.dispatch/2
    (telemetry 1.3.0) /Users/amartsinovich/Library/Caches/mix/installs/elixir-1.17.3-erts-15.1.3/5a1480805a98346fc77508013de370cc/deps/telemetry/src/telemetry.erl:324: :telemetry.span/3
    /Users/amartsinovich/Library/Caches/mix/installs/elixir-1.17.3-erts-15.1.3/5a1480805a98346fc77508013de370cc/deps/plug/lib/plug/router.ex:242: Router.dispatch/2
    bandit.exs:6: Router.plug_builder_call/2
    (bandit 1.6.1) lib/bandit/pipeline.ex:127: Bandit.Pipeline.call_plug!/2
    (bandit 1.6.1) lib/bandit/pipeline.ex:36: Bandit.Pipeline.run/4
    (bandit 1.6.1) lib/bandit/http1/handler.ex:12: Bandit.HTTP1.Handler.handle_data/3
Last message: {:continue, :handle_connection}
State: {%ThousandIsland.Socket{socket: #Port<0.22>, transport_module: ThousandIsland.Transports.TCP, read_timeout: 60000, silent_terminate_on_error: false, span: %ThousandIsland.Telemetry{span_name: :connection, telemetry_span_context: #Reference<0.2186918947.463994885.242980>, start_time: -576460734287898010, start_metadata: %{remote_address: {127, 0, 0, 1}, remote_port: 53034, telemetry_span_context: #Reference<0.2186918947.463994885.242980>, parent_telemetry_span_context: #Reference<0.2186918947.463994885.242884>}}}, %{opts: %{http: [], websocket: [], http_1: [], http_2: []}, plug: {Router, []}, handler_module: Bandit.InitialHandler, http_1_enabled: true, http_2_enabled: true}}

My understanding is that there’s not much Bandit can do here in terms of error handling and in general using exit isn’t the best way of handling this kind of case. But I was wondering, if it’s a good idea to define terminate callback for handler and issue a telemetry event? WDYT?

Yep, we can do better here. The work that grzuy did in fix: throwing plug properly handled and returns 500 by grzuy · Pull Request #411 · mtrudel/bandit · GitHub makes this nearly trivial. I’ll get a PR worked up to improve this.

Fixed on main. Will go out in the next release.

Thanks for bringing this up @martosaur !

Hey Matt,
Any thoughts on what may have been the bottleneck here? Seems we ran up the CPU as well.

Tough to tell. A few observations:

• BLUF: Of course this is an apples and oranges comparison. I personally couldn’t give two hoots how well we perform against other languages (especially natively compiled ones running on base libraries); that’s not a game I have any interest in playing or one that has any winners. It’s MONGODB IS WEB SCALE all over again and I have better things to do with my time than to engage in the comparative aspects of this. The two contestants aren’t even playing the same sport

• The PR’s setup looks fine (it’s not a matter of app configuration). I didn’t look at any of the lower level OS / BEAM tuning details

• He’s using m7a.large instances, which at first glance look like they’d perform a smidge worse than the instances we use for microbenchmarks in CI. From that perspective the results seem roughly correspondent with what I’d expect in absolute terms.

• I’m a little worried by the growth numbers that Bandit demonstrates. CPU usage shouldn’t be growing without bound like that, and (as he states) that’s likely the root cause of the lacklustre numbers elsewhere

• There’s not really a whole lot of actionable steps to take based on this data. We really do need a better benchmarking environment (ideally one that runs as part of CI), as the microbenchmarking setup we use now just doesn’t get to the absolute scale needed to reproduce these sorts of situations ‘in the lab’, which is a necessary precondition to be able to improve them in Bandit. If anyone is looking for a place to help, that’s probably the highest value way to do so