I’m using Caddy as reverse proxy and it works very well for my projects. Not using Xamal, but I guess my setup is similar.
The socket/concurrency concern is theoretical, not practical. Caddy proxying to BEAM adds one local TCP hop per connection. On a single VPS serving a consumer SaaS app, you’d need tens of thousands of concurrent connections before this matters. The Linux kernel handles local socket pairs extremely efficiently - this is the same path nginx/HAProxy/Caddy take for millions of production apps.
The CPU overhead is negligible. Caddy’s reverse proxy for HTTP is minimal - it’s just shuffling bytes between sockets. The TLS termination has to happen somewhere, and Caddy does it well with automatic cert management via Cloudflare DNS challenge (how I configured it).
Yes, around 65k connections before you hit the ephemeral port limit. I build websocket and TCP proxy services, so tens of thousands is normal, hundreds of thousands, also not a big deal. That type of concurrency is no problem for the Beam on dedicated hardware with a few CPU cores and some memory and a rather appealing use-case for Elixir/Erlang in general. There’s been some nice blog posts on millions of connections with Phoenix.
Caddy (and Traefik) can consume a lot of CPU/memory when handling 30-50k persistent connections, on a smaller Digital Ocean VPS Traefik was using half the machine resources to do it. We had to remove it and have the service directly on the host with no reverse proxy in the path due to its resource usage and concurrent connection limitation.
My company is currently setting up our own version of a deployment system for our Elixir apps, so when I see projects like this I always hope that they take an approach that at least allows for the full capability of the Beam to shine without restrictions that reduce its utility for my use-case so maybe we won’t need our own custom system.
I can’t comment whether Xamal wants to solve it or not. But the way I see it - this solution is not for the kind of scale that you are talking about. But it is great for solo devs, small/medium companies. It is a nice way to have a solid hosting, without the complexity (my deployment setup is like 200-300 lines of bash). And it’s also a very flexible solution that you can add any other web server behind Caddy, it doesn’t care if it’s Elixir or not.
This is correct. I am not using this for the scale @bbangert is talking about. So haven’t even thought of optimization along those lines.
But now that he has brought it up… it is interesting. If Xamal could maintain the same experience/functionality and improve in this aspect I’m all for it. I’m not very familiar with the performance/scale/internals of Caddy/etc or eBPF.
If we let Cloudflare say handle the TLS and just have the server connect with cloudflared (which is how I have my apps configured currently) - then the only thing Caddy is doing is routing. @bbangert would you mind educating me more on what your approach would look like/what your thoughts are?
Sure @Dmk, the approach that I was thinking would replace Caddy with a different eBPF service that just exposes an API similar to Caddy to control eBPF switching. A new app release would run the app on a new port, validate its health, then make the API call to the eBPF service to switch the routing to the new port.
Note that there’s limitations to this approach, as eBPF is at the Layer 2 level, it can just route packets. If a goal of using Caddy was to use its SNI so you could dispatch different domains to different apps, eBPF won’t work for that.
The benefit is that if you are using different IP’s and/or you’re fine terminating SSL because you are only running a single app on the VPS or bare metal, eBPF, especially with XDP (packet handling logic before it hits the linux kernel right at the NIC) isn’t going to be beat. Just microseconds of overhead vs. a dozen or more milliseconds with Caddy, and no extra RAM/CPU usage.
I think for many, using Caddy is likely fine, but the folks running large websocket/TCP services (or even DNS with say, erldns and such), the eBPF/XDP route is great. My company is already going this path, we just don’t have a great app deployment story right now. I’ll be happy to circle back with more concrete code and examples of the eBPF/XDP side when we have it running, as I still haven’t figured out the best way to build that eBPF service API (Honey Potion is a bit alpha).
Edit: My bad, looks like enough packet logic can run in eBPF to look at the SNI header and route appropriately.
).
