bismark

bismark

Phoenix Socket draining

We’re attempting to make our deployment process a little less… disruptive, so we’d like to find a way of manually draining all open websockets.

Background

We’re running on AWS autoscaling groups with an Application Load Balancer. When we deploy, we register the a new instance with the load balancer target group, then deregister the old one. Unfortunately, target groups deregistration is ignorant of websockets, so it waits the 300s for all open connections to complete (which of course the websockets do not do), and then forcefully closes all of the connections. This causes a bit of a stampede (yes, yes, this is partly client issue, let’s just assume the clients are bad actors) and we much prefer to do this more gently.

Question

Is there any “official” way of traversing the open websockets and closing them down? Aside from spelunking through the Supervision tree that is.

Though I’m open to hearing about other options :slight_smile:.

Most Liked

bismark

bismark

We’ve come up with a not-so-great workaround for now.

SocketDrano had a couple issues for us:

  • it relied on channel join telemetry events, so it would miss sockets that do not join a channel
  • it was oriented around draining upon sigterm, but we’re relying on CodeDeploy lifecycle events for triggering the draining

So, we ended up “extending” Phoenix.Socket by creating our own Socket module that has overrideable callbacks:

Then we create a custom init callback:

@impl Phoenix.Socket.Transport
   def init(state) do
     super(state)
     |> tap(fn
       {:ok, {_state, %Phoenix.Socket{transport_pid: pid}}} when is_pid(pid) ->
         track_socket(pid)

       _ ->
         :ok
     end)
   end

track_socket then does essentially what SocketDrano does (monitor the socket pids, disconnects them when told to).

I’ve opened a thread on the Phoenix Core mailing list asking about better solutions being added Phoenix itself: https://groups.google.com/g/phoenix-core/c/1umhh2X1oAM/m/_sfxU8xIBgAJ

mweidner

mweidner

For folks coming across this recently: Phoenix sockets now have built-in connection draining (since 1.7.2). You can configure it with the drainer option to socket/3: Phoenix.Endpoint — Phoenix v1.8.8

rjk

rjk

Presence.track can store key/values so you could store hostname and userid? So on a quit signal do a lookup on hostname and you have a list of connected sockets for that node? And you could use Phoenix.PubSub — Phoenix.PubSub v2.2.0 to keep the broadcast local to the node.

Where Next?

Popular in Questions Top

_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
JulienCorb
I am trying to implement my new.html.eex file to create new posts on my website. new.html.eex: <h1>Create Post</h1> <%= ...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New

Other popular topics Top

vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
AngeloChecked
What learn first? Rust or Elixir Hi Elixir community! I’m here because i want learn a new language. I’m a junior developer and mainly i ...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New
komlanvi
Hi everyone, I was playing with phoenix liveView but I run into an issue. I have a form and want to validate each input text when the te...
New
AstonJ
We’ve put together this wiki for Phoenix LiveView - please feel free to add any info you feel is worth including. What is Phoenix LiveV...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

We're in Beta

About us Mission Statement