alexcastano

alexcastano

I’ve been working on a pool of remote sessions, where each session is created and closed with HTTP requests. Sessions are limited in number, so I figured, why not implement a pool? An additional requirement is that these sessions must be refreshed every 15 minutes if idle.

I decided to use a GenServer to handle the sessions. When it starts, it makes a request to create a session and stores the secret. On termination, it makes another request to close the session. It has a public API that makes an HTTP request using the session, parses the response, and sends it back to the client. And if it’s not used for 15 minutes, it’ll refresh the session itself.

To manage the pool, I’ve got a Supervisor looking after two children:

  1. Finch HTTP pool
  2. Poolboy worker pool

So when a Poolboy worker needs to make a request, it grabs a connection from the Finch HTTP pool. However, things start to go south when it’s time to shut everything down.

When the app needs to shut down, the Supervisor sends a shutdown signal to Poolboy. But it seems Poolboy isn’t that patient and doesn’t wait for its children to stop. From what I can tell, the exit/2 function works asynchronously. So, the children get the shutdown signal, but the Poolboy worker pool doesn’t stick around.

https://github.com/devinus/poolboy/blob/master/src/poolboy.erl#L285

Then the Finch pool shuts down, the Supervisor follows suit, and finally, the workers. But, here’s the twist: the workers need the Finch pool to send out a final request to close the remote session. So, they crash without closing the remote sessions.

What’s throwing me for a loop is why a library as popular as Poolboy would behave like this. Am I missing something here?

I also thought about using NimblePool, but decided against it because:

NimblePool may not be a good option to manage processes. After all, the goal of NimblePool is to avoid creating processes for resources. If you already have a process, using a process-based pool such as poolboy will provide a better abstraction.

Any ideas on this? Maybe a different approach?

PD: I opened an issue in Poolex :slight_smile:

Most Liked

mpope

mpope

If you need a solution before a fix is released from the libraries you could use a process that each worker links with called the WorkerManager. This process can add each registerd worker to a set for tracking, and listen for their deaths. Once all of the processes that were linked die, then this process can terminate itsself. You can use a supervisor is has a rest_for_one strategy, then I think if you have the order: [FinchPool, WorkerManager, WorkerPool]. The WorkerPool can shutdown, the manger will wait for all the workers, then terminate once the final worker dies, and then finally the FinchPool can shutdown. This ordering should allow for a worker that is shutting down to grab a Finch connection.

cmo

cmo

pooler exists too

alexcastano

alexcastano OP

Thank you for your proposal. Last commit in Poolboy repo was more than 4 years ago, so I’m hopeless. I’ll try your solution if I cannot find a good alternative to Poolboy.

I don’t like that the worker has to know about the manager to link, but I’m sure it will eradicate the bug :slight_smile:

Last Post!

dimitarvp

dimitarvp

In that case I think that the Erlang :jobs library could serve you because it also tries to auto-adapt depending on the observed (BEAM) system health.

Where Next? Top

Trending in Questions Top

RSP87
I’m working on a project that simulates the bumbl example in the programming phoenix book. It acts almost like an email client. We have a...
New
kszambelanczyk
Hello! Could someone please give me a help/sample code, how to delete a file from s3 using waffle/waffle_ecto from Phoenix app. I creat...
New
RemyXRenard
I’m seeing that a list inside a Kino.DataTable will be interpreted as a charlist, even if the Kino.configure() is set to charlists: :as_l...
New
velrest
So my question is quite simple and i have found no conclusive answer on forum, google or AI. Should we use :erlang.float for Integer to ...
New
samoloth
Hi, I’ve just set up an application with ash_authentication. There is only magic link strategy for now, so there is no confirmation add o...
New
nseaSeb
Hello, I know there is an approach for handling lists that allows for optimized traversal, but I can’t recall the specific method (somet...
New
FlyingNoodle
If a change or preparation module uses Ash.Changeset.get_argument/2 or Ash.Query.get_argument/2 (or any of the other get_argument functio...
New

Other Trending Topics Top

mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Dmk
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews