Poolex - A library for managing pools of workers

Your Libraries & OS Mentoring Libraries
, ,
1

Hi :wave:t2:

Poolex is an Elixir library for managing pools of workers.

In short, this is a poolboy written in Elixir.

When I started this project, I had the following goals:

  • To solve the problem of missing documentation of public interfaces and “official” library usage examples.
  • To bring this library back to life. poolboy is not actively maintained. Even if poolboy is written perfectly, there may be a chance of incompatible OTP changes in the future or the appearance of new features we’d like to use.
  • Try to rewrite this library in Elixir. It’s not a problem, but I’d like to use Elixir dependencies when I’m writing in Elixir.
  • To add the ability to use different strategies for getting a worker. I think a developer may have more needs than just choosing a LIFO / FIFO. So I added the ability to describe and use implementations for operating with worker and caller process queues.

Some project links:

12 Likes
2

Release 0.7.0

Added FIFO implementation for getting workers from the pool. This is the same mechanism as the :fifo strategy in poolboy.

3 Likes
3

Release 0.8.0

Since the latest major release, many improvements have been made:

  • Shutting down the pool and its workers is more accurate.
  • The work has been optimized by eliminating unnecessary ETS tables.
  • Fixed many bugs with handling exit of workers and callers processes.
  • Added missed validation for some initialization options.

Also, there are some breaking changes. I will quote the release note for 0.8.0:

  • Option :timeout renamed to :checkout_timeout.

    • Reason: This option configures only the waiting time for worker from the pool, not the task’s work time. This naming should be more understandable on the call site.

      # Before
Poolex.run(:my_awesome_pool, fn worker -> some_work(worker) end, timeout: 10_000)

# After
Poolex.run(:my_awesome_pool, fn worker -> some_work(worker) end, checkout_timeout: 10_000)

  • Poolex.run/3 returns tuple {:error, :checkout_timeout} instead of :all_workers_are_busy.

    • Reason: It is easier to understand the uniform format of the response from the function: {:ok, result} or {:error, reason}.

  • Poolex.caller() type replaced with struct defined in Poolex.Caller.t().

    • Reason: We need to save unique caller references.

  • Poolex.run!/3 was removed in favor of Poolex.run/3. The new unified function returns {:ok, result} or {:error, :checkout_timeout} and not handles runtime errors anymore.

    • Reason: We should not catch errors in the caller process. The caller process itself must choose how to handle exceptions and exit signals.

As always, here are some important links:

2 Likes