MassiveFermion
Understanding nimble_pool internals
Hi
I’ve been working on a resource pool package for gleam for some time and this is what I have right now.
Currently it accepts a pool size and a function that creates a resource(can be anything) and creates a pool size number of processes that each has a resource as the state.
Then people can use an apply function that accepts a function that acts on the resource and returns a result. apply will checkout a process, send a message to it with the function so that the process can use that function on the resource, get the result and return it to the manager process, which will checkin the process and return the result to the user.
But right now this approach does not handle the situation in which either the worker process or the user process crashes and it seems to me that I won’t be able to handle those situations and still be able to provide parallel resource usage with this approach.
So I’ve been trying to understand how nimble_pool works so that I can implement a similar approach. But I’m having a hard time following what’s going on in the code(I’m a nodejs developer by day and do BEAM stuff as side projects, so not really proficient in OTP).
The only thing I’m sure about right now, is that nimble_pool will directly give the resource to the user. It also seems like there is a timeout after which it just checks the resource in, regardless of what the user has done with it but I’m not really sure about it or anything else.
I’d appreciate any guidance about this, thanks
First Post!
dimitarvp
I don’t know the internals of nimble_pool but I remember it saying that you should accept a single-process access to your resource. Meaning that if you have 5 processes each needing access to the resource they’ll have to wait for each other to finish. Is this what you require?
As for handling crashes, you can simply spawn your workers under a Supervisor or DynamicSupervisor that you configure and they’ll get automatically restarted – though I am not sure that is your intended behavior either.
Can you talk a little more about your exact ideal scenario? There are good options for many requirements but we need to know a little more about what you need.
Most Liked
christhekeele
Note, I only recently discovered the new stdlib PartitionSupervisor but it looks like it does this exact thing for you out-of-the-box.
dimitarvp
the primary reason I wrote it is to use it to add connection pooling
Then definitely start with :poolboy because nimble_pool is much more manual and can confuse you.
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex









