Released v0.2.5-alpha Release v0.2.5-alpha · alfetahe/process-hub · GitHub
This release includes bug fixes and documentation improvements.
anuaralfetahe
michalmuskala
this is a bit naive question, but how does this compare to using the built-in pg module for keeping track of processes in a cluster and doing service discovery that way?
3
anuaralfetahe
This is a very good question! My initial plan was to use the pg module to track the registered processes.
Using pg would solve some of the issues, the biggest probably being keeping track of registered processes across the cluster. However, I would still need to figure out the underlying mechanisms to distribute processes, react to cluster changes, migrate processes to keep the load balanced, handle process state handovers, manage process replications, deal with network splits, and more.
My initial requirements kept growing, and I realized I needed all of that. At this stage, using pg to track processes was not an option anymore because I needed a way to store much more information regarding the registered processes. I could have used pg along with my custom ETS tables, but I decided to store all the data in the ETS tables and synchronize the table myself.
In fact, ProcessHub relies on another library I built which is based on pg. This same mechanism is used to synchronize the ETS tables between the nodes. Basically, I use the pg module to emit process registration/unregistration events to keep the ETS tables in sync. I use the same library for other pub/sub related jobs as well.
I would recommend others to use the pg module when starting out if the distribution requirements are not that strict because it is a really simple and reliable option. Once the distribution requirements become more stringent, there are other mature libraries that can help, and ProcessHub aims to be one in the future.
A small fact: I wasn’t initially planning to build such a library. I am actually writing software that needs this functionality, and at some point, I decided to extract the code into separate libraries. That’s how ProcessHub was born. 
2
anuaralfetahe
Version 0.2.6-alpha released
This version includes bug fixes, new API functions, and minor improvements to the documentation.
Changed
- Replaced
Cachexwith our custom implementation to enhance performance. - Updated the default values for
max_restartsandmax_secondsto 100 and 4, respectively. - Storage module now accepts the table identifier as the first parameter, allowing it to be used with multiple tables.
Added
- Introduced new API functions
get_pids/2andget_pid/2to get the pid/s bychild_id. - New guide page for interacting with the process registry.
Fixed
- Corrected an issue where local supervisor restarts were not properly updating the global registry.
- Fixed various typos and errors in the documentation.
2
shifters98
Thanks for another update!
In my current project I use :syn (GitHub - ostinelli/syn: A scalable global Process Registry and Process Group manager for Erlang and Elixir. · GitHub) for process registry for a pub/sub mechanism and :libcluster (GitHub - bitwalker/libcluster: Automatic cluster formation/healing for Elixir applications · GitHub) as as an automated clustering mechanism as well as ProcessHub for process distribution and control. It does seem like there is overlap (there is!) but each does a good job for what i use it for i.e. i am using selected bits of each one as they all have there strengths and weaknesses.
I was thinking is it worth making the process tracking method selectable so you can use/make different implementations if you wish? i.e. yours, pg, syn, custom etc as each has trade offs.
My project is just a hobby project but its intended to make a changing cluster of machines look like a single machine with a single memory space and have the system behave as if its a single machine with automated handling of nodes joining and leaving the cluster and auto migration of processes to keep the system complete and running.
Its a play thing and speed and memory constraints are not of any concern and i intend to use a lot of processes like objects (similar to GitHub - wojtekmach/oop: OOP in Elixir! · GitHub) but with all the genserver stuff hidden behind a macro DSL. You could then submit a program to the cluster without knowing or caring about where it is running.
Well enough rambling (sorry!), the joys of retirement is you can play with all these interesting techs!!
Best wishes
Shifters
4
anuaralfetahe
I was thinking is it worth making the process tracking method selectable so you can use/make different implementations if you wish? i.e. yours, pg, syn, custom etc as each has trade offs.
While I wouldn’t rule out the possibility of making it configurable in the future, given that the project is still evolving, the process tracking and process registry are currently very coupled with other sections of the code, making it difficult to make configurable at this moment.
Well enough rambling (sorry!), the joys of retirement is you can play with all these interesting techs!!
I wish I can get there myself too
1
anuaralfetahe
Good evening, version 0.2.9-alpha out!
Includes some bugfixes related with binary child_ids.
3
jaybe78
Hello,
Any idea when your library will be ready for production use ? Any other library you would suggest in the meantime ?
0
dimitarvp
anuaralfetahe
Hi, I plan to put the library through a testing phase in a real system within 6 months, but there are no guarantees.
The Horde library is very popular, and you might find it useful. Another option I recommend is building your own solution using Erlang’s :pg module.
0