Hi all

Context
I am learning Elixir, Phoenix and LiveView by making a hobby project on yet another ChatGPT wrapper. I’ve got to a point where I feel comfortable updating LiveView the way I want and now need to figure the how to manage and limit access to OpenAI. I have also used poolboy for resource management on another small project (where poolboy workers were used for contacting a constantly running Python process synchronously).

Where I want to apply limits
OpenAI API has limits for concurrency (at least on some models) and I want to apply limits on how many requests are in parallel and per minute (and also per day/month as it’s a hobby or half-hobby project, so I need a hard limit not to get broke suddenly).

Possible architecture
I am thinking about creating an API module with rate limiter such as ExRated (probably for start it will just immediately reject if we are over one of rate limit).

Then it would grab one of e.g. 5 workers controller by poolboy and return pass LiveView request to it (basically a set of conversation messages between user and bot). A worker would use something like openai_ex to contact open ai and pass stream of updates back to LiveView.

The problem
With my not too high elixir skills I have issues figuring how to do interprocess communication. I am thinking about the following interactions to be possible:

• A. LiveView shall be able to pass initial request to OpenAI worker
• B. Worker shall be able to update LiveView many times as chat continuation tokens are received
• C. LiveView shall be able to cancel the stream, because e.g. user decided he doesn’t want to continue this chat

Point A seems to be easy as I have already made a functional poolboy worker in the past and pretty much along the lines of this guide I can pass to transaction() whichever Map I like, but with points points B and C, I am not sure what’s the proper way to establish communication between a pool worker and a LiveView process.

• Shall I pass LiveView process pid to worker, so it can send “some more words received from Open AI” messages via GenServer.cast to be handled via LiveView’s handle_cast?
  • And first such message to contain worker’s pid, so LiveView could cancel via same GenServer.cast?
• Or am I better arrange a Phoenix PubSub communications with topics such as “openai_worker_1_commands” and “openai_worker_1_responses” so worker and LiveView processes would subscribe to them on initialization and then update each other?
• Or should I use something completely different? Like maybe there are even ready-made libraries for such cases?

As it’s my first real case of asynchronous interprocess elixir communication I am not sure where to start from?
What would you do?