Seeking thoughts on the design of my umbrella project: Multinode WhatsApp Gateway and Bot App

Node-per-tenant is somewhat unusual; more commonly you’d a setup where a DynamicSupervisor on a single node manages one tree of processes per bot, roughly like:

Screenshot 2024-10-31 at 7.47.36 PM810×521 29.4 KB

This lets all the instances of Bot share the rest of the BEAM support machinery, versus having separate copies in each client’s node.

This may not be possible if instances of Bot need a lot of resources - for instance, if you’re running LLM inference on a GPU in them - in which case splitting onto different physical nodes may be the only option.

But you need to restart the app to add a business. That is what he tries to avoid.

Why won’t OP be able to do it without a restart? They can use the remote shell to trigger a function for creating a business. There’s a myriad other ways.

Sorry, I was already half sleeping and missed the ‘Dynamic’ part of ‘DynamicSupervisor’

Thanks @al2o3cr so much for the insight. I think I need to dust off my notes from Sasa Juric’s book to fully wrap my head around the DynamicSupervisor setup you described. Appreciate the guidance, and I’ll definitely consider the resource implications for the bot instances.

Totally get it @BartOtten, no one wants to restart the app mid-convo

@krasenyp I’ve explored a solution that involves storing a table in memory (using :mnesia), connecting to this remote-node, and establishing a business account relationship with the new node.

The downside is that the entire app doesn’t interrupt conversations, plus the ability to create new business accounts ‘on the fly’. Each bot app has its own state, conversation history, and client profile, and it’s crucial that this info is isolated between business accounts.

On top of that, I’m using rpc.call rpc.cast to route to the destination node (The bot). The bot calls a LLM, and depending on the state, it performs long-running processes, image and audio processing, external API calls, and even calls a Pytorch model.

I’m sharing a snippet of the code that decides which node to send the request to. Guys, it’s working for now, but I’m not sure if it’s good practice. Any recommendations are welcome!

bussines_account_id = "...." # get from Meta request

case MyApp.discover_node_target(bussines_account_id) do # read from :mnesia
      [_, target_node, :target_app_name] ->
        :rpc.cast(
          target_node,
          MyApp.StateMachine,
          :new,
          [
            bussines_account_id,
            Keyword.get(sender, :sender_phone_number),
            Keyword.get(sender, :message),
            Keyword.get(sender, :wa_message_id),
            Keyword.get(sender, :flow),
            Keyword.get(sender, :audio_id),
            Keyword.get(sender, :scheduled),
            Keyword.get(sender, :forwarded)
          ]
        )
...

# output example
iex(node1@10.0.0.28)1> MyApp.discover_node_target("742393808949918") 
["742393808949918", :"customer_service@10.0.0.28", :target_app_name]

I’m curious whether you’ve tested how Mnesia behaves under heavy concurrent bot traffic across nodes. I’ve seen some folks switch to ETS plus a lightweight sync process because Mnesia got chatty in clustered setups. Did you run into anything like that, or does your layout keep the load balanced well enough?