Scalable architecture for chatbot

General Prog (Archived)
1

Hello, my name is Ran and this is my first post here!

I’ve been learning Elixir for the last 6 months and so far I really like it! It just fits into my brain and I’m eager to start building my first project, a chatbot :slight_smile:

I just read an article about challenges to face when building a chatbot and I like to to talk about the “Scalable architecture for handling messages” part which I hope I can solve with Elixir.

From the article:

1. Instant handling – The server sitting behind the webhook should handle incoming messages instantly by moving them straight to the message queue. Some chat platforms may be pretty touchy about delays, so better safe than sorry.

2. Assigned worker – The message queue should guarantee that each user will always be handled by the same worker (consumer in message queue lingo). This is necessary to ensure that messages sent by a user are always handled in the order in which they are sent.

3. Load-balancing – The message queue should also perform load-balancing to keep the workers reasonably occupied with tasks.

4. Propagation – If a worker dies, the queue should be able to transfer its responsibilities to another worker.

5. Event processing loop – A worker’s event processing loop includes these steps: pop a message from the queue, fetch the conversation state associated with the user, handle the message, possibly replying to the user, and push the updated state back.

6. Storage – The storage place for the per-user conversation state should allow flexible structure and fast read/write access. The best choice seems to be a NoSQL database. We opted for MongoDB because its simple and extensible document format seems perfect for keeping track of the conversation state.

According to the article, I should probably use a message broker, like RabbitMQ but as the article states, it doesn’t support consuming the messages from the same user in order.

So I was thinking that a GenServer process has a mailbox, it receives messages and handle a single message at a time, so If I will create a process for every user who uses my bot and propagate the message to the correct process it will solve both number 2 and number 3 since every user gets a dedicated process.

The problem with this approach is that If the process dies, all the messages in the mailbox will be lost, so I now need some way to handle this situation.

My questions are:

  1. Is it a good idea to try and implement it by myself?
  2. Are there any existing solutions for these problems in Elixir?
  3. In general, am I in the right direction? tips?
  4. Elixir is a good fit for this use case?

Thanks!

2 Likes
2

  • Use a GenServer to serialize the messages.

  • Keep a mapping of ID -> GenServer, I’d use ETS.

  • As for storing messages in case a GenServer dies, well you can wrap it with your own exception handler, or have a trampoline process that passes to the GenServer (via pulling or so when it is idle), or just stuff all pending messages in to ETS too.

So yes, this is a good task to learn, and yes, this is exactly the kind of stuff that the BEAM excels at. :slight_smile:

1 Like
3

Thanks for for the reply!

It kinda feels like I’m re-inventing the wheel.
Are there any existing libraries that help with persisting a process mailbox?
It seems like a common thing that will be useful to a lot of people.

Another thing, where should I store the conversation state?
I need a fast read/write, flexible schema and persistence DB.
The options are: Postgres (jsonb column), MongoDB, Mnesia, any recommendation?

4

Hey Ran,

You can have a GenStage producer holding messages in a Erlang :queue and then partition the messages by the users unique ID. :slight_smile:

As for tge state persistance, the easiest way would probably be to use DETS or if you want to be distributed then :riak_core :smiley:

Hope this helps :slight_smile:

5

Thanks! I’ll look into GenStage :ok_hand: