Best DB for chat

Then probably genserver + postgres will work fine.

Hello,
- how import is the chat history (if not at all I would just keep it in genserver)
The chat history is important ( Purchases are made by chat )

- how long to you want to save the history
The chat history will be stored for at least 6-12 months.

- how do you work the historical data afterwards (searching, scrolling)
Minimum, just to find some order, or some times open some chat history to check the order.

It will be a chat 1 to 1 ( agent to client or bot to client) and it is expected to have a high number of conversations at the same time

In general : Do Not Use Cassandra except if you have a really really good team of devs or a really really good team of Operators/SRE. And i would probably put a and here instead of a or if i could.

Cassandra seems nice from the outside, but it is really hard to operate and really hard to develop for, especially if you care about not losing data or getting acceptable time frame for answers.

Discord uses Elixir / Phoenix and switched from The Mongo to Cassandra. We don’t really know if they’re having good luck with it, but I think there is a high probability of that.

I’d tend to agree with this assessment.

Plus, you also need to think about how the data is related before you think about scaling strategies. With a Chat app every message in the system is basically grouped by “room”. Whether that room accounts for 2 people or a group of people depends on your interface.

When it comes to scaling it, this is a data structure that is very, very easy to shard because you can shard by “room”. All that’s required to make it work is to have a central point to act as a directory of which rooms is on which database and then you can query conversation from there.

What I’m getting at is just, don’t dive to deep into trying to find some ultimate database that can spread itself across hundreds of machines when your data naturally separates into manageable chunks. Those “spreading” solutions become necessary when the data doesn’t separate so easily.

All I’m saying is that there is no reason not to just start with Mnesia. If you grow to a point where you need more then switch at that time. Whatsapp was serving over 600 million users on Mnesia so it’s understandable that it started to become a pain point. If your service gets that big then you can afford to spend some time changing your storage options.

Mnesia requires little to no setup. It’s more important to get your product working and out there than spending time deciding which database to use.

And let the content push to S3 when chat is inactive for a couple of miniutes. And kick the genserver. When they start chatting again pull the history form S3 again.

a genserver per chat would be fine, I think.

I would think about the use cases you want to implement:

• how import is the chat history (if not at all I would just keep it in genserver)
• how long to you want to save the history
• how do you work the historical data afterwards (searching, scrolling)
• how big will the history get
• how many users
• how can you partition stuff (maybe there are groups that can be separated for easy charding)

When you know more about the use cases, the amount and the frequencies you can think about the technology.

If you want to get started quickly I would just use what you have, like postgresql and separate the storage from the core app. Then change to something more powerful when needed.

Marcus

I do like me some k8s and terraform from time to time …