Should I use Redis or a global GenServer for data between applications?

Questions / Help
,
1

I think the title says it all, but let me write a bit of an explanation.

Say I have an app A which does some work and it hosted on one machine. App B is a phoenix application which is the user dashboard, etc. and is ran on a totally different server. And I want app B to access some data which is stored in app A’s GenServer. Should I use clustering here (if possible, I think it is) or create an element in Redis which is connected to the GenServer.

I’ve no experience with clustering so I don’t know if it’s good for me to use a GenServer to share data accross or use Redis. (Also if it’s possible to cluster 2 different applications)

2

This feels like an XY problem to me. What kind of information does each application have, and where does each application store that information for its own purposes today?

2 Likes
3

As @benwilson512 says, you are describing to us the problems with one potential solution of your original problem – without telling us what that original problem is.

What are you looking to solve exactly? Redis is an amazing piece of software but global state in general should be avoided as much as possible (easy to say from an academic point of view of course; but it’s a good principle to remember).

1 Like
4

Okay. In that case I’m going to be giving information of what I actually want to create:

The worker/app A is a discord music bot. It will listen to message events off from Discords’ API and, with the correct input from the user, store specified links in a playlist. Currently the playlist is saved via a GenServer.

I will be making a Phoenix application which will give the user the ability to show the playlist in a nice web UI.

I don’t need any long-lasting persistence, i.e. that the playlist needs to be permanently stored.

Edit:

I think it should be noted that I plan to cluster the worker app.

1 Like
5

Couldn’t this just be a monolith? What are you trying to achieve by clustering?

1 Like
6

  1. I would like to distribute the workload so that not everything is ran from a single node. Audio can and will stutter if the app were to simultaneously stream, let’s say, 2k audio streams.

  2. I’ve never actually clustered any kind of app and I think it would be nice thing to learn.

7

I’ll preface this by saying that I don’t a ton of experience and I’m going off what I’ve seen on the forum and elsewhere. If I were you I’d just put it all in a single node and collect some metrics on that; the streaming workload might be heavy but the web workload should have a minimal impact on that. Remember that you can also easily scale vertically and it’s way easier than scaling horizontally. This would also let you organize all your modules, functions and the structure of your app in general; afterwards you can split it in order to experiment with clusters.

Keep in mind that I, too, haven’t clustered any apps but it seems to me like Redis would be a better option in this case so you only focus on sharing a decoupled datastore and have a bit more fun while splitting the app.

1 Like