Looking feedback on a multi-node deployment setup

sodapopcan

I’m looking for some feedback/validation on a deployment setup. It’s a few months out still, just asking early.

My main app is a mini monolith. It’s a relatively small e-commerce web app with an admin area and some other features all contained within the same Application (not an umbrella). Everything is LiveView so there is no web API involved (thanks LiveView, ya da best <3). Apart from that, there is a separate tiny Application that handles image processing. It’s developed as a separate app for two reasons: a) The Security considerations that come with running libvips as BEAM NIFs, and b) I want to be able to run a copy of it locally as we do a bunch of image experiments that don’t need to run in the cloud.

My original thought was to have a little web server for the image processor and communicate with it that way. Then I remembered: “Oh right, BEAM. I can just run the two apps on separate nodes and connect them, right?”. After doing some reading and some light experiments, it seems that I would want to use libcluster to have the two apps running on different nodes find each other. I can then communicate between the two with plain ol’ message passing. Does this make sense? Am I on the right track?

Sorry if this is all coming off as incredibly obvious—I have absolutely no experience with distributed Elixir and while it seems right based on reading and trying stuff out, I just wanted to say it “out loud” to someone since I’m working solo here. My brain is still very much stuck in the ways of Rails when it comes to deployments.

I’m not looking for any how-to answers or anything, I just feedback as to if I’m on the right track and if there is anything I’m not considering and and maybe some links to some resources to RTF I do know where the official docs are which is what I’ve been reading.

Thanks!

4 comments

/phoenix #testing #uploads

1 1411 4

2023-04-26 17:25:55 UTC

Showing Posts 1 to 4

Show Best Posts
Show All (oldest first)
Show All (newest first)

JEG2

Author of Designing Elixir Systems with OTP

I’m sure you know this, but libcluster isn’t required to connect nodes.

Post #1

sodapopcan

Hi James!

I do indeed know that but wasn’t sure whether it was better to make them auto-discoverable or not, even at small scale. Of course, I’m trying to go for the absolute simplest setup. I’ll likely be only running the two or three nodes, so my initial assumption was to give them explicit names and pass messages via {NodeName, host} tuples, which is what I’ve done while experimenting. I take it that should suffice? I assume I can just use something like monit for bringing the image node up on the off-chance it actually crashes.

Thanks!

Post #2

schneebyte

I would recommend the excellent :pg module.

I’ve been using it a lot lately with an umbrella app that runs on one node locally and gets deployed to multiple nodes in a cluster.
I just have my processes/services join a :pg group. Then use :pg.get_members to get a list of pid’s. Don’t have to worry about node names.
If a process goes down it automatically gets removed from the group.

Now for security considerations, as far as i know if an exploit gains access to one app it could just access the whole BEAM VM including all other apps and all connected nodes.

Post #3

sodapopcan

Awesome, thanks for the reply and suggestion! I’ll take a look as that sounds pretty simple for my usecase.

As for security considerations, I just meant those related to using Vix. It’s possible for massive images to crash the whole BEAM (as opposed to just a process). I’ll be doing my best to guard against that but I feel much safe running on it on its own node as I’d rather not have the whole app taken down, even temporarily.

Post #4

— All posts loaded —