How to start workers only on specific server?

I have an application that performs daily imports of data. This needs extra RAM. I want the import process to start only once in the cluster. Also I would like to have 1 node with extra RAM, and all other nodes for Phoenix and other stuff with less memory.

How can I have this setup that certain processes start on specific server/node. Is it done with an extra release that I deploy to my special server? Or something else?

A separate release configuration is definitely how I would do it, or slightly less preferred, using a runtime configuration variable that determines whether or not to start the batch work as part of the normal supervision tree, so that it becomes opt-in.

How would it work with separate releases?

Here is a similar structure to my project:

my_app_umbrella/
  apps/
    my_app_core/
    my_app_db/ # <- import workers live here
    my_app_web/

Right now the import workers start when db app boots. I guess I need to move them to a separate app first? Something like this:

my_app_umbrella/
  apps/
    my_app_core/
    my_app_data_import/ # <- workers now live here
    my_app_db/
    my_app_web/

Then I’ll create 2 releases:

releases: [
  web: [
    applications: [
      my_app_core: :permanent,
      my_app_db: :permanent,
      my_app_web: :permanent
    ]
  ],

  data_import: [
    applications: [
      my_app_db: :permanent
      my_app_data_import: :permanent
    ]
  ]
]

Is that sort of what you meant?

You can also do this with just 1 release as long as the server with extra RAM can provide some kind of flag (system environment variable, whatever) to indicate what it is. Then you run the same code on all servers. The release boots and goes “Do I see the flag?”. If not, it doesn’t boot the child. If it does, it boots the child.

This is the “runtime configuration variable” part that @shanesveller was talking about, right?

Ah basically yes. Personally that’s my preferred approach, but it depends on how radically different the supervision trees are. If it’s just 1 child, I’d use the runtime flag. If it’s basically a different tree entirely, then different app config / releases.

1 Like

Thanks. I think this sounds like a more appropriate solution for my case. But I still want to understand the releases part as well. What do you think about my understanding above?