Sharing schemas across applications?

What I have done in this situation is create a mix project that contains the migrations and schemas (and usually some of the common interaction functions, like ones to generate useful changesets) and then use that as a dependency in all the individual projects.

During development, I replace the usual git path to that library in the mix dependency with a local path, e.g. this:

 {:my_database, git: "git@git.somewhere.com:path/to/my_database"}

becomes

{:my_database, path: "../my_database"}

Then changes during devel to the schemas gets picked up in further calls to mix compile without having to commit, push, mix deps.update etc. Very handy! Then before committing into the feature branch of the application repo(s), I revert the mix.exs change (git stash is pretty handy here), so that the path version does not polute the app repo.

I feel like you might be forcing separation of concerns without reason. If all the apps rely on the same schemas and database - why not just join them into a singular app? If you want some nodes to run certain services, but not others (ie. Your background workers to not run your endpoint) - you can just use env flags to control what is started in your supervision tree.

This is how I’ve setup my current worker/web (as worker currently relies on schemas in web).

It has resulted in:

  def start(_type, _args) do
    import Supervisor.Spec

    children = if app_worker?() do
      [
        supervisor(Verk.Supervisor, []),
        supervisor(Pingerly.Repo, []),
        {Task.Supervisor, name: Pingerly.Task.Supervisor}
      ]
    else
      [
        supervisor(Pingerly.Repo, []),
        supervisor(PingerlyWeb.Endpoint, []),
        worker(Redix, [redis_url(), [name: Verk.Redis]], id: Verk.Redis)
      ]
    end

    opts = [strategy: :one_for_one, name: Pingerly.Supervisor]
    Supervisor.start_link(children, opts)
  end

Not entirely happy scaling that for every new application I introduce. Maybe if there were just two apps (worker/web), but not now I have a scheduler and [potentially] more apps down the line. The apps have separate responsibilities along with dependencies and, as you say, supervision trees, different configs.. very messy to dump all that in a singular app.

But fair point.

Cheers. I think this is likely what I’ll end up doing.

It does get a bit ugly as more and more business logic related to a variety of modules and config ends up in the supervisor(s). One possible route is to Enum.reduce a children list by calling a function in each of the children which returns a spec or not, and is thereby included in the children list or not, e.g.:

Enum.reduce([Verk.Supervisor, Pingerly.Repo], [],
            fn module, children -> 
              case module.should_start?(args) do
                nil -> children
                spec -> [spec | children]
            end) |> Enum.reverse()

.. or some such. At least then the decision making logic is in each child. For child implementations you don’t control (e.g. deps), creating a factory module that produces child specs based on env would do (the module in the children list does not need to return a child spec that resolves to itself!)

In fact, I wonder if this couldn’t be added as a general feature in the standard library. I know I would have used it in projects already if it were .. something like this patch .. @josevalim: what do you think? Obviously that patch only covers Supervisor and would need similar implementations in all the child_spec() implementations and supervisors…

Don’t forget that a mix.exs file is code, you can always conditionally pull a local path or git, or from an environment, or whatever.

Folks tend to forget you can return :ignore from a start_link function. GenServer — Elixir v1.20.2. If you return :ignore from a start link it’ll just get ignored. This means you can have your worker list be a nice full worker list, and then the start_link function of each child can determine whether it should actually start or not.

This tactic works great for sort of “feature flag” style workers where they’re all pretty independent. Not so great for tightly coupled workers.