zamith

zamith

Hi,

I’m implementing a system that implements multitenancy via multiple databases. For business reasons I cannot do away with prefixes, which I’d prefer. In any case, I was able to make it work by using put_dynamic_repo, but am concerned it might be a bit too “magic”.

I’m doing this with a Phoenix app, so I set the repo on the request process, any sub process will have to set the repo again, which is easy to forget.

I’ve been pointed out to an alternative that is to wrap the Ecto API and require a pid for every call, which seems more obvious but both more labor intensive as well as brittle, since any change in the API breaks the code.

I’m very torn in terms of what’s the best option (and maybe there’s even another that I’m missing). Any advice would be great.

Here’s what I’m using to manage the repos, note that the tenants are agencies and I’m using a GenServer to cache the dynamic repos that were created. Also have a function to create a new database and run the migrations if needed.

defmodule MyApp.RepoManager do
  use GenServer

  alias MyApp.{
    Application,
    Repo
  }

  def start_link(settings) when is_list(settings) do
    GenServer.start_link(__MODULE__, settings, name: __MODULE__)
  end

  def set_agency_repo(agency, ensure_exists \\ false) do
    if ensure_exists do
      ensure_repo_exists(agency)
    end

    repo_pid = GenServer.call(__MODULE__, {:get_dynamic_repo, agency})
    Repo.put_dynamic_repo(repo_pid)

    {:ok, repo_pid}
  end

  def unset_agency_repo do
    repo_pid = Repo.put_dynamic_repo(Repo)

    {:ok, repo_pid}
  end

  def init(_opts) do
    {:ok, %{repos: %{}}}
  end

  def handle_call({:get_dynamic_repo, agency}, _from, state) do
    case state.repos[get_database_name(agency)] do
      nil ->
        {:ok, repo_pid} = Repo.start_link(get_connection_options(agency))

        {:reply, repo_pid,
         %{state | repos: Map.put_new(state.repos, get_database_name(agency), repo_pid)}}

      repo_pid ->
        {:reply, repo_pid, state}
    end
  end

  defp get_database_name(agency) do
    "my_app_#{agency.slug}"
  end

  defp get_connection_options(agency) do
    [
      name: nil,
      pool_size: 2,
      database: get_database_name(agency)
    ] ++ Application.db_config()
  end

  defp ensure_repo_exists(agency) do
    options = get_connection_options(agency)
    Repo.__adapter__().storage_up(options)
    {:ok, repo_pid} = Repo.start_link(options)
    Repo.put_dynamic_repo(repo_pid)
    Ecto.Migrator.run(Repo, :up, all: true, dynamic_repo: repo_pid)

    Repo.stop(1000)
    Repo.put_dynamic_repo(Repo)
  end

  # TODO: Manage pool size better
  # TODO: Support removing repos
end

Showing Posts 11 to 20

zamith

zamith OP

Exactly, that was my thought process. My main concern is the fact that any sub process does not use the dynamic_repo. That’s why the wrapper you mention might be a good option, even though it’s more code to write every time.

sasajuric

sasajuric

Author of Elixir In Action

This is a bit annoying but I’m not particularly worried about it, because there’s no global default, so failing to set the repo will raise an exception, and this should be caught by tests.

zamith

zamith OP

How do you ensure that there is no global default? At the supervisor level?

sasajuric

sasajuric

Author of Elixir In Action

We set name: nil repo option, and yeah there’s no global tenant repo instance running. The set of current tenants are read from some place (in the current draft it’s the “main” database powered by a different repo which is global) and corresponding instances are started during the boot, but they are nameless so they won’t be used by default.

zamith

zamith OP

Right, that was what I was thinking. Do have any “smart” way of handling migrations so that you can have tenant only migrations, and “main” database migrations that run separately?

LostKobrakai

LostKobrakai

That sounds like something you could have a look at Triplex for inspiration, which handles multi-tenancy using schemas.

sasajuric

sasajuric

Author of Elixir In Action

We have some helper functions, basically wrappers around Ecto.Migrator, which first migrate the main db, then all known tenants. It’s hard to say yet if that will suffice for production, but they seem to be good enough for local dev. We defined a couple of mix aliases, such as “ecto.reset” and “ecto.migrate” to make it fit with the flow of other projects.

For tests, we precreate a single tenant db in test_helper.exs (or migrate it if the db already exists). That way, most of the tests can work on that single tenant db, and so they can run in a sandbox with async: true. There is one test module which tests dynamic addition and removal of repos, and that one has to be async: false b/c afaik db creation & migration can’t run on a sandbox.

There seems to be enough material here for a blog post, I only need to find some time to write it up :slight_smile:

zamith

zamith OP

When testing, where do you clean up that tenant db, since you don’t have the on_exit callback available on test_helper.exs, or am I missing something?

You can just leave it there, is that the idea?

sasajuric

sasajuric

Author of Elixir In Action

Yeah, we leave it there, which is actually good b/c we don’t have to recreate it on every test run.

zamith

zamith OP

How do you manage the connections to the DB with this approach? Each dynamic repo will create it’s own connection pool, so the connections per tenant will grow with the number of tenants. If those tenants are databases in the same database server, this can become an issue.

Do you close the connections after each call? Is there a smarter way to manage these connections?

Where Next? Top

Trending in Questions Top

katta
I having some trouble figuring out if I have set myself too strict of standards for my production server. Currently I can handle 75% of r...
New
achenet
Hello, I’m trying to build a basic Phoenix web-app, and I’d like to use Tailwind. However, when I launch mix phx.server, I get an error...
New
Cxx-mlr
I’m working on a small exercise involving update_in/3, and I came up with this solution: data = %{ name: "Periodic Table", category:...
New
ChrisAmelia
I’ve got trouble wrapping my head around the order in which functions are called in this snippet (from Phoenix’s authentication): toke...
New
dillonoconnor
Is there any way to avoid the Hologram compiler running when using iex? It seems like the front-end code could potentially be disregarded...
New
thiagogsr
** (ArgumentError) expected :max_attempts to be a positive integer, got: {:@, [line: 10, column: 19], [{:max_attempts, [line: 10, column:...
New
unaware8150
Hello folks! So at work, we are seeing some situations where we have to define some “fixed” strings that are used across the codebase in...
New

Other Trending Topics Top

GenericJam
Edit: 2026 May 15 - This post is archived. Mob is alive!! Main docs: mob v0.7.11 — Documentation A bit of explanation for the slightly c...
New
mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
garrison
Hobbes is a low-level distributed database for the Elixir programming language. Hobbes provides a simple, safe, and scalable storage lay...
New
budgie
A little off-topic, but I feel like people here have a good head on their shoulders. I used to be quite good at making software. Was luc...
New
KristerV
Hey. Is there anyone here who creates agents in their apps? Not talking about using agents, but creating them. I’m finding it pretty diff...
New
mcass19
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews