zamith

zamith

Hi,

I’m implementing a system that implements multitenancy via multiple databases. For business reasons I cannot do away with prefixes, which I’d prefer. In any case, I was able to make it work by using put_dynamic_repo, but am concerned it might be a bit too “magic”.

I’m doing this with a Phoenix app, so I set the repo on the request process, any sub process will have to set the repo again, which is easy to forget.

I’ve been pointed out to an alternative that is to wrap the Ecto API and require a pid for every call, which seems more obvious but both more labor intensive as well as brittle, since any change in the API breaks the code.

I’m very torn in terms of what’s the best option (and maybe there’s even another that I’m missing). Any advice would be great.

Here’s what I’m using to manage the repos, note that the tenants are agencies and I’m using a GenServer to cache the dynamic repos that were created. Also have a function to create a new database and run the migrations if needed.

defmodule MyApp.RepoManager do
  use GenServer

  alias MyApp.{
    Application,
    Repo
  }

  def start_link(settings) when is_list(settings) do
    GenServer.start_link(__MODULE__, settings, name: __MODULE__)
  end

  def set_agency_repo(agency, ensure_exists \\ false) do
    if ensure_exists do
      ensure_repo_exists(agency)
    end

    repo_pid = GenServer.call(__MODULE__, {:get_dynamic_repo, agency})
    Repo.put_dynamic_repo(repo_pid)

    {:ok, repo_pid}
  end

  def unset_agency_repo do
    repo_pid = Repo.put_dynamic_repo(Repo)

    {:ok, repo_pid}
  end

  def init(_opts) do
    {:ok, %{repos: %{}}}
  end

  def handle_call({:get_dynamic_repo, agency}, _from, state) do
    case state.repos[get_database_name(agency)] do
      nil ->
        {:ok, repo_pid} = Repo.start_link(get_connection_options(agency))

        {:reply, repo_pid,
         %{state | repos: Map.put_new(state.repos, get_database_name(agency), repo_pid)}}

      repo_pid ->
        {:reply, repo_pid, state}
    end
  end

  defp get_database_name(agency) do
    "my_app_#{agency.slug}"
  end

  defp get_connection_options(agency) do
    [
      name: nil,
      pool_size: 2,
      database: get_database_name(agency)
    ] ++ Application.db_config()
  end

  defp ensure_repo_exists(agency) do
    options = get_connection_options(agency)
    Repo.__adapter__().storage_up(options)
    {:ok, repo_pid} = Repo.start_link(options)
    Repo.put_dynamic_repo(repo_pid)
    Ecto.Migrator.run(Repo, :up, all: true, dynamic_repo: repo_pid)

    Repo.stop(1000)
    Repo.put_dynamic_repo(Repo)
  end

  # TODO: Manage pool size better
  # TODO: Support removing repos
end

Showing Posts 22 to 13

DidactMacros

DidactMacros

apoorv-2204

apoorv-2204

where can i find the beginners guide to this>?

zamith

zamith OP

How do you manage the connections to the DB with this approach? Each dynamic repo will create it’s own connection pool, so the connections per tenant will grow with the number of tenants. If those tenants are databases in the same database server, this can become an issue.

Do you close the connections after each call? Is there a smarter way to manage these connections?

sasajuric

sasajuric

Author of Elixir In Action

Yeah, we leave it there, which is actually good b/c we don’t have to recreate it on every test run.

zamith

zamith OP

When testing, where do you clean up that tenant db, since you don’t have the on_exit callback available on test_helper.exs, or am I missing something?

You can just leave it there, is that the idea?

sasajuric

sasajuric

Author of Elixir In Action

We have some helper functions, basically wrappers around Ecto.Migrator, which first migrate the main db, then all known tenants. It’s hard to say yet if that will suffice for production, but they seem to be good enough for local dev. We defined a couple of mix aliases, such as “ecto.reset” and “ecto.migrate” to make it fit with the flow of other projects.

For tests, we precreate a single tenant db in test_helper.exs (or migrate it if the db already exists). That way, most of the tests can work on that single tenant db, and so they can run in a sandbox with async: true. There is one test module which tests dynamic addition and removal of repos, and that one has to be async: false b/c afaik db creation & migration can’t run on a sandbox.

There seems to be enough material here for a blog post, I only need to find some time to write it up :slight_smile:

LostKobrakai

LostKobrakai

That sounds like something you could have a look at Triplex for inspiration, which handles multi-tenancy using schemas.

zamith

zamith OP

Right, that was what I was thinking. Do have any “smart” way of handling migrations so that you can have tenant only migrations, and “main” database migrations that run separately?

sasajuric

sasajuric

Author of Elixir In Action

We set name: nil repo option, and yeah there’s no global tenant repo instance running. The set of current tenants are read from some place (in the current draft it’s the “main” database powered by a different repo which is global) and corresponding instances are started during the boot, but they are nameless so they won’t be used by default.

zamith

zamith OP

How do you ensure that there is no global default? At the supervisor level?

Where Next? Top

Trending in Questions Top

stjefim
Hello! Suppose you are building workflow (order / task / payment) processing system with the following requirements: Each workflow con...
New
water
I’m aware of 2 such projects, are there others? git: GitHub - team-alembic/realworld: A fullstack Phoenix LiveView application with back...
New
itzmidinesh
Hi all, I’m working on a Phoenix project where I need to make external HTTP requests and I’m trying to choose between Req and HTTPoison....
New
peck
Hi everyone. My team and I have been working on a fairly modest app based around video streaming and chat, but we’ve landed a customer t...
New
leoschweizer
Hi, we run different LLM models across a cluster, with each node loading a configurable subset. Model-specific queues let nodes consume ...
New
csmr
Hi all, I’m new here and would like to hear peoples views on my Elixir/Phoenix based game-engine! :smiley: Mutonex gamedesign was finish...
New
perzanko
Do we have maintainer of absinthe-graphql/dataloader here? We have a PR that we’d like to merge to the library. Relatively small additive...
New

Other Trending Topics Top

dominicletz
Hey there, I haven’t seen a list of useful Elixir specific skills for the popular AI tools yet so I’m starting one here with my first. ...
New
mudasobwa
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
KristerV
Hey. Is there anyone here who creates agents in their apps? Not talking about using agents, but creating them. I’m finding it pretty diff...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
webofbits
With AI doing more of the implementation work, I’ve been wondering how much coding I should deliberately keep doing myself. My main conc...
#ai
New
OndrejValenta
(I just needed to vent somewhere and LinkedIn is full of hope, or hype, I’m not sure which exactly) AI dream has many faces, but general...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews