zamith

zamith

Hi,

I’m implementing a system that implements multitenancy via multiple databases. For business reasons I cannot do away with prefixes, which I’d prefer. In any case, I was able to make it work by using put_dynamic_repo, but am concerned it might be a bit too “magic”.

I’m doing this with a Phoenix app, so I set the repo on the request process, any sub process will have to set the repo again, which is easy to forget.

I’ve been pointed out to an alternative that is to wrap the Ecto API and require a pid for every call, which seems more obvious but both more labor intensive as well as brittle, since any change in the API breaks the code.

I’m very torn in terms of what’s the best option (and maybe there’s even another that I’m missing). Any advice would be great.

Here’s what I’m using to manage the repos, note that the tenants are agencies and I’m using a GenServer to cache the dynamic repos that were created. Also have a function to create a new database and run the migrations if needed.

defmodule MyApp.RepoManager do
  use GenServer

  alias MyApp.{
    Application,
    Repo
  }

  def start_link(settings) when is_list(settings) do
    GenServer.start_link(__MODULE__, settings, name: __MODULE__)
  end

  def set_agency_repo(agency, ensure_exists \\ false) do
    if ensure_exists do
      ensure_repo_exists(agency)
    end

    repo_pid = GenServer.call(__MODULE__, {:get_dynamic_repo, agency})
    Repo.put_dynamic_repo(repo_pid)

    {:ok, repo_pid}
  end

  def unset_agency_repo do
    repo_pid = Repo.put_dynamic_repo(Repo)

    {:ok, repo_pid}
  end

  def init(_opts) do
    {:ok, %{repos: %{}}}
  end

  def handle_call({:get_dynamic_repo, agency}, _from, state) do
    case state.repos[get_database_name(agency)] do
      nil ->
        {:ok, repo_pid} = Repo.start_link(get_connection_options(agency))

        {:reply, repo_pid,
         %{state | repos: Map.put_new(state.repos, get_database_name(agency), repo_pid)}}

      repo_pid ->
        {:reply, repo_pid, state}
    end
  end

  defp get_database_name(agency) do
    "my_app_#{agency.slug}"
  end

  defp get_connection_options(agency) do
    [
      name: nil,
      pool_size: 2,
      database: get_database_name(agency)
    ] ++ Application.db_config()
  end

  defp ensure_repo_exists(agency) do
    options = get_connection_options(agency)
    Repo.__adapter__().storage_up(options)
    {:ok, repo_pid} = Repo.start_link(options)
    Repo.put_dynamic_repo(repo_pid)
    Ecto.Migrator.run(Repo, :up, all: true, dynamic_repo: repo_pid)

    Repo.stop(1000)
    Repo.put_dynamic_repo(Repo)
  end

  # TODO: Manage pool size better
  # TODO: Support removing repos
end

Showing Posts 1 to 10

LostKobrakai

LostKobrakai

If you immitate the Ecto.Repo behaviour, just with a pid as first parameter everywhere needed this would be just as brittle as the MyApp.Repo api. Not sure why you’d feel one is more brittle than the other.

zamith

zamith OP

What I meant was that if the Ecto internal API changes (Ecto.Adapter.Schema and the sorts), it would break my app, which is different from the public API.

But is that the approach you would recommend? What issues are there with using put_dynamic_repo that I’m missing?

LostKobrakai

LostKobrakai

Why would you need to depend on ecto internal api? The public api of ecto doesn’t change without a major version bump and your wrapper could just manage the “dynamic repo” without making callers be concerned with it.

zamith

zamith OP

Ok, let me share some code to ensure we’re talking about the same thing:

defmodule MyApp.DynamicRepo do
  def insert(pid, struct, opts \\ []) do
    Ecto.Repo.Schema.insert(pid, pid, struct, opts)
  end

  def update(pid, struct, opts \\ []) do
    Ecto.Repo.Schema.update(pid, pid, struct, opts)
  end

  def delete(pid, struct, opts \\ []) do
    Ecto.Repo.Schema.delete(pid, pid, struct, opts)
  end
end

Something like this. is Ecto.Repo.Schema considered public? Also, it means that every call to Ecto needs to use this wrapper.

Regardless of the brittleness or not, is this the preferred approach for multitenancy with multiple databases?

LostKobrakai

LostKobrakai

Nope
https://github.com/elixir-ecto/ecto/blob/5ec093becad402903774aa5c28e17fdfff9d7f2a/lib/ecto/repo/schema.ex#L4

Therefore you should rather make it so that your wrapper sets a dynamic repo on each call, like this:

def insert(pid, struct, opts \\ []) do
  current = MyApp.Repo.get_dynamic_repo()
  MyApp.Repo.put_dynamic_repo(pid)
  MyApp.Repo.insert(struct, opts)
  MyApp.Repo.put_dynamic_repo(current)
end

Dynamic repo support was added for the usecase of multiple dynamically setup repos, so yes.

kip

kip

ex_cldr Core Team

May I clarify what constitutes a database in this context? For example, a Postgres server can host multiple databases and therefore only one Repo would be necessary. But perhaps you are referring to different servers (rather than databases) which would require multiple Repos.

zamith

zamith OP

Ah, yes. It might need to run in different servers, yes.

zamith

zamith OP

Ah, I see. Why do you think this is best that put_dynamic_repo at the beginning of the request, which would affect all Ecto calls?

LostKobrakai

LostKobrakai

Because this will leave the current dynamic repo for processes intact. You could skip it if you’re not using Ecto directly anymore. Otherwise afaik this is all you can do using public api.

sasajuric

sasajuric

Author of Elixir In Action

My clients are currently doing the same things. I did an initial sketch of this, and our current draft is to use a small wrapper around put_dynamic_repo at the top-level context functions. Basically each top-level context function would have to do something like Tenant.with(tenant_id, fn -> do_something end). Given their usage patterns, perhaps we end up setting a global default from some phoenix plug or absinthe middleware, b/c a single request handler doesn’t have to deal with multiple tenants, so there should be no risk in working with a wrong db.

As an aside, there was a small bug in EctoSandbox which didn’t honor the dynamic_repo setting. The fix has been merged and I think it will be included in the upcoming Ecto 3.5.0.

Where Next? Top

Trending in Questions Top

katta
I having some trouble figuring out if I have set myself too strict of standards for my production server. Currently I can handle 75% of r...
New
peck
Hi everyone. My team and I have been working on a fairly modest app based around video streaming and chat, but we’ve landed a customer t...
New
achenet
Hello, I’m trying to build a basic Phoenix web-app, and I’d like to use Tailwind. However, when I launch mix phx.server, I get an error...
New
bradley
I really like the adapter patterns that ecto, nebulex, waffle, etc. use and would love find something similar for a key management servic...
New
unaware8150
Hello folks! So at work, we are seeing some situations where we have to define some “fixed” strings that are used across the codebase in...
New
Alvinkariuki
How Can I Optimise Compile Time Dependencies I have been building an elixir application for about 2 years now. Many modules and files ha...
New

Other Trending Topics Top

budgie
A little off-topic, but I feel like people here have a good head on their shoulders. I used to be quite good at making software. Was luc...
New
KristerV
Hey. Is there anyone here who creates agents in their apps? Not talking about using agents, but creating them. I’m finding it pretty diff...
New
mudasobwa
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
juhalehtonen
There has been a thread to discuss the Stack Overflow Developer Survey on this forum every year since 2018, so here’s yet another one for...
New
budgie
I love Elixir. It’s one of 2 programming languages I’ve ever fallen in love with. But I don’t use it anymore. Serverless was the promis...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews