zamith
Hi,
I’m implementing a system that implements multitenancy via multiple databases. For business reasons I cannot do away with prefixes, which I’d prefer. In any case, I was able to make it work by using put_dynamic_repo, but am concerned it might be a bit too “magic”.
I’m doing this with a Phoenix app, so I set the repo on the request process, any sub process will have to set the repo again, which is easy to forget.
I’ve been pointed out to an alternative that is to wrap the Ecto API and require a pid for every call, which seems more obvious but both more labor intensive as well as brittle, since any change in the API breaks the code.
I’m very torn in terms of what’s the best option (and maybe there’s even another that I’m missing). Any advice would be great.
Here’s what I’m using to manage the repos, note that the tenants are agencies and I’m using a GenServer to cache the dynamic repos that were created. Also have a function to create a new database and run the migrations if needed.
defmodule MyApp.RepoManager do
use GenServer
alias MyApp.{
Application,
Repo
}
def start_link(settings) when is_list(settings) do
GenServer.start_link(__MODULE__, settings, name: __MODULE__)
end
def set_agency_repo(agency, ensure_exists \\ false) do
if ensure_exists do
ensure_repo_exists(agency)
end
repo_pid = GenServer.call(__MODULE__, {:get_dynamic_repo, agency})
Repo.put_dynamic_repo(repo_pid)
{:ok, repo_pid}
end
def unset_agency_repo do
repo_pid = Repo.put_dynamic_repo(Repo)
{:ok, repo_pid}
end
def init(_opts) do
{:ok, %{repos: %{}}}
end
def handle_call({:get_dynamic_repo, agency}, _from, state) do
case state.repos[get_database_name(agency)] do
nil ->
{:ok, repo_pid} = Repo.start_link(get_connection_options(agency))
{:reply, repo_pid,
%{state | repos: Map.put_new(state.repos, get_database_name(agency), repo_pid)}}
repo_pid ->
{:reply, repo_pid, state}
end
end
defp get_database_name(agency) do
"my_app_#{agency.slug}"
end
defp get_connection_options(agency) do
[
name: nil,
pool_size: 2,
database: get_database_name(agency)
] ++ Application.db_config()
end
defp ensure_repo_exists(agency) do
options = get_connection_options(agency)
Repo.__adapter__().storage_up(options)
{:ok, repo_pid} = Repo.start_link(options)
Repo.put_dynamic_repo(repo_pid)
Ecto.Migrator.run(Repo, :up, all: true, dynamic_repo: repo_pid)
Repo.stop(1000)
Repo.put_dynamic_repo(Repo)
end
# TODO: Manage pool size better
# TODO: Support removing repos
end
Trending in Questions
Other Trending Topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ai
- #ecto-query
- #elixirconf-us
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #elixirconf-eu
- #api
- #forms
- #metaprogramming
- #hex










Showing Posts 11 to 20- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
zamith
Exactly, that was my thought process. My main concern is the fact that any sub process does not use the dynamic_repo. That’s why the wrapper you mention might be a good option, even though it’s more code to write every time.
sasajuric
This is a bit annoying but I’m not particularly worried about it, because there’s no global default, so failing to set the repo will raise an exception, and this should be caught by tests.
zamith
How do you ensure that there is no global default? At the supervisor level?
sasajuric
We set
name: nilrepo option, and yeah there’s no global tenant repo instance running. The set of current tenants are read from some place (in the current draft it’s the “main” database powered by a different repo which is global) and corresponding instances are started during the boot, but they are nameless so they won’t be used by default.zamith
Right, that was what I was thinking. Do have any “smart” way of handling migrations so that you can have tenant only migrations, and “main” database migrations that run separately?
LostKobrakai
That sounds like something you could have a look at Triplex for inspiration, which handles multi-tenancy using schemas.
sasajuric
We have some helper functions, basically wrappers around Ecto.Migrator, which first migrate the main db, then all known tenants. It’s hard to say yet if that will suffice for production, but they seem to be good enough for local dev. We defined a couple of mix aliases, such as “ecto.reset” and “ecto.migrate” to make it fit with the flow of other projects.
For tests, we precreate a single tenant db in test_helper.exs (or migrate it if the db already exists). That way, most of the tests can work on that single tenant db, and so they can run in a sandbox with
async: true. There is one test module which tests dynamic addition and removal of repos, and that one has to beasync: falseb/c afaik db creation & migration can’t run on a sandbox.There seems to be enough material here for a blog post, I only need to find some time to write it up
zamith
When testing, where do you clean up that tenant db, since you don’t have the
on_exitcallback available ontest_helper.exs, or am I missing something?You can just leave it there, is that the idea?
sasajuric
Yeah, we leave it there, which is actually good b/c we don’t have to recreate it on every test run.
zamith
How do you manage the connections to the DB with this approach? Each dynamic repo will create it’s own connection pool, so the connections per tenant will grow with the number of tenants. If those tenants are databases in the same database server, this can become an issue.
Do you close the connections after each call? Is there a smarter way to manage these connections?