Arch Question: Tired of passing this same parameter around for every method call

Situation

I’m working on a multi-tenant library for Ecto. Ecto structs and queries let you set a prefix in the meta area, which is awesome. But it’s getting really verbose to send every query and changeset through a set_prefix() method before forwarding it along to the Repo.

Thoughts / Ideas

  1. This may simply be the elixir/functional way. After all, less magic is good, and not hiding your function inputs is good too. :slight_smile:

  2. Currying - Currying isn’t quite what I want because I’m currying a module, not a function. (Also, I’m not changing the number of parameters required)

  3. Wrapping - I have a repo wrapper that verifies that a prefix was set on relevant Ecto structs. This is easy because I’m just verifying that SOMETHING was set, which means I can build it at compile time with macros. But what I’d really like to be able to do is:

    repo = Repo.prefixify(123)
    user = User.changeset(%User{}, %{name: “joe”})
    repo.insert(user)

and have it sent to the 123 prefix. Automatically. Then later,

loaded_user = repo.get!(User, user.id)

and have it know to pull from prefix 123.

So repo itself is a module.

Agents seem like the closest strategy here. However, it seems like with an agent, you have to keep track of your agent’s pid. Then you call your regular module, just including the pid as an argument. But at this point, all I’ve done is traded passing in a prefix for a pid.

Is there any way to pass this in once, and be done with it? If not, no worries. I just don’t want to let my unfamiliarity with the language limit the easy of use of my library.

1 Like

Nov 2, 2016 Update

So indeed my tests were where I was feeling the most pain.

I worked out a solution in 2 parts for that.

  1. I’m running a test_seeds.exs script at the beginning of all of my specs to do standard setup. That’s reduced general duplication quite a bit, and sped the specs up tremendously as well (9 sec -> .8, because the specs were actually creating tenants)
  2. Where I am creating extra setup data, I was unable to use the Ecto strategy for ex_machina, because I couldn’t set the prefix on those structs. So I created a PR for ex_machina (ExMachine PR) that kinda feels like ‘traits’ from factory girl.

When I get back to this project, I’ll look into @josevalim’s insight about Ecto 2.1, and its ability to accept the prefix as a keyword to Repo operations. That is probably the ‘good enough’ solution right there!

Oct 31, 2016 Update

Update for future readers. I realized that the place I was noticing all of this painful duplication was in my specs. But my specs are usually only using a single prefix, and part of the struggle was with ex_machina needing a new (prefix) parameter that hosed its lovely strategy pattern. So the compromise solution I’m going with now is to create a test-only import file that will wrap ex_machina, letting me pass in a tenant that is defaulted to a ‘test’ tenant. And then also wrappers around the Repo that do the same. So far (this is a work in process), I’m not planning to use the wrappers in my production code, because my methods are short, and so I’m comfortable seeing the specification of the tenant. Hopefully, once this is is all complete, I’ll bake it in to the library docs.

That said, if there’s a solution to my original question that I’ve missed, I’d still really like to know!

1 Like

You could use the pipe operqator |> and instead of:

do

repo = Repo.prefixify(123)
%User{}
|> User.changeset(%{name: "joe"})
|> repo.insert

Also if the module where you to this is User specific you might consider import User
EDIT: Expanded original answer
Regarding the prefix: How about using something like this:

defmodule User do
  defmacro __using__(prefix) do
    quote do
      use Ecto.Schema
      @schema_prefix prefix || Application.get_env(:ex_machina, :prefix, "public")
      #... all the other user stuff
  end
  end
end

and then

defmodule MyPrefixUser do
  use User :my_prefix
end
1 Like

One option is to use the process dictionary along side custom functions in the repository. So you would do:

defmodule MyApp.Repo do
  use Ecto.Repo, otp_app: :my_app

  def put_prefix(prefix) do
    Process.put({MyApp.Repo, :prefix}, prefix)
  end

  def tenant_get(query, opts) do
     get(query, Keyword.put_new(opts, :prefix, get_prefix())
  end

  defp get_prefix do
     Process.get({MyApp.Repo, :prefix}) || raise "no Repo prefix set"
  end
end

And then:

MyApp.Repo.tenant_get(User, id)

Keep in mind I am using Ecto 2.1 (currently out as a release candidate) ability to pass the prefix as an option to all Repo operations.

4 Likes

Thank you for the detailed writeup, @pba. I wasn’t as clear as I should have been on what I was trying to achieve. The problem I’m struggling with was how to write the function Repo.prefixify(123), such that it returned a version of the Repo that would apply the given prefix to either the query or changeset appropriately. The problem is that the prefix will change based on each request. I’m using postgresql’s schemas to isolate client data. As a result, I can’t set the prefix in a config. Poor explanation on my part. Thank you for the input!

1 Like

@josevalim Great to know about 2.1! To be honest, that might just represent a solid solution right there.

The process solution seems ideal, depending on how processes work in Phoenix. Does each request get its own process? (ie: Could I write a plug that would call MyApp.Repo.put_prefix, and not have to worry that that setting could impact concurrent processes? I would assume there’s a pool of these. Is there a way to ensure I clear that setting before checking it back into the pool? (In case of an exception)

Each request gets its own process. The Erlang scheduler ensures nothing shared between processes. There is no pool of processes (rather a pool of memory, it is low level stuff, just assume erlang does things right, because it does ^.^), no need to clear it unless you want it cleared for your current request for some reason. :slight_smile:

1 Like

I’d say we missed the obvious solution here - anonymous functions.

If we were talking about a single function instead of whole repo - this would be obvious with using partial application, wouldn’t it?

prefixed_insert = &Repo.insert(&1, prefix: "foo")
prefixed_insert.(my_data)

So can we do something similar for a module? We need to do some changes, mainly because now we need to decide at runtime which function to call, fortunately we can use apply/3.

def prefixify(prefix) do
  fn fun, args ->
    [opts | rest] = Enum.reverse(args)
    apply(Repo, fun, Enum.reverse(rest, [Keyword.put_new(opts, :prefix, prefix)])
  end
end

This allows us to call:

prefixed = Repo.prefixify("foo")
prefixed.(:insert, [my_data, []])
prefixed.(:all, [some_query, []])

It’s a bit different than the original, but achieves the goal. Is it worth it and should be done? That’s a completely different question :wink:

3 Likes

Awesome! Great to know more about the innards there. Thank you, @OvermindDL1!

1 Like

prefixed_insert = &Repo.insert(&1, prefix: "foo") <-- :heart_eyes:

I didn’t know that was possible. Very slick. I think the drawback here (using apply on the entire module) is that you lose the use of the |> operator.

I think Ecto 2.1’s new method signature is likely to be a good enough solution here. Unfortunately, it looks like my ex_machina PR (https://github.com/thoughtbot/ex_machina/pull/176) is going to be rejected. Not quite sure how I’ll work around that.