victorolinasc

victorolinasc

Library configuration - multi-instance

Hello people!

I am about to write a “SDK” for a service. Before I got started, I was looking for some general patterns and best practices.

One thing I noticed is that most of them don’t worry about multiple “instances”. Almost all of them that I’ve used or have seen are using the single instance approach. By that I mean: they have a config like:

# Configuration
config :my_library, client_id: id, client_key: key

defmodule MyApplication do
  # possibly add MyLibrary to the application supervision
end

# Then on application code
MyLibrary.make_some_call(params)

Examples: :ex_aws, :ex_twilio, :sendgrid, etc…

Ecto uses a different approach that facilitates the multiple instances approach but it makes it a bit more verbose to configure it.

# Configuration
config :my_app, MyRepo1, options

config :my_app, MyRepo2, other_options

defmodule MyRepo1 do
  use Ecto.Repo, opts # where otp_app: :my_app
end

defmodule MyRepo2 do
  use Ecto.Repo, opts # where otp_app: :my_app
end

defmodule MyApplication do
  # add MyRepo1 and MyRepo2 to the application supervision
end

# Then on application code
some_query |> MyRepo1.all()
some_query |> MyRepo2.all()

With the first approach, I don’t think there is a good way to have two instances of the library running at the same time. With the second approach we end up needing some macros…

I would like to hear people’s opinion on this:

  1. Should we, as library authors, strive to always provide a multi-instance configuration?
  2. Are there other ways to do this cleanly? Even if that demands Elixir 1.9+…

Cheers!

Most Liked

keathley

keathley

Generally, libraries shouldn’t rely on application env at all. A lot of libraries continue to use that pattern because it was popular in the early days of Elixir. But its a bad model and shouldn’t be copied. If your library needs to use processes, then users should be able to pass in arguments when they start the process in their supervisor. Redix is an excellent example of how to do this well.

There are exceptions to this rule. Sometimes it makes the most sense to provide a full-blown OTP Application. I typically do this because there’s nothing in the application to configure, the supervision hierarchy is complex, or if there’s no benefit to the user managing the lifecycle of the process.

Building a library that is re-usable without collisions is often much more difficult. But I think it pays off in the long run.

Last Post!

keathley

keathley

Let me see if I can clarify this a bit. If we have this code:

config :my_lib, MyRepo,
  adapter: Sandbox # value is determined at compile time

defmodule MyRepo do
  defp adapter do
    # Even if next line is called during app boot, it uses a compile time value
    Application.get_env(:my_lib, MyRepo)[:adapter]
  end
end

The adapter isn’t actually being “compiled” into the MyRepo module. Its always being fetched from the application env. That means its possible to do something like this:

# The adapter is set from when `config.exs` was initially run
Sandbox = MyRepo.adapter()

# Update the config...
Application.put_env(:my_lib, MyRepo, [adapter: DBAdapter])

# Now the adapter has changed
DBAdapter = MyRepo.adapter()

Its possible to compile the adapter directly into the module doing something like this:

defmodule Repo do
  defmacro __using__(_opts) do
    quote do
      @before_compile unquote(__MODULE__)
    end
  end

  defmacro __before_compile__(env) do
    a = Application.get_env(:my_lib, Repo)[:adapter]

    quote do
      def adapter do
        unquote(a)
      end
    end
  end
end

defmodule MyRepo do
  use Repo
end

This will cause the adapter function to be hard-coded with the adapter module into MyRepo.

Hopefully that provides some clarity :).

Everything else seems spot on to me :+1:.

In my applications and services I’ll often use this pattern:

defmodule Server do
  def start_link(opts) do
    GenServer.start_link(__MODULE__, state, name: opts[:name] || __MODULE__)
  end

  def foo(server \\ __MODULE__, args) do
    GenServer.cast(server, {:foo, args})
  end
end

This gives me a lot of flexibility when testing the server and provides reuse. But, I don’t tend to use this pattern in libraries because it can create name conflicts if 2 different users try to use the default name. Passing the name around can definitely be annoying, but I haven’t found a better pattern that still provides the same level of flexibility. At this point I’m used to passing the name or a pid as the first argument to my function calls.

Likewise :slight_smile:

Where Next?

Popular in Discussions Top

MarioFlach
Hello, I want to share a project I’ve been working on for a while: https://github.com/almightycouch/gitgud Background Some time ago I ...
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New
gausby
I asked this very same question on twitter and got some interesting feedback, but I thought it would be a good question to ask here as we...
1207 40082 209
New
lucaong
Hello Elixir and Nerves community, I have been working for a while on an open-source embedded key-value database for Elixir, that I call...
230 14362 124
New
Fl4m3Ph03n1x
Background A few days ago I was listening to The future of Elixir from Elixir Talks, with Dave Thomas (@pragdave ) and Brian Mitchell. I...
New
nunobernardes99
Hi there Elixir friends :vulcan_salute: In a recent task I was on, I needed to check in two dates which of them is the maximum and which...
New
arcanemachine
https://nitter.net/josevalim/status/1744395345872683471 https://twitter.com/josevalim/status/1744395345872683471
New

Other popular topics Top

JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New

We're in Beta

About us Mission Statement