Iteratively calling "use" on a list of modules?

I have this situation where I have a list of modules in a module attribute to be used for another purpose and I also want to “use” them, so I thought it would be nice to have a single place in the module to enable/disable their functionality. Unfortunately I have been unable to find a way to make it work, but it doesn’t provide me any errors.

Is there any way to make this work as I expect?

defmodule Example do

@mods [
  Example.One
  Example.Two
]

def something() do
  # When called, enumerate over @mods here for another purpose
end

# this doesn't work
for mod <- @mods, do: quote do: use unquote(mod)

end

It doesn’t error, but it definitely is not calling the __using__ of that module. Explicitly setting the use works, even if you set it after this, so it’s definitely not working.

Any thoughts?

for mod <- @mods, do: quote do: use unquote(mod)

is equivalent to

for mod <- @mods do 
  quote do 
    use unquote(mod)
  end
end

Note that the quote do there just causes the AST A.K.A “quoted code” to be returned. Nothing is actually evaluating it.

Try this (haven’t verified that it works). In a different module, make a macro called use_all.

defmodule SomeModule do
  # explicitly expects a module attribute to be given to it
  defmacro use_all({:@, _, [{name, _, _}]}) do
      for module <- Module.get_attribute(__CALLER__.module, name, []) do
        quote do
           use unquote(module)
        end
      end
    end
  end
end

Then in your module you can use

SomeModule.use_all(@attr)

There is a simpler way that is perhaps a bit less hygienic as its typically recommended to avoid Code.eval_quoted, but you could do this:

for mod <- @mods do
  quote do
    use unquote(mod)
  end
  |> Code.eval_quoted([], __ENV__)
end

Thanks! I knew I was on the right track and I was staring at the source code for use in the Kernel but hadn’t remembered that the AST I was generating wasn’t being evaluated…

I haven’t tested the more complex solution but I’ll give it a spin tomorrow and report back for future readers.

Honestly I think it may just be the right way. I’m not sure why I was thinking one should avoid Code.eval_quoted, but I think that usage of it is perfectly reasonable.

I can’t get this one to work. Isn’t this going to result in each use being scoped to for’s block? I have a poor mental model of using quote directly in a module body, though I was trying other stuff earlier (somewhat akin to your first answer, though not quite) and no luck there either :cold_sweat: I actually can’t get the first example working either, says that @mods is unused.

Oh that would be funny if I found a bug in the compiler or something. I’m already using that @mods equivalent in another function so it wouldn’t error as unused. See if that solves it for you?

You’re right :slight_smile:

Here is a working example:

defmodule Foo do
  defmacro __using__(_) do
    quote do
      import Integer
    end
  end
end

defmodule MultiUse do
  # must be a literal list
  defmacro __using__(mods) do
    using_code =
      for module <- Macro.expand_literals(mods, __CALLER__) do
        quote do
          use unquote(module)
        end
      end

    setter = quote do
      @using_modules unquote(mods)
    end

    [setter | using_code]
  end
end

defmodule Bar do
  use MultiUse, [Foo]
  @using_modules |> IO.inspect()
  digits(12)
end

This demonstrates that you can use the list again later w/ @using_modules, and things like import affect the top level scope.

Yes, sorry, I shouldn’t have focused on the warning. It doesn’t accomplish the use for me is the main thing. I could be doing it wrong though, it’s getting a bit late for me.

Oh, this is clever! This code (without the @using_modules part) is very close to what I had working with the problem being that it failed passing it a module attribute. Instead of expand_literals, though, I used expand_once. Is there a big difference here? I never quite understood the docs of expand_literals though it actually sounds like might be equivalent in this case? Both have to do with expanding aliases properly?

I think they’re equivalent here, yeah.

1 Like

Thanks! I ask because this is one of those things I thought I knew the answer to then 30 mins later I still going :upside_down_face:

This one works, but not if the list you’re passing is a module attribute like:

@mods [Foo]
use MultiUse, @mods

In that case it errors like this:

** (Protocol.UndefinedError) protocol Enumerable not implemented for {:@, [line: 12, column: 28], [{:mods, [line: 12, column: 29], nil}]} of type Tuple

That’s why he set the module attribute within the macro.

The problem is that use, as well as import and alias, are scoped to the block they are called in, so you have to get around that somehow and it’s a little tricky.

For example:

defmodule Foo do
  if true do
    import String
    capitalize("foo") # all good!
  end

  capitalize("foo") # Error: `capitalize` doesn't exists
end
2 Likes

But I need the module attribute in the other module. Let me just lay my cards out here, this is for Telemetry/Logging. The goal is to have a telemetry module that receives events and turns them into log messages. I don’t want to end up with a 2000 line file or be forced to have multiple modules duplicating the same code

So in start/2 of application.ex:

# register the handlers
Project.Telemetry.Logs.setup()

and then in the module that wires everything up:

defmodule Project.Telemetry.Logs do
  @moduledoc "Transforms telemetry events to logs"

  require Logger

  @loggers     [
      Project.Telemetry.Logs.FileStore,
      Project.Telemetry.Logs.Oban,
      Project.Telemetry.Logs.Tesla
    ]

  # I need to have a use here that can "use" each of those @loggers modules
  # which will inject their handle_event/4 functions into this file

  def setup do
    events =
      Enum.reduce(@loggers, [], fn logger, acc ->
        acc ++ logger.events()
      end)

    Enum.each(events, fn event ->
      :telemetry.attach({__MODULE__, event}, event, &__MODULE__.handle_event/4, [])
    end)
  end

  # Catchall
  def handle_event(event, measurements, metadata, _) do
    Logger.error(fn ->
      "Unhandled telemetry event #{inspect(event)} with measurements #{inspect(measurements)} and metadata #{inspect(metadata)}"
    end)

    :ok
  end
end

example logger module:

defmodule Project.Telemetry.Logs.Oban do
  @moduledoc false

  @events [
    [:oban, :job, :exception]
  ]

  def events(), do: @events

  defmacro __using__(_opts) do
    quote do
      def handle_event([:oban, :job, :exception], _measure, meta, _) do
        Logger.error(fn ->
          "[Oban] #{meta.worker} error: #{inspect(meta.error)} args: #{inspect(meta.args)}"
        end)
      end
    end
  end
end

Zach’s solution does this, though, it just does it sneakily. You could change @using_modules to @loggers in MultiUse.__using__ then do:

def Example do
  use MultiUse, [Foo]
end

and this will create a @loggers attribute will be available in Example. You could even change MutiUse to set the module name, like use MultiUse, {[Foo, Bar], set_attribute: :logger}) or something. This is why it’s tricky.

My feeling is that there is a better way to do this that still accomplishes your end goal, especially since calling use, import, alias in loops isn’t something that comes up often at all. I’ve never faced it myself in 5+ years of Elixir. I’d have to think on it myself or perhaps someone who has more experience with :telemetry can advise.

Oh I see it now, I just needed a second cup of coffee. :coffee:

So I could make the canonical list of modules the one I pass to use MultiUse and it injects the module attribute back into the module for me to use.

I might be okay with that but it’s obscure enough that I’d have to leave breadcrumbs for other people working on this codebase :slight_smile:

Another thought I had this morning was to just loop through a list modules in the application.ex setup/2, perhaps read via Application.get_env, and make that the source of truth for which telemetry-derived logs are enabled.

edit: yeah, I can make it work this way by putting the use at the bottom of each module that registers for telemetry events so the catchall will still work. This way I’m not fighting with the compiler to do something it’s not meant to do.

I’m going to leave this as the solution: “Just don’t try this. It’s not worth it.”

2 Likes

Maybe this solution was not the right one, but there might be others. Maybe we can find one in another direction?

Edit: I’m on mobile, @sodapopcan is in meeting. Frustrating hours :sweat_smile:

1 Like

Personally, I agree that “how to call use in a loop” is “just not worth it.” There are maybe some situations in libraries where it could be? I dunno, I just wouldn’t ever suggest doing it. But I’m actually quite interested in whatever the solution to OP’s problem ends up being—that is if there is a clean way to not have to maintain two lists of the same modules. I assume this would involve registering the event handlers in a different way (but I’ve given it no thought, I’m responding here in between meetings).