Pass argument to use statement

Hi All,
I’m using a library (rabbit_mq) which has some arguments that i’d like to pass in via the child spec if possible or some other way vs being hardcoded in the source file. Thanks for any and all help.

use RabbitMQ.Producer, exchange: “example_queue”, worker_count: 3

The library has the following:

defmacro __using__(opts) do
    quote do
      alias RabbitMQ.Producer

      require Logger

      @confirm_type unquote(Keyword.get(opts, :confirm_type, :async))
      @exchange unquote(Keyword.fetch!(opts, :exchange))
      @worker_count unquote(Keyword.get(opts, :worker_count, 3))
      @this_module __MODULE__

      @behaviour Producer

      ##############
      # Public API #
      ##############

      def child_spec(opts) do
        if @worker_count > max_workers() do
          raise """
          Cannot start #{@worker_count} workers, maximum is #{max_workers()}.
          You can configure this value as shown below;
            config :rabbit_mq, max_channels_per_connection: 16
          As a rule of thumb, most applications can use a single digit number of channels per connection.
          For details, please consult the official RabbitMQ docs: https://www.rabbitmq.com/channels.html#channel-max.
          """
        end

        config = %{
          confirm_type: @confirm_type,
          exchange: @exchange,
          nack_cb: &on_publisher_nack/1,
          worker_count: @worker_count
        }

....

It is not hardcoded. What you are seeing is a convenience code that gets evaluated with the values you are passing to the use statement. Your worker_count parameter will be honoured.

Yes, those module attributes will get injected in the using module. That’s a fairly normal pattern for libraries utilising metaprogramming and that’s why they usually recommend you put the use statement in a dedicated single-responsibility module. Ecto.Repo does this as well.

Do not get worried about implementation details. Does the library work for your scenario?

Forgot to answer your question:

Yes, the library works great. First library i found that handles rabbitmq messages reliably – e.g. when the rabbitmq broker goes down, it maintains a list of messages it has received and will send them after reconnect.

I was looking into the implementation to see a way around passing in the params differently. My way of trying to understand the language/platform. This is my first elixir project.

Thank you for the response. I think i’ve poorly stated my goal. I understand i can change those parameters, such as worker_count, but that would still reside inside my .ex file that contains the use statement.

Since I want to access more than one queue, I would need to have two separate source files with the exchange param set individually (that’s what i meant by hardcoded). My goal would be to have one source file launched twice from two entries in the supervisor child spec passing in these parameters vs having two copies of the same source file where the only difference is the parameter for the use statement.

Hopefully that makes more sense. I appreciate the help.

You can have one file with one module and two submodules inside then?

Something like this? Separates configuration from generation:

defmodule Queue do
  def configuration do
    %{
      My.Queue_1  => [exchange: "example_queue", worker_count: 3],
      My.Queue_2  => [exchange: "other_queue", worker_count: 2]
    }
  end
end

defmodule Rabbit do
  for {module, options} <- Queue.configuration() do
    defmodule module do
      use RabbitMQ.Producer, unquote(options)
    end
  end
end
1 Like

This looks like a great approach and the kind of thing i was hoping for!

The module creation seems to work but I’m getting an error for the options and not sure how to fix:

{:unquote, [line: 8], [{:options, [line: 8], nil}]}

defmodule Tmp2.QConfig do

  def configuration do

    %{
      Tmp2.Foo.Queue1  => [exchange: "example_queue", worker_count: 3],
      Tmp2.Foo.Queue2  => [exchange: "other_queue", worker_count: 2]
    }

  end
end

defmodule Tmp2.Foo do

  for {module, options} <- Tmp2.QConfig.configuration() do

    defmodule module do
        #use RabbitMQ.Producer,  exchange: "example_queue", worker_count: 3
        #use RabbitMQ.Producer, [exchange: "example_queue", worker_count: 3]
        use RabbitMQ.Producer, unquote(options)

        @impl true
        def handle_publisher_nack(unackd_messages) do
          Logger.error("Failed to publish messages: #{inspect(unackd_messages)}")
        end

    end

  end

end

Full error:

== Compilation error in file lib/tmp2.ex ==
** (FunctionClauseError) no function clause matching in Keyword.get/3

The following arguments were given to Keyword.get/3:
 
    # 1
    {:unquote, [line: 8], [{:options, [line: 8], nil}]}

    # 2
    :confirm_type

    # 3
    :async

Attempted function clauses (showing 1 out of 1):

    def get(keywords, key, default) when is_list(keywords) and is_atom(key)

(elixir 1.10.1) lib/keyword.ex:201: Keyword.get/3
expanding macro: RabbitMQ.Producer.__using__/1
lib/tmp2.ex:8: Tmp2.Foo.Queue1 (module)
(elixir 1.10.1) expanding macro: Kernel.use/2
lib/tmp2.ex:8: Tmp2.Foo.Queue1 (module)

Seems as if the library does not call Macro.expand/2 on its use arguments. Not actually sure how to proceed in that case. Perhaps the rest of the community will have some ideas (and I’ll keep thinking on it).

@Denny I believe the issue requires a PR to the rabbit_mq lib. Specifically to change the RabbitMQ.Producer:

Change from:

      @confirm_type unquote(Keyword.get(opts, :confirm_type, :async))
      @exchange unquote(Keyword.fetch!(opts, :exchange))
      @worker_count unquote(Keyword.get(opts, :worker_count, 3))

To:

      @confirm_type Keyword.get(unquote(opts), :confirm_type, :async)
      @exchange Keyword.fetch!(unquote(opts), :exchange)
      @worker_count Keyword.get(unquote(opts), :worker_count, 3)

Seems to then work. Basically the the unquote is unwrapping the wrong part of the expression.

1 Like

Ok, that fixed the errors, thank you. When i run it now, the for loop only seems to generate one module not the qty in the config (2 for now). Any ideas?

Seems like this is the issue. Thanks again for your help.

I am seeing both modules be generated:

Interactive Elixir (1.10.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> Tmp2.Foo.Queue
Queue1    Queue2

My code is:

defmodule Tmp2.QConfig do
  def configuration do
    %{
      Tmp2.Foo.Queue1  => [exchange: "example_queue", worker_count: 3],
      Tmp2.Foo.Queue2  => [exchange: "other_queue", worker_count: 2]
    }
  end
end

defmodule Tmp2.Foo do
  for {module, options} <- Tmp2.QConfig.configuration() do
    defmodule module do
      use RabbitMQ.Producer, options
      require Logger

      @impl true
      def handle_publisher_nack(unackd_messages) do
        Logger.error("Failed to publish messages: #{inspect(unackd_messages)}")
      end
    end
  end
end

Just in case we diverged somewhere…

Interesting. I have the same code but only see one module. I’m embarrassed to say that I don’t know what ‘Tmp2.Foo.Queue’ does. I use:

Enum.map(Enum.sort(Process.registered()), fn x -> IO.inspect(x) end)

and the observer to look at the running processes.

This is the list for registered processes:
Tmp2.Foo.Queue2
Tmp2.Supervisor

I appreciate all your effort, you’ve gone above and beyond. Quite impressive.

I’m running supervised, and had one Q for testing, not both. Solved! Thanks again for all your help. Very much appreciated.

Glad to hear you’re up and running. Be great if you can send a PR to the upstream lib. If not I’ll try to do so tomorrow.

I passed it along last night. Thanks again.

1 Like