Start_supervised! macro can't start worker with start_link/1

Hello, everyone

I’ve faced with the issue of running worker with start_supervised! macro in ExUnit tests, but can’t figure out what is the actual problem is and how to fix it.

Before writing code for application, I’ve prepared specific package for my own needs so that it will be shared across other modules, that gonna be used later on. The code is quite simple (full source of this worker is available here):

defmodule Spotter.Worker do
  @moduledoc """
  Base worker module that works with AMQP.
  """
  @doc """
  Create a link to worker process. Used in supervisors.
  """
  @callback start_link :: Supervisor.on_start

  @doc """
  Get queue status.
  """
  @callback status :: {:ok, %{consumer_count: integer, message_count: integer, queue: String.t()}} | {:error, String.t()}

  @doc false
  defmacro __using__(opts) do
    quote bind_quoted: [opts: opts] do
      use GenServer
      use Confex, Keyword.delete(opts, :connection)
      require Logger

      @connection Keyword.get(opts, :connection) || @module_config[:connection]
      @channel_name String.to_atom("#{__MODULE__}.Channel")

      unless @connection do
        raise "You need to implement connection module and pass it in :connection option."
      end

      def start_link() do
        GenServer.start_link(__MODULE__, %{config: config(), opts: []})
      end

      def start_link(opts) do
        GenServer.start_link(__MODULE__, %{config: config(), opts: opts})
      end

      # Other implemented methods for worker class 
      # ...
       
      defoverridable [configure: 2, validate_config!: 1]
    end
  end
end

Then, in application I written special module that defines extra functionality (sources here):

defmodule Matchmaking.AMQP.Worker do
  @moduledoc """
  Base module for implementing workers of Mathcmaking microservice
  """
  @doc false
  defmacro __using__(opts) do
    quote bind_quoted: [opts: opts] do
      use AMQP
      use Spotter.Worker,
      otp_app: :matchmaking,
      connection: Matchmaking.AMQP.Connection,
      queue: Keyword.get(opts, :queue, []),
      exchange: Keyword.get(opts, :exchange, []),
      qos: Keyword.get(opts, :qos, [])

      unless opts[:queue] do
        raise "You need to configure queue in #{__MODULE__} options."
      end

      unless opts[:queue][:name] do
        raise "You need to configure queue[:name] in #{__MODULE__} options."
      end

      # Other implemented functions are skipped here
    end
  end
end

After is this module shares the implementation across other modules and each of them provide its own logic for something with processing incoming messages from RabbitMQ queues. It looks like this:

defmodule Matchmaking.Middleware.Worker do
  @moduledoc false

  @exchange_request "open-matchmaking.direct"
  @queue_request "matchmaking.games.search"

  require Logger

  use AMQP
  use Matchmaking.AMQP.Worker.Consumer,
  queue: [
    name: @queue_request,
    routing_key: @queue_request,
    durable: true,
    passive: true
  ],
  exchange: [
    name: @exchange_request,
    type: :direct,
    durable: true,
    passive: true
  ],
  qos: [
    prefetch_count: 10
  ]

   # Do something useful with messages
   # ...
end

After it I started writing tests for modules, but stuck with issue that happen on setup stage. Running the tests with the same child specification as it used in actual run via mix run or iex -S mix commands, leads to the following error:

  1) test Middleware pushes the prepared data about the player to the next stage (MiddlewareWorkerTest)                      
     test/middleware_worker_test.exs:80                                                                                      
     ** (RuntimeError) failed to start child with the spec %{id: Matchmaking.Middleware.Worker, restart: :transient, start: {
Spotter.Worker, :start_link, [[channel_name: Matchmaking.Middleware.Worker.Channel]]}}.                                      
     Reason: an exception was raised:                                                                                        
         ** (UndefinedFunctionError) function Spotter.Worker.start_link/1 is undefined or private                            
             (spotter) Spotter.Worker.start_link([channel_name: Matchmaking.Middleware.Worker.Channel])                      
             (stdlib) supervisor.erl:379: :supervisor.do_start_child_i/3                                                     
             (stdlib) supervisor.erl:365: :supervisor.do_start_child/2                                                       
             (stdlib) supervisor.erl:671: :supervisor.handle_start_child/2                                                   
             (stdlib) supervisor.erl:420: :supervisor.handle_call/3                                                          
             (stdlib) gen_server.erl:661: :gen_server.try_handle_call/4                                                      
             (stdlib) gen_server.erl:690: :gen_server.handle_msg/6                                                           
             (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3                                                          
     stacktrace:                                                                                                             
       (ex_unit) lib/ex_unit/callbacks.ex:373: ExUnit.Callbacks.start_supervised!/2                                          
       test/middleware_worker_test.exs:63: MiddlewareWorkerTest.__ex_unit_setup_0/1                                          
       test/middleware_worker_test.exs:1: MiddlewareWorkerTest.__ex_unit__/2                                                                                                                    

In tests there’s nothing special, just running middleware worker with the same specification as for actual run (sources):

  setup_with_mocks([
    {
      MiddlewareWorker,
      [],
      [add_user_to_queue: fn(_user_id) -> {:ok, :added} end,
       get_player_statistics: fn(_channel_name, _user_id) -> {:ok, @client_data} end]
    }
  ]) do
    middleware_worker = start_supervised!(%{
      id: Matchmaking.Middleware.Worker,
      restart: :transient,
      start: {Matchmaking.Middleware.Worker, :start_link, [[channel_name: MiddlewareWorker.Channel]]}
    })

    {:ok, rabbitmq_client} = start_supervised({AmqpBlockingClient, @rabbitmq_options})
    AmqpBlockingClient.configure_channel(rabbitmq_client, @rabbitmq_options)

    {:ok, [worker: middleware_worker, client: rabbitmq_client]}
  end

Anyone has any ideas why test runner can’t invoke the start_link/1 function, although it truely exists in the module namespace (checked via Matchmaking.Middleware.Worker.module_info call in iex)?

         ** (UndefinedFunctionError) function Spotter.Worker.start_link/1 is undefined or private                            

The function it’s trying to call is Spotter.Worker.start_link which doesn’t exist at least in the code you’re showing. Can you show the actual test module?

I can’t figure out what it means, but the child spec passed to start_supervised! doesn’t match the one in the error (Matchmaking.Middleware.Worker vs Spotter.Worker in start). How does that happen?

The function it’s trying to call is Spotter.Worker.start_link which doesn’t exist at least in the code you’re showing.

I also though that is doesn’t exist or so, however module_info shows that it truely exists for Matchmaking.Middleware.Worker module and works correctly when the application started with mix run or iex -S mix commands:

iex(3)> Matchmaking.Middleware.Worker.module_info                                
[                                                                                
  module: Matchmaking.Middleware.Worker,                                         
  exports: [                                                                     
    __info__: 1,                                                                 
    ack: 2,                                                                      
    add_user_to_queue: 1,                                                        
    channel_config: 1,                                                           
    check_permissions: 2,                                                        
    configure: 2,                                                                
    consume: 4,                                                                  
    create_consumer: 2,                                                          
    get_channel: 1,                                                              
    get_endpoint: 1,                                                             
    get_player_statistics: 2,                                                    
    handle_call: 3,                                                              
    handle_info: 2,                                                              
    init: 1,                                                                     
    nack: 2,                                                                     
    safe_run: 2,                                                                 
    send_request: 5,                                                             
    send_request_to_microservice: 3,                                             
    send_request_to_microservice: 4,                                             
    send_response: 4,                                                            
    send_rpc_request: 2,                                                         
    send_rpc_request: 3,                                                         
    start_link: 0,                                                               
    start_link: 1,                                                               
    status: 0,                                                                   
    validate_config!: 1,                                                         
    child_spec: 1,                                                               
    code_change: 3,                                                              
    config: 0,                                                                   
    handle_cast: 2,                                                              
    terminate: 2,                                                                
    module_info: 0,                                                              
    module_info: 1                                                               
  ],                                                                             
  attributes: [                                                                  
    vsn: [30058795809176950487363593847644817736],                               
    behaviour: [GenServer]                                                       
  ],                                                                             
  compile: [                                                                     
    version: '7.4.4',                                                            
    options: [:no_spawn_compiler_process, :from_core, :no_auto_import],          
    source: '/app/lib/middleware/worker.ex'                                      
  ],                                                                             
  native: false,                                                                 
  md5: <<22, 157, 29, 13, 243, 57, 94, 254, 217, 206, 105, 41, 192, 95, 9, 72>>  
]                                                                                

The source code of the Spotter.Worker module you can see here. So, other modules that were mentioned here just import this module via use Spotter.Worker ... and define their own variable or like so. As you can see from the source code, the Spotter.Worker module has two start_link/0 and start_link/1 methods for further usage.

Can you show the actual test module?

Yes, of course. You can view this module here

I can’t figure out what it means, but the child spec passed to start_supervised! doesn’t match the one in the error ( Matchmaking.Middleware.Worker vs Spotter.Worker in start ). How does that happen?

Because I’m re-using the Spotter.Worker module as the base for custom worker implementation, it leads to the following “inheriance tree”, like this:

Spotter.Worker
┖ Matchmaking.AMQP.Worker
  ┖ Matchmaking.AMQP.Worker.Consumer
      ┖ Matchmaking.Middleware.Worker

For this case, only the Spotter.Worker module provides start_link/0 and start_link/1 calls for the further usage. All other modules only extends the functionality of its parent. The code for Matchmaking.AMQP.Worker and Matchmaking.AMQP.Worker.Consumer modules you can find here.

To be clear, modules do not have inheritance trees. You can inject code into them, but that isn’t the same as inheriting anything. Importantly the difference is that the quoted def start_link inside Spotter.Worker is not a function inside the Spotter.Worker module, which is why you have errors when it tries to call that function.

Just that the function is in Matchmaking.Middleware.Worker doesn’t mean it’s in Spotter.Worker, and Spotter.Worker is the one the error is about. Your source code for https://github.com/OpenMatchmaking/spotter/blob/master/lib/worker.ex also shows that it isn’t in there.

Can you show the code for All of the other relevant modules you list?

Of course, I can share the links to other modules as well:

  1. Matchmaking.AMQP.Worker module can be found here
  2. Matchmaking.AMQP.Worker.Consumer module can be found here
  3. And the main Matchmaking.Middleware.Worker module (with which I have troubles) can be found here