Restarting a process?

You could check the mix env:

opts = if Mix.env() == :test do
  [strategy: :one_for_one, name: My.Supervisor, ...]
else
  [strategy: :one_for_one, name: My.Supervisor]
end

But beware that mix is not available in releases.

Sorry, misunderstood your problem, please ignore the suggestion below

I’d consider using

# config.exs

config :my, My.Pipe, enable: true # enable by default

# test.exs

config :my, My.Pipe, enable: false

# application.ex

  def start(_type, _args) do
    children = [
      if Application.get_env(:my, My.Pipe).enable do
        {My.Pipe, []}
      end
    ] |> Enum.reject(&is_nil/1)
    opts = [strategy: :one_for_one, name: My.Supervisor]
    Supervisor.start_link(children, opts)
  end

I tend to avoid testing processes started by the applications supervision tree. I start a separate (set) of them manually and without global naming and pass the pid around to tests. This way I often get the ability to run tests async as well.

I’m not sure how well known it is, but there’s start_supervised/2 for the approach described by @LostKobrakai.

I’m not sure if I follow how you’d do this… when I try to (re)start the Broadway pipeline for testing, e.g.

test "start_link something" do
      {:ok, _pid} =
        MyPipeline.start_link(
          producer: Broadway.DummyProducer,
          other: "overides",
          # ... etc ...
        )

I get an error:

{:error, {:already_started, #PID<0.280.0>}}

Likewise, I can’t figure out the syntax required to make the start_supervised option work.

start_supervised(%{
       id: My.Application,
       start: {My.Application, :start_link, [{My.Pipeline,
         [
           queue_url: "fake-url",
           producer: Broadway.DummyProducer,
         ]}]}
     })

Returns {:error, {{:EXIT, {:undef, ...

Same for:

start_supervised(
 %{
   id: My.Pipeline,
   start: {My.Pipeline, :start_link,
     [
       queue_url: "fake-url",
       producer: Broadway.DummyProducer,
     ]}
 }
)

Am I close?

That would work in a pinch, but it pegs all options to the environment, and the tests need to vary the options in order to fully test the results…

A clean way to do this is to just bake the possibility to switch to the special testing settings into your code.
One possibility would be to add one (or a few) optional arguments that can override the ‘default’ settings that are used.

The advantage on this over relying on Application.get_env/put_env is that it is completely separate between test-cases (it is ‘functionally pure’). This helps reasoning about the tests, but also e.g. allows your tests to run concurrently.

That’s the route I’m going down… but I still don’t see room for overrides when the process has already started BEFORE I execute my test code.

For example, if I want to override a callback that would normally query an external API with a mock callback that mimics a bad response from the API, I can do that in an individual function by using Application.get_env/put_env/3… but when those options need to be overridden inside the start_link/1 implementation and the process is already started… there isn’t a way to override the behavior. Even making use of Broadway.DummyProducer is pretty difficult when the process has already been started.

So I feel like I’m running around in circles. It feels like there is a way to do this, I just can’t make enough sense of the docs to pull the rabbit out of the hat.

I think I found one way of working with this based on this post.

First was to make the Broadway :name option overrideable by using Keyword.get/3 with a default value:

defmodule My.Pipeline do
    use Broadway

    def start_link(opts) do
        Broadway.start_link(__MODULE__,
        name: Keyword.get(opts, :name, __MODULE__),
        # ... etc ...
     )
   end
   # ... 
end

Then I can pass a unique :name option to my pipeline when I call start_supervised in my test:

test "start_link with custom options"
    {:ok, pid} = start_supervised({My.Pipeline,
                   [
                     queue_url: "fake-url",
                     producer: Broadway.DummyProducer,
                     # ... etc ...
                     name: :example
                   ]}
end

and it works equally well when I call start_link/1 directly:

{:ok, pid} = My.Pipeline.start_link(
                producer: Broadway.DummyProducer,
                queue_url: "fake-url",
              name: :something
              )

This works and I will probably use this method… the only downside I can see is that the regular built-in process in the application’s supervisor always starts… but we kinda just ignore it. That’s not a problem for my use case, however.

I think the docs for https://hexdocs.pm/ex_unit/master/ExUnit.Callbacks.html?#start_supervised/2 need some updating because they don’t really explain how to work with this common use case… I’m not sure what I did is even the proper way to work with that function… it just feels like a work-around more than anything else. Thoughts?

Just in case, here’s how I’d use start_supervised. Not sure if it addresses the problem.

Relevant bits:

• in test env the “real” pipeline is disabled in the config
• if config says the pipeline is disabled, it’s not added to the in the root supervisor, so that it doesn’t clash with any pipelines started in tests
• the pipeline callback module has default opts which can be overwritten in tests