Restarting a process?

Questions / Help
,
1

I’ve got a simple app that starts up a supervised Broadway pipeline. This is fine for regular use, but it makes testing difficult:

  def start(_type, _args) do
    children = [{My.Pipe, []}]
    opts = [strategy: :one_for_one, name: My.Supervisor]
    Supervisor.start_link(children, opts)
  end

I’m wondering if there’s a way to re-start the process with different options specifically for testing? I would like to send some different options to the start_link/1 function specifically to help override things for testing.

Is this possible? Or should I rely instead on using Application.put_env/3 to modify settings at (testing) runtime?

2

You could check the mix env:

opts = if Mix.env() == :test do
  [strategy: :one_for_one, name: My.Supervisor, ...]
else
  [strategy: :one_for_one, name: My.Supervisor]
end
3

But beware that mix is not available in releases.

Sorry, misunderstood your problem, please ignore the suggestion below :frowning:

I’d consider using

# config.exs

config :my, My.Pipe, enable: true # enable by default

# test.exs

config :my, My.Pipe, enable: false

# application.ex

  def start(_type, _args) do
    children = [
      if Application.get_env(:my, My.Pipe).enable do
        {My.Pipe, []}
      end
    ] |> Enum.reject(&is_nil/1)
    opts = [strategy: :one_for_one, name: My.Supervisor]
    Supervisor.start_link(children, opts)
  end
1 Like
4

I tend to avoid testing processes started by the applications supervision tree. I start a separate (set) of them manually and without global naming and pass the pid around to tests. This way I often get the ability to run tests async as well.

4 Likes
5

I’m not sure how well known it is, but there’s start_supervised/2 for the approach described by @LostKobrakai.

6

I’m not sure if I follow how you’d do this… when I try to (re)start the Broadway pipeline for testing, e.g.

test "start_link something" do
      {:ok, _pid} =
        MyPipeline.start_link(
          producer: Broadway.DummyProducer,
          other: "overides",
          # ... etc ...
        )

I get an error:

{:error, {:already_started, #PID<0.280.0>}}

Likewise, I can’t figure out the syntax required to make the start_supervised option work.

start_supervised(%{
       id: My.Application,
       start: {My.Application, :start_link, [{My.Pipeline,
         [
           queue_url: "fake-url",
           producer: Broadway.DummyProducer,
         ]}]}
     })

Returns {:error, {{:EXIT, {:undef, ...

Same for:

start_supervised(
 %{
   id: My.Pipeline,
   start: {My.Pipeline, :start_link,
     [
       queue_url: "fake-url",
       producer: Broadway.DummyProducer,
     ]}
 }
)

Am I close?

7

That would work in a pinch, but it pegs all options to the environment, and the tests need to vary the options in order to fully test the results…

8

A clean way to do this is to just bake the possibility to switch to the special testing settings into your code.
One possibility would be to add one (or a few) optional arguments that can override the ‘default’ settings that are used.

The advantage on this over relying on Application.get_env/put_env is that it is completely separate between test-cases (it is ‘functionally pure’). This helps reasoning about the tests, but also e.g. allows your tests to run concurrently.

9

That’s the route I’m going down… but I still don’t see room for overrides when the process has already started BEFORE I execute my test code.

For example, if I want to override a callback that would normally query an external API with a mock callback that mimics a bad response from the API, I can do that in an individual function by using Application.get_env/put_env/3… but when those options need to be overridden inside the start_link/1 implementation and the process is already started… there isn’t a way to override the behavior. Even making use of Broadway.DummyProducer is pretty difficult when the process has already been started.

So I feel like I’m running around in circles. It feels like there is a way to do this, I just can’t make enough sense of the docs to pull the rabbit out of the hat.

10

I think I found one way of working with this based on this post.

First was to make the Broadway :name option overrideable by using Keyword.get/3 with a default value:

defmodule My.Pipeline do
    use Broadway

    def start_link(opts) do
        Broadway.start_link(__MODULE__,
        name: Keyword.get(opts, :name, __MODULE__),
        # ... etc ...
     )
   end
   # ... 
end

Then I can pass a unique :name option to my pipeline when I call start_supervised in my test:

test "start_link with custom options"
    {:ok, pid} = start_supervised({My.Pipeline,
                   [
                     queue_url: "fake-url",
                     producer: Broadway.DummyProducer,
                     # ... etc ...
                     name: :example
                   ]}
end

and it works equally well when I call start_link/1 directly:

{:ok, pid} = My.Pipeline.start_link(
                producer: Broadway.DummyProducer,
                queue_url: "fake-url",
              name: :something
              )

This works and I will probably use this method… the only downside I can see is that the regular built-in process in the application’s supervisor always starts… but we kinda just ignore it. That’s not a problem for my use case, however.

I think the docs for https://hexdocs.pm/ex_unit/master/ExUnit.Callbacks.html?#start_supervised/2 need some updating because they don’t really explain how to work with this common use case… I’m not sure what I did is even the proper way to work with that function… it just feels like a work-around more than anything else. Thoughts?

2 Likes
11

Just in case, here’s how I’d use start_supervised. Not sure if it addresses the problem.

Relevant bits:

  • in test env the “real” pipeline is disabled in the config
  • if config says the pipeline is disabled, it’s not added to the in the root supervisor, so that it doesn’t clash with any pipelines started in tests
  • the pipeline callback module has default opts which can be overwritten in tests
3 Likes
12

:wave: 3 years later! spotted this post while dealing with some legacy code.

If you really need to restart a process that’s part of the app sup tree, say in the setup block of your test, you can do that with:

:ok = Supervisor.terminate_child(My.Supervisor, My.Pipe)
{:ok, _} = Supervisor.restart_child(My.Supervisor, My.Pipe)

And in order to make it work you’d need to name your Supervisor in the application start, like:

Supervisor.start_link(children, name: My.Supervisor)

Note: restarting child would use original child spec defined in your app. So if you’d like to override some options passed into a supervised module you’d need to not rely on actual options in the child spec, but on the configuration of your supervised module that’s pulled from the app configs. In the original example that topic started brought up that My.Pipe is getting started with empty options.

I saw that approach recommended by @sasajuric a while back and I would suggest you do that only if you have no other way and you just need to get things done quick. Of course, most likely you wouldn’t like to have such test running async.

But the approach with start_supervised that @LostKobrakai and @idi527 pointed out above is what is probably an ideal way of doing things :+1:

1 Like