Proper way to test Membrane pipelines?

Elixir Framework Forums Membrane Forum
,
1

Hi there,

As I’m starting to implement Membrane pipelines (initially as one-off scripts like the one below), I wonder what are the good pipeline testing strategies?

I am more interested by pipelines testing (where the SUT = System Under Test is the pipeline itself, which is supervised in the example) than by individual components, where I have pretty good ideas on how to do that.

With this example:

Mix.install([
  :membrane_hackney_plugin,
  :membrane_mp3_mad_plugin,
  :membrane_portaudio_plugin
])

defmodule MyPipeline do
  use Membrane.Pipeline

  @impl true
  def handle_init(_ctx, mp3_url) do
    spec =
      child(%Membrane.Hackney.Source{
        location: mp3_url,
        hackney_opts: [follow_redirect: true]
      })
      |> child(Membrane.MP3.MAD.Decoder)
      |> child(Membrane.PortAudio.Sink)

    {[spec: spec], %{}}
  end
end

mp3_url =
  "https://raw.githubusercontent.com/membraneframework/membrane_demo/master/simple_pipeline/sample.mp3"

Membrane.Pipeline.start_link(MyPipeline, mp3_url)

What testing can be done?

I realise that ultimately I will use a “test sink” for these tests, and also mock out the HTTP source, but you see the general idea.

I will look into the Membrane codebase, but happy to bounce ideas here.

2

:wave: When testing pipelines, I try to avoid mocking and create E2E tests whenever possible :wink: I use snapshot testing extensively as well. In this case, I’d set up a simple HTTP server based on Bandit or inets that would serve the mp3 and a virtual sound card that would capture the output and save it to a file. At first, you can manually play the file and ensure it’s correct. Then, you can use it as a snapshot to compare automatically. I think If we had an HTTP source based on Req, it would be easier to test, as Req seems to have great testing features built-in :thinking:

Testing source and sink are actually useful for making artificial pipelines to test particular elements by feeding them with specific inputs and asserting on outputs. If you replace elements in a real pipeline with them, remember that you only test a part of the original pipeline.

1 Like