SeanShubin

SeanShubin

How do I test things like IO, File, and System?

New to elixir, looking for guidance regarding how to test things like IO, File, and System

Whenever I learn a new language, I always make sure I know how to test it by implementing the following “Hello, world!” application.

defmodule HelloApp do
  def main() do
    time_unit = :microsecond
    microseconds_before = System.monotonic_time time_unit
    target = System.argv |> List.first |> File.read!
    IO.puts "Hello, #{target}!"
    microseconds_after = System.monotonic_time time_unit
    microseconds_duration = microseconds_after - microseconds_before
    IO.puts "Took #{microseconds_duration} microseconds"
  end
end

I intentionally choose the requirements for this application because they have the most extreme combination of easy-to-write yet hard-to-test that I can think of.
As it is in any language, the trick is to isolate the side effects behind an abstraction of some kind.
With Elixir this seems harder to do than normal, but perhaps that is simply because I don’t know the proper language constructs to hide the side effects behind an abstraction.
I don’t see a way to swap out System, IO, and File with stub versions because they are sitting in a global namespace.
I did look at ExUnit.CaptureIO, but rejected it for 2 reasons.
First reason is that as the capture is happening globally, I become vulnerable to one test affecting another.
Second reason is that CaptureIO is not generalizable to File and System.

Ideally I would like each test to validate the implementation in a sandbox independent of other tests with all the side-effecting modules swapped out.

I was able to get 100% test coverage by inverting the dependencies, so it looks like this may indeed be the right answer.
However, since this is my first ever Elixir program, I want to check with people with more Elixir experience than me to make sure I am on the right track.

What do you think of this solution to getting IO, File, and System under 100% test coverage?
Can you guide me to a better solution?

For the implementation, I invert each side-effecting dependency

defmodule HelloAppInverted1 do
  def main(collaborators) do
    system = collaborators.system
    file = collaborators.file
    io = collaborators.io
    time_unit = :microsecond
    microseconds_before = system.monotonic_time.(time_unit)
    target = system.argv.() |> List.first |> file.read!.()
    io.puts.("Hello, #{target}!")
    microseconds_after = system.monotonic_time.(time_unit)
    microseconds_duration = microseconds_after - microseconds_before
    io.puts.("Took #{microseconds_duration} microseconds")
  end
end

For production, I configure to use the real thing:

defmodule HelloAppEntry1 do
  def main() do
    system = %{
      :monotonic_time => &System.monotonic_time/1,
      :argv => &System.argv/0
    }
    file = %{
      :read! => &File.read!/1
    }
    io = %{
      :puts => &IO.puts/1
    }
    collaborators = %{
      :system => system,
      :file => file,
      :io => io
    }
    HelloAppInverted1.main(collaborators)
  end
end

For testing, I configure to use stubs:

defmodule HelloAppInverted1Test do
  use ExUnit.Case
  @moduletag timeout: 1_000

  test "say hello to world" do
    tester = create_tester(
      %{
        :command_line_arguments => ["configuration.txt"],
        :remaining_monotonic_time_values => [1000, 1234],
        :file_contents_by_name => %{
          "configuration.txt" => "world"
        },
        :lines_emitted => []
      }
    )

    tester.run.()

    assert tester.lines_emitted.() == ["Hello, world!", "Took 234 microseconds"]
  end

  def create_tester(initial_state) do
    state_process = create_process(initial_state)

    monotonic_time = fn time_unit ->
      send(state_process, {:get_monotonic_time, time_unit, self()})
      receive do
        x -> x
      end
    end

    argv = fn ->
      send(state_process, {:get_argv, self()})
      receive do
        x -> x
      end
    end

    read! = fn file_name ->
      send(state_process, {:get_file_contents, file_name, self()})
      receive do
        x -> x
      end
    end

    puts = fn output_string ->
      send(state_process, {:puts, output_string})
    end

    lines_emitted = fn ->
      send(state_process, {:get_lines_emitted, self()})
      receive do
        x -> x
      end
    end

    system = %{
      :monotonic_time => monotonic_time,
      :argv => argv
    }
    file = %{
      :read! => read!
    }
    io = %{
      :puts => puts
    }
    collaborators = %{
      :system => system,
      :file => file,
      :io => io
    }

    run = fn ->
      HelloAppInverted1.main(collaborators)
    end

    %{
      :run => run,
      :lines_emitted => lines_emitted
    }
  end

  def create_process(state) do
    spawn_link(fn -> loop(state) end)
  end

  def consume_monotonic_time(state) do
    [time_value | remaining_time_values ] = state.remaining_monotonic_time_values
    new_state = Map.replace(state, :remaining_monotonic_time_values, remaining_time_values)
    {new_state, time_value}
  end

  def append_line(state, line) do
    new_lines_emitted = [line | state.lines_emitted]
    Map.replace(state, :lines_emitted, new_lines_emitted)
  end

  def loop(state)do
    %{
      :command_line_arguments => ["configuration.txt"],
      :remaining_monotonic_time_values => [1000, 1234],
      :file_contents_by_name => %{
        "configuration.txt" => "world"
      },
      :lines_emitted => []
    }
    receive do
      {:get_lines_emitted, caller} ->
        send(caller, Enum.reverse(state.lines_emitted))
        loop(state)
      {:get_monotonic_time, :microsecond, caller} ->
        {new_state, monotonic_time_value} = consume_monotonic_time(state)
        send(caller, monotonic_time_value)
        loop(new_state)
      {:get_argv, caller} ->
        send(caller, state.command_line_arguments)
        loop(state)
      {:get_file_contents, file_name, caller} ->
        file_contents = state.file_contents_by_name[file_name]
        send(caller, file_contents)
        loop(state)
      {:puts, line} ->
        new_state = append_line(state, line)
        loop(new_state)
      x ->
        raise "unmatched pattern #{inspect x}"
    end
  end
end
  • edit, I said production twice, the last snippet was for testing, not production

Most Liked

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Hey @SeanShubin welcome!

While true, I think there’s more to say here. As a functional language, the first thing to do wouldn’t be to reach for an abstraction, it would be to reach for extraction (I couldn’t resist). Basically, work to extract the code with side effects from the logic of your program.

Then much of your logic can be tested just with pure inputs and outputs. In particular the values you’re reaching for like the path to read or write a file to should definitely not be produced in the same function that actually does the writing or reading to the file.

sodapopcan

sodapopcan

Hello @SeanShubin and welcome.

Your target audience is a bit unclear here. Your writing style is attracting responses from seasoned Elixir developers and seasoned Elixir developers are well aware they are not working in a purely functional language. Erlang was developed to solve a practical problem and its functional properties emerged as a consequence of that. It’s never been academic. Elixir developers generally favour pure functions for the most part and push all side-effects to a particular boundary which isn’t necessarily the top boundary (depending on how you think about your system).

I’d say go for it if you want. You will generally find people around here encouraging others to code however they see fit. If you look through some prominent open source Elixir projects there are all sorts of different styles.

dimitarvp

dimitarvp

FWIW, what you are describing is more in the lane of integration / end-to-end testing than unit testing. Having pure functions that don’t rely on side effects and testing them with various inputs is what I would do in an imperative / OOP language as well, not only in an FP one (like Elixir). Testing those and/or their modules is what’s unit testing.

If you anticipate wildcard events – like time jumps due to DST (which just happened last night in Europe) – then craft a specific test that mimics the scenario?

As for files, I can partially side with you because it’s not a one-minute job to imitate conditions with no disk space remaining, exhausted file descriptors etc. but with the help of the already mentioned Mox – or Patch – you can find the underlying Erlang code or dig in the docs for the right error responses when these things happen and just return them straight away in a mock.

(Me and many others in the Elixir community use this technique to a crushing success, very often, e.g. when you have an API client contacting 3rd party servers you can just directly return 404, 500 or a validation error in your mock. Or 429 for too many responses etc. You can exercise the unhappy paths very easily. Not super quickly, mind you, it’s still a job where you must apply rigor and cover your expected error conditions exhaustively. But it’s possible and it’s done regularly in commercial projects.)

I don’t think there’s a magic silver bullet here. As programming matures as a profession / discipline, people at large mostly understand that 99% of your code should be pure and the rest 1% should be well-covered with integration tests utilizing mocking.

Last Post!

SeanShubin

SeanShubin

I think that is a consequence of me being new to Elixir while at the same time having such ambitious goals regarding testability. When I start coding in a programming language that is new to me, I want to get certain foundations nailed down first.

I have found the ability to write code in a pure functional style has very little to do with the language. I can write any program in a language such as Java in 100% pure functional style and Java is not even trying to be a functional programming language. I have also seen code in Haskell, a pure functional programming language, where the side-effecting IO Monad has so permeated the entire code base that I can’t even comprehend what it means to call that code functional.

The reason why I am so focused on precision regarding what it means to be functional, is that I am trying to parse out the justification for when to take a functional vs non-functional approach. Sometimes it is because of language limitations that ruin the aesthetics of certain styles, sometimes it is the way people are used to coding, sometimes there are well established conventions, and I guess that comment was a challenge to see if the only reason the functional approach was not chosen in a particular case is because it doesn’t occur to people that it is even possible. If you tell me I can’t have pure functional, or maybe it doesn’t make sense to strive for pure functional, that may very well be true, I just want to know why.

I am not seeking any more clarification right now though, I got enough suggestions to keep me busy. I am glad you let me know about the history of Elixir being practical and becoming functional as a side effect, not the other way around.

Where Next?

Popular in Questions Top

rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
Lily
In templates/appointment/index.html.eex: <%= for appointment <- @appointments do %> <tr> <td><%= appoi...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New

Other popular topics Top

electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New

We're in Beta

About us Mission Statement