SeanShubin
How do I test things like IO, File, and System?
New to elixir, looking for guidance regarding how to test things like IO, File, and System
Whenever I learn a new language, I always make sure I know how to test it by implementing the following “Hello, world!” application.
defmodule HelloApp do
def main() do
time_unit = :microsecond
microseconds_before = System.monotonic_time time_unit
target = System.argv |> List.first |> File.read!
IO.puts "Hello, #{target}!"
microseconds_after = System.monotonic_time time_unit
microseconds_duration = microseconds_after - microseconds_before
IO.puts "Took #{microseconds_duration} microseconds"
end
end
I intentionally choose the requirements for this application because they have the most extreme combination of easy-to-write yet hard-to-test that I can think of.
As it is in any language, the trick is to isolate the side effects behind an abstraction of some kind.
With Elixir this seems harder to do than normal, but perhaps that is simply because I don’t know the proper language constructs to hide the side effects behind an abstraction.
I don’t see a way to swap out System, IO, and File with stub versions because they are sitting in a global namespace.
I did look at ExUnit.CaptureIO, but rejected it for 2 reasons.
First reason is that as the capture is happening globally, I become vulnerable to one test affecting another.
Second reason is that CaptureIO is not generalizable to File and System.
Ideally I would like each test to validate the implementation in a sandbox independent of other tests with all the side-effecting modules swapped out.
I was able to get 100% test coverage by inverting the dependencies, so it looks like this may indeed be the right answer.
However, since this is my first ever Elixir program, I want to check with people with more Elixir experience than me to make sure I am on the right track.
What do you think of this solution to getting IO, File, and System under 100% test coverage?
Can you guide me to a better solution?
For the implementation, I invert each side-effecting dependency
defmodule HelloAppInverted1 do
def main(collaborators) do
system = collaborators.system
file = collaborators.file
io = collaborators.io
time_unit = :microsecond
microseconds_before = system.monotonic_time.(time_unit)
target = system.argv.() |> List.first |> file.read!.()
io.puts.("Hello, #{target}!")
microseconds_after = system.monotonic_time.(time_unit)
microseconds_duration = microseconds_after - microseconds_before
io.puts.("Took #{microseconds_duration} microseconds")
end
end
For production, I configure to use the real thing:
defmodule HelloAppEntry1 do
def main() do
system = %{
:monotonic_time => &System.monotonic_time/1,
:argv => &System.argv/0
}
file = %{
:read! => &File.read!/1
}
io = %{
:puts => &IO.puts/1
}
collaborators = %{
:system => system,
:file => file,
:io => io
}
HelloAppInverted1.main(collaborators)
end
end
For testing, I configure to use stubs:
defmodule HelloAppInverted1Test do
use ExUnit.Case
@moduletag timeout: 1_000
test "say hello to world" do
tester = create_tester(
%{
:command_line_arguments => ["configuration.txt"],
:remaining_monotonic_time_values => [1000, 1234],
:file_contents_by_name => %{
"configuration.txt" => "world"
},
:lines_emitted => []
}
)
tester.run.()
assert tester.lines_emitted.() == ["Hello, world!", "Took 234 microseconds"]
end
def create_tester(initial_state) do
state_process = create_process(initial_state)
monotonic_time = fn time_unit ->
send(state_process, {:get_monotonic_time, time_unit, self()})
receive do
x -> x
end
end
argv = fn ->
send(state_process, {:get_argv, self()})
receive do
x -> x
end
end
read! = fn file_name ->
send(state_process, {:get_file_contents, file_name, self()})
receive do
x -> x
end
end
puts = fn output_string ->
send(state_process, {:puts, output_string})
end
lines_emitted = fn ->
send(state_process, {:get_lines_emitted, self()})
receive do
x -> x
end
end
system = %{
:monotonic_time => monotonic_time,
:argv => argv
}
file = %{
:read! => read!
}
io = %{
:puts => puts
}
collaborators = %{
:system => system,
:file => file,
:io => io
}
run = fn ->
HelloAppInverted1.main(collaborators)
end
%{
:run => run,
:lines_emitted => lines_emitted
}
end
def create_process(state) do
spawn_link(fn -> loop(state) end)
end
def consume_monotonic_time(state) do
[time_value | remaining_time_values ] = state.remaining_monotonic_time_values
new_state = Map.replace(state, :remaining_monotonic_time_values, remaining_time_values)
{new_state, time_value}
end
def append_line(state, line) do
new_lines_emitted = [line | state.lines_emitted]
Map.replace(state, :lines_emitted, new_lines_emitted)
end
def loop(state)do
%{
:command_line_arguments => ["configuration.txt"],
:remaining_monotonic_time_values => [1000, 1234],
:file_contents_by_name => %{
"configuration.txt" => "world"
},
:lines_emitted => []
}
receive do
{:get_lines_emitted, caller} ->
send(caller, Enum.reverse(state.lines_emitted))
loop(state)
{:get_monotonic_time, :microsecond, caller} ->
{new_state, monotonic_time_value} = consume_monotonic_time(state)
send(caller, monotonic_time_value)
loop(new_state)
{:get_argv, caller} ->
send(caller, state.command_line_arguments)
loop(state)
{:get_file_contents, file_name, caller} ->
file_contents = state.file_contents_by_name[file_name]
send(caller, file_contents)
loop(state)
{:puts, line} ->
new_state = append_line(state, line)
loop(new_state)
x ->
raise "unmatched pattern #{inspect x}"
end
end
end
- edit, I said production twice, the last snippet was for testing, not production
Most Liked
benwilson512
Hey @SeanShubin welcome!
While true, I think there’s more to say here. As a functional language, the first thing to do wouldn’t be to reach for an abstraction, it would be to reach for extraction (I couldn’t resist). Basically, work to extract the code with side effects from the logic of your program.
Then much of your logic can be tested just with pure inputs and outputs. In particular the values you’re reaching for like the path to read or write a file to should definitely not be produced in the same function that actually does the writing or reading to the file.
sodapopcan
Hello @SeanShubin and welcome.
Your target audience is a bit unclear here. Your writing style is attracting responses from seasoned Elixir developers and seasoned Elixir developers are well aware they are not working in a purely functional language. Erlang was developed to solve a practical problem and its functional properties emerged as a consequence of that. It’s never been academic. Elixir developers generally favour pure functions for the most part and push all side-effects to a particular boundary which isn’t necessarily the top boundary (depending on how you think about your system).
I’d say go for it if you want. You will generally find people around here encouraging others to code however they see fit. If you look through some prominent open source Elixir projects there are all sorts of different styles.
dimitarvp
FWIW, what you are describing is more in the lane of integration / end-to-end testing than unit testing. Having pure functions that don’t rely on side effects and testing them with various inputs is what I would do in an imperative / OOP language as well, not only in an FP one (like Elixir). Testing those and/or their modules is what’s unit testing.
If you anticipate wildcard events – like time jumps due to DST (which just happened last night in Europe) – then craft a specific test that mimics the scenario?
As for files, I can partially side with you because it’s not a one-minute job to imitate conditions with no disk space remaining, exhausted file descriptors etc. but with the help of the already mentioned Mox – or Patch – you can find the underlying Erlang code or dig in the docs for the right error responses when these things happen and just return them straight away in a mock.
(Me and many others in the Elixir community use this technique to a crushing success, very often, e.g. when you have an API client contacting 3rd party servers you can just directly return 404, 500 or a validation error in your mock. Or 429 for too many responses etc. You can exercise the unhappy paths very easily. Not super quickly, mind you, it’s still a job where you must apply rigor and cover your expected error conditions exhaustively. But it’s possible and it’s done regularly in commercial projects.)
I don’t think there’s a magic silver bullet here. As programming matures as a profession / discipline, people at large mostly understand that 99% of your code should be pure and the rest 1% should be well-covered with integration tests utilizing mocking.
Last Post!
SeanShubin
I think that is a consequence of me being new to Elixir while at the same time having such ambitious goals regarding testability. When I start coding in a programming language that is new to me, I want to get certain foundations nailed down first.
I have found the ability to write code in a pure functional style has very little to do with the language. I can write any program in a language such as Java in 100% pure functional style and Java is not even trying to be a functional programming language. I have also seen code in Haskell, a pure functional programming language, where the side-effecting IO Monad has so permeated the entire code base that I can’t even comprehend what it means to call that code functional.
The reason why I am so focused on precision regarding what it means to be functional, is that I am trying to parse out the justification for when to take a functional vs non-functional approach. Sometimes it is because of language limitations that ruin the aesthetics of certain styles, sometimes it is the way people are used to coding, sometimes there are well established conventions, and I guess that comment was a challenge to see if the only reason the functional approach was not chosen in a particular case is because it doesn’t occur to people that it is even possible. If you tell me I can’t have pure functional, or maybe it doesn’t make sense to strive for pure functional, that may very well be true, I just want to know why.
I am not seeking any more clarification right now though, I got enough suggestions to keep me busy. I am glad you let me know about the history of Elixir being practical and becoming functional as a side effect, not the other way around.
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #hex









