SeanShubin
Is this a better way to test Elixir?
This is a followup to my earlier attempt here
The difference I am experimenting with here is to see if swapping out entire modules rather than individual functions is less intrusive on the code under test.
The problem of separating determinism from nondeterminism has been solved for decades with dependency injection,
and until someone is able to point out a better way,
I am sticking with the theory of dependency injection for now.
The idea is to pass modules into the parameter list in production,
and make stubs and fakes look like modules for test.
This turned out to be much more difficult than I expected, but I am not sure if that is because it is not easy in Elixir, or if I am not aware of the proper language constructs.
I ended up dynamically naming my fake and stub modules to keep their state isolated from each other.
I refactored duplication with a macro.
I pasted the IOFake module to the end of test/test_helper.exs because I couldn’t figure out another way to get the other tests to see it.
Although I would prefer a solution that is more simple and less clever,
this does still seem, to me, to be an objectively superior way to test non-determinism in Elixir code than what I have seen so far in the ecosystem.
I can run my tests in parallel without worrying about colliding state.
I have taken complete control of all non-determinism.
The code under test does not need to change much (I could even leave the parameters capitalized, making them look like modules instead of parameters).
I don’t have to make excuses regarding why this or that is too hard to test, I just test everything.
I only did IO.puts here, but you can see how this can be extended to any non-determinism.
For those of you that are interested in this kind of thing, do you see any flaws in my logic?
Or perhaps I missed a better way because I don’t know elixir that well yet?
defmodule ConcurrentA do
def send_line(line, io) do
io.puts(line)
end
end
defmodule ConcurrentB do
def send_line(line, io) do
io.puts(line)
end
end
defmodule ConcurrentATest do
use ExUnit.Case, async: true
test "send line" do
Fake.with_io fn io ->
ConcurrentA.send_line("hello", io)
ConcurrentA.send_line("world", io)
actual = io.get_lines()
expected = ["hello", "world"]
assert expected == actual
end
end
end
defmodule ConcurrentBTest do
use ExUnit.Case, async: true
test "send line" do
Fake.with_io fn io ->
ConcurrentB.send_line("hello", io)
ConcurrentB.send_line("world", io)
actual = io.get_lines()
expected = ["hello", "world"]
assert expected == actual
end
end
end
# Pasted at the end of test_helper.exs
defmodule IOFake do
defmacro __using__(_) do
quote do
def loop(lines) do
receive do
{:get_lines, caller} ->
send(caller, Enum.reverse(lines))
loop(lines)
{:puts, line} ->
new_lines = [line | lines]
loop(new_lines)
{:stop, caller} ->
IO.puts("stopped #{__MODULE__}")
send(caller, :stopped)
x -> raise "unmatched pattern #{inspect x}"
end
end
def puts(line) do
send(__MODULE__, {:puts, line})
end
def get_lines() do
send(__MODULE__, {:get_lines, self()})
receive do x -> x end
end
def start() do
process = spawn_link(fn -> __MODULE__.loop([]) end)
Process.register(process, __MODULE__)
IO.puts("started #{__MODULE__}")
end
def stop() do
send(__MODULE__, {:stop, self()})
receive do x -> x end
end
end
end
end
defmodule Fake do
def with_io(f) do
id = System.unique_integer([:positive])
{_, io, _, _} = defmodule String.to_atom("IOFake#{id}") do
use IOFake
end
io.start()
result = f.(io)
io.stop()
result
end
end
Trending in Discussions
Other Trending Topics
Chat & Discussions>Discussions
Latest on Elixir Forum
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #elixirconf-us
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #performance
- #security










First Post!
JohnnyCurran
If this is what you want to do, I think you should read:
Doing something like:
then in your app config you can:
And then in your app code you can:
If you’re leaning towards passing around side effecting modules to all of your code - I would reconsider
Unless you are talking about testing the actual implementations of
IO,Filethen I can’t be of more help. But if you want to make your tests have deterministic side effects then I think the article will be helpful for youMost Liked
benwilson512
OK here we go!
https://github.com/benwilson512/mox_demo/blob/main/lib/mox_demo.ex
This has your
mainfunction. As you can see I have swapped out theSystemIOandFileto the general IO behavior module I define here. mox_demo/lib/io.ex at main · benwilson512/mox_demo · GitHubAs I note in that module, it’s a bit simplistic to put all the IO functions in one module. In the real world you’d probably define multiple behaviors, but I wanted to keep this simple.
Then we have the test:
https://github.com/benwilson512/mox_demo/blob/main/test/mox_demo_test.exs
As you can see I can define test specific expectations. In the end I didn’t even really need to manage any state, since the ability to chain expectations meant that when you call eg
monotonic_timetwice I just define two expectations, one with the first value, and one with the second.I don’t have additional tests in there right now but these expectations are 100% isolated from any other tests that would be happening.
One nice consequence of this approach is that backend being parameterized you still get strong compiler and language server support since it resolves to a compile time value.
This is barely scratching the surface of what’s possible though and ironically, a lot of what’s possible is made possible by how minimalist Mox is. Here is the same test set up a different way where it’s more interactive. Basically instead of setting up all the IO ahead of time and calling
main(), we putmainin a task and then have it basically “talk to” our test process as if it is the IO world:https://github.com/benwilson512/mox_demo/blob/main/test/interactive_test.exs
All I’m doing here is combining core Elixir primitives like
sendandreceivewith Mox. Mox doesn’t really have any state to track here other than knowing which functions to call for this particular invocation ofmain. Then the “state” of the IO is in this case managed by my test process itself by sending messages to the mock at the appropriate times.And of course you’ll note that you can just run
mix testin the project root directory and these two test files will run at the same time and do not interfere. Both approaches are perfectly valid. I tend to favor theMox.expectapproach when I just have some outside system that I need a value or two from, or that I want to assert we pushed a value to. I tend to use the second approach if I’m writing more of a “simulator” test where there is some IO heavy piece of code and I want to step through it. This particular example makes that look sort of verbose but thatsetupblock is generic. Once you write your little stub you’re done, and each test just gets to really focus on the interaction between the function under testing and the IO.SeanShubin
Thanks so much for putting the work in, that was really helpful for my prototyping. You can see my various attempts here:
Now that I got the mechanics down I can try to figure out which I like best and why, but I am too tired for analysis now. I wanted to post my results here in case anyone else is interested.
ityonemo
Mox lets you do concurrent testing. Each expect is bound to the lifetime of the test, and processes can figure out which test they map to to get where they need to go.
Instead of passing modules around, a better choice is to assign the module to a module attribute so it will be set at compiletime. This is extremely unobtrusive and basically replaces
Modulenamefor@modulename. If you need to use a default impl aways except for when an expect is explicit, you can use stub_with to set it.If you are wondering about the mechanics of how concurrency works with mox, here:
Last Post!
SeanShubin
Thanks, I have made the update.
I have posted my analysis here:
https://forum.elixirforum.com/t/analysis-of-testing-styles-in-elixir/57329
I found the intention of my analysis sufficiently different than my intention for the discussion here to warrant a separate thread.