Blog Post: Some Elixir Testing Tricks

This is my current command to run the tests:

mix test --failed --max-failures 1 --seed $(odo -c _build/seed.txt) && \
mix test --stale  --max-failures 1 --seed $(odo -c _build/seed.txt) && \
odo _build/seed.txt

odo is a simple counter that reads or increments a number from a file.

So what is does is:

• as long as I have a failing test, it will only run that test, thanks to the seed
• when this test passes, it will now fail repeatedly on the next failing test
• when all my previously failing tests pass, it will run all the stale tests. If one of them fails, the loop will start over.
• if everything passes, we increment the seed to preserve a minimum of entropy like designed in ExUnit.

A simple shortcut for vscode:

  {
    "key": "ctrl+alt+u",
    "command": "workbench.action.terminal.sendSequence",
    "args": {
      "text": "mix test --failed --max-failures 1 --seed $(odo -c _build/seed.txt) && mix test --stale --max-failures 1 --seed $(odo -c _build/seed.txt) && odo _build/seed.txt\n"
    }
  },

Thanks. I asked because I’ve been writing this kind of helpers for at least a couple of years now and I never thought about writing them as macros, so I was wondering if there was a deeper reason behind your choice.

Thats mostly out of a bit of laziness, to allow the module that’s defining the macros to not have to import the parts of ExUnit that does the assertions.

If you wanted to do them as pure functions, you’d have to import ExUnit.Assertions into the module you’re defining them in.

I’d written up an explanation on what I thought was the right answer, and even went to update the blog post to show you could use functions, but thought I should test it, and then on the first run of the test suite, got errors from no function named assert. So there you have it!

Speaking of macros, I’m curious why you use macros for your helpers instead of regular functions?

Yeah that one is pretty cute; works by passing around a tuple of the things you’d need for each one. Pretty clean way of doing it without much macro abuse

Ya I get it. I sometimes feel my tests get a little verbose with lv or view piping into everything so it wouldn’t take much to get me to buy into this if I were on a team that wanted to.! I was really interested in this project. I still am but I got really distracted by work and it just kind of slipped my mind.

It was mostly to reduce the large maintenance burden of writing

view |> element("button", "Create post") |> render_click()

everywhere (over 10k tests!)

We defined two versions, however, one that does “magic” and one that takes the view in as its first argument, so you can override as you need.

Looking at the calls, we use the explicit one in a couple places, and the implicit one everywhere else.

And its noted in the documentation using an Admonition/Nutrition facts thing, and failure to have a view variable set causes a compile error.

Hey if a team can decide and stick to it that’s totally cool! I’m solo right now but don’t trust myself not to die by a thousand cuts I also want less to document and explain if anyone else ever joins the project so I’ve been avoiding even the most seemingly innocuous things.

We use some custom click/form fill macros that pull it out of the current space at my current job, but my team is a two man team, and we both agreed that it was better than having to write lv all over the place.

All our assertions still require you to pass a html by hand.

Yep—it crossed my mind to do that and considered mentioning it here but didn’t want to give anyone any ideas I think it’s a bit gnarly and of course it stays pipeable if you directly accept the dependency. I’ve abandon every idea I’ve had like this after shooting myself in the foot with Ruby over the years. You technically don’t even need a macro to make it work as it is, but it’s a little bit simpler sorta.