Dynamic test setup using tags?

Questions / Help
1

Before coming to Elixir I used FactoryGirl in pretty much every project that had data to generate test setup state, so naturally when I started my first Phoenix project I used Bamboo. I didn’t really like the experience nearly as much because you have to manage all the changset related logic yourself which means factories are a lot harder to maintain and/or they’re not as useful. So on my latest project I decided to try using vanilla ExUnit. The main feature Bamboo gives you is the ability to override specific fields with data relevant to what you want to test. You can of course write a function to generate that data but it’s not very DRY. You end up with tests with a half dozen or more functions just to set specific fields, and then you have to either duplicate that in other tests or move them to a shared module. Normally I actively avoid DRY tests, but helper functions feels a bit different.

Of course I could write my own set of very generic functions that accept attrs and call them in the tests so at least there would be less duplication, but I had the thought of using tags to do something similar with even less boiler plate. Basically the idea is to create functions like you normally would, but check for certain tag values in context:

  # Helper module
  def create_source(%{source_count: count} = context) do
    sources =
      0..(count - 1) |> Enum.to_list() |> Enum.map(fn _ -> create_source(context |> Map.drop([:source_count])) end)

    {:ok, sources: sources}
  end

  def create_source(%{user: user} = context) do
    source =
      Repo.insert!(%Source{} |> Map.merge(context |> Map.get(:source_attrs, %{}))
      )

    {:ok, source: source}
  end

  # some test file
  @tag source_count: 5, source_attrs: %{name: "test"}
  test "something about when user already has 5 sources named test", %{user: user} do
    assert %{user: _} = Source.changeset(%Source{user: user}, %{}).errors |> Enum.into(%{})
  end

This isn’t a pattern I’ve used very much so I’m not sure how it will end up working but it feels a lot cleaner to me. What do people think? Is this an abuse of tags? Is there a better way?

2

We use tags a lot for test setup at work, some thoughts:

  • it’s good, but can get unwieldy if you try to bunch too many unrelated tests under the same setup - lots of “optional” branches in the setup makes it harder to tell what will actually happen for a test. If you ever want to to write a TEST for your setup, You’re In The Bad Place.

  • overloading a name like create_source seems like a recipe for confusion. If some tests need many sources, split that out to create_sources or something that clearly distinguishes one-vs-many

  • passing the whole context around is usually not what you want since there’s ExUnit plumbing in there (like context.test). Even if it’s a little more verbose, future readers will appreciate if you call out exactly which keys are important (for instance, source_attrs and user in the recursive call to create_source)