ryanzidago
Each tests should own its setup
Hi all,
In this article, I make the case for each test owning its setup.
Usually I forbid my AI agents to use the setup callbacks; I much prefer when tests are portable.
For example at work during code reviews, I often ask the AI agent to prove its finding by writing a failing test. Once the test is drafted, I ask it to publish it as a GitHub inline comment:
- I use the test as a tool to illustrate the agent’s point
- If the test assertions are valid, the author of the PR just needs to fix it
- the setup does not get in the way because the test is self-sufficient to be understood, you can just read the comment and get it, instead of trying to find the corresponding setup.
Even with huge setups, I prefer no common setup callbacks and have each test manually defininig their setup.
Let me know what you think!
Most Liked
Awlexus
Nitpick for the first example, I think you can completely side-step problem with the magic number by using the product price, like the test said it would
test "uses the product price as the order total", %{
customer: customer,
product: product
} do
assert {:ok, order} = Orders.create(customer, product)
assert order.total == product.price
end
wrt to large, shared setup blocks like the one in your example
setup do
account = insert(:account)
admin = insert(:user, role: :admin, account: account)
employee = insert(:user, role: :employee, account: account)
subscription = insert(:subscription, account: account)
product = insert(:product)
%{
account: account,
admin: admin,
employee: employee,
subscription: subscription,
product: product
}
end
Elixir already provides tools that make it possible to balance the of setup blocks
Setup functions can be changed
Instead of writing one large block you can build composable setup blocks. for example
setup [:with_account, :with_role, :with_subscription, :with_product]
Then define the setup blocks
def with_account(_context) do
{:ok, account: insert(:account)}
end
def with_subscription(context) do
{:ok, subscription: insert(:subscription, account: account)}
end
Using tags to modify setups per test
using @tag we can inject some values into the context ourselves, which we can use to modify shared setups to the needs of the test.
def with_product(context) do
{:ok, product: insert(:product, Map.get(context, :product_data, %{}))
end
Using these features the first test could be rewritten like this, allowing you to keep the tests (Not that I recommend this in particular, because using the product price directly would be preferred)
@tag product_data: %{price: 1000}
test "uses the product price as the order total", %{
customer: customer,
product: product
} do
assert {:ok, order} = Orders.create(customer, product)
assert order.total == 1000
end
I do not disagree with the points presented, but I prefer the ability to compose and to extend when possible. Also dicipline your Agents early on with DRY or else you’ll find the same function 5 times in different places and with different names.
krasenyp
Good article but there’s an issue with the examples. Too much reliance on ExMachina. This can bite you pretty bad because inserts like this might not encode invariants which the business logic expects. Use your app’s API to prepare the data for your tests. If you can’t then you have a big problem.
nathanl
I strongly agree. If we want to DRY up duplication, we can do it like we do anywhere else, using functions.
test "does a thing" do
%{stuff: stuff} = setup_the_stuff(%{which: :stuff})
# test content here
end
With this approach, if I want to know what stuff was setup, I know to inspect the function(s) that the test called and/or the data they returned.
Another article that argues the same case:
Last Post!
fuelen
I argued the general case for this in another thread so I won’t repeat it.
I’d say mocks are a one-time cost with Mox rather than a per-test one. Global stub in setup, expect only in the tests that assert on the interaction.
Side effects are real, and my answer is the same as in that thread. Skip the normal flow for the one step that’s too heavy, with a comment saying why.
I’d flip this around. The absence of it is the bigger cost. The schema grows an intermediate table, the real flow starts creating a membership, and insert(:user, company: company) stays green. Nothing failed, so nothing told you that your setup now describes a state your application no longer produces. Setup that goes through your context functions either breaks loudly when the flow changes or picks up the change on its own.
The “centralized setup layer built on functions” I recommended there is a library I maintain, I just didn’t name it at the time: seed_factory. It resolves dependent entities automatically, which is what makes going through business logic affordable.
Which brings me back to your original point. Rewriting ex_machina tests this way over the last three years, the setup often comes out shorter. I noticed that we mostly stopped moving setup into setup blocks. My read is that people extract setup mainly once it gets long, so shared setup is partly a symptom of expensive setup.
One day I’ll write the blog post about the road from ex_machina to seed_factory ![]()
Popular in Blog Posts
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex









