Depends on whether you’re using your schemas for all CRUD operations or mostly R operations. I have (some) schemas that are readonly. They still link like they should, but I also don’t always put all belongs_to or has_many relationships in all of my schema (because while the relationship exists and is enforced by the DB schema and the part of the schema I use for CRUD capabilities), not all schema are equal.
halostatue
0
TheFirstAvenger
My .02 on fixtures vs factories:
My problem with Fixtures is that it buries the logic that you are testing. Repeatedly in my career I have jumped in to work on a fixture-using project, and without fail I end up wasting way too much time on what should be a simple task of fixing or updating a test, because the logic of what exactly is being tested was not clear in the test, it was buried implicitly in the fixture data.
For example, if you were to come across this test which is all of a sudden failing, which relies on fixtures, what would you do to fix it?
test “returns confrazzled users” do
assert [%{id: 1}, %{id: 3}] = Confrazzler.get_confrazzled_users()
end
From the test, it is unclear what is being tested, or why it is failing. You could look in the fixtures, but even then it most likely wouldn’t be clear what exactly is supposed to be testing. You would have to dig into the actual code to figure out what get_confrazzled_users actually does. In a recent project, the function being tested used a 100 line SQL statement. It was basically impossible to reverse engineer what exactly was being tested.
Now, if it were written with factories, you would instead see this code:
test “returns confrazzled users“ do
%{id: id1} = insert(:user, frazzled: “con”)
%{id: id2} = insert(:user, frazzled: “pro”)
%{id: id3} = insert(:user, frazzled: “con”)
assert [%{id: ida}, %{id: idb}] = Confrazzler.get_confrazzled_users()
assert Enum.sort([ida, idb]) == Enum.sort([id1, id2])
end
In the factory test, the setup is clear and explicit, and part of the test. There is no digging in fixtures to figure out what the implicit assumptions are, or combing through 20 fields to try and figure out which field(s) are actually being tested in the current test, because the answer to that is clear in the setup inside the test. It is easy to understand that this test is testing that get_confrazzled_users returns users where their frazzled field is “con”.
In the fixture test, frazzled: “con” was set for id 1 and 3 in the fixture. That fact was known by the person who wrote the test initially, but it is hidden knowledge to everyone else reading it, and hidden to anyone who subsequently needs to update the fixtures.
Beyond the issue of fixtures burying logic, there have been plenty of issues I have caught because the factory was using randomly generated data and triggered an edge case I didn’t think of, which the static fixture never would have covered.
9
dimitarvp
Yep, you pretty much summed my experience with Ruby on Rails testing (and occasionally with PHP and some Java). The only good usage of fixtures I ever found was to pre-seed the test DB with data since there are plenty of apps where proper testing requires non-empty DB (namely context outside of what you current test is testing).
2
TheFirstAvenger
I would argue that if data in a context outside of what your current test is testing is necessary to the test, then it isn’t actually outside of the context of what you are testing. If a test of context A requires 50 data points in context B, then that should be part of the setup for testing context A, because if someone goes and changes the global fixtures, and now all of a sudden there are only 49 matching items in the fixture, then test A will start failing when the developer didn’t change any code related to context A or it’s test, and they are now on the hook for figuring out how to fix that broken test which they know nothing about.
Furthermore, if Context A starts failing in production, and someone investigating it looks at the properly set up version of the test, then it will be immediately apparent that “this fails without 50 items in Context B”, and maybe that is the cause in production.
1
dimitarvp
Sure, I am not contending that. But let’s include high developer velocity in the equation. In several teams I’ve been in, the people were happy to take some shortcuts for that cause.
I do personally prefer crystal-clear dependency graphs – especially for data – but often times it wasn’t up to me and my experience and past expertise have been disregarded as “pursuing academic excellence as opposed to being productive”.
I did eventually sneak a lot of such fixes behind the backs of the team and the manager for which I wasn’t congratulated, even if that resulted in eliminating 20+ intermittent CI/CD test errors. Oh well. 
1
tfwright
This seems spot on. Working with Rails, fixtures clashed hard with the convention over configuration philosophy, and practically speaking, when you’re working on “rapidly prototyping” a lot of brand new apps, it was much better to have everything self contained in each test. Conversely working on a project that did fixtures poorly was a hellish experience.
But getting acquainted with Elixir/Phoenix has really made me see how much of the “magic” you get with factory generators (like FactoryGirl) is unnecessary an d a potential source of silly gotchas. Putting a little more thought and discipline at the beginning can pay off big, while still allowing the vast majority of the convenience. In the Rails world there was a lot of talk about the 80/20 rule in terms of features, I feel like a similar rule should be applied to convenience: you get 80% of the benefit from 20% of the magic…
1