Test performance

ecto
phoenix
testing

#1

Hi all! :grin:
I already posted this question in the Elixir - Phoenix Slack Channel and there were a few helpful answers. But maybe some of you guys know whats up with my tests/maybe I am doing something wrong.

I am trying to unit test some of my entities (basically I have users, collections, items and items-in-collections).
For the items in collections I need I have a test setup in my items-in-collections test suite which looks somehow like this:

setup do
      user = create_user_with_valid_attributes()
      collection = create_collection_for_user(user)
      item = create_item_with_valid_attributes()
      {:ok, %{collection: collection, item: item}}
    end

These are all functions which are similiar to the generated fixture functions you get if generate some of your entities via contexts.
The items-in-collections entity has two foreign key contraints (collections and items) - and the collection belongs to a user.

Running this test suite (with 18 tests in total) takes about 4-5s - which is pretty long if you ask me. Of course, for each test, all of these entities need to be created at first.

If I copy the logger config of my config/dev into my config/test and run a single test in my test suite, the database statments take roughly about 2-5ms, including logged dband queue time.

Btw, the test passes, however once, there is an QUERY ERROR log which looks the same like other QUERY OK logs.

Is there so much time left between, setting up the test, building a db connection such that the tests take that long?

Thanks in advance!


#2

Hi there,

without the test code it’s very hard to figure out what the problem is :thinking:

I know it’s a lot to ask but recreating the problem in an app you can open source would be the easiest way to find out what’s wrong here. Alternatively I’d recommend you to profile the test run/multiple test runs to find the bottleneck: https://hexdocs.pm/mix/Mix.Tasks.Profile.Eprof.html#profile/2


#3

Hi @PragTob,

I was about to recreate the issue like you mentioned in a new repository. Thanks to your suggestion I found the problem! :raised_hands: :raised_hands: :raised_hands:

Solution:
For every test where I need a user, I create one on the fly. I did this via my application layer which involved using Comeonin.Bcrypt. This resulted in creating a hash for every user I inserted this way.
After changing my create_test_user helper function to work around this hashing algorithm (for I most of the tests I just need a user in the database) I decreased test execution time from 7.5s to 3s (~250 tests).
Also, the other tests are executing in a reasonable time now.


Btw: During my reserach I also came across this medium post: https://medium.com/adorableio/bcrypt-hashing-in-phoenix-tests-2532f07645b

In my case however, when I added

config :comeonin, :bcrypt_log_rounds, 4

to my config/test file, the test execution time did not decrease remarkably, if at all.

Do you know of additional :comeonin config with which I can achieve this?


#4

My config is a little different to yours, and it definitely makes a difference if I comment the line out. Try:

config :bcrypt_elixir, log_rounds: 4

#5

@amarraja: Thanks! it worked for me as well! :smiley:

Reduced test execution time further down to 1.5s.
Considering that it took > 20s last week I am more than happy