Testing with Ash authentication is extremely slow even after configuring `config :bcrypt_elixir, :log_rounds, 1`

Ash Forum Ash Questions / Help
,
1

I have been experiencing extremely slow tests for each test behind a protected route with Ash authentication. 54 tests are taking 47 seconds. I tried various tips available online including config :bcrypt_elixir, :log_rounds, 1 but it didn’t help. Meanwhile others are ble to run 376 tests in 7 seconds. How do you speed up your tests?

376 tests in 7 seconds: x.com

My experience:
image

2

:thinking: @jimsynz may have some thoughts on speeding that process up, but I believe that most people are not actually doing password authentication in the vast majority of cases.

They will simply generate a token for the user and set it into the conn, for example, as opposed to first going through authentication flow.

1 Like
3

The approach I am using is to create a user, then add a user_subject in the connection that will be used for protected routes.

I could also reduce this time if there was a way of signing in once at the beginning for all tests instead of having to sign in on every test. Do you know to sign in once before all tests begin?

Something like setup function be for all tests in the suite.

4

https://hexdocs.pm/ex_unit/ExUnit.Callbacks.html#setup_all/2

There is a setup_all that you can run at the start of each test to ensure your test user is created. It doesn’t happen in the sandbox, so won’t be rolled back on each test. So each test could check first if the user exists and create it otherwise, or do an upsert into the user’s table.

And your config :bcrypt_elixir, log_rounds: 1 is in config/test.exs?

1 Like
5

Yes. I have it configured. See: hr/config/test.exs at dev · kamaroly/hr · GitHub

6

kk. And you’re able to confirm that the majority of your test duration is taken up doing sign in?

7

Yes. like 90% of my tests require login.

8

You should just be able to generate a token for your test user. There is no need to validate their password. How are you generating the user_subject? bcrypt shouldn’t even really come into the picture right? Well…I guess it does to hash their password. But with log_rounds configured to 1 it should be very fast. Feels like something else might be a factor?

9

I’d like also to mention that I am using Ash multitenancy, thus, here’s what happens every time I need to tests a protected age:

  1. Create user
  2. Add user to organisation
  3. Add user to owner’s group
  4. Add all permissions to owner’s group
  5. Login the user and return authorized conn.
  6. Test pages behind protected routes.

I think if there’re could be a way of doing step 1 to 5 once for all. It would significantly speed up the tests.

Here’s the test login function: hr/test/support/auth_case.ex at 005b42a185106442b8f8820bfd85c04f4d38192b · kamaroly/hr · GitHub

Here’s a form test:

10

you can do that for each test using setup_all, or for all tests in your test_helper file IIRC.

1 Like
11

Here is what I did to reduce the testing time.

I created a fake hash provider to skip the bycrypt hashing in while testing. This improved test speed to over 50%.

defmodule MyApp.Accounts.User.HashProviders.TestHashProvider do
  @behaviour AshAuthentication.HashProvider

  @impl true
  def hash(input) when is_binary(input) do
    # Fake hashing logic: prefix the input with "fake_hash_"
    {:ok, "fake_hash_" <> input}
  end

  @impl true
  def valid?(input, "fake_hash_" <> input), do: true
  def valid?(_input, _hash), do: true

  @impl true
  def simulate do
    # For testing purposes, return false to simulate a hash check
    false
  end
end

Then, I made it configurable in config/test.exs like the following

# Do NOT set this value for production
config :ash_authentication, hash_provider: MyApp.Accounts.User.HashProviders.TestHashProvider
  1. I told User resource to pick the hash provider from configuration if available like the following
# /lib/my_app/accounts/user.ex
defmodule MyApp.Accounts.User do
  #...
  authentication do
    strategies do
      password :password do
        identity_field :email
        # Tell AshAuthentication to use a faker hash provider to speed up tests
        hash_provider Application.compile_env(
                        :ash_authentication,
                        :hash_provider,
                        AshAuthentication.BcryptProvider
                      )
      end
    end
 #...
end
12

This feels like it shouldn’t be necessary. :thinking:

13

This feels like it shouldn’t be necessary. :thinking:

It is, however, a nice work around.

@kamaroly - do you actually need to register a user for most of your test cases, or do you just need a user record to exist? If the latter then I would suggest just inserting one with Ash.Seed.seed!.

1 Like
14

Yes. I need to create a user at least in the setup_all. How have you been doing it?

15

I’d suggest Ash.Seed.seed! as well.

1 Like