Global State in tests

Questions / Help
1

I’m doing evaluation of using elixir for some e2e api tests for a distributed application not written in elixir.

My plan is to use ExUnit to run these tests concurrently with the Req library. I think it will showcase elixir’s strength in parallelism.

The problem i’m facing at the moment is I’d like some global state which contains the bearer tokens for the rest of the tests. I can do a login in a setup_all function, in each test module, but that means we login a bunch more times than is needed. The applicaiton will create a new bearer token each login, and i really want to share one between modules for the same user.

I just wondered if anyone has encountered this type of thing before and how they solved it? Ideally ExUnit.start would have an option of a state, which gets merged with the state from setup/setup_all

Am I thinking about this wrong?

2

You should be able to setup req with the required auth headers once then all your test cases just use that req struct to do their requests:

https://hexdocs.pm/req/Req.Steps.html

3

I guess you mean like

req = Req.new(base_url: "https://api.github.com", auth: "token")

or something, but then how do i get this global req into all test modules? That’s really the guts of my question.

In my case specifically, login is particularly expensive so i’d like to do it once globally per user, and push this into some sort of global state

4

It might help if you read the ExUnit documentation, it provides example of setting up the test context and shows how it is available to all your tests.

5

Hacky, but you could declare a module in your test_helper.exs that logs in and stores the token in a function. Something like:

defmodule Login do
@token login_me_in()
  def token, do: @token

  def log_me_in do
      Your login code here
  end
end

This would get executed once before all the tests.

You would call it as Login.token()

You could play around with @tag and :included in ExUnit.configuration() to only conditionally do the login, so eg use a tag :external and only really perform the login if it is included, otherwise return a dummy token.

6

Hermanverschooten, you sir, are a legend. This is exactly the sort of hack i was looking for.

Ideally ExUnit would cover this type of use case but i realise it might be fringe, so this will work in the meantime. Thank you!

7

sorry andrewh but i don’t understand how what you linked to sets up a global context, just the context for a module. Please let me know if i’m mistaken.

I read the docs several times before posting this.

8

@adw632’s solution would work if all your tests that need the token are in the same testfile. setup_all would run just once before all these tests, but as far as I know there is currently no way to set a “before” hook to run once before all the tests, there is one for after all tests after_suite/1.

1 Like
9

Yes ExUnit is designed to isolate tests and test suites so you don’t get leakage and side effects between tests.

It even randomizes the order to help catch unintended dependencies and side effects to make testing robust.

You could use the test_helper compile time “evil” that @Hermanverschooten showed to create a globally fixed value.

You could also use many of the nicely explained flakey test approaches to undo all the goodness that ExUnit provides and make your tests flakey and evil too.

Such evil can include stashing a Req structure in the Application environment or Ets. So in your setup_all callback you try and get the Req struct and only create and set it if it is doesn’t yet exist.

You will likely introduce a race condition on the create which you thoughly deserve for doing such evil things.

I would just use setup_all to do the authentication once per test suite. How many test suites do you have?

Perhaps you need to think about Mocking?

1 Like
10

You are right, though maybe a bit strong in wording.
I’d consider (this case) more as a seed.
Mocking may indeed be something to look into, I use it extensively.

1 Like
11

If i end up using elixir for this (currently they are a mix of karate and postman tests) then it is likely that i’d have 80+ suites/modules

I fully understand the issues with race conditions and such -I think it’s somewhat par for the course with system testing.

I will look at mocking but we already have application mocks - that’s generally integration testing or service level testing if the library mocks in the general sense. the point of system/e2e testing is to not mock, eg. to be as close to our deployed system as possible, and have all the real components under test.

12

My solution to this to get the token in test_helper.exs put it in the env and then in test’s setup pick the token from the env for the tests that need it.

Application.put_env(
  :app,
  :token_key,
  get_token()
)
// test
  setup_all do
    token = Application.get_env(:app, :token_key)
2 Likes