Testing Phoenix Services

How do you test services that talk to external components such as DB or Email?

Say I have UserController.create that calls UserManager.register_and_notify. This UserManager talks to Ecto.Repo and maybe Bamboo. Should I treat this (and all services) like “controller extension” and write integration tests? or is it advisable to simply mock these dependencies and write unit tests?

I’m asking this because I got confused by @josevalim’s reply in this discussion. I fully agree with his explanation that it doesn’t make sense to mock dependencies (or just Ecto.Repo?) in controller [integration] tests since that’s the whole point of the controller - to integrate. However, he didn’t cover how we should approach testing the services that talk to external dependencies which is a common way to simplify our controllers.

3 Likes

Have you read the link at the end of José’s post? I find that piece very valuable in explaining about the use of explicit contracts in order to have clear boundaries of your code (which, in turn, simplifies testing).

This is my 2c, but rather than focusing on how you test something, it might be better to have a clear view of what you want to test, or why are you testing a certain component. It’s usually a matter of asking, “what is it that I really want to know with this test?”

Let’s say you define a contract of UserManager such that the register_and_notify function takes a couple args and returns a tuple. With such contract, we have the boundaries as such:

[UserController] -> [UserManager (contract)]

and

[UserManager (impl)] -> [Ecto.Repo & Bamboo]

With this contract, we decouple the controller with the service. Now, let’s ask some questions. Do you:

  1. Want to test that the controller calls the right function of UserManager with correct arguments? It makes sense to create a mock of UserManager via environment config (such as the TwitterClient in the post) and unit test the controller. Notice that we don’t care whether the call to UserManager inserts to DB or not, since we only interested in the code paths of the controller.
  2. Want to test that UserManager.register_and_notify hits the database and the mailer correctly? You need integration tests for that service (not the controller), no unit test will suffice. Bamboo has assert_delivered_email for that.
  3. Want to see if a call to a URL calls the correct controller function and hits the DB and mailer as well as returning the correct response? Write an end-to-end test.
  4. and more questions!

Of course, the only thing that certain is that this will not cover all cases, but I think thinking about and solving those edge cases is what we are being paid for as software engineers :slight_smile:

Given that, I’m still learning too so suggestions and critics to my view are welcomed!

2 Likes

Thanks for your insights! Got more follow ups:

Nailed it. I think this is the one I need.

Now, what about if I want my services to be testable without actually touching the database? In other words, I just want to check if I’m passing the correct parameters to the dependencies. How do we do this in practice? Let’s tackle Repo as a dependency first. Do we pass Repo from controller to the UserManager service so in our test we can just inject a stub? (just like the simple mock as a noun example in Jose’s blog post). Is dependency injection even a thing in Elixir or FP?

Or we should just mock (as a verb) Repo from UserManagerTest itself and just check the passed parameters?

Or more broadly, does this thought process even makes sense? I’m aware that this kind of unit testing might lead to errors that can only be seen in production but I still think our services (that rely to external dependencies) should just be unit tested and check if correct parameters to correct functions are being passed. Needless to say, this will allow us to run tests async: true.

IMO, Integration tests should be left for testing the controllers.

1 Like

Based on my understanding of what dependency injection means, I believe that the post introduced at least three ways to inject your dependencies to the function that needs it:

  1. via env config and Application.get_env

  2. via passing the callback function as argument with defaults

    def my_function(heavy_work \\ &SomeDependency.heavy_work/2)
    
  3. via passing the module fulfilling a contract as argument with defaults

    def my_function(dependency \\ SomeDependency)
    

I think I’m kinda lost here, I assume you mean if you want your services to be testable without touching the DB?

If we’re talking about Ecto, then if we want to check the parameters, I think the way to go is to test the your changeset validations. We’re passing changesets as the arguments to Repo operations, then it is reasonable to test whether the changeset functions generate the right validations. We are sure that the Repo function would convert the changesets to a proper query, since Ecto should have tested that for us.

It makes little sense for me to test whether the query generated by Ecto is correct by, for example, testing if the generated query matches an SQL string; that’s kinda like you’re testing Ecto’s query generators, which should be done by the Ecto authors. The only way to test if your query logic is correct is by providing some test db, run the query against it, and assert the results.

I believe Ecto and Phoenix already allow async DB tests?

From my POV, integration tests should be done for the integrators. José’s reply on the mailing list are structured as such because in that post’s context, the controller is the integrator, such that we call Repo functions from inside the controller function.

However, your code extracts the integrator role of the controller to your UserManager service. Therefore, the integration test should be done against your service. Your controller should have faith that the service would fulfil its own contract.

Do correct me if I’m missing something!

1 Like

Yes, but I don’t think it’s practical since your tests may become inconsistent from one test to another. Say one prior test inserted to database, no take down happened, then next case will check the number of records in the database, it will fail if it’s not expecting some other tests prior its execution time.

Agreed. I made a poor example by choosing Repo but I get what you meant here - if one would like to be confident if Repo is executing the correct query, there’s no unit tests that would suffice that but only a[n integration] test that actually writes to the DB. And you perfectly generalized this by saying:

Now to conclude this productive discussion (at least for me, thanks a lot), here’s what I’m going to do:

# test/controllers/user_controller_test.exs
defmodule MyApp.UserControllerTest do
  use MyApp.ConnCase
  describe "user signup via POST" do
    test "user creation" do
      # end-to-end test here. no mocks.
      # checks the formatting of the resulting data
      # checks db if the expected state is attained
      response =
        conn
        |> post(v1_user_path(conn, :create), user_params(@valid_attrs))
        |> json_response(201)
    end
  end
end
# test/services/user_manager_test.exs
# note: register_and_notify will do (for the sake of example)
# 1. Creates a user changeset according to the params
# 2. Writes to the DB using Repo
# 3. Sends Email using Bamboo
# 4. Award user using an internal service UserAward.give
defmodule MyApp.UserManagerTest do
  use MyApp.ServiceCase
  alias MyApp.UserManager

  describe "register_and_notify/2" do
    test "with valid parameters" do
      # do integration test
      # test and make actual db writes to verify the behavior of Repo
      # check if the expected user record is inserted in the test db
      # check assert_delivered_mail for Bamboo
      # create a UserAwardMock module and check if the correct parameters are being passed
      UserManager.register_and_notify(UserAwardMock, params)
   end
  end
end

Now, I find this awkward UserManager.register_and_notify(UserAwardMock, params), is this really the way to go? I’m aware that the design is inappropriate, the call for UserAward service should be done in the controller or by whoever consumer. But let’s just say that it really needs to be in register_and_notify, how do we deal with this in practice?

1 Like

This is not an issue with concurrent tests with Ecto since every test runs in it’s own transaction that is never committed. This leverages the database guarantees and allows us to be sure that the tests don’t influence one another. Also the cleanup - once test is finished is guaranteed by the database.

1 Like

Ah. Really? Then I have to recheck my newly created phoenix project since the data during testing are being persisted in the test database. Thank you @michalmuskala.

@michalmuskala, just want to update you that I created a new thread since I can’t really get this ecto transactions you mentioned to work. Test data is persisting making the tests non-repeatable

@nmcalabroso can you post Jose’s answer here?

I’m having this problem to open the discussion: https://github.com/elixir-ecto/ecto/issues/2999