Consumer driven contract testing in Elixir

Chat / Discussions
1

Hi all, i am currently researching consumer driven contract testing (in particular pact.io) in hope that it would make better development collaboration for frontend - backend or service to service in microservice architecture. However, even after reading through blogs and docs regarding pact.io, i am still struggling to fully understand it’s benefit. Could you share your experience?:

  1. Worked with consumer driven contract testing before? do you find it helpful?
  2. Does consumer driven contract ensure no breaking change? Does it improve no-breaking possibility much compared to using strongly typed schema (GraphQL/gRPC) combined with schema evolution?
  3. Does consumer driven contract enable better autonomy? Enable frontend to be developed separately from backend?
  4. Does there is consumer driven contract library in elixir, especially one that follow pact specification?
2 Likes
2

I did work in a team that had multiple services in different technologies and there was a push to do consumer-driven contract testing with Pact and it got implemented for Ruby services. At that point the tooling for Elixir was not that great so we didn’t do it in the end, but I think the biggest reason was that it was always a low priority item for us - we did have an extensive set of tests for the API, so I saw little to no value (but significant effort) in adding Pact tests.

Instead of Pact I’d suggest writing unit tests (note that here the unit of the test is a behavior/endpoint, not a module/function) this way:

describe "GET /api/foobars" do
  test "requires authorization" do
    assert %{status: 401, body: body} = get(nil, "/api/foobars")
    assert %{"error" => "Unauthorized"} = body   
  end

  test "returns a list of foobars" do
    assert %{"foobars" => []} = get!(api_key, "/api/foobars")
  end
end

The idea here is to write your tests as if they were written using an external tool (you’ll need to add some helpers for making HTTP requests), but run them inside your normal test suite (mix test) to leverage all the ExUnit goodies and keep them fast.

2 Likes
3

I don’t know if you can call these unit tests, I usually refer to these kind of tests as integration tests, as they test the system interactions as a whole, not just isolated parts.

1 Like
4

I certainly would call them unit tests, but then we get to the point of what is the unit of a test? I think the common approach of treating a class, module or a function a unit of a test is not really useful. For me the unit of a test is a single feature/behaviour.

In my books, an integration test is one where multiple independently-deployable services are involved.

2 Likes
5

@stefanchrobot thanks for detailed reply! Do you know impact of Pact on the Ruby services? Does using Pact on these services manifest significant positive impact (in term of ability to change API/enable autonomy for Frontend/Backend) that outweigh its implementation and maintenance cost?

6

Not sure, unfortunately. This initiative was pushed by the QA team. It did increase test coverage of the APIs, but from my perspective, I’d rather invest more in those end-to-end tests.

1 Like