GenServer: avoid refresh running in tests

Questions / Help
1

I have a GenSever that hits an external API every 5 secs. Because of the limited number of API requests, I’m using Docker to run the app locally.
Now even when running tests and even using Mimic library, I need to have Docker API instance running.
I have several questions regarding how to avoid starting/hitting the API locally:

  • define a behavior, then 2 different implementations for prod and dev/tests and tweak the config settings to separate the implementation modules depending on the environment
  • check somewhere in the code the value of the MIX_ENV to define what to do.
  • any other solutions?

Thank you.

2

Yep, with e.g. mox to have contracts and then have dual implementation. I personally dislike that, too much writing and the result somehow does not impress because you also have to have compile-time configuration for test and non-test environments to pick an implementation. To me it’s an expensive abstraction.

I prefer mock or patch these days. They are much more transparent and stay out of your way.

Mmmm, don’t. No point. If you are going to do that then you might as well use mox because it’s almost the same. Though with mox you still keep the two implementations around… but with compilation paths that are different per environment – a standard practice in Elixir, and even newly generated projects have that feature, check elixirc_paths in mix.exs).

So I day don’t.

Maybe bypass? I am not impressed by it but many colleagues prefer it, arguing that it’s better to have a half HTTP server than to have none and just mock responses. Huge debate that we best not get into but I thought it’s best to give you another perspective.

My personal preference was mock, also used patch once and liked it. They are interchangeable.

Though have in mind the following, taken from patch README:

Since the global execution environment is altered by Patch, Patch is not compatible with async: true.

Same applies for mock.

mox allows for async tests.

2 Likes
3

Not yet tinkered with the combination but this might fit: async: false is the worst. Here’s how to never need it again.

4

You cannot work yourself out of the limitations of meck/patch. They’re replacing modules within the beam runtime wholesale to implement their behaviour, which by definition means you cannot use them with concurrent tests.

It’s interesting how dividing this can be. A mock is going to be another implementation and making this a behaviour instead of trying to ignore it will make things more obvious imo. Having a proper interface should also make it clear how each end of the interface is meant to be covered by tests.

There’s also no need to configure things at compile time. Mox doesn’t care how you switch out implementations, only that you do so. E.g. see the mentioned blog post around ProcessTree, where you could pass implementations in with the process start arguments.

2 Likes
5

Isn’t that distinction strictly academical?

I mean OK, I find the following worse:

GithubClient.impl().get_forks(username)

…than this:

GithubClient.get_forks(client, username)

But in the end it’s still a visually noisy abstraction. We shouldn’t need to look at our code and say “this is fine”.

You mean the one above my previous comment where Process.put is utilized? I mean I get it, I’ve used it myself when I really wanted something done and/or it got faster that way (or less visually noisy indeed) but if I can avoid it I’ll absolutely will.

BTW I am really curious if you can offer other patterns for having an implementation vary depending on Mix.env – any others except the two I mentioned? Admittedly I haven’t shopped around for such practices in a long time so might be one of my blind spots.

6

I think we got them enumerated here: Compile time selection through app env, runtime selection through app env, process dict (+ hierarchy) or plain passed as value and I guess runtime selection through switching out the module itself instead of the module referenced. Each of them come with their own set of tradeoffs.

Personally I think one should choose the option most suitable to when and how implementations need to be switched out. Though I’m not a fan of meck/patch because they hide the fact that there are multiple implementations from the codebase. Looking at the code you expect Module A to be called, but the code in Module A might never be called by tests. I very much prefer explicit polymorphism of implementations.

Also as with all places where polymorphism is involved you want a properly documented interface, so you can test the code using multiple implementations against the abstract interface as well as test that the actual implementations adhere to the interface. Without that I wouldn’t trust myself debugging a failing test a few month down the line and being able to figure out if the code is wrong or the mock implementation is wrong.

Lastly I tend to find that the things I need to mock are often also things I don’t have access to for dev work as well, so I need to be able to work with multiple implementations anyways.

7

You can have and use a @behaviour even if you don’t use mox. :person_shrugging: And I agree that when it comes to stuff that’s out of our hands – like 3rd party networked services – then it’s best if we have good contracts, parsing / validation, the whole bag of goodies.

Well OK, it’s hard to disagree on this but how do you cope with the visual noise? Can you give an example with your code? As mentioned above, I find it very non-ergonomic to not just directly call my module with arguments but having to go through a layer of indirection. Would you say you don’t find it as annoying as I do?

8

That. I tend to put selections behind impl functions in some places, but more recently I tend to prefer passing those implementations around top to bottom. I guess all OTP behaviours work like that as well. You never call MyGenServer.handle_info either and :gen_server just passes around a variable for the module being the behaviour implementation.

1 Like
9

Hmmm okay, I can see that, though in the case of GenServer we’re talking about a “live” entity – a separate process – and in our case we’re talking about a “dead” module that has to invoke the network when its functions are called.

But that distinction is likely irrelevant in regards to whether a 3rd party network dependency should accept an implementation as a first argument.

I don’t find the current state of affairs satisfying and I strongly dislike leaky abstractions but I’ll admit that passing an implementation around seems like the least worst way of handling 3rd party network dependencies.

10

Nothing on the beam runs outside a process though. The question is how much work it is to configure the process and if it’s possible to then pass said configuration down the callstack.

I really like that we can be that explicit in elixir especially over abominations like IoC containers, which you might see in OOP languages. I know it takes effort to add the posibility for switching out an implementation for what was previously a static function call, but imo trying to let it stay a “static function call” when it’s no longer that is even worse.

1 Like
11

I basically don’t see it. My regular code just calls (as an example) FlightTracker.get_flights(tracking_number). That module both defines the behavior but also does

def get_flights(number) do
  impl().get_flights(number)
end

So there is one basically boilerplate module that wraps it all and then the rest of my code doesn’t care. For that, I get async tests, a dev implementation that has to be different than prod anyway due to API limits, and all the caller code is still perfectly clear.

3 Likes
12

This is what I eventually settled on as well but it feels very… unclean. Both a behaviour module and an implementation dispatcher. Two in one.

But I am the first guy around here to make a huge stink about having too many small files so yeah, I settled on that option and made myself hate it the least. :grin:

13

I try to avoid introducing behaviour if this is only for test. If the behaviour already exists because the production code uses different implementations, then Mox all the way, but otherwise no. If the code calls an external thing such as an HTTP API we mock that with a real HTTP server. Even if it is controversial ; it works well in our case.

GenServers that do polling are just not started in test environment, and tested only in their own test. If the rest of the app must interact with it then we would rather have the GenServer only handle process stuff and delegate the logic to a functional module that will store the data somewhere, so we can call that directly in the tests to setup the data, and have the app interact with the data.

1 Like
14

Wow, I provoked a huge wave of discussions, some of them are still beyond of my understanding. Thank you all for bringing your experience and knowledge stones to the base.
What confuses me most is that n my simple app, I have a GenServer defies to be started as a child process wen the app starts:

# application.ex
@impl true
  def start(_type, _args) do
...
  {PaymentServer.Exchange.RatesMonitor, []}
]
end

where the module RatesMonitor is a GenServer which hits an external API server every ‘X’ minutes.

I tried to define a behaviour module as follows:

defmodule PaymentServer.Exchange.MonitorApi do
  @moduledoc """
  Handles Exchange server API calls
  """
  @callback get_rates(from_currency :: String.t(), to_currency :: String.t()) ::
              {:ok, any()} | {:error, any()}
end

Then in a real API module I declare this behavour:

defmodule PaymentServer.Exchange.AlphaVantageApi do
  @behaviour PaymentServer.Exchange.MonitorApi
  @moduledoc """
  This is a Alpha Vantage API module which provides the exchange rates data
  """
  @api_url "http://localhost:4001/query"

  @impl true
  def get_rates(from_currency \\ "USD", to_currency \\ "JPY") do
    url = build_search_url(from_currency, to_currency)

    case Req.get(url) do
      {:error, reason} -> {:error, reason}
      {:ok, response} -> response
    end
  end
...
end

Then in the GenServer module, I call it as follows:

defmodule PaymentServer.Exchange.RatesMonitor do
  @moduledoc """
    This is the implementation of an exchange rates monitor server
  """
  use GenServer
...
defp fetch_rates(%{from_currency: from_currency, to_currency: to_currency} = state) do
    api_module =
      Application.get_env(:payment_server, :api_module, PaymentServer.Exchange.AlphaVantageApi)

    response = api_module.get_rates(from_currency, to_currency)
    rate = Parser.parse(response)

    %{state | from_currency: from_currency, to_currency: to_currency, rate: rate}
  end
end

To be able to use a mock API module in tests, I created the following module:

defmodule PaymentServer.Exchange.MockAlphaVantageApi do
  @behaviour PaymentServer.Exchange.MonitorApi

  @impl true
  def get_rates(from_currency, to_currency) do
    """
    status: 200,
    body: %{
    "Realtime Currency Exchange Rate" => %{
      "1. From_Currency Code" => "EUR",
      "2. From_Currency Name" => "Euro",
      "3. To_Currency Code" => "USD",
      "4. To_Currency Name" => "US Dollar",
      "5. Exchange Rate" => "3.2045",
      "6. Last Refreshed" => "2024-03-31 16:47:21.920729Z",
      "7. Time Zone" => "UTC",
      "8. Bid Price" => "3.2045",
      "9. Ask Price" => "3.2045"
    }
    },
    """
  end
end

and put these lines in the config/tests.exs file:

config :payment_server,
  api_module: PaymentServer.Exchange.MockAlphaVantageApi

A simple tests where I tried to check if the mock module would be used:

defmodule PaymentServer.Exchange.MockAlphaVantageApiTest do
  use ExUnit.Case, async: true

  test "it should call mocked API" do
    response = PaymentServer.Exchange.MonitorApi.get_rates("eur", "usd")
    IO.inspect response
  end
end

Before running the test, I stopped the API running in a Docker container and run mix test test/payment_server/exchange/mock_alpha_vantage_api_test.exs which failed with:

** (Mix) Could not start application payment_server: PaymentServer.Application.start(:normal, []) returned an error: shutdown: failed to start child: PaymentServer.Exchange.RatesMonitor
    ** (EXIT) an exception was raised:
        ** (UndefinedFunctionError) function PaymentServer.Exchange.MockAlphaVantageApi.get_rates/2 is undefined (module PaymentServer.Exchange.MockAlphaVantageApi is not available)
            PaymentServer.Exchange.MockAlphaVantageApi.get_rates(:usd, :eur)
            (payment_server 0.1.0) lib/payment_server/exchange/rates_monitor.ex:64: PaymentServer.Exchange.RatesMonitor.fetch_rates/1
            (payment_server 0.1.0) lib/payment_server/exchange/rates_monitor.ex:36: PaymentServer.Exchange.RatesMonitor.init/1
            (stdlib 4.3.1.2) gen_server.erl:851: :gen_server.init_it/2
            (stdlib 4.3.1.2) gen_server.erl:814: :gen_server.init_it/6
            (stdlib 4.3.1.2) proc_lib.erl:240: :proc_lib.init_p_do_apply/3

So the question is why, despite all the above settings, the app could not start the GenServer module, PaymentServer.Exchange.RatesMonitor ?
It bothers me why to run tests, I still need to have API instance running in a Docker container? Even when using the above Mimic lib, (which mock fine the requests), it is still needed.

15

On a technical level this is indeed different, but on a functional level you have the interface in the form of an http api specification. Instead of switching out a module you’ll be switching out an endpoint. The only downside I see to this is that while you have an interface you’re not in control over the interface. The provider of the http api changing requirements breaks not just the integration with the actual API, but also all the rest of the codebase depending on the interface. If you wrap things in your own abstraction you can limit that blast radius.

This might also be a small push to define the interface by what the application needs rather than what the (current) provider provides.

module PaymentServer.Exchange.MockAlphaVantageApi is not available. This part of the error message is the key. It’s trying to call the correct module, but it’s not available. So the question would be where you put the module definition.

2 Likes
16

PaymentServer.Exchange.MockAlphaVantageApi module is defined in test/payment_server/exchange/mock_alpha_vantage_api.ex. Should it be absolutely somewhere in the lib directory?

17

By default only files in lib/ are compiled. Test files are scripts and called by the test runner. However in mix.exs you can add additional folders for source files to be compiled. Many elixir projects configure test/support to be such a folder for the test mix env: Add Elixir files to your compiled list - Today I Learned

2 Likes
18

Ah, I got it, thanks a lot!

19

On a technical level this is indeed different, but on a functional level you have the interface in the form of an http api specification. Instead of switching out a module you’ll be switching out an endpoint.

Yes but this interface has a single implementation in production code and so we test that only.

The only downside I see to this is that while you have an interface you’re not in control over the interface. The provider of the http api changing requirements breaks not just the integration with the actual API, but also all the rest of the codebase depending on the interface.

Indeed, if the HTTP API we depend on changes, we are screwed and have to chage the code. I don’t see how adding more layers to the story changes that infortunately. However I understand that you may get more time to change with more layers.

If you wrap things in your own abstraction you can limit that blast radius. This might also be a small push to define the interface by what the application needs rather than what the (current) provider provides.

Our production code that we test with a real http endpoint as a mock is our own abstraction, and provides what we need from that external dependency. If you add one more layer you still need to implement and test what to do when you get 400, 500, 403 errors etc.

What you get when splitting that in two parts is test speed, I concede, at the cost of more complexity.

Now if the endpoint returns {"hello": "world"} and our client returns %Greeting{who: "world"} then indeed we cannot just directly tell in the tests that the client will return that second form. But we can just pass that second form to the code that should handle it and test that it does. We just need one more test to be sure that there is “something” that will call the client and then pass that second form to the other part of the code.

In our app it works well like that because the “handler” part is dynamic and we can mock that with a simple module that will ensure that it receives the good data from a defined endpoint returned body. It depends on how things are layed out and there are so many possible ways to do things. Bottom line is I’d rather not add behaviours for the sake of tests.

1 Like
20

The thing that actually sold me on passing an implementation as a function argument is the fact that in tests I can have it both ways: both async: true and clean-ish production code. I don’t like the idea very much but still, we can always have one more wrapping module e.g.

defmodule MyApp.GithubAPI do
  def impl(), do: # fetch from env or w/e.

  def get_forks(repo) do
    MyApp.GithubClient.get_forks(impl(), repo)
  end
end

This allows (more) parallel tests as it doesn’t modify the modules globally and also calms down readability fanatics like myself – we don’t have to carry implementations around like the useless friend who always gets drunk and has to be carried / driven home.

1 Like