In the absence of macros keyword lists are cheaper to create than maps. That way the production code pays the minimum of overhead while the cost is moved to the testing code.
If you tell me of a real use case where using maps (instead of keywords) made your application sluggish and nightmare slow for your clients to use, then I will refactor the code to use keyword lists.
Until then, I shall treat the Maps VS Keywords debate as I treat everything else before doing performance testing and finding actual pain points: as premature optimization.
Still, it’s good to know this detail about Keywords, thanks for chipping in!
I don’t want to test handle_continue, I just want it to run before ExUnit starts verifying the assertions in the tests
If yes, and these are unit tests, then my answer is - do not start the server at all. Just call functions and provide your own state directly. In that way you do not have the problem with asynchronous testing.
If that process isn’t SUT then mock it by starting different process with the same name
If these tests are integration tests then sleep in the test setup and then perform all actions.
Nothing prevents you from instead of using GenServer.call(my_server_pid, :message) use MyServer.handle_call(:message, make_ref(), state).
I can’t remember where I saw it now, but I’m sure I saw a post about using the :erlang.trace function to set up tracing on the SUT from the test. i.e. setup tracing for YourModule.handle_continue function and when it’s called tear down the tracing and continue with the assertions.
I can’t remember any more details than that, and I haven’t tried it myself but it seems possible to know the function has been called without having to add an extra message just for testing.
But what do you want to test then? Showing it might help understanding how to do it? If you want to test that a call made after the full initialisation (meaning init + :continue) results in either A or B according to the state set after the :continue executes, then doing a call with a big enough timeout should be (for practical purposes) deterministic (say 10secs timeout). If you want to test that side effects to the process state happen, then probably chaining manual calls to the handle methods should be enough, etc
defmodule Tracetest.Server do
use GenServer
def start_link(arg) do
GenServer.start_link(__MODULE__, [arg])
end
@impl true
def init([arg]) do
{:ok, :some_state, {:continue, arg}}
end
@impl true
def handle_continue(_, state) do
{:noreply, state}
end
end
The test:
test "can know when handle_continue finished" do
:erlang.trace(:new, true, [:call, :return_to])
:erlang.trace_pattern({Tracetest.Server, :handle_continue, 2}, true, [:local])
{:ok, pid} = Tracetest.Server.start_link(:foo)
assert_receive {:trace, ^pid, :call,
{Tracetest.Server, :handle_continue, [:foo, :some_state]}}
assert_receive {:trace, ^pid, :return_to, {:gen_server, :try_dispatch, 4}}
assert true == true
end
I setup the same tracing calls in the iex console, started the server there, and ran flush to see what trace messages got received. There were only two so it was straightforward to match on them. I am sure your real code is a lot more complicated than this but perhaps just waiting till the method had been called rather than a return would be enough to avoid the race condition.
The last few topics that you have posted suggest to me that you need a new boundary - for example between the worker process and the http client library.
Essentially that puts the http client library in the horrible outside world and the new module becomes the way your application wants to interact with “the service” - i.e. it becomes a contract that is specified by your application.
Once that boundary is established a test double can be slipped in that the worker process uses to complete its flow to produce results (e.g. a response message) that are directly observable by the test case process.
Depending on the necessary sophistication of the test double “test first” may need to take a back seat for a while.
This provide interface similar to the gen_server but functions are run synchronously instead of asynchronously. This should make testing of gen_server a little bit easier (however there still is no timeout, so you cannot test that).
When working with a lot of asynchronous calls (handle_continue, handle_info, :timer.send_interval, etc) I often make use of a helper to automate testing for results repeatedly. This avoids using fixed Process.sleep/1 calls where the sleep period is statically defined and either too long (which is pointlessly slow) or too short (which causes flickering tests):
def with_backoff(opts \\ [], fun) do
total = Keyword.get(opts, :total, 50)
sleep = Keyword.get(opts, :sleep, 10)
with_backoff(fun, 0, total, sleep)
end
def with_backoff(fun, count, total, sleep) do
fun.()
rescue
exception in [ExUnit.AssertionError] ->
if count < total do
Process.sleep(sleep)
with_backoff(fun, count + 1, total, sleep)
else
reraise(exception, __STACKTRACE__)
end
end
I typically use this from higher level integration tests, where it isn’t desirable to call handle_ functions directly.
In your production system how do you know that the workers are available to run? Also what type of failure do you get in your tests when the workers aren’t finished initializing? If you’re just doing a handle_call to the workers then I would expect it to work just fine (unless the initialization takes more than 5 seconds which would cause you to exceed the default handle_call timeout)
I feel there is some clarification needed here. The Worker process is a GenServer, that does, some work.
As all GenServers, it’s code is divided into 3 parts:
The public API that clients can call
GenServer calls to the process itself
handle_x calls which do the real work
In the tests I am performing right now, I am testing the public API. Say I have the following code:
defmodule Clint.Worker do
use GenServer
defmodule State do
defstruct conn_pid: nil,
active_streams: [],
worker_id: nil,
opts: %{},
deps: %{}
end
###############
# Public API #
###############
@spec start_link(map) :: Supervisor.on_start
def start_link(args) do
deps =
args
|> Map.get(:deps, %{})
|> build_deps()
worker_id = Map.fetch!(args, :worker_id)
opts = Map.fetch!(args, :opts)
Logger.debug("Starting worker #{inspect worker_id }")
init_state = %State{worker_id: worker_id, opts: opts, deps: deps}
pname = :bananas
GenServer.start_link(__MODULE__, init_state, name: pname)
end
@spec request(atom, integer, charlist, map) :: {:ok, :received}
def request(group_name, worker_id, url, injected_deps \\ %{}) do
deps = build_deps(injected_deps)
GenServer.cast(:bananas, {:fire, url})
{:ok, :received}
end
@spec build_deps(map) :: map
defp build_deps(injected_deps), do:
Map.merge(@default_deps, injected_deps, fn _, a, b -> Map.merge(a, b) end)
#############
# Callbacks #
#############
@impl GenServer
def init(state) do
Process.flag(:trap_exit, true)
{:ok, state, {:continue, :establish_conn}}
end
@impl GenServer
def handle_continue(:establish_conn, state) do
with {:ok, conn_pid} <- Logic.establish_connection do
new_state = %{state | conn_pid: conn_pid}
{:noreply, new_state}
else
{:error, reason} -> {:stop, reason, state}
end
end
@impl GenServer
def handle_cast({:fire, url}, state) do
stream_ref = state.deps.http.get.(state.conn_pid, url)
new_state = %{state | active_streams: [stream_ref | state.active_streams]}
{:noreply, new_state}
end
end
Here the public API (what clients will call) are the start_link and request functions. I am testing the Public API of the worker, the face it shows to the world.
This fits nicely into the “Test the interface, not the implementation” rule of testing, but it is insufficient. For example, using this methodology I can’t test any handle_info calls, unless I create a worker and then send him the exact messages that trigger handle_info (which according to some community members, I should do. I am now trying out this strategy).
Is this Unit testing? Is this integration?
I argue that the Worker is my unit, so I can argue this is unit testing. Some of you will say “You are mixing your state tests with GenServer OTP and therefore this is integration testing”. I wouldn’t say you are wrong, but I will definitely say this is a very thin line, at least for me.
@chrismcg I have never used trace for testing before, and I fail to see how I could apply it here. I need to invest more time into this idea, as it looks really cool!
I am however not sure why I need to wait for the
:erlang.trace(:new, true, [:call, :return_to])
call.
I usually (try to) make my questions and topics as small and isolated as possible, to make the load on the people reading smaller. Sometimes, this comes at the cost of clarity, which I believe is the case here.
My worker does not depend on any HTTP client. It depends on an HTTP contract, which can be implemented by any client I wish. The boundary is well defined, the way I see it
Perhaps, the trouble here is in making it clear it is a boundary. Perhaps you believe I am testing too much the details of my worker which leads you to think I have no boundaries defined. You are one of the most well educated people in testing I have seen and I find it curious how your opinions differ from mine in so many areas. All I can say to defend myself is that I am following a traditional London style TDD, while injecting the dependencies directly without creating Mock modules, because I believe functional injection is the best way of passing dependencies.
Test first is not always the solution, true. This is actually the refactor of a project that I made and that I consider is a complete disaster. No better time to fix it than now
In fact, one of the many problem is that this project has not tests at all
Oh my God. Thank you so much. I don;t want to sound … ungrateful, but I am not well versed in erlang yet, so I have trouble understanding what this actually accomplishes. I can only hope this comes with great documentation for dummies like me
@sorentwo So you let the tests fail repeatedly until you hit a timeout that is slow enough? I understand you provide a maximum number of tries for the test to run, correct (50 times) ?
