Repatch - Everything you need for mocking and patching

[Asd]

Hi,

I am happy to release the Repatch library for mocking and patching implementation in tests and anywhere else. It brings new possibilities which make it possible to make literally any test async.

You can install it and start trying it out in iex right now, while reading the post!

For example,

iex(1)> Mix.install [repatch: "~> 1.1"]
Resolving Hex dependencies...
Resolution completed in 0.01s
New:
  repatch 1.1.0
* Getting repatch (Hex package)
==> repatch
Compiling 3 files (.ex)
Generated repatch app
:ok
iex(2)> Repatch.setup
:ok
iex(3)> Repatch.patch Range, :new, fn l, r -> Enum.to_list(Repatch.super(Range, :new, [l, r])) end
:ok
iex(4)> Range.new 1, 10
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Continue in the docs: Repatch — Repatch v1.6.1

Features

1. Patch any function or macro (except NIF and BIF). Elixir or Erlang, private or public, it can be patched!

2. Designed to work with async: true. Has 3 isolation levels for testing multi-processes scenarios.

3. Requires no boilerplate or explicit DI. Though, you are completely free to write in this style with Repatch!

4. Every patch is consistent and applies to direct or indirect calls and any process you choose.

5. Powerful call history tracking.

6. super and real helpers for calling original functions.

7. Works with other testing frameworks and even in environments like iex or remote shell.

8. Battle-tested in real world production project. Using Repatch instead of Patch reduced time of the whole test suite by 10-20% (depending on amount of cores on the test runner machine) by reducing the time of sync tests by 2 times.

Quick comparison with other tools

1. Mox is a great library but it works only with behaviours which is a good approach from the architectural point of view, but fails to work in situations when you’re using 3rd-party library without behaviours, what makes you wrap every function you want to mock into the behaviour. However, Repatch inherits allowances and global mode approaches from Mox, but implements them in more efficient genserver-less way.
2. Patch is a great library too and it has been a good example and Repatch borrows some code from it. Patch doesn’t work with async: true what has been a problem because test suite was getting slower every time it was using Patch. However, Repatch shares same ideas as Patch does.
3. Mimic is a library I’ve used and loved too and it shares most of it’s features with Repatch, however Repatch is more efficient in local-only (aka private in Mimic terms) patching and has a shared mode. Also, Repatch performs less amount of module recompilations (every module is recompiled at most once) in runtime.

New horizons

Features I describe here are unique to Repatch and they bring new possibilities to write faster tests. Here is a short list of ideas:

• Patching Application and :application modules for async-friendly application env. I did this in my production project and I was able to make every sync test in the project I have to become async. I did not finish the work and benchmarking this approach yet, but I expect around 30s speedup in the whole suite, which is a lot.
• Patching DateTime module for time travelling in timestamps. Yes, it’s possible and works for some tests, but I haven’t figured a way to make it work for functions like :erlang.monotonic_time(), so be careful when using these.
• Patching System for async-friendly system env. I don’t usually write code which reads from system env in non-config files, but some people do and now you can test system env in async tests
• Patching Process.sleep and Process.send_after for timeouts manipulation. This is also a risky thing to do, but, as the old Russian saying goes “Those who do not risk, do not drink champagne”!

Afterwords

Don’t be afraid to experiment with Repatch and don’t be afraid to experiment in general. This library was born from understanding that long-lived tools and approaches are limited and there is always a way to move forward.

[axelson]

Thank you for the library, it looks quite interesting. I like how it’s functionality builds upon other Elixir mocking libraries. Can you talk a bit about how it’s implemented? I’m really curious how it’s able to safely override any call to external modules.

[Asd]

Sure, here it is: How it works — Repatch v1.1.0
Codebase is really small, around 1.5k lines, so you can read it in half an hour

[Asd]

Repatch 1.5.0

This is a big update which adds helpers on top of existing patch logic to support

1. Mox-style define/expect explicit DI mocking for behaviors.
2. ProtoMock-style define/expect explicit DI mocking for protocols.

That means that you no longer need to install a bunch of separate libraries, some of which require explicit dependency injection boilerplate, others don’t work with async tests, and so on. You no longer need to keep in mind the different limitations of these libraries with Repatch.

I’ve tried this update on the project of mine, I replaced all TestServer/Bypass, Mox, Patch and ProtoMock code with Repatch, made all of the tests async: true and tests now run 2 times faster, while testing code looks much more cleaner as a bonus.

[dimitarvp]

@Asd

I’ve tried this update on the project of mine, I replaced all TestServer/Bypass, Mox, Patch and ProtoMock code with Repatch, made all of the tests async: true and tests now run 2 times faster, while testing code looks much more cleaner as a bonus.

Oh, I would love examples! Any links to PRs / commits to demonstrate a before/after?

[mudasobwa]

The library looks great. For those who are more of self-controlled code, there is also Bex library to generate behaviours for whatever to be mox’ed.

[Asd]

Unfortunately, I can’t show the project, but I can give the numbers. It used to be 3 minutes sync tests + 1 minute async tests and now it is just 2 minutes async tests + a couple of seconds of sync tests. My goal is to make it possible to rewrite all tests with async: true.

The only tests I can’t make async are those which perform series of DDL transactions in postgres. Well, I mean, I can make them async by using unique database in each test, but it would consume more time to setup then time saved by async

[spicychickensauce]

Thanks for creating Repatch!
I think this is the best of all the mocking libraries with the least restrictions, and it should deserve a lot more recognition from the community.

Patching DateTime module for time travelling in timestamps

Do you have a helper module / small library that you use for this which you can share?
I needed time travel testing in literally every project at some point and I hate to reinvent it every time.

With Repatch I’m currently just doing this, but it’s probably too simple for most situations:

    {:ok, time} = Agent.start_link(fn -> ~N[1990-01-01 00:00:00] end)
    Repatch.patch(NaiveDateTime, :utc_now, fn ->
      Agent.get(time, fn t -> t end)
    end)
    # ...
    Agent.update(time, fn _ -> ~N[2010-01-01 00:00:00] end)

[Asd]

Hi, uhh, I mentioned it in “New horizons” because it was worth exploring. And I did some exploration and found out that

1. DateTime.utc_now uses System.os_time which uses :os.system_time. And :os.system_time can’t be patched
2. We can patch DateTime.utc_now but it will only replace this call, since, for example, erlang dependencies will use :os.system_time which will always return unpatched, real values
3. And there are also other functions like :erlang.monotonic_time which return different kinds of time

So my conclusion is that it’s not possible to use Repatch for 100% compatible time traveling. And it is not possible to use anything else, since erts implementation calls posix gettime directly, without any way to change it on erlang side only for some specific process.

My suggestion for you is to try to approach the problem another way: instead of travelling in time in order to make sure that some timers are triggered, you should try to make timeouts for these timers configurable and then set them up in tests accordingly.

[spicychickensauce]

Hmm, ok, thanks for digging.

However, so far, all my use cases of time traveling were possible by patching just NaiveDateTime and similar.
In my projects it mostly comes up with logic that involves ecto timestamps, and for those it is enough.
So I’d be happy with a good enough solution that only patches what’s possible.

But yeah, ideally we’d have travel_to like in rails that just works across the whole ecosystem..

[Asd]

Well, only thing that I can suggest to improve then is to use Repatch.patch(NaiveDateTime, :utc_now, fn -> ~N[1990-01-01 00:00:00] end) without an agent. You can then overwrite this patch with [force: true] option in the third argument.

But to be honest, I’d have a O_O look on my face if saw a test code like this.

[Asd]

Repatch 1.6.0

Coverage

This update adds first-class coverage support for patched modules. It utilized new OTP 27 native coverage and supports non-native coverage for OTP versions prior to OTP 26. Coverage support is introduced in compatible way without :cover module private functions hacking like in mex and mimic libraries.

Feel free to try it out in your project. doc.

Repatch.notify

A function which sends a message to the calling process the specified function is called. Current implementation uses Repatch patching, but future implementations will use :trace and :erlang.trace which are a little bit more efficient.

Repatch.history

The name speaks for itself, since this function just allows user to access the full history (or some filtered part of it) of all calls to the patched modules.

[murrgelb]

Interesting approach with the recompiler, this makes the library very powerful.

In Efx - A library to declaratively write testable effects we used process dictionaries to achieve async testing. What mechanism do you use?

You may want to add this library to your comparison as it is very very similar. It is more lightweight but we need to be explicit in code, about what we want to be bindable.

[Asd]

Hi, thanks!

Repatch currently has 3 isolation modes

• local mode which applies patch only to the single process. It uses process dictionary as the storage for patches and is the fastest one in terms of Repatch overhead
• global and shared mode which apply patch to all or some processes respectively. These modes use global ets tables as the storage for patches

[aifrak]

Thank you for this library!

I have a small issue where I use Repatch to test the today’s date inside a LiveView.

It works well if I use mode: :shared for each test, but I wanted to make it :shared for the whole LiveView test. So I thought I could do use Repatch.ExUnit, isolate_env: :shared as written in the documentation, but this does not seem work.

I am not sure if I am doing something wrong or if it does not work with expectations or even it is actually normal.

I have published a repository to repository to reproduce this case here. The line were Repatch is used is here.

[Asd]

Hi, thanks for the feedback, I appreciate it

1. isolate_env option is an option to isolate application env. That’s to limit the scope of Application.get_env and Application.put_env functions. See Repatch.Application — Repatch v1.6.1
2. You should pass a mode option to the expect call, like

   expect(Date, :utc_today, [mode: :shared, exactly: 2], fn -> ~D[2023-04-06] end)
   

So

• I will improve the documentation about this isolate_env option.

• And since I can see that you wanted to pass the default mode option on the use Repatch.ExUnit I think I will consider something like default_mode option, but I my biggest argument against this is that it is less explicit and reader of the test will be confused if this patch is local, shared or global unless they check one of the many line in the head of the module

  Perhaps I can implement some setup in the use Repatch.ExUnit to take the mode from test tag, like

  @tag repatch_mode: :shared
  test "..." do
    Repatch.patch(DateTime, :utc_now, fn -> :ok end) # <- this patch is shared mode now
  end
  

Share your ideas if you have any

[aifrak]

Thanks for your answer!

I will improve the documentation about this isolate_env option.

Thank you for that!

Perhaps I can implement some setup in the use Repatch.ExUnit to take the mode from test tag, like

@tag repatch_mode: :shared
test "..." do
  Repatch.patch(DateTime, :utc_now, fn -> :ok end) # <- this patch is shared mode now
end

I think it is fine to keep all options together like you previously wrote:

expect(Date, :utc_today, [mode: :shared, exactly: 2], fn -> ~D[2023-04-06] end)

It is indeed more explicit as you said. I just wish that we could add @tag at the top of describe but I don’t think it is possible to do that.

I was just searching a way to avoid adding mode: :shared to all expect inside a LiveView test, because everything inside a LiveView is in the LiveView process or in its child processes. As result, all tests for LiveView will have all have `mode: :shared`.

The trick that I have done so far to avoid repetition of the options was something like this:

@repatch_options [mode: shared, at_least: :once]

Then inside the test:

expect(Date, :utc_today, @repatch_options, fn -> ~D[2023-04-06] end

But maybe this is an anti-pattern as it is then less explicit.

I will follow your way and make it more explicit when I use expect by adding the options [mode: :shared, …] to all my tests.

[krasenyp]

If I define a mock for a protocol, should I allow for it to be used in different processes or it’s the same as in Promox - the struct carries some reference and no allowances are needed?

[Asd]

Mock for a protocol is just a module with defstruct plus the module which implmenets the protocol for this mock.

Consider this example

Mix.install [:repatch]

Repatch.setup()

defprotocol Proto do
  def f(x, y)
end

Repatch.Mock.define(My, protocol: Proto)
# It is essentially just the `defmodule` with `defstruct` and `defimpl Proto` inside

my = struct(My, [])

IO.inspect Proto.f(my, 123)
# raises
# ** (Repatch.Mock.NotImplemented) Not implemented
#     Proto.My."REPATCH-f"/2
#     file.ex:11: (file)

Repatch.patch(Proto.My, :f, fn _x, y -> y * 2 end)
# Note that we patch Proto.My module here. It is the module
# which contains specific implementation and it is automatically
# generated and compiled for every `defimpl` (which was done
# during the `Repatch.Mock.define`)
# It is `local` by deafult. If you want, you can override the scope
# and use allowances just like with any other patch

IO.inspect Proto.f(my, 123)
# prints 246

Repatch does not store the reference to the patch state in the structure, it is intentional.

---

If you could describe your initial problem, I may be able to come up with the way I’d solve it with Repatch