Mock depends on :meck, which does swap out the complete module within the runtime – as in make the VM unload the existing module and load the module with the mock code. That architecture cannot support concurrent tests. The best improvement to get would be better errors or disclaimers.

What you call an issue is imo a good driver for well rounded mocking.

There’s the guideline of “don’t mock what you don’t own”. You don’t own the API of File – the core team does. Elixir does well with not doing backwards incompatible changes, but they could always add new return values and such. You might not become aware of those additional return values, so you won’t be testing for those, which might break your code in production while even well setup tests – working on an incomplete assumption of the interface – would suggest everything is fine.

Instead you want to create your own interface (in the form of a behaviour), around the actual usecases you have for interacting with the filesystem. Let’s call it MyApp.FileStorage. Then you own the interface between your code and the underlying implementation using File’s API (MyApp.FileStorage.LocalFiles), as well as the implementation you use in tests (MyApp.FileStorage.Mock).

Changes in File’s APIs then no longer affect your mocked interface MyApp.FileStorage. They only affect the implementation MyApp.FileStorage.LocalFiles, which you hopefully tested separately without mocks to ensure it works correctly. Those tests hopefully fail before you push to production. All tests using the mock implementation would be unaffected.

One sideeffect of that approach is also that your interface might becomes smaller. Instead of the whole File API you’ll likely shrink the mocked interface to a few more select things your codebase actually needs, potentially even shrinking the number of possible parameters and return values as well. Complex tasks, which require multiple calls to File API might become a single callback on your behaviour, again simplifying the interface and how much work it would be to mock.

I certainly have made my point and I’ll stop adding comments in regards to that going forward. Though I’d argue that my points will aid in having more code under control, and therefore being able to be tested as much as possible, rather than less. The gaps will be exactly the impossible to cover gaps one has with any approach. So I think we do align on the goal.

So you cannot use automated tests – that’s a real and valid limitation. But even if you cannot have that specific benefit of testing the production implementation (which you couldn’t test anyways) you still get all the benefits of having the mocked interface not mapping to that external http API.

The only thing changes in that http api can break is your implementation of your own interface with said http API. It however cannot change your interface itself and to a certain degree your mock can get away not being affected as well.

There’s a few ways your external http api can change:

1. Endpoints change, but there are alternatives – potentially doing multiple requests – still using the same data as before, ultimately returning all the date you need.
   1. HTTP API is the interface → You need to adjust all code/tests/mocks interacting with that externally supplied interface
   2. You have your own interface → You adjust the implementation with the http API only.
2. Endpoints change and either need more data than before or no longer return data you depended on before
   1. HTTP API is the interface → You need to adjust all code/tests/mocks interacting with that externally supplied interface
   2. You have your own interface → You need to adjust your interface by the minimal amount of changes the http api changes pushed onto your interface (might just be a subset of the http api changes)
3. Stateful interaction between multiple API calls change
   1. HTTP API is the interface → You need to adjust all code/tests/mocks interacting with that externally supplied interface
   2. You have your own interface → If those changes are contained within a single callback of your interface, then only the implementation with the http api might need to change. If it affects interaction between multiple callbacks of the interface, then any interaction with your interface might need adjustment. You might however also be able to just adjust the implementation with the http api by introducing your own stateful handling correcting for the changes vs how your interface is used.

Actual real life changes might be a combination of those cases.

Having you own interface here is therefore an effective means of limiting the effects external changes can have onto your system. That’s essentially the idea around anti-corruption layers you might be reading about in hexagonal or onion architecture, though they usually explain that in regards to data not computation.