How do I write Elixir tests?

Destroyed and replaced with cassettes, recorded from actual calls. I’d love to see a generic recorder for “any” elixir module/function call and response to etf or similar format. This has only become more important as use of llms grows.

I really don’t like describe "foo/2", test that the requirements are met, not the implementations. I’d rather group the tests by goals though I still use the function name for helpers or convenience public API functions.

Also fully on board for simpler database seeding, one of our staff engineers introduced ex_machina everywhere but to me it is just a headache factory.

@subject module attribute for module under test

I picked this up from you a few years ago when looking at your projectionist config and still use it to this day!

I think it depends on the “level” you’re testing at. I largely agree with you and yet I still find myself doing this all the time. As a minor note: the one thing I don’t do is include the arity. In my experience, it always gets out of sync and people don’t seem to notice.

Re: ex_machina, I agree it makes little sense in Elixir. With FactoryBot your factories would run through validation (which you touched on) and callbacks and I’ve often found myself essentially re-implementing the context function in my factories.

Ash’s generators have hit the right note for me since you get a changeset_for_* function for every domain (context) function. Ash has callbacks which you do need to call manually from generators, but this often a good thing if the callback isn’t essential for your tests (no silver bullet). I’ve always felt keeping as much logic as possible in changesets is good practice in Phoenix anyway so changesets could be used in plain ol’ Phoenix test as well.

If you prefer context function with MyApp.Blog.create_comment_for instead of ExMachina, and dislike mocking, how do you deal with complex context functions that needs to do a multitude of side effects such as:

• capture audit log changes via carbonite
• validate the user permission
• trigger webhook or API calls
• etc …

I understand wanting to test via the application functions but doesn’t it make the test suite much more complex and heavier?

Of course, it does not mean that you even need ExMachine, you could directly insert via Repo but you wrote that you prefer context functions so very curious if/how you handle that.

I tend to agree with everything short of:

In Elixir we have bunch of the mocking libraries out there, but most of them have quite substantial issue for me - these prevent me from using async: true for my tests.

Huh? The argument sounds to me as “we have several orange suppliers, the most of them supply apples instead, that’s why I don’t buy oranges at all.” To my best knowledge, each project needs one mocking library, and mox is carefully designed to allow running everything in parallel with a help of nimble_ownership.

In finitomata there is a whole testing framework allowing to test all the possible transitioning scenarios in FSM (async, in parallel, of course), and without mox it could not be even possible, because finitomata allows ‘determined’ transitions, which are executed instantly once the ‘from’ state for them is reached.

As I said in the article - these are guides not rules. There are situations when you need to break rules, but I find that often using any mocking library (whether it is Mox, Repatch, Patch, or any other) while sometimes allows doing tests in async manner, it often quickly break when you want to cross process boundaries. And if you want to use such mocks there, then you need to resolve to manually passing function anyway, which is not really different from “functional DI”.

However, as these are guides (again), there obviously will be points where you will break them. It is normal and expected. These are meant to be “high level suggestions” which I prefer to use unless I am forced to do otherwise.

How to test effects or the integrations like with external services and such?

Repatch mentioned

Repatch works with cross-process scenarios, but you need to find the processes to add to async group, which is the biggest problem. If you could point out the difficult parts, maybe I could add some functionality

That is the problem - you need to find processes to add them to group. And if you can do so, then IMHO patching isn’t a problem, but at the same time, in such cases you can often find a solution without mocking anything at all.

I would argue that in my personal experience the sole fact that difficulties do arise is a clear sign of the bad design, namely a superfluous coupling. Mocking here plays the same role as Boundary lib for static code.

I agree, but perhaps I could introduce helpers to add all processes linked to some process to group, or maybe add all children of some supervisor, or maybe introduce something based on tracing. I am curious to hear your pain points