How do you test? What do you test?

Hello! there are really tons of resources about testing with elixir, from courses, books, post, you name it. @germsvel, is the guy that you can check out, he had put a lot of effort on this topic, I will name a few testing live view is a paid resource, the former employer bought it and it has pretty good material, or a free one testing with phoenix, also has a talk: Using DDD concepts to create better Phoenix Contexts, anyway there are a lot of resources. I just name a few that In my experience bring a lot of value

I think you might be over-analyzing it. IMO just start and then ask yourself: “Which parts I want to make sure keep working after I change code in places X and Y?”.

This is honestly the best way to gauge that I ever found. There have been metrics tons of material written about religiously testing everything or testing the bare minimum. Truth is always in the middle as I think most mature people know – so use the above benchmark.

There’s no end of articles telling you the “right way” to test, but IMO you’ll be better-served by writing ANY tests and learning from there.

The simplest possible ExUnit test could look something like this:

defmodule AssertionTest do
  use ExUnit.Case

  test "the truth" do
    result = 2 + 2

    assert result == 4
  end
end

There are plenty of tools in the ExUnit toolbox to make writing related tests more efficient (setup blocks, tags, etc) but none of them are required to get started.

If you aren’t writing tests currently, then presumably you are running your code somehow to verify that it works the way you intend - for instance, by poking around in iex.

One approach to start writing tests is to literally just PASTE that poking around into a test block; that way you can run them over and over again without having to manually set things up or worry about missing steps.

At the beginning, copy-paste with wild abandon - you’ll eventually want to reduce duplication, but it will be much easier to spot “what’s common” vs “what’s unique” when there are more examples to review.

This realization is what got me into testing and what sold me on “test first” (which I am not religious about). Afterwards I have a written test that I can choose to keep or not. I still poke around in the REPL to start, though.

Yep! It’s just one of those things that requires practice where you can sniff out what is necessary to test. I use tests to help guide design but ultimately I only really care about regression/acceptance tests.

A good metric to think about is how you feel about refactoring at any given moment. If the answer is “confident” then you probably have the “right” tests. If the answer is “not at all” then you don’t have enough tests. If the answer is “I’m afraid to because of all the tests I’m going to have to fix,” then you have too many tests/too fined-grained tests. This is, of course, isn’t perfect in of itself. Unfortunately it really is one of those complex topics you just have to start doing to get a feel for because ya, you’re going to get a lot of conflicting advice.

EDIT: Quoting @al2o3cr but generally responding to @RicoTrevisan.

As you yourself said below, regression & acceptance tests are super important. And as I said above, I got “sold” on testing when I realized that I liked something that worked and didn’t want it to stop working so I… discovered testing back then, long long time ago.

So a repeated message to OP: don’t get religious about testing, just test what you think is important to keep working in your project.

There’s a fairly popular-with-the-kids YouTuber who tells people not to write tests at all so

Well, who are we to stop them. Maybe that’s a shortcut to making more money with software? These days I am more result-oriented.

Yep, that’s more or less what I’m getting at. I wouldn’t want to work that way, but lots of people have very different ways of working and somehow still produce profitable software. There’s so much context around all of this stuff but people don’t like hearing that and just want to be told The One True Way.

Yep, agreed, we should throw away any religion-like thinking from our profession. The only hill I am willing to die on these days is – strong static typing. I’ve witnessed the bug reduction of that first-hand, a number of times, and I am now tired of replicating the work that really smart and good compilers do, with my own hands, in dynamic languages like Elixir.

But I can’t deny Elixir’s REPL and very strong poke-ability (if that’s a word) are still keeping me hooked to it. And being able to just get inside a k8s pod and connect to the real prod DB and fix a problem with Elixir code (and not with Postgres SQL statements inside psql) is something I am still not willing to give up – though I haven’t given a fair chance to the alternatives either…

But it’s still a coin toss. F.ex. we like fiddling in iex and prototype stuff and I cringed not having that in Rust… but in Rust you are thus encouraged to write your stuff in a modular and decoupled manner and have tests for it almost right away, and this flow becomes your “iex”. Because if you don’t do it like that then you can code for a full week without being able to see if your idea works. But if you write in certain ways you can have almost the same iteration speed as with Elixir.

Both approaches netted me very good software so again, in the end it’s just preference. I at least still haven’t found OneTrueWay™ to do things. I lean on the side of messing around in the REPL but I’ve seen several other approaches and they worked just as good as well.

That’s funny because I come from startup environments where tests are not written usually because “we don’t have time”, which results in at least 3x time spent afterwards on debugging stuff, stuff that is as stupid as passing parameters in wrong order for a function.

All software should be written with tests, otherwise development velocity is not possible, that is true for 100% of untested projects I worked on.