Seriously, my “technical journey” (air-quotes) has been: VLOOKUP on Excel → no-code (Bubble.io) → snorted some JavaScript → and now I’m mainlining Elixir & Phoenix on everything.
I’ve heard about given-when-then, exploratory testing, and I’ve seen people go through a test suite. I think I kinda understand as an end user testing, but I’m not sure I get it when it comes to the code side of it. I tried to get ChatGPT and Claude to ELI5 but I always have this sneaking suspicion that might not be the best instructors.
So, imagine you had a young brother who knows some ideas of programming, but he’s missing the foundation: where should he start? Which concepts, thought-technologies, frameworks should he start with? Are there books / videos / channels you recommend?
Hello! there are really tons of resources about testing with elixir, from courses, books, post, you name it. @germsvel, is the guy that you can check out, he had put a lot of effort on this topic, I will name a few testing live view is a paid resource, the former employer bought it and it has pretty good material, or a free one testing with phoenix, also has a talk: Using DDD concepts to create better Phoenix Contexts, anyway there are a lot of resources. I just name a few that In my experience bring a lot of value
I think you might be over-analyzing it. IMO just start and then ask yourself: “Which parts I want to make sure keep working after I change code in places X and Y?”.
This is honestly the best way to gauge that I ever found. There have been metrics tons of material written about religiously testing everything or testing the bare minimum. Truth is always in the middle as I think most mature people know – so use the above benchmark.
There’s no end of articles telling you the “right way” to test, but IMO you’ll be better-served by writing ANY tests and learning from there.
The simplest possible ExUnit test could look something like this:
defmodule AssertionTest do
use ExUnit.Case
test "the truth" do
result = 2 + 2
assert result == 4
end
end
There are plenty of tools in the ExUnit toolbox to make writing related tests more efficient (setup blocks, tags, etc) but none of them are required to get started.
If you aren’t writing tests currently, then presumably you are running your code somehow to verify that it works the way you intend - for instance, by poking around in iex.
One approach to start writing tests is to literally just PASTE that poking around into a test block; that way you can run them over and over again without having to manually set things up or worry about missing steps.
At the beginning, copy-paste with wild abandon - you’ll eventually want to reduce duplication, but it will be much easier to spot “what’s common” vs “what’s unique” when there are more examples to review.
This realization is what got me into testing and what sold me on “test first” (which I am not religious about). Afterwards I have a written test that I can choose to keep or not. I still poke around in the REPL to start, though.
Yep! It’s just one of those things that requires practice where you can sniff out what is necessary to test. I use tests to help guide design but ultimately I only really care about regression/acceptance tests.
A good metric to think about is how you feel about refactoring at any given moment. If the answer is “confident” then you probably have the “right” tests. If the answer is “not at all” then you don’t have enough tests. If the answer is “I’m afraid to because of all the tests I’m going to have to fix,” then you have too many tests/too fined-grained tests. This is, of course, isn’t perfect in of itself. Unfortunately it really is one of those complex topics you just have to start doing to get a feel for because ya, you’re going to get a lot of conflicting advice.
As you yourself said below, regression & acceptance tests are super important. And as I said above, I got “sold” on testing when I realized that I liked something that worked and didn’t want it to stop working so I… discovered testing back then, long long time ago.
So a repeated message to OP: don’t get religious about testing, just test what you think is important to keep working in your project.
Yep, that’s more or less what I’m getting at. I wouldn’t want to work that way, but lots of people have very different ways of working and somehow still produce profitable software. There’s so much context around all of this stuff but people don’t like hearing that and just want to be told The One True Way.
Yep, agreed, we should throw away any religion-like thinking from our profession. The only hill I am willing to die on these days is – strong static typing. I’ve witnessed the bug reduction of that first-hand, a number of times, and I am now tired of replicating the work that really smart and good compilers do, with my own hands, in dynamic languages like Elixir.
But I can’t deny Elixir’s REPL and very strong poke-ability (if that’s a word) are still keeping me hooked to it. And being able to just get inside a k8s pod and connect to the real prod DB and fix a problem with Elixir code (and not with Postgres SQL statements inside psql) is something I am still not willing to give up – though I haven’t given a fair chance to the alternatives either…
But it’s still a coin toss. F.ex. we like fiddling in iex and prototype stuff and I cringed not having that in Rust… but in Rust you are thus encouraged to write your stuff in a modular and decoupled manner and have tests for it almost right away, and this flow becomes your “iex”. Because if you don’t do it like that then you can code for a full week without being able to see if your idea works. But if you write in certain ways you can have almost the same iteration speed as with Elixir.
Both approaches netted me very good software so again, in the end it’s just preference. I at least still haven’t found OneTrueWay™ to do things. I lean on the side of messing around in the REPL but I’ve seen several other approaches and they worked just as good as well.
That’s funny because I come from startup environments where tests are not written usually because “we don’t have time”, which results in at least 3x time spent afterwards on debugging stuff, stuff that is as stupid as passing parameters in wrong order for a function.
All software should be written with tests, otherwise development velocity is not possible, that is true for 100% of untested projects I worked on.
I cringe at this as much as you do but for the 20-year old MBAs that make the calls that’s valuable because it reduces time to market.
And sadly for me and you and others of the same mind, this actually works. If you produce a product that works well 70% of the time and people pay for it, they are willing to forgive its defects until you fix them (if you ever do).
Maybe we’re just badly adapted to this world, I fear sometimes.
The reality is more simple, software industry has such a huge margin of profit, that it is actually encouraged to write bad software, just later ti switch to yet another sponsored solution that fixes it all.
Imagine if all concurrent java codebases were to run stable on erlang VM, how many artists were to lose their paychecks?
Well that’s kind of what I said – this approach that you and me and others hate actually works in terms of a business model. Which means that people will practice it until it stops working.
As a funny (second hand) side story, my friend told me a pretty ridiculous story from when he worked on a very large enterprise application. He was advocating for doing some preparatory refactoring to make a new feature easier to implement. The response was, “We don’t refactor here because if we do then we’re required to run ALL the tests.” My friend asked why that was a big deal and how long that would take and the answer was “3 months.” “Running all the tests” meant having the QA team work through a playbook of thousands-upon-thousands of hand-written scenarios. Whatever works, I guess
I look at my code and see what things I expect to do. I may also add tests for things I expect the code NOT to do. Then I make a test for each of those things.
Like, say you have an add/2 function that adds 2 integers. What do you want to be sure will happen when the function will run?
You want it to:
Give the correct result
Raise an ArgumentError if it does not receive integers as its inputs
So each of those is a testable thing, and you could have a test block for each item (I would give more in-depth examples but I’m on mobile).
So it’s basically like that for each “thing” your code does. Every time you run the tests, you are essentially writing some (test) code that ensures that the (production) code does what you think it does. Your tests are like little minions that ensure that nothing breaks when you change something.
Your tests are like a mirror image of your code. In the code you implement a given feature. In the tests, you verify that each feature works as expected.
Typically, I prefer to write a given feature or module, then write the tests (as opposed to writing one piece of code, then one test, etc.).
With good tests in place, if you change the code and tests break, you’ll know that either your code is wrong and need to be fixed, or the tests have become wrong and need to be fixed.
