Good Elixir TDD resources?

Learning Questions & Discussions Resources Chat/Questions
, ,
1

Background

After following the communitiy suggestion, I bought the Elixir in Action 2nd Edition book and I am about to finish it now.

I must say it really helped me out and I am quite glad I bought it. Now I need to look forward towards my next endeavor - testing.

Why testing?

Due to the nature of my work, I do TDD, which is of huge help to me. In my new job using Elixir I can see that TDD also is a good fit for the company and for it’s projects so now I need a way to learn on how to do TDD with Elixir.

What am I looking for?

I am looking for a book, a video course or any set of resources that can teach me how to do TDD the Elixir way. Paying is not an issue as long as it falls bellow the 20 euros mark ( or 25 dollars ).

You guys know this community and its resources the best, what would you recommend?

13 Likes
2

There are a lot of books with a dedicated chapter on testing, but a book dedicated only on tdd, not really…

I think there is a tdd based development in Phoenix Inside Out, but I did not read it, and maybe someone can confirm.

There is a book dedicated on testing only, it is

But it’s all about property based testing :slight_smile: which seems to be a new way of testing.

Anyway You will see that testing in a functional world is a breeze (mostly no side effect), and tests are incredibly fast (run concurrently).

2 Likes
3

Are there any other resources you know of? I have the feeling there is a distinct lack of resources about testing in this community.

4

You also have ExUnit Official Documentation to start with.

I do not know a book dedicated to TDD in Elixir, but testing is not an issue for me, it’s more like a pleasure after using RSpec/Rails. It’s really damn fast. You don’t even have time to get a coffee while running the full tests :slight_smile:

But You need to test different things, like macro testing, OTP testing (You can assert that a process has/has not being called).

Others resources You might find useful.

You might ping @peerreynders for a wise advice on the subject.

5 Likes
5

I can confirm the author @shankardevy follows TDD in his Mastering Phoenix Framework book where you develop an e-commerce site called Mango. Each chapter begins with an iteration planning section including user stories, then writing acceptance (browser) tests using Hound, then ExUnit tests, finally writing the implementation until the tests pass green. You can see the code of the finished product/site here: GitHub - shankardevy/mango: Ecommerce site that you will build while learning Mastering Phoenix Framework

I’ve collected pretty much all Elixir/Phoenix books by this point and his is the only one I’ve found that is strict about TDD/writing tests first and I have to highly recommend it.

2 Likes
6

I’m not exactly sure what you are expecting.

GOTO 2015 • Agile is Dead • Pragmatic Dave Thomas:

(I seem to recall that he stated of typically having about 60% test coverage.)

Anyway - TDD is a practice and if you have a competent testing framework like ExUnit you should be good to go - what other language specific support do you expect? And thanks to the Erlang community there is PropEr to replace some tedious static test cases.

Keep in mind that any practice should not be taken to the extreme of religion or dogma - that is counter-productive.

Practice TDD when it makes sense but keep in mind that it is just as important to delete those tests once they taught you all they can so that you don’t have to pay the cost of maintaining them (and don’t make the mistake of skipping Refactor in: Red, Green, Refactor - Why Refactor? - Economics).

DevTernity 2017: Ian Cooper - TDD, Where Did It All Go Wrong

5 Likes
7

So, in JavaScript where I originally come from, I used to do TDD using a variety of tools: Mocha, Sinon, Chai, supertest, etc.

These tools used to help me with spies, stubs and mocks, servers and you name it. Some people will tell you that they just use the Node.js assert, and only use that function to test everything and anything, but I would respectfully disagree with them.

Regardless of that, I learn how to do dependency injection, inversion of control and a myriad of other techniques using such tools to test my applications. As a consequence, none of the apps I tested well ever had issues. In fact, they saved my skin.

Oh, so you are a fanatic!

Well, well, let’s calm down. I have actually seen GOTO 2015 • Agile is Dead • Pragmatic Dave Thomas before. I don’t agree with many of his opinions, but he also makes some pretty good points I agree with.

I don’t see myself as a fanatic. I am of the belief that having 100% coverage is ncie but I will definitely not fret if I don’t reach it. I rather have tests that make sense rather than tests that just bump up a random percentage.

However, given the projects I am in and how much I have benefit from TDD in the past, I am of the strong belief that applying it in my projects makes sense and fits.

I don’t say TDD is the holy grail of salvation. It won’t work for everyone. But for the things I work on, it’s actually pretty great.

Fl4m3 Snow, you know nothing!

I can pierce into some of your brains and see what you are thinking:

You only think you know TDD, but you don’t. In reality you have depended on a set of tools to do testing, while you needed only a basic assert, so without those tools you feel lost because you know nothing!

Well, I wouldn’t say I know nothing. The way I see it,I know some stuff, but I wouldn’t say I am a master either.

YEah, yeah, what do you want?

Well, I was looking for a resource that would present me with a similar set of tools to the one I used in the past. A library to fake spies and stubs and mocks, an assertion library, and explaining how to do tests with such tools and the conventions the Elixir community uses. I want to write Elixir tests after all, not JavaScript tests written in Elixir.

I hope this sheds some light on the kind of person I am and on what I am looking for! :stuck_out_tongue:

@kokolegorille For now, I just want to learn how to do TDD in Elixir. No Phoenix or frameworks. That’s for another time :smiley:

I do have to say that the ExUnit docs are rather … well for me, it is hard just by reading the docs to see how I can stitch up each function and make a coherent test suite. As for testing in FP… side effects are always present.

It is my challenge to isolate them and test them properly that led me to the magic world of Monads, which I am still considering ( but I take it most people here don’t really like Monads? ).

@tme_317
@kokolegorille
Also the book Mastering Elixir looks nice, but the testing chapter is mainly a warmup for how to test using Phoenix. Since we don’t use it in the project, I don’t believe it makes much sense.
Maybe I missed something from the online preview?

8

This one comes to mind: Why I use Tape Instead of Mocha & So Should You

Anyway Elixir has a set of "assert"s under ExUnit.Assertions - including ones to test the presence or absence of mailbox messages.

And there are utilities that people “just whip up” when and if the need arises, example: TaskAfter.

Sometimes it’s just more effective to create exactly what you need yourself, rather than to having to learn some other tool or more general library when you only intend to use 5% of its capability.

I learn how to do dependency injection, inversion of control and a myriad of other techniques using such tools to test my applications.

While those design concepts are helpful when it comes to testing and designing for testability, they are not testing centric but rather general software design concepts. For example: Inversion of Control Containers and the Dependency Injection pattern.

To get a notion of the general attitude towards mocking see Mocking and Explicit Contracts - and there is Mox for mocking behaviours (and there have been discussions around Mox like this one).

I do have to say that the ExUnit docs are rather … well for me, it is hard just by reading the docs to see how I can stitch up each function and make a coherent test suite. As for testing in FP… side effects are always present.

You could always try “learn by example” - e.g. pick a project (e.g. Plug or start small with ExUnits own tests) and investigate how testing is approached there. By the time you’re done, you’ll likely know the project as well.

1 Like
9

Already searched the forum with tdd as keyword? You will find BDD / TDD criticized . About the ruby community and testing: see links to DHH’s criticism in that thread also (I at least think I pasted them there).

10

Nope.

You can blame Steve Freeman for that answer, he planted it in my mind. We had a coffee (or, probably more likely, a beer) and were chatting about JMock (created by him and others to basically help out with “London School” style TDD-ing) versus all the other mocking libraries and frameworks that were out there.

A lot of users of the other frameworks mocked (couldn’t resist) JMock for being so “primitive”, sometimes going ad hominem, blaming its authors for not understand enough Java to do a “proper” implementation.

Knowing Steve, I knew that that was false. So I brought it up and he said that JMock was intentionally kept simple, and they tried to tune its functionality to do one thing:

Drive good designs.

The goal is that if you cannot JMock it, that’s not a library limitation, it’s a design smell in your code and you need to go and fix it. The other libraries don’t help you there, as they will allow you to mock pretty much anything under the sun (he advised that you should use JMock for your own code and grab one of the other ones in case you need to mock library stuff, because most libraries in Java are just horribly written).

That made something click in my mind, and since then - I think it was a bit under ten years ago - I’ve been looking at how people test in various languages; and they’re on to something. The more powerful the tools, the worse the test suites and the code design (Rails tops it; never, ever, listen to people called DHH for any programming advice). These libraries and frameworks can just walz through anything; they’re akin to just using heavy powertools while trying to learn fine woodworking.

I think that ExUnit is sufficient; the only thing that’s usually hard to test is a GenServer, or Agent, etc; either I call the server-side methods directly, or I pull the business logic out and then the GenServer itself just forwards messages and then becomes “obviously correct” - a term I use for code that doesn’t need testing. Dependencies can be passed in, it usually is not a lot of work. Most of the tools, like ex_mock, come with serious drawbacks and let you get away with murder. Don’t give in to their temptation :wink:

3 Likes
11

Many developers focus on unit testing (for clearity: “a level of software testing where individual units/ components of a software are tested. The purpose is to validate that each unit of the software performs as designed. A unit is the smallest testable part of any software. It usually has one or a few inputs and usually a single output.”). DHH warns against harming architecture while making software unit-testable. Coplien does that too: “If you find your testers splitting up functions to support the testing process, you’re destroying your systemarchitecture and code comprehension along with it. Test at a coarser level of granularity.”. And: “You’ll probably get better return on your investment by automating
integration tests, bug regression tests, and system tests than by automating unit tests.” See the link in my previous message for the references.

12

Andrea Leopardi and I are currently writing a book on testing elixir for PragProg. While it does not emphasize TDD, I’m a strict test-driver. I’m not sure what resources need to be dedicated specifically to TDD if you are already familiar with the concepts from another language.

Let me know how I can help and I’m happy to. I do not know of any specific guides or posts.

12 Likes
13

Oo nice! Looking forward to seeing it :023:

I would love to see it cover the topic from the ground up. So assuming that people have not tested previously - even if you speed through such parts. I think having beginner material not only widens the audience but would make the book feel more complete and accessible too.

I would not shy away from opinions either. As you’ve probably noticed there has been a lot of discussion about testing on the forum, and so referencing the position of others and allowing the user to follow their own path could be a nice take on the topic? Either way interested in hearing your plans for the book! :003:

Personally I read a book on testing when I was learning Ruby but never really bothered with testing. I’d like to change that with my Elixir apps :slight_smile:

1 Like
14

The thing I miss the most would be spies and stubs ( looking at sinon.js ). Most people end up creating mocks when in reality they don’t need them. It just happens they don’t know about spies for example.
I am a believer that when it comes to designing tests, code and architecture, the simplest solution is usually the best. That’s why in over 2 years of programming I have never used a mock.
In Elixir I find that the solution for everything is just a mock ( mocks should be used carefully ).

Another thing I miss are the basic concepts that made ( at least for me) TDD viable, such Dependency injection and Inversion of control. I know I how to them JavaScript style, but as I said before, I want to write Elixir instead of writing JavaScript code using Elixir.

A section detailing any of these would clearly up my interest as I have found little information regarding the application of such techniques in the Elixir community.

I have also recently learned about Elixir’s love for property testing which I am new at ( coming from other backgrounds it is the first time I hear about it). A comparison between TDD and PT or how they can be combined together would be interesting to me.

Do note these are my personal opinions based on my personal struggles with Elixir. I have been a seasoned developer in other languages, but coming here I feel there is a lot of adaption needed.

I may not be a good representative of your target audience, you may not be writing a book for people who moved to Elixir after years of JavaScript or something else.

Everyone is entitled to their opinion and as I stated, TDD / BDD is not the holy grail. It may not save you from your demons, but I assure you it has saved me from many ugly demons over the years. It works for me and for the work I do, so I’ll keep defending :stuck_out_tongue:

2 Likes
15

At the risk of stating the obvious:

  • When dealing with function values nothing much changes. Pass in your fake function instead of the real function and have the fake do what it needs to do.

  • However Elixir doesn’t have Objects. In JavaScript Objects conflate (mutable) state and behaviour. In Elixir state is stored in immutable data structures while behaviour resides in modules.

So it’s:

someValue = state.method(...params); // spread params array over arguments

versus

new_state = Module.function(old_state, params) # params is a parameter key list

So in Elixir the “fake object” situation may require a fake data structure (though at times you can simply use the real one) and will require a fake module, which means that your mockable boundary cannot hard code the module so that we can write:

new_state = apply(Module, :function, [old_state, params])

with the rough JavaScript equivalent:

someValue = (state.method).apply(state, params);

Now in Elixir that Module value has to come from somewhere and typically that is resolved at compile time (something that JavaScript doesn’t have) by grabbing the Module from the compilation environment - i.e. compilation in preparation for running tests versus running the application.

This is essentially what Mocking and Explicit Contracts and Mox are based on.

Consequently choosing mockable boundaries in Elixir is a much more deliberate process (hence “Explicit Contracts”) than it typically is in JavaScript with the ad hoc creation of injectable spies (where state has customized behaviour already bound to it).

(Another example)

3 Likes
16

I wonder whether building contractors have to defend using a double mitre saw over a skilsaw for certain jobs; Similarly, when I worked in the pharmaceutical industry, I never had lab managers argue the virtues of liquid over gas chromatography because they have their clear uses. I know that these sorts of discussions happen everywhere but us programmers seem particularly attached to just a subset of all possible tools.

TDD, BDD, test-after-the-fact, the-code-is-so-simple-I’m-not-even-gonna-test-it - all tools in your toolbox. Use them carefully; the trick is knowing what to apply when ;-). Usually, during my rare full days of coding, I go through them all.

2 Likes
17

With TDD you can write and rewrite a function until it passes your test while you are not understanding what you are doing. It is important to understand what you need. When you can formulate (above code level) what you need you can write maximally comprehensible code (and design the best tests).

1 Like
18

Can you give a specific example on what exactly are you mocking?

19

Databases, APIs, Cache, basically entire services. The idea here (a according to the elixir community ) is to create a contract and then create an implementation for that contract as well as a mock ( or a fake implementation ).

Then depending on the environment you either use the real object or a mock that implements the contract. ( do note that by definition, your real object also implements the contract ).

I am still not convinced with mocks, I pretty much prefer to simply pass the functions I am going to need down the line ( Functional DI, shout out to @peerreynders for the video link ).

The issue with this is that your higher level functions will have a ton of dependencies, no matter what … So I guess this is where the argument for mocks could take place …

20

Why do you need test/mock database (or 3rd party api, cache and etc.)? Given that you’re not writing database, 3rd party api or cache library.

You either receive correct response from them or you don’t. You need to mock data, not services.

But that’s just my (incorrect?) opinion and one of the main reasons I’ve fell in love with fp/elixir.
It’s just functions (grouped in modules) that work on data - simple and elegant.