Good Elixir TDD resources?

With TDD you can write and rewrite a function until it passes your test while you are not understanding what you are doing. It is important to understand what you need. When you can formulate (above code level) what you need you can write maximally comprehensible code (and design the best tests).

1 Like

Can you give a specific example on what exactly are you mocking?

Databases, APIs, Cache, basically entire services. The idea here (a according to the elixir community ) is to create a contract and then create an implementation for that contract as well as a mock ( or a fake implementation ).

Then depending on the environment you either use the real object or a mock that implements the contract. ( do note that by definition, your real object also implements the contract ).

I am still not convinced with mocks, I pretty much prefer to simply pass the functions I am going to need down the line ( Functional DI, shout out to @peerreynders for the video link ).

The issue with this is that your higher level functions will have a ton of dependencies, no matter what … So I guess this is where the argument for mocks could take place …

Why do you need test/mock database (or 3rd party api, cache and etc.)? Given that you’re not writing database, 3rd party api or cache library.

You either receive correct response from them or you don’t. You need to mock data, not services.

But that’s just my (incorrect?) opinion and one of the main reasons I’ve fell in love with fp/elixir.
It’s just functions (grouped in modules) that work on data - simple and elegant.

You need to mock them so you can do what we call “contract tests”. In short you need to:

  1. make sure you call the contract with the correct parameters
  2. make sure you handle the data returned by those who implement the contract correctly

If you don’t understand what i mean by any of this, I recommend you watch “Integration tests are a scam”.

1st item is not possible in elixir, since you can’t call function with wrong parameters (it won’t compile). By “wrong” I mean too much/too little params.

This leaves us to 2nd item - making sure you handle data correctly. That’s why you mock data, not services.

Btw:
I googled “integration tests are a scam” because that title seemed strange. The title appeared a mistake: https://blog.thecodewhisperer.com/permalink/clearing-up-the-integrated-tests-scam

I am not talking about a signature ( you even have spec to help with that ) I am talking about calling the contract with params that make no sense. Like wrong values and such.

Really, go watch the video conference. It will help you understand the context of this discussion:

Property testing to the rescue! :slight_smile:

Edit: also dialyzer.

@Egis Dialyzer won’t save you. In fact, if you are using Elixir 1.7.X it will even hurt you, as it doesn’t even work.
Even if ti did work, it wouldn’t matter. Dialyzer can’t understand that a method being called with “foo” should be instead called with “baz”. For Dialyzer, both are strings and the function’s signature is correct, so it passes.

As for property testing, I am not convinced. I am actually reading a book that mentions it ( The Little Elixir and OTP Guidebook ) but the examples are rather contrived and perfected to make sense using property testing. The applications to the real world are … considerably more limited.

Still, this is a discussion for another topic. I don’t think property testing is a good as people say ( feel free to create a new discussion referring me if you want to try and convince me) but I am still studying it’s potential applications.

This discussion is about TDD and good resources for those who value it.

Please do not tell it like this, as dialyzer works with 1.7.x if the OTP versions match. This is not an issue of dialyzer or elixir, this is an issue of installing the correct things in the correct versions.

This sounds to me as if the types haven’t specified correctly.

Sad. You should give it a go. For algorithms that have well defined properties this is a nice thing.

Eg. list == list |> reverse |> reverse or a + b == b + a or len(a++b) == len(a) + len(b). Having those is really nice. Of course it requires a different set of thinking than example based testing.

No it doesn’t: Which version of elixir-otp-erlang can we use with dialyzer? - Stack Overflow

Dialyzer checks types. Doesn’t check values. I could have a function add return 1+1=3 and it would pass. Why? both are numbers, return a number and the function takes the correct number of parameters. It is still wrong. Dialyzer doesn’t know any better because it doesn’t check values. That’s the job of TDD. To say that Dialyzer can replace TDD is just flat out wrong. It can help, sure, but never replace.

Yo are correct. However I found two main issues:

  1. Few times we have code that deals with nice algorithms that deal with very well defined properties. Sure, if you want to re-invent the wheel and reverse lists it will work fine but real life apps are messy and full of side effects and edge cases that don’t quite fit into PT.
  2. The code you write with PT is actually … quite complex. I am not even talking about recursive generators nor trees, but even the most simple things, like applying the reverse algorithm, require you to code both the solution and the counter solution. If I don’t know my solution works well, how am I supposed to know my counter-solution used to test my solution does? It makes no sense to me ( PS: counter solution = inverse function, inverse algorithm, equivalent of the undo operation ).

I am not giving up on PT, not yet. I want to delve deeper into it, however at this point in time, I am still not convinced :stuck_out_tongue:

But power to you if you make it work for you. Perhaps you have some resources I could use?

The counter should generally be part of the API as well for simple property tests. However property tests are not just about reverse testing but about invariant testing, like have it generate inputs into a state system, test that the state of it is valid each time and make sure it ends at a proper place given the inputs, even without knowing the values you catch a significant amount of bugs.

As a good example of state testing see the Elevator example for the C side of the QuickCheck library (which uses Erlang to do the testing) for how to think about it. :slight_smile:

Aspect: Testing as One Driver of Design

My exposure to property testing is minimal (e.g. worked through an elaborate a QuickCheck demonstration in Haskell) and for the time being Elixir Forum Property-Based Testing with PropEr, Erlang, and Elixir is a bit of a specialist topic for me to fully commit - but the reason it is staying on my radar is that I suspect that beyond the primary benefits it may impose beneficial design pressures on the code.

you find a rule that dictates the behavior that should always be the same no matter what sample input you give to your program, and encode that into some executable code—a property.

  1. In order to use property based testing the STU has to be designed against somewhat coherent (and explicitly stated) rules that the tests can be based on (now if the same was only true for “business” logic). So if you want to leverage property based testing it seems the need for “rules” improves coherence. The flip side is that if you can’t be bothered to discover/refine those “rules”, you are not going to see the value in property based testing. With example based testing there is never any real incentive to unify fragmented, incoherent logic.

  2. While this rant states that “You Don’t Need Referential Transparency” I’d be inclined to believe that property based testing would drive you towards preferring referential transparency.

However I still see property based testing as a rather specialized tool in the “testing” part of software development, likely useful for micro tests, possibly useful with collaboration tests, likely not appropriate for contract tests.

Some approaches to testing can be inefficient/ineffective (similar to the situation in process and documentation). Hypothetically property based testing should be more efficient than example based testing because it can cover more values with less effort. But the issue is that it can’t replace example based testing.

Given that developers will be exposed to example based testing first (and testing is a means to an end, rather than an end in itself - unless you are specializing in testing) they may never get exposed to property based testing.

Then there is the possibility that property based testing is only applicable and effective for code bases that already meet certain quality standards.

Which begs the question: could some software benefit from being “perfected” (i.e. designed) to be more amenable to property based testing and would that improve the software’s design from functional and maintenance perspective?

1 Like

Chiming in, as I am also a big fan of TDD but not fanatical in my implementation of it.

I have experienced the same thing @Fl4m3Ph03n1x , testing in Elixir is requires a lot of arcane knowledge that will not arise from getting into the code and writing the tests. Elixir lacks a popular opinionated testing framework and it is easy to do wrong, or to architect your code base to make testing a slog.

I’m working on a blog post about this exact subject, so I will share some of my notes.

Macros make the dream work

Keep in mind: Macros make the dream work! They let you write amazing abstractions for your code and I highly recommend getting VERY familiar.

There is a book by THE Chris McCord https://pragprog.com/book/cmelixir/metaprogramming-elixir

The (new) Facade Pattern

First and foremost, there is a pattern we have yet to name but is mentioned over and over in the community to enable clean mocking and testing. You can find resources on it here:

Mocking using a fake server process

a good reference impl of this is bamboo.

Other resources

Additionally, I do property based checking and macro away as much boilerplate as I can.

1 Like

Hi Pedro,

Probably I am bit late here, but I was just googling for “TDD Elixir” and I found this thread. I am deeply interested in topics like these.

Few years ago I started a Twitch Channel with the intent to show others how it is possible to apply practices like Clean Code, TDD and Refactoring as a support to learn a new programming language.

I decided to learn Elixir: starting from the basics of the language, the I did some programming session on Code Katas, and at the end I had the chance to push some little contributions to different open source projects, meet and learn from more experienced people. In general, an 100% nice experience! Here you can access to several examples of my journey in learning Elixir with TDD.

As a result, I wanted to create a talk (unfortunately it’s only available in Italian) where I shared the process that I followed to learn a new programming language doing TDD:


Eventually, I would like to propose an updated version of this talk, in english!

I remember also a talk from @gpad on TDD and Elixir, from the last Code Beam STO 2019

Now, after more than one year I started to work for a company where we play daily with Elixir and I learned something more about how is it possible to apply TDD (new tools, new libraries, what works, what don’t, etc …). And it’s time to share something, again.

I will give a speech at the next Beam Languages United - Stockholm Meetup with a practical example of TDD Outside-In in Elixir.

The session should be recorded and then uploaded on YouTube, from the organizers of the meetup. I will ask.

By the way, I will then upload an offline session on YouTube also, with only the desktop, the code and my voice. Something similar to this.

I am still learning, it’s an interesting journey and I am absolute available to share experience with anyone else interested in these topics.

1 Like

Hello @joebew42 I am happy to know you are also interested in the Elixir meetup, I truly hope you have a great experience.

Regarding the talks, I recommend you post them to the Talks section of this forum, where more people are likely to see them!

1 Like

At this point what is your favorite Phoenix book? Elixir book?

Hi @mchean,

My favorite Phoenix book is Programming Phoenix >= 1.4. Elixir books are Programming Elixir >= 1.6 and Elixir in Action, 2nd ed when starting out and I’m really enjoying Designing Elixir Systems with OTP for more advanced architectural patterns. Finally Programming Ecto is excellent.

In my earlier post (since the thread relates to TDD specifically) I also recommended Mastering Phoenix Framework since @shankardevy uses TDD on every iteration including E2E testing with Hound. Other books cover testing but I haven’t found any that focus on it as comprehensively as he does. Hope this helps!

2 Likes

Just to add that the book

is free to read here…

https://shankardevy.com/phoenix-inside-out-mpf/#mastering-phoenix-framework

as mentionned in this post.

1 Like