jivko-chobanov

jivko-chobanov

Criticize my idea for integration testing with automatically managed assertions of the whole HTML

I want to speed up my integration testing by making tests in the following way:

1/ setup by creating records in the database with fixed data. Even fixing (setting) the current time, so the inserted_at values are the same on each test run.

2/ do some actions (load a LV/non-LV page, click a link or trigger LV action)

3/ KEY PART: instead of multiple assertions of phrases or html tags/attributes, I call my CustomTesting.assert(html, call_identifier) function that compares the html output (the whole webpage) to the previously stored html output during the prevous successful test run for this particular assertion call (identified by call_identifier).

  • during the first run the html output is sent to the browser for me to see it and visually inspect that everything is fine. Then I click “Store” button at the very bottom to store this html output for future test runs.

  • when such assertion fails it will open two tabs in the browser, making a diff (like kdiff, git diff). The first tab will load the html from the test run and for each difference it will surround it with a div tag having a thick red border. In the second tab the same will happen for the expected html with green border. Below I will have a button “Store” to replace the expected html with the newly produced html if I want to.

4/ I can continue with more actions and CustomTesting.assert() calls.

Advantages that I see over using several (sometimes more than 10) small assertions to test that the key elements of the webpage are ok:

1/ I don’t have to write these mini assertions, I don’t have to update them manually when my main code changes. I don’t have to think what is important to assert and what isn’t and many other details like - is it possible in the sidebar to have the same phrase I assert for (now or in the future version of my app).

2/ The full html assertions are easy to manage even though they are brittle (easy to break). If I change my footer, for example, all the tests will fail, but in CustomTesting I can have a func to run all the tests and replace all the HTML output (because I know that the footer change is the only change and it will not break my actual code, so I don’t have to inspect each test manually).

3/ Manual inspections of failed tests happen in the browser with regions in red/green to guide me where the difference is. I can fix my main code, I can fix my tests setup/actions or I can click the “Store” button at the bottom to fix the test assertion.

What bothers me:

  • I can’t find similar approach or testing tool being used in any programming language (maybe there is). If there isn’t - there is probably a good reason why? The only similar thing I found is Snapshot Testing (an overkill in my opinion) - it takes snapshots of the page (like .jpg) and then compares the images.

Can you think of why this approach is a bad idea and stop me, before I start implementing it?

Most Liked

linusdm

linusdm

Maybe you can take some inspiration from mneme, which implements snapshot testing (but not entirely in the way you describe).

LostKobrakai

LostKobrakai

Snapshot testing (also found it called approval testing) is afaik the name for the more general idea of storing a previous result to compare to a later ones. The implementation of that idea might be comparing jpegs, html or really anything you can compare.

However I’m not aware of any library, which would implement it using the approach you described. With automated testing I think that approaches, which take manual input, are generally not as popular.

dimitarvp

dimitarvp

Some rare apps regenerate the CSRF hidden tag value often. If yours were to be one of them, a complete snapshot of the page will not be valuable for comparison.

I do sympathize that you don’t want to manually assert on a bunch of things but you can always make helpers for it? E.g. assert_breadcrumbs or assert_login_links etc.?

Where Next?

Popular in Discussions Top

PragTob
Hello everyone, I know we had quite some threads (read through lots of them) about background job processing but it remains a hotly deba...
New
WildYorkies
It seems that the more I read, the more I find Elixir users speaking about all the ways that Elixir is not good for x, y, and z use cases...
New
jeramyRR
This is an interesting article to read. Elixir’s performance, like usual, is excellent. However, it seems like the high CPU usage is co...
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New
sergio
There’s a new TIOBE index report that came out that shows Elixir is still not in the top 50 used languages. It also goes on to call Elix...
New
restack_oslo
Hello, Please pardon me for any faux paux. I am 46 and this is my first time on a forum of any kind. I wanted to to get answers from tho...
New
100phlecs
Are there any downsides, like perf issues, to putting all functional components in CoreComponents as long as you prefix it with the conte...
New
MarioFlach
Hello, I want to share a project I’ve been working on for a while: https://github.com/almightycouch/gitgud Background Some time ago I ...
New
fireproofsocks
This is more of a general question, but I’m wondering how other people in the community think about the pattern matching in function sign...
New
Markusxmr
Since Drab has been developed for a while in the open, introducing the Liveview functionality in a way it happend appears to undermine th...
New

Other popular topics Top

aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
Nvim
Anybody knows a comprehensive comparison of Django and Phoenix, thanks for the help. Where are they similar? Where do they differ the m...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New
komlanvi
Hi everyone, I was playing with phoenix liveView but I run into an issue. I have a form and want to validate each input text when the te...
New
Qqwy
Update: How to use the Blogs & Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3271 127089 1222
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New

Latest on Elixir Forum

Elixir Forum

We're in Beta

About us Mission Statement