Improving LiveView test reading & writing with a set of helper functions

Hello, all! I want to share a pattern that I have found useful and get your feedback. I am relatively new to LiveView, so I am doing so humbly at risk of beginners naivety.

In short, as I began writing tests of LiveViews, I found it difficult to keep the different parts of the test state organized in my mind. Those being: 1) the conn, 2) the “live”, and 3) the “html” doc that is often asserted against. This would distract me from the actual interactions that the test is meant to describe.

Here is a brief example (contrived, but hopefully realistic):

import Phoenix.LiveViewTest

test "live view", %{conn: conn} do
  {:ok, live, html} = live(conn, "/")
  assert html =~ "New"
  assert live |> element("a.new") |> render_click() =~ "Enter new"
  assert_patch(live, "/new")
  {:ok, _, html} = live |> form(".form", @attrs) |> render_submit() |> follow_redirect(conn)
  assert html =~ "success"
end

By my judgement, this is pretty typical of LiveView tests. In the test, the value that is being handled switches between the conn (starting the view and following redirect), the live view (clicking the link and submitting the form) and the HTML docs (asserting initial page content and changes as links and forms are interacted with). The clicks and forms I found particularly confusing, because the data of focus changes from live view to HTML through the pipeline. Ultimately I just want to focus on the user interactions.

That’s where some helper functions come in. Essentially their purpose is to bundle this context together and make everything a pipeline so that the reader is relieved of mentally juggling the state.

Here’s the same example refactoring with these helpers:

use Iamvery.Phoenix.LiveView.TestHelpers

test "live view", %{conn: conn} do
  start(conn, "/")
  |> assert_html("New")
  |> click("a.new")
  |> assert_html("Enter new")
  |> assert_path("/new")
  |> submit_form(".form", @attrs)
  |> assert_html("success")
end

As you can see, I’ve bundled these up (for now) as a part of a utility package that I maintain, but if they’re well-received, I could see them becoming a part of the LiveView package itself. With some more work, of course, they’re not meant to be a complete set at this time.

Here is a link to the package source and a blog post I put together as well:

Let me know what you think!

6 Likes

Also, fwiw, the only thing I’m really concerned about here is developer experience. I’m not deeply attached to the naming or API of any of this. In fact, I all but assume there’s a better way to express and implement the purpose given more eyes and experience on the problem itself.

1 Like

I love it, really cool idea! Nice work :clap:

  1. When a test fails, do I see which step in pipeline caused it to fail?

  2. I would personally love to have an assert_flash :slight_smile:

3 Likes

Great question! And the answer is: There is room for improvement. Honestly, I’m not in love with the way the macro generations functions. I didn’t put a ton of thought into it, so there’s probably a better API for the helpers.

Here’s an example failure:

  1) test copying a stream from source (MyApp.SomeTest)
     test/features/some_test.exs:13
     Assertion with =~ failed
     code:  assert html =~ expected_html
     left:  "<html>..."
     right: "lolwat"
     stacktrace:
       test/features/some_test.exs:4: MyApp.SomeTest.assert_html/2
       test/features/some_test.exs:28: (test)

The first line in the trace is the use call, but the second line does point to the correct place in the test file, i.e. the call to assert_html.

Great idea! I’m sure there are lot’s of other things that could be helpful :slight_smile:

1 Like

Some quick reactions:

  • asserting on the HTML with =~ is brittle if your data sometimes contains characters that are escaped in HTML; a favorite offender is Faker which will occasionally generate last names like O'Henry which become O&apos;Henry. I’ve personally been using has_element? on the view which deals with details like that automatically.

  • having the :ok part in the function heads feels strange; if the call to live/2 in start doesn’t return the expected shape, it will result in a MatchError that points at the next function in the chain.

  • I have a vague recollection that stacktraces didn’t always used to point to the correct line in older Elixir versions, but I don’t have any evidence to support that (and might be misremembering a factoid about Ruby method chains!). That may be part of why this style isn’t widely used.

1 Like

I ran into that too, which is why I included an escape/1 function that’s just a delegate to Plug.HTML.html_escape/1. It can be used like:

|> assert_html(escape("can't be saved"))

More generally though, I also suspected that =~ was a little too prescriptive for a function with such a general name as “assert”. Perhaps match_html is a better name? Or coming up with a way of providing the full strategy. Here’s a (probably bad) example:

|> assert_html(& &1 =~ escape("can't be saved"))

Update: Here’s an issue for this idea Is assert_html too prescriptive? · Issue #4 · iamvery/iamvery-elixir · GitHub

Great feedback! That makes a lot of sense to me. For whatever reason this hasn’t come up yet in my application of the pattern, but my guess would be to provide more function heads to handle not-OK scenarios in the pipeline. I’ll try to find some practical examples and consider solutions.

Update: I put this in an issue What if things are not :ok? · Issue #11 · iamvery/skipper · GitHub.

I would love to get feedback on other ways to include the helpers. Something still doesn’t sit right with me on generating the functions with the “using” macro, but other efforts fell flat at the time. Still very open to improving that.

Thanks for the thoughtful reply!

I started an issue about the assumed :ok in the function heads. What if things are not :ok? · Issue #3 · iamvery/iamvery-elixir · GitHub

I’m just finding this now—it’s cool! I like it! I want it!

Something I’d like to have which addresses some concerns above would be assert_html_within/3:

start(conn, "/")
|> submit_form("#new-message-form", %{body: "Oh hi there"})
|> assert_html_within("#messages-container", "Oh hi there")

@sodapopcan awesome! I actually made some changes just a few days ago that includes this. Check out the release candidate:

Pretty sure the specific thing you are looking for was already in assert_visible/2, but this RC is much better and includes docs.

You can do basically exactly what you’re referring to, and since it uses element/2 under the hood, it gives you nicely scoped errors when the assertions fail, e.g.

Assertion with =~ failed
     code:  assert element_html =~ expected_html
     left:  "<div data-test=\"placement\">Foo-bar</div>"
     right: "Foobar"
     stacktrace:
       (iamvery 0.11.0-rc1) lib/iamvery/phoenix/live_view/test_helpers.ex:124: Iamvery.Phoenix.LiveView.TestHelpers.assert_visible/3
       test/foo_web/live/foo_live_test.exs:273: (test)

Hopefully I can release this sooner rather than later. It would be slightly breaking, and I’m not sure how many folks are using the library. Further, I’m wondering if it’s time to go ahead and split it off as its own lib. I’d love to get the attention of the LiveView core team who may be interested in bringing these helpers into the library proper. Probably time I updated the blog post.

1 Like

Ah yes, indeed it does! I don’t, however, think assert_visible is the best name. I know it’s a bit different for LiveView but generally in these acceptance-style tests, assert_visible means that an element is visible on the page, ie, not hidden by CSS or JS, whereas this function is asserting that HTML was patched into the DOM.

Just cut a 0.1.0 release and then you can break it to your heart’s content and no one can complain until you get to 1.0! :slight_smile:

Ya, go for it!

I wouldn’t worry about this until you get to 1.0. Not to be negative but I also wouldn’t count on it as the current API is quite good and I couldn’t see them wanting to maintain two styles. These types of things are best left up to the community to maintain!

Lastly another suggestion: I think a better name for start would be visit. This could be problematic if people mix LV and Wallaby tests in the same module but, like, people really shouldn’t do that :upside_down_face:

Fair, I had opened a issue similar to this 🤔 The assert_visible functions still match non-visible markup · Issue #8 · iamvery/skipper · GitHub.

It’s true that pre-release minor versions are breakable. Just trying to be a good citizen. I’m using the RC for now.

:slight_smile:

:heart:

I like it a lot! Consider: Rename `start` function to `visit` · Issue #6 · iamvery/skipper · GitHub

1 Like

Awesome!

Lemme know if you split off a new project or what and I’m happy to start using it in my current project and give feedback.

Sorry I didn’t look more closely at the repo issues as I was leaving work and trying to write quickly (and caused me to write “I don’t think assert_visible isn’t the best good name.” which I had to edit—thank you for not quoting that part, lol).

1 Like

PS, wanted to point out you can just call it a release and not a pre-release. Case in point, LiveView is 5 years old and still not at v1. A lot of useful libs are still not at v1, you just gotta get people using it and contributing!

1 Like

All: any recommendation on naming for such a package? “live view test helpers” is somewhat descriptive, but pretty vague…

  • live_view_testing
  • live_view_test_pipeline
  • liveline
  • live_line
  • ???

Interesting… GitHub - batteries-included/heyya: Heyya the snapshot testing utility for Phoenix framework components

Since I’m partial to clever names, I like liveline. I did a quick search for what you call testing IRL pipelines and “hydrostatic test” came up. Hydrostatic could be a fun name as the first sentence I found describing it was: “Hydrostatic tests are some of the most common pipeline testing methods.” :joy: It’s not as discoverable, though, at least by name alone.

EDIT: Snapshot testing is not for me but it’s cool to know that Heya exists!

1 Like

How about testline?

1 Like

Good ideas! I thought of skipper today:

  1. it’s a “helper” / “driver” (like a boat’s skipper)
  2. and it enables you to “skip” some of the liveview internals in tests

Certainly this falls into the category of “clever”, but I feel a little different about clever naming when it comes to public projects.

  • i like having “test” in the name, but it’s hard to pull off. i think this does it pretty well
  • i like the -line suffix that feels a little bit like “throw me a line”
  • it’s pretty searchable
  • i’m not sure any intended reference to “pipeline” would be noticed

overall, i like it!

Ha, I’m the totally opposite when it comes to clever: I’m all for it for libraries but I can’t stand when people name internal business concepts dumb names that I can’t decipher :sweat_smile:

I left some github comments and have a couple of more but I’m getting 500s from github rn.

PS: I like skipper. testline is good too.

Ya I saw some of them. Looks like GH is having an incident this morning. Thanks for all the great input!

1 Like