Mirage - Browserless testing for Hologram pages and components

Now that I’ve used this in anger for a week, I’d like to introduce 0.1.0 of Mirage, a browerless testing framework for Hologram pages and components.

I still consider this library highly experimental, so please see “goals” below.

Motivation

There are a few things about LiveView that I love equally and one of those things is LiveViewTest. As a solo dev and avid TDD’r, I often start with a LiveView test and work my way down. Having something akin to a full browser test finish in milliseconds was one of the experiences that got me addicted to LiveView.

I’ve recently become extremely enamoured with Hologram and wanted a similar experience. With Hologram being isomorphic, it lends itself to getting even closer to that “almost a real browser test” than LiveView! Mirage aims to provide a similar experience and does so with an API similar to that of PhoenixTest.

Goals

This is my first heavily LLM-assisted library and would basically not exist (yet) without them. I’ve still put many, many hours into this, learning about Hologram internals, and deciding how this should work. This contains pieces that are everywhere from hand-written to totally vibed (though it’s mostly in-between those). Hologram is a very young framework, and it’s not (yet) written in a way that easily facilitates easily writing a library like Mirage. For that reason, Mirage not only contains code copied directly from Hologram, it has a high level re-implementation of Hologram’s client side Runtime in Elixir. While this is somewhat sustainable thanks to LLMs, it’s certainly not ideal.

Therefore, I would like to make it clear that, at least for now, Mirage is meant for the Hologram community to experiment with this type of testing library and figure out if it’s something we really want and what it should ultimately look/feel like. I know Bart would eventually like an e2e solution built into Hologram, but I ultimately want browerless more than anything, so I really wanted to start exploring it now.

Notes

I announced Mirage last week in the Hologram Discord. If anyone has taken it for a test drive yet, be aware that there is a significant breaking change in 0.1.0.

Examples

defmodule MyApp.HomePageTest do
  use ExUnit.Case, async: true
  use Mirage.Page

  test "sign up", %{server: server} do
    server
    |> visit(MyApp.HomePage, my_param: "some-param")
    |> click_link("Sign-up")
    |> fill_in("Name", with: "Bender Bending Rodríguez")
    |> fill_in("Password", with: "wanna-kill-all-humans?")
    |> click_button("Submit")
    |> assert_page(MyApp.WelcomePage)
    |> assert_has("p", "Welcome, Bender!")
  end
end

You can also test components in isolation:

defmodule MyApp.Components.PoplarTrackerTest do
  use ExUnit.Case, async: true
  use Mirage.Component

  test "it counts" do
    ~HOLO"""
    <MyApp.Components.PoplarTracker cid="counter" eaten={0}>
      <p>{@user.name} eats too many poplars.</p>
    </MyApp.Components.PoplarTracker>
    """
    |> mount({MyApp, user: current_user})
    |> click_button("Eat a poplar")
    |> assert_has("p", "Number of poplars eaten: 1")
  end
end

Get it!

11 Likes

Oops, I guess because I stupidly used my human brain instead of getting the machine to do it, we’re actually at v0.5.0, not v0.1.0 :upside_down_face:

Nice - I’m glad you are working on this! I agree that LiveViewTest is one of the best things about LV. I wouldn’t want to migrate over to hologram until it has a similar testing story. Being able to test if my page is working instantly without leaving the terminal is amazing.

I’m trying to wrap my ahead around how this could work in Hologram….

It seems like you would need to either:

  1. Run a JS interpreter in the test framework so that it can run the transpiled elixir → js code and apply dom updates / send events
  2. Capture the elixir before its transpiled to JS, and then build functions that can manipulate the DOM/send events with the frontend-elixir directly

From your README, it seems like you went with something like 2. Is that accurate? Naively, it seems like for that to work, you would need to essentially implement the entire browser API inside of elixir?

I haven’t tried hologram yet, so apologies if my questions don’t make sense :slight_smile:

Hey!

Mirage works by rendering your template into Hologram’s AST (which is its own thing). The test helpers then just walk that and execute the appropriate events on any nodes, calling the appropriate actions and commands (which often includes a chain of them). This means, however, that Mirage includes superficial re-implementation of Hologram’s client-side runtime in Elixir. The CSS selection code is taken from meeseeks with code that maps them to Hologram AST (this is the part that is fully vibe-coded in that my prompt was, “Just make this work” and I still haven’t looked at the resulting code yet).

So ya, porting Hologram’s client-side runtime is obviously far from ideal, but I’ve gained a decent understanding of it and LLMs make it quite tenable to keep up with it for now. There are some things Hologram could do to clean things up in Mirage (for example Hologram doesn’t expose a way to get its AST, only render to string, so that part had to be copied) but as far triggering events directly from Hologram’s AST, Hologram isn’t really built for that (although could be another way to do this I’m unaware of). My only thought of how it could work cleanly is if (when?) it gets to the point where it makes sense for Hologram’s client side JS to be written in Elixir. There would need to be a big demand for a library like this for that to get prioritized (Hologram has a pretty big roadmap) so for the time being, I’m happy to make sure Mirage stays in sync with Hologram’s runtime, but it’s definitely not good for long term sustainability.

Oh yes, speaking of the README I do need to give that some love (most documentation is something that is not LLM-generated).

There are some purposeful design decisions that I haven’t mentioned anywhere, mainly that commands in Hologram are async but in Mirage they are not. This is done for ease of my mental model, though of course something could fairly easily be implemented to fake a webserver… like some kind of “generic server” if you will… :thinking: :wink:

Congrats on the release, @sodapopcan! It’s exciting to see libraries starting to pop up around Hologram!

On testing in general: I do envision Hologram eventually shipping proper browser-based e2e tooling, but there’s absolutely a place for fast component output and behaviour testing alongside it. Honestly, I’d expect most tests in a typical Hologram app to be written that way - they’re cheap, fast, and cover the bulk of what you actually want to assert. You’d still want some portion of tests exercising the full pipeline end-to-end though, especially anywhere JS interop is involved, since that’s where the isomorphic story can leak.

I’m really curious to see what testing patterns emerge through Mirage as people use it on real projects. That kind of in-the-wild feedback is hard to get any other way, so I’ll definitely be watching how it evolves - and credit to you for taking the leap and figuring out so much of this on your own, especially given how rough the internals must have been to work against.

On making life easier for library authors: it’s something I care about and want to address, but some things genuinely need to happen in a specific order. A lot of this - middleware, permissions/authorization, proper e2e testing - intersects with the local-first direction Hologram is heading in, and local-first is pretty new territory for web dev, so the design space is hard to pin down until some foundational pieces settle. I’d rather not publish new APIs prematurely either - anything that ships has to be maintained, and I’d like to open up surfaces that fit the long-term vision instead of patching in extension points that might not survive. So bear with me on that front.

Thanks again for putting the work in on this - looking forward to seeing where it goes :slight_smile:

3 Likes

Absolutely to all of this. My strategy in LiveView is to treat LiveViewTests more like unit tests, testing single pages, usually with at most one redirect. I like these to be super thorough so that I can refactor and run the whole “unit” test suite frequently and not be met with a whole bunch of errors when I finally run the e2es. I use e2es for user journey tests that flow through multiple pages and, as a consequence, generally tests all the JS.

Working with internals has really not been that bad at all! LLMs make it relatively easy to ensure I’m covering my bases (although I know a few things are still missing). Please don’t get the impression that I’m even mildly trying to apply some passive aggressive pressure! I think this type of library is a perfect use-case for heavy LLM-use where, so long as its behaviour is correct, code quality doesn’t matter so much (although I still go through a lot of it and get either the LLM of myself to clean stuff up when the mess gets out of control). Of course, that “so long as its behaviour is correct” is incredibly important for a testing library, but if enough are interested in Mirage and use it, it should be easy to get any kinks sorted out.

For me the ultimate goal for Mirage would be for a to be invalidated by an official solution baked in Hologram, but no rush there! It also depends on how Hologram does this on whether Mirage would still need to exist or not. For example, LiveViewTest provides more “low level” primitives while PhoenixTest provides the higher level feature-test-like API that Mirage uses, so maybe it would make sense for Hologram to provide something more akin to LiveViewTest? I have no idea. There are lots of questions there, but the whole point of Mirage is so that I, and anyone who wants it, can have browless testing NOW and you can put off thinking about it for a while :slight_smile:

2 Likes