I’m using Wallaby (0.17.0) for acceptance tests with phantomjs (2.1.1), and the latest Elixir (1.4.5) and Erlang (20). I’ve written a few acceptance tests, each of which passes when run individually, but when I run them together (via
mix test test/acceptance) they fail because things that should be on the page aren’t. It’s as if all the tests are running in the same browser, even though Wallaby should “manage multiple browsers for you.”
I believe I’ve set everything up according to the setup section of the README, although I’m not running the tests with
async: true because my app uses sockets and
Phoenix.Ecto.SQL.Sandbox doesn’t play well with async. To be clear, the tests also fail with
async: true, I just also get
DBConnection.OwnershipErrors in each when attempting to connect to the socket.
Is there anything else necessary to ensure each test is run in its own phantomjs browser, or to clear out sessions between test runs?
I’ve posted this to StackOverflow as well in someone would prefer to answer there for the points
Can you reproduce it in a sample application? As long as you call
Wallaby.start_session per setup and have
async: false, you should be covered.
I’ll try to reproduce it in a sample application, but in the meantime will note that I was able to resolve the issue (and get all Wallaby tests passing when run together with
mix test test/acceptance) by manually logging out (
click(link("Log Out"))) at the end of each test case.
I store an authentication token in LocalStorage, and when I print it to the page (and take a screenshot in each test case), I can see that the token is the same for all of my test cases, even though I’m calling
Wallaby.start_session before each test case. I would assume that, as a result of this PR, Wallaby sessions should not be sharing LocalStorage:
I’ll continue investigating…
I believe this may be a bug with Wallaby. When I
IO.puts both the
session.id and result of
execute_script("return localStorage.getItem('token')" at the start of each test, it’s clear that each test is run in a differens phantomjs session (the ids differ) but that the
token I’m setting with
localStorage.setItem is persisting across sessions.
I’ve reproduced this behavior in a minimal sample app: https://github.com/stevegrossi/wallaby_test and have submitted an issue to Wallaby: https://github.com/keathley/wallaby/issues/274