I’m using Wallaby (0.17.0) for acceptance tests with phantomjs (2.1.1), and the latest Elixir (1.4.5) and Erlang (20). I’ve written a few acceptance tests, each of which passes when run individually, but when I run them together (via mix test test/acceptance) they fail because things that should be on the page aren’t. It’s as if all the tests are running in the same browser, even though Wallaby should “manage multiple browsers for you.”
I believe I’ve set everything up according to the setup section of the README, although I’m not running the tests with async: true because my app uses sockets and Phoenix.Ecto.SQL.Sandbox doesn’t play well with async. To be clear, the tests also fail with async: true, I just also get DBConnection.OwnershipErrors in each when attempting to connect to the socket.
Is there anything else necessary to ensure each test is run in its own phantomjs browser, or to clear out sessions between test runs?
I’ve posted this to StackOverflow as well in someone would prefer to answer there for the points 
Can you reproduce it in a sample application? As long as you call Wallaby.start_session per setup and have async: false, you should be covered.
I’ll try to reproduce it in a sample application, but in the meantime will note that I was able to resolve the issue (and get all Wallaby tests passing when run together with mix test test/acceptance) by manually logging out (click(link("Log Out"))) at the end of each test case.
I store an authentication token in LocalStorage, and when I print it to the page (and take a screenshot in each test case), I can see that the token is the same for all of my test cases, even though I’m calling Wallaby.start_session before each test case. I would assume that, as a result of this PR, Wallaby sessions should not be sharing LocalStorage:
https://github.com/keathley/wallaby/pull/93
I’ll continue investigating…
I believe this may be a bug with Wallaby. When I IO.puts both the session.id and result of execute_script("return localStorage.getItem('token')" at the start of each test, it’s clear that each test is run in a differens phantomjs session (the ids differ) but that the token I’m setting with localStorage.setItem is persisting across sessions.
I’ve reproduced this behavior in a minimal sample app: https://github.com/stevegrossi/wallaby_test and have submitted an issue to Wallaby: https://github.com/keathley/wallaby/issues/274
1 Like
