Async Hound tests failing sporadically with ChromeDriver

Questions / Help
1

I’m having issues getting async Hound test cases to work while my synchronous Hound tests are working perfectly with ChromeDriver. I’ve searched the forums and GH issues and can’t find the answer and hoping someone has seen this.

Here are the simplified test cases I am trying to run concurrently:

defmodule MyAppWeb.HomePageFeatureTest do
  use MyAppWeb.HoundCase, async: true
  @moduletag :hound
  test "homepage renders with hero", %{} do
    navigate_to("/")
    assert visible_text({:css, ".hero-body > .container > .title"}) == "Welcome!"
  end
end

and the duplicate…

defmodule MyAppWeb.HomePageFeatureTest2 do
  use MyAppWeb.HoundCase, async: true
  @moduletag :hound
  test "homepage renders with hero", %{} do
    navigate_to("/")
    assert visible_text({:css, ".hero-body > .container > .title"}) == "Welcome!"
  end
end

Here is the used HoundCase module:

defmodule MyAppWeb.HoundCase do
  use ExUnit.CaseTemplate

  using do
    quote do
      use Hound.Helpers
      alias MyApp.Repo
      import Ecto
      import Ecto.Changeset
      import Ecto.Query
      import MyAppWeb.Router.Helpers
      @endpoint MyAppWeb.Endpoint
    end
  end

  setup tags do
    :ok = Ecto.Adapters.SQL.Sandbox.checkout(MyApp.Repo)
    unless tags[:async] do
      Ecto.Adapters.SQL.Sandbox.mode(MyApp.Repo, {:shared, self()})
    end

    metadata = Phoenix.Ecto.SQL.Sandbox.metadata_for(MyApp.Repo, self())
    Hound.start_session(additional_capabilities: %{
                          chromeOptions: %{ "args" => [
                            "--user-agent=#{Hound.Browser.user_agent(:chrome) |> Hound.Metadata.append(metadata)}",
                            "--headless",
                            "--disable-gpu"
                            ]}
                          })

    parent = self()
    on_exit(fn -> Hound.end_session(parent) end)
    :ok
  end
end

When I run these with async: true it succeeds about 1/3 of the time. Another 1/3 the first test passes and the second one basically renders a blank page then fails trying to find the css selector. The remaining 1/3 fails like this (even though the webdriver is running):

     ** (exit) exited in: GenServer.call(Hound.SessionServer, {:change_session, #PID<0.682.0>, :default, [additional_capabilities: %{chromeOptions: %{"args" => ["--user-agent=Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36/BeamMetadata (...long_string_here...=)", "--headless", "--disable-gpu"]}}]}, 60000)
         ** (EXIT) an exception was raised:
             ** (RuntimeError) could not create a new session: econnreset, check webdriver is running
12:19:52.398 [error] GenServer Hound.SessionServer terminating
** (RuntimeError) could not create a new session: econnreset, check webdriver is running
    (hound) lib/hound/session_server.ex:101: Hound.SessionServer.create_session/2

When I change to async: false the tests pass 100% of the time as expected.
Also, although I am using headless Chrome settings here it exhibits the same behavior using the regular Chrome settings (i.e. using Hound.start_session(metadata: metadata)). I haven’t tried other drivers (such as Selenium or Phantom).

I’m running MacOS Sierra 10.12.6, Elixir 1.5.1 w/OTP 20, latest Hound 1.0.4, the latest Chromedriver 2.32 (installed by homebrew), and the latest official build Chrome 61.0.3163.79.

Thanks!

2

I am seeing the exact same behavior. Did you find a solution?

3

No unfortunately and I’ve spent a lot of time trying to find the solution. Right now I am running all the tests synchronously but the time it takes to run the whole suite is growing fast so I generally run one test or module at a time at this point.

When I was evaluating Wallaby earlier it handled async testing with chromedriver just fine but I could never figure out what Hound was doing differently. You might want to check it out if you haven’t already written a bunch of tests.