Does anyone have a working setup to do end to end testing using hound with github-actions?
I’ve got it working locally , and its a great way to sanity check the multi-step liveview features on this project but when running on the github-action I get a
** (RuntimeError) invalid element state for each of those tests.
When using hounds fill_field function.
I’ve tried several different combinations of things last night and this morning and seem to be at a deadend, and I’m unsure what else to try.
My workflow.yaml is below.
name: Master PR
on:
pull_request:
branches:
- master
jobs:
build:
runs-on: ubuntu-18.04
services:
postgres:
image: postgres:10.8
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: larder_test
ports:
- 5432:5432
# needed because the postgres container does not provide a healthcheck
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
steps:
- uses: actions/checkout@v1
- uses: actions/setup-elixir@v1.1.0
with:
otp-version: 22.x
elixir-version: 1.9.x
- name: Install Dependencies
run: |
mix local.rebar --force
mix local.hex --force
mix deps.get
- name: Setup Node
uses: actions/setup-node@v1
- name: Install ChromeDriver
run: |
CHROME_VERSION=$(google-chrome --version | cut -f 3 -d ' ' | cut -d '.' -f 1) \
&& CHROMEDRIVER_RELEASE=$(curl --location --fail --retry 3 http://chromedriver.storage.googleapis.com/LATEST_RELEASE_${CHROME_VERSION}) \
&& curl --silent --show-error --location --fail --retry 3 --output /tmp/chromedriver_linux64.zip "http://chromedriver.storage.googleapis.com/$CHROMEDRIVER_RELEASE/chromedriver_linux64.zip" \
&& cd /tmp \
&& unzip chromedriver_linux64.zip \
&& rm -rf chromedriver_linux64.zip \
&& sudo mv chromedriver /usr/local/bin/chromedriver \
&& sudo chmod +x /usr/local/bin/chromedriver \
&& /usr/local/bin/chromedriver --version
- name: Run Tests
run: |
export DISPLAY=:99
/usr/local/bin/chromedriver --port=9515 &
curl localhost:9515
sudo xvfb -ac :99 -screen 0 1280x1024x24 > /dev/null 2>&1 &
mix compile
mix test
env:
# use postgres for the host here because we have specified a container for the job.
# If we were running the job on the VM this would be localhost
DB_USER: postgres
DB_PASS: postgres
DB_NAME: larder_test
or alternatively, I’m open to swapping to Wallaby if anyone has had luck there.
I was reading some posts last night on Cypress as well, and it seems very nice, but keeping everything in the exunit/elixir world is something I’d like to do if possible to make it one less thing to onboard others to the project.
@PJUllrich thanks so much for sharing that. That gave me a great head start for figuring out caching if nothing else so totally worth it taking my build/test time down!
One question, I don’t see anything about starting chromedriver after its installed, are you starting that in your mix test task or something else?
Yes, chromedriver is started by Wallaby. You don’t have to start anything manually.
Cypress as @alanvardy mentioned is also really, really nice! I can highly recommend it if you want to test your system as a black box, so exclusively test through the UI. Writing UI tests is a breeze using Cypress. In my case however, I needed to do “grey box”-testing, so, I had to both call some Elixir functions like e.g. insert(:post) and “click through the UI”. Since wallaby is written Elixir I could easily do both with it. I’m not sure about this, but maybe @alanvardy knows whether it’s possible to call Elixir functions using Cypress?
Awesome. I managed to get everything working last night, thanks for sharing those github action spec, that helped a ton at getting it working.
After adding a line to start chromedriver and background it, it seems to be working as expected.
I’m intrigued by cypress, it looks like a good approach and looks very clean. I’m going continue to explore there, but for now I’m glad to have hound rolling. I mostly wanted to stay in the elixir world due to the gray-box testing you mentioned, specifically for my project some “shopping cart”-type functionality that happens over several steps with items/quantities and persists between sessions. Its nice to be able to write assertions around what should be in the database.
I am not able to call any elixir functions within the code (I think), and pretty much black box the whole thing.
I have Cypress register an account, log out, log in, create and view resources etc. I made a 4th environment called :system_test and then can mock out anything I need to within the code (i.e. not requiring a confirmation email on sign up).
I try to cover as much as possible with the unit and integration tests because they are a lot faster, and then have Cypress for covering the happy path and the things that are impossible to test any other way… like if LiveView is still working after my last change
, and its a great way to sanity check the multi-step liveview features on this project but when running on the github-action I get a
