Do you want to use only hound? or are you open to other solutions for scraping?
If you are not please show config.exs
Do you have this in config exs
config :hound, driver: "phantomjs"
?
wolfiton
1
pillaiindu
It’s an Elixir Project (not a Phoenix one), and it have the following file tree (excluding _build and deps directories), and it doesn’t have a config.exs file.
➜ scrap tree .
.
├── lib
│ └── scrap.ex
├── mix.exs
├── mix.lock
├── README.md
└── test
├── scrap_test.exs
└── test_helper.exs
0
NobbZ
Of course :hound will fallback then to the defaults.
If you do not want to use the defaults, you need to configure them.
Though I’d check if there is some API for hound that allows for runtime configuration of single scrapers rather than a global boot time configuration…
Phoenix is just a library. It doesn’t make your projects different in any way, they are just plain OTP applications.
3
NobbZ
Seems like you can use Hound.Helpers.Session.start_session/1 to start a runtime configured session.
PS: I really don’t like hounds API, as it has a lot of hidden magic, I prefer wallabys explicit passing around of the responsible client/connection.
3
ityonemo
Just my experience, I use hound for testing in the same way you do (it’s a django app that I’m testing), with chromedriver. I have written Chromedriver.ensure_started and Chromedriver.ensure_dead functions which use System.cmd/2 to make sure chromedriver is running before running the test suite.
Also I prefer wallabys syntax but find wallaby fails sometimes on our django/angilar stack (it has some poorly written parts and even 750ms latencies which don’t play well I think with wallaby’s “wait for all JavaScript to finish”. I’m also not the best at frontend so there’s maybe just something I’m missing.
0
wolfiton
The problem is that he hasn’t configured his hound with phantomjs or any other web driver
Try this GitHub - jaydorsey/elixir_scraper: Elixir/Hound web scraper example · GitHub it is also an elixir project using hound.
0
DevotionGeo
I don’t know if it’s from the start or it changed in the recent versions (I predominantly created Phoenix apps, not bare Elixir apps), but a mix new something won’t add the config.exs by default.
Add a config.exs under a config\ folder with the following code. If you’re using something other than phantomjs edit the code accordingly.
use Mix.Config
config :hound, driver: "phantomjs"
Also let us know if it worked or not.
1
pillaiindu
Now the output is:
iex(1)> Scrap.start
starting
** (Hound.NoSuchElementError) No element found for name 'email'
(hound) lib/hound/helpers/page.ex:51: Hound.Helpers.Page.find_element/3
(scrap) lib/scrap.ex:9: Scrap.start/0
iex(1)>
Which shows that the webdriver error is gone.
Thank you!
0