protoboolean
I’ve implemented my first web-scraper with Crawly following the tutorial published on the official docs.
I’m at the point where I start a single crawler module from iex and it seems it’s successfully crawling but it’s not writing to disk.
I’ve checked it’s crawling the website and extracting some items using IO.inspect.
I’ve limited the total number of items to make sure it completed gracefully a run.
The Crawly.Pipelines.WriteToFile module is on the pipeline and it’s configured to write to the user-writable folder /tmp/crawly, but at the end of the run it’s as empty as ever.
I’ve enabled logging pointing in the same folder but that’s not written either.
I’ve never debugged an Elixir/Erlang app and I still find the idea a bit daunting.
Anyone has a clue why it’s not writing to disk?
Where Next?
Trending in Questions
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
At WyeWorks, we believe great software is built by combining technical excellence with genuine human connection
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixirconf-us
- #ai
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #hex
- #security
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Practical resources that improve the lives of professional developers.
Producing high quality Elixir screencasts since 2017.
**Building the future of software with AI, trust, and human connection.**
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 1- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
kip
I haven’t use crawly so have no direct experience. But I do notice your configuration file is called
config.exwhen it should beconfig.exswhich may be part of the issue.