edxofp
How to read the file and make a list
I read my file in Elixir with the following code
It takes a long time to read the file
File.stream!("big.txt") |> Enum.map(&String.trim/1)
But in Python, I read the file in the fastest possible way with the following code
[line.rstrip() for line in open("big.txt")]
Do you have a better offer?
Thanks
Marked As Solved
antoine-duchenet
On a dummy 100MB text file, the use of a 8kB buffer seems ~25x faster than the default line per line behavior on my machine.
File.stream!("big.txt", [:read], 8192) |> Stream.map(&String.trim/1) |> Enum.to_list()
Also Liked
LostKobrakai
If I understand the python docs correctly it reads in 4kB - 8kB buffers and not individual lines. The elixir code you posted actually reads the file line by line.
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Supporting innovation across the BEAM ecosystem.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking for Elixir devs who love to ship. Start your free account.
Practical resources that improve the lives of professional developers.
Develop your skills with books, videos, and courses.
Producing high quality Elixir screencasts since 2017.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"