How to read the file and make a list

I read my file in Elixir with the following code
It takes a long time to read the file!("big.txt") |>

But in Python, I read the file in the fastest possible way with the following code

[line.rstrip() for line in open("big.txt")]

Do you have a better offer?


not a Stream expert here, but I’d try something like this!("big.txt") |> |> Enum.to_list()

:thinking: this seems equivalent to your code though…

I load in Python in less than 4 seconds
But in Elixir over 500 seconds and more

If I understand the python docs correctly it reads in 4kB - 8kB buffers and not individual lines. The elixir code you posted actually reads the file line by line.


On a dummy 100MB text file, the use of a 8kB buffer seems ~25x faster than the default line per line behavior on my machine.!("big.txt", [:read], 8192) |> |> Enum.to_list()

Thank you very much dear friends