cjbottaro
Hello,
Why is this Elixir code:
defmodule Foo do
def run(file_name) do
File.open! file_name, [:read], fn f ->
IO.stream(f, :line) |> Enum.each(&process_line/1)
end
end
defp process_line(line) do
String.rstrip(line) |> String.split(",")
end
end
[ file_name | _ ] = System.argv
Foo.run(file_name)
So much slow than this Ruby code:
def run(file_name)
File.open file_name, "r" do |f|
f.each_line{ |l| process_line(l) }
end
end
def process_line(line)
line.chomp.split(",")
end
run(ARGV[0])
Elixir:
$ time elixir test.exs ../data_gen/posts.csv
real 0m24.496s
user 0m23.527s
sys 0m1.983s
Ruby:
$ time ruby test.rb ../data_gen/posts.csv
real 0m6.556s
user 0m6.444s
sys 0m0.100s
I suspect it’s because I’m not streaming the lines properly?
Thanks for the help,
– C
Where Next?
Trending in Questions
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
Producing high quality Elixir screencasts since 2017.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #blog-post
- #phoenix_html
- #iex
- #graphql
- #ai
- #genstage
- #elixirconf-us
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Bringing learning, work and compliance together since 1987.
Producing high quality Elixir screencasts since 2017.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
First 10 of 36 Posts
uranther
Try
File.stream!/3instead ofFile.open!andIO.streamcjbottaro
Thanks! It’s twice as fast now:
(using this code)
Still twice as slow as Ruby though…
Any other tips? Thanks again.
uranther
With
String.rstrip(line) |> String.split(","), you are ripping through the line multiple times. Try using thetrim:option instead:cjbottaro
That change made no significant difference… :sad face:
sotojuan
Wild/dumb guess but maybe because it has to spin up a VM to run?
cjbottaro
I assume that accounts for milliseconds, not many seconds…
NobbZ
I’m not sure how fast BEAM boots up, but have you tried profiling using fprof?
dom
What version of Ruby is that? Not all of them will use UTF-8 by default.
You could also load the whole file into memory first, rather than stream. If I’m not mistaken that’s what the Ruby version does.
andre1sk
build executable with escript?
NobbZ
Running an elixirscript means to compile and then call into the compiled module. I do not think, that for that short script the compiletime will make any significant difference.