http402
If I have a mix project with two libraries that I want to comparatively benchmark - is it fair to call each of them one at a time from an IEx shell (started with iex -S mix) and compare their execution times? Is this significantly different than say, making a mix task, and running it? How much of the IEx experience is interpreted rather than compiled?
Where Next?
Trending in Questions
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
Supporting innovation across the BEAM ecosystem.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixirconf-us
- #ai
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #hex
- #security
- #metaprogramming
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Practical resources that improve the lives of professional developers.
Bringing learning, work and compliance together since 1987.
Producing high quality Elixir screencasts since 2017.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 4- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
OvermindDL1
I may be wrong, but I think iex is entirely interpreted.
Do note, that means that
defmodule’ing inside iex will interpret the defmodule call, but that really does compile a module and code run ‘inside’ those functions are fast.net
If you’re trying to benchmark libraries you’ll have much better and more useful results with a benchmarking library like benchee.
bbense
In any VM environment benchmarking is very tricky, you never know if you are benchmarking your actual code or the implementation of the VM. You really need to do many invocations and do a statistical analysis. Using :tc.timer in iex can provide very misleading results.
env MIX_ENV=prodalso effects the timing results.Benchee , Benchfella and other similar libraries help, but even with those there are some gotchas. If you are simply looking for order of magnitude results, then :tc.timer in iex is probably ok, but I would not trust it beyond that.
http402
Thanks everyone. I’ll definitely try to check out some of the benching libraries to see what they do for this.