creusefond
Benchmark : String templating at runtime
Hi all,
I am facing the issue of having large template strings containing an unknown number of variables, and those templates need to be bound thousands of times.
Due to performance constraints, I would like for this procedure to be optimized. I had multiple ways of doing it.
Threads such as this one suggest to use EEx. I saw that I could use EEx.compile_string to get a quoted expression, that I can resolve with Code.eval_quoted (on the benchmark : eex). I also benchmarked a direct call to EEx.eval_string (eex_eval_string).
The previous implementation of this was to simply parse the content of the string for each variable, using a Regex. I added that method to the benchmark (base).
I also tried a manual implementation. The template is parsed once, and broken down into parts. During the binding, I simply append each variable value and each part (manual).
You can see the implementation here, the benchmark here and the results here.
Conclusions :
- Manual implementation outperforms the rest, by a wide margin.
- EEx seriously underperforms for this task (40x-170x slower than manual). Maybe I’m not using it right.
- Creating this benchmark was incredibly simple. Kudos to Benchee.
Please do not hesitate if you have feedback, or if you’d like to add another method to the test.
Most Liked
Qqwy
The way you were using EEx indeed does not result in very good performance: It re-compiles the EEx template on every benchmarking iteration.
I’ve sent you a PR which contains an implementation using EEx’s function_from_string to compile the EEx template once and then use it during the benchmark run.
Running the benchmark on my computer shows the following results:
$ mix run benchmarks/string_templating.ex
Compiling 1 file (.ex)
"ALL THE FOLLOWING SHOULD BE TRUE"
true
true
true
true
Operating System: Linux
CPU Information: Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
Number of Available Cores: 8
Available memory: 7.60 GB
Elixir 1.10.2
Erlang 22.3.4.1
Benchmark suite executing with the following configuration:
warmup: 2 s
time: 5 s
memory time: 0 ns
parallel: 1
inputs: none specified
Estimated total run time: 35 s
Benchmarking base...
Benchmarking eex...
Benchmarking eex_compiled...
Benchmarking eex_eval_string...
Benchmarking manual...
Name ips average deviation median 99th %
eex_compiled 758.61 1.32 ms ±20.23% 1.27 ms 2.41 ms
manual 125.65 7.96 ms ±14.40% 7.89 ms 11.80 ms
base 17.07 58.57 ms ±5.70% 58.51 ms 65.98 ms
eex 2.22 450.36 ms ±5.85% 442.40 ms 500.30 ms
eex_eval_string 1.42 704.79 ms ±9.19% 714.84 ms 787.16 ms
Comparison:
eex_compiled 758.61
manual 125.65 - 6.04x slower +6.64 ms
base 17.07 - 44.43x slower +57.25 ms
eex 2.22 - 341.65x slower +449.04 ms
eex_eval_string 1.42 - 534.66x slower +703.47 ms
warning: redefining module EexExampleModule (current version defined in memory)
lib/templating_benchmarks.ex:27
Operating System: Linux
CPU Information: Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
Number of Available Cores: 8
Available memory: 7.60 GB
Elixir 1.10.2
Erlang 22.3.4.1
Benchmark suite executing with the following configuration:
warmup: 2 s
time: 5 s
memory time: 0 ns
parallel: 1
inputs: none specified
Estimated total run time: 35 s
Benchmarking base...
Benchmarking eex...
Benchmarking eex_compiled...
Benchmarking eex_eval_string...
Benchmarking manual...
Name ips average deviation median 99th %
eex_compiled 5.23 0.191 s ±41.72% 0.175 s 0.47 s
manual 2.43 0.41 s ±7.35% 0.41 s 0.47 s
base 0.199 5.02 s ±0.00% 5.02 s 5.02 s
eex 0.0213 47.00 s ±0.00% 47.00 s 47.00 s
eex_eval_string 0.0142 70.61 s ±0.00% 70.61 s 70.61 s
Comparison:
eex_compiled 5.23
manual 2.43 - 2.15x slower +0.22 s
base 0.199 - 26.24x slower +4.83 s
eex 0.0213 - 245.76x slower +46.81 s
eex_eval_string 0.0142 - 369.18x slower +70.42 s
josevalim
It is worth adding that there are improvements in Erlang/OTP 24 and Elixir master in regards to code evaluation with many vars which should improve it about 5-6x at least.
axelson
Not a direct answer, but have you looked at using iolists for this? Here’s a nice article that touches on the performance benefits of iolists:
https://www.bignerdranch.com/blog/elixir-and-io-lists-part-1-building-output-efficiently/
The hard part would probably be parsing the “template” into something you can use to build an iolist. Also if you gave a few examples of the types of input you expect that would probably help people give you suggestions.
Popular in Discussions
Other popular topics
Chat & Discussions>Discussions
Latest on Elixir Forum
Sponsor Spotlight
Enabling companies to succeed by building software people love.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking for Elixir devs who love to ship. Start your free account.
Practical resources that improve the lives of professional developers.
Develop your skills with books, videos, and courses.
Enabling companies to succeed by building software people love.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"