dgmcguire
when does deduplication happen for IOLists…if I create a massive IOList with a bunch of macros expanding inside of each other, will there be a final pass that deduplicates binary segments?
e.g.
1. EExFun.outter() #=> [
"<div>",
EExFun.inner() #=> ["<div> some string </div>"],
"</div>"
]
2. ["<div>", "<div> some string </div>", "</div>"]
3. [ref_div_open(), [ref_div_open(), "some string", ref_div_close()], ref_div_close()]
Where Next?
Trending in Questions
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
Producing high quality Elixir screencasts since 2017.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #blog-post
- #phoenix_html
- #iex
- #graphql
- #genstage
- #ai
- #elixirconf-us
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Practical resources that improve the lives of professional developers.
Producing high quality Elixir screencasts since 2017.
Enabling companies to succeed by building software people love.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
First 7 of 7 Posts
dimitarvp
If I’m understanding your question correctly then what you ask for is already happening internally in the runtime: repeated strings are a part of a reference-counted pool.
NobbZ
What do you mean by dedupping?
Having 2 immediate values that represent the same string might be optimised into a single immediate value by the compiler, this is considered an optimisation and the observable effect of the program stays the same whether or not those values get optimised into one or two immediates.
At runtime, any binary value though, that is bigger than a certain threshold gets to the binary heap. It’s recounted there. But just because you create some string somewhere else that happens to have the exact same value, it won’t count to the old ref counted string, it would create a second copy of it.
So, there is no magic deduplication as I would understand it, and also I think that is fine, as I would dare the runtime implications of comparing all existing strings on the binheap when we create/build a new string/binary.
axelson
If you want to poke around with how the vm will make use of your strings. There’s a tool linked from this article that may be of use: Elixir RAM and the Template of Doom – Evan Miller
dgmcguire
Thanks, that’s actually the article that got me interested in how it was happening.
dgmcguire
Gotcha, the ref counted pool gives me something to google as I’m trying to understand the specifics of when it happens. Thanks!
dgmcguire
yeah, unfortunately I dunno if the linux version for dtrace is ready yet? Maybe it is I should probably check, but linux does have bpftrace, I’m working on setting that up now
dgmcguire
Okay, that’s really good to know and clears up a misconception I had, thanks