vans163
So Iv been working on Morphlings diff algo to create a dom differential to send over the wire wasting as little CPU/Memory as possible and wanted to talk about a point when dealing with binaries.
This is how you shoot yourself in the foot and then turn around and say BEAM/Erlang/Elixir is very slow and bad.
100x Slower code with linear slowdown
fn_chunk = fn(bin,parts)->
size = byte_size(bin)
segment_size = div(size, parts)+1
{chunks,_} = Enum.reduce_while(0..(parts*2), {[], bin}, fn(idx,{a,bin})->
piece = String.slice(bin, 0, segment_size)
case piece do
"" -> {:halt, {a,bin}}
_ ->
a = a ++ [%{old_pos: idx*segment_size, size: segment_size, binary: piece}]
{:cont, {a, String.slice(bin, segment_size, size)}}
end
end)
chunks
end
Fast code
fn_chunk = fn(bin,parts)->
size = byte_size(bin)
segment_size = div(size, parts)+1
{chunks,_} = Enum.reduce_while(0..(parts*2), {[], 0}, fn(_,{a,idx})->
to_take = min(byte_size(bin)-idx, segment_size)
piece = :binary.part(bin, idx, to_take)
case piece do
"" -> {:halt, {a,bin}}
_ ->
a = a ++ [%{old_pos: idx, size: to_take, binary: piece}]
{:cont, {a, idx+to_take}}
end
end)
chunks
end
What this piece does is, it splits a binary into x amount of equal parts. On a binary of size 20k, the above slow code takes 100ms, on a 5k size binary takes 43ms.
The fast code takes 0-1ms on a 20k size binary.
What is the difference?
Trending in Discussions
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
New
Hey there,
It’s been more than a year since we started using LiveView as our main UI library and building a whole library of UI componen...
New
I am happy to introduce the very α version of the new programming language compiled to BEAM.
Welcome Cure.
It has literally three kille...
New
Quite interesting article Google brought me. Didn’t find any mentions about it here.
What do you think in general? Would you use togethe...
New
Hi everyone!
The first release candidate for the Expert language server project is now available!
We’ve published a press release detai...
New
Since we have deprecated our Erlang sections (as we have dedicated Erlang Forums now) let’s add this thread for those who’d like to post ...
New
:warning: Security advisory: Decimal DoS vulnerability
A vulnerability has been published for decimal where very large exponents can cau...
New
Other Trending Topics
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
Hi there! We created Gust: A task orchestrator inspired by Airflow.
For those who have never heard about Aiflow, it’s a Python-based wor...
New
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
With AI doing more of the implementation work, I’ve been wondering how much coding I should deliberately keep doing myself.
My main conc...
New
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixirconf-us
- #ai
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #hex
- #security










Showing Posts 1 to 4- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
sneako
String.slice/3 works on Unicode graphemes vs :binary.part/3 which is just looking at raw bytes. The documentation mentions this and suggests Kernel.binary_part/3 if you dont care about graphemes String — Elixir v1.20.2
xlphs
What if you just pattern match?
<< part :: binary-size(n), rest :: binary>> = binwhere n is number of bytes for each part.vans163
There is a lot of options to consider, what I was getting at is its a little confusing to know which operations make copies and which refer to the original binary on the shared binary heap. And this was a good performance refresher for me.
ananthakumaran
I would assume the second version is using sub binary. Instead of allocating a new binary, a reference to the original is created. I agree, from the documentation it’s not clear which function will allocate a new one vs share the original binary.
Be careful though, the original binary will not be garbage collected till all the sub binaries are garbage collected. This has a tendency to cause memory leak. Let’s say you take the header part of a large binary and keep it in genserver state, the garbage collection of the large binary will be delayed.
binary:copy/1can be used to create a independent copy. NimbleCSV — NimbleCSV v1.3.0