What is the most efficient way to fill a binary with constant values?

Questions / Help
,
1

Hey all,

As the title says, what do you think is the most efficient way to fill a binary with all constant values? I usually do the following:

for _i <- 1..n, into: "", do: <<0xFF>>

But when using a large-ish number for n, such as 10_000, it takes a couple of seconds to fill the buffer (on my low-powered Nerves device).

The following is actually noticeably faster:

(for _i <- 1..n, do: 0xFF) |> :erlang.list_to_binary

Some timings:

iex(23)> :timer.tc(fn ->
...(23)>     for _i <- 1..10_000, into: "", do: <<0xFF>>
...(23)> end)
{1184934,
 <<255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
   255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
   255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
   255, 255, 255, ...>>}

iex(24)> :timer.tc(fn ->
...(24)>     (for _i <- 1..10_000, do: 0xFF) |> :erlang.list_to_binary
...(24)> end)
{64366,
 <<255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
   255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
   255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
   255, 255, 255, ...>>}

That’s a big difference - 1.1 seconds vs 64ms!

I assume the reason for the first one being so slow is because each iteration of the for comprehension does a binary concatenation, whereas the second version appends to the linked list, then accumulates everything at the end into the binary.

This has me wondering - is this the best method for creating large binaries with constant values, or is there something better?

2

You can use String.duplicate/2 / :binary.copy/2.

6 Likes
3

Oh nice, I had no clue those existed! Much faster. Looks like they probably use the same function under the hood:

iex(30)> :timer.tc(fn ->
...(30)>     :binary.copy(<<0xFF>>, 10_000)
...(30)> end)
{726,
 <<255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
   255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
   255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
   255, 255, 255, ...>>}

iex(32)> :timer.tc(fn ->
...(32)>    String.duplicate(<<0xFF>>, 10_000)
...(32)> end)
{724,
 <<255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
   255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
   255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
   255, 255, 255, ...>>}
1 Like