mjadczak

mjadczak

TL;DR: are tuples much faster to construct and access than maps?

I am working on a project where I am passing elements through GenStage and processing them in each stage. I am implementing a data processing system (the Dataflow / Beam model of processing) on top of GenStage. Therefore, as well as actual data/values, I have a fair amount of metadata riding along in these events.

Currently, I am using a 4-tuple to represent these events. This came by as initially it was a two-tuple, then three, then four, etc. I probably would have chosen maps initially had I known, but now I ask the question—will maps slow me down in the case where I’m pattern matching / constructing tens of thousands of them?

I don’t actually know where I get this impression from, but I thought that since maps are relatively new in Erlang, they may be slower than tuples. Now I realised that perhaps it shouldn’t be a concern.

Does anyone have any experience with this / are there any benchmarks or results out there to support or refute this?

Showing Posts 1 to 5

Eiji

Eiji

@mjadczak: If you can’t say how many items your tuple will have in future then I suggest you to combine struct and tuple like:

defmodule MyApp.Event do
  defstruct [:your, :fields, :here]
end
defmodule MyApp.Example do
  alias MyApp.Event

  def sample do
    {:ok, %Event{your: "", fields: 0, here: true)
  end
end

I’m not sure about performance, but when you are building API and changing it from 2 tuple items (initial version) to 4 and more tuple items (in next versions) then it’s easier for someone that uses your API to have always 2 tuple items with map or struct as a second tuple item.

In your case:

alias MyApp.Example

{:ok, your, fields, _here} = Example.sample
# then in next versions:
{:ok, _here, your, fields, _something, _more} = Example.sample

do_something_with(your, fields)

As I suggest:

alias MyApp.Example

{:ok, event} = Example.sample
# then in next versions:
{:ok, event} = Example.sample

do_something_with(event.your, event.fields)
mjadczak

mjadczak OP

Thanks, just to clarify, these tuples aren’t actually exposed to users, they’re just to contain all of the information internally. The purpose of moving to maps would just be to aid further development for myself, and make it easier to refactor in the future.

xlphs

xlphs

They are equally difficult to construct, and tuple should be faster to access than map. I refer to how BEAM implements tuple:

tuple is very easy to access at arbitrary index, and very hard to grow

from http://beam-wisdoms.clau.se/en/latest/indepth-memory-layout.html#tuple-arityval-0

The reason map is slower to access is because of arbitrary key type.

Also, I would argue that unless your map is going to contain thousands of elements there is no point in worrying about performance.

bbense

bbense

FWIW, Maps of less than X elements ( currently X is 32 ) are just sorted Lists of Key Value tuples, beyond that they are Hash Mapped Trie’s. Hash array mapped trie - Wikipedia

Tuples are implemented under the covers as a C array of pointers to Terms, thus tuples are fast for access if you know the element number, but are slow if you need to modify them.

If you always prepend to a List, they are fast and memory efficient to modify, but access to specific elements is slow.

Maps are somewhere in between, access to a specific element is much faster than a List, but modification is more expensive than a List, but less than a Tuple.

Without knowing the exact access patterns and transformations, it’s hard to know what is best for your case. For “small” values it’s extremely hard to make any general statements about performance of various data types. You really need to benchmark the code to know.

This is what I try to do with Elixir code:

“Make it right, Make it pretty, Make it fast ( if you need to)”

I’d pick the data type that makes working on the code as easy as possible.

OvermindDL1

OvermindDL1

For note, if you want a struct-like thing that is built for reading speed, that is precisely what Records are, Records are just tuples with named slots.

Altering a map can be faster than a tuples if you have more that a couple elements though (entire tuple has to be recreated, map can be altered faster), but for few elements even a tuple will always be faster.

But yeah in general do not worry about performance unless you find a specific need.

— All posts loaded —

Where Next? Top

Trending in Questions Top

Blokh
Hey guys, I’ve got a huge CSV ( around 10 GB ) that needs to be processed hourly Do you guys have any suggestions what is the best prac...
New
kszambelanczyk
Hello! Could someone please give me a help/sample code, how to delete a file from s3 using waffle/waffle_ecto from Phoenix app. I creat...
New
Onor.io
I have what I’ve heard referred to as a “lookup table” in my database. This is a way of assigning codes to common values. One common lo...
New
Trolleger
What approach to take when sending live updates to “random” users Hi! I have a question, I have a little chat app, and when I create a DM...
New
matt-savvy
Anyone here using Honeybadger? My Honeybadger account is being overwhelmed with noise from some bots. Seeing a lot of Bandit.HTTPError...
New
RemyXRenard
I’m seeing that a list inside a Kino.DataTable will be interpreted as a charlist, even if the Kino.configure() is set to charlists: :as_l...
New
samoloth
Hi, I’ve just set up an application with ash_authentication. There is only magic link strategy for now, so there is no confirmation add o...
New

Other Trending Topics Top

mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
garrison
Hobbes is a low-level distributed database for the Elixir programming language. Hobbes provides a simple, safe, and scalable storage lay...
New
mcass19
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
wintermeyer
There are three potential reasons for members of this forum to have a look at https://vutuv.de You are tired or annoyed of LinkedIn. Yo...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews