mjadczak

mjadczak

TL;DR: are tuples much faster to construct and access than maps?

I am working on a project where I am passing elements through GenStage and processing them in each stage. I am implementing a data processing system (the Dataflow / Beam model of processing) on top of GenStage. Therefore, as well as actual data/values, I have a fair amount of metadata riding along in these events.

Currently, I am using a 4-tuple to represent these events. This came by as initially it was a two-tuple, then three, then four, etc. I probably would have chosen maps initially had I known, but now I ask the question—will maps slow me down in the case where I’m pattern matching / constructing tens of thousands of them?

I don’t actually know where I get this impression from, but I thought that since maps are relatively new in Erlang, they may be slower than tuples. Now I realised that perhaps it shouldn’t be a concern.

Does anyone have any experience with this / are there any benchmarks or results out there to support or refute this?

Showing Posts 1 to 5

Eiji

Eiji

@mjadczak: If you can’t say how many items your tuple will have in future then I suggest you to combine struct and tuple like:

defmodule MyApp.Event do
  defstruct [:your, :fields, :here]
end
defmodule MyApp.Example do
  alias MyApp.Event

  def sample do
    {:ok, %Event{your: "", fields: 0, here: true)
  end
end

I’m not sure about performance, but when you are building API and changing it from 2 tuple items (initial version) to 4 and more tuple items (in next versions) then it’s easier for someone that uses your API to have always 2 tuple items with map or struct as a second tuple item.

In your case:

alias MyApp.Example

{:ok, your, fields, _here} = Example.sample
# then in next versions:
{:ok, _here, your, fields, _something, _more} = Example.sample

do_something_with(your, fields)

As I suggest:

alias MyApp.Example

{:ok, event} = Example.sample
# then in next versions:
{:ok, event} = Example.sample

do_something_with(event.your, event.fields)
mjadczak

mjadczak OP

Thanks, just to clarify, these tuples aren’t actually exposed to users, they’re just to contain all of the information internally. The purpose of moving to maps would just be to aid further development for myself, and make it easier to refactor in the future.

xlphs

xlphs

They are equally difficult to construct, and tuple should be faster to access than map. I refer to how BEAM implements tuple:

tuple is very easy to access at arbitrary index, and very hard to grow

from http://beam-wisdoms.clau.se/en/latest/indepth-memory-layout.html#tuple-arityval-0

The reason map is slower to access is because of arbitrary key type.

Also, I would argue that unless your map is going to contain thousands of elements there is no point in worrying about performance.

bbense

bbense

FWIW, Maps of less than X elements ( currently X is 32 ) are just sorted Lists of Key Value tuples, beyond that they are Hash Mapped Trie’s. Hash array mapped trie - Wikipedia

Tuples are implemented under the covers as a C array of pointers to Terms, thus tuples are fast for access if you know the element number, but are slow if you need to modify them.

If you always prepend to a List, they are fast and memory efficient to modify, but access to specific elements is slow.

Maps are somewhere in between, access to a specific element is much faster than a List, but modification is more expensive than a List, but less than a Tuple.

Without knowing the exact access patterns and transformations, it’s hard to know what is best for your case. For “small” values it’s extremely hard to make any general statements about performance of various data types. You really need to benchmark the code to know.

This is what I try to do with Elixir code:

“Make it right, Make it pretty, Make it fast ( if you need to)”

I’d pick the data type that makes working on the code as easy as possible.

OvermindDL1

OvermindDL1

For note, if you want a struct-like thing that is built for reading speed, that is precisely what Records are, Records are just tuples with named slots.

Altering a map can be faster than a tuples if you have more that a couple elements though (entire tuple has to be recreated, map can be altered faster), but for few elements even a tuple will always be faster.

But yeah in general do not worry about performance unless you find a specific need.

— All posts loaded —

Where Next? Top

Trending in Questions Top

stjefim
Hello! Suppose you are building workflow (order / task / payment) processing system with the following requirements: Each workflow con...
New
water
I’m aware of 2 such projects, are there others? git: GitHub - team-alembic/realworld: A fullstack Phoenix LiveView application with back...
New
itzmidinesh
Hi all, I’m working on a Phoenix project where I need to make external HTTP requests and I’m trying to choose between Req and HTTPoison....
New
peck
Hi everyone. My team and I have been working on a fairly modest app based around video streaming and chat, but we’ve landed a customer t...
New
leoschweizer
Hi, we run different LLM models across a cluster, with each node loading a configurable subset. Model-specific queues let nodes consume ...
New
csmr
Hi all, I’m new here and would like to hear peoples views on my Elixir/Phoenix based game-engine! :smiley: Mutonex gamedesign was finish...
New
Stef-Verniers
Hello I’ve been coding in Elixir for almost two years now because of my current job where we implemented Elixir in our application. But ...
New

Other Trending Topics Top

dominicletz
Hey there, I haven’t seen a list of useful Elixir specific skills for the popular AI tools yet so I’m starting one here with my first. ...
New
raa-ziq
Hi everyone :waving_hand: I wanted to share something I’ve been building — NowBlind, a random chat app designed to help people discover ...
New
mudasobwa
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
KristerV
Hey. Is there anyone here who creates agents in their apps? Not talking about using agents, but creating them. I’m finding it pretty diff...
New
OndrejValenta
(I just needed to vent somewhere and LinkedIn is full of hope, or hype, I’m not sure which exactly) AI dream has many faces, but general...
New
webofbits
With AI doing more of the implementation work, I’ve been wondering how much coding I should deliberately keep doing myself. My main conc...
#ai
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews