seanmor5

seanmor5

Author of Genetic Algorithms in Elixir

I came across something weird today as I was doing some optimization.

I have a function that splits a list at some point. I was exploring the differences in using Enum.take/2 and Enum.drop/2. I ran some benchmarks on a list of 1,000,000 elements with 4 different scenarios:

  1. Getting the first 500,000 elements using Enum.take(list, 500_000)
  2. Getting the first 500,000 elements using Enum.drop(list, -500_000)
  3. Getting the last 500,000 elements using Enum.take(list, -500_000)
  4. Getting the last 500,000 elements using Enum.drop(list, 500_000)
Name                                          ips        average  deviation         median         99th %
Enum.drop/2 second 500,000 elements        142.99        6.99 ms     ±3.27%        6.97 ms        7.22 ms
Enum.take/2 first 500,000 elements          50.22       19.91 ms    ±33.95%       19.52 ms       26.40 ms
Enum.drop/2 first 500,000 elements          30.62       32.66 ms    ±24.34%       32.35 ms       77.36 ms
Enum.take/2 second 500,000 elements         23.11       43.28 ms    ±19.79%       42.81 ms      112.65 ms

Comparison: 
Enum.drop/2 get second 500,000 elements        142.99
Enum.take/2 get first 500,000 elements          50.22 - 2.85x slower +12.92 ms
Enum.drop/2 get first 500,000 elements          30.62 - 4.67x slower +25.66 ms
Enum.take/2 get second 500,000 elements         23.11 - 6.19x slower +36.28 ms

The results are KIND of as expected. Take is better at retrieving the first half of the list and drop is better at receiving the back half of the list. My real misunderstanding is WHY is drop/2 so much faster than take/2 at doing what it’s intended to do vs. what take is intended to do.

Also, shouldn’t the benchmarks be pretty similar no matter whether you’re dropping or taking forward or backward? Wouldn’t it make sense to just call drop/2 when the second input for take/2 is a negative integer and vice versa with drop/2?

First 6 of 6 Posts Switch mode

kokolegorille

kokolegorille

Given that lists are linked lists, it makes sense that taking from start is faster than taking from middle. But if You want to optimize this, I would suggest Stream instam of Enum, because it implements lazy loading, while Enum is loading all the list in memory.

I would always add from start, never from the end, then I would reverse the list if needed.

It might not be useful if You just want to split the list…

seanmor5

seanmor5 OP

Author of Genetic Algorithms in Elixir

For my purposes I’ve settled on using Enum.split/2 rather than either take/2 or drop/2. I was just confused as to why there’s such a discrepancy in the performance of these two functions.

What I mean is, why wouldn’t an implementation of drop and take when n is less than 0 look like:

def take(list, n) when n < 0 do
   drop(list, n)
end

def drop(list, n) when n < 0 do
   take(list, n)
end

Shouldn’t this theoretically close the performance gap?

kokolegorille

kokolegorille

I try to avoid this with lists (taking from the middle).

Ankhers

Ankhers

This exact code would never return if you were to pass a negative n because it would continually match the n < 0 guard on both functions. If you meant

def take(list, n) when n < 0 do
  drop(list, n * -1)
end

def drop(list, n) when n < 0 do
  take(list, n * -1)
end

then I’m not sure that would give you what you are looking for. If I had some list of size 10 and wanted the last 2 elements, I would do Enum.take(list, -2) which would end up becoming drop(list, 2) which would return the first 8 elements of the list.

Unless I am misinterpreting what you meant?

seanmor5

seanmor5 OP

Author of Genetic Algorithms in Elixir

Sorry I wasn’t thinking too hard about the actual implementation.

I think this:

def take(list, n) when n < 0 do
   drop(list, length(list)-n)
end

def drop(list, n) when n < 0 do
   take(list, length(list)-n)
end

Was what I meant (or something in the same spirit as this, I haven’t actually tested if this works).

OvermindDL1

OvermindDL1

To sum it up:

drop with a positive value: Just returns the tail at that position
drop with negative value: Has to rebuild the front list
take with a positive value: Has to rebuild the front list
take with a a negative value: Is currently rebuilding the back list. Now technically it shouldn’t necessarily need to if it were implemented differently (implement as Enum.drop in that case?), a PR could fix it. ^.^

— All posts loaded —

Where Next? Top

Trending in Discussions Top

AstonJ
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
2977 91898 914
New
AstonJ
The obligatory hello world thread! Who are you and where are you from? :stuck_out_tongue:
4616 55835 594
New
byu
@chrismccord : I just saw the Extract AGENTS.md from Phoenix.new into phx.new generator commit to the phoenix project. My initial shotgu...
New
arcanemachine
I was working on an Ecto migration and I needed a timestamp. So, for the nth time, I looked up the different data types for timestamps, a...
New
AstonJ
Just a general thread to post chat/news/info relating to AI/ML stuff that may be relevant for Nx now or in the future. Got anything to sh...
New
juhalehtonen
There has been a thread to discuss the Stack Overflow Developer Survey on this forum every year since 2018, so here’s yet another one for...
New
type1fool
I just stumbled on a newly redesigned elixir-lang.org. :tada: It looks like @Software_Mansion did the work, and I think it is generally a...
New

Other Trending Topics Top

JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge &amp; Solve. They are GUI (Emerge) and State management (S...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
ausimian
Emily is an Elixir library that runs Nx computations on Apple’s MLX. Install it as the default Nx backend and Nx, defn, Axon, Nx.Serving,...
New
mudasobwa
While I am working on the Language Agnostic Code Audit SaaS, which uses MetaAST (spoiler: I am expecting it to be in a good shape for ann...
New

We're in Beta

About us Mission Statement