Elixir structs vs Erlang records

Indeed, here it is in Benchee, I tried to prevent certain optimizations from happening by interning the test data in different modules than that which is accessed so things like the record macro’s don’t get optimized out and so forth (records were a lot faster than structs before I made that change).

Code struct_record_bench.exs:

defmodule AStruct1 do
  defstruct [a: 1]
  def news1(), do: %__MODULE__{}
end

defmodule AStruct9 do
  defstruct [a: 1, b: 2, c: 3, d: 4, e: 5, f: 6, g: 7, h: 8, i: 9]
  def news9(), do: %__MODULE__{}
end

defmodule ARecords do
  import Record
  defrecord :aRecord1, [a: 1]
  defrecord :aRecord9, [a: 1, b: 2, c: 3, d: 4, e: 5, f: 6, g: 7, h: 8, i: 9]
  def newr1(), do: aRecord1()
  def newr9(), do: aRecord9()
end

defmodule StructRecordBench do
  import AStruct1
  import AStruct9
  import ARecords

  def classifiers(), do: [:get, :put]

  def time_mult(_), do: 2

  def inputs(_) do
    nil
  end

  def actions(:get) do
    %{
      "Struct1" => fn -> news1().a end,
      "Struct9-first" => fn -> news9().a end,
      "Struct9-last" => fn -> news9().i end,
      "Record1" => fn -> aRecord1(newr1(), :a) end,
      "Record9-first" => fn -> aRecord9(newr9(), :a) end,
      "Record9-last" => fn -> aRecord9(newr9(), :i) end,
    }
  end

  def actions(:put) do
    %{
      "Struct1" => fn -> %{news1() | a: 42} end,
      "Struct1-opt" => fn -> %AStruct1{news1() | a: 42} end,
      "Struct9-first" => fn -> %{news9() | a: 42} end,
      "Struct9-first-opt" => fn -> %AStruct9{news9() | a: 42} end,
      "Struct9-last" => fn -> %{news9() | i: 42} end,
      "Struct9-last-opt" => fn -> %AStruct9{news9() | i: 42} end,
      "Record1" => fn -> aRecord1(newr1(), a: 42) end,
      "Record9-first" => fn -> aRecord9(newr9(), a: 42) end,
      "Record9-last" => fn -> aRecord9(newr9(), i: 42) end,
    }
  end
end

Results:

╰─➤  mix bench struct_record           

Benchmarking Classifier:  get
=============================

Operating System: Linux"
CPU Information: AMD Phenom(tm) II X6 1090T Processor
Number of Available Cores: 6
Available memory: 15.67 GB
Elixir 1.7.4
Erlang 21.1.1

Benchmark suite executing with the following configuration:
warmup: 2 s
time: 2 s
memory time: 2 s
parallel: 1
inputs: none specified
Estimated total run time: 36 s

Benchmarking Record1...
Benchmarking Record9-first...
Benchmarking Record9-last...
Benchmarking Struct1...
Benchmarking Struct9-first...
Benchmarking Struct9-last...

Name                    ips        average  deviation         median         99th %
Record9-last        23.29 M      0.0429 μs   ±107.83%      0.0400 μs      0.0700 μs
Record9-first       22.06 M      0.0453 μs     ±5.87%      0.0440 μs      0.0550 μs
Record1             21.61 M      0.0463 μs    ±10.12%      0.0450 μs      0.0640 μs
Struct9-first       18.14 M      0.0551 μs    ±19.80%      0.0560 μs      0.0660 μs
Struct1             17.93 M      0.0558 μs     ±8.76%      0.0560 μs      0.0700 μs
Struct9-last        17.46 M      0.0573 μs     ±6.31%      0.0560 μs      0.0720 μs

Comparison:  
Record9-last        23.29 M
Record9-first       22.06 M - 1.06x slower
Record1             21.61 M - 1.08x slower
Struct9-first       18.14 M - 1.28x slower
Struct1             17.93 M - 1.30x slower
Struct9-last        17.46 M - 1.33x slower

Memory usage statistics:

Name             Memory usage
Record9-last             72 B
Record9-first            72 B - 1.00x memory usage
Record1                  72 B - 1.00x memory usage
Struct9-first            72 B - 1.00x memory usage
Struct1                  72 B - 1.00x memory usage
Struct9-last             72 B - 1.00x memory usage

**All measurements for memory usage were the same**

Benchmarking Classifier:  put
=============================

Operating System: Linux"
CPU Information: AMD Phenom(tm) II X6 1090T Processor
Number of Available Cores: 6
Available memory: 15.67 GB
Elixir 1.7.4
Erlang 21.1.1

Benchmark suite executing with the following configuration:
warmup: 2 s
time: 2 s
memory time: 2 s
parallel: 1
inputs: none specified
Estimated total run time: 54 s

Benchmarking Record1...
Benchmarking Record9-first...
Benchmarking Record9-last...
Benchmarking Struct1...
Benchmarking Struct1-opt...
Benchmarking Struct9-first...
Benchmarking Struct9-first-opt...
Benchmarking Struct9-last...
Benchmarking Struct9-last-opt...

Name                        ips        average  deviation         median         99th %
Record1                 18.38 M      0.0544 μs   ±691.86%      0.0500 μs       0.120 μs
Record9-first           15.52 M      0.0644 μs   ±497.05%      0.0600 μs       0.140 μs
Record9-last            15.45 M      0.0647 μs   ±539.09%      0.0600 μs       0.150 μs
Struct1                 15.13 M      0.0661 μs   ±647.84%      0.0600 μs       0.130 μs
Struct1-opt             14.48 M      0.0691 μs   ±327.02%      0.0600 μs       0.110 μs
Struct9-first           14.31 M      0.0699 μs   ±309.43%      0.0600 μs       0.160 μs
Struct9-first-opt       12.73 M      0.0786 μs   ±328.39%      0.0700 μs       0.130 μs
Struct9-last            11.66 M      0.0857 μs   ±414.23%      0.0800 μs        0.21 μs
Struct9-last-opt        10.74 M      0.0931 μs   ±322.63%      0.0800 μs        0.21 μs

Comparison:  
Record1                 18.38 M
Record9-first           15.52 M - 1.18x slower
Record9-last            15.45 M - 1.19x slower
Struct1                 15.13 M - 1.21x slower
Struct1-opt             14.48 M - 1.27x slower
Struct9-first           14.31 M - 1.28x slower
Struct9-first-opt       12.73 M - 1.44x slower
Struct9-last            11.66 M - 1.58x slower
Struct9-last-opt        10.74 M - 1.71x slower

Memory usage statistics:

Name                 Memory usage
Record1                      96 B
Record9-first               160 B - 1.67x memory usage
Record9-last                160 B - 1.67x memory usage
Struct1                     112 B - 1.17x memory usage
Struct1-opt                 112 B - 1.17x memory usage
Struct9-first               176 B - 1.83x memory usage
Struct9-first-opt           176 B - 1.83x memory usage
Struct9-last                176 B - 1.83x memory usage
Struct9-last-opt            176 B - 1.83x memory usage

**All measurements for memory usage were the same**

So Records are faster in general (in all tested cases here actually) than Structs, but only marginally so, so much so that only the most performance sensitive code would really care, so in general most people shouldn’t care. :slight_smile:

I’m surprised that putting the struct type in the update syntax doesn’t make it faster actually as it could infer some existing structure, but I guess that all it would be doing is adding an extra runtime check or so (hence the module-optional variants are slower in the end).

EDIT: Personal Opinion time: Personally I’d prefer records were ubiquitous and used struct syntax (first class records in other words). Records in every language I’ve seen are statically sized, there is no point in them being maps, especially if they ‘own’ their module definition as structs do now then all the proper accessors for Access and extra data would all be accessible as they are for structs as well and as such by using those generated macro’s then you could generate getting/setting code that would be even more efficient than how structs work now. HOWEVER, Elixir is extremely poorly typed and doesn’t know what the type of a given thing would be, and Erlang works around that by requiring using the record name at all uses of a record variable, Elixir tries to be a little more succinct, and that succinctness is at odds with efficiency, and so the first-class syntax uses the slightly less efficient version in order for ease of use and relegates the more efficient version to a side set of macro’s since you require the names anyway. If Elixir had a decent typing system then you’d be able to have both efficiency and succinctness, but maybe that’s for an Elixir 2.0 or something. ^.^

EDIT: Hmm, a possible workaround for the first-class syntax access would be just dispatching based on the ‘module’ in the type-tag of the record, it would be a ‘remote call’ on the BEAM but might be good… I should test…

5 Likes