Choosing between maps, structs, records etc

As a long time erlang user I like Records, but maps/structs have supplanted their use.

Just note, a record is just a tuple with some compile-time names given. I’d have to know more about your use-case and why maps are slow for you, and given that I’m not sure records would be much faster. Can you give an abbreviated example of what you are trying to accomplish and why there would be so many storage things?

AWS API responses are XML

E.g.

<DescribeKeyPairsResponse xmlns="http://ec2.amazonaws.com/doc/2015-10-01/">
  <requestId>7a62c49f-347e-4fc4-9331-6e8eEXAMPLE</requestId>
  <keySet>
    <item>
      <keyName>gsg-keypair</keyName>
      <keyFingerprint>
         00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
      </keyFingerprint>
    </item>
  </keySet>
</DescribeKeyPairsResponse>

Which I think could easily be expressed as structures.
Some good discussion about records and structs:

The number of structures is dictated by the number of data types that the api description files define - 276 for the rds api , 536 for the ec2 one, 225 for IAM etc. Across 72 apis / api versions that adds up to a lot of structs.

I’m not sure about runtime slowness - I haven’t written enough of this api client that you can actually use it to do anything. I’m just noticing that the large numbers of structures I’m creating is resulting in these slow compile times (like I said, 18 seconds for just one of the was apis), which is annoying on its own, and possibly a warning that I’m not doing things right.

The sheer amount of data types seems to make the structs road unfeasible. I would go with maps because they are still typo safe: map.foo will raise if the field foo does not exist, as well as pattern matching on %{foo: foo}. It does suffer a bit on discoverability when compared to structs.

That makes sense. I can populate the maps with nils for fields that are optional in api responses.

One other thing, aside from what is the correct interface, why is it that defining lots of structs causing slow compile times?

Is it because each struct is module and there is a cost to pay for each module? Is this this something to worry about normally or is only edge cases with hundreds of modules per file that are affected?

Yes.

Only when defining hundreds.

It looks like Postgrex makes heavy use of records for a similar use case. Is there a distinction that I’m missing? Or would you recommend this be built differently if done today?

Maps are a fairly recent development in the BEAM world, most uses of Records are better served by Structs nowadays, and Postgrex has been around a long while.

Records are still technically better in some ways, like accessing a field in it is ‘slightly’ faster than a map, but inserting an update in a map ‘might’ be slightly faster than a record ‘if’ the record is decently sized.

Think of records as glorified tuples. When you need to handle multiple different tuples, which are private to a module, records work great. Maps would also work in the example above, but they wouldn’t give the compile time guarantee of records. Structs would be too wasteful though for those cases. Those data structures are never really “exported”, so using multiple modules for representing them is quite unnecessary.