Improper structs after code change & de-serialization from file

I have a problem that I did anticipate will happen at some point. I am persisting some Elixir structs to files. DETS, disk_log, or term_to_binary, whatever - the result is the same: a binary snapshot of my data is stored in file.

Let’s say I have the following struct:

defmodule Foo
  defstruct [:bar]
end

it gets serialized and de-serialized properly. When I do make a code change, however, and change the fields in my Foo struct, add some, remove some etc. - the de-serialization results in a struct that’s somehow improper. The iex prints it as such, so I can see it’s improper because it is printed out as struct:

%{__struct__: Foo, bar: "foo"}

How can I detect that my de-serialized structs are out of sync? I assumed I can pattern match against empty %Foo{}, and it would not match - but it does even that the struct is not a valid Foo anymore.

The naive fix I am doing is to treat everything I de-serialize as a map, and create new structs each time, by copying over the fields that do exist in new struct and do exist in old struct. it’s just that I am doing a lot of that unnecessarily because I don’t know which structs need correcting, and which ones are fine.

This is where I really like versioned tuples and just pass them through converters on every import.

But in your case I’d add a version field to the struct and test it, if it matches the program pass it through, if not then transform it to update it, would be cheaper then. :slight_smile:

1 Like

Tell me more - is it just {something, version} ? I think that would work for me.

Basically yep, though I’ve always put version first. ^.^

That was also back in the time that maps did not exist, so nowadays I just add a :version key to the struct itself.