We ran into an issue today with structs that I found a bit surprising.
The essence of the issue is:
A struct is a Map under the hood
you can manipulate a struct with Map functions, deleting keys that should be there and adding keys that shouldn’t
as long as the __struct__ key is still there, the Map is treated as a struct including in Elixir pattern matching
Here’s some code which demonstrates this:
defmodule Sandbox do
@enforce_keys [:mandatory]
defstruct [:mandatory, :optional]
def struct_foo do
s1 = %Sandbox{mandatory: "must have this"}
do_something("s1", s1)
s2 = Map.delete(s1, :mandatory)
do_something("s2", s2)
s3 = Map.put(s1, :bogus, "blah")
do_something("s3", s3)
end
def do_something(s, %Sandbox{} = thing) do
IO.puts "#{s} #{inspect thing} is a %Sandbox"
end
def do_something(s, thing) do
IO.puts "#{s} #{inspect thing} is just a thing"
end
end
iex(24)> Sandbox.struct_foo()
s1 %Sandbox{mandatory: "must have this", optional: nil} is a %Sandbox
s2 %{__struct__: Sandbox, optional: nil} is a %Sandbox
s3 %{__struct__: Sandbox, bogus: "blah", mandatory: "must have this", optional: nil} is a %Sandbox
One interesting thing is that IO.inspect seems to know the difference between the original struct and one that’s been hacked into a Map, but Elixir pattern matching doesn’t.
So you almost certainly should not be manipulating a struct in this way, and you certainly shouldn’t be deleting keys from structs. The question is: should (could) Elixir do anything to stop you doing this? Should Map functions check for __struct__ and behave differently if they are asked to do something nefarious to a struct? Should a Map function which manipulates a struct at the very least also remove the __struct__ key so it returns a Map that is not treated as a struct any more?
Well I don’t have code like that – that’s just sample code to demonstrate the underlying behaviour: that Elixir allows you to manipulate a struct into something that looks nothing like the struct, but it still treats it as a struct.
(For the curious, the issue was that somewhere in a codebase someone had, with good intentions, deleted from a struct all keys whose value was nil. This caused issues elsewhere in the codebase – specifically when relying on the default @derive implementation of Jason.Encoder – where code assumed that if something was a struct then all the keys should be there.)
I know you shouldn’t write code like that; my question was whether Elixir should - or indeed could - be doing something to stop you.
So if you are dealing with a larger code base and more developers then I’d say it makes sense to convert the strict structs to loose maps on their way out of the code you control. Let the other modules / projects do whatever with them and then try and convert them back to strict structs via the struct function on their way in back to your code.
If the type is not opaque what could dialyzer catch here? It is not even aware of structs. Making a struct opaque is possible but then you have to write accessors for every piece of data you might want to use.
I don’t think any error will be raised in the above code. The Map.put option succeeds, so what is there for dialyzer to complain about? The elixir compiler does complain about access or pattern match of non-existing fields, but dialyzer will never even see that code. It should catch cases that must lead to an error in exactly the same circumstances that it would for a bare map, but no others unless user-defined types are supplied.
One of the things I find odd is that IO.inspect somehow knows the difference between a struct and the Map with a __struct__ key that is returned from e.g. Map.delete:
struct: %Sandbox{mandatory: "must have this", optional: nil} is a %Sandbox
map with a __struct__ key: %{__struct__: Sandbox, optional: nil} is a %Sandbox
It could but it would make all maps operations more expensive, which is mostly why we don’t. If we had a static type system, doing Map.put/3 on a struct would certainly fail.
If there is a typespec in the function, dialyzer will fail
Example:
defmodule Test do
defstruct a: 0, b: nil, c: nil
@type t :: %__MODULE__{a: non_neg_integer(), b: nil | String.t(), c: atom()}
@spec foo() :: t
def foo do
%Test{
a: 1,
b: "aha",
c: :foo
}
|> Map.put(:bar, "bar")
end
end
$ mix dialyzer
lib/test.ex:6:invalid_contract
The @spec for the function does not match the success typing of the function.
Function:
Test.foo/0
Success typing:
@spec foo() :: %Test{:a => 1, :b => <<_::24>>, :bar => <<_::24>>, :c => :foo}
________________________________________________________________________________
Adding to this point wanted to say I found this library to be awesome for succinctly defining typespecs, enforced keys, and default values on structs so I now use it for every struct… https://github.com/ejpcmac/typed_struct
It doesn’t solve all the runtime concerns of the OP but it helps dialyzer help you without too much ceremony!
A follow up on the issue described above: turns out it was really a bug in erlang, and Hans Bolinder from Ericsson fixed it today https://github.com/erlang/otp/pull/2392
