Has anyone come up with a good way to create some form of module registry that can be used to lookup a module by name. My use case is to have a module that can deserialize a list of heterogeneous items. I would like the different items to be added to a registry automatically. Ultimately, it would be nice to ‘auto’ generate the Filters module below.
defmodule Filters do
@moduledoc """
Module that aids in dealing with filters. It acts as a registry that
knows about all the filter types at compile time.
Supports serializing and deserializing a list of heterogeneous filter types.
"""
# Serialize a specific filter, adding name to serialized value.
# this can be better handled with `Protocols` and defimpl in __using__ macro.
# This is not the difficult part.
def serialize(%Filter1{} = filter) do
%{
type: "filter1",
value: Filter1.serialize(filter)
} |> Jason.encode!()
end
def deserialize(s) when is_binary(s) do
# s is a json encoded string that needs to be
# deserialized into one of the filter modules below
deserialize(Jason.decode!(s))
end
# THIS IS THE FUNCTION THAT I WOULD LIKE TO GET AUTO GENERATED, or have
# some kind of registry that can give me the module for some arbitrary name.
#
# Some means to create a specific struct given some name, in this case, "filter1"
def deserialize(%{type: "filter1", value: value}) do
Filter1.deserialize(value)
end
end
defmodule Filter1 do
# can something be done in the declaration of this module
# to add it to the Filters module?
def serialize(filter), do...
def deserialize(data), do...
end
defmodule Filter2 do
...
end
It’s the deserializing bit that I am having problems with. Given some name, what module handles the deserialization? I modified my post to try and clarify this better.
It does for serialize but not for the deserialize part. When deserializing, I don’t know what the type of struct I am deserializing into. @entone suggests adding a type to the map when serializing, then this can be used in the deserializing bit. I think this will work for me, I was just hoping to map an arbitrary name to a specific Elixir struct.
You can put the type wherever you want. You were putting it in a wrapper map, which would work fine as well. The main part is capturing the type at serialization. Doesn’t matter where you store it.
defmodule Filter1 do
@derive Jason.Encoder
defstruct [:name]
def serialize(%__MODULE__{} = struct) do
Jason.encode!(struct)
end
def deserialize(json) do
map = Jason.decode!(json, keys: :atoms!)
struct(__MODULE__, map)
end
end
defmodule Filter2 do
@derive Jason.Encoder
defstruct [:name]
def serialize(%__MODULE__{} = struct) do
Jason.encode!(struct)
end
def deserialize(json) do
map = Jason.decode!(json, keys: :atoms!)
struct(__MODULE__, map)
end
end
defmodule Filters do
def serialize(%{__struct__: type} = struct) do
%{type: type, value: apply(type, :serialize, [struct])}
end
def deserialize(%{type: type, value: json}) do
apply(type, :deserialize, [json])
end
end
Other than a GenServer (Agent) or Registry I’m not sure how to do this. Ideally you want your filters to “register” themselves somewhere and for the Filters module to have a way to look them up.
You can write a function and hook it into your application startup that scans the list of modules and registers them if they are filters?
Having gone down this road many times (trying to build lists of modules dynamically based on properties) I’d advise against it. It can work well enough for production, but in development mode things can get very frustrating very quickly because of incremental recompilation.
My general suggestion for this is to instead have a module that lists all of these, i.e
defmodule MyFilters do
def filters, do: [Filter1, Filter2, Filter3]
end
And then add this:
defmodule Filter do
defmacro __using__(_) do
quote do
@behaviour Filter
unless __MODULE__ in MyFilter.filters() do
raise "Hey, don't forget to put this module into the list in MyFilter"
end
end
end
end
Then in each filter, you say
use Filter
which sets up the behaviour and also validates that you are in the registry of filters. This way, when someone inevitably copies one and renames it, they get a helpful error. Its not as convenient as having a module magically detect other modules, but trust me that kind of thing has so many edge cases.
In the end I chose to give up on the ‘arbitrary’ naming of a serialized item and just added the name of the struct to the map being serialized. I was concerned about adding the full path of the module because if I ever refactor the code and move the module, I would no longer be able to deserialize the data.
defprotocol Serializer do
def serialize(data)
end
defmodule MyApp.Filter do
@callback deserialize(data :: map()) :: struct()
@callback serialize(filter :: struct()) :: map()
defmacro __using__(opts) do
quote do
@behaviour MyApp.Filter
defimpl Serializer do
def serialize(filter) do
apply(@for, :serialize, [filter])
|> Map.put(:__type__, Module.split(@for) |> List.last())
end
end
end
end
def serialize(filter) do
Serializer.serialize(filter)
end
def deserialize(%{"__type__" => type} = data) do
mod = Module.safe_concat(__MODULE__, type)
apply(mod, :deserialize, [data])
end
end
defmodule MyApp.Filter.SomeFilter do
use Filter
defstruct [:id]
def serialize(%__MODULE__{id: id}) do
%{
id: id
}
end
def deserialize(%{"id" => id}) when is_binary(id) do
%__MODULE__{
id: id
}
end
end
This allows me to serialize/deserialize a any filter through the MyApp.Filter module.
filter = %MyApp.Filter.SomeFilter{id: :some_id}
data = MyApp.Filter.serialize(filter)
filter = MyApp.Filter.deserialize(data)
There are a few details omitted which are not important to the discussion at hand. This serialization/deserialization does not do the JSON encoding at this level since this is only part of the the overall structure that is getting serialized.
