User defined tags and type-safe structs for domain modelling (Domo library)

There is one long term request - to have strict type checking in Elixir.
As far as I remember, @bcardarella mentioned that in the talk at Lonestar ElixirConf 2019. And after reading the How to make Dialyzer more strict? it’s clear that dialyzer is good to have and at the same time it’s not enough to check all possible function calls automatically. And eventually, there is no “one-button” tool to verify the data consistency throughout the app at the compile-time.
We have no Haskel like type inference with dialyzer types, that seems required to do that. And according to @josevalim, it seems impossible to introduce a native type system into the Elixir language itself without breaking compatibility with existing applications code.
There are several projects for building a compile-time type-safe BEAM language from the ground-up like Alpaca, Gleam, and some others Statically typed languages on BEAM. Same time, integration with other BEAM running software can be an issue according to @rvirding.
There are Elixir libraries f.e. ExType by @gyson, and proof of concepts like TypedElixir and MLElixir by @OvermindDL1. These seem are for making the strict equivalent of the dialyzer tool.
It looks like a not too optimistic situation.

It’d be good to look at the problem from a different angle. For what we need strict types when writing an application?
Usually, to check the consistency of states of business data, we operate in the app.
To model the data, we make structures, take out values from them, process them, and assemble back.
Here is an idea - we can match the value type with defined field type each time we assemble a struct at run-time, and raise or return an error on mismatch. And to differentiate the same typed values by the business meaning, we can wrap them into tagged tuples. That is, to keep the link of the separated value to the concrete struct with a tuple’s tag.
With means of the Domo library, that I’m glad to present, it’s expressed like the following.

defmodule Order do
  use Domo

  alias Measure.{Kilograms, Units}

  deftag Id, for_type: String.t()
  deftag Quantity, for_type: Kilograms.t() | Units.t()
  deftag Note, for_type: :none | String.t()

  typedstruct do
    field :id, Id.t()
    field :quantity, Quantity.t()
    field :note, Note.t(), default: Note --- :none
  end
end

Order.new!(
  id: Id --- "156",
  quantity: Quantity --- Kilograms --- 2.5
  note: Note --- "Deliver on Tue"
)

The dialyzer can check the correctness of data assembly in the context of struct-value relation at the compile-time. At the run-time, the autogenerated new!/3, put!/3, and merge!/3 functions of the struct automatically matches values themselves against field type.

In combination with the expat library by @vic , the processing of the tags can be neat. Domo is fully compatible with the existing Elixir code bases and can be introduced in existing projects gracefully. More examples are in the Domo library’s documentation (the master version of Elixir is necessary to run). There is an example app included in the library’s repo.

Folks, is it an absurd idea to ask you to give feedback on the approach presented, and how it can be made even better?

3 Likes

Add proper encapsulation) I had some fun with solving similar issues in Elixir, and encapsulation and new types was the most interesting, finally kinda got solved with this (link to article with explanation)

And then I used Calculus to build functional programming library

But at some point, I just realised that it’s easier just to use Haskell where all this kind of problems are solved many years ago)

It seems the Haskel was your tool of choice. :grinning:

The Domo library is about flexibility - to give a possibility of type-safe structs in existing codebases written in Elixir.

Yeah) Once I had started with Haskell, I always wanted more - more newtypes, smart constructors, type classes, generics, dependent types, type families… ohhh boyyy, it’s sooo huge unknown universe. Even after a year of Haskell production I think I know (understand) just around 25% of already existing stuff there, even if we exclude some overwhelming things related to Idris and Agda advanced type systems.

But regarding Erlang/Elixir I kinda gave up finally, it’s untyped languages anyway whatever you are doing with them. But it’s still cool to see people there are looking into type systems.

I think that library can work quite well. I don’t have any immediate remarks on it.

The main trouble is going to be social, not technical however. This effectively kind of changes Elixir (if I am understanding your Order.new! snippet correctly) and many people wouldn’t agree to adopt such a new syntax in their projects.

I also kind of gave up on the idea that the BEAM languages will have actual strong static typing for now (except maybe for Gleam that draws inspiration from Rust and thus from the ML languages as well; but it’s still figuring out its message sending interaction with the strong static typing IIRC… @lpil, am I off the mark here?).

What I do in my projects is just conservatively use as much pattern matching as possible without losing [a lot of] productivity.

Your idea is good but (a) introduces extra coding (the deftag thingies) and (b) a rather new syntax. IMO dedicated teams can make your library work for their process quite well but many others will choose not to.


If one day I have enough time and energy, I’d invest in a linter (likely as a part of credo) that could e.g. yell at you if you do this:

def do_business_stuff() do
  # ...
  Repo.insert(...)
  |> elem(1)
end

…which wrongly assumes success and blindly returns the second element of the tuple, ignoring the fact that the insert might fail and a Changeset be returned instead of the expected successfully inserted in the DB struct. (And I’ve seen Elixir code exactly like this one.)

But… that particular case could likely be covered with Dialyzer if you only add @spec to this function. Then Dialyzer could tell you that you are expecting a User to be returned but you could also get Changeset. So still not sure. As I said, just ideas that I might get to one day, life allowing (which it absolutely doesn’t for months now).


TL;DR: Looks pretty good but I am not sure it will gain traction with the requirements you put on your users.

2 Likes

We’ve got statically typed messages and processes working great! There’s a few options on the table currently, so now we’re figuring out which is best and what the API should be,

4 Likes

Thanks for kind words.

The Domo library suggests using additional syntax for making/pattern-matching of Tags and for defining type-safe structs.

I usually address the new concept adoption issue in a team with turning up the following idea - give it a try for half an hour, and if you definitely don’t like it it’s always possible to go with something else.

I am also looking to have typed structs in Elixir to be happy enough with the language, but the boilerplate necessary always put me off, and I just go with using pattern matching and guard clauses as much as possible, just like @dimitarvp.

What I am looking for is a cleaner solution:

defmodule AuthIt.UserProfile do
  @enforce_keys [
    id: is_integer/1,
    name: is_binary/1,
  ]
  defstruct @enforce_keys
end

But I guess that this will never be possible :frowning:

1 Like

One possible way to do this is with Domo in more declarative style like this:

➜  domo git:(master) cd example_app
➜  example_app git:(master) iex -S mix
Erlang/OTP 22 [erts-10.6.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe]

Interactive Elixir (1.11.0-dev) - press Ctrl+C to exit (type h() ENTER for help)

iex(1)> defmodule AuthIt.UserProfile do
...(1)>   use Domo
...(1)>
...(1)>   typedstruct do
...(1)>     field :id, integer
...(1)>     field :name, binary
...(1)>   end
...(1)> end
{:module, AuthIt.UserProfile,
 <<70, 79, 82, 49, 0, 0, 31, 176, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 3, 2,
   0, 0, 0, 63, 25, 69, 108, 105, 120, 105, 114, 46, 65, 117, 116, 104, 73, 116,
   46, 85, 115, 101, 114, 80, 114, 111, 102, ...>>, :ok}

iex(2)> alias AuthIt.UserProfile
AuthIt.UserProfile

iex(3)> UserProfile.new!(id: 1, name: "John")
%AuthIt.UserProfile{id: 1, name: "John"}

iex(4)> UserProfile.new!(id: "John", name: 1)
** (ArgumentError) Can't construct %AuthIt.UserProfile{...} with new!([id: "John", name: 1])
    Unexpected value type for the field :id. The value "John" doesn't match the integer type.
    Unexpected value type for the field :name. The value 1 doesn't match the binary type.
    lib/domo/struct_functions_generator.ex:20: AuthIt.UserProfile.new!/1

The field types in the field macro are from the usual @type spec.

If you may want to distinguish between different ids in the app, then tags come into play. With

deftag AuthIt.UserProfile.Id, for_type: integer

you can do the following: UserProfile.new!(id: UserProfile.Id --- 1, name: "John") and get the exception/error for UserProfile.new!(id: 1, name: "John") where 1 is not a tagged integer. :blush:

2 Likes

I come from a background of dynamic interpreted languages, all this compilers stuff is new to me, thus sometimes I get things in the wrong way.

This being said I like your declarative way example of using the example I gave, but this:

I just don’t get it, and this tags was what made me to not like Domo in the moment I saw it. I just don’t grasp them and they have a weird syntax.

That’s a good example and it’s something that I could see myself doing because I go 99% the way anyway (by using guards in new / create functions). Yours is a bit cleaner. :slight_smile:

2 Likes

Thanks for pointing out the confusion with the example.

Domo brings several possible benefits. And more explicit version of the example can be like this:

mix new try_domo && cd try_domo && asdf local elixir master

sed -i '' 's/# {:dep_from_hexpm, "~> 0.3.0"}/{:domo, ">= 0.0.0"}/g' ./mix.exs

echo """
defmodule Order do
  use Domo

  deftag Id, for_type: String.t()

  typedstruct do
    field :id, Id.t()
    field :note, :none | String.t(), default: :none
    field :quantity, integer
  end
end
""" > lib/order.ex

echo """
defmodule User do
  use Domo

  deftag Id, for_type: String.t()

  typedstruct do
    field :id, Id.t()
    field :name, String.t()
  end
end
""" > lib/user.ex

mix deps.get && iex -S mix

iex(1)> import Domo
iex(2)> alias Order.Id
iex(3)> Order.new!(
...(3)>   id: Id --- "156",
...(3)>   quantity: 150
...(3)> )
%Order{
  id: {Order.Id, "156"},
  note: :none,
  quantity: 150
}
iex(4)> Order.new!(
...(4)>   id: Id --- "157",
...(4)>   note: "Clear out the delivery date",
...(4)>   quantity: 220
...(4)> )
%Order{
  id: {Order.Id, "157"},
  note: "Clear out the delivery date",
  quantity: 220
}
iex(5)> Order.new!(
...(5)>   id: User.Id --- "157",
...(5)>   quantity: 540
...(5)> )
** (ArgumentError) Can't construct %Order{...} with new!([id: {User.Id, "157"}, quantity: 540])
    Unexpected value type for the field :id. The value {User.Id, "157"} doesn't match the Id.t() type.
    (try_domo 0.1.0) lib/domo/struct_functions_generator.ex:20: Order.new!/1
    (stdlib 3.11.1) erl_eval.erl:680: :erl_eval.do_apply/6
    (iex 1.11.0-dev) lib/iex/evaluator.ex:258: IEx.Evaluator.handle_eval/5
iex(6)> Order.new!(
...(6)>   id: "156",
...(6)>   note: nil,
...(6)>   quantity: "One hundred fifty"
...(6)> )
** (ArgumentError) Can't construct %Order{...} with new!([id: "156", note: nil, quantity: "One hundred fifty"])
    Unexpected value type for the field :id. The value "156" doesn't match the Id.t() type.
    Unexpected value type for the field :note. The value nil doesn't match the :none | String.t() type.
    Unexpected value type for the field :quantity. The value "One hundred fifty" doesn't match the integer type.
    (try_domo 0.1.0) lib/domo/struct_functions_generator.ex:20: Order.new!/1
    (stdlib 3.11.1) erl_eval.erl:680: :erl_eval.do_apply/6
    (iex 1.11.0-dev) lib/iex/evaluator.ex:258: IEx.Evaluator.handle_eval/5
1 Like