Zero-cost abstraction for NewTypes in Elixir

Another approach would be to use a 2-tuple with the first element denoting the “newtype”:

defmodule Location do
  def new(name), do: {:location, name}
end

(resemblance to Erlang records not entirely accidental)

Then the callsite looks the same as in the struct case:

MyModule.new("Metallica", Location.new("U.S."), Genre.new("Heavy Metal"))

But the implementation is a little different:

@spec new(artist :: Artist.t, country :: Country.t, genre :: Genre.t) :: Artist.t
def new({:artist, n}, {:country, c}, {:genre, g}) do
  %Artist{
    name: n, country: c, genre: g
  }
end

An additional thought: that signature for new/3 looks a lot like a keyword list without the list-ness.

Named arguments wouldn’t prevent mis-configuration quite as well as types, but would produce a moderately-string error signal since writing artist: params[:country] looks weird

2 Likes

Mixing your suggestion with @LostKobrakai suggestion, a possible implementation would be:

defmodule Location do
   @type t :: {:location, String.t}

   def new(name), do: {:location, name}
  
   def get({:location, name}), do: name
end

User code:

loc = Location.new("U.S.")

# Instead of Scala's `loc.name` we would do `Location.get(loc)`
locataion_name = Location.get(loc) 

In MyModule:

@spec new(artist_name :: String.t, country :: Country.t, genre :: Genre.t) :: Artist.t
def new(artist_name, country, genre) do
  %Artist{
    name: artist_name, country: Country.get(country), genre: Genre.get(genre)
  }
end

MyModule.new("Metallica", Location.new("U.S."), Genre.new("Heavy Metal"))

I can honestly see both options working.
In both cases, we would get a compiler warning (via Dialyzer or Gradient) for calling the function with parameters swapped.

You can look at Record, which would remove a bunch of the boilerplate and give you a common API.

1 Like

I checked out Record and tried to use it, but unfortunately it has one fundamental flaw for this specific use case:

  • It requires every field has a default value

So for example:

defmodule Genre do
  require Record

  Record.defrecord(:genre, :name)

  @type t :: record(name: :hard_rock | :heavy_metal | :pop)
end

Won’t compile, because :name has no default.
you could argue “Just use nil as a default”, but I really don’t want that to be possible. In this case, for example, a genre can be 1 of three things, nil is not one of them.

However because something like this is possible:

import Genre

# To create records
record = genre()        # this should not be possible 
record = genre(name: :hard_rock) #=> {:name, :hard_rock}

The idea falls apart. In contrast, with the previous approaches, dialyzer would pick up such cases and report them as incorrect.

This is unfortunate, as this is the almost perfect solution for the NewType abstraction I am looking for.

This is very interesting. Is a zero-cost NewType abstraction similar to a value object in OOP?

1 Like

I thought @opaque types were intended for this purpose. So I tried the following example but unfortunately it seems it’s not working and both Dialyzer and Elixir-LS don’t report a warning:

  @opaque artist :: binary
  @opaque title :: binary

  @spec artist!(binary) :: artist
  def artist!(a), do: a
  @spec title!(binary) :: title
  def title!(t), do: t

  @spec create(artist, title) :: %{name: artist, title: title}
  def create(name, title), do: %{name: name, title: title}

  def test() do
    # Wrong order of arguments, but no warning/error. :(
    create(title!("Title"), artist!("Artist"))
  end
1 Like

There was a discussion recently about changing the way dialyser matches on specs. Perhaps it might help you here?

Of course not, everything happens in the same module, and the module that defines an @opaque type has access to it’s internals, and is therefore allowed to use any binary without explicitly “converting” it.

2 Likes

No, NewType is more like a new primitive type, like Strings or Integers. Value objects are a different concept not related to algebraic data types.

Interesting read though!

2 Likes

That discussion is not about “changing dialyzer”, but more about finding flags to use in order to detect specific error cases.

This discussion is geared more towards finding a cheap way to define a new type in Elixir. Since it is a new type, it means that dialzyer’s default algorithm would always be able to find incorrect invocations, without the need for underspec or overpsec flags.

1 Like

I’d go with @aziz solution. I usually define the API for other modules to consume so it doesn’t hurt that much that the solution does not work in the same module. It works in other modules:

defmodule Title do
  @opaque t :: binary
  @spec new(binary) :: t()
  def new(t), do: t
end

defmodule Artist do
  @opaque t :: binary
  @spec new(binary) :: t()
  def new(a), do: a
end

defmodule Song do
  @opaque t :: %{artist: Artist.t(), title: Title.t()}
  @spec create(Artist.t, Title.t) :: t()
  def create(artist, title), do: %{artist: artist, title: title}
end

defmodule Test do
  alias Title
  alias Artist
  alias Song

  def test() do
    title = Title.new("Title")
    artist = Artist.new("Artist")

    # Wrong order of arguments causes Dialzyer error
    Song.create(title, artist)
  end
end

mix dialyzer produces

lib/zero_cost.ex:24:no_return
Function test/0 has no local return.
________________________________________________________________________________
lib/zero_cost.ex:29:call_without_opaque
Function call without opaqueness type mismatch.

Call does not have expected opaque terms in the 1st and 2nd position.

Song.create(_title :: Title.t(), _artist :: Artist.t())

________________________________________________________________________________
done (warnings were emitted)
Halting VM with exit status 2

Rafał Studnicki used this idea in one of his projects: Rafal Studnicki - The Alchemist's Code: Bringing More Value with Less Magic | Code Elixir LDN 19 - YouTube He goes even further and with those types :slight_smile:

However, there is some boilerplate involved and with dialyzer cryptic errors, I don’t see this solution getting too much traction :slight_smile:

7 Likes

That’s cool, @tomekowal! :+1: I wonder if nested modules would work? I’m sure the boilerplate code can be reduced significantly with macros. And the dialyzer error messages being cryptic… well, our brains get used to it after some time and we don’t try to understand the words anymore and just take the line number and try to fix the issue there :joy:. (Of course, if you never started out with dialyzer it’s gonna be tough. :confounded:)

Thanks for the pointer to video!

Edit: Yep, it works also when you move the Title & Artist modules into the Song module. :ok_hand:

Just made a reduced example that works wonderfully.

defmodule Song do
  @opaque title :: binary
  @spec title!(binary) :: title()
  def title!("" <> t), do: t

  @opaque artist :: binary
  @spec artist!(binary) :: artist()
  def artist!("" <> a), do: a

  @type t :: %{artist: artist, title: title}
  @spec create(artist, title) :: t()
  def create(artist, title), do: %{artist: artist, title: title}
end

defmodule Test do
  def test() do
    # No error:
    Song.create(Song.artist!("Artist"), Song.title!("Title"))
    # Wrong order of arguments causes Dialyzer error:
    Song.create(Song.title!("Title"), Song.artist!("Artist"))
  end
end

Edit: Song.t() can be a normal type, no need for it to be opaque.

2 Likes

Just to make it obvious (because I was fooled by it) @tomekowal’s code causes the dialyzer errors for actually mismatched types, not for actually correct code. I totally didn’t see the comment in Test.test/0.

1 Like

Before you jump around euphorically and introduce this idea in your code base, here are some issues you should be aware of:

defmodule Zero_Cost_Type_Wrapper_Issues do
  def issues() do
    song = Song.create(Song.artist!("Artist"), Song.title!("Title"))

    # warning Function issues/0 has no local return
    _issue1 = "#{inspect(song.title)} by #{song.artist}"
    _fix1a = "#{inspect(song.title)} by #{as_string(song.artist)}"
    _fix1b = "#{inspect(song.title)} by #{song.artist <> ""}"

    case song.title do
      # warning The attempt to match a term of type
      #                   'Elixir.Song':title() against the pattern
      #                   <<65,_/binary>> breaks the opacity of the term
      "A" <> _issue2 -> :A
      _ -> nil
    end

    # Fix:
    case as_string(song.title) do
      "A" <> _fix2 -> :A
      _ -> nil
    end
  end

  @spec as_string(any) :: binary
  def as_string(s), do: s
end

These are 2 issues I could discover immediately. It’s very likely there are a few others. Another solution is to simply use binary as the type of the struct/map attributes and only give the opaque types to the function parameters.

Yeah, that is the strength of opaque type. It enforces proper encapsulation. Basically, you cannot rely anymore on the type being binary. You need song!\1 or Song.new\1 to tell Dialyzer "I am explicitly converting binary to song. Then, you’d have to do the other way round. Song.get(song) :: binary. That’s why I’d rather define each opaque type as its own separate module. You’ll need constructor and getters.

It is great when you have a struct with private fields that you want to ensure, that nobody else uses. Or if you think the internal implementation can change (e.g. using map vs keyword list) and you don’t want users to rely on it. It is also great when you have some kind of “handle” that can be a PID or an in-memory struct.

However, it is not exactly the same as zero-cost abstraction in other langs.

1 Like

After reading more I finally came up with a small macro that can be used as a proof of concept, a trivial implementation of the NewType in Elixir.

While it is not a zero cost abstraction, it is very lightweight and I seriously doubt anyone would run into considerable performance issues using it.

I expanded over the idea of @tomekowal and used tuples, mainly to check for type validation. I am sure there is another way of doing the same is_type?/2 function without using tuples, but I think this is fast enough and the code is clear enough, so no harm done:

defmodule NewType do
  defmacro deftype(name, type) do
    quote do
      defmodule unquote(name) do
        @opaque t :: {unquote(name), unquote(type)}

        @spec new(value :: unquote(type)) :: t
        def new(value), do: {unquote(name), value}

        @spec extract(new_type :: t) :: unquote(type)
        def extract({unquote(name), value}), do: value
      end
    end
  end

  @spec is_type?(data :: {atom, any}, new_type :: atom) :: boolean
  def is_type?({type, _data}, new_type) when type == new_type, do: true
  def is_type?(_data, _new_type), do: false
end

To see the full context:

1 Like

Personally I do not see much value in this over just using a tagged tuple directly.

Though it seems you want to have it like this, as it suites some usecase you have, I am totally fine with that.

Still I have some nitpick about the is_type?/2 predicates naming. has_ and is_ prefixes are only used for guards, while ? suffix is left out for guards. As you have a non guard function, you should therefore name it type?.

Also the function might be more idiomatically written by fully using pattern match rather than quards:

  @spec is_type?(data :: {atom, any}, new_type :: atom) :: boolean
  def is_type?({type, _data}, type), do: true
  def is_type?(_data, _new_type), do: false

Please be also aware, that calling the predicate with any @opaque type will make dialyzer complain.

2 Likes

What about Map.has_key?
I do like you suggestion though :smiley:

Agreed !

This is quite an interesting conversation. I very much like the example with opaque Dialyzer types which @tomekowal provided.

One thing that might be tried, is adding @compile :inline to the module(s) in which these functions (like Song.new/1 and Song.get/1) are used. This might* convince the Erlang compiler to inline their definitions, which would truly make them “zero cost” at runtime.

*: The rules for when particular functions are and aren’t inlined are a bit vague. Local functions are very easily inlined. Remote functions only sometimes, and I am not entirely sure when (the documentation is not very clear on this, or rather, what I see in practice does not seems to always follow the documentation. If some expert can shed light on it, I’d be very grateful!)

2 Likes