Fl4m3Ph03n1x

Fl4m3Ph03n1x

Zero-cost abstraction for NewTypes in Elixir

Background

Recently I have discovered this notion of “zero cost type wrappers”. Basically what this means is that you can create a new type by wrapping a primitive type (and the cost of doing this is low to non-existent). This new type would serve as an additional layer of abstraction and prevent certain categories of bugs at compile time.

For example, let’s assume we have this function (assume we have an Artist struct):

@spec new(artist :: String.t, country :: String.t, genre :: String.t) :: Artist.t
def new(n, c, g) do
  %Artist{
    name: n, country: c, genre: g
  }
end

Now obviously I added the specs here for help. But you will notice that everything is String.t. This basically means I can incorrectly invoke this function:

MyModule.new("U.S.", "Metallica", "Heavy Metal") # name and country and swapped

The compiler would not complain.

NewType abstraction

To solve this issue, some people came up with this notion of wrapping primitive types into an abstraction. If you are from Scala you may know this as “Zero-cost abstraction for NewTypes”, if you are from Rust you may know it as the NewType Pattern and so on (this is a feature present in many languages these days).

scala code

opaque type Location = String
object Location{
  def apply(value: String): Location = value
  extension(a: Location) def name: String = a
}

This would create a new type called Location that wraps the String primitive type.

In Elixir, our function’s signature would now be:

@spec new(artist :: String.t, country :: Location.t, genre :: String.t) :: Artist.t

(you can also do the same for genre)

Elixir NewType wrappers?

Now, using the power of typespecs I could do something like:

@type location :: String.t()

And use it in my specs. But this would serve merely as documentation and would prevent no types of errors whatsoever.

The closest thing that comes to my mind, would be to define a struct:

defmodule Location do
  defstruct [:name]

  @type t :: %__MODULE__{name: String.t()}

  @spec new(name :: String.t()) :: __MODULE__.t()
  def new(name), do: %__MODULE__{name: name}
end

Ignoring the boilerplate code (we can just create a macro for that!) I think this is the closest I can get to having something like the NewType abstraction.

This would allow us to invoke the function like this:

MyModule.new("Metallica", Location.new("U.S."), Genre.new("Heavy Metal"))

We can’t swap parameters and have thus eliminated a category of errors. Further more, we did this at compile time.

Would it be zero-cost? I don’t think so, since I am replacing a String.t with a map that has 1 key. The overhead would probably be minimal, but I don’t think I could call it zero cost.

Questions

  • How would you implement this abstraction in Elixir?
  • Are there any optimizations one could do here?
  • Is it possible to have a compile time check that prevents this category of errors using typespecs only? (I don’t think so, but please feel free to prove me wrong)

First 10 of 28 Posts Switch mode

csadewa

csadewa

Hmmm, zero-runtime cost mean somehow the work is done on compile time? i wonder if it’s possible to do this via macro, which run at compile time. but i think in that case the obvious limitation would be that the checking would only work if when the value is known at compile time (static).

LostKobrakai

LostKobrakai

While I can see the usefulness of the general pattern one has to acknowledge that elixir is not a statically typed language. The compiler only has limited knowledge around the types of data at compile time (especially around everything message passing). So the remaining options are runtime checks.

As you noted things won’t be “zero cost” at runtime. The smallest way to add information to a piece of data would be a number per type or on the beam an atom, which at runtime is also basically a number. Erlang usually uses records for that {:user, "someone"}. Tuples afaik come with very little overhead in actual memory over just the two values itself (1). In elixir we usually don’t use tuples/records that much, we use maps/structs. Luckily small maps (<32 keys) are stored in (again afaik) a similar memory layout to tuples. Still a bit more overhead, but less than for large maps.

So those are the options to look at imo for deciding if the runtime hit is worthwhile. Generally I feel like structs are a good way to “type” data, but I wouldn’t do it for every scalar floating around in your system, but rather things which are reasonable entities or values in your system. E.g. Location in an event booking system makes sense. Wrapping every city string probably not.

[1] Memory Usage — Erlang System Documentation v29.0.2

Fl4m3Ph03n1x

Fl4m3Ph03n1x OP

This is very interesting. But if I were to use a tuple, my signature would have to be:

@spec new(artist :: String.t, country :: {:name, String.t}, ... ) :: Artist.t

Instead of:

@spec new(artist :: String.t, country :: Location.t, ...) :: Artist.t

Right?

I would also need to extract the value via elem/2.

An idea worth exploring though, thanks !

LostKobrakai

LostKobrakai

defmodule Location do
  @type t :: {:location, String.t}
end

But yes you’d have to unwrap the value - just like with a struct. This is runtime data we’re dealing with. You cannot implicitly wrap a string to be tagged “a location” and the runtime would infer that tag from the plain string received. There’s things you can do in a statically typed language you simply cannot do if you don’t have a statically typed language.

Exadra37

Exadra37

I will try to help, but after so many months out of Elixir my understanding may be cloudy…

I kind of tried to achieve this on my own in the past, but then resorted to use the Domo library.

I also know of the Typed Struct library that I have not tried yet.

Does any of this libs can help you achieve what you want?

Fl4m3Ph03n1x

Fl4m3Ph03n1x OP

Thank you for trying!

When I want structs about something, I usually use TypedStruct. Some people I know use Embedded Ecto schemas.

However, here the purpose is different. Even though I am using a struct, my objective is not to make “using structs easier” (like it is with TypedStruct). My objective here is to simply wrap a primitive value into an abstraction that allows dialyzer (or gradient) to complain.

If anything, structs are an implementation detail that I would hide under the hood of a macro :smiley:

mat-hek

mat-hek

Membrane Core Team

I usually use a single argument that’s a keyword list, map or struct in such cases, like

@spec new(artist: String.t, country: String.t, genre: String.t) :: Artist.t

or

@spec new(%{artist: String.t, country: String.t, genre: String.t}) :: Artist.t

or

@spec new(%ArtistConfig{artist: String.t, country: String.t, genre: String.t}) :: Artist.t

Though it’s more verbose, I find it more readable and Dialyxir is theoretically able to find bugs there

Exadra37

Exadra37

So, what I use currently to avoid bad data popping in at runtime is this approach:

defmodule TypeIt.Progress do

  use Domo

  @all_states %{
    backlog: "Backlog",
    todo: "Todo",
    doing: "Doing",
    pending: "Pending",
    done: "Done",
    archived: "Archived",
  }

  @states Map.keys(@all_states)

  typedstruct do
    field :state, :backlog | :todo | :doing | :pending | :done | :archived
    field :title, String.t()
    field :since, NaiveDateTime.t()
  end

  def default(), do: new_for!(:todo)

  def next_state(:backlog), do: :todo
  def next_state(:todo), do: :done
  def next_state(:done), do: :todo

  def new_for!(state), do: new!(state: state, title: @all_states[state], since: NaiveDateTime.utc_now())
  def new_for!(state, since: since), do: new!(state: state, title: @all_states[state], since: since)
  def new_for!(state, title: title), do: new!(state: state, title: title, since: NaiveDateTime.utc_now())

  def states() do
    @states
  end

  def all() do
    @all_states
  end
end

For what I understand you want to make it possible only with compile time checks, but from my understanding that’s not possible in the BEAM, but I really hope you find a solution to be only compile time check. Let me know when you find it that I can help you testing it.

The Domo library I use here adds type specs for me when the code is compiled to help Dialyzer to catch as much as possible, and the rest I have to code it by my self to be checked at runtime, but with the caveat that one using the Struct can always bypass what I implemented by not calling the provided functions to create and manipulate the struct.

al2o3cr

al2o3cr

Another approach would be to use a 2-tuple with the first element denoting the “newtype”:

defmodule Location do
  def new(name), do: {:location, name}
end

(resemblance to Erlang records not entirely accidental)

Then the callsite looks the same as in the struct case:

MyModule.new("Metallica", Location.new("U.S."), Genre.new("Heavy Metal"))

But the implementation is a little different:

@spec new(artist :: Artist.t, country :: Country.t, genre :: Genre.t) :: Artist.t
def new({:artist, n}, {:country, c}, {:genre, g}) do
  %Artist{
    name: n, country: c, genre: g
  }
end

An additional thought: that signature for new/3 looks a lot like a keyword list without the list-ness.

Named arguments wouldn’t prevent mis-configuration quite as well as types, but would produce a moderately-string error signal since writing artist: params[:country] looks weird

Fl4m3Ph03n1x

Fl4m3Ph03n1x OP

Mixing your suggestion with @LostKobrakai suggestion, a possible implementation would be:

defmodule Location do
   @type t :: {:location, String.t}

   def new(name), do: {:location, name}
  
   def get({:location, name}), do: name
end

User code:

loc = Location.new("U.S.")

# Instead of Scala's `loc.name` we would do `Location.get(loc)`
locataion_name = Location.get(loc) 

In MyModule:

@spec new(artist_name :: String.t, country :: Country.t, genre :: Genre.t) :: Artist.t
def new(artist_name, country, genre) do
  %Artist{
    name: artist_name, country: Country.get(country), genre: Genre.get(genre)
  }
end

MyModule.new("Metallica", Location.new("U.S."), Genre.new("Heavy Metal"))

I can honestly see both options working.
In both cases, we would get a compiler warning (via Dialyzer or Gradient) for calling the function with parameters swapped.

Where Next?

Trending in Discussions Top

AstonJ
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
2977 91561 914
New
byu
@chrismccord : I just saw the Extract AGENTS.md from Phoenix.new into phx.new generator commit to the phoenix project. My initial shotgu...
New
arcanemachine
I was working on an Ecto migration and I needed a timestamp. So, for the nth time, I looked up the different data types for timestamps, a...
New
AstonJ
Just a general thread to post chat/news/info relating to AI/ML stuff that may be relevant for Nx now or in the future. Got anything to sh...
New
type1fool
I just stumbled on a newly redesigned elixir-lang.org. :tada: It looks like @Software_Mansion did the work, and I think it is generally a...
New
juhalehtonen
There has been a thread to discuss the Stack Overflow Developer Survey on this forum every year since 2018, so here’s yet another one for...
New
alexslade
Fly’s CEO posted this recently - Turn And Face The Strange · The Fly Blog It says that Fly is going all-in on sprites, which is a worry ...
New

Other Trending Topics Top

JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge &amp; Solve. They are GUI (Emerge) and State management (S...
New
ausimian
Emily is an Elixir library that runs Nx computations on Apple’s MLX. Install it as the default Nx backend and Nx, defn, Axon, Nx.Serving,...
New
akoutmos
@hugobarauna and I (Alex Koutmos) have been hard at work on writing a book on Nerves that takes you from simply blinking LEDs to building...
New
bjorng
We want to introduce a new native datatype to Erlang: native records. Although replacing all tuple records with native records is not our...
New

We're in Beta

About us Mission Statement