TypeCheck - Fast and flexible runtime type-checking for your Elixir projects

[Qqwy]

TypeCheck: Fast and flexible runtime type-checking for your Elixir projects.

Core ideas

• Type- and function specifications are constructed using (essentially) the same syntax as Elixir’s built-in typespecs.
• When a value does not match a type check, the user is shown human-friendly error messages.
• Types and type-checks are generated at compiletime.
  • This means type-checking code is optimized rigorously by the compiler.
• Property-checking generators can be extracted from type specifications without extra work.
• Flexibility to add custom checks: Subparts of a type can be named, and ‘type guards’ can be specified to restrict what values are allowed to match that refer to these types.

Usage Example

defmodule User do
  use TypeCheck
  defstruct [:name, :age]

  @type! t :: %User{name: binary, age: integer}
end

defmodule AgeCheck do
  use TypeCheck

  @spec! user_older_than?(User.t, integer) :: boolean
  def user_older_than?(user, age) do
    user.age >= age
  end
end

Now we can try the following:

iex> AgeCheck.user_older_than?(%User{name: "Qqwy", age: 11}, 10)
true
iex> AgeCheck.user_older_than?(%User{name: "Qqwy", age: 9}, 10)
false

So far so good. Now let’s see what happens when we pass values that are incorrect:

iex> AgeCheck.user_older_than?("foobar", 42)
** (TypeCheck.TypeError) At lib/type_check_example.ex:28:
The call to `user_older_than?/2` failed,
because parameter no. 1 does not adhere to the spec `%User{age: integer(), name: binary()}`.
Rather, its value is: `"foobar"`.
Details:
  The call `user_older_than?("foobar", 42)` 
  does not adhere to spec `user_older_than?(%User{age: integer(), name: binary()},  integer()) :: boolean()`. Reason:
    parameter no. 1:
      `"foobar"` does not check against `%User{age: integer(), name: binary()}`. Reason:
        `"foobar"` is not a map.
    (type_check_example 0.1.0) lib/type_check_example.ex:28: AgeCheck.user_older_than?/2

iex> AgeCheck.user_older_than?(%User{name: nil, age: 11}, 10)
** (TypeCheck.TypeError) At lib/type_check_example.ex:28:
The call to `user_older_than?/2` failed,
because parameter no. 1 does not adhere to the spec `%User{age: integer(), name: binary()}`.
Rather, its value is: `%User{age: 11, name: nil}`.
Details:
  The call `user_older_than?(%User{age: 11, name: nil}, 10)` 
  does not adhere to spec `user_older_than?(%User{age: integer(), name: binary()},  integer()) :: boolean()`. Reason:
    parameter no. 1:
      `%User{age: 11, name: nil}` does not check against `%User{age: integer(), name: binary()}`. Reason:
        under key `:name`:
          `nil` is not a binary.
    (type_check_example 0.1.0) lib/type_check_example.ex:28: AgeCheck.user_older_than?/2

iex> AgeCheck.user_older_than?(%User{name: "Aaron", age: nil}, 10) 
** (TypeCheck.TypeError) At lib/type_check_example.ex:28:
The call to `user_older_than?/2` failed,
because parameter no. 1 does not adhere to the spec `%User{age: integer(), name: binary()}`.
Rather, its value is: `%User{age: nil, name: "Aaron"}`.
Details:
  The call `user_older_than?(%User{age: nil, name: "Aaron"}, 10)` 
  does not adhere to spec `user_older_than?(%User{age: integer(), name: binary()},  integer()) :: boolean()`. Reason:
    parameter no. 1:
      `%User{age: nil, name: "Aaron"}` does not check against `%User{age: integer(), name: binary()}`. Reason:
        under key `:age`:
          `nil` is not an integer.
    (type_check_example 0.1.0) lib/type_check_example.ex:28: AgeCheck.user_older_than?/2

    
iex> AgeCheck.user_older_than?(%User{name: "José", age: 11}, 10.0) 
** (TypeCheck.TypeError) At lib/type_check_example.ex:28:
The call to `user_older_than?/2` failed,
because parameter no. 2 does not adhere to the spec `integer()`.
Rather, its value is: `10.0`.
Details:
  The call `user_older_than?(%User{age: 11, name: "José"}, 10.0)` 
  does not adhere to spec `user_older_than?(%User{age: integer(), name: binary()},  integer()) :: boolean()`. Reason:
    parameter no. 2:
      `10.0` is not an integer.
    (type_check_example 0.1.0) lib/type_check_example.ex:28: AgeCheck.user_older_than?/2

And if we were to introduce an error in the function definition:

defmodule AgeCheck do
  use TypeCheck

  @spec! user_older_than?(User.t, integer) :: boolean
  def user_older_than?(user, age) do
    user.age
  end
end

Then we get a nice error message explaining that problem as well:

** (TypeCheck.TypeError) The call to `user_older_than?/2` failed,
because the returned result does not adhere to the spec `boolean()`.
Rather, its value is: `26`.
Details:
  The result of calling `user_older_than?(%User{age: 26, name: "Marten"}, 10)` 
  does not adhere to spec `user_older_than?(%User{age: integer(), name: binary()},  integer()) :: boolean()`. Reason:
    Returned result:
      `26` is not a boolean.
    (type_check_example 0.1.0) lib/type_check_example.ex:28: AgeCheck.user_older_than?/2

---

While TypeCheck is not stable yet, it is mature enough to be used for simple tasks.

Please try it out and share your experiences and feedback here!

---

If you like videos, also see my 2022 ElixirConf.EU talk about TypeCheck:

TypeCheck Effortless Runtime Type Checking | Wiebe-Marten Wijnja | ElixirConf EU 2022

[hauleth]

What is runtime penalty for using this?

[ityonemo]

I recommend checking the @spec values themselves. If you’re curious how to do this I made this library (which I later decided I hated) which can inspect the type specs of remote modules: https://github.com/ityonemo/typed_headers

Also this is superficial but in your example note the convention with regard to is_ functions and ? functions.

[Qqwy]

Great question! Because the type-checking code is generated and added at compile-time it gets optimized by the compiler, which means that redundant checks are eliminated from the code and you end up with something that is quite performant.

How fast it will be exactly of course depends on what kind of type you are checking against. TypeCheck performs exhaustive checking of e.g. the elements in a list, which takes longer when the list is longer.
For simple datatypes, the type-check is a constant-time operation.

As an example, the following:

defmodule Example do
  use TypeCheck
  spec add(number(), number()) :: number()
  def add(a, b) do
    a + b
  end
end

is turned into something like:

defmodule Example do
  use TypeCheck
  @spec add(number(), number()) :: number()
  def add(a, b) do
    unless is_number(a) do
      raise "param a does not match type number()"
    end
    unless is_number(b) do
      raise "param b does not match type number()"
    end

    result = a + b

    unless is_number(result) do
      raise "the result does not match type number()"
    end
  end
end

(This is somewhat simplified; in the actual implementation we pass nested errors back up as values before they are raised as exception to be able to have human-friendly error messages.)

So a general answer would be ‘for most situations the performance penalty is probably negligible’.

I definitely want to write a couple of benchmarks to be able to give a more scientific answer in the future. And it might also be able to further optimize the code that is generated (allowing the compiler to do even more optimizations) in the future as well.

[Qqwy]

@ityonemo thank you! The example in the first post (and the README) has been updated. That’s what you get when writing documentation while burning the midnight oil .

The main reason I chose to not read the @spec-values themselves is to allow people to ‘opt-out’ of the @spec/@type etc. that are being generated, which can be useful in some cases. It also allows support for certain kinds of extensions that Elixir’s builtin typespecs cannot handle, like adding named types, ‘type guards’ or certain type-shorthands like tuple(3).

I do wonder what happened which made you dislike your library later on; it seems like quite a bit of effort went into writing that!

---

@Qqwy

As an example, the following:

defmodule Example do
  use TypeCheck
  spec add(number(), number()) :: number()
  def add(a, b) do
    a + b
  end
end

For the curious, here is an example of what kind of BEAM code ends up being generated for an example like this.

Something cool that you can see here is for instance that the return-type check is completely elided since the compiler sees that add will always return a number.

While I’m already quite happy with this, there are a couple of potential improvements for the future I’d like to make as well:

• Potentially inline the wrapped function (but how inlining might work when combined with defoverridable is something I need to still figure out)
• Potentially strip away the code related to tracking ‘named bindings’ wherever they are not used: Currently we return {:ok, []} everywhere, but we can at compile-time figure out whether/where we can just use :ok which would make the job for the compiler a lot simpler; passing a static atom around vs a 2-tuple probably makes quite a difference.
• Simplify the error-response code by hiding the gist of it in an internal function. I hope this might reduce the number of :jump calls that are added to the bytecode, since I want the ‘happy path’ to be as fast as possible.

[baldwindavid]

@Qqwy I’m currently using Norm, but this might end up being a better fit for my needs. I’m solely using Norm for the @contract specifications as a replacement for typespecs, but need to create my own functions to represent everything in typespecs, which feels a bit like recreating the wheel.

Are the specs that can be created in any way limited? One of the things I like about Norm is that anything can be used. Is TypeCheck simply optimizing the parts it can and letting the rest run as it normally would?

I also had the question whether it would be possible to check @spec/@type. It appears that it is and @ityonemo has done it, but that is not a goal here.

I can’t help but envision a package that could check regular @spec/@type at runtime. Those standard typespecs could be used most of the time, but when you need to do something special you can remove the @ and write more complex specifications. If spec could use remote @types that would also be cool, but no idea if that would be possible.

[Qqwy]

@baldwindavid

Are the specs that can be created in any way limited? One of the things I like about Norm is that anything can be used. Is TypeCheck simply optimizing the parts it can and letting the rest run as it normally would?

Correct. TypeCheck optimizes the parts of the type it understands, and if you want to add extra checks you can add a ‘type guard’ which allows arbitrary Elixir code:

type sorted_pair :: {lower :: number(), higher :: number()} when lower <= higher

This indeed optimizes to code that does, in order, the following:

• Checking that we have a tuple.
• Checking that the tuple has 2 elements.
• Checking that the first element is a number (and binding that element to the name ‘lower’)
• Checking that the second element is a number (and binding that element to the name ‘higher’)
• Running the guard code, in this case ‘lower <= higher’.

Does that answer your question?

[ityonemo]

I disliked it for the following reasons.

1. adding typing to headers gets burdensome to read. Surprisingly, I actually rather like @type being outside of the headers now.
2. I stopped and thought, “Do I really need this”? Well. Dialyzer is atrocious, but it’s strictly a matter of ergonomics, and the ergonomics, I find are completely solved by vs_code and elixir_ls. I don’t even bother running dialyzer anymore, my code is fairly well typed and dialyzer really does catch 90% probably of typing errors. If vscode/elixir_ls ever stops making the type suggestions “annoyingly a different size than the code” I will be upset, because that minor “annoyance” is really the thing that drives me to write typespecs. Don’t change that, ever, vscode team.
3. I did still have one typing error make it to prod in like 20k LOC. I found it months later because i was idle and just killing bugs that cropping up on appsignal. BEAM will really save your ass in these sorts of situations, and honestly, it was really “not a big deal” Of course, in this case, too, vs code had that yellow squiggly that I had just overlooked.
4. Elixir is already helpfully stronger in typing, if only culturally, in ways that Erlang never was (with the way that structs really nudge you to type out), for example, or how Mocks in Mox require you to define the contract.
5. the typing things that I think are “the hardest” are “message passing APIs” and dynamically bound modules. But in 99% of cases your entire message passing should live in the same module that defines the GenServer interface, so if you’re careful and organized it shouldn’t be a problem. I also use a coding style where my handle_*s are “as dumb as possible” and follow a strict convention, and my GenServer API functions call a private implementation function that I write directly underneath. Ex: https://github.com/ityonemo/erps/blob/master/lib/erps/server.ex#L223 and so this is not a problem. I haven’t found a solution to “dynamically bound modules” yet.
6. It’s bad form to redefine kernel macros (like def/2) unless you really really really have a good reason to. On the other hand, if I were to rename it (defc/2) or something it could cause the reader to have to think a bit. In the end, since I’m hiring out my team, my preference is to stick as much as possible to the existing elixir standard to prevent confusion ahead of having to grow my team (especially as the hires will probably not know elixir to start off).

[baldwindavid]

This is a really impressive and useful package. I tested it out on quite a bit of code (about 400 functions) and didn’t really run into any issues with the code. Ergonomics are the only issue for me in that spec is not syntax highlighted as nicely as something like @spec/@contract and empty newlines appear between the spec and multiline functions, which is almost always for me.

I can’t help but wonder if something like @contract or @check rather than spec might solve these problems.

I still think the ultimate would be if this could additionally do type checking on @spec and @type. Teams could install it and immediately get the benefit of runtime checks that are optimized. And if someone needs to do more, just remove the @ and spec unleashes the power of what this package already does.

Regardless, really nice work!

[Qqwy]

@baldwindavid

This is a really impressive and useful package. I tested it out on quite a bit of code (about 400 functions) and didn’t really run into any issues with the code.

Thank you, great to hear!

@baldwindavid

Ergonomics are the only issue for me in that spec is not syntax highlighted as nicely as something like @spec / @contract and empty newlines appear between the spec and multiline functions, which is almost always for me.

I am currently trying out some tests to extract the specs from the @spec/@type/@typep/@opaque attributes directly. It seems promising, but I’ll need to do some more tests before I’m sure that it (i.e. overriding @) does not have some weird edge cases.

[baldwindavid]

I went ahead and tried out your test branch applying it across my entire codebase switching all uses of spec/type to @spec/@type. My test suite continues to pass and I don’t seem to have hit any edge cases. I’m not sure how it could get much better than this just using @spec/@type everywhere. Would be neat if Elixir just worked like this out of the box (maybe with ability to disable in prod for zero cost).

[baldwindavid]

I’m thinking typespecs with runtime checks (“typespecs with teeth”) provide the opportunity to cleanup a bit of duplication in my function signatures. Suppose a function had the following signature:

@spec do_a_thing(User.t(), Project.t()) :: binary()
def do_a_thing(%User{} = user, %Project{} = project) do

I tend to pattern match on struct arguments for enforcement and explicitness. However, now that these typespecs can provide the same, is there value in continuing to include them?

I will continue to pattern match when necessary for multiple function heads with the same name, but beyond that seems a bit redundant. The following seems cleaner and easier to maintain:

@spec do_a_thing(User.t(), Project.t()) :: binary()
def do_a_thing(user, project) do

[Qqwy]

Thanks to some discussion with @baldwindavid we’ve settled on a new naming scheme:

Instead of using type, spec etc. the new scheme uses @type!, @spec! etc.

From the docs:

Using these forms has two advantages over using the direct calls:

1. Syntax highlighting will highlight the types correctly and the Elixir formatter will not mess with the way you write your type.
2. It is clear to people who have not heard of TypeCheck before that @type! and @spec! will work similarly to resp. @type and @spec .

Version 0.2.0 has been released with this change. It is for obvious reasons not backwards-compatible.

[baldwindavid]

Really loving this @spec!/@type! syntax. The established semantics of the bang hopefully make it unsurprising. This should also make it easy for developers to try this out one function at a time without feeling like they need to immediately buy in for an entire file or codebase.

On another note, I’m curious if your intent is that TypeCheck becomes a superset of type specs. It already supports a good bit of the @spec/@type syntax and it seems like you are hoping to support most of the rest of the builtins. It already even has some extensions like fixed-size lists with types and lazy types. One notable absence though is the when keyword for @spec!. The docs note that with the following:

Note that TypeCheck does not allow the when keyword to be used to restrict the types of recurring type variables (which Elixir’s builtin Typespecs allow). This is because:

• Usually it is more clear to give a recurring type an explicit name.
• The when keyword is used instead for TypeCheck’s type guards’. (See TypeCheck.Builtin.guarded_by/2 for more information.)

The first seems like helpful guidance and the second notes the very important addition of the type guard extension. Even so, I’m still thinking there are times when being able to use when on @spec! would be useful. Moreso, I just think it’s easier to understand/remember things when working with a superset than with something that adds some things, but removes others. Is the absence more about guidance or would it also be technically problematic to implement spec guards?

[Qqwy]

@baldwindavid The idea is indeed that TypeCheck accepts a superset of the built-in Elixir typespecs, which are themselves based on Erlang’s builtin typespecs. There are currently a couple of places where this is not yet 100% the case (some rarer literal type-syntaxes are not implemented in TypeCheck yet) but I do want to reach that.

However, when with a keyword list might be an exception here:

Currently, TypeCheck will raise a compile-time error when when with a keyword list is used, hinting people towards the correct usage (defining recurring types as a dedicated named type), making it impossible for people to have TypeCheck do the wrong thing by accident.
In the future, we might alter this behaviour and implement when to work both with ‘recurring type variables’ as well as for type guards. The reason this is not a priority however is because usage of when to re-use recurring type variables is rare; for instance there currently is an open issue on ExDoc because it also is not able to handle it (in all cases).

[baldwindavid]

My usage of “superset” might not even be the right term here. What I’m really getting at is supporting the same general constructs of regular typespecs rather than necessarily the exact same syntax. I just think it would be nice if @spec! allowed when and it worked the exact same way as for @type! in accepting arbitrary code. Contrived example:

@spec! in_magic_range?(the_number :: non_negative_integer()) :: boolean()
       when the_number < 42 && the_number > 21 && the_number != 22 && the_number != 33

I kind of liked how Norm allowed this for one-off specifications that aren’t used in multiple places even if it is not that common.

[Qqwy]

Ah! I see. Yes, currently it would be required to add the when to the typing of the_number

So this variant already works today (maybe some extra parentheses are required; I did not test this but go from memory here):

@spec! in_magic_range?(the_number :: non_negative_integer() when the_number < 42 && the_number > 21 && the_number != 22 && the_number != 33) :: boolean()

We could enhance it to allow the syntax you propose as well. Feel free to open a PR if you want; it’s unlikely that I’ll get to it myself in the near future as there are other features that I would like to add which I’d give higher priority.

[baldwindavid]

Oh, interesting. I’ve tried a few different forms of parens and can’t quite get that to work. That’s okay though. I was more curious if you were dead set against when for @spec! and it sounds like you are not against it and that it potentially already works in some form.

I do not have the requisite metaprogramming chops to make my proposed syntax happen (and it might not even be the best syntax), but is a part of the language I really want to explore.

Regardless, the library already handles all the things I need and I look forward to seeing it evolve as you work through your roadmap.

[Qqwy]

Version 0.2.1 has been released which adds:

• The possibility to override the StreamData property-testing generators that are made from the types by default.
• The possibility to add type-guards using when to a type as a whole as well as to a parameter in a typespec without problems.

[baldwindavid]

Well, José announced that the compiler will soon check for the following…

def do_a_thing(%User{} = user, %Project{} = project) do
  user.attribute_that_does_not_exist
end

Thus, it appears that it will still be beneficial to pattern match against struct arguments rather than solely relying on TypeCheck for this purpose like mentioned earlier in this thread. A bit of duplication here will probably be worth it.