A case for inline type annotations

Thank you @smueller for providing this example. However, I must point out that you are not fully showcasing what you proposed, given several functions had their clauses elided, such as new, new!, parse, remove_dot_segments, merge, hex_to_dec, and unpercent.

I’d love if you could actually annotate the existing clauses in the URI module, at least for these functions, showing all clauses alongside patterns and guards (you can elide the parts inside the do-block with … for conciseness). It is important to preserve the clauses for inline annotations because one of the arguments against them is that they get repetitive. Otherwise, you are effectively keeping them separate from the existing code, which I must agree looks really good. :smiley:


Besides the above, here is a question you should answer: do you agree with the previous comment by Wojtek and myself, which said that the following clause is confusing (and I would say semantically invalid), because it says it accepts URI, String and nil, but in practice that particular clause only deals with String?

def hostname(url :: URI | String?) when is_binary(url) :: String?, do: hostname(URI.parse(url))
                    ^^^^^       ^  ^^^^^^^^^^^^^^^^^^^

Or do you believe such definitions should be accepted? I am asking semantically, disregard the syntax for now.


PS: type aliases are part of the type system. I am not convinced about String? though. I think it is worth being more explicit around nil usage, but this is something I can change my mind once we start using the type system in practice and it feels too verbose. Feel free to use it in your example, it is not relevant to the discussion at hand.

As promised, here is the version with the proposed syntax, keeping all of the existing clauses:

defmodule URI do
  @moduledoc """
  Utilities for working with URIs.
  """

  # Type aliases for better readability
  $ type scheme() = string() or nil
  $ type host() = string() or nil
  $ type port() = non_neg_integer() or nil
  $ type path() = string() or nil
  $ type query() = string() or nil
  $ type fragment() = string() or nil
  $ type userinfo() = string() or nil
  $ type uri_string() = string()
  $ type encoding_type() = :www_form or :rfc3986
  $ type query_map() = %{string() => string()}
  $ type query_enum() = enumerable.t()
  $ type query_pair() = {string(), string()}
  $ type predicate() = (byte() -> boolean())
  $ type authority() = string() or nil

  @derive {Inspect, optional: [:authority]}
  defstruct [:scheme, :authority, :userinfo, :host, :port, :path, :query, :fragment]

  $ string() -> port()
  def default_port(scheme) when is_binary(scheme), do: ...

  $ string(), port() and not nil -> :ok
  def default_port(scheme, port) when is_binary(scheme) and is_integer(port) and port >= 0, do: ...

  $ query_enum(), encoding_type() -> string()
  def encode_query(enumerable, encoding \\ :www_form), do: ...

  $ query_pair(), encoding_type() -> string()
  defp encode_kv_pair({key, _}, _encoding) when is_list(key), do: ...
  defp encode_kv_pair({_, value}, _encoding) when is_list(value), do: ...
  defp encode_kv_pair({key, value}, :rfc3986), do: ...
  defp encode_kv_pair({key, value}, :www_form), do: ...

  $ string(), map(), encoding_type() -> %{}
  def decode_query(query, map \\ %{}, encoding \\ :www_form)
  def decode_query(query, %_{} = dict, encoding) when is_binary(query), do: ...
  def decode_query(query, map, encoding) when is_binary(query) and is_map(map), do: ...
  def decode_query(query, dict, encoding) when is_binary(query), do: ...

  $ string(), query_map(), encoding_type() -> query_map()
  defp decode_query_into_map(query, map, encoding), do: ...

  $ string(), any(), encoding_type() -> any()
  defp decode_query_into_dict(query, dict, encoding), do: ...

  $ string(), encoding_type() -> enumerable.t()
  def query_decoder(query, encoding \\ :www_form) when is_binary(query), do: ...

  $ string(), encoding_type() -> {query_pair(), string()} or nil
  defp decode_next_query_pair("", _encoding), do: ...
  defp decode_next_query_pair(query, encoding), do: ...

  $ string(), encoding_type() -> string()
  defp decode_with_encoding(string, :www_form), do: ...
  defp decode_with_encoding(string, :rfc3986), do: ...

  $ byte() -> boolean()
  def char_reserved?(character), do: ...

  $ byte() -> boolean()
  def char_unreserved?(character), do: ...

  $ byte() -> boolean()
  def char_unescaped?(character), do: ...

  $ string(), predicate() -> string()
  def encode(string, predicate \\ &char_unescaped?/1)
      when is_binary(string) and is_function(predicate, 1), do: ...

  $ string() -> string()
  def encode_www_form(string) when is_binary(string), do: ...

  $ byte(), predicate() -> string()
  defp percent(char, predicate), do: ...

  $ byte() -> byte()
  defp hex(n) when n <= 9, do: ...
  defp hex(n), do: ...

  $ string() -> string()
  def decode(uri), do: ...

  $ string() -> string()
  def decode_www_form(string) when is_binary(string), do: ...

  $ string(), string(), boolean() -> string()
  defp unpercent(<<?+, tail::binary>>, acc, true), do: ...
  defp unpercent(<<?%, tail::binary>>, acc, spaces), do: ...
  defp unpercent(<<head, tail::binary>>, acc, spaces), do: ...
  defp unpercent(<<>>, acc, _spaces), do: ...

  $ byte() -> byte() or nil
  defp hex_to_dec(n) when n in ?A..?F, do: ...
  defp hex_to_dec(n) when n in ?a..?f, do: ...
  defp hex_to_dec(n) when n in ?0..?9, do: ...
  defp hex_to_dec(_n), do: ...

  $ t() or uri_string() -> {:ok, t()} or {:error, string()}
  def new(%URI{} = uri), do: ...
  def new(binary) when is_binary(binary), do: ...

  $ t() or uri_string() -> t()
  def new!(%URI{} = uri), do: ...
  def new!(binary) when is_binary(binary), do: ...

  $ map() -> t()
  defp uri_from_map(%{path: ""} = map), do: ...
  defp uri_from_map(map), do: ...

  $ t() or string() -> t()
  def parse(%URI{} = uri), do: ...
  def parse(string) when is_binary(string), do: ...

  $ string() -> query() or nil
  defp nilify_query("?" <> query), do: ...
  defp nilify_query(_other), do: ...

  $ string() -> {authority(), userinfo(), host(), port()}
  defp split_authority(""), do: ...
  defp split_authority("//"), do: ...
  defp split_authority("//" <> authority), do: ...

  $ string() -> string() or nil
  defp nilify(""), do: ...
  defp nilify(other), do: ...

  $ t() -> string()
  def to_string(uri), do: ...

  $ t() or string(), t() or string() -> t()
  def merge(uri, rel)
  def merge(%URI{scheme: nil}, _rel), do: ...
  def merge(_base, %URI{scheme: rel_scheme} = rel) when rel_scheme != nil, do: ...
  def merge(%URI{} = base, %URI{host: host} = rel) when host != nil, do: ...
  def merge(%URI{} = base, %URI{path: nil} = rel), do: ...
  def merge(%URI{host: nil, path: nil} = base, %URI{} = rel), do: ...
  def merge(%URI{} = base, %URI{} = rel), do: ...
  def merge(base, rel), do: ...

  $ path(), path() -> path()
  defp merge_paths(nil, rel_path), do: ...
  defp merge_paths(_, "/" <> _ = rel_path), do: ...
  defp merge_paths(base_path, rel_path), do: ...

  $ path() -> path()
  defp remove_dot_segments_from_path(nil), do: ...
  defp remove_dot_segments_from_path(path), do: ...

  $ string() -> [string() or atom()]
  defp path_to_segments(path), do: ...

  $ [string() or atom()], [string() or atom()] -> [string() or atom()]
  defp remove_dot_segments([], acc), do: ...
  defp remove_dot_segments([:/ | tail], acc), do: ...
  defp remove_dot_segments([_, :+ | tail], acc), do: ...
  defp remove_dot_segments(["."], acc), do: ...
  defp remove_dot_segments(["." | tail], acc), do: ...
  defp remove_dot_segments([".." | tail], [:/]), do: ...
  defp remove_dot_segments([".."], [_ | acc]), do: ...
  defp remove_dot_segments([".." | tail], [_ | acc]), do: ...
  defp remove_dot_segments([head | tail], acc), do: ...

  $ [atom() or string()] -> string()
  defp join_reversed_segments([:/]), do: ...
  defp join_reversed_segments(segments), do: ...

  $ t(), string() -> t()
  def append_query(%URI{} = uri, query) when is_binary(query) and uri.query in [nil, ""], do: ...
  def append_query(%URI{} = uri, query) when is_binary(query), do: ...

  $ t(), string() -> t()
  def append_path(%URI{}, "//" <> _ = path), do: ...
  def append_path(%URI{path: path} = uri, "/" <> rest = all), do: ...
  def append_path(%URI{}, path) when is_binary(path), do: ...
end

I suggest that you also preserve and annotate all of the existing clauses, so we can effectively compare how existing code will be typed.

8 Likes

To me it both reads and works like URI | (String | nil). But it is the same with or without parens.

It seems like inline type annotations–even if better–would be a more intrusive change to the language. You’re updating the core syntax of Elixir rather than introducing an optional header to each function.

I’m personally against making such a deep change to the language when the type system is still experimental and early stages.

And there are real downsides. For example, adding type information adds quite noise to the function signature (and seems like a few others feel in this thread the same way!). If you want “inline” typing, why not use pattern matching instead and let the compiler infer types from those patterns?

In my daily coding, I’m not constantly asking myself “what is the type of this variable”? It’s a secondary question when I’m reading code. I’d rather hover over a variable to understand its type than get overwhelmed with a function signature that is 2x the size.

Sure, I acknowledge there’s a downside to integer() -> integer() syntax. I have to do an Enum.zip in my head to match the corresponding variables to the corresponding types. But can’t this be solved at the IDE level?

For example, in Typescript, I hover over a variable and it tells me the type.

Similarly, in Elixir we could show type hints on hover. It doesn’t have to be solved at the language level.

4 Likes

Just curious, are all the parentheses needed for the $ types? I think it’d be far more readable without them. Generally though this approach seems more readable than the inline types which imo work well for simple things but can become a monstrosity.

Agree, let the compiler do the work. Taken to an extreme, it seems like specifying types may only be useful for external data at the boundary. If the compiler can make that a reality, then the out-of-band $ types make even more sense as they’d be used selectively instead of throughout your codebase (unlike the other languages).

4 Likes

Maybe a separate topic, but I don’t think nil should be declared as part of a type alias (like ever). Instead it should find its place in the type spec above function headers (on case by case basis), because enabling something to be nil does not mean it can be of any type but instead permits a particular argument or return value to be undefined.

One instance that particularly bothers me in this regard can be found in the LiveView docs for Phoenix.Component.attr/3 that reads:

Note only `:any` and `:atom` expect the value to be set to `nil`.`

According to the above excerpt from the docs, for each attribute that I know for sure is of a particular type or module, but can also be undefined, I should define it as :any. IMO, this is plain wrong and I am violating this on purpose in pretty much every single function component.

Me neither. In fact I rarely read @specs except for the odd hairy function. In fact I used to highlight them as comments because for the most part I see them as pure noise (this broke and I haven’t bothered to fix it). And yet, with clear variable names, pattern matching and guards, as well as low-bar good coding practices like not making “rabbit hole” function calls, I can always easily identify the types.

All that coupled with Elixir (likely/hopefully) getting type inference, there is absolutely no need for inline types. We can use our modern editors to give us clarity in the times we’re confused. As @jam says, let the compiler do the work! I also don’t appreciate this rhetoric that inline types are “clearly superior” as it doesn’t lend to good faith debates. In my mind, having them them on their own line is clearly superior. If you’re having trouble reconciling them then you’re stuffing way too much in your function head.

As far as “familiarity in the name of adoption” goes, I’m also not convinced of this. Elixir is already different in many weird and wonderful ways. For example, it makes the “obviously superior” decision to have separate syntax for blocks vs data structures (oh the horrors that poor overloaded curly brace is forced to endure in all those “modern” languages!) This is to say that anyone moving over to Elixir is already going to have to have an open mind or they’re going to have a very bad time (as we see on this forum from time to time). Adding inline types almost feels like a step towards the normalization of proposals to “make Elixir less like Elixir.” If you want a “familiar” syntax on the BEAM, there is the fabulous Gleam project! Though you’d be missing out on Elixir’s exceptional macro system meaning you’ll never have awesome projects like Ash. I should say, though, that I have no idea what José’s adoption goals are, these are just my feelings.

That said, in the name of community and open ideas, I love that this is being entertained. If it is found to be possible and this is what the community wants, then so be it. But also, please no :frowning:


If we’re all going to be vibe coders by next year, does any of this even matter? :grin: :grimacing: :upside_down_face:

2 Likes

Doesn’t that imply different semantics from the Kotlin example? It is not obvious from the type signature that the return type is constrained by the argument types.

I’m not sure the type system will go into clauses like dialyzer would do. The return type is binary or nil in all cases.

Here’s a more comprehensive version with inline types that keeps all existing clauses:

defmodule URI do
  @moduledoc """
  Utilities for working with URIs.
  """

  # Type aliases only when they add real clarity
  typealias EncodingType = :www_form | :rfc3986
  typealias QueryMap = %{String => String}
  typealias QueryPair = {String, String}
  typealias Predicate = (byte) -> boolean
  typealias Segments = List of (String | atom)

  @derive {Inspect, optional: [:authority]}
  defstruct [
    scheme: String?,
    authority: String?,
    userinfo: String?,
    host: String?,
    port: non_neg_integer?,
    path: String?,
    query: String?,
    fragment: String?
  ]

  # All function clauses with clean inline types

  def default_port(scheme: String) -> non_neg_integer? when is_binary(scheme)

  def default_port(scheme: String, port: non_neg_integer) -> :ok 
    when is_binary(scheme) and is_integer(port) and port >= 0

  def encode_query(enumerable: Enumerable of QueryPair, encoding: EncodingType \\ :www_form) -> String

  defp encode_kv_pair({key, _}: QueryPair, _encoding: EncodingType) -> no_return when is_list(key)
  defp encode_kv_pair({_, value}: QueryPair, _encoding: EncodingType) -> no_return when is_list(value)
  defp encode_kv_pair({key, value}: QueryPair, :rfc3986) -> String
  defp encode_kv_pair({key, value}: QueryPair, :www_form) -> String

  def decode_query(query: String, map: QueryMap \\ %{}, encoding: EncodingType \\ :www_form) -> QueryMap
  def decode_query(query: String, %_{} = dict: any, encoding: EncodingType) -> any when is_binary(query)
  def decode_query(query: String, map: QueryMap, encoding: EncodingType) -> QueryMap 
    when is_binary(query) and is_map(map)
  def decode_query(query: String, dict: any, encoding: EncodingType) -> any when is_binary(query)

  defp decode_query_into_map(query: String, map: QueryMap, encoding: EncodingType) -> QueryMap

  defp decode_query_into_dict(query: String, dict: any, encoding: EncodingType) -> any

  def query_decoder(query: String, encoding: EncodingType \\ :www_form) -> Enumerable of QueryPair 
    when is_binary(query)

  defp decode_next_query_pair("", _encoding: EncodingType) -> nil
  defp decode_next_query_pair(query: String, encoding: EncodingType) -> {QueryPair, String}?

  defp decode_with_encoding(string: String, :www_form) -> String
  defp decode_with_encoding(string: String, :rfc3986) -> String

  def char_reserved?(character: byte) -> boolean

  def char_unreserved?(character: byte) -> boolean

  def char_unescaped?(character: byte) -> boolean

  def encode(string: String, predicate: Predicate \\ &char_unescaped?/1) -> String
    when is_binary(string) and is_function(predicate, 1)

  def encode_www_form(string: String) -> String when is_binary(string)

  defp percent(char: byte, predicate: Predicate) -> String

  defp hex(n: byte) -> byte when n <= 9
  defp hex(n: byte) -> byte

  def decode(uri: String) -> String

  def decode_www_form(string: String) -> String when is_binary(string)

  defp unpercent(<<?+, tail: binary>, acc: String, true) -> String
  defp unpercent(<<?%, tail: binary>, acc: String, spaces: boolean) -> String
  defp unpercent(<<head, tail: binary>, acc: String, spaces: boolean) -> String
  defp unpercent(<<>>, acc: String, _spaces: boolean) -> String

  defp hex_to_dec(n: byte) -> byte? when n in ?A..?F
  defp hex_to_dec(n: byte) -> byte? when n in ?a..?f
  defp hex_to_dec(n: byte) -> byte? when n in ?0..?9
  defp hex_to_dec(_n: byte) -> nil

  def new(%URI{} = uri) -> {:ok, t}
  def new(binary: String) -> {:ok, t} | {:error, String} when is_binary(binary)

  def new!(%URI{} = uri) -> t
  def new!(binary: String) -> t when is_binary(binary)

  defp uri_from_map(%{path: ""} = map) -> t
  defp uri_from_map(map) -> t

  def parse(%URI{} = uri) -> t
  def parse(string: String) -> t when is_binary(string)

  defp nilify_query("?" <> query: String) -> String
  defp nilify_query(_other: any) -> nil

  defp split_authority("") -> {nil, nil, nil, nil}
  defp split_authority("//") -> {String?, nil, String, nil}
  defp split_authority("//" <> authority: String) -> {String?, String?, String?, non_neg_integer?}

  defp nilify("") -> nil
  defp nilify(other: any) -> any

  def to_string(uri: t) -> String

  def merge(base_uri: t | String, relative_uri: t | String) -> t
  def merge(%URI{scheme: nil} = uri, _rel: t | String) -> no_return
  def merge(base: t | String, %URI{scheme: rel_scheme} = rel) -> t when rel_scheme != nil
  def merge(%URI{} = base, %URI{host: host} = rel) -> t when host != nil
  def merge(%URI{} = base, %URI{path: nil} = rel) -> t
  def merge(%URI{host: nil, path: nil} = base, %URI{} = rel) -> t
  def merge(%URI{} = base, %URI{} = rel) -> t
  def merge(base: String, rel: String) -> t

  defp merge_paths(base_path: nil, rel_path: String?) -> String?
  defp merge_paths(base_path: String?, "/" <> _ = rel_path: String?) -> String?
  defp merge_paths(base_path: String?, rel_path: String?) -> String?

  defp remove_dot_segments_from_path(path: nil) -> nil
  defp remove_dot_segments_from_path(path: String) -> String

  defp path_to_segments(path: String) -> Segments

  defp remove_dot_segments([]: Segments, acc: Segments) -> Segments
  defp remove_dot_segments([:/ | tail]: Segments, acc: Segments) -> Segments
  defp remove_dot_segments([_, :+ | tail]: Segments, acc: Segments) -> Segments
  defp remove_dot_segments(["."]: Segments, acc: Segments) -> Segments
  defp remove_dot_segments(["." | tail]: Segments, acc: Segments) -> Segments
  defp remove_dot_segments([".." | tail]: Segments, [:/]: Segments) -> Segments
  defp remove_dot_segments([".."]: Segments, [_ | acc]: Segments) -> Segments
  defp remove_dot_segments([".." | tail]: Segments, [_ | acc]: Segments) -> Segments
  defp remove_dot_segments([head | tail]: Segments, acc: Segments) -> Segments

  defp join_reversed_segments(segments: [:/]) -> String
  defp join_reversed_segments(segments: Segments) -> String

  def append_query(%URI{} = uri, query: String) -> t 
    when is_binary(query) and uri.query in [nil, ""]
  def append_query(%URI{} = uri, query: String) -> t when is_binary(query)

  def append_path(uri: t, "//" <> _ = path) -> no_return
  def append_path(%URI{path: path} = uri, "/" <> rest = all: String) -> t
  def append_path(uri: t, path: String) -> no_return when is_binary(path)
end

defimpl String.Chars, for: URI do
  def to_string(uri: URI.t()) -> String

  defp extract_authority(%URI{host: nil, authority: authority}) -> String?
  defp extract_authority(%URI{host: host, userinfo: userinfo, port: port}) -> iodata
end

Leveraged Improvements:

Inline Type Syntax (can be changed to :: if necessary)

# Parameters use single colon (:) with no spaces
def encode_query(enumerable: Enumerable of QueryPair, encoding: EncodingType) -> String

# Return types use arrow (->)
def default_port(scheme: String) -> non_neg_integer?

Optional Types with ?

# Instead of: String.t() | nil
def default_port(scheme: String) -> non_neg_integer?

# In struct definitions
defstruct [
  scheme: String?,
  host: String?,
  port: non_neg_integer?
]

Collection Type Constraints with of

# Clear collection contents without full generics
def query_decoder(query: String) -> Enumerable of QueryPair
typealias Segments: List of (String | atom)

Type Inference from Patterns

# Struct patterns - type obvious from %URI{} match
def merge(%URI{} = base, %URI{} = rel) -> t
def new(%URI{} = uri) -> {:ok, t}

# Literal patterns - type obvious from literal value  
defp split_authority("") -> {nil, nil, nil, nil}
defp split_authority("//") -> {String?, nil, String, nil}
defp hex_to_dec(_n: byte) -> nil

Improved Struct Definitions

# DRY: types defined once in defstruct, not separately in @type
defstruct [
  scheme: String?,
  authority: String?,
  userinfo: String?,
  host: String?,
  port: non_neg_integer?,
  path: String?,
  query: String?,
  fragment: String?
]
# @type t automatically generated

Consistent Typealias Syntax

typealias EncodingType = :www_form | :rfc3986     # Union type
typealias QueryPair =  {String, String}             # Semantic meaning, contained complexity

Simplified Type Names

# Sugar makes using Types more concise
String?          # not String.t() | nil
Enumerable      # not Enumerable.t()

Collectively, these constructs achieve:

  • Locality: Types next to parameters they describe
  • Familiarity: Syntax similar to Swift, TypeScript, Rust, Kotlin
  • Conciseness: ? sugar, pattern inference, of constraints
  • Consistency: Single : for all type annotations, -> for returns
  • Maintainability: DRY struct definitions, smart typealias usage
  • Readability: Clear without being verbose

In comparison to $

  • No mental mapping of positional types to parameters
  • Self-documenting function signatures
  • Easier refactoring - types move with parameters
  • Better IDE support potential for inline type information
  • Pattern-aware - leverages Elixir’s existing pattern matching strengths
1 Like

At the risk of sounding sarcastic, if we are genuinely considering the “function juggling” argument then we’re likely going to want to also consider the following scenarios:

@doc [foo: 2], """
Do a foo.
"""
def foo(a, b), do: a + b
@tag "writes to the given file", :tmp_dir
test "writes to the given file", %{tmp_dir: tmp_dir} do
  # ...
end
attr :avatar, :user, User, required: true
slot :avatar, :inner_block

def avatar(assigns) do
  ~H"""
  <%!-- -->
  """
end

I actually do think that:

def foo do
  @doc "I'm a foo that does foo things!"
  "foo"
end

would be sorta be nice as it is consistent with how @moduledoc works. I’m sure there was a good reason it wasn’t done this way but it is something I think about sometimes when I write a @doc. I’m pretty happy with status quo, though.

Also, if you use vanilla Vim you can use my mixer.vim plugin which has a text object that includes all function heads with all and their annotations (including attr and slot)—refactoring woes solved! :stuck_out_tongue: Figured I’d may as well include some self-promo at this point :grinning_cat:

@smueller thank you. Btw, please don’t use type inference in the examples, as the goal is to validate the type syntax. There are several cases where it can be omitted in signatures - not all - but when exploring the syntax it should be done fully.

Even languages with complete inference, many teams prefer to explicitly type all signatures. If the syntax requires inference to be palatable, then that’s not a good sign.

EDIT: also note you cannot use the foo: Type as that is already available today for pattern matching in keyword lists. Such change would be ambiguous and backwards incompatible.

After sleeping on it, here is my criticism of inline annotations.

Too much repetition

For example, take encode_kv_pair. The different clauses are effectively variations of the same type, which are repeated over and over again:

  defp encode_kv_pair({key, _}: QueryPair, _encoding: EncodingType) -> no_return when is_list(key)
  defp encode_kv_pair({_, value}: QueryPair, _encoding: EncodingType) -> no_return when is_list(value)
  defp encode_kv_pair({key, value}: QueryPair, :rfc3986) -> String
  defp encode_kv_pair({key, value}: QueryPair, :www_form) -> String

In the example above, there was even a need to rely on inference to reduce some of the repetition. With a separate clause, the type signature is defined once:

  $ query_pair(), encoding_type() -> string()
  defp encode_kv_pair({key, _}, _encoding) when is_list(key)
  defp encode_kv_pair({_, value}, _encoding) when is_list(value)
  defp encode_kv_pair({key, value}, :rfc3986)
  defp encode_kv_pair({key, value}, :www_form)

While I definitely prefer the second one, and some of it can be described as taste, there is no discussion the second one is more concise and less repetitive. You can see this happening over and over when comparing snippets, in almost every function that has more than one clause.

It obscures the actual signature

One of the benefits of type signatures is to provide a brief description of what the function does. However, if you only have inline type annotations, this can become very hard. For example, let’s look at your merge function and have it with the actual code:

  def merge(%URI{scheme: nil} = uri, _rel: t | String) -> no_return do
    raise ArgumentError, "you must merge onto an absolute URI"
  end

  def merge(base: t | String, %URI{scheme: rel_scheme} = rel) -> t when rel_scheme != nil do
    %{rel | path: remove_dot_segments_from_path(rel.path)}
  end

  def merge(%URI{} = base, %URI{host: host} = rel) -> t when host != nil  do
    %{rel | scheme: base.scheme, path: remove_dot_segments_from_path(rel.path)}
  end

  def merge(%URI{} = base, %URI{path: nil} = rel) -> t  do
    %{base | query: rel.query || base.query, fragment: rel.fragment}
  end

  def merge(%URI{host: nil, path: nil} = base, %URI{} = rel) -> t do
    %{
      base
      | path: remove_dot_segments_from_path(rel.path),
        query: rel.query,
        fragment: rel.fragment
    }
  end

  def merge(%URI{} = base, %URI{} = rel) -> t do
    new_path = merge_paths(base.path, rel.path)
    %{base | path: new_path, query: rel.query, fragment: rel.fragment}
  end
  
  def merge(base: String, rel: String) -> t do
    merge(parse(base), parse(rel))
  end

It is really hard to tell the arguments it receives and the expected return types. You need to go clause by clause, build which one overlaps and which ones do not in our head. Maybe each of them handle a different type, maybe not.

However, if you have a separate type declaration at the top, regardless of the syntax, then it is immediately clear:

  def merge(base_uri: t | String, relative_uri: t | String) -> t
  
  # or
  
  $ t | String, t | String -> t
  def merge(uri_or_string, uri_or_string)

Incorrect type annotations

Some of your examples have clearly invalid type annotations. Let’s keep annotation and code together once more:

  defp merge_paths(base_path: nil, rel_path: String?) -> String? do
    merge_paths("/", rel_path)
  end

  defp merge_paths(base_path: String?, "/" <> _ = rel_path: String?) -> String? do
    remove_dot_segments_from_path(rel_path)
  end

  defp merge_paths(base_path: String?, rel_path: String?) -> String? do
    (path_to_segments(base_path) ++ [:+] ++ path_to_segments(rel_path))
    |> remove_dot_segments([])
    |> join_reversed_segments()
  end

The second clause says "/" <> _ = rel_path: String? but it clearly cannot handle nils. The last clause says it deals with nils, but it does not. Here is another one:

  # First argument doesn't deal with all Segments, only with part of them (empty lists)
  defp remove_dot_segments([]: Segments, acc: Segments) -> Segments

We see a similar mistakes in the return types to split_authority and hex_to_dec. Furthermore, because you were relying on inference, many of these mistakes have been hidden. If you fully type the arguments, you will see this popping up more and more.

Of course, the type system would find these bugs in practice, but the fact those issues are popping so frequently shows there are fundamental semantic inconsistencies with inline type annotations, which we will explore next.

Type aliases awkwardness

Inference is a great feature to have and most of our work so far has been on inference. However, many teams on statically typed languages prefer to rely on inference as little as possible. Especially because inference may hide bugs in certian cases. For example, you chose to rely on inference for decode_with_encoding, to avoid repetition:

  defp decode_with_encoding(string: String, :www_form) -> String
  defp decode_with_encoding(string: String, :rfc3986) -> String

However, this clause has one issue: if you change EncodingType to have a new entry, you won’t have a typing violation in this function, because nowhere you defined it is supposed to handle all EncodingTypes. While in this case you will likely get a warning anyway, because it is all defined in the same module, you won’t be able to rely on inference when implementing clauses for a type alias defined in another module (as I showed in log_payment_status earlier).

Of course, the answer would be to annotate those types:

  defp decode_with_encoding(string: String, :www_form: EncodingType) -> String
  defp decode_with_encoding(string: String, :rfc3986: EncodingType) -> String

However, per the previous section, the definition above is invalid. A type alias means, by definition, that if you have typealias Alias = Type1 or Type2, you can replace the Alias by Type1 or Type2. That’s how they behave in all typed programming languages. This means you literally wrote this signature:

  defp decode_with_encoding(string: String, :www_form: :www_form | :rfc3986) -> String
  defp decode_with_encoding(string: String, :rfc3986: :www_form | :rfc3986) -> String

which, per above, is clearly wrong.

This puts us in a pickle. If type inference won’t help us catch type alias changing and we cannot use the type alias, there is literally nowhere we can annotate that this function is meant to handle all EncodingType, unless we define the type annotation separately. This is what Wojtek and I referred to earlier in this thread in the hostname function. The issue was not the syntax, the issue is that inline annotations are semantically at odds with pattern matching, which is accentuated by type aliases.

Summary

There are a couple other issues with your inline type annotations, such as the syntax being fundamentally incompatible (anyone who disagrees is welcome to change the parser and prove me wrong), but hopefully the above is enough to show they have enough syntactical and semantic issues when typing existing code. And that’s within a single module! The Elixir repository has 447 modules and over 5700 public functions, while the URI module has 29 of them. So these issued popped up when typing 0.5% of a single codebase.

But how can we be certain that having type annotations apart is better? Well, that’s how we have been adding annotations for the last 10+ years via typespecs, and new type system is meant to improve on the flaws of the existing typespecs. I acknowledge there are a separate discussion to have about syntax, in this thread someone already asked about parens being required or optional, or even the $ of itself, but it is clear having type annotations separate from clauses is the superior choice.

Finally, I have to say it is a bit frustrating that at no moment none of the cons above have been mentioned, posing inline type annotations as having only benefits and no trade-offs. In particular, I disagree with almost all items from your summary:

  • No mental mapping of positional types to parameters

I actually agree with this one, having to positionally map types to arguments is one of the downsides of having them separate.

  • Self-documenting function signatures

As shown above, that’s clearly false. Whenever a function has more than one clause, I need to parse through every single annotation to figure out the types it accepts and returns. The combination of inline annotations with type inference in your latest snippet only makes this harder and defeats any self-documenting purpose.

  • Easier refactoring - types move with parameters

I disagree. While inline annotations makes it easier to move code to a new place, it comes with the huge downside that you can break code when you move it around, because you change the specification and implementation at the same time. Given the purpose of types is to help us find bugs, I’d rather err on making sure bugs do not go undetected.

  • Better IDE support potential for inline type information

I disagree. I see no reason why IDEs would struggle with any of the approaches.

  • Pattern-aware - leverages Elixir’s existing pattern matching strengths

Strongly disagree. Mixing inline annotations with pattern matching leads to excessive typing violations, as explained at length above, especially when aliases are used. And when relying on type inference to avoid repetition, as done above, allows bugs to creep in.

Still, I appreciate the time for this discussion because I know others would have the same questions. Now we can hopefully put this particular topic past us and focus on approaches more suitable to the language. Thank you.

26 Likes

I think this is the best argument in favor of inlining the types (I don’t find the others convincing). In particular, when updating a function signature it can be easy to accidentally forget to update the corresponding type signature. This happens to me with reasonable frequency.

However, this is something which can be solved with tooling. In practice the compiler catches almost all of these immediately (if the arity doesn’t match) and I see an error via ElixirLS. For more complex (typing) mismatches Dialyzer will generally catch them, and again the LSP will inline a warning.

Given that the main benefit of the inline syntax would be to catch an arity mismatch, and given that the existing tooling already catches those 100% of the time before the code is even run, I don’t think it’s a big deal. The new type system (and new LSP) will only improve this further.

2 Likes

It’s pretty surprising how readable this version is.

I apologize if I intrude to give the type-theorist viewpoint. José knows I am quite fond of having inline type annotations (if used with parsimony), but not at the expense of whole type annotations. Giving the whole type annotation is far more expressive than just typing function parameters (besides all the advantages already evoked in this thread). The simplest example I can think of is the or function or as it is defined in JavaScript: it takes two arguments and returns the first one if it is truthy, otherwise it returns the second one. In Elixir, we can write this as follows (assuming that falsy values are 0, “”, 0.0, and :false):

def or(x,y) do
  if x != 0 and x!= "" and x != 0.0 and x != false do
    x
  else
    y
  end
end

You can define a type annotation that precisely describes the function:

$ type Falsy = 0 or "" or 0.0 or :false
$ type Truthy = not Falsy

$ ((Falsy, a) -> a) and ((Truthy and b, term()) -> (Truthy and b)) 
  when a: term(), b: term()

where a and b are type variables.

I do not pretend that the type above is readable, but it exactly states what I wrote in English: “the function returns its second argument if the first one is falsy, otherwise it returns the first argument”. While this is not implemented in Elixir (yet?), we have running prototypes that can reconstruct this type for the unannotated code [see POPL25 conference]. There is no way to obtain such precision (crucial to precisely track types in branching) by typing the function parameters (unless, of course, you use two distinct function clauses, which is not the point).
The only type we can give to the parameters x and y is term() therefore deducing for or the type term(), term() -> term(), the one of all binary functions.

To summarize, whole type annotations are necessary to fully exploit the expressiveness of set-theoretic types.

12 Likes