Type system updates: moving from research into development

I don’t like this approach of casting data too early in the system, at least in my experience it drive to unnecessarily rigid systems.
I was expecting to be able to express something like:

$ %{"foo" => string, "bar" => integer} -> success()
$ map -> error()

So i could be strict on success and loose on failures. :thinking:

1 Like

We already have Dialyzer and it’s more lax, no need to duplicate that behaviour IMO.

Let’s have the new type system be more strict. It would also be a good step towards exhaustive pattern matching. Whoever does not want “annoyance” will simply not use the new system.

Though if it will replace the current type specs then I see your point.

10 Likes

Compiler switch / mix config key. Legacy mode and modern mode.

Non-exhaustive pattern matching is a fact of life in code bases that make money. I’ve been in several teams where the policy was “just match on these three patterns and if we get an error in our monitoring system then we’ll add more”. Can’t say how often that happens but Elixir being lax on this until an actual crash happens is something people definitely are making use of.

7 Likes

I like this.

$ :foo or :bar -> string
def foo_or_bar(:foo), do: ...
def foo_or_bar(:bar), do: ...

Elixir pattern matching would loose its charm, if we have to add types in the functions themselves.

It’s always fascinating to see new approaches. :star_struck:


It would be great if the type definitions are just like glorified guards, with constraints baked in.
And we are just supposed to pattern match on the types as well. :sunglasses:

$ integer, :foo | :bar, ~W(asc desc), 1..10 -> str | atom

def stringify(num, foo_bar, sort, selected_num), …
def stringify(_, _, _, _), do: :halt

€ _, _, _, _ -> atom

I wouldn’t mind sprinkling these in. :nerd_face:

4 Likes

I big :+1: for literals support. However I can agree that “just adding it” isn’t good. I would expect that in the documentation for types before first mention of literals support there should be explanation like in your post. :see_no_evil:

In case we go for literals support, I wonder if also supported would be overlapped domains (which is how dialyzer calls it). Currently multiple @spec are supported by ex_doc, but not by dialyzer. :thinking:

This is important to clearly say what kind of input would give what result, for example:

@spec sample(input :: atom) :: {:ok, output :: String.t()}
@spec sample(input :: any) :: {:error, :not_yet_supported}

Of course it’s not an example from prod, but imagine we use function generators and we want to do above i.e. explain what kind of input would work and what would fail:

for {input, output} <- data do
  $ unquote(input) -> unquote(output)
  def sample(unquote(input)), do: unquote(output)
end

$ any -> :error
def sample(_input), do: :error

# alternatively when output literals are not needed:

input_or = data |> Map.keys() |> Enum.reduce(&{:or, [], [&2, &1]})
$ unquote(input_or) -> {:ok, string}

for {input, output} <- data do
  def sample(unquote(input)), do: unquote(output)
end

$ any -> :error
def sample(_input), do: :error

Also let’s say we write code like:

@spec sample(any) :: :{ok, any}
$ any -> {:ok, any}
def sample(data), do: {:ok, data}

How would it look like in documentation generated by ex_doc? Would it prefer $ over @spec or maybe it would use both? :icon_question:

I know it’s a beginning, but I believe such questions may be helpful when making a decision about final form and answers or code examples may give us some general preview of the whole idea, so we can may think about edge cases from our experience. :brain:

For example here is one. How 0-arity functions would look like in $ notation?

# if we have such general form:
$ input -> output
# then 0-arity annotation would be:
$ -> output
# or maybe
$ () -> output

Looking forward seeing it’s final form. :eyes:

4 Likes

It is not about duplicating Dialyzer behaviour. My argument is precisely that the behaviour is already more strict, so adding more on that may (or may not) push too forward.

General rules such as “make it as strict as you can” can sound good on paper but not in practice. So it is important not to rush. We are being very deliberate in our decisions. :slight_smile:

100% agree. For those cases, I would say case! would be further beneficial, even without a type system, because you would communicate this intent (similar to File.read! and other bang functions).

14 Likes

Good luck! If you manage to get this right, you might pave a road for how programming languages of the future should look like, having a hybrid system without the downsides of strict types will be truly amazing.

2 Likes

I would phrase this as «and the compiler will assist you in refactoring» which is the only thing I am missing in Elixir and I hope the type system will give that to me :slightly_smiling_face:

6 Likes

That’s on you and the team to decide – still I’d be worried about adding extra syntax, IMO it will just confuse people and will introduce separate schools of thought and practices in real-world Elixir code. “We don’t use case! here” – I can already hear that during a starting phase of an interview.

It’s a super tricky situation admittedly. Naively I’d opt for compiler switches / mix config options that change the behavior of case. That way you have an off-ramp for the older way of doing things, and you can hard-deprecate (remove) it e.g. 2 years after the new typing system lands.

I know all of this is easier said than done, I am simply addressing needs that I’ve seen in teams (and in myself).

And I stand with @olivermt here; giving better hints during refactoring is what I also miss from Elixir. If it achieves that then it would be poised to eat part of the lunch of Golang, Rust and Typescript.

4 Likes

Just out of curiosity since I’m not 100% sure what exhaustive pattern matching looks like, would that mean that File.read would need to look like this not to error:

case! File.read("~/Desktop/passwords.txt") do
  {:ok, contents} -> contents
  {:error, :enoent} -> # ...
  {:error, :eacces} -> # ...
  {:error, :eisdir} -> # ...
  {:error, :enotdir} -> # ...
  {:error, :enomem} -> # ...
end

or could you still do a catch-all on the error reason part ({:error, reason}) since the :error part matches. IE, would would it just prevent this:

case! File.read("~Desktop/passwords.txt") do
  {:ok, contents} -> contents
  error -> error
end

…which is just a single-clause with anyway. Unless, of course, We Don’t Use Single Clause withs Here ™ :wink:

2 Likes

You could have one clause to match all errors, but the very idiomatic pattern that would not work anymore is this:

{:ok, contents} = File.read("...")

You would have to explicitly deal with the possible errors. That’s something that people used to statically typed languages will often encourage, but it’s clearly not how Elixir (and Erlang) are being used. If you can’t really deal with the error, then handling it explicitly and just propagating it is basically more work for no benefit.

On the other hand, it would be nice to have the compiler check that a function is implemented for all possibilities where it makes sense. Like if an area function expects a shape which could be any of {:square, number} | {:rectangle, number, number} | {:circle, number} and then we add another shape, our program will start crashing unless we implement area for the new shape as well. And the compiler could catch that.

I guess the difficult part is figuring out how to tell if exhaustivness is desired or not in the specific situation.

6 Likes

Ah, mostly gotcha, but just for clarity:

Does this mean that just adding a catch-all would be considered exhaustive? IE, would it have to be {:error, reason} -> reason or would a simple error -> error work, or even _ -> nil work?

1 Like

Even _ -> nil would be fine, exhaustive pattern matching means that a clause will always match and you will never get a runtime error there. Of course, it’s up to you to propagate the error up or manually raise. This would make it more tedious to handle errors and it would remove part of the magic of quickly developing in Elixir.

5 Likes

Now I gotcha, thanks!

I agree though this is why I am intrigued by the prospect of case! as it would allow you to opt-in.

2 Likes

I agree, I don’t really see another way to reconcile these two opposing points of view other than letting the programmer choose which one they want.

Although, maybe there could be an argument for making case always exhaustive (still, it would break existing code), while pattern matching in function heads and single line matching like our File example would allow non-exhaustive matches, since that is how people seem to use these slightly different mechanisms.

1 Like

I don’t think that’s the differenciator. Single line is just the “I want to handle 1 case” vs. a non exhaustive case is the “I want to handle n out of m cases”.

1 Like

Mostly yes but again, if you can only expect :ok / :error tuples then just matching {:ok, value} and {:error, reason} is still exhaustive pattern-matching even if you don’t handle the N possible error cases.

1 Like

That’s true. I’m just going with what I’ve realistically seen in the code that I’ve read and maybe this would be an acceptable trade off? Single line matching is super common, while case usually seems to be meant to be exhaustive. And if not, either handling the error explicitly or extracting the case into a helper function with multiple heads could be good enough.

2 Likes

Curious if you are borrowing any ideas from Gleam, it’s kinda neat syntax-wise and not cluttered and easy to read.

1 Like

It honestly never occurred to me to make case non-exhaustive until the other day (I just didn’t have the vocabulary to describe it), hence my curiosity here. Is a bare {:ok, contents} = File.read("...") really that common, though? That’s what File.read! is for. If those types of things could be prevented I’d be pretty into that, though not at the expense of now having to type everything to get it. I have very little experience with statically-typed languages (mostly OCaml) so I don’t really know what I’m talking about. I share a bit of @D4no0’s concerns but also share their optimism. It’s clear that types are what (most of?) the people want and intense diligence is being exercised here, so I’m mostly optimistic!

3 Likes