From my experience as a solo dev, the most important tests are black-box integration-level tests that exercise the public contract of the app (UI if it’s a web app, the APIs if it’s a backend service). These tests are implementation-agnostic, so whether I’m using types or not has zero effect on them.

My app is a server-side rendered Phoenix app. I’m testing it by making multiple HTTP requests and asserting on the content of the responses. I have 173 of those tests plus 9 doctests and my suite takes ~6s to run. I threw away all other tests (no unit tests for schema or contexts). This gives me immense freedom in refactoring and rewriting the code.

My main point is that if your tests can be replaced by types, I would argue they are most likely tests not worth writing anyway. For example, I rarely see the purpose in checking for FunctionClausError (and similar).

On the flip side, believing types replaces tests (and docs), will lead you to lacking tests (and lacking docs).

I don’t love complex type systems. My preference is for the types to fit the testing trophy, which to me means that the purpose of the types is to give the quickest feedback possible (right in the IDE) that something is off, before the code is even run. From that perspective, types supplement the developer’s experience.

My view is that testing and strong typing both help with lowering the cost of change, but they do so in different ways, and are therefore not a substitute of each other.

Strong typing pins down the basic interface of each callable function, defining what is expected as input and output. If a change breaks such expectation, the type system tells us. Additionally, when used properly, types also provide documentation, specifying explicitly what a function takes as input and what it returns as output.

Good tests pin down invariant properties of the system behavior, so that if a change breaks such invariants, the test tells us. Ideally, a test would assert an invariant property without restricting the specific implementation, so that one could change the implementation and still pass the tests, if the invariant is respected.

Here’s a toy example to illustrate what I mean. Suppose I want to implement a sort of key-value store, with two functions: set(key, value) that sets a key to a value, and get(key), that gets the value associated to a certain key, if any.

The type system can specify what keys and values are supposed to be, and even what happens if I try to get a key that is not associated to any value.

On the other hand, types won’t help in ensuring that if I set a certain key K to a certain value V, and then get the key K, I get exactly the value V and not another one of the same type. For that, we need a test like “set K to V. Then get K, and assert that the result is V”.

Such test asserts an invariant that the system should respect, without specifying what the implementation should be. Under the hood the system could be implemented as a hash map, as a database call, etc. without failing the test. This is easier said than done in more realistic cases, but that’s the aspiration.

Sometimes, a bug is discovered, and a test can be added to ensure that a similar bug cannot be re-introduced with future changes. That’s protecting against regressions, and again, ideally it would pin down some invariant that should hold for some corner case, rather than a specific implementation.

I don’t think that tests (or types) help much with ensuring correctness. What really matters for me is to lower the cost of change, and both tests and a good type system help, in complementary ways.

I guess it’s a problem of differing terminology but I don’t see what Gleam has to do with strong typing as it compiles to Erlang (strong typing) and JavaScript (weak typing), at least in the traditional meaning of the terms. But the type inference in Gleam is not complete and the compiler asks you to add types if it can’t deduce what you are doing.

If we look at some real Gleam code I wrote: src/glemplate/renderer.gleam · 5fae861c8649506405e88c5b79214979a20c8b85 · Mikko Ahlroth / Glemplate · GitLab

We can see that mostly the types can be left out in two cases: variable assignment and return values. These can usually be inferred by the compiler, but it’s still very much static typing, since static typing is not an alternative to type inference (Gleam has both).

If you want type inference that can guarantee that the compiler knows what types your variables / arguments / return values are, then it sounds to me that you want static typing with type inference, like Gleam or TypeScript do. I’m not sure type inference can meaningfully be done without static typing – we have surely seen with Erlang and Elixir that Dialyzer is not enough, and misses very obvious error cases (as it just doesn’t have enough information).