Hot reloading and static types

himaraya · December 25, 2018, 1:08am

I read this thread from last year and got curious about the mentioned black-boxing “fix” for hot reloading with static types. How feasible is it? I read that a statically typed version of Erlang was abandoned years ago due to speed and complexity concerns.

Edit: Another thread mentions other obstacles including backwards compatibility to statically typed Elixir. I also came across Elchemy, which doesn’t seem production ready. Thoughts appreciated!

lpil · December 25, 2018, 1:34am

At Facebook they actually do hot code swapping of Haskell, which is a statically typed language.

The rules they follow is that the swapped in object code must export functions of the same type as the previous version. There is nothing checking these types are correct other than human testing, so it’s effectively dynamically typed at this point.

Alternatively we could define an opaque “any” type and force the user to safely decode their data on code change. This isn’t an option for Haskell as their runtime needs to know the memory layout of the data as it is not designed for dynamic data.

I could imagine a system like this working for a typed Erlang. If I might self publicise a little, I’m working on a typed language for the Erlang VM at the moment, and I plan to experiment with this here https://github.com/lpil/gleam

For more info on the work at Facebook the keynote from Simon Marlow at this year’s Haskell exchange was very informative.

himaraya · December 25, 2018, 4:07am

Thanks! I just came across Elchemy – how’s Gleam different?

lpil · December 25, 2018, 9:03am

Elchemy is an alternatively compiler for the Elm programming language that outputs Elixir rather than Javascript, Gleam is an entirely new language.

The most obvious differences between Gleam and Elm are that Gleam has side effects, Elm does not, Elm has autocurrying, Gleam does not, Elm has a Haskell like syntax, Gleam has a Rust like syntax, and Gleam has a slightly more expressive (though unfinished) type system as it has typed modules. Elm’s compiler is written in Haskell, Elchemy’s compiler is written in Elm, Gleam’s compiler is written in Rust. Both languages are relatively simple functional programming languages that prioritise safety and clarity.

Elm and Elchemy are more mature than Gleam (which isn’t use useable currently). Gleam was designed to be a BEAM language while Elm was not- because of this I think that once ready Gleam will more suited to writing Erlang/Elixir style OTP programs than Elchemy is, though that’s possibly my personal bias.

Other projects worth checking out are the Alpaca language https://github.com/alpaca-lang/alpaca and the Purescript to Erlang compiler https://github.com/purerl/purescript

Qqwy · December 25, 2018, 5:51pm

It’s interesting to note that in Haskell, there is a library to facilitate upgradable types, called SafeCopy, which is used in some systems, but it does not seem to be entirely painless:

From user ‘Mightybyte’ on the Haskell ∪ Chat Discord group:

@Qqwy The safecopy approach can be a little cumbersome to use in practice.

Its migration requires you to define a function migrate :: a -> b

Data.SafeCopy

This means that if you add a new field to your Person data type, you have to keep the old Person data type around.

In practice, the easiest way for me to do this was to create a module Person.V1 where I put the old code.

Then migrate :: Person.V1.Person -> Person

At first glance the approach seemed nice to me, but after using it for awhile with larger production data types, it got pretty painful.

If you think of this problem as a graph where the nodes of the graph are your data types and the edges of the graph are migrations between different versions, I call this safecopy approach the node-oriented approach. You have to keep around all the nodes.

The other approach is what I call the edge-oriented approach, which is where you don’t store multiple versions of the data types, but rather you store code that performs the migration. [which would not use SafeCopy, but e.g. migration logic in plain SQL, and assumes that your data structures are all at rest when migrating.]

lpil · December 26, 2018, 11:11am

Another problem with that approach is that it depends on the old version being known. If you write the incorrect type there or the update is applied to a node running different code to what you expect then your upgrade will fail and likely lose state.