darkmarmot

darkmarmot

Someone recently asked “What feature would you most like removed from your language?”

And, for me, it has to be structs or the way Elixir currently handles them.

In my job, we do a lot of deployments on distributed clusters, either pushing hot code updates against running servers for small changes or running multiple versions of our code side-by-side in the cluster as we do rolling deployments for large changes.

Thus far, we’ve had zero downtime running with a cluster of roughly 30 servers (knock hard on wood) over the last year and a half (since we went into production).

Our biggest difficulty (and danger) has been Elixir’s structs. While Erlang was designed for our kind of environment, it feels like Elixir broke away from it with its struct implementation.

To explain, functions that handle structs don’t happily duck-type them. If you make changes to a struct definition and move it between nodes, you can’t match on it unless all the fields are in perfect agreement. So there’s no easy way to update them in the running system.

We don’t let any data that moves between nodes contain them.

And this is where it gets really ugly. A lot of Elixir’s base data types are implemented as structs, such as Range and DateTime.

So, what seems like a simple non-breaking change for the language, as when the recent version of Elixir added step to Range, could actually cause catastrophic chaos on our distributed system.

I would propose that Elixir consider making structs act as duck-typed maps in the future as I love the language, but I hate to see it limiting the power of the original VM and ecosystem.

Showing Posts 1 to 10

lpil

lpil

Creator of Gleam

Rather than removing structs I would be interested in hearing how structs and hot-upgrades could be improved to work better together. I think there’s value in both, and removing either would be a huge change that would likely make a large proportion of users (most users?) of Elixir unable to upgrade to this hypothetical version.

I would expect Erlang to be worse as records are even harder to match on structurally than Elixir’s structs are.

darkmarmot

darkmarmot OP

I would be happy to have things under the hood use the __struct__ field for things like protocols and types. I would just like to be able to mostly use them more like maps in general. As for records, they aren’t something that’s ever come up as an issue for us as we mostly don’t use them.

dorgan

dorgan

Removing structs = no protocols :cry:

cevado

cevado

but protocols are just “type”(thinking of structs as types) based dependency injection. I’d rather like an explicit options dependency injection. I think the only place they’re not interchangeable is in nested stuff.

darkmarmot

darkmarmot OP

Why can’t the language just dispatch on matching %{__struct__: some_name} and ignore the rest of the fields?

lpil

lpil

Creator of Gleam

They come up rather a lot in Erlang libraries and codebases in my experience, so I think Elixir may do better than Erlang here.

This would not work if a field is added as it would crash when the field is found to be missing. That or you would have corrupt data after the upgrade due to missing fields.

The struct pattern is effectively an assertion to say that your data conforms to the schema. If these assertions do not find the problem from the upgrade that did not correctly migrate the state to the new format then it would likely cause errors later when the data is attempted to use.

darkmarmot

darkmarmot OP

Yeah, that could definitely be from my comparative lack of experience in Erlang.

darkmarmot

darkmarmot OP

Perhaps structs could also include a __version__ number such that code could handle structs with different formats?

lpil

lpil

Creator of Gleam

There’s not any way to know when the version needs to be bumped or what should happen when there is a version mismatch. At best we can check the structure of the struct and crash if the data doesn’t match the schema, which is what structs currently do.

Until someone makes a sufficiently powerful type checking tool that could be run against both versions of the codebase the business of upgrading state in hot upgrades is going to be a manual process for the programmer to implement via the appropriate OTP callbacks. It’s a very challenging job, and I recall reading once that Ericsson spend as much time testing and developing their upgrades as their application code.

darkmarmot

darkmarmot OP

We often do daily multi-version deployments or hot code upgrades on our cluster. For us, the key is that the data transferred between nodes has to be to inferred (no structs, just maps) and we avoid hot code loads that modify GenServer states (these are usually based on modifying pure functions).

Where Next? Top

Trending in Discussions Top

cblavier
Hey there, It’s been more than a year since we started using LiveView as our main UI library and building a whole library of UI componen...
515 12931 101
New
wmnnd
Hey folks, I wanted to share with you a project I’ve been working on for a while: It’s an Open Source (AGPLv3) newsletter tool. Think Ma...
New
raa-ziq
Hi everyone :waving_hand: I wanted to share something I’ve been building — NowBlind, a random chat app designed to help people discover ...
New
axelson
Hi there! :wave: @frigidcode and I (but mostly him) have been running an Elixir Book club, we’re almost done with Designing Elixir Syste...
New
alexslade
Fly’s CEO posted this recently - Turn And Face The Strange · The Fly Blog It says that Fly is going all-in on sprites, which is a worry ...
New
sntran
In my third year of college, around 2010, I found Erlang. A few years later, I found Elixir. I liked the ideas at once: many small proces...
New
Ganna
Hi everyone, I have 8 years of production experience with Erlang/OTP and I’m just starting to get to know Elixir. What would you learn f...
New

Other Trending Topics Top

dominicletz
Hey there, I haven’t seen a list of useful Elixir specific skills for the popular AI tools yet so I’m starting one here with my first. ...
New
mudasobwa
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
KristerV
Hey. Is there anyone here who creates agents in their apps? Not talking about using agents, but creating them. I’m finding it pretty diff...
New
OndrejValenta
(I just needed to vent somewhere and LinkedIn is full of hope, or hype, I’m not sure which exactly) AI dream has many faces, but general...
New
webofbits
With AI doing more of the implementation work, I’ve been wondering how much coding I should deliberately keep doing myself. My main conc...
#ai
New
kip
In 2021 I started a new library called Tempo with the objective of modelling time as a set of intervals - not as instants. In 2022 I gave...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews