What are the remaining gaps in the elixir ecosystem?

[jam]

would love is something similar to Phoenix LiveView but JUST the data store itself

Sounds like LiveState as mentioned above. I think the same author built LiveSignals to sync the data to your frontend.

I haven’t explored either yet but I’m also not sure I want to keep a bunch of state on the server. Would be curious to hear other’s experiences with it. For now, I’m trying to keep my dependencies to a minimum.

Offline-first seems interesting, but for many apps it could be a lot more complex than necessary.

Yeah even if it’s not “offline-first”, I think there is value in thinking “client-first” particularly for applications with lots of interactivity as you mention.

[stevensonmt]

One of the stupidest most bike-sheddy things I absolutely loved about Elm was the commas prepended to new lines. Lists like:

[ :apple
, :banana
, :cherry
]

so that the elements are all left aligned and you never worry about trailing commas.

[sodapopcan]

Is that really how Elm works? Is it not this?:

[ :apple
, :banana
, :cherry
]

If so, that just translates the trailing comma problem to the preceeding comma problem. I’ll give you that it is less common to append than to prepend but it still doesn’t help the diff situation. Although, if it allows double commas like that, that is a whole other thing!

[Eiji]

Is it really about language or just a code formatting preference? I believe you can achieve that with a custom formatter plugin. That’s said I don’t really like this idea. If we would really go for a custom formatting plugin then I would simply allow trailing comma even on the last element (when the map or list has more than one line.

[stevensonmt]

haha, yeah that was my bad. It’s not double commas. But it’s also not an issue with the preceding comma because you’d have it on a whole new line and visually find it immediately if the formatter didn’t already handle it for you (which it does).

[sodapopcan]

@Eiji I was specifically talking about the doubled up commas @stevensonmt had erroneously included (and has now fixed).

I have no problem with this style and actually like it. My main point is that for me the real advantage of trailing commas is for cleaner diffs when appending to a list, this style gives you the same problem when prepending.

Otherwise, they are super convenient but I personally don’t care about too much about not having them. I would absolutely use them if the formatter allowed it but actually prefer it visually without them, so I have a love/hate relationship with the formatter there.

One thing that absolutely does drive me nuts and I can never get used to is this style of tuple return for LiveViews:

{:noreply,
 socket
 |> assign(:foo, "foo")
 |> assign(:bar, "bar")}

My team at work uses this and multiple times per week I go to append a new assign and add it after the }. It’s an easy fix but gives me a visceral reaction every time. I know it’s prevalent and have seen people preach it as GoodStyle, but I do not (and will not) understand the aversion to assigning variables.

EDIT: I wrote “arrows” instead of “commas” (WTH? )

[Eiji]

You can write a macro or a function to solve this depending on what you prefer.

# if block returns socket wrap it in noreply
handle_event :some_message, _params, socket do
  socket
  # …
  |> reply(some: :data)
end

# if returns a tuple leave it as-is
handle_event :other_message, _params, socket do
  socket
  # …
end

# otherwise warn/raise error
handle_event :bugged, _params, socket do
  :oops
end

defp reply(socket, data) do
  {:reply, data, socket}
end

It would be awesome if by default Phoenix would use :noreply and allow to reply with assign/2 or reply/2 function instead of wrapping it in the tuple.

assign(socket, :phx_reply, some: :data)
# or
reply(socket, some: :data)

[sodapopcan]

Ha, I tried those out for a while and didn’t like it. I much prefer re-binding a socket variable and doing {:noreply, socket} (though I will do {:noreply, assign(socket, :foo, :bar)} if there is only one assign). It’s verbose, but it’s explicit and readable. I have snippets that makes writing them painless, though I should really be using AI by this point—I’m just old and slow.

[eahanson]

For those that like this style, Moar.Sugar implements a bunch of these. You can add the library to your project or just copy the code.

[stevensonmt]

Very cool library. I do think the function names are not consistent with normal Elixir conventions. Like noreply/1 and noreply!/1 are not analogous to Map.fetch and Map.fetch!. The noreply/noreply! pair do inverse things to their arguments (wrap vs unwrap) whereas fetch/fetch! do the same things and differ in how they handle errors. Put another way, it’s the argument type that differs rather than the handling of the arguments. The specs from the Map module docs for fetch and fetch!:

@spec fetch(map(), key()) :: {:ok, value()} | :error


@spec fetch!(map(), key()) :: value()

versus for Moar.Sugar.noreply and Moar.Sugar.noreply!

@spec noreply(term()) :: {:noreply, term()}

@spec noreply!({:noreply, term()}) :: term()

Maybe this doesn’t matter to anyone else, but I would find it less confusing to have unwrap in the name of functions that unwrap tuples, with an unwrap! variant that raises an exception rather than returning an error if there is one. I do love little convenience libraries like this though.

[Eiji]

Looks like a perfect backstory for the creation of yet another library!

[sodapopcan]

You guys are uncovering another bullet point on my list of why I dislike these things

[eahanson]

@stevensonmt

I do think the function names are not consistent with normal Elixir conventions. Like noreply/1 and noreply!/1 are not analogous to Map.fetch and Map.fetch!. The noreply/noreply! pair do inverse things to their arguments (wrap vs unwrap) whereas fetch/fetch! do the same things and differ in how they handle errors.

It’s true that noreply and noreply! do opposite things, and I agree that it is not an ideal naming system. It partially follows the normal naming conventions in that noreply! will raise if it’s unwrapping a tuple that’s not {:noreply, term()}.

[spagyther]

There is Khepri, Riak and OpenRiak, the Couch family of DB tech, Hypermnesia (not much beyond poc stage), and probably I am missing a few. I think the BEAM is one of the ecosystems with most databases, tbf, what are you missing? I have used Mnesia in prod and yes, it’s a pain, you need to build a lot of boilerplate around it to keep it from corrupting your data for one reason or another, but Riak for example is fairly battle tested and works kind of out-of-the-box.

[garrison]

First of all thank you for replying I was hoping there would be some discussion on the topic after I posted, but the conversation went elsewhere.

First of all, I just want to point out that this is not necessarily “my opinion” that I’m arguing here. This point is indisputable: Elixir developers are not using BEAM databases. We’re pretty much all using Ecto with relational databases, particularly Postgres/SQLite. I would imagine Postgres users outnumber Mnesia users by at least a few orders of magnitude in this community. When I said we lacked “proper” databases, the word “proper” was doing a lot of work there.

I would say Riak is a “real” database, but architecturally my understanding is that it’s a clone of Dynamo (the original Dynamo at that, not DDB).

This generation of distributed databases was born out of the early 2000s with BigTable and spread to DynamoDB, Cassandra, HBase, etc. The problem is that it’s now 2025, a full two decades later, and frankly these designs turned out to be a bit of an architectural dead end.

First of all, they partition data across machines with no affordances for atomic transactions across partitions (which were often of a limited size). As a result they offer virtually nonexistent consistency guarantees.

Second, because of the “distributed hash table” approach they fail to offer good support for range queries, which are extremely important for managing relational data. So as a result they also fail to offer a relational model, and it is nearly impossible to implement the relational model on top of them.

Modern “NewSQL” databases or transactional K/V stores (like FoundationDB) offer much better consistency guarantees and are built on top of a range-partitioned storage engine (not hash) so that they can perform range scans and implement the relational model. This tech is over a decade old (Spanner 2012, FDB early 2010s), so we’re not talking bleeding edge here. There is nothing like this on the BEAM to my knowledge, but if there is I would of course love to hear about it.

Finally, Khepri seems cool but as I understand it’s just a replicated (not scalable) K/V store which runs all operations through MultiPaxos (actually Raft). This architecture is maybe okay for the old “distributed lock table” approach (which is actually a terrible idea btw), but it’s not appropriate for a “real” database for two reasons.

First, you still only have one table, so you can’t scale this approach to more than one core. And second, by running the log through MultiPaxos on every write you are eating double or triple round-trip latency to write, so it’s just going to be slow.

FoundationDB for example does not do this - there is not even a two-phase commit in the commit path. It’s a much better design.

[garrison]

@spagyther

what are you missing?

To give a direct answer, here’s what I want:

A database which is distributed, replicates data, scales horizontally across cores, is highly available, is range sharded, reshards data automatically, is highly available, has extremely strong consistency guarantees (strict serializability) across the entire keyspace, and runs on the BEAM with a simple native API.

From there, I want APIs which can stream updates to particular keys or ranges of keys with perfect consistency (always in-order, never missing updates).

And then on top of that, I want a relational layer with good Elixir integration and top-tier UX. I want strongly typed records (rows), relational queries, secondary indexes with perfect consistency (always up-to-date), and strongly typed schema migrations. I want to be able to perform live queries and receive updates to the result set as changes come in. I want all of this to scale horizontally.

I think this feature set would be enough to justify leaving Postgres. As far as I am aware there is nothing in the ecosystem that could even be used to build what I’ve described above, let alone provides the features I want.

Frankly we don’t even have anything that matches Postgres, a database which architecturally has not escaped the 20th century.

[D4no0]

@garrison

First of all, I just want to point out that this is not necessarily “my opinion” that I’m arguing here. This point is indisputable: Elixir developers are not using BEAM databases. We’re pretty much all using Ecto with relational databases, particularly Postgres/SQLite. I would imagine Postgres users outnumber Mnesia users by at least a few orders of magnitude in this community. When I said we lacked “proper” databases, the word “proper” was doing a lot of work there.

There are a few reasons why people don’t use mnesia, some of them:

1. Peculiar way to do queries. This can be worked around and I think there are a few libraries that do that in elixir pretty well already. Nonetheless this can never be compared to having a proper DSL like SQL;
2. No external tooling. For me this is the most important factor when it comes to databases. Having the ability to open a CLI or GUI to query and inspect the database is critical. I’ve even worked on projects where we had analysts that were querying the database, as that was 100x times more productive than trying to build for them some custom web interfaces;
3. Migrations and backups. Databases like postgres offer a variety of tools for migrations and backups. If you were to use something like mnesia, you would have to implement all of this from scratch;

@garrison

This generation of distributed databases was born out of the early 2000s with BigTable and spread to DynamoDB, Cassandra, HBase, etc. The problem is that it’s now 2025, a full two decades later, and frankly these designs turned out to be a bit of an architectural dead end.

Distributed databases are a very niche field and it’s blown out of proportion on how useful they actually are. What I’ve seen in practice is that 95%+ of projects won’t need a distributed database, ever. There are also things like replicated read-only databases that postgres offers, this strikes a perfect balance for cases where you want distribution, but you don’t want to deal with the horrible experience of making all of that stuff consistent.

[garrison]

@D4no0

There are a few reasons why people don’t use mnesia, some of them:

I was trying to avoid roasting mnesia because I feel like we’re all aware of the drawbacks, but I was surprised to see you left out what I thought the two biggest problems were:

1. Tables are limited to something like 2GB and then have to be partitioned, and once you partition them you lose all consistency guarantees, which is catastrophic. Also aren’t they stored in-memory?
2. In the event of a netsplit consistency guarantees are nonexistent, which is again catastrophic. From what I remember this is essentially unrecoverable?

Of course the standard disclaimer applies here: the people who built Mnesia weren’t idiots or anything, it’s just that a) it was a really long time ago and b) they were designing for specialized telcom hardware or whatever, not Phoenix webapps!

@D4no0

Distributed databases are a very niche field and it’s blown out of proportion on how useful they actually are.

I understand why you feel this way, but I disagree very strongly with this viewpoint. Distributed databases are very useful first because they can substantially improve durability and availability, both of which have proven to be very difficult to glue onto existing architectures (see the pain and suffering wrought by Flyio’s HA postgres attempts, for example).

But second, they are useful because Moore’s law ended a while ago and compute has primarily increased in parallel (more cores). We have reached a point where even your phone is almost certainly a distributed system, and so distributed databases must necessarily be the default. The ancient 20th century RDBMS architecture (Postgres) is extremely wasteful on multi-core hardware and spends a ludicrous amount of time dealing with lock contention instead of useful work. I don’t think this is something I need to justify because this is the Elixir community and I think we natively understand the benefits of parallel processing here

@D4no0

but you don’t want to deal with the horrible experience of making all of that stuff consistent

But that’s the funny thing, isn’t it. It’s like what Joe Armstrong always used to say about Erlang: all it takes is a few smart people to work out the “distributed” part, and then the rest of us benefit for free!

[D4no0]

@garrison

The ancient 20th century RDBMS architecture (Postgres) is extremely wasteful on multi-core hardware and spends a ludicrous amount of time dealing with lock contention instead of useful work.

No offense, but this is not an argument, more like a troll post I would see on linkedin.

Postgres excels at concurrent access to data, this is what it was built for in the first place. If you are in the search for the next hip technology, then I can understand that, but please don’t use the fact that postgres is mature as a argument to jump the ship. As any other tools, you as a professional need to assess where it fits and where it doesn’t.

@garrison

I don’t think this is something I need to justify because this is the Elixir community and I think we natively understand the benefits of parallel processing here

Is it though, or it’s about easy to manage concurrency? I see a lot of people using concurrent and parallel interchangeably these days, however by definition concurrent code is not necessarily parallel.

@garrison

But that’s the funny thing, isn’t it. It’s like what Joe Armstrong always used to say about Erlang: all it takes is a few smart people to work out the “distributed” part, and then the rest of us benefit for free!

Taking in consideration that distribution, redundancy and consistency is a much bigger and harder problem than concurrency, I am more than positive we are at least 20 years behind, or if we were to look at telecom industry, where these problems were prerequisites to make that industry work, maybe we wouldn’t have to reinvent the wheel everytime. I guess that is not fun because the industry is too old.

Look at the currently used programming languages and how bad they are at dealing with concurrency, transpose that technological gap to distribution and the answer is clear that we are not there yet.