I personally think it makes a lot of sense to implement lighter versions of such 3rd party products as BEAM libraries, with the goal of simplifying the operations and reducing the amount of moving parts. There are already many examples where we can opt for BEAM libraries instead of external tools & products, such as nginx, cron, or redis. The alternatives from the BEAM ecosystem don’t necessarily match these tools in terms of features or performance, but in many cases they can work just fine, and help simplifying the system architecture.

I’d like to see the ecosystem growing further in this area. For example, I’d love to see a relational database as a BEAM library. Something I can add as a lib dependency, start an instance (or multiple instances) somewhere in a supervision tree, and have SQL based persistence without needing to manage a separate database instance, roles, handle language ↔ db type mapping etc. When I occasionally mention this during my talks, I get some skeptical feedback along the lines of “Why would you want to reimplement databases such as PostgreSQL, MySQL, etc.?”. The point is not to reimplement or compete with established databases, but to have a lightweight alternative which would be more fitting in simpler scenarios.

Ideally, if I want to build a small to medium web-facing CRUD, I should be able to start using nothing but Elixir, and get the basic skeleton working within 15 minutes or so, with everything implemented in a single language (say Erlang and Elixir), inside a single project, running as a single OS process per each node in the cluster. As long as we’re not able to do this, I think there’s a lot of potential for improvements in our ecosystem, and implementing alternatives to established products makes a lot of sense

Phoenix’s PubSub does not do what kafka does.

I had need for a lightweight topic based log system and we created GitHub - erleans/vonnegut · GitHub

It is compatible with kafka on disk and on the wire but uses chain replication instead of Kafka’s ISR based replication. It is not ready as a kafka replacement, There is plenty to still do around membership and concensus but the internals are there. Figured I’d mention it in case anyone wanted to work on such a thing and it could be a useful base layer

We are 3 developers having less than 1000 users. I’m tired of “this is how Netflix does it” or “Google uses” etc.

As far as I experience it there are few software talks, courses and systems that focus on doing stuff at small scale yet still professional.

I like these E languages not because I have to solve 2 million concurrent users but because I’m hacking away trying to reap the benefits of the microservices style within my application.