francesco.pessina

francesco.pessina

Migrate from microservices to Elixir/Phoenix: is it possible?

Hello everyone,
I beginned recently to study and work in elixir, and I’m very curious about OTP, scalability and performances.
I’m writing this message to ask you if what I’m thinking makes sense or I am missing something.

By now I’m maintaining a platform made up of 6 microservices, all in Java (Quarkus/Spring), which intercommunicate with Kafka (both for data and events).
Every microservice has its own database; the platform asks data to an external APIs (through 3 “connectors” microservices) and write them into Kafka. Then another microservice reads this data and stores them into a database (with some logic), and exposes them through different queries.
Finally there is another service which acts as authentication server and gateway (with authorization).
All run in a Kubernetes cluster.

What I am thinking is: can I replace all this with a Phoenix application, where every microservice could be a separated app (maybe all under the same umbrella)?

These are possible pros:

  • One database for all (is this an anti-pattern?)
  • Get rid of Kafka (and Avro)
  • Simpler maintenance
  • Easy support to GraphQL (if I choose to make the switch, now it’s all rest)
  • Elixir

Possible cons:

  • Losing of scalability opportunities (with Kafka I can scale services as many as I need: how can I replace this with Phoenix? Do I need this?)
  • Minor decoupling of the system (I cannot “plug” an eventual microservice developed in another language)

Making the question in another way, Elixir applications and OTP could replace a platform made up of some microservices (all developed in the same programming language) which communicate with a message bus?

Thank you for any eventual response :slight_smile:

Most Liked

Qqwy

Qqwy

TypeCheck Core Team

Yes, this is possible. And maybe even more importantly, it might be possible to perform this change incrementally (switching over microservices one by one).

Depending on your needs you might be able to either augment or fully replace Kafka with Broadway.

It is very straightforward to create multiple releases from the same umbrella project, containing a different set of OTP-applications, and connect these together in a ‘distributed Erlang’ cluster. This will allow you to scale different parts of your platform independently (e.g. scaling the amount of webserver nodes independently from the amount of data ingestion nodes etc.), while not having to think hard about how these microservices need to communicate, as transparent everyday message-passing can be used. (Usually e.g. Phoenix.PubSub is used in these kinds of situations which allows for even further decoupling between services.)

Obviously this last part is only possible once everything has been moved over to Elixir. However, when rewriting incrementally you will still be able to remove more and more ‘manual communication’ code as time goes on.

Where Next?

Popular in Questions Top

sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
marius95
Hello everyone, I try to use an Javascript Event Handler in my root.html.leex file. Therefore I created a function in the app.js file: ...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
Lily
In templates/appointment/index.html.eex: <%= for appointment <- @appointments do %> <tr> <td><%= appoi...
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
nobody
Hi! In PHP: $SERVER['SERVERADDR'] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New

Other popular topics Top

mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I fore...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
chrismccord
This release brings a number of exciting features, including integration with the new Phoenix LiveDashboard and Phoenix LiveView. There h...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
hariharasudhan94
Lets say i have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => "XX...
New

We're in Beta

About us Mission Statement