a8t

a8t

How do you approach adding or migrating to Elixir from a different language?

To clarify the question, I would like to discuss the technical aspects of your approach to introducing Elixir in the context of a production system running (maybe poorly) in another set of languages or technologies. I understand that there are possibly more important social aspects to this question, but I’d like to focus on the technical aspects. I’m hoping not to discuss in abstract, but instead with concrete experiences migrating to or adopting Elixir.

I know that the decision to switch is a social AND technical decision, and I realize sometimes the answer is “let’s not do that”. But let’s take it as a black-box dependency. Once you have made that decision: In your experience, what are your short-term, medium-term, and long-term technical plans for introducing Elixir? Do you aim to fully replace the other parts of the system? Do you run Elixir services alongside other services? Do you prefer an Elixir monolith even if the existing system is built with microservices? How do pieces communicate?

My specific context at work is a NodeJS monolith connecting to a Mongo cluster, plus a few cloud infrastructure appendages like an Azure Service Bus. I’m happy to hear about whatever context was present in your experience, as deemed relevant by you.

I’m also happy to be pointed to a relevant thread or post to read!

In short, I feel that there are many resources on why folks have adopted Elixir. But I haven’t found many on how they’ve adopted Elixir!

Most Liked

John-Goff

John-Goff

So I can only speak to how I did it, but I have successfully migrated a rails monolith to an elixir monolith app, over the period of about a year and a half. So our stack at the beginning is nginx serving a static frontend, a rails json API, a MySQL database, and all of those are in docker containers inside docker-compose. Nginx is configured so any requests to /api are proxied to rails. At the end, we ended up with nginx serving the frontend, a phoenix json api, raills API for some legacy accounts, and a postgres database. The way we decided to do this is by using the Terraform library. Requests to the api first hit nginx, are proxied to phoenix, and if phoenix doesn’t understand it then it’s sent to rails using the terraform library.

So the first steps was setting up terraform, making sure rails and phoenix had the correct database credentials, and then defining some basic ecto schemas. I started here by copying out of my schema.rb file and pasting it into the ecto schema file, then changing the syntax so it matches ecto’s, which worked pretty well. Then I needed to configure the JWT library Joken so that it matched the same secret key and settings we were using with devise-jwt on the rails side. Now we can log a user in via rails and authenticate that person in elixir land. Yay, now it’s time for feature development!

The business case behind adopting phoenix for me was channels, so it makes sense that the first feature we shipped was a chat system. Then it was a slow process of new feature development happening in elixir, and any time I needed to fix a bug with some ruby code I tried to rewrite it in elixir. This was by far the longest step of the whole process.

The two lingering things in rails for a long time was image uploading, which we were doing in rails with ActiveStorage, and user registration/authentication, which we were doing with devise. To solve the image uploading issue, I used Waffle and waffle_ecto to upload any new images, and I dug into the activestorage source code to figure out how they were generating urls for the images, and I replicated that in elixir code so we could generate a URL in elixir based on the activestorage tables that were stored in the database. This was probably the only actually hard code to write in this whole process. I have considered open sourcing this, but I’m not sure if anything has changed in activestorage since I wrote that code, and I wouldn’t really want to maintain it as it’s kinda a one and done thing in terms of usefulness. This was with rails 5.3 at the time. So once we had that, it was basically just writing a function to check if there is a waffle uploaded image, if so use that url, and if not then call the activestorage url function. For passwords, I did something similar except using the phx_gen_auth library to create the new password tables. When you change your password or create a new account, it uses the phx_gen_auth code to set that. When logging in, it first checks if your password is valid with phx_gen_auth, if that fails it calls rails and does the check with devise.

I also migrated mysql to postgres during this time, and while not strictly related I thought I’d share that story as well. Elixir made it super easy, I added both the mysql driver and the postgres driver to my dependencies, then I renamed my repo module which was set up to work with mysql to OldRepo. Then I created a new repo module and configured that to work with postgres. I then wrote a function to call from IEx, which basically listed all the tables I wanted copied, and basically did Repo.insert_all(OldRepo.get(table)), though I think I needed a bit more code than that due to the limits postgres has on the number of items you can insert at one time with insert_all. All in all, pretty easy!

Hopefully this is what you were looking for!

al2o3cr

al2o3cr

FWIW, Adopting Elixir has some fairly in-depth discussions / interviews about this exact topic.

dimitarvp

dimitarvp

As @John-Goff pointed out, you absolutely need a solution to route to your old monolith and your gradually growing Elixir app on a per-route basis, and terraform is indeed one very good way to approach it. This is your very first stop.

Once you handle that successfully (try and just have a testing endpoint like /hello/test/elixir or some such as a start) then it’s a matter of just moving things endpoint by endpoint.

One key element is to keep the DB migrations in the old monolith for the time being and export a full SQL data structure after each migration is added and export that SQL file to the Elixir app so it knows the DB shape (although that’s strictly optional; as long as you define your Ecto shema modules well then it doesn’t really matter if you have a priv/repo/structure.sql file or not – this is important if you want to provision an empty DB while working only with the Elixir code).

Another key element is to keep the Elixir app on a separate server. Elixir (technically the BEAM VM) is by default tailored to use all CPU cores of the system so for best results keep it alone in a dedicated instance even if has like 2 cores and 256MB RAM.


  • SHORT TERM: Parallelism benefits: rewrite the parts of the old monolith that could benefit from heavy parallelism and/or are already struggling when a certain parallel load is going on. That should be your first priority! To make things better, the mere fact of using Ecto and Phoenix already gives you that automatically without you lifting a finger.

  • MID-TERM: Scaling. I got tired of trying to convince people that Ruby / PHP / JS / Python codebases usually use more CPU (and sometimes RAM but that’s less deterministic) resources than a 99% equivalent Elixir codebase so I started demonstrating it instead and except for one time (a custom-compiled Node.JS heavily tuned for I/O throughput) it was always a success that led to smaller hosting costs several months down the line. I know there are still many around here that would contest such a claim until the end of time but I know what I saw in various DataDog / NewRelic / other telemetry performance dashboard solutions.

  • LONG-TERM: Code maintainabiliy. This is not a given and depends on the team, a lot. One advice here is: ignore common wisdom and find your own best patterns. I know at least 4 different ways how people structure their project: none of them are “the best”! You should invest in quick code discoverability from the get go. Don’t get fixated on artificial constructs like models, controllers, views, templates, domain / business logic and many others – to clarify, they are a very good start but often get problematic once the project starts growing. Focus on this: “if I take a 3 months break, where I would first look for X if I have a task Y?” This question should be asked to each team member and the code should be organized according to a democratized average/mean value of the answers.

Last Post!

John-Goff

John-Goff

I have not no, but that’s a decent idea so I might. In the meantime it’s really super barebones, just calling tmux new-session and then tmux split-window a few times. At the end make sure you call tmux attach-session to connect to the session you just created. Whole thing is under 50 lines with the majority being setting up env vars to pass to the various servers depending on some flags you can give. Really like 6 lines of code are directly dealing with tmux and I’ve covered them all. You can read the tmux man pages for more info I’m sure, that’s where I got most all of this.

Where Next?

Popular in Discussions Top

AstonJ
If a newbie asked you about Phoenix Contexts, how would you explain the basics to them? Feel free to be as concise or in-depth as you li...
New
ben-pr-p
In general I’ve been sticking to this community style guide GitHub - christopheradams/elixir_style_guide: A community driven style guide ...
New
AstonJ
I’ve just started the Phoenix part of the utterly brilliant online course by @pragdave. On generating the Phoenix app he uses the --no-ec...
New
dogweather
I wrote this comment on r/haskell, and it’s not popular there. :wink: But I think I’m on to something… Haskell reminds me of Java, and e...
New
Nvim
Elixir appears to be a superior language to Python. I don’t see any advantage of Python over Elixir. Are there any?
New
Fl4m3Ph03n1x
Background A few days ago I was listening to The future of Elixir from Elixir Talks, with Dave Thomas (@pragdave ) and Brian Mitchell. I...
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New

Other popular topics Top

Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 44139 214
New
AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New

We're in Beta

About us Mission Statement