Elevator Pitching Elixir and Phoenix with live example: Need some advice

Hello, friends!

So where I work we have the massive entanglement of a Java + Vue application, that in a lot of aspects is a headache to make changes to and especially when business wants to make changes.

Later down the road I thought I’d pitch us moving to Elixir and Phoenix, for a few reasons.

  1. Live updates to data

    There’s quite a few times where there’s no reliable way to update certain data in our admin interface (due to the library we’re using and what version we’re on) other than SSH-ing to the production/testing site and running some SQL and then restarting Tomcat. This process is not only slow, but inefficient. With Elixir we can connect to a REPL and make updates with Ecto instead of just raw SQL and there’s no need to restart the server or rebuild the whole project. This is probably the biggest thing.

  2. Opportunity to create a better admin interface

    Our admin site is dated and not super efficient, but that’s in part due to a library we’re using. I know projects like Kaffy exist, but I’d want to be stay away from that unless I know that it’s totally flexible. This is the second biggest thing.

  3. High uptime (so long as AWS cooperates)

    Don’t need to explain this one to y’all.

  4. Better development tooling

    With the little experience I have with Elixir, I like its tooling far better than Java’s. It’s also a bit more flexible compared to what we’re using with Java. credo is very nice and mix xref is really neat to have. I loathe pom.xml, I’m very tired of XML in general and so does everyone else on the team, and I don’t really like Gradle. Mix is much nicer. Also the way we implement some server tasks is through some library that requires us to insert data into a table where one row contains a cron expression. Not terrible but I don’t really like that.

    Elixir’s more modern tooling would culminate into faster and more efficient development.

There’s also stuff like being able to more easily implement data validations via Ecto but off the top of my head those are the ones that business would care about.

The biggest thing I’d like to do is have an actual working example. I’d really only need

  • A database
  • API
  • Admin interface

All not hard. There are a few things that business would like that sort of complicate this though.

One is a publishing mechanism for data, or new entities in that database. Say we’re adding Somaliland as a new country (I don’t know why we would in our case, just the first thing that came to mind), and before it can be used for existing or new cities we add to our database it has to be reviewed before it’s published. I don’t really know what the Phoenix way of doing that would be. This is the biggest thing for the admin part of the project, the rest I can figure out.

This isn’t really an Elixir thing, it would be something I would have to present in the elevator pitch, is avoiding the possible headache of moving our data from our database and the structure that a software we use for our platform requires us to have. We’re have a MySQL database and at some point we had a PostgreSQL database. I don’t know why we changed to MySQL, but that was before I was brought on. I’m guessing there’s no easy way to migrate databases without creating very filtered SQL dumps. Maybe Elixir has something for this, but I wouldn’t really know. This would probably be the biggest thing to make the business reluctant.

The other thing is keeping the current Vue frontend for the site. I don’t know how Phoenix and Vue 3 work together currently, and currently we build that part of the project with Vite. So long as we have the API, we wouldn’t have to change anything in regards to the frontend or at least I’m 95% sure of that.

Anyway, sorry for the long post but I’m interested in reading any suggestions you give.

IMO this doesn’t sound like a Java problem, it sounds like a specific issue with that library. Nothing about Java itself would require this kind of behavior, and nothing about Elixir would prevent it - you could certainly write a Elixir process that read from the DB once at boot and then didn’t re-check.

I don’t know of a specific off-the-shelf solution for this, but it sounds like a fairly standard sort of “hide until approved” workflow. It’s likely that the “Phoenix way” would use the same database structures as the app does today.

Is your Java application currently sharing the database with something else? That’s going to make everything else more complicated if it’s happening…

I’ll leave the question of JS bundling to people who know more, but the API should be straightforward to wire up with Phoenix controllers.

1 Like

IMO this doesn’t sound like a Java problem, it sounds like a specific issue with that library.

Yea, it’s definitely more of a library thing. But we are also on Java 7 and updating to a new up-to-date Java version, new framework version, restructuring the app to meet with how the new version of the framework requires the app to be structured, restructuring the database to meet what the library requires and updating to new dependencies at this point would be more of a headache than just starting from scratch.

It’s likely that the “Phoenix way” would use the same database structures as the app does today.

Well, I don’t really know about that. The library we uses “sandboxes” data that someone has yet to be published, as opposed to having some column that you just set to true or whatever. I’m not 100% sure what it means by “sandbox” but there is an foreign key in many of the entity tables that points to some other table.

Is your Java application currently sharing the database with something else?

No, sorry. What I meant by “software” is the library we use. It’s Broadleaf.

i recently pitched Elixir to a team that would IMO benefot a lot from it. to my great surprise their response was ‘meh’. later, when talking to them, i realized why. because my pitch was oriented around elixir’s strengths rather than their weaknesses. back when elixir was pitched to me we just happened to have similar problems.

what i would do this time would to take a very specific problem the current codebase is having and code a solution for it. make it elegant, present it to the team. then when they’re still interested you can talk about the other stuff elixir does. but in this case you’d really have to focus on something Java can’t (or won’t) do. something that can’t just be refactored in the current codebase. or i suppose something that is easy to refactor in elixir, but hard on java.

just my two cents about an effective elevator pitch.

4 Likes

That’s a good point.

I think for us, the problem is the utter inflexible giant that is Java and the frameworks and libraries we’re using are compared to what Elixir gives us. The other thing is the nightmare that it would be to update to newer versions of Java, not allowing us to take advantage of newer features in the new version of Java and Java frameworks. Granted this could obviously be an issue with Elixir, but I feel like it’s far less a nightmare compared to Java. The other thing is the lack of supervision for each piece of the project. It’s all disjointed, and, sure, we could try to refactor to a more “microarchitecture” approach. However, even though we have moved to that somewhat recently, if one part of the site has a bug we can’t just push a simple hotfix and boom the project is fixed. We have to go through the whole rebuild of the project, as part of the CI pipeline, and with how slow Java is and how big of a mess the project is, this is a good 5-6 minute wait time. Where as if each part of the project is its own process, we could just update that part of the project without having to restart the entire application.

There’s other things little things like the library we use for our admin interface being very strict and customizing it is a pain. Adding brand new entities is time consuming. Thymeleaf is…yea. Configuring Java beans is a headache and autoconfiguration usually works but you are also slightly crossing your fingers. Java stacktraces, need I say more? Integration with new libraries is also a nightmare, requiring a lot of setup on our end before we can actually see if it will work for our use cases.

One question I had in relation to Elixir: in the case of having a real-world project that’s has

  • a database, using Ecto to interface with it
  • an admin site
  • an API

is it better to have it be an umbrella app, as opposed to one Phoenix project with separate libraries that’s something like

your_project/
-- lib/
---- your_project/
------ db/
------ admin/
------ client/
------ api/
---- your_project.ex

Or vice versa? Currently we just have a giant monolith of a project, but nearly each part has their own pom.xml so it’s kind of like a umbrella app? Maintaining dependencies for multiple sub-apps doesn’t sound too fun, even for an Elixir project.

You’ll need an umbrella project or, just a collection of classic projects. Though with Terraform and co. you can semi-automate all the blue-green deployments and restarts, so it’s gonna be mostly handled for you, should you take the time to set it up. IMO it’s very much worth it, I’ve seen it work.

To be fair, very small amount of stacks are better than this – Golang comes to mind with its super fast compilation. I’ve also seen Elixir projects needing 10 minutes for CI/CD though it got better in the last several years with GitHub / GitLab Actions and with more detailed caching phases. It’s very doable to have a PR compile in 2 minutes, I witnessed it in my last several contracts.

It’s a b*tch to set up though. I never tried to invest in knowing these YAML “programming languages” but knowing them pays off for sure.

I’ve personally never done anything with terraform, but maybe the senior developer has. Totally down to learn something new. Would the project be something like this?

Oh god, and here I thought I could be done with markup languages used for configurations.

As another note, I was considering using Ash and AshPostgres to represent our entities. I’ve looked at it several times and I like how it abstracts a lot of Ecto stuff into a easy to understand DSL. We have relationships between entities that we could fix/clean up/make more clear and we could really make use of Ash’s calculations in some places. It’d be great to use Cachex as well, but maybe Ash has something already where we can store calculations with ETS.

1 Like

Yeah, pretty much. You can segment your setup in TF for convenience and version-control it as it’s done in the example. If you know the language, it’s a fairly trivial stuff after. AWS is the biggest problem (as usual) but after a few trials and errors you’ll nail it.

I know right. :021: I hate them with a passion.

Hmm, there’s another thing with umbrella apps that I’m not too sure about. Can I really not have dependencies that are “global” to the whole umbrella app? So if I use credo, I can’t put it as a dependency in my_project/mix.exs and that be used project wide? I have to do it in each app’s mix.exs in apps/?

If so, that’s a bit of a bummer. I guess I could just install credo and have it in my PATH on my machine.

My advice: learn what this means. You’re going to need a deep understanding of “how things work now” to have any chance of a replacement project working out.

Similarly, you need to understand why the company chose to build on Broadleaf - does it solve important problems for the company? Was it easier to get started and now turned into a tarpit? Did the sales reps buy your C-suite a skiing trip to seal the deal? (that last one would suggest you have a non-technical challenge ahead :stuck_out_tongue: )

Usually you’d run mix credo from the top-level and it automatically recurses into the directories in app, Either way the files wind up downloaded to deps and compiled to _build exactly once. :man_shrugging:

I’d also recommend avoiding the temptation to dip into micro-optimizations like “you mean I might have to add ONE LINE to FOUR or FIVE config files?” - your pitch for switching isn’t going to hinge on them.

1 Like

Well this project was initially created by contract devs from a company that we stopped working with awhile ago. So that might give you an idea what the answer to the next question is.

Yes, haha.

Then don’t let bugs that are so easy to fix reach production.

This may sound harsh, but 6 minutes of CI should not be such a concern. A longer CI time makes a trial-and-error approach of production harder and makes everyone thinks twice before pushing code. Once a hotfix can be deployed in a minute, “just push it, if needed we can fix” becomes way too tempting!

In our company the CI kicks in as soon as a pull request is opened. Most of the times the CI already reports failure or success before a colleague has reviewed the. So no time is lost.

If only we had thought of that.

Sure, makes sense. I guess the CI isn’t as big of a concern as I thought it might be.

I wouldn’t personally choose umbrellas unless you absolutely need the interdependencies or you approach them like microservices. If each portion has its own team that makes sense but each one could just as easily be a mix project brought into a parent.

I primarily work in Laravel and I’ve seen approaches to segregate core layers but I’ve never needed them. Back when I did C# work I separated them and didn’t need to but it made sense logically and they were always very lightweight. I never replaced the database layer from ms sql to another rdbms even though that was my primary reason to segregate. I do create composer packages and leverage a not insignificant amount of dependencies over the fleet of projects I’ve touched but I would take that approach with Elixir and Phoenix until I had a compelling reason not to.

One hiccup I have seen that could be temporary is elixirls prefers I open vscode in each app directory in an umbrella. If I have to do that for productivity then I might as well split everything up as separate mix projects. Separate git repositories, versions, and potentially git submodules or subtrees all make sense here too me.

1 Like

Our team is too small to do that.

Yea in Emacs, ElixirLS seems to have a hard time with umbrella projects. But that may be due to that I haven’t set lsp-elixir-project-dir to "". I’ll have to see sometime this weekend if setting that variable fixes things.

I’d advise against umbrellas, for reasons I don’t feel like explaining right now – but there are several and you stumbled upon one of them: non-ideal editor support. Another is that mix xref gets confused with umbrellas (and it’s a very good idea to be able to analyze compilation graph at one point, if compilation becomes too slow).

Even with a small team, having 6-12 repositories / projects is very doable and I’ve seen it work well.

Yea I think I’m just going to start with a single project to just get the DB stuff with Ash and AshPostgres working with some of the calculations and aggregates we might need with a simple admin interface.

1 Like

In my experience, this is extremely difficult to do successfully. It doesn’t matter how sound your reasoning is, people are wedded to their tech stack, and if they do move, they want it to be something everyone else is using.

If you want to work with Elixir, a much better strategy is to start your own project, or join someone else’s where it is already being used. Then you are guaranteed to be using it and don’t have to prep yourself for a futile argument by making forum threads :wink:

2 Likes

Well fortunately we’re not really married to our tech stack. It arguably gets in our way at this point more than anything. It’s a small team and even the senior develop had mentioned that we could move away from Java given how we use our data.