Using lightweight libraries/tools instead of Phoenix

Hello!

I’ve been an avid Phoenix user for years now (let’s say ~4-5 years?). Have multiple installations running in production etc. I also have Ruby on Rails experience a way back (from Rails 4 already if I remember correctly up to Rails 6). Total professional programming experience is ~15 years and has been mostly about web-apps.

Usually when I use these frameworks which do all the things, I start to wonder at one point, how difficult would it be to create a web application using lower level tools instead of these heavy-weight champions. I’m happy to say that I find Phoenix much more lightweight compared to Rails, however it still gets once in a while in my way with weird problems, which are harder to grasp due to the overhead and creates additional work with its backwards-incompatible version releases if there is any urge to upgrade. In Ruby ecosystem I would use Sinatra (or something similar) with some templating engine and lower-level database-connection library to build a web application if I would not want to use Rails.

I’ve never done something similar in Elixir world yet, but started to think what does Phoenix offer me in the first place. As I understand then one big thing about Phoenix is LiveView, but I’ve never found a use for it since I like more “boring technologies” meaning server-side template rendering and only a little bit of JavaScript here and there where more dynamic functionality is needed (surprise, surprise - most of the time it is not needed). And of course Phoenix-like framework offers conventions of where to put your code, which is really good if you’re just starting coding career.

I started to ponder that thought a little bit more and found out that most of the things I thought being part of Phoenix is just part of Elixir itself (thing Config and Releases for example).

After some initial research I found out that I could use Plug with Cowboy, Config, Releases and EEx together with Ecto and its migrations. I would also like not to use Ecto and use something where I can write plain old SQL, but have not found anything to my liking yet in Elixir ecosystem.

Any thoughts or experience about going this road or any strong opinions against doing it? What does Phoenix offer in your mind, which is not easy to do when not using Phoenix? Any good light-weight library suggestions (especially about database related)?

Don’t get me wrong - Phoenix is a fantastic framework, which has managed to create a fork of RoR in a good way. Usually these forks fail miserably in other ecosystems/programming languages because for some reason good parts of RoR is not copied and many bad parts are copied instead :slight_smile: Thank you for Phoenix!

Jarmo

8 Likes

You should be able to get something quickly up and running using Plug (Plug.Router in particular) and Phoenix.Template+Phoenix.HTML (if you are doing HTML stuff). If you are doing APIs, then you will need Jason and a handful of helpers such as json/2, which receives the connection, data, and renders it as JSON. For queries, you can use Postgrex directly.

Thank you @josevalim for quick reply.

Why would you recommend using Phoenix.Template+Phoenix.HTML instead of plain old EEx? Can’t I just go with EEx for some reason?

And for Postgrex it seemed to me at first that it’s not that easy to use transactions and pooling (need to use ecto-dbconnection for that). Do you have any good guides for that? Also database migrations seem to require ecto-migration.

If you just want to run plain SQL you can use your database driver directly - for example, when you use PostgreSQL, Ecto uses postgrex under the hood. You could also try Erlang’d ODBC, I haven’t used it personally yet though :slight_smile:

As for light-weight library suggestions for the HTTP server itself, I’ve recently had good luck when building a simple mnesia benchmarking tool with using Erlang’s built-in HTTP server with mod_esi. You basically need to start :httpd with a config that tells it where to find your Erlang Script Interface (ESI) modules (Erlang Syntax):

    ServiceConfig = [
        {port, 4000},
        {bind_address, {127, 0, 0, 1}},
        {server_name, "mnesia_bench"},
        {document_root, filename:join(Here, "public")},
        {directory_index, ["index.html"]},
        {mime_types, [{"html", "text/html"}, 
                      {"css", "text/css"},
                      {"js", "application/javascript"}]},
        {server_root, "/tmp"},
        {error_log, "error.log"},
        {erl_script_alias, {"/esi", [mnesia_bench_esi]}}
    ],
    {ok, _Pid} = inets:start(httpd, ServiceConfig),

then if the mnesia_bench_esi module exports a function create_table/3, you can call it from your browser at the URL /esi/mnesia_bench_esi:create_table. I found this pretty simple to quickly build something with, and :httpd is pretty configurable, but it won’t compare with Phoenix of course :grin:

You can, however template engine used by phoenix first of all is html aware and secondly it generates safe escaped html so you don’t have things like XSS attacks.

5 Likes

Yeah it’s this bit specifically that makes EEx on its own a non starter for any app that will display user data.

An alternative over “raw” postgrex is to still use ecto, but only set up a repo and do migrations with it. You can do Repo.query or Repo.all(from u in "users", ...) and skip all the schema / changeset stuff if you want. This gets you all of the valuable things ecto does from a safety / migration / core ergonomics standpoint.

3 Likes

Thanks for the tip, already checked Phoenix.HTML code and it should be really trivial to take only relevant parts from there. For example form/tag builders are not needed when using “boring technologies”.

This is maybe too low-level already, but good thoughts anyway. I’d prefer Plug since it also offers Plug.Test and other nice things while still being quite low-level, but not too low.

1 Like

IMO about 80% of the developer effort spent on “keeping things lightweight” is waste; there was a huge fad back in the day of people going to incredible lengths to prove “they didn’t NEED a framework” that led to a lot of hard-to-maintain messes. I’ve worked on at least one $BIGCO codebase where they started by embedding Sinatra “to make an efficient API” then welded every other Rails feature back on.

IMO it’s good for everybody, because people farther along in their coding career should probably be thinking about “how to solve the customer’s problems” and less about “how to shuffle around some files”.

11 Likes

I’m only weighing in because you say you like “boring technologies” and the whole reason I use LiveView is because I consider it a “boring technology”. It gets out of my way more than any other framework I’ve used and is quite a bit simpler than controller+view. There are also fewer things to learn than in other frameworks. If you’re used to building apps with “sprinkles of JavaScript” it even abstracts away most of those sprinkles (what’s less than a sprinkle? lol). Also, LiveView templates are server-rendered. I would at least give it a chance!

1 Like

You’ll end up reinventing a smaller buggier Phoenix IMO.

I too think Phoenix can be too heavyweight on management – several different files in the right places, magic configurations with several levels of lists and tuples, code generators that historically have ended up producing outdated code you have to manually upgrade, and others I can’t remember right now – so I’ve been in projects where we’ve only used Plug for routing. However, these projects were serving JSON and had zero visual parts.

As much as I feel working with Phoenix is boring and annoying, it’s probably one of the best things we have on the planet in the web area.

6 Likes

And then another developers has to work at the code base and:

  • learn your new (likely less) documented custom conventions
  • can’t find answers at ElixirForum and Stackoverflow
  • can’t ask “Phoenix questions” but has to resort to “we’re using a combination of Plug and X”

Have seen companies go ‘bare bone’ and instead they just rebuild a less documented, less dev friendly, less secure, less optimized ‘fat bone’ over time.

There might be very rare cases where it is beneficial but in general it’s just not worth it.

7 Likes

It’s more a Want To upgrade, not a Have To upgrade. Isn’t it? Compared with the JS ecosystem, keeping up to date with Elixir libs is a breeze :slight_smile:

several different files in the right places

This one is annoying me a bit more indeed. I really like the decoupling of name spaces and file location (let the editor handle jumping around) and Phoenix is quite relaxed about it. But once in a while it DOES matter and then you trip over it.

2 Likes

Don’t forget about some nasty macros (I think all of us are guilty of creating them). But in general I totally agree, when I saw for first time proxies ( that of course became full fledged servers after a short period) in golang with some custom decoders, converters and middleware that never were abstracted in a separate layer, I just decided this is not worth investing my time, just nasty practices that cost tens of thousands of dollars.

Oh absolutely. It’s my personal and professional weakness that I get OCD when I know there’s a new version of something. I intuitively fear I am missing out which I’ll admit had led to not making strictly informed decisions before inflicting pain on my personal – and some professional – projects. What can I say. Still haven’t cured that defect of mine.

Also no need to compare with JS, we all know the horror show of programming. :grimacing:

Yeah that’s exactly it. You never truly exercise these things on the level to put them in long-term memory and they surprise you EVERY DAMN TIME. I hate it.

But I’ll say again – Phoenix is one of the best of the best web frameworks in the world. I am kind of piggybacking on this thread to just share quickly that I don’t have much interest in working with it. I know it well and whatever I managed to forget I remind myself of it in literal minutes but… I never enjoy working with Phoenix nowadays.

1 Like

If we are talking about education, personal projects or the likes it really doesn’t matter.
For anything else where more than a person needs to collaborate i agree with the previous replies.
I’ve seen my fair share of Express Frankensteins and Flask project’s than ended up being a worst Django by a large mile but without any proper docs.

Now, about lightweight, Phoenix works fine in one file for simple use cases.

3 Likes

I agree and I’ve seen ungodly Golang messes as well but there’s an upside to it: they are very rarely modified and are self-contained which is a boon for maintainability – if you are the original dev.

But yeah, if the original dev leaves then ouch, it’s bad for everyone else after.

2 Likes

The reality is that it doesn’t matter who left the turd, you come and usually as it happens the original dev is long gone and you have to make that mess work, without any documentation or guidance, you are lucky if that person used some logic behind his code structure, which is rarely the case for low paying dev positions as there are in my country.

One thing that set Phoenix apart from other frameworks is that it does not force you to adopt everything it offers. In one solo project I have, I use neither Phoenix.Template nor Phoenix.HTML, or any javascript from Phoenix for that matter. However, I still use Phoenix. The functionalities that I do not use don’t bother me; they are hardly noticeable.

Wow, this thread has gotten more traction than I initially hoped fo. Some good thoughts in here. Trying to reply to some of them.

Yes, this could happen. I’d argue that this only tends to happen if devs involved do not understand or agree with the good and bad parts of the frameworks used in the past and try to mimick them (also true for Rails/Phoenix forks). Not everything is good in every framework. It’s of course a subjective matter. Unfortunately it requires time and experience of multiple programming languages, their ecosystems and frameworks to have an objective opinion about good and bad parts and this is not something all devs (yet) have, thus it’s easy to get lost and build every other Rails feature back on, when in reality it’s not really needed.

When I started with Ruby and Rails 4 in my coding career, I came from a bloated Java EE world, I found Ruby and Rails to be like a fresh breath of air. Everything looked so clean and elegant. This was until some problem came along and I needed to troubleshoot it by going through all of the complex layers behind the curtains of Ruby and Rails magic. This made me understand that I liked magic at first because as long as you follow the road correctly then everything works as expected, but if you have a small misstep, then good luck understanding what’s wrong. You just can’t see this in the code because there’s so much magic going on and not that many lines of code. Rails became to me as one of these fancy frameworks which sells itself with the statement that developers can concentrate on solving customer’s business problem, but this is only true at the beginning of a project. At one point you need to start dealing with all of the extra-weight brought in with the choice of the framework devs took a long time ago. And then you need to upgrade it because you want to have latest and greatest features and run it on newest programming language runtime/compiler. A separate “upgrade framework” tickets will be created into project planning software and no-one has any ideas how long it will exactly take due to many unknowns and vigorous testing (not just unit-testing, since there might be production-only problems coming up).

I would disagree that LiveView is a boring technology. It looks like that at first, but i’d consider it still cool/fancy new technology compared to the old and boring ones. Instead of having a HTTP+HTML (and maybe a few JavaScript files) you end up with HTTP+WebSockets+proprietary protocol built on top of it+specifically written code for LiveView at server-side and frontend+JavaScript running in the frontend to patch DOM. There’s many layers which could go wrong and might be hard to debug when this happens as opposed to plain old HTTP+HTML where you can see request and response and understand what was or was not sent back to the browser.

I guess I might have not communicated clearly enough. My idea was never to build something from scratch, but instead of having a heavy-weight framework like Phoenix as a project dependency, I would use smaller and only necessary dependencies as a building blocks. This doesn’t mean that I need to write everything from scratch, but gives me freedom to change/update/test these parts separately if keeping code neatly designed. Also, less bugs and security problems not more due to having less complexity and ways to go wrong.

Agree, there’s nothing better (you who love SPA frameworks and their endless complexity, go back to drawing boards :stuck_out_tongue:). However, as you also stated then there’s some annoyances and problems of keeping it up to date. Someone wrote here that there’s no need to upgrade, but this is also not true since by not upgrading Phoenix you can’t upgrade Elixir at one point to get faster runtime/compilation etc. To upgrade Elixir, you might need to upgrade project depenendencies and by having smaller/less dependencies, it’s definitely easier/less error-prone and takes less time plus it’s much safer to deploy to production since rolling back is going to be much easier too.

This again tends to happen when less experienced developers try to do it without following good coding practices and want to create clever code full of magic. I, however prefer explicit code over implicit code at any time and would not include any magic whenever possible. Code is meant for reading most of the time not writing so having all of these extra characters there will help at one point. It’s one of the reasons I even started to look at Elixir and Phoenix at one point since it felt that there’s not that much of magic going on. I love Elixir compared to Ruby because it is quite explicit as long as macros are not used. In Ruby you never know what the value of self is in runtime. It’s even much worse compared to JavaScript this. Comparing Phoenix with Rails is a good example where Phoenix took mostly good parts from Rails by ignoring bad parts and one of the things is having less magic involved. There’s still some, but still a lot less in my opinion.

Again, happens when less experienced devs try to be smart without having a full understanding where the boundaries should be kept etc. I’ve done few projects in Java/Kotlin where similar approach was used - instead of going for de facto full-of-magic Spring Boot, which is used in Java world our team decided to spend more time at the beginning of project by creating a good foundation by using more simple Spark and/or ktor respectively (don’t recommend it, since it has some nasty bugs). Code was really explicit, easy to understand, maintain, test and easy to write due to help of the type system. Also, boundaries were in a way that it would have been quite easy to replace whatever framework was used since most of the code was plain-old Java/Kotlin without any external dependencies and dependencies were kept in a separate layers by wrapping them away. No documentation really needed since you can open up a router and follow from there downwards. No magic annotations or file locations required just a few familiar concepts if dev has any previous experience with MVC frameworks. I’ve also created something similar with Ruby + Sinatra, but it was more like a hobby project. With Phoenix I have few instances running in production where asset pipeline has been stripped and no LiveViews are used. Works awesomely.

It’s lightweight, but definitely not something I had in mind. I meant by lightweight of having fewer dependencies and less framework-specific code in my business logic layers. Lightweight still means that code should be loosely coupled and have a good design principles. Throwing everything into one file writing as few characters as possible is definitely not it :smiley:

For these cases it doesn’t really matter if a Phoenix-like framework was used or not. That kind of devs think that using a popular framework solves all their code-base problems and if they still happen to end up with a really smelly code-base then they blame the framework and go with the next best thing using the same undisciplined coding practices. Ironically it’s good in a way that this kind of devs are unable to write much on their own, which means that conventions created by a framework might help a little when it comes to understanding as long as some of them has been followed.

Yes, agree. I also didn’t use LiveView (as explained above), nor it’s asset pipeline, form/tag builders, code generators and few other things.

Just to conclude: for this particular case I have a feeling that there’s an easier way compared to Phoenix (why have any functionality in your code-base when it’s not used?), but it requires some discipline and experience, thus this road is definitely not for everyone. For this particular case I’m trying to experiment for my own solo project. I would not use this in production for a customer yet. However, after this experiment I will definitely have a better understanding of the lower level libraries/components already used by Phoenix, which in turn help me troubleshoot problems with Phoenix in a more efficient way. And maybe in my next customer project I will choose this path if experiment is successful :slight_smile:

Thank you for all the thoughts and replies so far!

1 Like