Does my frustration with Node merit switching to Elixir?

[lorenzo]

Hey everone!

I created a prototype for my app using Nodejs for the api. But the framework I chose wasnt great (in general theresnt any great node framework) and I find I’m wasting a lot of time setting everything up myself: an orm, testing, seeding, typescript, dependency injection etc. I have a basic setup going but it feels brittle. I really don’t know how people can be productive with nodejs. And it’s even worse since I chose to use graphql and most frameworks/docs/tutorials tailor towards REST.

My question is, is it worth switching to Elixir despite having more experience with Javascript/Node? They’re both performant, so my main concern I guess would be productivity. But part of me is thinking “the grass is always greener.” With Elixir I’d have to learn an entire new language and framework. But part of me feels that I’d be easier to learn well since everything fits nicely together.

Other factors:

• Apollo graphql has done wonders for node.js. It makes a huge framework less necessary, but you still have to figure out how to integrate it with your setup. While absinthe seems to fit nicely with Elixir/Phoenix out of the box.
• It’s nice to have everything in one monorepo and one language (Javadcript plus typescript). But part of me feels that trying to keep this is causing more headaches than its solving.
• Deployment with Node.js is a breeze - all it takes is a single command to deploy to zeit’s now. Whereas from what I’ve read deploying Elixir is not as fun.
• The node.js ecosystem is so fragmented that it’s hard to learn. There isn’t really any great books that guide you through the process since theres so many solutions for everything. Whereas with Elixir, I feel that the path would be more straight-forward. The question is time. I’ve already taken way too much time getting my app going and I feel like I’m running in circles at this point.

Anyway, I’d love to hear any advice or other considerations you may have on this whole backend fiasco, especially if any of you recently switched from Node.js as well.

Thanks!

[OvermindDL1]

@lorenzo

and the fact that I’m using graphql

Well hey, the Absinthe Elixir library has you covered there!

@lorenzo

My question is, is it worth switching to Elixir despite having more experience with Javascript/Node? They’re both performant, so my main concern I guess would be productivity. But part of me is thinking “the grass is always greener.” With Elixir I’d have to learn an entire new language and framework. But part of me feels that I’d be easier to learn well since everything fits nicely together.

If you use any of the immutable functional libraries in javascript then you’ve got the ideas of Elixir already down, if not then that will mean learning a bit new of a paradigm.

Elixir is worth learning due to the entire BEAM VM it runs on though, but honestly if your current system runs well and can be maintained without much issue I’d keep it. But if you want to learn a new way, or if you want to be able to scale well, or if you just likely want to shorten your code and improve maintainability then Elixir would be great to use. Just be aware of what you are doing, though porting an existing working project from something else to Elixir is definitely a good way to learn things.

@lorenzo

Apollo graphql has done wonders for node.js. It makes a huge framework less necessary, but you still have to figure out how to integrate it with your setup. While absinthe seems to fit nicely with Elixir/Phoenix out of the box.

Yep, Absinthe works with the Apollo client code just fine too!

@lorenzo

Deployment with Node.js is a breeze - all it takes is a single command to deploy to zeit’s now. Whereas from what I’ve read deploying Elixir is not as fun.

Never heard of zeit’s, but deploying my elixir server’s is just running a single shell script then bouncing the server via systemd (which I could easily automate too, but haven’t yet because eh… easy enough).

@lorenzo

The node.js ecosystem is so fragmented that it’s hard to learn. There isn’t really any great books that guide you through the process since theres so many solutions for everything. Whereas with Elixir, I feel that the path would be more straight-forward. The question is time. I’ve already taken way too much time getting my app going and I feel like I’m running in circles at this point.

Elixir is exceptionally well documented and has a fantastic community for sure!

But yeah, if your current thing works and you see it doing fine as it is in the future then I’d say just keep at it, only port it over if you want to learn. However if you see pain with it in the future then it may be worth porting to Elixir sooner.

[AstonJ]

I think many people here will say, yes

If you can spare the cash, I would recommend you get Elixir for Programmers (PragDave) though not to learn Elixir, but to get an insight into why working with Elixir and Phoenix is one of the most exciting ways to develop modern software …Elixir makes so many of the current best practices in development incredibly easy, making it a very natural fit it will make the hairs on the back of your neck stand up

[easco]

@OvermindDL1

Apollo graphql has done wonders for node.js. It makes a huge framework less necessary, but you still have to figure out how to integrate it with your setup. While absinthe seems to fit nicely with Elixir/Phoenix out of the box.

Yep, Absinthe works with the Apollo client code just fine too!

The one caveat to this is GraphQL subscriptions. The solution that the Absinthe community provides for subscriptions relies on Phoenix Channels as the transport mechanism between a Phoenix server and the client. That means that the GraphQL client must work with the Phoenix Channels transport mechanism and an out-of-the-box Apollo client will use the websocket protocol of the Apollo Server by default.

The Apollo JavaScript client is flexible though, and the Absinthe community has used that flexibility to create a JavaScript Apollo client that handles subscriptions through Phoenix channels (see GitHub - absinthe-graphql/absinthe-socket: Core JavaScript support for Absinthe WS-based operations). If you require subscriptions, and want to use the Apollo clients for other platforms (most notably the iOS and Android Apollo clients) they may require additional modification to tie into the transport that Absinthe Phoenix provides by default.

[seanc]

@lorenzo

Deployment with Node.js is a breeze - all it takes is a single command to deploy to zeit’s now. Whereas from what I’ve read deploying Elixir is not as fun.

I came to Elixir a little more than a year ago from JavaScript and I still share your thoughts here. I spent 30 hours one weekend trying to just research deploying an Elixir/Phoenix application. I’ve asked once on Slack for tips and wasn’t really helped other than a few things like “just do [thing JS devs never deal with]” and it was hard to grok since JS is just so easy to deploy pretty much anywhere. I’ve read a lot of posts here and in every conference talk whenever they mention JS they always make it a joke so I think a lot of Elixir devs just look down on JS in general and haven’t had the pleasure of deploying it

Ideally I would have liked to keep asking more questions, but I feel like I waste everyone’s time when I ask how to figure out this deployment stuff. I ended setting up an ECS instance for an EC2 cluster, then added an RDS Postgres instance, set up all the security groups and stuff, and put it all behind an Elastic Load Balancer which came with all of its configuration. If anything breaks I’m screwed because I have no idea how it works or how I got it to work in the first place lol.

Anyways, I really hope you find the answer to the deployment story. I’m still looking.

[lorenzo]

Hmm I wonder how this factors in with cross platform frameworks - e.g. can I still use it for react native or vue-nativescript?

[easco]

@lorenzo

Hmm I wonder how this factors in with cross platform frameworks - e.g. can I still use it for react native or vue-nativescript?

I’m afraid I do not have enough experience with either environment to say. I won’t let that stop me from speculating though. Based on what I do know, the data transport mechanism is generally a very small part of the overall system. Once the data has been squeezed from the server to the client through whatever pipe is used (Apollo’s socket protocol, Phoenix’s channel protocol… whatever) then feeding that data into the top of the React (or presumably Vue) data flow should be fairly straightforward.

[OvermindDL1]

@easco

The one caveat to this is GraphQL subscriptions. The solution that the Absinthe community provides for subscriptions relies on Phoenix Channels as the transport mechanism between a Phoenix server and the client. That means that the GraphQL client must work with the Phoenix Channels transport mechanism and an out-of-the-box Apollo client will use the websocket protocol of the Apollo Server by default.

I thought there was an apollo addon library that adds that though?

/me mostly uses graphql on the back-end for as much sense as that makes…

@easco

The Apollo JavaScript client is flexible though, and the Absinthe community has used that flexibility to create a JavaScript Apollo client that handles subscriptions through Phoenix channels (see GitHub - absinthe-graphql/absinthe-socket: Core JavaScript support for Absinthe WS-based operations ). If you require subscriptions, and want to use the Apollo clients for other platforms (most notably the iOS and Android Apollo clients) they may require additional modification to tie into the transport that Absinthe Phoenix provides by default.

Ah yep that’s it! ^.^

@seanc

Anyways, I really hope you find the answer to the deployment story. I’m still looking.

It’s always seemed easy for me? Just release, bind it to systemd, and let it go. Building a new release is just release, copy into the systemd accessed location, and tell systemd to restart it. It’s the same as any other unix style server?

/me is thinking they should make a blog post about systemd deployments sometime…

[zimt28]

I feel the same about deployments. Even though I’ve done it successfully, I don’t think it was that simple. You might want to look into https://nanobox.io/ – they offer a great and simple solution for deploying your apps.

Alternatively have a look at https://gigalixir.com/ or search our #deployment tag.

@OvermindDL1 Please write that post

[andre1sk]

I think it would be worth mentioning what exactly are you building? Node stuff is “easy” as long as you don’t have to deal with having to offload work to other node processes to keep event loop from blocking. The deployment story is easier for Node but deployment is a kind of thing you setup hopefully once vs development that you have to do every day. As you mentioned Elixir ecosystem is much less fragmented and it is very easy to get going vs Node. The most refreshing thing to me is the ability to write sync code and block in Elixir so it’s much easier to reason about what is going on vs JS/TS. Async/await might look nice but you still better understand what is really happening behind that syntactic sugar. Also on large node projects arch. tends to get vary hairy I am working on large enterprise node app if it was done with Elixir it would literally be 1/4 the size and much easier to understand.

[supernova32]

Regarding deployments, Gigalixir and Heroku are great if you don’t care about the underlying platform. You just push your code and they take care of the rest. (This is pretty much what Zeit Now is, so it is just as easy to deploy Elixir to a hosted service)

In my opinion, doing it yourself is far more satisfying, and allows you to take advantage of the full set of Elixir/Erlang features that come into play once your code is running, like hot code reloading for zero downtime upgrades.

For me the best way to deploy Elixir would be to setup a Digital Ocean droplet, install Elixir, PostgreSQL, and anything else you might need. Then setup edeliver to push your code to the server and run it.

You could also just use Docker to build an image of your project, and Docker Compose to orchestrate the services needed and where to deploy, or build a release manually and use scp to copy it over to the server (provided your local machine and the target machine share the same environment).

It all might sound complicated, but a lot of the steps mentioned above would also have to be performed for Node as well, if you are not deploying to a ready-made PaaS.

[madasebrof]

Re: deployment.

+1 for https://nanobox.io (deploying to Digital Ocean)

We are using for it staging & production deployment, works AMAZINGLY well. (Large Elixir/Phoenix/Elm app/site). Highly recommend.

And yes, switch to Elixir/Phoenix (and Elm!). You’ll be super-psyched.

[ranyefet]

I also came to Elixir from Node.js and I shared the same issues that you mentioned with Node.
Fragile deps, manual wiring all the different libs.

In Node.js its really easy to get started, but is harder to maintain the system in the long run.
There’s is a learning curve, but once you it “clicks” you will never want to go back

If you’re just getting started, I suggest you make your Elixir app stateless (just like in Node.js), don’t use ETS or try to hot-code reload, keep all your state in external DB (Postgres, redis) it will make your life easier when trying to deploy your app.

Once you feel comfortable, you can try the cool features of Elixir

[aaron-price]

TLDR: My biggest regret was not switching to elixir sooner. I suggest following the udemy course by Stephen Grider. It’s a bit old, but it should give you enough to decide whether you want to make the leap or not.

I also came from node (nextjs + redux + mongodb + heroku) for many of the same reasons.
My current project is Elixir + phoenix + react + redux + postgres (for users) + arangodb (for graph data) + graphql inside arangodb. I don’t bother with absinthe, just send a POST request with a string.

For my own project I wanted something really performant, and the js crowd seems to think that js is performant because it’s async. But dealing with async was a huge pain my rear. Especially when you start using things like mongoose, where it uses it’s own promise library, but parts of it you can do with async/await (promises), but there’s the occasional undocumented thing in mongoose that ONLY works with old school callbacks… That lead to a great deal of frustration.

Elixir is more performant. WAY more. And its async system (processes) is WAY easier to reason about. Processes are also truly asyncronous, while the js event loop is technically just a highly optimized, yet synchronous system.

Then you have immutability. I remember the time I spent days trying to figure out why my array was changing by itself. Oh, you used an Array.prototype method on a different array? Too bad!

That is not an issue in elixir.

Or how about setting things up. Expressjs is awesome because it’s so lightweight and lets you roll your own stack… but eventually I found that to be its greatest weakness. There’s several different ways to do anything in node. When you look for help, you often have to explain your whole setup, and pray that someone had the same setup with the same problem.

Elixir comes with testing. I find it better than jest, jasmine, or mocha.
Elixir has a rails-like framework (phoenix). Built in CSRF validation and sessions, ORM, and an intuitive organization of code. Of course if you really want to roll your own, that’s still easy enough.

[aamerabbas]

Hi, I switched from Node to Elixir. I am generally happy with the switch, but it really just depends on what your needs and expectations are. I personally really like the coherence of the community and the batteries-included approach of Phoenix. The community has basically taken the opinionated stance of Rails, but fixed most of the shortcomings.

Anyway, I’m sorry people are not answering your questions seriously. We all have to start somewhere and it’s very frustrating when people trivialize these challenges. I went through some of these struggles myself. Let me tell you my setup, and I hope it helps you out.

I am essentially doing the following for my Phoenix application:

• I build my Elixir code using the “mix release” task
• I get all the dependencies and package them up in a container
• I deploy using Kubernetes (in my case, on Google Cloud)
• I use BitBucket pipelines to automate the testing and deployment process.

My Dockerfile looks something like this:

FROM elixir:1.7.1-alpine
ARG APP_NAME=my_app
ARG PHOENIX_SUBDIR=.
ENV MIX_ENV=prod REPLACE_OS_VARS=true TERM=xterm
WORKDIR /opt/app
RUN apk update \
    && apk --no-cache --update add nodejs nodejs-npm make g++ python git imagemagick \
    && mix local.rebar --force \
    && mix local.hex --force
COPY . .
RUN mix do deps.get, deps.compile, compile
RUN cd ${PHOENIX_SUBDIR}/assets \
    && npm install \
    && ./node_modules/brunch/bin/brunch build -p \
    && cd .. \
    && mix phx.digest
RUN mix release --env=prod --verbose \
    && mv _build/prod/rel/${APP_NAME} /opt/release
FROM alpine:latest
RUN apk update && apk --no-cache --update add bash openssl-dev imagemagick
ENV PORT=8080 MIX_ENV=prod REPLACE_OS_VARS=true
WORKDIR /opt/app
EXPOSE ${PORT}
COPY --from=0 /opt/release .
CMD ["/opt/app/bin/my_app", "migrate_seed_and_start"]

My bitbucket pipeline config looks something like this

image: elixir:1.7.1-alpine

pipelines:
  default:
    - step:
        name: Test
        script:
          - apk update
          - apk --no-cache --update add coreutils nodejs nodejs-npm make g++ python git imagemagick bash curl tar
          - MIX_ENV=test mix local.rebar --force
          - MIX_ENV=test mix local.hex --force
          - MIX_ENV=test mix deps.get
          - MIX_ENV=test mix deps.compile
          - MIX_ENV=test mix ecto.create
          - MIX_ENV=test mix ecto.migrate
          - mix test
        services:
          - database
    - step:
        name: Build and Deploy
        trigger: manual
        script:
          - apk update
          - apk --no-cache --update add coreutils nodejs nodejs-npm make g++ python git imagemagick bash curl tar
          # Installing gcloud
          - curl -o $HOME/google-cloud-sdk.tar.gz https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-207.0.0-linux-x86_64.tar.gz
          - tar -xvf $HOME/google-cloud-sdk.tar.gz -C $HOME/
          - $HOME/google-cloud-sdk/install.sh
          - source $HOME/google-cloud-sdk/path.bash.inc
          - export PATH="$HOME/google-cloud-sdk/bin:$PATH"
          - gcloud -v -q
          # Installing kubectl
          - gcloud components install kubectl -q
          # Authentication
          - echo $GCLOUD_API_KEYFILE | base64 --decode --ignore-garbage > ./gcloud-api-key.json
          - gcloud auth activate-service-account --key-file gcloud-api-key.json
          - gcloud config set project my_app
          - gcloud container clusters get-credentials my_app --zone=europe-west1-d
          - kubectl version
          # Deploy
          - source $BITBUCKET_CLONE_DIR/build.sh $BITBUCKET_BUILD_NUMBER
          - source $BITBUCKET_CLONE_DIR/deploy.sh $BITBUCKET_BUILD_NUMBER
        services:
          - docker
              
definitions:
  services:
    database:
      image: postgres:9.6
      environment:
        POSTGRES_PASSWORD: secret
    docker:
      memory: 2048

Also, you will notice a build.sh and deploy.sh file referenced here. These are small simple scripts I made. They look like this:

Build.sh

VERSION="$1";

if [ -z "$VERSION" ]
then
  echo "Must specify a version to tag the docker container";
  exit;
fi

docker build --no-cache -t "eu.gcr.io/my_app/my_app:$VERSION" . && \
gcloud docker -- push "eu.gcr.io/my_app/my_app:$VERSION"

Deploy.sh

#!/bin/bash

VERSION="$1";

if [ -z "$VERSION" ]
then
  echo "Must specify a version to tag the docker container";
  exit;
fi

kubectl set image deployment/my_app my_app=eu.gcr.io/my_app/my_app:$VERSION

[alamba78]

Has anyone ever gone from Elixir to Node? That’s the interesting question. LOL.

[akoutmos]

Haha. I had to make the switch for my day job as the tech stack does not include Elixir :’(.

The things you take for granted in Elixir are serious concerns in the Node world. The obvious example being hogging the event loop. Also, while not limited to Node, immutability is something I greatly miss especially in a team environment. You never really know if an object will change out from under you if you pass it around to other functions. If I had to choose between “developer diligence” and “baked in”, I would chose baked in.