We’re a small team in large company that for acquisition reasons uses Elixir instead of dotnet/nodejs like the rest of the company. Elixir is currently considered an unknown/risk in terms of performance and stability (despite having very respectable load testing numbers for our service).
How would you market Elixir as a solution for a high traffic service? (references to large companies that use it, benchmarks and conversion stories would be very appreciated)
Their concerns
unknown quantity
unproven tech (this seems patently unfair but need some material to back up Elixir’s credentials)
hard to hire (though they accept that cross training Ruby of functional developers wouldn’t be so hard)
Some of the big companies would be Pinterest, Discord, Whatsapp that have adopted Elixir / Erlang. Discord is quite nice to blog about it as well.
As for the hard to hire part, feel free to mention UCLL (college) @ Belgium. Starting from this academic year, we yearly provide junior devs with a fundamental knowledge of Elixir (distributed) and Phoenix.
I’d say: it cannot be such a huge problem since you acquired us
But really, as with everything you have to sell: What’s in for the company?
A few questions you could ask yourself:
What would be the costs to migrate this service to another technology?
Are there things that are much harder to accomplish in another technology? (E. g. clustering of machines, or highly stateful/soft-realtime services)
How proficient is your team in the companies default tech stack? Is it worth the cost of retraining?
How would the company safe money by utilizing Elixir?
Are there complex monitoring systems in place that keep the backend running? (E. g. Kubernetes)
Is there downtime due to poorly managed Node.js servers? Does this have consequences for the cash-flow?
Are projects suffering from too many layers of indirection, which slows the system down, which in turn costs leads? 1 (This is, in my experience, mainly a problem of Java and .Net projects)
To the concerns:
I’m not sure what you mean by “unknown quantity”
Unproven tech is a straw man argument. As @WFransen pointed out: simply Discord or Whatsapp should be enough justification in that area.
In practice that is rarely a problem since the language is really easy to get started with: There is fantastic documentation available for almost every package out there. Much better docs than in nearly any other ecosystem I’ve used. Also the code is very easy to read, even for beginners. And last but not least: the community is, and I cannot stress this enough, awesome.
Having considered all this, please also consider another thing: why would you want to keep Elixir? Maybe the best business decision is, indeed, to rewrite your service inside the tech stack of the company in order to minimize friction and to keep the business running.
Because, as much as I love obsessing over tech, in the end we, as engineers, are there to solve problems. Not to create them.
I think what is important to clarify from the beginning - is the company giving the teams freedom to chose their own stack. Or it is preferred that there is one way to do things. Because if the freedom is not there, it doesn’t matter how good your stack is. You will lose.
If the freedom is there, still doesn’t mean you’ve won. The problem of hiring / maintenance / interop with existing stack / and a lot of other questions need to be addressed. Probably some politics will be involved as well, you have to find a higher-up decision maker that has you back.
It will be a long battle
EDIT:
Also, in my opinion it doesn’t make any difference to persuade .net and nodejs devs. 99% of them will not care and will see a new stack as a threat. That’s why a decision maker who can over-rule all of them is important.
You also never know when you’re even getting a fair shot to make a case. In many large companies you’ll see a central decision maker looking for uniformity regardless of any other trade offs.
it’s a two-way street, unfortunately. A developer that was open-minded enough to pick elixir in the first type is more likely to be the type that is also pragmatic enough to cave in to demands of more closed-minded developers.
We’re in the same boat at my Fortune 100. We’re currently using Elixir to handle jobs where the existing services (written with “standard” platforms) were getting crushed and/or had no possibility of realistically meeting the engineering needs.
I would suggest looking for things that your company would like to do but can’t do with their existing stacks. Marketing against “it’s impossible” is easier than marketing against “it’s safer”. Use the strangler pattern to entrench yourself, then try to expand.
Otherwise, Discord and Whatsapp make great mission critical, high volume business cases.
This article had some nice success stories as well.
Bleacher Report in particular is an interesting case, because it is not necessarily a use case where another stack couldn’t do, but Elixir still turned out to be a huge boost acroos all of those dimensions: productivity (less code, less tech debt), reliability (the spikes in traffic went from devops nightmare to non-events), response times and infra costs.
(Unfortunately this link which was referenced a lot is now broken)
Being familiar running nodejs in production, its non-blocking I/O model is pretty powerful for I/O concurrency and performs pretty well, but once you have some kind of spike or issue that starves your event loop everything bursts into flames. From there it can be hard to pinpoint the root cause. I don’t have the same level of experience with Elixir yet, but from what I could gather the BEAM seems really good at isolating failure and offers more gradual degradation in performance under load. Less alerts and less downtime seem like very compelling reasons to pick Elixir over Node.
First of all if you wish to build a robust system that is scalable in a short amount of time. There is no other choice. I’ve seen our competitors left in the dust because they chose the common technologies like ‘dotnet’ and ‘nodejs’.
The argument that Elixir is hard to hire for is a false belief. Companies that value employee development and have strong leadership that knows Elixir well will not feel this way because they can train people in a very short amount of time (which is what we do).
Out of the developers we hire, we’ve found NodeJS / JS developers are generally of very low quality we interview 10 and find one that maybe ‘adequate’, even then we have to teach them a lot of things like design patterns, do’s / don’ts . Generally they do not know best practices because in the JS universe everyone does what they want, and everyone changes their mind very often, they spend a lot of time ‘refactoring’ code which has never shipped, and optimizes prematurely on a regular basis. They also don’t test things well.
If you want to waste time / money configuring things, over and over, and have developers ‘have fun’ discussing which package manager / build tool / framework to use, and refactor code that doesn’t ship / scale then hire JS / Node developers they will serve you well in that respect.
If you want to have developers focus on application / business logic and discuss sytem design and architecture and finding solutions to new problems then Elixir / Ruby / Phoenix / Rails developers are much much better.
When we hire people we look for people with solid principles in programming
They know / value ORMs
They know the value of frameworks like Ruby on Rails
They value system design / architecture
They write tests for their code
They don’t get attracted to fancy new tech
They are result driven
They are attracted to Convention over configuration
They know how to solve problems ‘simply’ (NodeJS devs don’t know how to do this)
They ship
Developers with these qualities will make great elixir developers because they will be able to pick things up very quickly.
As an Elixir developer who started working part time for a company/team that uses C# and .NET (with no knowledge of any other languages or tools) I think you’re in for a long battle.
After I started getting to grips with Windows and C# one of the other developers quipped
“Now you’re starting to get it. Now you can start building real business applications, not toys”
Unfortunately this myopic perspective seems entrenched and is a parody of “nobody got fired for choosing Microsoft”. I’ve just accepted it now.
I think that’s partly why, but the juice also needs to be REALLY worth the squeeze to drop your stack and move to something else, and have it be done by the masses because it’s so compelling.
It’s really tough to build something that’s so much better than something similar that it becomes a no brainer decision to switch, even if it means relearning a ton of new things.
Most web apps can be built quickly with any popular web framework today and it’ll be “good enough”, where good enough means you’ll be able to serve p95 requests in 100ms or less to hundreds of thousands of daily visitors on a single $20 / month server with a UI that feels nice to use without making a SPA. You also have container orchestration tools for scaling that works with every stack if you need it.
At the end of the day if every tech can do that then it becomes figuring out how to do this with the least amount of effort using a language that you enjoy using it.
The point I would emphasize is that BEAM processes are the secret sauce of Elixir / Erlang. Every web request, web socket connection, background job, database connection, or file handle will use a separate process, and you can have millions of them. Benefits include (from the first article):
Non-blocking IO : when a process needs to do input or output — like reading from or writing to a file or database or making a network request — that process can go to sleep while it waits for a reply, letting other processes get work done in the meantime.
Non-blocking computation : when one process gets stuck with a heavy workload, the Erlang scheduler ensures that it performs that work in bursts; between them, other processes get a chance to run. So a small number of expensive requests can’t bog down the whole system.
Error isolation : if an error occurs in one process, it can fail or be restarted without other processes being affected at all. If need be, other processes can be notified and react accordingly.
The Erlang Solutions case study on Bleacher Report has some great scaling figures for you in terms of load, response time, and resource usage. Other resources are linked in the first article.
I think that’s partly why, but the juice also needs to be REALLY worth the squeeze to drop your stack and move to something else.
The other problem is that Elixir’s main benefits (high velocity to code and near-zero maintenance burden) are VERY difficult to quantify. As a solo dev, I run a distributed cloud orchestration system in Elixir, and I spend about 1% (and decreasing) amount of time maintaining an (necessarily complex) codebase while shipping features. If I have to push a change/debug I don’t spend very much time at all scratching my head finding where the change needs to happen, even for code I haven’t touched in 6-7 months.
But how do you sell that concept? It might be even harder in big shops, where they are already used to and have personnel and dev infrastructure to handle high maintenance burdens.
Cmon seriously - the Erlang VM is 30 years old and Elixir is just a (very nice and fancy) preprocessor that generates Erlang bytecode. Unproven tech. Man I nearly fell of my chair!
To many programmers in a big shop a stack requiring a lot of personnel to run would be a feature of the stack. In my limited experience working with C# stack it seems MSFT rewrites the dotnet web server stack every few years.
