That is why I am working on libraries like systemd to provide better integration with systemd, so the Docker will not be needed (and even troublesome). I still need to write article about how to deploy Elixir application with systemd, as it could make the process clearer. Not only about running the application itself, but also it will provide tooling for simpler configuration management (as you can run script to generate sys.config before start of the application instead of relying on “hacks” in the core, like config/runtime.exs).
hauleth
9
dimitarvp
Because now we have several slightly differently working compile-time / boot-time / run-time configuration policies and it becomes a legacy support burden. When you get into a new project as a consultant / freelancer / employee you’ll have to take into account if they use distillery or releases and what Elixir version. A lot of project’s configs are very specific and people are understandably unwilling to change them because of language or library version upgrades. I faced this often and the result was an ugly pile of .exs files that nobody except people in the team could decipher and modify.
I am with you here and hence I adopted a policy to mainly use Elixir for glue and orchestration and iteration speed but reach for Rust when things need to be faster / better type-checked / venture into an area that’s a non-goal for the BEAM.
Not exactly, Rust and OCaml enforce exhaustive pattern matching.
4
dimitarvp
I applaud you for the good summary and the courage to post it. The linked GitHub gist is also an awesome read!
There are aspects where Elixir seems to not be able to have a strong opinion (configuration being a prominent example). And those papercuts indeed do build up.
Without this, any language is eventually doomed. Platforms like Render and Gigalixir have emerged to make this easier – but they only manage to herd you into their own garden where deployment is still a service-specific affair. I’d like one Deployfile that can be used for all deployments! How hard can it be? Maybe Dockerfile-s can be transparently translated into actual scripts for Docker-less deployments?
This and the rest of the paragraph is spot on. As a guy who was a noob (and still is) in many technologies, it’s really not helpful to have many ways of doing things and relying on people to discover for themselves. Some direction and “blessed” ways that must be very front-and-center-and-impossible-to-miss is a very much needed change! It’s one thing to play with a language, but a completely different story when you are working with it. Newbies need direction.
I’ve entertained the idea of making a website with brutally short snippets for common scenarios (sort of like StackOverflow but much more minimal and to the point, and without comments too) but life, work and health takes precedence and such a project might never come to fruition.
Yep. Good practice advices baked right into the language’s tooling I found to be an amazing way to train newcomers and experienced veterans alike.
I work with Rust more and more lately (it’s my new job) and I have to say that beyond OTP, brief syntax and fantastic libraries like Phoenix and Ecto, I don’t feel much pull back to Elixir. OTP is definitely unbeatable for the moment and that’s reason enough but I wonder how long will that last.
Apologies to everyone for dumping 3 posts in a row.
6
cpgo
I agree with this.
elixir-ls is awesome when it works (most of the time), but when it doesnt the experience is not very good.
Usualy deleting the build directories fix the issue, but still I feel we should give more love and support to the language server maintaners (thanks @JakeBecker, @axelson and the other contributors).
On a side note, its interesting to see that the Go team has officially stepped up as the new maintainer of the Go extension.
Im not saying that the elixir core team should maintain the vscode (or any) extensions, just pointing out that editor support is very important for the language.
10
derek-zhou
While I agree that the “blessed” ways should be promoted more, I like to have choices (I used to program in perl). For example I don’t like eex (or any template in general) and elixir/phoenix gives me the option to hand roll html in render/2
0
hauleth
My problem is that even the “blessed” approach seems a little bit “unnatural” for me. The main problem there is that there are some options that cannot be set via config/config.exs but I think (but I am not sure) that these options can be set via config/runtime.exs. And to be able to configure these options via config/runtime.exs the VM needs to be started twice during startup of the application… Additionally while config/*.exs files looks like code, these cannot contain all BEAM types (for example functions, I am not sure how about PIDs or refs) nor it can use dependencies, at least not all of them. This makes the configuration here quite confusing, in my humble opinion.
2
shanesveller
Several other people have alluded to this in the further replies, but hot upgrades do not come for free despite being readily available in the technical sense. You need to be very intentional in how you plan and execute such transitions, and you often need to write specific support code to make it successful, and these do not have much in the way of prior art for folks coming from other languages, so this is often a new set of skills and instincts to nurture. Even some of the strongest proponents of hot upgrades will readily admit that they don’t use it 100% of the time for every revision - sometimes they still just do a full restart of each BEAM node.
Most businesses’ uptime expectations are about the customer impact, not about individual runtime instances, and you can use bog-standard horizontal scaling and load balancers to achieve this in a language-agnostic way that still feels available to the user while you’re doing your shuffle behind the scenes. This skillset is more readily available in the broader industry with lots of prior art and tooling. Most products don’t actually have the strict tolerances that call for true hot upgrades as a strict necessity like a telecom would have in the original use-case for Erlang.
All of that is also just dealing with the BEAM implications, and you yourself acknowledged down-thread that a lot of the difficulty actually comes from how you manage your off-BEAM stateful database content during evolutions. That bit doesn’t really change one way or the other with hot upgrades and it’s the piece that most orgs struggle with IME, and I think it’s a more productive place to spend your time, energy, and educational efforts.
1
shanesveller
I think releases.exs has not moved the state of the art forward from what we already had with Distillery, but I am quite looking forward to runtime.exs when Elixir 1.11 ships in ~October. That will really drive the conversation forward and introduce the required learnings earlier in a project’s lifetime, which is all to the good. The version of releases introduced natively with Elixir 1.9 does a minimal subset of what Distillery provides, and some of the things that were omitted are things I still want for every project today. To be honest, I think most people switched to native releases not because Distillery’s documentation was hard or its features were overwrought, but because it has every appearance of being unmaintained. I hope that’s not true, but there’s been relatively little communication around it since Paul mostly left the Elixir ecosystem. I’m grateful to his incredible work on Distillery and libcluster, but it feels like we’re on shaky ground where the new solution is incomplete and the incumbent is stagnating.
Hot code upgrades have no analogue in other languages, and edeliver is basically a Capistrano clone, which in hindsight requires a sliver of Ansible to reimplement if you enjoy that deployment model. I emphatically don’t. These tools and techniques cannot be adapted to a second or third language at a polyglot organization, or at least I feel attempting to do so is not a good use of time. If you are at an org that allows that fluidity of language use, you have to optimize your deployment practices for the lowest common denominator and start communicating and working with a standardized process and vocabulary around deploys. There’s no space for bespoke single-language stuff, given my belief that hot upgrades are not actually necessary for delivering business value in most products/orgs.
I don’t have the goal of arriving at something that is like-for-like with the BEAM. If I valued all of its designs and principles that highly, I would stick with Elixir and be happy. I value languages that let me do meaningful work, do it well and comfortably, and avoid needing superfluous hardware to power it. I value code that is resilient to human and technical failure. The BEAM does not have a monopoly on those properties.
I think most of the rest of your post is actually supporting my own stance, highlighting that I value things and want to pursue areas of work that are implicitly or explicitly non-goals for Elixir. It’s okay for those things to be non-goals for Elixir, but it’s also okay for me to recognize that disconnect, and that the values driving the design decisions are not my own. That means I probably have to look elsewhere to avoid suffocating as a developer.
Totally happy with your points and your tone, thought it was all in the spirit of respectful challenge and I appreciate that. Lots of great links, too.
4
josevalim
Creator of Elixir
I would love to know what is missing because the only aspect of Distillery we haven’t brought over at this point is hot code upgrades, which you have mentioned multiple times you don’t care for.
That’s my impression from reading your post too. And, to be fair, it is not necessarily about those being non-goals for Elixir. Some of your concerns are really only addressed in the most popular languages (or the heavily backed ones), and even then, not all of them would check all of the boxes.
Edit: Comments on the gist.
if a module is loaded in your copy of the BEAM you can call it day long, even if you shouldn’t be.
Improved in Elixir v1.11 which will require those to be at least listed in your application dependencies. This will address issues such as poor boundaries between (umbrella) applications. It is a direction we are actively exploring.
There’s a lot of use-cases it’s frankly just plain unsuitable for. CLIs, game development, and machine learning applications are some common examples.
To be honest, very few languages would be suitable for CLIs, game development, and machine learning, in addition to the areas Elixir already provides a good foundation for, such as data ingestion, web apps, embedded systems, and distributed systems.
Even if we consider your three leading entries, I am struggling to come up with a language that currently excels on all three.
Node and Java beat us handily in a lot of scenarioso, and anything in the C- family or Rust can do much more and with way less resources required.
Agreed on C, Rust, and Java, but I can’t think of where Node would handily beat us and we are generally being used at.
10
shanesveller
Thanks for your attention here!
Looking at the latest documentation from the Mix hexdocs, I think the only outstanding bits are things that you could have in spirit with native releases, just not in identical forms. The configuration providers have excellent parity already, and the tarball support came not too much later.
- I get a lot of mileage out of custom subcommands over
evalandrpccalls, to reduce cognitive overhead during operational or triage tasks. This is also a place where documentation and runbooks could supplement about as well. - I’ve seen them used for significant evil but book hooks are a nice model sometimes. I expect that the guideline for this is to use configuration providers as much as possible, and for anything else just roll them into your supervision tree directly, perhaps with an
:ignoreresult as appropriate. - I academically miss the prospect of release plugins for a way for Hex packages to incorporate into your release build more intrusively without a lot of copypasta, but have not actually seen them used in practice so far. Like the boot hooks, I think this probably just evolves into supervision tree entries as well, plus documentation.
I hope it doesn’t seem petty that I say I miss these things in this specific form, but also the Mix.Release module docs and mix task docs leave them out as topics thus far. For most folks, they’d need to be aware of the existence of these features as prior art in order to synthesize the native solutions above from first principles, and part of my stance is that deployments are already hard to teach good instincts for. Not having a canonical reference for these things in the first-party material means I need to say things like “also check out the Distillery docs even though you may not use it, there’s some good ideas to steal from”.
I’m just as excited for this as I am for runtime.exs! 
Very true, I won’t get everything I mentioned from any one place. I don’t want to turn this into an advertisement thread for Rust so I will just say it already has some traction in most of the spaces I’ve mentioned having an unfulfilled interest in, with ML being probably the weakest and webdev having some ergonomic concerns in its current form.
1