Qqwy
Docker in combination with Hot-Code-Swapping?
The current cloud infrastructure seems very much based around the concept of containerization: Running each service in its own virtual environment. The main advantage seem to be that you can normalize configuration and storage options for these services, regardless of how they have been built themselves, and that you are able to automate the spinning up of new containers when necessary to handle increased load (as well as stopping them if no longer necessary).
Containers are usually very stateless, with a database as source of truth, meaning that you clusters are usually updated by (one by one) starting the new version of the software in a new container in the backround, and then replacing the old container by this one.
Unfortunately, this is sort of orthogonal with Erlang’s own way of doing Hot-Code Upgrades (‘code replacement’), because these are all about treating your cluster as a stateful system (which is obviously necessary to keep existing sockets etc. alive). Or at least, that is what I thought at first.
(See also: this question on StackOverflow)
I think that there are two ways to upgrade a system that have the same result:
A)
- Move the new release inside the current docker container running Erlang.
- Do a hot-code Erlang upgrade to the new version.
Your docker container now runs the new software update, or at least until restarted.
B)
- In the background (e.g. on your build machine) start a new docker container with your current software.
- Do a ‘hot’ Erlang upgrade of this system.
- Commit this docker container as a new version of the docker image.
You now have a new docker container that might replace the old production nodes, when they have to be restarted.
It seems to me that, if these two techniques are used together, we can have hot-code upgrades of a running Docker cluster, where if nodes might need to be added/removed at a later time (or when you later have to restart larger parts of your cluster) the cluster will still be running the latest version of your software.
The reason this is under ‘Discissions’ is because I do not have enough experience with Docker and Kuberneters (and my knowledge of Distillery is not that extensive either), so I would like to know if this idea is sound. Also, I’d like to know if there are other people that think that this is an exciting idea as well, and if there are other alternatives that might make more sense 
.
Most Liked
keathley
You can have persistent connections and immutable deploys. These are not mutually exclusive. It does mean that you need to add orchestration at the systems level. But this is how every other runtime that utilizes RPC over persistent connections works.
peerreynders
Currently my position is that code replacement in a containerized environment is a “square peg in a round hole” kind of situation. Code replacement has its use cases where a typical cut-over is just not feasible, be it on telephone switches before containers, upgrading a drone in mid-flight or upgrading high availability IoT devices.
In any case certain modules/processes have to be written with an awareness that code replacement is a possibility in order for the upgrade to be successful which tends to increase the complexity at the lowest level. So while the VM has the basic capability for code replacement, it isn’t a capability that can be taken advantage of “for free” on the system level - it adds yet another concern that winds through the entire codebase.
With containers you have the opportunity to address that complexity explicitly through architecture. Daniel Azuma describes one approach taken in tanx that uses Horde supervisors.
tristan
Yea, it would be neat if the new Erlang socket module was able to create a socket from a file descriptor like wat is done in other languages to use systemd’s socket activation. This would allow for full node restart without losing the listen socket.
Popular in Discussions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Deploy Phoenix to your own VPS with PaaS ease: push-to-deploy, zero downtime, auto SSL, managed Postgres. 14-day free trial at potions.io
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Real-time error tracking, performance insights, and observability for devs.
Develop your skills with books, videos, and courses.
Producing high quality Elixir screencasts since 2017.
The deployment platform built for Elixir: PaaS ease, VPS control.
Courses that'll move you from confusion to "Aha, now I get it!"