Qqwy
The current cloud infrastructure seems very much based around the concept of containerization: Running each service in its own virtual environment. The main advantage seem to be that you can normalize configuration and storage options for these services, regardless of how they have been built themselves, and that you are able to automate the spinning up of new containers when necessary to handle increased load (as well as stopping them if no longer necessary).
Containers are usually very stateless, with a database as source of truth, meaning that you clusters are usually updated by (one by one) starting the new version of the software in a new container in the backround, and then replacing the old container by this one.
Unfortunately, this is sort of orthogonal with Erlang’s own way of doing Hot-Code Upgrades (‘code replacement’), because these are all about treating your cluster as a stateful system (which is obviously necessary to keep existing sockets etc. alive). Or at least, that is what I thought at first.
(See also: this question on StackOverflow)
I think that there are two ways to upgrade a system that have the same result:
A)
- Move the new release inside the current docker container running Erlang.
- Do a hot-code Erlang upgrade to the new version.
Your docker container now runs the new software update, or at least until restarted.
B)
- In the background (e.g. on your build machine) start a new docker container with your current software.
- Do a ‘hot’ Erlang upgrade of this system.
- Commit this docker container as a new version of the docker image.
You now have a new docker container that might replace the old production nodes, when they have to be restarted.
It seems to me that, if these two techniques are used together, we can have hot-code upgrades of a running Docker cluster, where if nodes might need to be added/removed at a later time (or when you later have to restart larger parts of your cluster) the cluster will still be running the latest version of your software.
The reason this is under ‘Discissions’ is because I do not have enough experience with Docker and Kuberneters (and my knowledge of Distillery is not that extensive either), so I would like to know if this idea is sound. Also, I’d like to know if there are other people that think that this is an exciting idea as well, and if there are other alternatives that might make more sense 
.
Where Next?
Trending in Discussions
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
Build Elixir applications with speed and confidence.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #blog-post
- #elixir-ls
- #ai
- #elixirconf-us
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #hex
- #security
- #metaprogramming
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Practical resources that improve the lives of professional developers.
Enabling companies to succeed by building software people love.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 10- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
lpil
Both seem viable though I would ask why use hot code upgrades when you’re in an environment where you can cycle nodes and perform immutable deploys?
What business problem does hot code loading solve for you that makes it worth the additional complexity?
amarraja
I’ve never tried it, but gigalixir has support for hot upgrades in a container, and seems closely aligned to your first suggestion. On the topic of restarts, I guess a new container starting will take the latest image anyway so there will never be a “rolled back” state.
sribe
If you have anything where clients need stateful persistent connections. Now, web clients are not written that way, but not everything is an HTTP service. Of course, much other software also uses stateless connections, but…
Consider the classic use of Erlang: phone service
keathley
You can have persistent connections and immutable deploys. These are not mutually exclusive. It does mean that you need to add orchestration at the systems level. But this is how every other runtime that utilizes RPC over persistent connections works.
peerreynders
Currently my position is that code replacement in a containerized environment is a “square peg in a round hole” kind of situation. Code replacement has its use cases where a typical cut-over is just not feasible, be it on telephone switches before containers, upgrading a drone in mid-flight or upgrading high availability IoT devices.
In any case certain modules/processes have to be written with an awareness that code replacement is a possibility in order for the upgrade to be successful which tends to increase the complexity at the lowest level. So while the VM has the basic capability for code replacement, it isn’t a capability that can be taken advantage of “for free” on the system level - it adds yet another concern that winds through the entire codebase.
With containers you have the opportunity to address that complexity explicitly through architecture. Daniel Azuma describes one approach taken in tanx that uses Horde supervisors.
TANSTAAFL
tristan
Yea, it would be neat if the new Erlang socket module was able to create a socket from a file descriptor like wat is done in other languages to use systemd’s socket activation. This would allow for full node restart without losing the listen socket.
AlchemistCamp
I use docker for CI/CD on Gitlab and then deploy from that container to a (non-dockerized) DO droplet via hot upgrades.
I don’t actually need the hot upgrades, though. The driving motivation of the setup was simplicity. I don’t need to run docker (thus it’s not a problem working from my windows laptop), deployments are automatic whenever I merge to master and tests pass, and all I had to set up was Edeliver and a .gitlab-ci.yml file.
YMMV if you’re in a more enterprisey setting.
sribe
Well, yeah–not trivial, but completely possible.
lpil
Hot upgrades are also not trivial, especially if your application is stateful. According to the Learn You Some Erlang book at Erricson they spend as much time testing their upgrades as they do their actual application code, effectively doubling the amount of work.
garazdawi
Can’t you just use the
:gen_tcp.connect/listenfdoption? Or am I missing something?