Note: Please read the question and details, and think about it before reading any of the replies.
There are numerous resources (books, video tutorials/courses, blog-posts etc) which will teach a complete noob how to create a web application based on Phoenix from start to finish, but there is no single resource which will teach her to deploy that same application (in a way as if it’s taught to a 5 / 6 years old).
I had high hopes from the book Designing Elixir Systems with OTP, but when I read its contents, it doesn’t teach how to actually deploy the SCALABLE application you created after reading that book, and I think nobody designs a SCALABLE application just to keep it in her harddisk and brag about it in front of her friends.
Is there any book which will teach me how to deploy an Elixir / Phoenix application in a way as if Jose or Chris is deploying himself or someone from Plataformatec or DockYard in the best scalable way on a group of bare-metal servers rented at Hetzner or a cluster of VPS instances from DO, Linode (again) Hetzner or other cheap (but reliable) host?
I checked Rails community have some nice books (and some dedicated coruses) on this topic which teaches a noob how to deploy a Rails application to a VPS (with and without Docker/Kubernetes etc) she just created.
Even though I didn’t read this specific tutorial yet (I bookmarked it to read it later), but as many tutorials I read teaches to deploy Phoenix applications as if you’re deploying some CodeIgniter or Laravel application on a single instance as if you’re expecting 10 users per day.
A side note: I won’t prefer digital ocean for any serious application after reading this Twitter thread, https://twitter.com/w3Nicolas/status/1134529316904153089
It doesn’t mean that Rails developers are completely clueless about Linux and other stuff, but it’s always great to have resources like that because not everyone wants to deploy applications to AWS etc, and there are many people who’ll want to take the full advantage of Elixir and OTP’s strengths by deploying their applications to bare-metal servers and / or dedicated VPSes. thank you for the replies!
That talk is helpful if you know over half of the things he is talking about.
I think the OP is right – written documentation about deployment is scarce, and I have also noticed that, but thought maybe I did not search hard enough.
Ideally, such documentation would answer the following questions:
What options do I have to release software?
What is considered good practice, and which approaches are bad or even dangerous, and why?
What if I want to deploy without docker, ansible, chef or something? You know, the least-tools-possible-learn-everything-unix-shell way?
I hope for the 1.9.0 release feature to standardize things, but have not found blog posts yet.
And as a last nitpick, and that is purely my taste: “There is no written documentation, but in this two-hour youtube video, maybe there are ten minutes about what you need” frustrates me to no end.
(Clarification: No one has said this quite in that way in this forum, and I am not aiming for provocation. I want to emphasize what is not helpful to me, personally. Youtube videos are probably better than nothing at all for many people.)
Hopefully some day we will have official, standardized documentation which explains it all. I mean, not only what you can do, but what you should (and more importantly should not) do.
I found this write up extremely useful, along with relevant chapters from Elixir in action, the many posts by Fred Hebert and the many videos from beam confs and the likes.
However I doubt that there will ever be a book or resource explaining how to deploy for scalability in the abstract. Apps, even running on the beam, tend to be complex systems, with highly heterogeneous requirements. Thus, the meaning of scaling, and availability will be highly dependent on the app and the requirements.
As an example, we’ve shipped an app recently, it receives events from sensors that produce (after heavy processing) data that we use to update the database.
With the current implementation, on our server, we can process 10000 events per minute, but it’ll take an hour to update the db accordingly. So in fact we can process 10k events per hour. We need consistency, so a distributed system with eventual consistency can’t work for us. So, for us scaling will mean a bigger connection pool, or improving our updates (possibly with caching) or getting rid of the updates.
Scaling is hard, even with the beam.
That being said, we currently get 30 to 50 events per hour, so I’m not worried.
Is there any book which will teach me how to deploy an Elixir / Phoenix application in a way as if Jose or Chris is deploying himself or someone from Plataformatec or DockYard in the best scalable way on a group of bare-metal servers rented at Hetzner or a cluster of VPS instances from DO, Linode (again) Hetzner or other cheap (but reliable) host?
No, I don’t recall there are any books currently that is dedicated to deploying let alone one that is just for scalable deployment. Maybe there might be an Erlang book somewhere.
I think it’s too niche with such a small dev base.
There are many blog articles out there for k8 + dockers and CI deployment if that’s your definition of scalable.
If it’s just straight up putting an app on a VPS there are blogs out there too. I just spent a day deploying mine yesterday (2nd time deploying).
No, I meant something which someone like me (a noob in deployment) reads and then deploys Whatsapp, like it was deployed before being bought by Facebook (then they used few big servers), and again deploys it the way Facebook deploys it now (many small servers), without using k8 or docker.
If I’m not able to do that with Elixir, then I should have used Rails (if I needed Rapid development), or Go (if I needed higher performance).
Edit: I’m sorry if this looked too much, I wasn’t able to explain what type of detailed book (on deployment) we should have as a community running on BEAM.
Bundle the data needed to be pushed to your servers
In Elixir/Erlang usually you build an otp release to be deployed. There is documentation for mix release in elixir 1.9 and also quite a bunch of documentation around distillery. In erlang I’m not sure which tool would create the release, but it’s a similar process.
Getting said data to the server and running it on there.
This part is actually hardly specific to elixir/erlang at all, there are a few requirements to releases (same architecture/os/native libraries where the release is build and deployed), but other than that it can be as simple as sftp’ing the release to the server or wrapping it in a docker file and using k8s to move it to the servers it should run on. Starting the release is actually just calling a binary with a command of how you’d like to run the release (daemonized vs in foreground). Those commands are also well documented.
(Extra)
Erlang Clustering: You can find a chapter on how it works in Designing for Scalability with Erlang/OTP. There’s also something in the erlang documentation itself. How you configure clusters depends quite a bit on how you manage servers and how dynamic/static your clusters are/should be.
Mnesia: If you use mnesia the above afaik becomes more tricky as mnesia is said to be not well suited for nodes joining/leaving often.
Other external dependencies to run: Look for “How to deploy external dependency”
I hope the above makes the point clear that when it comes to deployment a good chunk is not really about ruby or elixir or go, but about “do we buy a few bare metal servers and configure everything statically” or “we manage k8s clusters of 1000 nodes”; “Do we use CI/CD or deploy from developer machines?”. I’m not aware of books specifically for elixir (yet), but you should be able to read e.g. a generic “this is how you do CI/CD” book and adapt it to how elixir fit’s into the picture.
Also if you could specify which of the above topics you have most problems with people can also be more precise in suggesting reading material on the topic.
Aren’t Gigalixir and Render very expensive compared to a 10$/month VPS where you can run any number of Elixir apps, plus databases, plus any PHP/static/whatever app altogether ?
But according to the rumours I heard, they come with close to zero configuration and just work, while you need to do a bunch of setup and hardening the server on your own when you choose the 10$ VPS.
