What do you need from a deployment tool?

deployment

#62

I figured I should update everyone here, but I accepted an offer to join DockYard to fill the position they opened up for this. I’ll be formally starting November 3rd, and once I do I’ll be jumping into this head first.

I’ll be posting more in the forum once I have some more concrete steps fleshed out, partly to keep everyone up to date, but also as a means of gathering feedback on what’s planned. More to come later!


#63

That’s awesome news - congrats! :023:

Perfect person for the job :003:


#64

Congrats!!!


#65

There is no better person for the job. Distillery and exrm are great tools, I can’t wait to see the third iteration in the family.

Congrats @bitwalker, looking forward to your updates :grinning:


#67

so… it’s going to be a completely new implementation, separated from distillery? @bitwalker can you reveal something more already? :slight_smile:


#68

I’ve being a bit out of touch due to moving, switching work projects and dabbling in Rust a bit. Now looking too start a new personal proj. in Elixir again was wondering are there any updates on this new tool ?


#69

And then there should be a book by @josevalim and @chrismccord called “Reliably Deploying Elixir and Phoenix Applications using deployex”. :slight_smile:
(book name copied from Reliably deploying rails applications :smile: ).


#70

Nice update about deploy tools! Thanks @bitwalker

Sounds like a big step in the right direction.


#71

This is great news! I struggled with Elixir’s config vs releases myself (and my early projects used them totally wrong) and have only recently gotten into actually using releases. I sense this friction between your normal development vs deployment affects many others too, so anything that helps the situation is welcome.

As I understood it, the plan would be to have config.exs read on application init and so you could directly get env vars there? That would be sweet and save from a lot of hassle.


#72

Brilliant update - really excited by everything you’re planning @bitwalker :023:


#73

Thanks for the feedback everyone! I’m glad you all like the direction :slight_smile:

More or less, to be clearer, the plan is to have Elixir’s configuration file be evaluated and pushed into the application env (i.e. which is exposed by Application.get_env/3) right after Elixir is loaded and started, and before any other applications are started. This mirrors how Mix handles things today, and would allow you to access the application env in your own application right away without having to do any of your own fetching/transformation.

Of course, that’s only true for things which use the config file, for those use cases where you need to fetch configuration from external system, say etcd, you’d still have to follow more or less the existing approach to that scenario. That said, I’m hoping to provide an extension point which allows one to extend the configuration system with custom providers, so that by the time applications start booting, all the configuration is done. I don’t know what that will look like yet, but I think it’s an important part of consolidating configuration into a single approach.

I know José is planning to move what is currently Mix.Config out of Mix and into Elixir as Application.Config, and while initially it should just mirror how Mix.Config works today, I suspect that would provide us a foundation for extension like I mentioned above.


#74

Great news! Unifying Mix and OTP releases will be amazing. Any plans to tackle the process of getting your release onto servers, updating, smoke testing etc? Personally I’m happy to use language agnostic tools like ansible for such tasks, but I get the impression many devs are looking for an elixir-native solution.


#75

My dream deployment tool is one that would allow me to easily update a running server with git.

In simpler setups, for those small site that have yet to reach a size where they need multiple servers, the deployment server and the build server are one and the same. I would just ssh into the server do a git pull and then build the assets, digest, migrate and then restart the phoenix server.

It would be wonderful if I could just add the deployment server as a remote to the dev machine repository and then just push to that remote in order to trigger an update. Ideally, the deployment tool would have this use case in mind and would automatically do what is needed based on sensible configuration.

I don’t know what percentage of sites developed with elixir fall into this category of small, single server sites but I doubt it is a trivial percentage.


#76

I have several small web projects and I find it easier to build in a vm (vagrant) or a container than on the deployment server. It can also be automated with git hooks and a shell script.

commit -> spin up a container -> build release -> push tarball -> upgrade


#77

Once you know how to do it, it becomes very simple but to a person that might not have all the knowledge this might look scarily complex.

I have never used vagrant. Do you have your setup described in some post? I would love to take a look.


#78

Once you know how to do it, it becomes very simple but to a person that might not have all the knowledge this might look scarily complex.

I don’t say it is simple, far from it, the fact that I have to spin anything up already makes it very complicated.

I just stated my preferences – that I find one approach easier for myself than the other.

I have never used vagrant. Do you have your setup described in some post? I would love to take a look.

No, I don’t. But I use the default vagrant file for freebsd (but setting config.vm.base_mac = "080027D14C66" but it’s freebsd-specific). For building releases for linux, it’s probably easier to use docker.

Vagrant.configure("2") do |config|
  config.vm.box = "freebsd/FreeBSD-11.1-RELEASE"
  config.vm.network "forwarded_port", guest: 4000, host: 4444
  config.vm.synced_folder ".", "/vagrant", id: "vagrant-root", disabled: true

  config.ssh.shell = "sh"
  config.vm.base_mac = "080027D14C66" # freebsd-specific

  # the rest comes from the default template

  config.vm.provider :virtualbox do |vb|
    vb.customize ["modifyvm", :id, "--memory", "1024"]
    vb.customize ["modifyvm", :id, "--cpus", "1"]
    vb.customize ["modifyvm", :id, "--hwvirtex", "on"]
    vb.customize ["modifyvm", :id, "--audio", "none"]
    vb.customize ["modifyvm", :id, "--nictype1", "virtio"]
    vb.customize ["modifyvm", :id, "--nictype2", "virtio"]
  end
end

And there are several blog posts on how to build releases in docker containers. Here’re the ones that I’ve seen being mentioned on the forum:


#79

I find deploying pretty easy. I use distillary to build (via a script), rsync the release over (same script), run a systemd command via ssh to restart to the new distribution (same script again). It’s very short. :slight_smile:


#80

Hey there @bitwalker, just saw that elixir 1.7 dev release notes came out a few days ago:

I watched your presentation on what’s next for deployment, and many of us are very eager to start using your work in this area. You mentioned in the talk that the first version may end up in 1.7, but I saw no mention in the notes. Is there still a chance it’ll make it in, or is there a new timeline going forward?


#81

we should do a jail version of this for FreeBSD … interested?


#82

:wave:

Do you mean a blog post? I’m not sure I have enough “tips” for deploying to freebsd for an additional blog post. Since what I’ve posted above my approach in its entirety: develop on mac, run a shell script to build a release in a VM, deploy.

I’d be very interested in reading more about your workflow, though.