sashaafm
Nomad - Create cloud portable Elixir and Phoenix apps. Write once, use everywhere!
Hello everyone, this is a project I’ve been working for my Master’s Thesis for the last few months (since December roughly). It’s an SDK for the cloud that also has deployment capabilites. The project can be seen here Nomad at GitHub
Nomad
Create cloud portable Elixir and Phoenix apps. Write once, use everywhere!
What is the Nomad Project?
Nomad is an Open-Source API for Elixir that enables developer to use the most popular cloud providers interchangeabily at will without having to change any of their code. Nomad provides generic API’s for cloud services (mainly Virtual Machines, Storage, SQL and JSON datastores). These API’s offer the most useful features and functionalities that are common across the same kind of service between cloud providers. They are meant to be simple and user friendly. This enables applications to be portable and allows the migration of applications from one provider to the next.
Moreso Nomad offers Mix Tasks for automatic management of cloud resources for your application. Need a SQL database in the Cloud for your application? Create it and automatically configure your app to use the one you just created. Want to host your Phoenix app in the Cloud? Launch a Virtual Machine through Nomad Mix Tasks and let it automatically deploy a production release of your app in said VM.
This project is my subject for my Master’s Thesis but once it’s delivered it will be a pure Open Source application.
Disclaimer: Nomad is under heavy development and is prone to changes
What are the use cases for Nomad?
The main goal of the Nomad project is to defeat Cloud Vendor Lock-in in the Elixir ecosystem. With the rise in popularity of cloud services Elixir users should not be restricted to one provider and should be able to make the most out of Cloud Computing. Some of the identified use cases are the following (but there are many more):
- You’re using S3 storage for a Phoenix application but a few months later Google has better pricing? Just let Nomad automatically copy your whole S3 bucket to a Google Storage bucket and relaunch the app with Google configurations.
- Recently got a new userbase in a remote region? Launch a new Virtual Machine with your application in the closest available region.
- Need a powerful SQL database but you’re not really knowledgeable in Cloud related affairs? Just use Nomad’s user friendly API and it will get done in a jiffy!
How is this achieved?
Nomad takes a lot of inspiration from Ecto and uses adapters in a similar fashion. First and foremost the needed API functions are identified based on what the cloud providers’ APIs have to offer and on what are the most recurring use cases and used functionalities. From this a general Elixir Behaviour is defined. These Behaviours must be implemented by each provider’s adapter.
Each adapter will implement the APIs for each cloud service (Virtual Machines, SQL, Storage and Datastore). Upon starting up an application a simple config in the desired environment config is needed - mainly cloud account credentials and a key to identify the chosen provider.
Finally through macros and metaprogramming, the adapters logic will be generated for the Nomad main modules (VirtualMachines, SQL, Storage and Datastore modules).
Roadmap
SDK
- Storage API
- SQL API
- Virtual Machines API
- Datastore API
- Friendly Mode and Full Control Mode (return parsed response vs. complete HTTP response)
Mix Tasks/Deployment Tool
- Database Listing
- Database Creation
- Database Deletion
- Database Reboot
- Automatic Deployment
- Virtual Machine Creation
- Virtual Machine Reboot
- Virtual Machine Deletion
- Storage Creation
- Storage Deletion
Trending in Announcing
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
Practical resources that improve the lives of professional developers.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #elixirconf-us
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #performance
- #security
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Practical resources that improve the lives of professional developers.
Develop your skills with books, videos, and courses.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
First 10 of 12 Posts!
sashaafm
I’ve updated the README with better descriptions and with links for tutorials.
The tutorials can be found here.
greyhwndz
Hmm. This looks interesting @sashaafm
sashaafm
@greyhwndz Hi! I’ve been working really hard on it. It’s my baby right now ahah. Could you please give me some insight on why you find it interesting? I’ve been looking for feedback from other developers.
Right now I’m doing the API for virtual machines. It’s been a bit more difficult than the other APIs but it’s doable. After that I’ll work on one for datastorage (JSON).
Ultimately my goal for my demo at university will be to apply it to a web app I’m working on in my internship and show how I can migrate from AWS to Google and back without code changing. Even for people not interested in migrating this library will be useful for regular cloud interaction since it leverages the most relevant use cases from the cloud services.
I’m also thinking about building Mix Tasks like for automatic deploying prod releases on cloud VM’s created on the fly and other things like creating a database in the cloud for usage with Ecto, etc.. (I’ve actually built some of these but they still need to be better tested).
It’s been quite interesting. Specially since 6 months ago I barely knew how the Cloud worked. I’ve been learning a ton.
sashaafm
Nomad has just been updated with the Virtual Machines API. You can now interact with Google Compute Engine and Amazon EC2 interchangeably!
Qqwy
This looks so amazing.
Once my application becomes deploy-ready I’ll definitely try it out.
What a wonderful idea!
sashaafm
Thank you @Qqwy! Please feel free to give me any suggestions, feedback or opinions. I’m really looking forward to making this the best tool possible.
I’ve got a ton of ideas for it and things to build with it. I just need time
tell me if you need any helping setting it up or using it!
sashaafm
Automatic Mix Tasks for virtual machines have now been added to master. You can now create, delete, restart and list virtual machines through a mix task.
Full control mode will appear sometime this week or the next one. Full control will be bang versions of the functions where the results are not parsed and the HTTP reply (HTTPoison) is returned fully . So instead of something like
create_storagereturning:oktheHTTPoison.HTTPResponsetype will be returned.sashaafm
Full control mode has been added to all APIs (Storage, SQL and Virtual Machines). Full control mode are bang versions of the API functions which return the complete HTTP reply (with
HTTPoison) instead of the specification defined return type.list_virtual_machines :: [binary] | binary=>list_virtual_machines! :: HTTPoison.Response.tThis of course breaks the abstraction layer since the HTTP replies from different cloud providers will differ between each other. However, I’ve found some use cases where this may be relevant to have and having more options doesn’t hurt. Mind that using the non-bang versions is still the default way to don’t break cloud compatibility.
With this Nomad has been bumped to version 0.7.0.
Next steps are:
Mix Task for automatic generation of applications’ prod releases and uploading to a given VM (also a variant of this Mix Task where the VM is created on the fly)
Storage migration (copying all files from one Storage service to another in order to be feasible and easier to change between clouds)
cmelgarejo
Awesome! Great tool you’re making @sashaafm ! Looking forward to use it on my elixir pet project
sashaafm
@cmelgarejo thank you! I’ve been finishing my Msc. Thesis (which Nomad is part of) and from August I’ll start implementing a lot more features.
Do you mind sharing how you’re gonna be using Nomad (what kind of application/main use case for Nomad)? If you need any help setting up or using Nomad please contact me anytime here, GitHub, Slack or Twitter
Last Post!
sashaafm
Hey again @Crowdhailer
I haven’t thought about integrating into
distilleryorexrmdirectly. However, I plan on automating a task similar to what you described:Please any feedback you may have feel free to tell me. The version I have on GitHub is not the most up to date, because I’m working on a new version privately to first send it for a conference and then I’ll merge into the public repo.