43) ElixirConf US 2018 – Docker and OTP: Friends or Foes? – Daniel Azuma

Some links related to the talk are at http://daniel-azuma.com/elixirconf2018

Some questions I got afterward that might be worth discussing since I don’t have a final answer to them include:

1. How can we scale out a solution like what we did with the tank game? How well will Horde’s mechanisms handle thousands or millions of processes or what changes might be needed to the approach?

2. What are the trade-offs of just using a database like Redis for the handoff? The current CRDT solution is extremely fast, but could Redis work better at larger scale? I don’t know, but I suspect since the usage would be so bursty, the contention might be its downfall. There’s also mnesia, which isn’t really designed for this kind of application but since it’s also in-OTP, it might be interesting to evaluate.

3. Horde vs Swarm. I actually tried both for the demo, and ended up using Horde mostly because it handled shutdown a lot better for this case. Also because it adopted the OTP concepts of supervisor and registry and so was easier to describe and reason about, while Swarm implements an odd hybrid of the two. But Swarm has interesting capabilities as well, such as built-in handoff, and is currently much more mature.

I’d also love to hear how others have solved similar problems. The cloud/containers world is important, especially as our Elixir/OTP based systems need to interoperate with other platforms, and I really think our community needs to engage more and figure out our contribution to it.

LibCluster, Swarm & Horde, dist_agent & other Raft implementations

The distributed systems libraries are starting to become scattered to where its hard to keep up.

Are there any plans for an official distributed Elixir OTP team to consolidate these efforts - maybe like the upcoming Phoenix Firenest?

Also a great talk and examples @dazuma - I enjoyed walking away with just as many questions as answers.