19) ElixirConf 2017 - Scaling up Elixir at TpT - Shanti Chellaram

[axelson]

Here’s the last ElixirConf 2017 video of day one!

ElixirConf 2017 - Scaling up Elixir at TpT - by @shanti

Introducing a new language into an organization with a large existing codebase is a major effort. Teachers Pay Teachers recently made that transition from PHP to Elixir and NodeJS last year, and I’ll share the process, the struggles, and the triumphs we experienced, and the lessons we learned.

Audience: Beginner

All talks are available in the Elixir Conf 2017 Talks List or via the #elixirconf2017 tag

[krapans]

Great talk, however, I would love to know more about their production setup, servers, and architecture overall.

[axelson]

Yeah I’m really curious if they’re using clustering/distributed erlang at all.

[shanti]

Hi there! Sorry for the super-delayed followup.

We’re currently building on an internally-managed Kubernetes cluster, and we have multiple pods housing our API servers, which are all written in Elixir. The client-facing code executes on JS in the Browser + on Node.js workers and hits the API via intra-cluster RPC calls. We run with 8-12 pods on normal daily traffic, spiking to dozens during sale days. For the most part the servers themselves are somewhat-stateless, and we use MySQL as our primary data store. We also use message queues backed by Redis. The legacy system’s PHP codebase and Memcache sometimes access but are never accessed by the API.

[shanti]

We’re not using any distributed erlang facilities right now. If we had to set up a distributed lock manager, we’d probably rely on either Kubernetes’ built-in etcd instance or create a ZK instance. In-memory caches would be a strong contender for distribution within our API cluster, but we have no current need to consider those approaches.

In both cases, the additional operational overhead is a major factor against adopting more involved distributed architectures.

[krapans]

Is there any plans for making blog post about how you set up Kubernetes + Elixir for production with tips and tricks what other should know?