34) ElixirConf 2017 - Elixir with Kafka - Mathew Gardner

Today’s talk is another one from Toyota Connected

ElixirConf 2017 - Elixir with Kafka - Mathew Gardner

Elixir with Apache Kafka is a distributed streaming platform which is great at building real-time, web-scale streaming data pipelines that reliably deliver data between producers and consumers. It’s a core component of many large big-data pipelines and sounds a lot like our favorite language, Elixir! Toyota Connected’s role as the global hub for Toyota’s vehicle data means that hundreds of thousands of data points are handled each second. Kafka allows us to maintain a constant stream of these messages into the application without being overwhelmed.

We will show how we used Elixir to handle this scale of throughput and how we interoperated with the other components of the big data ecosystem here at TC. We will show pros and cons with our approach and discuss the alternatives.

Audience: Intermediate-Advanced

All talks are available in the Elixir Conf 2017 Talks List or via the elixirconf2017 tag