###QCon London 2017 - Outlyer: Why We Chose Erlang over Java, Scala, Go, C
- Colin Hemmings, CTO of Outlyer (formerly Dataloop.IO) - Cloud-Scale Infrastructure Monitoring designed for customization
- Outlyer is a SaaS infrastructure monitoring tool, processing and storing time-series data, currently at 100K points per second and growing.
- Replaces Nagios, Graphite, Statsd (1:52)
- Building for a Startup (2:30)
- V1: The monolith (5:23)
- Using Python Agent/Node.JS/Mongo DB for quick delivery (6:27)
- Growing the Monolith (6:57)
- Microservices? (8:01)
- Structured Monolith - Break up the code; Separate state; Separate Services; Separate deployment (10:45)
- Separating State (11:37)
- Dirty Secret - Mongo DB (12:32)
- Scale state (13:41)
- V2: Microservices (15:56)
- Splitting Services (16:00)
- Central Bus based on RabbitMQ (may move to Kafka) (16:40)
- Time series: Riak (16:52)
- Scaled well at first - then onboarded 2 large customers which lead to constant fire fighting (18:02)
- Redis to the rescue - but a more permanent solution was still needed (18.42)
- The Erlang Story (19:58)
- Requirements: Distributed for scale and HA; Reliable; Good Performance; Production visibility (20:03)
- “… contains an ad hoc informally-specified bug-ridden slow implementation of half of Erlang” (20:14)
- Option 1 - Java (22:02)
- Option 2 - Scala (22:49)
- Option 3 - Go (23:13)
- Option 4 - C (23:51)
- Honourable mention: Rust (24:25)
- Erlang: State and Concurrency (24:37)
- Erlang: Supervisors (25:48)
- Erlang: Behaviours (27:02)
- Erlang: Visibility (28:47)
- Erlang: Applications as Microservices (29:45)
- Erlang: The rest (31:11)
- Smaller Community: Erlang developers not as prevalent but “the ones you do find tend to be really good” (32:42)
- Elixir - will adopt in the future (33:32)
- Scalable Time Series (34:38)
- Dalmatiner DB Riak Core based, using ZFS (35:55)
- How it works (36:56)
- Riak: As previously mentioned (37:34)
- Riak Core (37:58)
- Benchmarks: Top 10 Time Series Databases (38:48)
- Trade offs (40:21)
- Now (roughly) (42:35)
- What we’ve learnt (43:29)
- "… didn’t realize how passionate and strong the Erlang community is … growing in numbers with the adoption of Elixir …" (44:57)
- What’s Next: Erlang all the things; Elixir; More live code reloading (45:57)
- Lessons from a small startup (46:19)