Outlyer: Why We Chose Erlang over Java, Scala, Go, C

###QCon London 2017 - Outlyer: Why We Chose Erlang over Java, Scala, Go, C

  • Colin Hemmings, CTO of Outlyer (formerly Dataloop.IO) - Cloud-Scale Infrastructure Monitoring designed for customization
  • Outlyer is a SaaS infrastructure monitoring tool, processing and storing time-series data, currently at 100K points per second and growing.
  • Replaces Nagios, Graphite, Statsd (1:52)
  • Building for a Startup (2:30)
  • V1: The monolith (5:23)
  • Using Python Agent/Node.JS/Mongo DB for quick delivery (6:27)
  • Growing the Monolith (6:57)
  • Microservices? (8:01)
  • Structured Monolith - Break up the code; Separate state; Separate Services; Separate deployment (10:45)
  • Separating State (11:37)
  • Dirty Secret - Mongo DB (12:32)
  • Scale state (13:41)
  • V2: Microservices (15:56)
  • Splitting Services (16:00)
  • Central Bus based on RabbitMQ (may move to Kafka) (16:40)
  • Time series: Riak (16:52)
  • Scaled well at first - then onboarded 2 large customers which lead to constant fire fighting (18:02)
  • Redis to the rescue - but a more permanent solution was still needed (18.42)
  • The Erlang Story (19:58)
  • Requirements: Distributed for scale and HA; Reliable; Good Performance; Production visibility (20:03)
  • “… contains an ad hoc informally-specified bug-ridden slow implementation of half of Erlang” (20:14)
  • Option 1 - Java (22:02)
  • Option 2 - Scala (22:49)
  • Option 3 - Go (23:13)
  • Option 4 - C (23:51)
  • Honourable mention: Rust (24:25)
  • Erlang: State and Concurrency (24:37)
  • Erlang: Supervisors (25:48)
  • Erlang: Behaviours (27:02)
  • Erlang: Visibility (28:47)
  • Erlang: Applications as Microservices (29:45)
  • Erlang: The rest (31:11)
  • Smaller Community: Erlang developers not as prevalent but “the ones you do find tend to be really good” (32:42)
  • Elixir - will adopt in the future (33:32)
  • Scalable Time Series (34:38)
  • Dalmatiner DB Riak Core based, using ZFS (35:55)
  • How it works (36:56)
  • Riak: As previously mentioned (37:34)
  • Riak Core (37:58)
  • Benchmarks: Top 10 Time Series Databases (38:48)
  • Trade offs (40:21)
  • Now (roughly) (42:35)
  • What we’ve learnt (43:29)
  • "… didn’t realize how passionate and strong the Erlang community is … growing in numbers with the adoption of Elixir …" (44:57)
  • What’s Next: Erlang all the things; Elixir; More live code reloading (45:57)
  • Lessons from a small startup (46:19)
11 Likes

I found his interest in live code reloading a bit strange as most try to avoid it if possible. It works of course but can be a bit sensitive. Of course it depends what he means by live core reloading.

1 Like

Yeah, hot code reloading is nice when needed, but if you don’t need max uptime and can do rolling restarts then it should be avoided (especially considering that almost no one versions their data properly for transformation between versions).