jdumont

jdumont

TL;DR What facet of a computer really moves the needle in development performance (compiling/tests/etc)?

So, I know that an Elixir app uses its BEAM based magic to spread itself over all the available cores when running, and that a certain amount of memory will always be needed that scales with concurrent users or workload. But is the same true during compilation and testing?

I only ask because I’ve been recently working on an iPad using a remote DigitalOcean droplet. It’s a modest one ($20, 2vCPU, 4GB RAM) and I’ve found my workflow to be quite OK despite the low specs. I’ve also been using a 2014 Mac Mini on occasion which only has a paltry 1.4Ghz dual core, and again it performs perfectly well. In either case I’m never left waiting too long for builds, but tests do seem to be taking longer now that I’m getting to a decent amount.

We’ve bought a house and I’m looking to end the nomadic working life and setup a nice office with a desktop. I’ve also considered acquiring a dedicated server to continue working over an SSH connection as I’ve really grown to like it’s benefits.

I’ll be looking to spec a nice large monitor and good keyboard as I think it’s a given that these are good for productivity/developer health; but the other hardware is still up for debate.

I tried scaling my current droplet up to get a feel for the performance increases I could expect to see. I bumped it up to the $320 per month (insane amount of money!) CPU optimised server with 16vCPU and 32GB RAM.

In an admittedly non-scientific and basic test I compared compilation times from a clean state. The $20 server built my current project in about 35 seconds, but surprisingly the $320 took 22 seconds. I was expecting a much larger change to be honest. I rarely have to run a full build, so those 13 seconds a few times a month definitely aren’t worth $300.

Now, these being VPS’ I might be losing some of the raw power I could get from a dedicated server or desktop running similar specs - more testing needed. I had a octa-core Xeon server from Hetzner kicking around until last month, but never tested Elixir on it. I may provision one just to have a play.

In short, is it worth building a powerful desktop (or renting a powerful dedicated server) for Elixir dev? Or is it a case that my current DO droplet is “enough” and I should keep my hard-earned in my pocket and look for efficiencies elsewhere?

First Post!

ardhitama

ardhitama

You won’t get perfect answer, but I always get the latest mainstream cpu, gpu, ssd, ram, mb, psu, monitor, keyboard, and mouse I can get at that time.

Just make sure you have >= 16GB ram and the others scale them proportionally to your budget, but usually it will be >1K USD to get future proof desktop.

Most Liked

jdumont

jdumont OP

Just thought that I’d expand on my earlier findings. I’m still using the same method and project to test different hardware configurations, but the project has grown so compile times are quite different now, and the majority of the tests are disabled as the app is being rewritten at the moment.

Here’s the latest tests:

$5 DigitalOcean droplet - 1 vCPU & 1GB RAM
The single thread spent all of it’s compile time pinned at 98-100%. RAM usage was variable but stayed around 280MB.
Compile time: 2min 06sec
Test time: 3.6sec

$15 DigitalOcean droplet - 2 vCPU & 2GB RAM
Both threads varied between 50 - 95% during compilation. Rarely was a single thread pinned at 100%. RAM usage was the same as above.
Compile time: 1min 10sec
Test time: 1.2sec

$40 DigitalOcean droplet - 4 vCPU & 8GB RAM
All threads varied between 30-80%. Never pinned at 100%. RAM same as above.
Compile time: 1min 01sec
Test time: 1.1sec

$40 CPU-optimised DigitalOcean droplet - 2 dedicated vCPU & 4GB RAM
Both threads varied between 50 - 95% during compilation.
Compile time: 51sec
Test time: 0.8sec

$320 DigitalOcean droplet - 16 vCPU & 64GB RAM
Often threads were left unused, but sometimes all used at peaking at ~40%. Spikes to 60% on individual threads, but infrequent and short. RAM usage was slightly higher.
Compile time: 52sec
Test time: 0.9sec
!~ Interestingly one of my tests started failing from here on in. Issue with isolation when running tests in parallel most likely.

$320 CPU-optimised DigitalOcean droplet - 16 dedicated vCPU & 32GB RAM
Very similar to regular $320 server. Spikes were at 50% instead and RAM use climbed further to around ~650MB.
Compile time: 38sec
Test time: 0.6sec

c1.xlarge.x86 - Xeon E5-2640 (2x), 16 core (32 thread) at 2.6GHz & 128GB RAM, 1.6TB NVMe drive. $1300 p/m
Work spread across all cores, with single threads pinned at 100% for brief moments. Usually around 10-30%.
Compile time: 48sec
Test time: 0.9sec

t1.small.x86 - Atom C2550, 4 core @ 2.4GHz & 8GB RAM, 80GB SSD.
Huge variability in thread usage. Occasionally a single thread will be pinned at 100% whilst the other three are doing nothing. Only server to display this behaviour.
Compile time: 1min 54sec
Test time: 3.2sec

I had wanted to try out their Cavium server with 96 cores @ 2.0Ghz, but I found getting the project setup on the ARM version of Ubuntu quite tricky, and unfortunately I didn’t have the spare time to go configuring an environment I wasn’t going to use again. Given the relatively small changes in performance between 2 threads and 32, I don’t foresee 96 cores doing much for compilation but I imagine it would rock as the production server!

Reference: DO standard clock speed is 1.8GHz and 2.6GHz according to a blog post they made in 2018.

Conclusion: For remote development, I think DO’s $15 droplet hits the sweet spot. Spending a bit more on either of the $40 might be justified, but there’s very little time-savings thereafter. I think this probably carries through to local dev too, where the point of diminishing returns comes quite quickly. I think if I were using the full test suite I’d see more dramatic improvements in test time as threads increased.

For Elixir development I think you’re in pretty good standing as long as you’ve got more than one thread (surely that’s everyone these days) and clock speeds around 2.5GHz and above just to help with NodeJS compilation. It’s very impressive just how fast the core team have managed to make working with this language. Realistically, most local environments are going to be constrained by RAM more than their processor due to browser tabs, music playing, etc.

So in short, for all of you out there thinking that an iMac Pro or even the new Mac Pro is justifiable because of the time-savings you’ll see, I’m afraid it’s probably not. The reality is that unless you do something else like photo or video editing with your computer, we really don’t need powerful hardware. We’re more likely to find time savings in learning our tools (editors, linters, test suites, etc) better. Guess I’ll have to start taking more photos to justify my hardware-lusting!

jdumont

jdumont OP

@AstonJ and everyone else here:

I’ve written up a wiki about working from an iPad / on a remote server so that this thread can remain focussed on the hardware choices that help with Elixir performance.

jdumont

jdumont OP

Unfortunately I don’t have anywhere that a full on rack could live (noise, space, heat) in my current property, so it would have to be a tower. Given those benchmarks above, I’d be inclined to go for a Threadripper 2950X over the Xeons, but there is still the issue of Zen 2 making any purchases at the moment unwise.

I may have to do these. Coconut battery puts it at 81% of original capacity which isn’t bad, but it feels worse than that.

@peerreynders & @Exadra37 I really wanted to avoid this discussion becoming a Mac vs. Linux vs. X pissing contest like so many threads have before (including this one). I wanted to discuss hardware in quite broad (but also specific terms) and OS is a part of that. Yes, some of us agree and some of us disagree, but I’m sure that we can all agree that this great holy OS war is dull now.

We all use the tools we want to use. We don’t all need to be told we’re wrong, just because our priorities and preferences differ from yours.

Last Post!

jdumont

jdumont OP

Sorry for taking so long to reply. Funnily enough I’ve been working on iOS apps lately — so haven’t been on the forum — and bought an M1 MacBook Pro on release day to use for these projects.

Having just picked up another Elixir project I decided to have a go with the M1 rather than my Ryzen-based desktop just to benefit from that mobility (and the insane battery life).

As I’m sure you’ll know by now from other threads/blog posts, Elixir runs brilliantly on Apple Silicon. With brew now supporting AS natively, install and setup is easy and I’ve yet to have any problems. I’m using PostgreSQL through Postgres.app which does use Rosetta, but as-per pretty much every other Rosetta app, it’s been problem-free.

Performance is really good. The Erlang VM sees (and appears to use) all 8 cores and compilation speed seems pretty comparable to my Ryzen 3600. No complaints from me!

Where Next? Top

Trending in Discussions Top

AstonJ
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
2977 94592 917
New
cblavier
Hey there, It’s been more than a year since we started using LiveView as our main UI library and building a whole library of UI componen...
New
mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
heathen
Quite interesting article Google brought me. Didn’t find any mentions about it here. What do you think in general? Would you use togethe...
New
mhanberg
Hi everyone! The first release candidate for the Expert language server project is now available! We’ve published a press release detai...
New
AstonJ
Since we have deprecated our Erlang sections (as we have dedicated Erlang Forums now) let’s add this thread for those who’d like to post ...
New
Null-logic-0
What IDE or editor are you using for Elixir development? Personally, I use Zed, and I really like it, but sometimes I wish there were a ...
New

Other Trending Topics Top

JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Dmk
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
webofbits
With AI doing more of the implementation work, I’ve been wondering how much coding I should deliberately keep doing myself. My main conc...
#ai
New

Latest on Elixir Forum

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews