In an admittedly non-scientific and basic test I compared compilation times from a clean state. The $20 server built my current project in about 35 seconds, but surprisingly the $320 took 22 seconds. I was expecting a much larger change to be honest. I rarely have to run a full build, so those 13 seconds a few times a month definitely aren’t worth $300.
You will see diminishing returns because of network latency. I use a first gen retina 2012 MB Pro and a 2016 MB Pro, both with 8 gigs of ram, and I run around 500 tests in circa ~5 seconds.
I also find that using structs when the DB isn’t necessary to be effective in reducing overall time.
If I wanted to change, I would consider a new MB Pro, but they’re very pricey.
Network latency? As in latency within DO’s infrastructure? I assume this is where dedicated hardware over virtualised CPUs would make a difference?
I try to use structs in my tests where I can, otherwise I’m just testing Ecto. It’s only when I’m having to use Ecto’s escape hatch to SQL do I actually run the test against the DB.
Indeed, however I think the more abstract things like battery life, screen quality and a lighter OS come into play. It’s pretty subjective and probably a case of familiarity. You could run an SSH session from almost anything and get the same benefits. In many ways that’s the point of working in remote environments, the client really doesn’t matter.
Just done a few comparisons between DigitalOcean’s Droplets and some hardware I have access to. I compiled my current project - still quite small - after having run mix clean --deps, and then ran time mix. Seconds are the “real” time.
Caveat: Yes, I realise this test is abstract and only one data point, but I feel it’s sufficiently real-world to answer my questions.
$20 DO server
2 vCPU - 4GB
34.56 seconds
$320 DO CPU optimised server
16 vCPU - 32GB
21.18 seconds
2014 Mac Mini on High Sierra
1.4Ghz dual-core Intel i5 - 4GB 1600MHz DDR3
28.69 seconds
2015 15" MacBook Pro on High Sierra
2.5Ghz quad-core Intel i7 - 16GB 16MHz DDR3
18.2 seconds
On that basis, there’s either a hefty cost in running Ubuntu (unlikely) or the virtualisation of Digital Ocean VPS’ is pretty costly. Whether that’s just the general overhead of a VPS, or the actual clock-speed of those virtual cores is severely limited. I’d guess at the latter?
I may provision a dedicated server with similar specs to my proposed desktop just to see what sort of performance is gained by running on bare-metal vs. cloud/VPS hardware.
UPDATE: 2 x Xeon E5-2640 provisioned from Packet
2.6Ghz 16-core Xeon - 128GB RAM
18.87 seconds
Personally, I think that lends credence to the idea that clock-speed matters far for compilation than multiple cores. Tests of course ran a fair bit quicker than the MacBook Pro though.
I went from a MacPro to a MBA (8gb ram 250gb ssd and with external display, keyboard and trackpad for most of the day) and loved my MBA so much that I sold my MP. The MP was the most powerful computer I have ever owned, but the MBA was the ‘best’ (i.e overall favourite) computer I have ever owned.
Going from 11" screen to the Retina 15" was probably the most obvious change. It was mainly the extra vertical height that I noticed. I felt like I had to spend less time managing windows and adjusting side panels to squeeze everything into view at once.
The performance was obviously better, but it was mainly the screen size/quality that was the “Wow” feature for me.
Edited to add:
Equally, the iPad clearly isn’t the most powerful computer I’ve used, but it has become my favourite.
I have the latest Macbook Pro, well… nothing really special about it. Just a goddamn boring machine with lots of its ports removed, leaving only 2 USB C ports and a headphone jack. But it is so damn stable and I have no other issue since the first day.
But really, I think any machine that is built after 2011 will work just fine for web development.
And I still have a 2012 Acer laptop with Linux Mint on it, currently using it right now, currently compiling a Phoenix project while browsing, running a rails app, and viewing Youtube. It’s pretty hot (I mean it’s hot because of 100% CPU usage right now).
Macbook Pro => Boring, stable machine, no customization, no need to install any driver, so compact yet so stupid because when I need to plug in an HDMI port, I have to find an adapter for it.
Linux => A tons of distro to choose from, a tons of customisation to choose from, but yet some hardware need to find a correct driver in order to run smoothly.
Currently using Macbook pro 2017 15 . Before it. I was using linux on sony laptop.
My biggest complain was linux is not stable. My wifi stops working and I have to install it again. Imagine you are on a deadline for some specific task of your project. And then you have to install the linux and then the applications all over again. Really frustrating. Keyboard suddenly stop working after you wakeup laptop from sleep.
But i did phoenix web development on that machine. It was core i 3 with 4gigs of ram. It only slows down if i opened up multiple tabs in chrome.
But macbooks are stable. I didn’t run any such problems that would cause problems in my daily work. But they are very expensive compared to windows. They are good but prices are just absurd.
I wish people could stick to the topic. But anyway: I looked online and found a linux kernel compilation time benchmark results chart which suggests that more cores is more important. Maybe with your tests there were other bottlenecks, like drive or memory speed / latency.
edit: The above looks like user-submitted results. Phoronix uses this benchmark to test processors regularly, results there should be more accurate (see second chart). Recent server-grade processors are here - 128 threads allowed to compile the Linux Kernel under 30s! Nuts!
Agreed, we’ve sorted of descended into “I use X…” I’m probably responsible too because of all the iPad chat. (Writing up that wiki that @AstonJ suggested - maybe could do some cleanup once it’s published)
That’s the sort of thing I was after! I had only searched for Elixir specific benchmarks so hadn’t come across that one. It definitely suggests that cores are the driving factor for kernel compilation.
Could compiling Elixir be much different?
What’s interesting though is that Intel iXs seem to outperform Ryzen processors, even when the AMD chip has both a core and clock-speed advantage. Perhaps things like L1/L2 cache, and maybe even cooling are coming into play? I’ve heard that Ryzens throttle themselves quite aggressively when the cooling isn’t adequate.
Very possibly. My test was only very superficial and didn’t really have enough data-points to form a comparison. Equally, it’s hard to know what sort of hardware you’re running on with DO droplets — kind of the point I guess… — which makes them hard to compare.
I didn’t want to keep that Xeon server around too long as it was quite pricey, so didn’t run too many tests on it. I wouldn’t have thought that there would be any meaningful bottlenecks on a server like that, particularly when compared to a 2015 MacBook Pro.
I realise that all this comparison of hardware on a task like compilation and test running is somewhat irrelevant unless you are on old hardware and the tasks are obnoxiously slow. I was mainly curious whether there was a case for fast hardware the same as there is for video rendering, and whether there was a point of diminishing returns.
I’ve written up a wiki about working from an iPad / on a remote server so that this thread can remain focussed on the hardware choices that help with Elixir performance.
Just a thought: Elixir and Erlang scale across multiple cores quite nicely, but if you’re working with many front-end assets, that means you’ve probably got Node.js in your build pipeline, which is essentially single-threaded, so peak clock counts more than number of cores when you run that. Sigh…
They certainly do for running applications and tests, but I’m still unclear how multiple cores are used in compilation. So far the tests that I’ve done seem pretty inconclusive due to too many variables in play and potential bottlenecks elsewhere in the system.
To be honest, I’m fairly certain that some of the Elixir core team would know the answer to this. If any are reading this, please jump in and put me out of my misery/curiosity!
I made certain to remove it from the codebase I was testing with. It’s a good point though. Realworld work is going to depend on frontend build tools for most people, so clock speed is something well worth considering regardless of how well Elixir compiles across cores.
That’s what I’ve been doing, but across 4 different systems I’ve got very varied results. One didn’t utilise all the cores, one saturates a few cores leaving the rest at ~50% load, another saturated all the cores, etc.
It’s not doubt — I believe and know that Elixir can use multiple cores during compilation — but I’m trying to understand what else is impacting performance and where these variances come from.
Yes, I’m probably asking questions that no one else cares about, but I’m curious about the bottlenecks and roughly where the point of diminishing returns kicks in.
