Awesome. The old-schoolness of this makes my curmudgeoness reduces ever so slightly.
What did you buy and where are you co-locating it?
1 Like
haha “curmudgeonous” was a word I had to lookup what it meant 
The gear I bought was AdLinks server type devkit. It includes a motherboard and a cpu Ampere Altra 128core. The spec is as described above.
I have been looking for a better alternative for colocation and found a small business located in the middle of Sweden (was trying to find the name of the business, will have to locate my notes 
) but their cost model was awesome. It included 1gbit bandwidth, not sure right now how much traffic(but I came to the conclusion its more then enough) and power. edit found the name - I think its inleed I have been in contact with and thinking about to moving to.
I must say that the AdLink gear so far works just great. Noticed that Asrock has a bundle available on newegg(i think thats the name) with motherboard and cpu(I believe it was a 32/64core not sure which) to a much better price then Adlink, but what Im aware of no 128core alternatives at the moment.
If you are curious about the Adlink-system Jeff Gerling has a extensive review of it on youtube.
Old-school =) I dont know. perhaps. For me its just economy and control of my working environment.
Still scared shitless I will end up with a monster bill from google cloud because of me f*ing up some budget setting for the services that I haven’t migrated yet.
1 Like
What about temperature? This is kinda the deal breaker when it comes to self-hosting. Now it shouldn’t be a problem, but once it starts to get hot outside, your servers will start to throttle in a warm room.
Yea… Its sweden. Worst case scenario - open the windows 
1 Like
Cooling can be sorted out but hopefully the fans are not of the jet engine type. 5 minutes of that and I am ready to jump out the window… 
1 Like
This is awesome.
Definitely old-school. 
15 years ago, I setup multiple data centers. One DC had over 3TB of ram (16 servers x 96GB x 2 chassis), just in two blade servers that could fit in a single rack. Now on AWS, 3TB of ram with 24 x2gd.large instances is $5k/month and that doesn’t include anything else.
We had 6 racks of hardware in our main DC and I think it was $5k / month. The exact same price and we had probably 10x-20x the hardware. Really, not a contest. And when you buy that much hardware, you get exceptional discounts if you go shopping around the year-end. And nice dinners, too.
When looking for a provider, I would suggest you look for:
- multiple-uplinks for blended bandwidth
- make sure their UPS looks good
- review their core network ingress to you as you should have multiple servers that can failover.
There’s a bunch of stuff that needs to be done to make sure things load-balance on real hardware. We had F5’s, but I’m pretty sure there’s a way to just use clustering with BEAM, or maybe failover with floating-ips, or even bgp-failover.
Anyway, super-awesome and feel free to ping me if you have any questions / need any help.
2 Likes
Wait what? You go somewhere, buy second-hand tech for $5k - $10k, and the salesman takes you to dinner?!
1 Like
Currently on Gigalixir. Been meaning to give fly.io a try. Wish there was a “one-click migrate” option though.
Nah, even better. Pre-negotiation dinner. Post-deal dinner. Several sports games in the big booths with free food and beer, too.
The HP deals were the best. I think their year-end was November, and we’d order about 200k worth of stuff and get a 40% discount. Sales guys need to make their numbers EOY. We had a bit more leeway on our spend since our year-end was April, but we had to spend our budget regardless (those corporate games where if you don’t use it, you lose it capex). Good times.
You know, as opposed to AWS who sends solution architects that confound your teams with architectures with lambda and sqs and other nonsense and next thing you know you’re locked in and held over a barrel and mugged. And is more expensive. And slower. And you need even more headcount to operate it. And you got to re-architect your stuff to work because it isn’t reliable.
Damn, I’m being a curmudgeon again.
Before I get re-educated, cough, all hail our cloud overlords. The cloud is fantastic and I love all things cloudy and could never imagine anything possibly better.
5 Likes
I like fly.io. What I dont like, is how dependent they are on docker locally on the dev machine to function… and for what I can understand - for no real good reason other then that the tools they rely on assumes it.
I mean how complex can you make the deployment process
But perhaps its just me not understanding the process.
I’m definitely a member of the KISS camp.
1 Like
It needs to build a docker container eventually given that’s the input they generalized for, but you can let fly build that container on their servers instead of locally. I don’t think there’s any requirement to run docker locally.
1 Like
I tried to make the tooling work under freebsd. But perhaps it was pure skill issues at play on my part.
What you say sounds reasonable, and that was what I was hoping for… but… =)
Something failed anyhow, perhaps it was more tools in the chain that only works with Lindows. I dont really remember all the details on my endeavor to make it work.
But I did spend more then reasonable amount of time to try make it work =D
The remote builder works flawlessly on macOS; on other OSes, it may be enabled by passing --remote-only to the fly deploy.
Their support is amazing.
But
Their LHR region is the worst PaaS I ever used.
Starting from suspicious network latencies with GCP eu-west-2 region to advice from their support to redeploy until the VM is allocated on a server with enough free memory to accommodate RAM increase request 
, to service disruption when clients from EU were served times faster than from UK and LiveView was timing out for them, till suspicious RAM usage spikes and OOM crashes in the out-of-office hours on the internal tool.
They have the best DX out there, and flyctl is next to perfect, but using them is what actually makes me consider self-hosting.
Going back to the topic I want to mention, Ansible for DevOps by Jeff Geerling [Leanpub PDF/iPad/Kindle] can help set up reusable and declarative deployment scripts.
2 Likes
