So, I bought a used Dell PowerEdge R720
One of the things I do a lot of in my job is compiling things. Especially Yocto/OpenEmbedded Linux images for various clients and projects.
Anyone who does these knows that they can take a long time to complete, and that the more cores and memory you have, the faster these builds will go.
For the last 4 or 5 years, I’ve been using my primary workstation for these builds. It’s an i7-4770 (4C/8T) based machine with 32GB of memory. At the time it did a great job. Well, I guess it still does, but when I’m doing them, there’s not much else I can do on the machine, unless I restrict the number of cores the build will use to at least give me one core that I can do email, editing, and other typical tasks.
I thought it was time to build a new machine – this time, one dedicated to these kinds of builds, so no need for fancy graphics cards and the like. Just lots of cores and memory, plus at least 8TB of storage – each build can suck up 100GB or more depending on configuration and packages built.
At first, I was looking at the usual suspects - Intels latest generation with at least 8C/16T or similar from AMD, such as their ThreadRipper series. ThreadRipper seems the best option given the number of cores, support for ECC RAM and a way more generous price point per core than similar offerings from Intel.
The problem with ThreadRipper (and Intel) is that you pretty much need a liquid cooling system for for “good” core counts. A 16C/32T system would have been ideal, but Intel was too expensive and I just do not trust liquid cooling. Yet. At some point I’ll probably have no choice.
Meanwhile, a colleague of mine who works in Poland suggested a used “datacenter quality” machine using Intel Xeon processors. He had a 16C/16T machine that beat my poor old workstation by a significant amount in build times. This could not stand.
So, I started looking - on Amazon of all places. There, I found a Dell PowerEdge R720 machine (19” rack mount), refurbished (“renewed” in Amazon-speak), with a 90 day no-questions-asked return policy.
It came configured with:
- 2x 2.60Ghz Intel E5-2670 processors, each supporting 8C/16T for a total of 16C/32T
- 192 GB ECC DDR3 RAM
- 8x 2TB SAS hard drives in 8 hotswap bays (7200RPM)
- H710P hardware RAID controller with 512MB cache
- iDRAC (integrated Dell Remote Access Controller) card with full Enterprise license installed
- 4-port 1Gb Intel Ethernet PCIe card
- Dual redundant hot swappable 750W power supplies
- DVD R/W
And it cost me under $800.
So I had to try it, and I am really glad I did.
When I got it, my first task was to get an OS on it I could actually use (it came with Windows Server 2012 on it).
All 8 drives were configured in a RAID0 (stripe) configuration, which meant that if I lost one disk, the entire array was gone.
I also had to learn how to use it’s “BIOS”. I put that in quotes because it is way more advanced than the typical PC BIOS you may be familiar with.
Fortunately manuals, specs, updated firmware and the like are all available from the dell.com website.
So I destroyed the existing “virtual” drive in the RAID controller’s firmware, and created 2 new virtual drives. One made of the first two drives in a RAID1 mirror configuration (1.8TB usable space), and the other 6 drives in a RAID5 configuration (9.1TB usable space).
Oh yes.
Then I loaded Ubuntu 18.04 LTS Server onto it. The BIOS must be set to UEFI for it to work - otherwise I could never get it to recognize and boot/install the media (tried both USB stick and DVD).
No problem, UEFI is fine and I have no idea why it was configured otherwise – probably a Windows requirement. Anyway, Ubuntu Server installed with no problems.
Then I wanted to try some builds. As expected, they blew my workstation away. Although each core only runs at 2.6Ghz, compared to the 3.40Ghz of the i7-4770 on my workstation, the sheer number of cores and the memory bandwidth (twice that of the i7) were more than sufficient to chew through those builds with reckless abandon.
One fairly huge vendor demo image took about 6.5 hours to build on my workstation. The R720 did it in 2.5 hours. It was an amazing thing to see 32 cores running full tilt. Bitbake never seemed so happy :)
There are a couple of downsides:
- Power. This thing eats a lot of power. It will consume 196W at idle, and 460-500+ watts at full tilt.
- Boot speed. The BIOS takes nearly 5 minutes from power-on to do all of its checks and inventories.
- Heavy… It weighs 64lbs. You want this in a proper rack or sturdy table.
The boot speed isn’t really a problem. It’s not something you have to do very often, and rarely do you need to care how long it takes to come up - as long as it actually does.
The power consumption can be problematic though. Connected to my UPS, it brought its runtime on battery from about 35 minutes to 11 minutes. Ooops.
Fortunately, using the included iDRAC, I can remotely power up the machine when needed. If I don’t need to do any builds for awile, I just shut it down.
I also have it automatically shut down on a power fail indication from the UPS, so the other machines can live longer :) I only use it for builds, so there is no impact on the rest of the network when it is not available.
So in short, if you are looking for a workhorse like I was, but didn’t really want to spend thousands on it, check out a machine like this one.
There are many available and they are solid performers in my short experience with them. Now I no longer need to abuse my workstation, and I can keep the build machine in the machine room where it’s cool, and remote – since it can get a little loud (though less than expected) depending on load and ambient temperature.