Getting a Sense of Relative Performance
The Arm world is, well interesting. We were asked to use GeekBench and both the single and multi-core numbers of ThunderX2 VMs versus Raspberry Pi 4’s using the geomeans of the single and multi-core scores. Things got a little bit strange when we did this request since we could not get a direct Raspberry Pi 4 Android score, but we could get a Raspberry Pi 4 Linux score.
What we essentially did, and the company we did this for thought it was close enough to be valid, was to take Raspberry Pi 3+ and Raspberry Pi 4 4GB Linux scores. We found that the Raspberry Pi 4 4GB was over twice the speed of its predecessor. We then ran GeekBench 2 and 3 on the Raspberry Pi 3+ and a ThunderX2 single thread VMs using dual 32 core / 128 thread parts. This meant we could have up to 256 of these VMs per machine, however, one generally does not want to fully load a server with VMs to leave some room for local tasks. The ThunderX2 single thread VM was just under twice the speed of a Raspberry Pi 3+ in Android. The ratios looked something like this:
The net impact is that a single ThunderX2 thread was running about 86% as fast as a Raspberry Pi 4 4GB. That came out to just over 220 RPi 4’s worth of performance per dual 32-core Marvell ThunderX2 server.
Using 220 felt wrong on both sides. First, a key principle of virtualization is overprovisioning CPU resources. Not all VMs will be running at all times. Larger virtual machine hosts help administrators balance workloads and oversubscribe vCPUs. That is Virtualization 101. On that side, we should be using more than 220. A counterpoint is that if they are all running full speed simultaneously, 220 is too many. The VM host needs some resources and running a machine at 100% is also in Virtualization 101 as a bad idea.
We are going to use 190 VMs as our number. We first took 10% off of the number of VMs which got us to 198. When we did 4GB of memory x 198 we got 792GB of total ram. That was close to what we could get with 768GB of memory in a system (192x 4GB) and we then left 8GB for the host. That got us to 190 4GB VMs to replace 4GB Raspberry Pi 4’s. This number also fits nicely into 8x 24-port PoE switches or 4x 48 port PoE switches.
One could argue that we could over-provision memory, but since we are not doing that on the CPU side, it felt like we should not be doing that on the memory side. Now that we have performance and RAM capacity targets for the ThunderX2 server, all that is left is coming up with the ThunderX2 system cost and doing a comparison to our Raspberry Pi node cost.
Marvell ThunderX2 System Pricing
The equivalent of the Gigabyte server we used in our Cavium ThunderX2 Review loaded with RAM at a lower-margin reseller, we are told, prices out to around $9000. We are going to add another $2200 for four 10TB hard drives and a 100GbE NIC.
One could argue that we do not need the 100GbE NIC if we are strictly comparing to the 190 Raspberry Pi 4’s, but we should have this capability. The 10TB hard drives give us more raw and usable capacity than the 24TB FreeNAS NAS we priced into our nodes. If we were building our own system, we would use SSDs, but we are using hard drives here.
We are also adding $300 for “extra bits.” That includes a management network port, a provisioning port, an upstream 100GbE port, and any other needed parts.
Total cost we are putting this as $11500. That will vary based on the discount you can get, but it seems reasonable.
Now that we have all of this, it is time to put it together and compare the costs.