Rackspace Cloud Benchmarks – 512MB through 30GB Instances Compared

Today we are looking at the performance of Rackspace cloud instances using our new Linux benchmark suite. Recently we have used our new Linux benchmark suite to test new systems from both Amazon and Rackspace as well as a few non-cloud servers such as the Intel Atom S1260 platform. The results have shown both significant scaling as well as some differences between the two providers. Today, in the spirit of rapidly publishing the data we are collecting, we are releasing benchmark results on all of the Rackspace cloud instance types. These benchmarks scale from the small 512MB RAM instance to the 30GB of RAM instance which is a fairly wide performance spectrum.

Test Configuration

Each Rackspace instance was powered by an AMD Opteron 4170 HE, a two generations old predecessor to the AMD Opteron 4376 HE processor we recently benchmarked. We started five Next Generation Dallas instances and five Next Generation Chicago instances and got the same configuration. For this test, we utilized five different Next Generation Dallas instances. Here is the quick screenshot of the instance size selector.

As one can see, the costs certainly do rise with instance sizes. At $1.20/hr the 30GB instance will cost users approximately $876/ month. Let’s take a look at how the instances perform. We are including a few smaller Amazon AWS EC2 instances as a point of comparison as well as the Intel Atom S1260 Centerton SoC dedicated server.

Rackspace Cloud Benchmarks

We are utilizing our new Linux benchmark suite and Ubuntu Server 12.04 LTS to take a look specifically at the compute performance of these instances.

Hardinfo Performance

hardinfo is a well known Linux benchmark that has been around for years. It tests a number of CPU performance aspects.

Here the general trend is that the larger instances are providing performance speed-ups in most cases. The overall test result set that the heatmap is generated from utilizes a wide array of hardware and cloud instance types. The green shading on the 8GB to 30GB Rackspace instances shows that these instances are performing nearer to the top of the pack.

UnixBench 5.1.3 Performance

UnixBench may be a defacto standard for Linux benchmarking these days. There are two main versions, one that tests single CPU performance on that tests multiple CPU performance. UnixBench segments these results. We run both sets of CPU tests. Here are the single threaded results:

In the single threaded benchmark we see two things happening. First, the Rackspace cloud instances often have very similar performance due to the same underlying architecture. Second, we see some variability in the test numbers. For each cloud instance type test we are using five instances (with the same underlying hardware) and five iterations each. The sample size is not huge, but large enough that we are seeing variations in the Rackspace cloud (and Amazon cloud for that matter.) The important thing to remember here is that if one wants very consistent performance dedicated hardware will be a bit better.

Here we see significant performance gains moving up to the 8GB Rackspace Cloud instance. Likewise we are seeing the increments continue at the 15GB instance type and 30GB instance type.

c-ray 1.1 Performance

c-ray is a very interesting ray tracing benchmark. It provides both consistent results and some clear separation. Ray tracing is generally a great multithreaded CPU benchmark. For this test we use both a simple 7500×3500 render and a more complex 1920×1200 render. Here are the Rackspace cloud instances running the benchmark:

Here we see a nice distribution of cloud instance performance. Traditionally ray tracing is an application where scaling is shown very well and that proves true here. The sample instance types above show the CPU scaling we would expect from scaling the Rackspace Cloud instance size.

Crafty Chess Performance

Crafty is a well known chess benchmark. It is also one where we saw issues last time with the Phoronix Test Suite and running on ARM CPUs. Here are the Crafty Chess results from simply running “crafty bench”:

In terms of the simple “crafty bench” test, we are clearly seeing the Rackspace cloud showing a near flat line performance based on underlying CPU architecture.

Phoronix Test Suite Performance

We are using four tests from the Phoronix Test Suite: pts/stream, pts/compress-7zip, pts/openssl and pts/pybench.

• STREAM by John D. McCalpin, Ph.D. is a very well known benchmark. So much so that the benchmark is highly optimized when results are presented. Vendors use the STREAM benchmark to show architectural advantages. Needless to say, oftentimes these results are hard to reproduce unless one is trying to squeeze every ounce of performance from a machine they are optimizing for. Since the rest of the industry does that, we are taking a different approach. One installation run across all systems. For this we are using the Phoronix Test Suite version pts/stream. Specifically we are using the lowest bandwidth figure Triad test to compare memory throughput. 
• 7-zip compression benchmarks were a mainstay in our Windows suite so we are including it again on the Linux side as a compression benchmark. 
• The pts/openssl benchmark is one we found a small issue on with the Amazon AWS EC2 t1.micro instance earlier. The micro instance had nice burst speed but fell off dramatically after that. Looping the test gave consistent results. 
• Python is a widely used scripting language. In fact, it is arguably becoming even more popular these days so we decided to include a pyBench benchmark in the results.

Here are the results of the Phoronix Test Suite benchmarks:

As one can see the larger instances are peaking around 9GB/s memory bandwidth, much higher than most of our previous tests. In terms of raw compression speed we are seeing nice scaling. The pts/openssl benchmark is oftentimes dictated by architecture, so the fact that we see nearly flat results across the Rackspace cloud is largely a function of the fact that each instance tested was an Opteron 4170 HE series instance. Likewise, pts/pybench is showing very linear results. The important thing to note here is that the underlying architecture of any cloud instance can have a significant impact on scaling.

Conclusion

Key takeaways from the Rackspace Cloud benchmarks are that instance speed does indeed increase as one scales instances. Unlike the 512MB to 1GB and 2GB to 4GB increments where performance is largely the same between the Rackspace cloud instances, 8GB to 30GB scaling is well defined. This makes sense as the Rackspace cloud instance documentation clearly shows where extra compute resources are added.

One other point to remember is that we are looking at raw compute with these benchmarks.