Of BBQ and Virtualization Why Large Nodes Reign

12
Of Virtualzation And BBQ Cover
Of Virtualzation And BBQ Cover

Given the US holiday weekend coming up, we have a fun piece on virtualization in the context of a favorite 4th of July activity: BBQ (or barbecue/ barbeque depending on who you ask.) A question we often are asked at STH is why are lower-end single-socket nodes such as Intel Xeon E-2100 and Xeon E-2200 servers not more popular. After all, in many product segments, the lower-cost products are also the higher volume product segments. That is not the case with virtualization servers, and we are going to have a bit of fun and use BBQ to explain why.

Of BBQ and Virtualization Video

This year we are doing more video content and this one was easy enough to do so we took the cameras outside and took a few snaps to illustrate the point.

We always have an article to accompany our videos for those who prefer to read. Normally I advocate that the article is more in-depth. In this case, the video may be more fun.

Of BBQ and Virtualization: Why Large Nodes Reign

Setting up the common question we get, “why does the industry gravitate to larger servers for virtualization?” Indeed, the Xeon E-2200 series often offers a lower cost per each GHz its CPU cores are running at than higher-end Xeon Platinum SKUs. You can see this in our Intel Xeon E-2200 Series SKU List and Value Analysis and recent 3rd Gen Intel Xeon Scalable Cooper Lake SKU List and Value Analysis pieces. In some markets, like the EDA market, it is actually common to use clusters of these lower-end CPUs to take advantage of that price/ performance ratio.

Dell EMC PowerEdge R240 Internal
Dell EMC PowerEdge R240 Internal

There are a number of reasons that despite higher costs per clock, the larger Xeons dominate the virtualization market. A big part of that is simple consolidation of NICs and the rest of a data center network stack. One has fewer per-node licenses, management ports, cables, PDU ports, motherboards, backplanes, and a denser footprint. There is however one case that is so unique, that it goes beyond mere cost drivers. That case is larger VMs. That is where BBQ comes in to illustrate.

Supermicro 2049P TN8R CPU Area Light
Supermicro 2049P TN8R CPU Area Light

For those who have not seen some of my posts, a few weeks ago, I purchased my first smoker. It is a Pit Boss 4 Pro Series vertical wood pellet smoker that costs around $400 at Lowes. The vertical pellet smoker seems like a great idea. There are five racks with around 1300 square inches of cooking area or about 260 square inches each.

Pit Boss Pro 4 Series Smoker
Pit Boss Pro 4 Series Smoker

Like looking at lower-cost higher clock speed CPUs, the square inches per dollar you get with this is relatively amazing. In a way, this is like having five cooking surfaces that are small like having a cluster of five small servers.

Pit Boss Pro 4 Series Smoker In Action
Pit Boss Pro 4 Series Smoker In Action

For those that are into smoking meats and BBQ and are thinking why not an offset smoker, there is a simple explanation. If I had to watch an offset smoker, I might use it once a year. With a pellet smoker, it is easy to set and go do other things such as edit articles for STH so I thought I would use it more.

Smoked 28 Day Dry Aged Steak
Smoked 28 Day Dry Aged Roast

For the first few weeks, the Pit Boss Pro 4 series worked very well. I was cooking baby back ribs, small briskets, hot links, pulled pork, you name it very easily. My results were great.

BBQ Beef Ribs
BBQ Beef Ribs

Then I ran into a challenge. It all started with a day I wanted to cook some amazing briskets and ribs. Even though I had seen either no briskets or possibly one brisket at Costco (and none at our local supermarkets) for weeks, on a single trip I managed to use my three beef item limit to get three large briskets, and some pork spare ribs.

35lbs Of Briskets For BBQ
35lbs Of Briskets For BBQ

When I got home, I quickly realized why vertical smokers are not more popular and why my 1300 square inches of cooking area was not the same as a trailer smoker with the same cooking area.

Starting with the beautiful USDA Prime 14lb brisket, about a pound or two larger than anything I had cooked thus far, I quickly noticed that it was going to just barely fit in the smoker, and likely not fit properly. Even in its vacuum-sealed bag, it was touching on one side and just barely fit on the other. While you typically trim a lot off of a full packer brisket like this, it usually has a larger footprint after trimming than it has in the bag.

Pit Boss Smoker USDA Prime 14lb Brisket Barely Fitting Still In Bag
Pit Boss Smoker USDA Prime 14lb Brisket Barely Fitting Still In Bag

If we compare that USDA Prime brisket to a virtualization server, then that brisket would be maybe a 62GB virtual machine running in a smaller 64GB Xeon E-2200 series server. In our example, cooking grates are going to be the servers and VMs are the meat. That brisket may or may not fit but it is right on the edge of what will fit, just like the 62GB VM.

HPE ProLiant DL20 Gen10 Internal Roadmap
HPE ProLiant DL20 Gen10 Internal Roadmap

Then when we got to the ribs, one can see that they were a bit too large. They may fit with a little bit of trimming since they are close, but we make a trade-off to do so. Let us call the ribs a 64GB VM in a 64GB machine. Of course, there is overhead, so that is just a little bit too big.

Pit Boss Smoker Spare Ribs Too Big Left Side
Pit Boss Smoker Spare Ribs Too Big Left Side

Now the other two packer briskets here are something special. They are both beautiful Imperial American Wagu Reserve briskets. They are big and have excellent intermuscular fat which will lead to a juicy brisket. One is just barely under 16lbs and the other is just barely under 19lbs.

American Wagyu Reserve Brisket 16lb
American Wagyu Reserve Brisket 16lb

If you were curious, brisket comes from around a cow’s chest. There are generally two muscles in these packer briskets that you can think of like pectoral muscles on a human. Since cows walk on all fours, these get worked a lot and are tough. That is why they are common for smoking. The American Wagyu means that these briskets come from cows that are bred partly from Japanese Wagyu cow lineage imported years ago along with American cattle. That is why they tend to have more marbling and a different taste.

American Wagyu Reserve Brisket 19lb
American Wagyu Reserve Brisket 19lb

As you can see, these briskets even at Costco were not inexpensive.

Since these are big briskets, they are like big and valuable virtual machines. Indeed, when we tried to fit them on the small cooking grates of the Pit Boss smoker, they did not even come close to fitting. This is just like virtual machines. If you have a VM that needs 96GB or 128GB of memory, then there is no way it will fit on a 64GB of RAM machine.

Pit Boss Pro 4 Series American Wagyu Reserve Brisket 16lb And 19lb
Pit Boss Pro 4 Series American Wagyu Reserve Brisket 16lb And 19lb

At this point, I came to a tangential similarity between BBQ and virtualization. With around $400 of meat to cook, plus fuel, accessories, and time, the least expensive part of the entire operation was actually the smoker itself. So I abruptly went out and got a Green Mountain Grills Jim Bowie Prime Plus just as one would get a larger server.

GMG Jim Bowie Empty
GMG Jim Bowie Empty

Now at first, the Jim Bowie Prime Plus may seem like a much smaller unit with only 658 square inches of cooking area it almost sounds like half the size. Instead, that 658 square inches are all on just one cooking surface. As a result, the two 16lb and 19lb briskets fit on the cooking surface with plenty of room to spare. This is similar to how one can have enough memory to fit larger VMs in a dual-socket server that would not fit in a cluster of smaller nodes.

Gigabyte H262 Node Rear CPU Heatsink With RAM
Gigabyte H262 Node Rear CPU Heatsink With RAM

You may be thinking sure you can fit the larger briskets (or VMs) onto the larger grill (or server) but you surely cannot fit as much. Simply using a few grill racks, we can easily fit all of the meat from the Pit Boss in the GMG, and in fact more.

GMG Jim Bowie With All The Meat
GMG Jim Bowie With All The Meat

Like using a larger virtualization server instead of a cluster of smaller nodes, we get more efficient use of space. For example, if one has only 48GB VMs but a 64GB server, there is a fairly large amount of wasted space. That is similar to the cluster of smaller cooking surfaces where we had five racks with extra space where we could not fit other cuts, but we still had briskets and ribs that did not fit.

GMG Jim Bowie Full Next To Pit Boss Pro 4 Series
GMG Jim Bowie Full Next To Pit Boss Pro 4 Series

A key reason that cloud providers are so successful in virtualization is that they are very good at filling up all of the space in the larger footprint servers. Just like using an array of grill racks in the smoker to get higher space utilization.

GMG WiFi Controlled App
GMG WiFi Controlled App

Before we close this out, there is one other aspect to the comparison we can compare. The new GMG Jim Bowie Prime Plus has integrated WiFi. You can see the grill temperatures and the meat probe temperatures from your house or from even the grocery store. You can turn the grill temperature up or down, remotely turn it on or off, and you get temperature increments of a single degree. In comparison, the Pit Boss is not designed to get anywhere near as hot, and you only get 25F temperature increments with manual controls at the device.

This is just like larger virtualization nodes where because of consolidation, you can afford to use more robust networking, storage, and other elements because those costs are being deferred over a larger number of VMs.

Dell EMC PowerEdge R740xd Internal Overview And CPUs
Dell EMC PowerEdge R740xd Internal Overview And CPUs

Final Words

This was a fun piece for the holiday weekend. Still, it illustrates that clearly, and conclusively that virtualization is just like BBQ. You can even make a living being very good at one or the other. Of course, BBQ usually tastes better than a VMware boot drive.

BBQ Ribs Brisket Link Day
BBQ Ribs Brisket Link Day

At the end of the day, economies of scale in virtualization, like smokers, mean that it is more efficient to use larger systems instead of clusters of smaller systems. Those larger nodes drive higher efficiencies and help deliver features and functionality that are not possible on smaller systems.

BBQ Brisket 1
BBQ Brisket 1

If you are at a cookout this summer, feel free to pass this along to show how grills and smokers are just like virtualization.

12 COMMENTS

  1. Would it be possible to cut the brisket in half so it fits in two single socket servers?

    What if the briskets are owned by noncooperative entities who need single-tenant nodes to avoid side-channel leakage of secret sauces in order to cook their food? In such cases, smaller physical nodes would provide the needed physical isolation and security without requiring each tenant to rent a huge cooker that they don’t need all for themselves.

  2. Life is like a box of chocolates, your HBA is going to fail.

    Thanks Patrick, it’s nice to see you guys have a sense of humor about yourselves.

  3. I saw this and I’ve gone out and bought a GMG JB Prime. It’s cheaper than servers!

    You’re right. That’s a fun piece.

  4. Wow. One can say much bad and clichè things about Amercian cuisine, but concerning meat and BBQ you are surely on top of the world. 😉
    Just made my first pulled porked days ago and it was amazing (poor mans version in an oven with low temps).
    And now i am even more sad i dont have a smoker or a place for it living in a flat without a garden. 😉

    Bon appetit and a good 4th of July (even though i am too late). 🙂

  5. It appears you may have purchased beef beyond its sale date (or froze it, which is worse).

    You may want to remove or cover your price stickers ($111.49 and 132.32 !!!).

    Finally, when did you last have your cholesterol checked, Patrick?

  6. @Eric Olson:
    That depends on what you are asking. If you want to span the VM across two hosts, that isn’t possible. A workload on the other can can be split or clustered between multiple hosts. You see that in HPC, massive DBs, and Super Computers. At the scale you are looking at, this isn’t feasible.

    For your second question there is already a fix for that in VMware. If you are running ESXi 6.7 U2 or later on Intel hosts you have a choice of two different setups. One choice restricts VMs to physical cores only and while that VM is using those CPU resources everything else is locked from accessing that CPU. This turns your 8c/16t CPU into an 8c/8t CPU, there by cutting your capacity in half. The second option keeps Hyperthreading but locks the VM to physical cores and that core’s hyperthreads. Alternatively one can use Epyc CPUs and not have this issues.

  7. We bought a davy crockett this weekend. The only reason is this article. It was great for the 4th of July barbecue

  8. So hungry now….
    All I have round me are servers – I tried eating the the smoked innards of one, but its just not the same as smoked brisket. Even with sauce.
    I may be one of those people that has difficulties with analogies.

    An amusing piece, enjoyable, thanks!

    Of course, it could be sumarized:
    ‘Big servers are popular because you can fit occasional huge VMs if you need to, or efficiently divide the space into smaller VM’s. A cluster of small servers simply can’t fit the occasional large VM, and you loose efficiency at the hardware boundaries if your VM’s don’t fit neatly into each small server’

    But thats a lot less fun. 🙂

    Seriously – I want a piece of your $132 brisket! Now!

LEAVE A REPLY

Please enter your comment!
Please enter your name here