Our first two colocation builds were relatively low power affairs. Both our Las Vegas, Nevada facility and Fremont, California facility offer 15A 120V of power per cabinet. Our latest facility in Sunnyvale, California has 208V 30A per rack. In our original lab and in the subsequent facilities, testing has traditionally occurred on 120V circuits which are standard in the USA. With 208V we have noticed some significant changes.
120V or 208V which is better?
120V (if you live in the USA) is extremely easy to find. 15A 120V wiring has been standard for residential and many office buildings for some time now. In the Silicon Valley, California, many small offices are abuzz with the sound of servers and workstations because 120V is available.
In terms of power supply efficiency, 208V is generally around 2% more efficient with figures commonly quoted in the 1-3% range. That degree of impact is noticeable but is not necessarily a game changer, especially in low density racks. The difference between 200w and 204w for a server is often negligible on a single server level. At a rack level, with 5000w usable (as an example) 2% better efficiency might mean about 100w difference, or enough to power a handful of very low power nodes.
Operating at a higher voltage requires less current to drive the same amount of power. For example, a 400w dual processor server might require around 3.33A on a 120V circuit but would only require about 1.92A on a 208V circuit. The simple formula you can use for most basic “plug a box into a datacenter PDU” scenarios is: Volts x Amps = Watts. Here is an example of how a 400w server varies based on input voltage.
Given the varying power situations servers are installed in, we now have the ability to test on both 208V and 120V racks for our testing. Moving up to 220V-240V generally yields another 0.25% or so gain in efficiency. There are more exotic power setups but if you are using standard colocation, there is a good chance you will use one of these options.
How does this impact how much power I can use for my servers and networking gear?
The difference between 120V and 208V practically is that 208V circuits tend to deliver more power and are (generally) designed to be more robust. One must take into account that only about 80% of the total power will be considered usable for a variety of reasons. Take two common examples:
- The standard 15A 120V circuit has capacity for 1.8kW * 80% = 1.44kW
- The standard 30A 208V circuit has capacity for 6.24kW * 80% = (about) 5kW
That is a huge difference. Also do remember that the 208V is more power efficient to the tune of 2%. 120V is considered a lower density option but is readily available. An example of this is the 8x GPU server running in Bank 1 of our Schneider Electric APC AP8441 PDU is using 10.3A or roughly 10.3A * 208V = 2143w.
When you have systems this large, they cannot be run on a standard 15A 120V circuit.
120V or 208V which can my server/ storage/ switch use?
Generally speaking, most modern servers are rated for 100-240V or 250V operation, much like the majority of today’s consumer electronics. The simple fact is that building a global power supply is oftentimes more cost efficient than building a narrow voltage input range power supply. There is great benefit in being able to simply match a power cord and get running. Do check the power supply (or manual) but if the included cable has a IEC320 C13 end, then you are likely OK on power.
While servers may have power supplies that can span voltage ranges, the power distribution unit and battery backup side of things can be a bit different. Those components due to plugs and inefficiencies are often rated at much narrower voltage ranges (e.g. 208V only, 208-240V and etc.) When we started building out the new Sunnyvale, California facility, we used Schneider Electric APC AP8441 PDUs which are rated for 200V/208V/230V 30A input.
This was not meant to be the most comprehensive guide on the subject by any means, but it should help with an extremely high-level overview of why one would use a higher-power rack and some of the implications. We do now have the capability to test machines in both low power 120V 15A environments as well as higher power 208V 30A environments. There are many more higher power and higher efficiency options out there but using general off the shelf hardware and colocation these are common options. One note is that lower-end gear with external power supplies may work fine on 208V but may require an adapter to work in the higher power PDUs. For example, the North American shipping Mikrotik CRS226-24G-2S+RM has an external power supply which is not easy to power using a standard IEC320 C14 outlet. More on this in the near future.