Power
Powering the next-generation of HPC systems is proving to be a limiting-factor when applied towards the growth of the HPC and computational science industries. For far too long, computer performance has been based on the raw measurement of speed in flop/s (floating point operations per second); while largely ignoring other performance metrics such as power efficiency, space efficiency, reliability, availability and usability. Based on this model, HPC systems have been on an ever increasing, yet unsustainable trajectory in their requirements for electrical power.
Current trends in microprocessor design have recognized this "energy crisis", and are addressing the need to maximize performance-per-watt as one of their primary goals. System optimization from a software perspective will also have to align with this current generation of multicore energy-efficient (EE) processors. Therefore, the next several years should see an increase in software parallelization, as more applications migrate away from inefficient serial algorithms and as multi-socket/mutlicore computing becomes standard. With that said, software design as a component of energy-efficiency in modern data centers is beyond the scope of this page. So one of the easiest ways to maximize energy efficiency is to look at the available power distribution models and determine the most energy efficient ways of delivering and converting power as required by the individual HPC components (i.e. servers).
Power distribution to HPC equipment can be accomplished using AC or DC power. Even with the "War of Currents" being revived, AC power is the most commonly used type of power; and is typically distributed at the local mains in voltages of 120 V, 208 V, or 240 V. DC power is more often used for telecommunications, with a standard voltage of 48 V. The actual energy efficiency of either AC or DC at various voltages is the subject of much debate. In the meantime it is important to understand your facility's ability to provide electrical power (and at what voltages), your cluster's power requirements and the most energy efficient ways to deliver that power via Uninterruptible Power Supplies (UPS), Power Distribution Units (PDU) and/or Power Supply Units (PSU).
UPS – The Uninterruptible Power Supply (UPS) converts unconditioned power to provide conditioned power
to critical loads. It contains an energy storage system, such as batteries, that can supply power to the load
when utility power is unavailable.
PDU – The main function of a Power Distribution Unit (PDU) is to house circuit breakers that are used to create
multiple branch circuits from a single feeder circuit. A secondary function of some PDUs is to convert voltage.
The AC voltage-converting PDUs contain either an isolation transformer or an autotransformer to step the
AC distribution voltage down to the power supply input voltage.
PSU – The Power Supply Unit (PSU) converts an input AC voltage to a regulated 12V DC output voltage.
Common power configurations for U.S. Data Centers:
480/208V AC Power Distribution
The present day
480/208V AC distribution design is stepped down by PDU transformers to three-phase 208/120V AC distribution for the IT loads. This configuration can be very inefficient, with up to 40% power utilization lost, if older/legacy power components are in use. However, by using the most efficient UPS and PDU components with this configuration, the overall
efficiency can reach 80%-85%.
480/277V AC Power Distribution
Utilities deliver three-phase 480/277V AC to many commercial and
IT facilities. These voltages are measured from Line-Neutral (277
V AC) and Line-Line (480 V AC). Instead of stepping down to
208V, the voltage remains constant through the PDU at 277V AC, improving the
efficiency of this distribution by up to 85-90%, depending on the load. Ideally, the solution is to auto transform the incoming 480V down to 415V, thus achieving the 240V which can be directly utilized by IT equipment. The alternative to reducing 480V to 415V, would be to have OEM manufacturers increase the voltage operating range of their equipment up to 277V. As most HPC equipment already operates at the 240V AC level, and with PSUs now universally adapting to input voltages of 120V or 240V, the 480V to 415V conversion may be the best option for now.
Common International Power configuration:
Another AC design, ubiquitous outside of North America and Japan, is the three-phase 400/230V AC. The 400/230V AC distribution eliminates the need for step down transformers, and if used in conjunction with high efficiency UPS and PSUs would greatly reduce power and the associated cooling costs of data centers. Some literature provides empirical results of 94% efficiency using this configuration.1 If this power configuration is available to you it should be leveraged for optimal power efficiency. 400/230V AC may become the standard for future data centers in the U.S., which would also help in standardizing power equipment across the globe.
Regardless of the power distribution available to you, it becomes paramount to upgrade and use the most efficient UPS and PDUs available. Aspen Systems can design your HPC system to operate at the most energy efficient level, while still delivering the highest performance necessary to run your applications.
See the Detailed Buyers Reference - Power section for a detailed discussion on 120V, 208V and 240V AC circuits, in addition to information about connection types for each circuit.
APC®, offers a number of free online Calculators that may also help you to further understand your energy requirements.
1 Rassmussen, N. (2007). A Quantitative Comparison of High Efficiency AC vs. DC Power Distribution for Data Centers.
Retrieved Septmeber 17, 2009, from APC Corporate Site: http://www.apcmedia.com/salestools/NRAN-76TTJY_R1_EN.pdf
