As high-performance computing systems incorporate higher-wattage processors, liquid cooling is gaining momentum.
The annual Supercomputing (SC18) conference is always a great showcase for leading-edge technologies for high-performance computing, and that will certainly be the case this year as SC18 gets underway in Dallas next week. One of these technologies in the spotlight will be liquid cooling, which is emerging as one of the keys to even-more-powerful HPC systems.
A convergence of trends is fueling the momentum for liquid cooling. For starters, the thirst for higher throughput in HPC clusters is unquenchable. With every new generation of systems, HPC users expect faster processing resulting in more heat, requiring more efficient cooling. Over the years, system builders have met this need with air cooling, yet data centers continue to push the limits of rack density. Processor capacity and wattages are also starting to bump up against the limits of conventional approaches to cooling.
Air cooling has worked OK for systems with CPUs that top out around 150 watts, which is what we have had for the past decade or so. But today, we’re seeing more HPC systems with processors that run at 200W and above, and the years ahead will bring even higher wattage processors. These are causing people to take a serious look at liquid cooling, which is a more efficient way to cool off the processors.
Among the organizations that have opted for liquid cooling is the Texas Advanced Computing Center (TACC), which is building a massive new supercomputer called Frontera. The primary system, supplied by Dell, will have more than 8,000 dual-socket Intel® Xeon® nodes with 205W processors, all packed into 91 racks. For a system of this size and wattage, air cooling is no longer practical. Instead, the primary compute racks will leverage liquid cooling from CoolIT, a Dell partner.
CoolIT’s Direct Contact Liquid Cooling (DCLC™) technology uses the exceptional thermal conductivity of liquid to provide dense, concentrated cooling to targeted surface areas. The technology uses warm or room-temperature water to cool CPUs, eliminating the need for chilled water and reducing the cost of cooling by as much as 56% over traditional cooling methods.
So how does it work? The concept is pretty simple, actually. The technology uses a metal plate, or cold plate, that functions as a heat sink. This plate is placed in direct contact with the processor casing surface. Water flows through micro-channels in the plate to capture heat from the processor, and then flows on to a heat exchanger, which dissipates the heat. The cooled water coming out of the heat exchanger travels back through the system, and the cycle repeats itself. It is a hermetically sealed closed system, so the processors are not in direct contact with the water.
While liquid cooling is a new concept for some people in the IT world, it’s actually been all around us for years. Liquid cooling is used widely in desktop systems with GPUs, which run hot — the current generation runs at 300W, and GPUs are expected to get even hotter in subsequent generations. Liquid cooling has also been used widely in desktop computers and various other devices.
At Dell, we now provide DCLC technology from CoolIT in our PowerEdge C6420 server, which is built for complex and demanding HPC and scale-out workloads. Liquid cooling allows the server to support higher wattage processors for increased performance, energy efficiency and rack-level density. If you’re at SC18, you can get a firsthand look at these servers and the cooling technology in the Dell booth.
So, is liquid cooling the only future for HPC systems? I wouldn’t go that far, but I would ask how many cars are still air cooled? I would also say that liquid cooling is a future for HPC, and an important one at that. The more we push for higher performance in systems, the more wattage we require in processors, and that equates to more heat. Liquid cooling is one of the ways to deal with that heat in an efficient and cost-effective manner.
For a closer look at liquid cooling in an HPC server, take a look at the Dell-CoolIT solutions brief. Find out a bit more in this quick video with CoolIT systems. You can also see liquid cooling in the HPC and AI Innovation Lab.
Follow the action at SC18 on Twitter at @Dellservers
