Cineca’s Tesla K20-Based “Eurora” Supercomputer Unveiled; Water Cooling Unlocks Extra Efficiency
by Ryan Smith on January 31, 2013 7:40 PM ESTWe typically don’t cover a lot of supercomputing news outside of the major Top500/Green500 announcements due to the fact that the launches of so many supercomputers are clustered around the official updates to those lists, but the following came across our desk earlier this afternoon from NVIDIA and caught our attention.
Italy’s Cineca computing center has unveiled their latest supercomputer today, a Eurotech created system built on a combination of Intel’s Xeon E5 CPUs and NVIDIA’s Tesla K20 GPUs. While the supercomputer itself is still considered a prototype and is fairly modest overall – its 110 TFLOPS would be good for roughly #250 on the most recent Top500 list – what makes this latest supercomputer particularly interesting (and just a bit snark-worthy) is not its peak performance, but rather its energy efficiency and cooling mechanism. In short, someone has finally paired a K20-based supercomputer with water cooling, with some interesting results.
Cineca's and Eurotech's Water Cooled Eurora Supercomputer
By using water cooling, Eurotech and Cineca have been able to capture the waste heat off of Eurora in a far more dense and more useful manner than air would allow, in turn allowing them to put that waste heat to productive use. By using that heat to not only heat their buildings but also to drive absorption chillers, they’ve been able to reduce the amount of heat that ultimately needs to be removed from the system, and consequently the amount of energy that is spent on cooling.
Adding an extra facet to this is the fact that due to the much greater heat capacity of water over air, using water for cooling – even warm-to-hot water as in the case of Eurora – results in lower processor temperatures than chilled air. In turn, thanks to the positive relationship in semiconductors between temperature and power consumption, these lower temperature processors consume less electricity and thereby produce less heat in the first place.
The end result is that between the use the water cooling and various other optimizations made by Eurotech, they’ve been able to produce a supercomputer significantly more power efficient than other K20 supercomputers, or any other supercomputer for that matter. At 110 TFLOPS performance for 34.7KW electricity consumption, Eurora has an energy efficiency of 3,150 MFLOPS/watt, almost 25% better than the current #1 system on the Green500 supercomputer list, a Xeon Phi system that operates at 2,500 MFLOPS/watt. This makes Eurora the most energy efficient supercomputer available by a wide margin for the moment, and while the next Green500 list won’t be published until June, if Eurora is accepted then will be a likely contender for the top spot on that list.
We won’t dwell on this too much more, but if you would like some more details there’s an interesting slide deck from Eurotech published over on NVIDIA’s website. Ultimately using waste heat for productive use is something of a one-trick pony – there aren’t too many productive ways to use that heat due to its relatively low temperature – but nevertheless it offers an interesting alternative into improving supercomputer performance within a fixed power envelope, particularly at a time where we’re beginning to see diminishing returns from smaller semiconductor manufacturing processes.
Source: NVIDIA
Facebook
Twitter
RSS
17 Comments
View All Comments
ShieTar - Friday, February 1, 2013 - link
I don't think it needs to be much more than would already circulate in a conventional gasburning heating system. You just replace the heating section by the rack. I'm sure the pipe-system of a usual office building is more extensive in terms of volume than the actual cooling system for the computer.jack.fxx - Sunday, February 3, 2013 - link
If you still remember something from high school physics courses, based on temperature difference and input power of the supercomputer, you can easily calculate flow rate of cooling water.Gadgety - Friday, February 1, 2013 - link
Great to see water cooling improving and getting an industrial slant rather than the homebuilt it was just 10 years ago. I'm a novice to water cooling, aiming for GPU and CPU loops and an external radiator in a separate room. I'll be getting heat in the basement as a bonus. No, it won't heat it a lot, but the basement is always cooler so the cooling will be more efficient. Provided the flow rates, connections, pumps etc are appropriate. This is where I find its gets tricky. I concur with Ian Cutress "tIt would be interesting to see the efficiency or flow rates of the pumps used, or the reservoir/pump combinations along the line."maximumGPU - Friday, February 1, 2013 - link
Nice Article!to all experienced watercoolers out there.. how silent does your system run?
I want to jump into watercooling but i'm big on silent pcs as well. At the moment i'm running a 3570K with a large Silver Arrow heatsink, together with the incredibly silent Asus DCUII GTX 670. So i end up with good temps and very low noise levels. My question is, with watercooling, could i possibly get even cooler temps for the same noise level as my air setup?
Death666Angel - Friday, February 1, 2013 - link
Yes? WC can always to cooler running and more quiet than any air coolers out there. The question is how much you want to spend, if you want to mount your radiators inside the case or if you are fine with external radiators etc. And if you have the space, you can always put the external radiator in another room where the noise doesn't bother anyone.I guess if you use a 480+ radiator with your rig, you could achieve very nice temperatures with minimal noise. How much better temps/more quiet operation? No idea, I don't know how loud your current rig is, how much you want to spend etc. Go visit a water cooling forum and get some info there. :)
MrSpadge - Saturday, February 2, 2013 - link
Intuitively I assume they could lower their power consumption further by running the 20°C water in parallel through the racks, instead of letting the water heat up serially. Depending on how many racks they're using (not many, for the prototype, I guess.. probably even just a single one) they could lower the operating temperature of the last racks significantly.ShieTar - Monday, February 4, 2013 - link
Sure that would lower the rack temperature, but you would end up with a lot of 25°C water useless for the purpose of heating. The main point of the setup is to use the heat from the racks for a purpose that needs heat anyways, in this example heating of office space. And that requires a reasonable temperature delta to be achieved. It's this kind of systematic approach to energy that helps to lower overall energy consumption without having any drawback whatsoever that is horribly under-utilized in todays world.Just think of the fridge in your home cooling the inside while heating the kitchen air even in summer, instead of heating your watersupply for your next shower. If I ever get around to building my own home, I will definitly try to connect all heat-generating systems to a common waterccoling/heating system.