Kicking off this week is the International Supercomputing Conference in Leipzig, Germany, one of the two major supercomputing/high performance computing conferences of the year. There will be several announcements coming out of ISC this week – a few of which we’ll see later this week – but perhaps the most visible announcement that comes out of ISC is the annual summer refresh of the Top500 supercomputer list.

With that in mind, the June 2013 Top500 list has been released, and as is usually the case a new supercomputer has entered the fray at the top of the list. The latest addition and new #1 supercomputer is the Chinese National University of Defense Technology's Tianhe-2 supercomputer, which hits the list with a measured Linpack performance of 33.86PFLOPS, nearly double the previous #1, Oak Ridge National Laboratory’s Titan. Like Titan, Tianhe-2 is another hybrid system, using a custom interconnect to tie together numerous systems containing both traditional Intel Xeon CPUs and Intel’s Xeon Phi co-processors.

Top500 Top 5 Supercomputers
Supercomputer Architecture Performance (Rmax, TFLOPS) Power Consumption (kW) Efficiency (MFLOPS/W)
Tianhe-2 Xeon + Xeon Phi 33862.7 17808 1901.5
Titan Opteron + Tesla 17590.0 8209 2142.7
Sequoia BlueGene/Q 17173.2 7890 2176.5
K Computer SPARC64 10510.0 12660 830.1
Mira BlueGene/Q 8586.6 3945 2176.5

Tianhe-2 is not the first Xeon Phi system in the list, but this marks the first time that a Xeon Phi system has taken the top spot. Next to the AMD Opteron plus NVIDIA Tesla based Titan, the #1 spot in recent editions of the list has also been held by IBM BlueGene/Q systems, and SPARC64 based systems, making Tianhe-2 the 4th architecture in 2 years to take the top spot. Tianhe-2 means that Xeon Phi now occupies two of the top 10 positions, with the previously launched Stampede system moving up the list from #7 to #6.

Looking at the latest data, what’s particularly astounding about Tianhe-2 is simply how large it is. Placing on the Top500 list requires both efficiency and brute force, and in the case of Tianhe-2 there’s an unprecedented amount of brute force in play. The official power consumption rating for Tianhe-2 is 17.8 megawatts, more than double Titan’s 8.2MW. Even going back several years, the second largest machine to hit the Top500 list – and thus far the only other machine over 10MW – is the K supercomputer, at 12.6MW. Simply put, from a power perspective Tianhe-2 is nearly 50% larger than any previous Top500 computer.

When it comes to efficiency the matter won’t technically be settled until the formal unveiling of the Green500 list later this week, but since it’s just a derivation of the Top500 it’s fairly easy to calculate. To that end while Tianhe-2 isn’t quite as efficient as the BlueGene/Q and Tesla systems that sit in the top 5, both of which are just over 2,100 MFLOPS/watt, at 1901 MFLOPS/watt Tianhe-2 is at least in the same general category and generation as these other systems. With all of these supercomputers interconnects play a big part in performance, so for a system as large as Tianhe-2 that’s especially true. Unfortunately very few details have been published about the custom interconnect Tianhe-2 uses, so there’s very little to say on the matte other than that whatever it is works well enough to scale well over Tianhe-2’s 16,000 nodes.

Wrapping things up, with the introduction of Tianhe-2 this marks the second hybrid system in as many editions of the list to take the top spot. Although hybrid systems have been in the Top500 for a few years now, they have continued to become more popular due to the high efficiency and density of GPUs and GPU-like processors. Though straight CPU systems such as BlueGene/Q and its ilk are by no means out of the running, it will be interesting to see if this is part of a larger trend. According to the Top500 list the number of hybrid systems is actually down – from 62 on the last list to 54 now – but on the other hand the continued improvement and increasing flexibility of GPUs and other co-processors, not to mention the fact that now even Intel is involved in this field, means that there’s more effort than ever going into developing these hybrid systems. So whether hybrids will continue to push out straight CPU systems at the top of the list will be an interesting thing to keep an eye on over the next couple of years.

Source: Top500; Tianhe-2 Image Courtesy Top500/Jack Dongarra

POST A COMMENT

16 Comments

View All Comments

  • marimvibe - Tuesday, June 18, 2013 - link

    It does, but compared to the two twelve core Xeons and 3 Phis, the interconnect power consumption is going to be miniscule.

    The BlueGene/Q systems are still my favorite. They've actually hit 14 PFlops in a benchmark run of a real application (not just Linpack,) and have actually used a million cores at once. The Sequoia system has as much memory as all of Tianhe-2. Until we get Graph500 results for Titan or Tianhe-2, I'm not going to be impressed. The entire architecture of BlueGene - the chips, the boards, the network topology, etc., are all designed to be scalable. Titan, Tianhe-2, etc., just hook everything up to a big router. Give me a torus over a fat tree any day.
    Reply
  • Ktracho - Monday, June 17, 2013 - link

    I wonder how much of the performance improvement is due to the Xeon CPUs vs. the AMD CPUs in the Titan. I've heard AMD does better in terms of floating-point performance per dollar, and perhaps even FP performance per watt, but I would imagine Intel could do better in terms of absolute FP performance. In other words, perhaps the increase in performance is not really due using Xeon Phi vs. NVIDIA Tesla, and maybe the decreased efficiency is due (at least partially) to Xeon CPUs? In any case, I don't think Cray is allowed to sell their computers to China. Reply
  • protomech - Monday, June 17, 2013 - link

    Here's a nice counter-point: In 2004 I helped build one of the bigger TOP500 computers. We saw about 20 MFLOPS/watt per node, 16 MFLOPS/watt for the entire system (not counting power used for facility cooling). Reply
  • hammer256 - Monday, June 17, 2013 - link

    So 2 orders of magnitude improvement in efficiency in 9 years. That's pretty sweet. Reply
  • marimvibe - Monday, June 17, 2013 - link

    Graph500 or GTFO. Reply
  • 1008anan - Tuesday, June 18, 2013 - link

    Ryan, it might be worth mentioning that the Tianhe-2 represents the first introduction of multi-socket Xeon Ivy Bridge Xeon E5-2692.Xeon Ivy Bridge E5s also powered the 54th and 330th fastest supercomputers on this list. No Xeon Ivy Bridge E5 models are listed on the latest Intel Price List.

    Tianhe-2 is the first ever top 10 supercomputer that consists entirely of processors fabricated at 22 nm.

    To my surprise there was no Oracle Sparc T5 supercomputer in the 500 list.

    Some other interesting observations:
    1) only 26 petaflop supercomputers on this list, up from 23 petaflop supercomputers in the last list. (would have thought that more than 3 petaflop supercomputers were purchased in 7 months)
    2) the 500th most powerful supercomputer only had 96.6 teraflops, up a measly 26% from 76.5 teraflops in the last list. This is the second straight list with modest growth in teraflops for the 500th most powerful system. What far will this trend go?
    3) The sum of computational power for all 500 supercomputer systems was 223 petaflops, up 38% from 162 petaflops in the November, 2012. (the November 2012 list showed in increase of 31% from 123 petaflops in the June, 2012 list)
    4) By performance, the most powerful supercomputer had 15.2% of the combined petaflops of all 500 supercomputers. This is almost certainly a record high percentage since the 500 list was first introduced in 1993.
    Reply

Log in

Don't have an account? Sign up now