Wednesday, February 04, 2009

LLNL's Sequoia Planned as 20 PetaFlops

Roadrunner and Jaguar, the DOE supercomputers that launched the petaflop era last year, will soon be eclipsed by new machines more than ten times as powerful. IBM and the US National Nuclear Security Administration (NNSA) announced on Tuesday that in 2011 Lawrence Livermore National Laboratory will install a 20 petaflop system to provide computational support for the country's aging nuclear weapons.

Building on its Blue Gene heritage, IBM will deliver "Dawn," a 500 teraflop Blue Gene/P system in the first quarter of this year, followed by "Sequoia," a 20 petaflop next-generation Blue Gene/Q machine for 2011. Sequoia is expected to officially go online in 2012. The new machines will take over Lawrence Livermore's weapon simulation codes that are being maintained under the Advanced Simulation and Computing (ASC) Program. Currently this work is being done with the existing capability supercomputers at the lab: the 100 teraflop ASC Purple and the 600 teraflop Blue Gene/L.

Dawn will act as an interim platform for porting and scaling the weapons codes. Once the Blue Gene/Q super comes online, those codes will be moved over to the bigger machine for production. The Dawn machine is in the process of being built right now, with about half of the machine already wired together at Lawrence Livermore. The lab is planning on getting the rest of the hardware over the next few months, with system acceptance scheduled for April.

[...]

Not only will Sequoia be more than ten times as powerful as the current crop of petaflop supercomputers, its energy efficiency will be much improved. According to IBM Deep Computing VP Dave Turek, Sequoia will consume around 6 megawatts, yielding an energy efficiency ratio of 3,000 FLOPS/watt. That represents a 7X improvement over the Blue Gene/P generation (440 FLOPS/watt), and is even better than the Cell-based Roadrunner system at Los Alamos (587 FLOPS/watt). For a starker comparison, the 1.6 petaflop Opteron-based Jaguar supercomputer installed at Oak Ridge National Laboratory uses about 8.5 megawatts (188 FLOPS/watt).

[...]

IBM is not releasing low-level details of the Blue Gene/Q architecture. However, since Sequoia will be composed of 98,304 compute nodes and contain a total of 1.6 million cores, one can surmise that a Blue Gene/Q node will contain 16 cores. Whether this is implemented as one 16-core chip or two 8-core chips (or even four quad-core chips) remains to be seen. Since Sequoia will sport 1.6 petabytes of memory, each node stands to have 16 GB. The current Blue Gene/P technology offers 4 cores and 4 GB of main memory per node.


1.6 million cores. I don't want to sound like a poo-poo-er, but... 1.6 MILLION CORES. There are exceedingly few algorithms or problem sets that will scale to that level. Climate. Some explosions. There are great deal of users here that are on the verge of being shuffled off to non HPC assets. If Sequoia is the wave of the future, these guys are toast. There's no way to gang together their moderately parallel apps into ones that will work on something that scale.

Some scattered notes and thoughts: per core memory is 1 GB (note: if you have more than 2 you are idling your CPU big time, folks). 400k processors means there will be thousands of them failing at a time: with the so-called 5 9s of reliability (99.999%) there will be 4 CPUs down at any given time. More likely, with a 97%, then 20k CPUs will be down and out. Flops per core doesn't seem to have improved much (no surprise). Power per core is improved though.

In the era of Sequoia, assuming it works, even Franklin (our Cray) will no longer be an HPC asset really. Supercomputer status is a moving target. Keep up or end up someone's desktop.

No comments: