China Might Beat the Rest of the World to an Exascale Supercomputer

China's rise to the top of the supercomputing world has been swift. It's only been in this decade that the country first claimed bragging rights to the world's most powerful supercomputer. It has held a tight grip on those rights the last few years. The United States is expected to reclaim the top spot later this year, as it is nearing completion of Summit, a supercomputer that is expected to be twice as powerful as its closest rival. Nevertheless, China is widely expected to produce the first supercomputer capable of carrying out 1 billion billion calculations per second, or 1 exaflop. If this exascale supercomputer comes online in 2020 as promised, that could put it a year or more ahead of similar exascale systems now being developed in the United States, Japan, and the European Union.

link.

When we get an Exascale Computer?

The global race to build more powerful supercomputers is focused on the next big milestone: a supercomputer capable of performing 1 million trillion floating-point operations per second (1 exaflops). Such a system will require a big overhaul of how these machines compute, how they move data, and how they’re programmed. It’s a process that might not reach its goal for eight years. But the seeds of future success are being designed into two machines that could arrive in just two years.

China and Japan each seem focused on building an exascale supercomputer by 2020. But the United States probably won’t build its first practical exascale supercomputer until 2023 at the earliest, experts say. To hit that target, engineers will need to do three things. First they’ll need new computer architectures capable of combining tens of thousands of CPUs and graphics-processor-based accelerators. Engineers will also need to deal with the growing energy costs required to move data from a supercomputer’s memory to the processors. Finally, software developers will have to learn how to build programs that can make use of the new architecture.

“To some degree it depends on how much money a country is willing to spend,” says Steve Scott, senior vice president and chief technology officer at Cray. “You could build an exaflop computer tomorrow, but it’d be a crazy thing to do because of the cost and energy required to run it.”

link.

Computer Science Seminar at LBNL Tomorrow: Krylov-based Methods for Future Extreme Computing

CS Seminar: Toward Smart-tuned Krylov-based Methods for Future Extreme Computing

Berkeley Lab – Computing Sciences Seminar

Date: Thursday, August 15, 2013

Time: 11:00am - 12:00pm

Location: Bldg. 50F, Room 1647

Speaker: Serge G. Petiton

Maison de la Simulation/CNRS and University Lille 1, Sciences et Technologies

Title: Toward Smart-tuned Krylov-based Methods for Future Extreme Computing

Abstract:

Exascale hypercomputers are expected to have highly hierarchical architectures with nodes composed by lot-of-core processors and accelerators. The different programming levels (from clusters of processors loosely connected to tightly connected lot-of-core processors and/or accelerators) will generate new difficult algorithm issues. New methods should be defined and evaluated with respect to modern state-of-the-art of applied mathematics and scientific methods.

Krylov linear methods such as GMRES and ERAM are now heavily used with success in various domains and industries despite their complexity. Their convergence and speed greatly depends on the hardware used and on the choice of the Krylov subspace size and other parameters which are difficult to determine efficiently in advance. Moreover, hybrid Krylov Methods would allow reducing the communications along all the cores, limiting the reduction only through subsets of these cores. Added to their numerical behaviours and their fault tolerance properties, these methods are interesting candidates for exascale/extreme matrix computing. Avoiding communication strategies may also be developed for each of the instance of these methods, generating complex methods but with high potential efficiencies. These methods have a lot of correlated parameter which may be optimized using auto/smart-tuning strategies to accelerate convergence, minimize storage space, data movements, and energy consumption.

In this talk, we first will present some basic matrix operations utilized on Krylov methods on clusters of accelerators, with respect to a few chosen sparse compressed formats. We will discuss some recent experiments on a cluster of accelerators concerning comparison between orthogonal, incompletely orthogonal and non-orthogonal Krylov Basis computing. Then, we will discuss some results obtained on a cluster of accelerators to compute eigenvalues using the MERAM method with respect to the restarting strategies. We will survey some auto/smarttunning strategies we proposed and evaluated for some of the Krylov method parameters. As a conclusion, we will propose auto-tuning strategies for future hybrid methods on post-petascale computers, on the road to exascale hybrid methods.

Joint wok with: Nahid Emad (U. Versailles), Leroy Drummond (LBNL), France Boillod and Christophe Calvin (CEA), Langshi Chen (CNRS), Maxime Hugu

Still No Exacscale For You (Nvidia's Bill Dally Weighs in)

NVIDIA’s Chief Scientist, Bill Dally, tackled this issue in the ISC conference keynote address he delivered at the big event, entitled “Future Challenges of Large-scale Computing.”

Presenting to some of the high performance computing (HPC) industry’s foremost experts, Dally outlined challenges the industry needs to overcome to reach exascale by the end of this decade.

It boils down, in his view, to two major issues: power and programming.

It’s About Power, Forget Process

Theoretically, an exascale system – 100 times more computing capability than today’s fastest systems – could be built with only x86 processors, but it would require as much as 2 gigawatts of power.

That’s the entire output of the Hoover Dam.

On the other hand, the GPUs in an exascale system built with NVIDIA Kepler K20 processors would consume about 150 megawatts. So, a hybrid system that efficiently utilizes CPUs with higher-performance GPU accelerators is the best bet to tackle the power problem.

GPUs are something I pushed here at the the Lab from the retirement of the  Cray vector machines until moving over to the data systems side.  However, CPU programming is still problematic in many ways.  Yet, this could be rectified.  On the flipside...how many sites can STILL afford 150 MW of power for a single machine.  Our new building won't be able to handle 300 MW of power (we keep two big machines on the floor at a time...)

I am refining the no singularity idea for the post.  Friends have seen the extremely rough outline of the idea in email.  it needs a couple more mental iterations before being committed to a post. 

The Race to Exascale...Fear Mongering Version!

Unlike China and Europe, the U.S. has yet to adopt and fund an exascale development program, and concerns about what that means to U.S. security are growing darker and more dire.

China's retaking of the global supercomputing crown was the starting point for discussion at an IBM-sponsored congressional forum this week on cognitive computing.

Cognitive computing systems have the capability of taking vast amounts of data and making what will be, for all intents, thoughtful decisions.

Efforts to draw attention to exascale in the U.S. House are being led Rep. Randy Hultgren (R-Ill.), who talked about China's new 33.89-petaflop system, Tianhe-2.

"It's important not to lose sight that the reality was that it was built by China's National University of Defense Technology," said Hultgren, who is finalizing a bill "that will push our nation toward exascale."

Hultgren is introducing legislation, the American Supercomputing Leadership Act, to require the U.S. Department of Energy to develop a coordinated exascale research program. The bill doesn't call for a specific spending level, but one source said about an annual appropriation of $200 million, if not more, will be sought.

That amount of money is well short of what's needed to build an exascale system, or a computer of 1,000 thousand petaflops. Each petaflop represents one thousand trillion floating point operations per second.

Earl Joseph, an HPC analyst at IDC, said that "$200 million is better than nothing, but compared to China and Europe it's at least 10 times too low."

Joseph said that it's his guess that the world will see an exascale system by 2015 or 2016 "installed outside the U.S. It will take a lot of power and it will be large, but it will provide a major capability."

I've seen this a few times before.  First it was the evil japanese (back in the early 90s through the early 00s) and now its the scary chinese!

I have to snigger at Mr Joseph's time frame.  oy.  The contract will have to already been signed...and...uh.