China Might Beat the Rest of the World to an Exascale Supercomputer

China's rise to the top of the supercomputing world has been swift. It's only been in this decade that the country first claimed bragging rights to the world's most powerful supercomputer. It has held a tight grip on those rights the last few years. The United States is expected to reclaim the top spot later this year, as it is nearing completion of Summit, a supercomputer that is expected to be twice as powerful as its closest rival. Nevertheless, China is widely expected to produce the first supercomputer capable of carrying out 1 billion billion calculations per second, or 1 exaflop. If this exascale supercomputer comes online in 2020 as promised, that could put it a year or more ahead of similar exascale systems now being developed in the United States, Japan, and the European Union.

link.

China's new Home-grown Supercomputer has a Theoretical Peak Performance of Almost 125 Petaflops

China on Monday revealed its latest supercomputer, a monolithic system with 10.65 million compute cores built entirely with Chinese microprocessors. This follows a U.S. government decision last year to deny China access to Intel's fastest microprocessors.

There is no U.S.-made system that comes close to the performance of China's new system, the Sunway TaihuLight. Its theoretical peak performance is 124.5 petaflops, according to the latest biannual release today of the world's Top500 supercomputers. It is the first system to exceed 100 petaflops. A petaflop equals one thousand trillion (one quadrillion) sustained floating-point operations per second.

The most important thing about Sunway TaihuLight may be its microprocessors. In the past, China has relied heavily on U.S. microprocessors in building its supercomputing capacity. The world's next fastest system, China's Tianhe-2, which has a peak performance of 54.9 petaflops, uses Intel Xeon processors.

link.

You Will NOT be Uploaded: a Most Excellent Rant about The Human Brain Project

This is a guest post [on a blog I am linking to, ed] by a neuroscientist who may or may not be a graduate student somewhere in Massachusetts.

You asked me about the Human Brain Project. Well, there is only one way to properly address that topic: with a rant.

Henry Markram at EPFL in Switzerland was the leader of the “Blue Brain” project, to simulate a brain (well, actually just one cubic millimeter of a mouse brain) on an IBM Blue-Gene supercomputer. He got tons of money for this project, including the IBM supercomputer for the simulations. Of course he never published anything showing that these simulations lead to any understanding of brain function whatsoever. But he did create a team of graphics professionals to make cool pictures of the simulations. Building on this “success”, he led the “Human Brain” EU flagship project into being funded by some miracle of bureaucratic gullibility. The clearly promised goal was simulating a human brain (hence the name of the project). Almost everyone in Europe publicly supported the project, although in private the neuroscientists (who, if they have done any simulations, know that the stated goal is completely absurd) would say something more like “hey, maybe it’s crazy, but it’ll bring a bunch of money.”

Now, some simple observations must be made, which are true now, and will still be true in ten years’ time, at the conclusion of this flagship project

link.

Simulating ~31,000 Neurons, .21% of a Rat's Brain Required a Supercomputer

Reconstruction and Simulation of Neocortical Microcircuitry

Authors:

Markram et al

Abstract:

We present a first-draft digital reconstruction of the microcircuitry of somatosensory cortex of juvenile rat. The reconstruction uses cellular and synaptic organizing principles to algorithmically reconstruct detailed anatomy and physiology from sparse experimental data. An objective anatomical method defines a neocortical volume of 0.29 ± 0.01 mm3 containing ∼31,000 neurons, and patch-clamp studies identify 55 layer-specific morphological and 207 morpho-electrical neuron subtypes. When digitally reconstructed neurons are positioned in the volume and synapse formation is restricted to biological bouton densities and numbers of synapses per connection, their overlapping arbors form ∼8 million connections with ∼37 million synapses. Simulations reproduce an array of in vitro and in vivo experiments without parameter tuning. Additionally, we find a spectrum of network states with a sharp transition from synchronous to asynchronous activity, modulated by physiological mechanisms. The spectrum of network states, dynamically reconfigured around this transition, supports diverse information processing strategies.

When we get an Exascale Computer?

The global race to build more powerful supercomputers is focused on the next big milestone: a supercomputer capable of performing 1 million trillion floating-point operations per second (1 exaflops). Such a system will require a big overhaul of how these machines compute, how they move data, and how they’re programmed. It’s a process that might not reach its goal for eight years. But the seeds of future success are being designed into two machines that could arrive in just two years.

China and Japan each seem focused on building an exascale supercomputer by 2020. But the United States probably won’t build its first practical exascale supercomputer until 2023 at the earliest, experts say. To hit that target, engineers will need to do three things. First they’ll need new computer architectures capable of combining tens of thousands of CPUs and graphics-processor-based accelerators. Engineers will also need to deal with the growing energy costs required to move data from a supercomputer’s memory to the processors. Finally, software developers will have to learn how to build programs that can make use of the new architecture.

“To some degree it depends on how much money a country is willing to spend,” says Steve Scott, senior vice president and chief technology officer at Cray. “You could build an exaflop computer tomorrow, but it’d be a crazy thing to do because of the cost and energy required to run it.”

link.

More on the IARPA's Cryogenic Supercomputer Effort

The Intelligence Advanced Research Projects Activity (IARPA) has officially commenced a multi-year research effort to develop a superconducting computer as a long-term solution to the power, cooling and space constraints that afflict modern high-performance computing. First revealed in February 2013, when the agency put out a call for proposals, the Cryogenic Computer Complexity (C3) program aims to pave the way for a new generation of superconducting supercomputers that are far more energy efficient than machines based on complementary metal oxide semiconductor (CMOS) technology.

Studies indicate the technology, which uses low temperatures in the 4-10 kelvin range to enable information to be transmitted with minimal energy loss, could yield one-petaflop systems that use just 25 kW and 100 petaflop systems that operate at 200 kW, including the cryogenic cooler. Compare this to the current greenest system, the L-CSC supercomputer from the GSI Helmholtz Center, which achieved 5.27 gigaflops-per-watt on the most-recent Green500 list. If scaled linearly to an exaflop supercomputing system, it would consume about 190 megawatts (MW), still quite a bit short of DARPA targets, which range from 20MW to 67MW.

link.

Beyond Exascale Supercomputers: IARPA Launches Cryogenic Computer Complexity Program

American intelligence agencies announced plans Friday to develop and build a new superconducting supercomputer, one which would increase current computing capacity while simultaneously reducing the energy consumption and physical footprint of the machines.

The Intelligence Advanced Research Projects Activity, a branch of the U.S. intelligence community, said in a press release that the agency has embarked on a multi-year research effort called the Cryogenic Computer Complexity program, or C3.

Current supercomputing utilizes technology that relies on tens of megawatts and requires large amounts of physical space to house the infrastructure and power and cool the components.

C3 hopes to use recent breakthroughs in supercomputing technologies — "new families of superconducting logic without static power dissipation and new ideas for energy efficient cryogenic memory" — to construct a superconducting supercomputer with "a simplified cooling infrastructure and a greatly reduced footprint."

"The power, space, and cooling requirements for current supercomputers based on complementary metal oxide semiconductor (CMOS) technology are becoming unmanageable," said Marc Manheimer, C3 program manager at IARPA.

"Computers based on superconducting logic integrated with new kinds of cryogenic memory will allow expansion of current computing facilities while staying within space and energy budgets, and may enable supercomputer development beyond the exascale," Manheimer said.

link.

cryogenics and hpc?  this will only end well.

LBNL-NERSC Climate Models now at 25 km Resolution, Generate 100 Terabytes of Data per run

Not long ago, it would have taken several years to run a high-resolution simulation on a global climate model. But using some of the most powerful supercomputers now available, Lawrence Berkeley National Laboratory (Berkeley Lab) climate scientist Michael Wehner was able to complete a run in just three months.

What he found was that not only were the simulations much closer to actual observations, but the high-resolution models were far better at reproducing intense storms, such as hurricanes and cyclones. The study, "The effect of horizontal resolution on simulation quality in the Community Atmospheric Model, CAM5.1," has been published online in the Journal of Advances in Modeling Earth Systems.

"I've been calling this a golden age for high-resolution climate modeling because these supercomputers are enabling us to do gee-whiz science in a way we haven't been able to do before," said Wehner, who was also a lead author for the recent Fifth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC). "These kinds of calculations have gone from basically intractable to heroic to now doable."

Using version 5.1 of the Community Atmospheric Model, developed by the Department of Energy (DOE) and the National Science Foundation (NSF) for use by the scientific community, Wehner and his co-authors conducted an analysis for the period 1979 to 2005 at three spatial resolutions: 25 km, 100 km, and 200 km. They then compared those results to each other and to observations.

One simulation generated 100 terabytes of data, or 100,000 gigabytes. The computing was performed at Berkeley Lab's National Energy Research Scientific Computing Center (NERSC), a DOE Office of Science User Facility. "I've literally waited my entire career to be able to do these simulations," Wehner said.

The higher resolution was particularly helpful in mountainous areas since the models take an average of the altitude in the grid (25 square km for high resolution, 200 square km for low resolution). With more accurate representation of mountainous terrain, the higher resolution model is better able to simulate snow and rain in those regions.

"High resolution gives us the ability to look at intense weather, like hurricanes," said Kevin Reed, a researcher at the National Center for Atmospheric Research (NCAR) and a co-author on the paper. "It also gives us the ability to look at things locally at a lot higher fidelity. Simulations are much more realistic at any given place, especially if that place has a lot of topography."

link.

IBM Wins Two Supercomputer Contracts Worth $325 Million

In a Department of Energy deal worth $325 million, IBM will build two massive supercomputers called Sierra and Summit that combine a new supercomputing approach from Big Blue with Nvidia processing accelerators and Mellanox high-speed networking.

The companies and US government agency announced the deal on Friday ahead of a twice-yearly supercomputing conference that begins Monday. The show focuses on the high-end systems -- sometimes as large as a basketball court -- that are used to calculate car aerodynamics, detect structural weaknesses in airplane designs and predict the performance of new drugs.

The funds will pay for two machines, one for civilian research at the Oak Ridge National Laboratory in Tennessee and one for nuclear weapons simulation at the Lawrence Livermore National Laboratory in California. They'll each clock in with a peak performance surpassing 100 petaflops -- that's a quadrillion calculations per second as measured in the Top500 list that ranks the world's fastest machines. Trying to do that with modern laptops would take something like 3 million of them, Nvidia estimates.

link.

Japan Selects Fujitsu to Build Exaflop Supercomputer by 2021

Japan has chosen Fujitsu to help it regain the top spot in the global supercomputer race with an exascale machine, which at 1,000 petaflops would be about 30 times faster than the leading supercomputer today.

The electronics giant said Wednesday it will work with the Riken research center to come up with a basic design for the supercomputer that would succeed the K computer, a machine they co-developed that grabbed the No. 1 spot in June 2011.

Riken has a mandate from Japan’s Ministry of Education, Culture, Sports, Science and Technology (MEXT) to develop a next-generation supercomputer, and said it chose Fujitsu following an open bidding process to develop a “post-K supercomputer.”

Fujitsu will initially collaborate with Riken on a basic design, working towards beginning operation of the post-K computer supercomputer by April 2021.

The Riken Advanced Institute for Computational Science did not specify a performance speed or other characteristics of the machine, which it is calling the FLAGSHIP 2020 Project.

link.

Monster Sized, 55,000 Solar Mass First Stars Left no Blackhole

Certain primordial stars—those between 55,000 and 56,000 times the mass of our Sun, or solar masses—may have died unusually. In death, these objects—among the Universe’s first-generation of stars—would have exploded as supernovae and burned completely, leaving no remnant black hole behind.

Astrophysicists at the University of California, Santa Cruz (UCSC) and the University of Minnesota came to this conclusion after running a number of supercomputer simulations at the Department of Energy’s (DOE's) National Energy Research Scientific Computing Center (NERSC) and Minnesota Supercomputing Institute at the University of Minnesota. They relied extensively on CASTRO, a compressible astrophysics code developed at DOE's Lawrence Berkeley National Laboratory’s (Berkeley Lab’s) Computational Research Division (CRD). Their findings were recently published in Astrophysical Journal (ApJ).

First-generation stars are especially interesting because they produced the first heavy elements, or chemical elements other than hydrogen and helium. In death, they sent their chemical creations into outer space, paving the way for subsequent generations of stars, solar systems and galaxies. With a greater understanding of how these first stars died, scientists hope to glean some insights about how the Universe, as we know it today, came to be.

“We found that there is a narrow window where supermassive stars could explode completely instead of becoming a supermassive black hole—no one has ever found this mechanism before,” says Ke-Jung Chen, a postdoctoral researcher at UCSC and lead author of the ApJ paper. “Without NERSC resources, it would have taken us a lot longer to reach this result. From a user perspective, the facility is run very efficiently and it is an extremely convenient place to do science.”

link.

paper link.

Professor Banned for Mining Bitcoin on NSF Supercomputers

The National Science Foundation has banned a researcher for using supercomputer resources to generate Bitcoin.

In the semiannual report to Congress by the NSF Office of Inspector General, the organization said it received reports of a researcher who was using NSF-funded supercomputers at two universities to mine Bitcoin.

Mining is a process to generate the digital currency that involves complex calculations. Bitcoin can be converted to traditional currencies, and 1 Bitcoin was worth roughly $654 on Friday, according to indexes on CoinDesk.

The computationally intensive mining took up about $150,000 worth of NSF-supported computer use at the two universities to generate bitcoins worth about $8,000 to $10,000, according to the report. It did not name the researcher or the universities.

The universities told the NSF that the work was unauthorized, reporting that the researcher accessed the computers remotely, even using a mirror site in Europe, possibly to conceal his identity.

The researcher said he was simply conducting tests, Inspector General Allison Lerner’s office wrote in the report, which covers six months to March 31.

“The researcher’s access to all NSF-funded supercomputer resources was terminated,” the office wrote. “In response to our recommendation, NSF suspended the researcher government-wide.”

link.

Meet Cori: The $70 Million Next Gen Supercomputer for the Day job @ NERSC

The U.S. Department of Energy’s (DOE) National Energy Research Scientific Computing (NERSC) Center and Cray Inc. announced today that they have signed a contract for a next generation of supercomputer to enable scientific discovery at the DOE’s Office of Science (DOE SC).

Lawrence Berkeley National Laboratory (Berkeley Lab), which manages NERSC, collaborated with Los Alamos National Laboratory and Sandia National Laboratories to develop the technical requirements for the system.

The new, next-generation Cray XC supercomputer will use Intel’s next-generation Intel® Xeon Phi™ processor –- code-named “Knights Landing” -- a self-hosted, manycore processor with on-package high bandwidth memory and delivers more than 3 teraFLOPS of double-precision peak performance per single socket node. Scheduled for delivery in mid-2016, the new system will deliver 10x the sustained computing capability of NERSC’s Hopper system, a Cray XE6 supercomputer.

NERSC serves as the DOE SC’s primary high performance computing (HPC) facility, supporting more than 5,000 scientists annually on over 700 projects. The $70 million plus contract represents the DOE SC’s ongoing commitment to enabling extreme-scale science to address challenges such as developing new energy sources, improving energy efficiency, understanding climate change, developing new materials and analyzing massive data sets from experimental facilities around the world.

[...]

To highlight its commitment to advancing research, NERSC names its supercomputers after noted scientists. The new system will be named “Cori” in honor of bio-chemist and Nobel Laureate Gerty Cori, the first American woman to receive a Nobel Prize in science.

Technical Highlights

Cori the supercomputer will have over 9300 Knights Landing compute nodes and provide over 400 gigabytes per second of I/O bandwidth and 28 petabytes of disk space. The contract also includes an option for a “Burst Buffer,” a layer of NVRAM that would move data more quickly between processor and disk, allowing users to make the most efficient use of the system while saving energy. The Cray XC system features the Aries high-performance interconnect linking the processors, which also increases efficiency. Cori will be installed directly into the new Computational Research and Theory facility currently being constructed on the main Berkeley Lab campus.

link.

End of a Personal Era

waiting to be disposed

This past week I uninstalled a lot of DDN 9550s.  These were the workhorse storage platforms we used for our global filesystem for years.  Too long, in fact, but budgets are what budgets are and storage normally gets the short end of the stick at HPC centers.  However, the 9550s were really, really stable.  They used 300 GB fiber channel drives to come to a total of somewhere around 720 terabytes of disk.  My loss rate for them was less than 10% - I believe somewhere around 2%, actually - of the newer 9900s with the 1 TB SATA drives.

my row of 9900s

The DDN 9900s are fine, really.  I manage 11 of them with those 1 TB SATA drives (see below).  This comes to about 3.3 petabytes worth of our global filesytems.  Interestingly, the problems which I have are not seen by my compatriots who work with the SAS drives.  However, their SAS drives are 300 GB, so if I were to substitute mine with those drives, while more reliable, they would give a capacity hit down to less than one petabyte. Those are not the newest systems, obviously, but they are useful.  Even if they give me most of my headaches.

my row of sfa12kes, only first 4 racks are mine

These days the new toys are my sfa12kes.  These are embedded servers with virtual machines which run on the disk controllers.  They have been remarkably stable and nonfussy.  I have my issues with them still (specifically their software stack (dude, rolling upgrades, really! need! this!)), but DDN is making progress on its development.  The five 12kes use 3 TB near-line SAS drives which gives me 5.1 petabytes of spinning disk (and I'd like to fill them out to their full capacity for 6 PB, but, again, a budgetary issue.  We spec'ed them for a specific bandwidth/capacity ratio.  Unlike the previous two types of DDN pictures here, not only are the 12kes with embedded servers, they have some spiffy FDR infiniband cards.  Half are tasked to be 10 Gig ethernet, a quarter are FDR IB and the rest of QDR IB (legacy systems).  Plans are afoot to move everything onto FDR IB and use ethernet gateways for the legacy systems.  We have to get rid of the QDR systems though.

 

  seismic isolation planks where the 9550s were

 

At any rate, this is an end of a personal era at NERSC.  No more FC drives for me.   And a glimpse of some of what I do at the dayjob.

Fast Eddy: Our New Supercomputer (at the day job) and some sad news

link.

This is tempered by the fact a coworker died.  He had a medical emergency at work and despite the efforts of folks here went into a coma.  He passed on friday.  I've known him for almost 13 years.  He even interviewed me when I came here and was very kind after a grueling day of interviews.  We worked together on the beginnings of the nersc global file system and then later when he was in the tape systems.  He always had a funny story or gentle, yet snarky comment which would lift my spirits.  He is survived by his wife.

Mike, we will miss you.  And, Mike, I already do.

What a 2 Petaflop (sustained) Supercomputer Looks like Once it is Skinned

More Details of the Environmental Degradation of the Permian Triassic Extinction

Acid rain and ozone depletion from pulsed Siberian Traps magmatism

Authors:

Black et al.

Abstract:

The Siberian Traps flood basalts have been invoked as a trigger for the catastrophic end-Permian mass extinction. Widespread aberrant plant remains across the Permian-Triassic boundary provide evidence that atmospheric stress contributed to the collapse in terrestrial diversity. We used detailed estimates of magmatic degassing from the Siberian Traps to complete the first three-dimensional global climate modeling of atmospheric chemistry during eruption of a large igneous province. Our results show that both strongly acidic rain and global ozone collapse are possible transient consequences of episodic pyroclastic volcanism and heating of volatile-rich Siberian country rocks. We suggest that in conjunction with abrupt warming from greenhouse gas emissions, these repeated, rapidly applied atmospheric stresses directly linked Siberian magmatism to end-Permian ecological failure on land. Our comprehensive modeling supplies the first picture of the global distribution and severity of acid rain and ozone depletion, providing testable predictions for the geography of end-Permian environmental proxies.

No Exoflops for You Part Duo: Moore's Law *IS* in Decline & it Hurts Supercomputers

Supercomputing users are relentless in their pursuit of compute power so they can run simulations of increasing complexity and scale to tackle mankind's truly big problems. But Moore's Law, once a reliable predictor of computing power's future, has reached its limits.

Supercomputing researchers aren't sure what's next.

Today, supercomputing relies on architectural changes, such as adding speedy GPUs, to boost performance. Researchers may increasingly turn to chips that integrate interconnects and memory to speed processing and reduce energy.

But the teams must also wrestle with the enormous costs of building -- and running -- multi-petaflop systems.

"We have reached the end of the technological era," said William Gropp, chairman of the SC13 conference and a computer science professor at the University of Illinois at Urbana-Champaign.

Gropp likened the supercomputer development terrain today to the advent of CMOS (complementary metal oxide semiconductor), the foundation of today's standard semiconductor technology. The arrival of CMOS was disruptive, but it fostered an expansive age of computing.

The problem is "we don't have a technology that is ready to be adopted as a replacement for CMOS," said Gropp. "We don't have anything at the level of maturity that allows you to bet your company on."

link.

Will Supercomputers Massively Change Material Science?

Engineered materials such as chip-grade silicon and fiber-optic glass underpin the modern world. Yet designing new materials has historically involved a frustrating and inefficient amount of guesswork.

Streamlined versions of the equations of quantum mechanics—along with supercomputers that, using those equations, virtually test thousands of materials at a time—are eliminating much of that guesswork.

Researchers are now using this method, called high-throughput computational materials design, to develop new batteries, solar cells, fuel cells, computer chips, and other technologies.

link.

Modeling the Locomotion of Sauropod Argentinosaurus huinculensis (with video)

March of the Titans: The Locomotor Capabilities of Sauropod Dinosaurs

Authors:

William Irvin Sellers, Lee Margetts, Rodolfo Anı´bal Coria and Phillip Lars Manning

Abstract:

Sauropod dinosaurs are the largest terrestrial vertebrate to have lived on Earth. This size must have posed special challenges for the musculoskeletal system. Scaling theory shows that body mass and hence the loads that must be overcome increases with body size more rapidly than either the ability of the muscles to generate force, or the ability of the skeleton to support these loads. Here we demonstrate how one of the very largest sauropods, Argentinosaurus huinculensis (40 metres long, weighing 83 tonnes), may have moved. A musculoskeletal model was generated using data captured by laser scanning a mounted skeleton and assigning muscle properties based on comparative data from living animals. Locomotion is generated using forward dynamic simulation to calculate the accelerations produced by the muscle forces, coupled with machine learning technique to find a control pattern that minimises metabolic cost. The simulation demonstrates that at such vast body size, joint range of motion needs to be restricted to allow sufficient force generation for an achievable muscle mass. However when this is done, a perfectly plausible gait can be generated relatively easily. Whilst this model represents the best current simulation of the gait of these giant animals, it is likely that there are as yet unknown mechanical mechanisms, possibly based on passive elastic structures that should be incorporated to increase the efficiency of the animal9s locomotion. It is certainly the case that these would need to be incorporated into the model to properly assess the full locomotor capabilities of the animal.