Walking Away

NERSC Supercomputer Seminar Tomorrow Webcast, Public may Join

CS/NERSC Seminar: NERSC's relationship with the Joint Genome Institute

Berkeley Lab – Computing Sciences Seminar Series
Date:
Tuesday, May 12, 2015

Time:
12:00pm - 1:00pm

Location:
NERSC OSF 943, Conference Room 238

Speaker:
Kjiersten Fagnan
JGI-NERSC

Title:
NERSC's relationship with the Joint Genome Institute

Abstract:

Have you ever wondered what the JGI does? Or why they generate so many tickets? Or why they need so many large file systems? In thisbrown bag I'll give an overview of the JGI, the science they do and how NERSC supports them. I'll cover their data management system, JAMO, some of the workflows and pipelines they run to process data and talk a little bit about the joint projects we have to improve the JGI's scientific computing practices.

password is 'Cori'.

LBNL-NERSC Climate Models now at 25 km Resolution, Generate 100 Terabytes of Data per run

Not long ago, it would have taken several years to run a high-resolution simulation on a global climate model. But using some of the most powerful supercomputers now available, Lawrence Berkeley National Laboratory (Berkeley Lab) climate scientist Michael Wehner was able to complete a run in just three months.

What he found was that not only were the simulations much closer to actual observations, but the high-resolution models were far better at reproducing intense storms, such as hurricanes and cyclones. The study, "The effect of horizontal resolution on simulation quality in the Community Atmospheric Model, CAM5.1," has been published online in the Journal of Advances in Modeling Earth Systems.

"I've been calling this a golden age for high-resolution climate modeling because these supercomputers are enabling us to do gee-whiz science in a way we haven't been able to do before," said Wehner, who was also a lead author for the recent Fifth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC). "These kinds of calculations have gone from basically intractable to heroic to now doable."

Using version 5.1 of the Community Atmospheric Model, developed by the Department of Energy (DOE) and the National Science Foundation (NSF) for use by the scientific community, Wehner and his co-authors conducted an analysis for the period 1979 to 2005 at three spatial resolutions: 25 km, 100 km, and 200 km. They then compared those results to each other and to observations.

One simulation generated 100 terabytes of data, or 100,000 gigabytes. The computing was performed at Berkeley Lab's National Energy Research Scientific Computing Center (NERSC), a DOE Office of Science User Facility. "I've literally waited my entire career to be able to do these simulations," Wehner said.

The higher resolution was particularly helpful in mountainous areas since the models take an average of the altitude in the grid (25 square km for high resolution, 200 square km for low resolution). With more accurate representation of mountainous terrain, the higher resolution model is better able to simulate snow and rain in those regions.

"High resolution gives us the ability to look at intense weather, like hurricanes," said Kevin Reed, a researcher at the National Center for Atmospheric Research (NCAR) and a co-author on the paper. "It also gives us the ability to look at things locally at a lot higher fidelity. Simulations are much more realistic at any given place, especially if that place has a lot of topography."

link.

Monster Sized, 55,000 Solar Mass First Stars Left no Blackhole

Certain primordial stars—those between 55,000 and 56,000 times the mass of our Sun, or solar masses—may have died unusually. In death, these objects—among the Universe’s first-generation of stars—would have exploded as supernovae and burned completely, leaving no remnant black hole behind.

Astrophysicists at the University of California, Santa Cruz (UCSC) and the University of Minnesota came to this conclusion after running a number of supercomputer simulations at the Department of Energy’s (DOE's) National Energy Research Scientific Computing Center (NERSC) and Minnesota Supercomputing Institute at the University of Minnesota. They relied extensively on CASTRO, a compressible astrophysics code developed at DOE's Lawrence Berkeley National Laboratory’s (Berkeley Lab’s) Computational Research Division (CRD). Their findings were recently published in Astrophysical Journal (ApJ).

First-generation stars are especially interesting because they produced the first heavy elements, or chemical elements other than hydrogen and helium. In death, they sent their chemical creations into outer space, paving the way for subsequent generations of stars, solar systems and galaxies. With a greater understanding of how these first stars died, scientists hope to glean some insights about how the Universe, as we know it today, came to be.

“We found that there is a narrow window where supermassive stars could explode completely instead of becoming a supermassive black hole—no one has ever found this mechanism before,” says Ke-Jung Chen, a postdoctoral researcher at UCSC and lead author of the ApJ paper. “Without NERSC resources, it would have taken us a lot longer to reach this result. From a user perspective, the facility is run very efficiently and it is an extremely convenient place to do science.”

link.

paper link.

Meet Cori: The $70 Million Next Gen Supercomputer for the Day job @ NERSC

The U.S. Department of Energy’s (DOE) National Energy Research Scientific Computing (NERSC) Center and Cray Inc. announced today that they have signed a contract for a next generation of supercomputer to enable scientific discovery at the DOE’s Office of Science (DOE SC).

Lawrence Berkeley National Laboratory (Berkeley Lab), which manages NERSC, collaborated with Los Alamos National Laboratory and Sandia National Laboratories to develop the technical requirements for the system.

The new, next-generation Cray XC supercomputer will use Intel’s next-generation Intel® Xeon Phi™ processor –- code-named “Knights Landing” -- a self-hosted, manycore processor with on-package high bandwidth memory and delivers more than 3 teraFLOPS of double-precision peak performance per single socket node. Scheduled for delivery in mid-2016, the new system will deliver 10x the sustained computing capability of NERSC’s Hopper system, a Cray XE6 supercomputer.

NERSC serves as the DOE SC’s primary high performance computing (HPC) facility, supporting more than 5,000 scientists annually on over 700 projects. The $70 million plus contract represents the DOE SC’s ongoing commitment to enabling extreme-scale science to address challenges such as developing new energy sources, improving energy efficiency, understanding climate change, developing new materials and analyzing massive data sets from experimental facilities around the world.

[...]

To highlight its commitment to advancing research, NERSC names its supercomputers after noted scientists. The new system will be named “Cori” in honor of bio-chemist and Nobel Laureate Gerty Cori, the first American woman to receive a Nobel Prize in science.

Technical Highlights

Cori the supercomputer will have over 9300 Knights Landing compute nodes and provide over 400 gigabytes per second of I/O bandwidth and 28 petabytes of disk space. The contract also includes an option for a “Burst Buffer,” a layer of NVRAM that would move data more quickly between processor and disk, allowing users to make the most efficient use of the system while saving energy. The Cray XC system features the Aries high-performance interconnect linking the processors, which also increases efficiency. Cori will be installed directly into the new Computational Research and Theory facility currently being constructed on the main Berkeley Lab campus.

link.

End of a Personal Era

waiting to be disposed

This past week I uninstalled a lot of DDN 9550s.  These were the workhorse storage platforms we used for our global filesystem for years.  Too long, in fact, but budgets are what budgets are and storage normally gets the short end of the stick at HPC centers.  However, the 9550s were really, really stable.  They used 300 GB fiber channel drives to come to a total of somewhere around 720 terabytes of disk.  My loss rate for them was less than 10% - I believe somewhere around 2%, actually - of the newer 9900s with the 1 TB SATA drives.

my row of 9900s

The DDN 9900s are fine, really.  I manage 11 of them with those 1 TB SATA drives (see below).  This comes to about 3.3 petabytes worth of our global filesytems.  Interestingly, the problems which I have are not seen by my compatriots who work with the SAS drives.  However, their SAS drives are 300 GB, so if I were to substitute mine with those drives, while more reliable, they would give a capacity hit down to less than one petabyte. Those are not the newest systems, obviously, but they are useful.  Even if they give me most of my headaches.

my row of sfa12kes, only first 4 racks are mine

These days the new toys are my sfa12kes.  These are embedded servers with virtual machines which run on the disk controllers.  They have been remarkably stable and nonfussy.  I have my issues with them still (specifically their software stack (dude, rolling upgrades, really! need! this!)), but DDN is making progress on its development.  The five 12kes use 3 TB near-line SAS drives which gives me 5.1 petabytes of spinning disk (and I'd like to fill them out to their full capacity for 6 PB, but, again, a budgetary issue.  We spec'ed them for a specific bandwidth/capacity ratio.  Unlike the previous two types of DDN pictures here, not only are the 12kes with embedded servers, they have some spiffy FDR infiniband cards.  Half are tasked to be 10 Gig ethernet, a quarter are FDR IB and the rest of QDR IB (legacy systems).  Plans are afoot to move everything onto FDR IB and use ethernet gateways for the legacy systems.  We have to get rid of the QDR systems though.

 

  seismic isolation planks where the 9550s were

 

At any rate, this is an end of a personal era at NERSC.  No more FC drives for me.   And a glimpse of some of what I do at the dayjob.

Fast Eddy: Our New Supercomputer (at the day job) and some sad news

link.

This is tempered by the fact a coworker died.  He had a medical emergency at work and despite the efforts of folks here went into a coma.  He passed on friday.  I've known him for almost 13 years.  He even interviewed me when I came here and was very kind after a grueling day of interviews.  We worked together on the beginnings of the nersc global file system and then later when he was in the tape systems.  He always had a funny story or gentle, yet snarky comment which would lift my spirits.  He is survived by his wife.

Mike, we will miss you.  And, Mike, I already do.

We had a Hand in it...but Mostly it was Erik's: Using NERSC Supercomputers to Find Habitable Zone, Terrestrial Exoplanets

One out of every five sun-like stars in our Milky Way galaxy has an Earth-sized planet orbiting it in the Goldilocks zone—not too hot, not too cold—where surface temperatures should be compatible with liquid water, according to a statistical analysis of data from NASA’s Kepler spacecraft by Erik Petigura, a graduate student at the University of California, Berkeley (UC Berkeley).

Petigura and his colleague Andrew Howard, now at the University of Hawaii, Manoa, spent three years developing a transit search pipeline called TERRA that is optimized for finding small planets. When they used this tool on supercomputers at the Department of Energy’s (DOE’s) National Energy Research Scientific Computing Center (NERSC) to analyze nearly four years of Kepler observations, the scientists determined that our galaxy could contain as many as 40 billion habitable Earth-sized planets.

link.


background link.

What a 2 Petaflop (sustained) Supercomputer Looks like Once it is Skinned

Crane Installed for LBNL/NERSC's New Building Named (yes) CRT

link to webcam.

Yes, We Were Hacked

Yep.  Our supercomputer, Franklin (affectionately called Cranky Franky, now decommissioned and replaced with Edison) was hacked.  Here's the story.  If I am given permission, I'll write up more: I need to tread very carefully here.  I will say there's a huge story which isn't in the Wired article. 

This man...

The one kneeling while doing the install of NERSC's first Cray-1 supercomputer back in 1978 is downstairs installing NERSC's newest supercomputer, or rather the expansion to it, yes, a cray.  He's one of the longest serving Cray employees.  He's been at Cray for 36 years.  He did the install above after his first year at Cray.  

We're honored to have him back.

If you watch, you might see him.

Supercomputer Construction Started

The drop cam is here.

Want to Watch a Supercomputer Being Built?

Go here.

Its a live feed of the machine room where our supercomputer, Edison, is going into phase 2.  The building ACTUALLY starts on the 26th.  

I question the sanity of management a bit.  NERSC is full of pranksters. 

I'll throw up the link again when trucks start arriving.

At the Day Job: LBNL's NERSC Releases 10 Year Plan

In its recently released strategic plan, the National Energy Research Scientific Computing Center (NERSC) outlines its goals for the next 10 years. The high-end scientific production computing facility for DOE’s Office of Science, NERSC supports 4,500 users working on more than 700 research projects and contributes to an average 1,500 publications annually. Go here to download the report.

PDF here.

No Exascale for You! Part Deux: The Interview

Although Horst Simon was named Deputy Director of Lawrence Berkeley National Laboratory, he maintains his strong ties to the scientific computing community as an editor of the TOP500 list and as an invited speaker at conferences.

Twice during the week of May 6, Simon gave back-to-back presentations of a new talk on “Why We Need Exascale and Why We Won’t Get There by 2020.” Not only was the talk a hit with conference attendees, but it also made its way onto Slashdot. In this HPCwire exclusive, Simon talks about his presentation with Jon Bashor of Berkeley Lab.

Simon is well positioned to discuss the path to exascale. An internationally recognized expert in computer science and applied mathematics, he joined Berkeley Lab in 1996 as director of the newly formed National Energy Research Scientific Computing Center (NERSC), and was one of the key architects in establishing NERSC at its new location in Berkeley. Under his leadership NERSC enabled important discoveries for research in fields ranging from global climate modeling to astrophysics. Simon was also the founding director of Berkeley Lab's Computational Research Division, which conducts applied research and development in computer science, computational science, and applied mathematics.

In his prior role as Associate Lab Director for Computing Sciences, Simon helped to establish Berkeley Lab as a world leader in providing supercomputing resources to support research across a wide spectrum of scientific disciplines. Simon’s research interests are in the development of sparse matrix algorithms, algorithms for large-scale eigenvalue problems, and domain decomposition algorithms for unstructured domains for parallel processing. His algorithm research efforts were honored with the 1988 and the 2009 Gordon Bell Prize for parallel processing research. He was also member of the NASA team that developed the NAS Parallel Benchmarks, a widely used standard for evaluating the performance of massively parallel systems. He is co-editor of the twice-yearly TOP500 list that tracks the most powerful supercomputers worldwide, as well as related architecture and technology trends.

heh. Wonder who was guilty of trolling /.?  ;)

No Exascale for You: Why We Will Not See an Exaflop System by 2020

The link to the presentation is in the title (like always). 

All of you who have been reading my blog for some time have noted that I am anti-Singularitarian.  I think the entire bit is bogus to the extreme.  I've talked about the 'Heat Death of the Singularity' and I have talked of the eye rolling Charlie's Accelerando (note: work is the distillation of the fantasies of the /. crowd) has induced (it stopped...eventually). I am also not concerned whatsoever about the Robopocalypse, my tag to the effect is as much a bit of humor as anything.

However, I'm a low level worker in the HPC field.  While, yes, I do move and shake a bit in the NewSpace world - its a much smaller one - I am not someone who, by career choice, is can have a big impact with my opinions in the world of supercomputing.  

Horst Simon, on the other hand, as a Deputy Director of Lawrence Berkeley National Lab, has far more impact when he stands up and says something.  He just has.  We will not be getting to exaflop machines by 2020, he says.  That takes guts to say.  Go read his prezo on the subject.  I wish there was a video of him presenting a version of it at the CUG LBNL just hosted.  I've been told it caused quite a stir.

DOE's Joint Bio Energy Institute and NERSC Advance Cellulosic Biofuel Production

Advanced biofuels – liquid fuels synthesized from the sugars in cellulosic biomass – offer a clean, green and renewable alternative to gasoline, diesel and jet fuels. Bringing the costs of producing these advanced biofuels down to competitive levels with petrofuels, however, is a major challenge. Researchers at the U.S. Department of Energy (DOE)'s Joint BioEnergy Institute (JBEI), a bioenergy research center led by Berkeley Lab, have taken another step towards meeting this challenge with the development of a new technique for pre-treating cellulosic biomass with ionic liquids - salts that are liquids rather than crystals at room temperature. This new technique requires none of the expensive enzymes used in previous ionic liquid pretreatments, and makes it easier to recover fuel sugars and recycle the ionic liquid.

"Most of our ionic liquid efforts at JBEI have focused on using enzymes to liberate fermentable sugars from lignocellulosic biomass after pretreatment, but with this new enzyme-free approach we use an acid as the catalyst for hydrolyzing biomass polysaccharides into a solution containing fermentable sugars," says Blake Simmons, a chemical engineer who heads JBEI's Deconstruction Division and was the leader of this research. "We're then able to separate the pretreatment solution into two phases, a sugar-rich water phase for recovery and a lignin-rich ionic liquid phase for recycling. As an added bonus, our new pretreatment technique uses a lot less water than previous pretreatments."

Simmons is the corresponding author of a paper describing this research that has been published in the journal Biotechnology for Biofuels. The paper is titled "Production and extraction of sugars from switchgrass hydrolyzed in ionic liquids." Co-authoring it were Ning Sun, Hanbin Liu Noppadon Sathitsuksanoh, Vitalie Stavila, Manali Sawant, Anaise Bonito, Kim Tran, Anthe George, Kenneth Sale, Seema Singh and Bradley Holmes.

With the burning of fossil fuels continuing to add 9 billion metric tons of excess carbon dioxide to the atmosphere each year, the need for carbon neutral, cost-competitive renewable alternative fuels has never been greater. Advanced biofuels, produced from the microbial fermentation of sugars in lignocellulosic biomass, could displace gasoline, diesel and jet fuel on a gallon-for-gallon basis and be directly dropped into today's engines and infrastructures without impacting performance. If done correctly, the use of advanced biofuels would not add excess carbon to the atmosphere.

Environmentally benign ionic liquids are used as green chemistry substitutes for volatile organic solvents. While showing great potential as a biomass pretreatment for dissolving lignocellulose and helping to hydrolyze the resulting aqueous solution into fuel sugars, the best of these ionic liquids so far have required the use of expensive enzymes. Recent studies have shown that acid catalysts, such as hydrochloric or Brønsted, can effectively replace enzyme-based hydrolysis, but the subsequent separation of sugars and ionic liquids becomes a difficult and expensive problem can require the use of significant amounts of water.

Guided by molecular dynamics simulations carried out at DOE's National Energy Research Scientific Computing Center (NERSC), Simmons and his colleagues at JBEI solved this problem by deploying the ionic liquid imidazolium chloride in tandem with an acid catalyst.

"Imidazolium is the most effective known ionic liquid for breaking down lignocellulose and the chloride anion is amenable with the acid catalyst," Simmons says. "The combination makes it easy to extract fermentable sugars that have been liberated from biomass and also easy to recover the ionic liquid for recycling. By eliminating the need for enzymes and decreasing the water consumption requirements of more traditional ionic liquid pretreatments we should be able to reduce the costs of sugar production from lignocellulose."

Complete separation of the pretreatment solution into sugar-rich water and lignin-rich ionic liquid phases was attained with the addition to the solution of sodium hydroxide. The optimized sodium hydroxide concentration for both phase separation and sugar extraction was 15-percent, resulting in the recovery of maximum yields of 54-percent glucose and 88-percent xylose. The JBEI researchers believe these sugar yields can be increased by optimizing the process conditions and using more advanced methods of phase separation and sugar recovery.

An Updated CAM 5.1 World Circulation Video

Mike responded and placed an updated version of his video I linked to earlier.  He shows the difference between the 25 km (new) and 200 km (old) resolutions. Watch in HD.

The Faces of the Machines of NERSC

Hopper and Edison.  The latter is slowly getting the nickname of Fast Eddie.  Fast Eddie got its first, early users as part of acceptance.  Both are Crays.  I'd put up pix of Carver and the other machines, but IBM doesn't do the nifty art like Cray does for their machines.