Higgs boson discovery has been confirmed, and two papers have been recently published in the Physics Letters B issue of September. It was no small task. The articles start with a list of hundreds of contributors around the world. The news made international headlines in July; almost fifty years after Peter Higgs predicted a new particle to explain how other elementary particles acquire mass.
The long-sought boson is extremely complicated to observe. It involves a global computing network to massage the vast amount of data generated by the Large Hadron Collider (LHC) near Geneva, Switzerland. Most of the researchers are actually in the US, and they are connected via the Energy Sciences network called ESnet. I decided to interview Bill Johnston from Berkeley Laboratory. He led the transformation of ESnet from “plumbing” to a “service” to the scientific community. Fasten your seatbelt! We are on for a ride at the frontiers of science and networking.
Ernest Lawrence pioneered Big Science in the thirties and the Department of Energy renamed the Berkeley national lab after him: Lawrence Berkeley National Laboratory (LBNL). I meet with Bill Johnston at LBNL where 4,000 researchers and staff work on the top of the University campus, which overlooks the Bay Area. Bill first comes across as one the “mad scientists” you would imagine working at a national lab. But the stereotypes dissolve quickly. His piercing eyes and patient kindness calm you right away.
Behind the headline news are two complex sensors that measured the boson independently: ATLAS and CMS, which cost billions of dollars. Why two expensive sensors? Well, it is not like a new particle today can be discovered under a powerful microscope. It is an evasive search based on statistical studies of a particle with complex properties and a very short life-time. See them as two independent witnesses of a car crash, except with a collision time of a tiny fraction of a second. Particle scientists have to repeat the collision again and again.
Bill Johnston shows me his watch and explains to me that studying particle physics is like taking a precisely crafted Swiss watch, and throwing it against the wall with sheer force. “What the sensors looked at are the multiple pieces that flew off from the collision” he adds, “Imagine you have to identify the watch from all the pieces”.
It is the observation of the effects of the particle that are consistent with the model of Higgs, and gives a high level of confidence that the LHC indeed discovered the last particle that completes the standard model of particle physics (picture left - courtesy of AAAS). Without it the other elementary particles would not have mass.
Both ATLAS and CMS teams reported data consistent with a neutral boson of a mass around 125 GeV. This is the culmination of decades of international scientific collaboration. No country could afford pursuing the experiment on its own.
The evolution of the Internet and ESnet
“It is a gradual effort” notes Bill Johnston who knows more than a thing or two about physics and networks. He is now retired but continues to serve as Advisor in the Computational Research Division of LBNL. He studied mathematics and physics and came to Berkeley in 1974 to work on computer graphics and systems. In particular, he started to work in the team that put LBNL among the first sixteen nodes on the Arpanet funded by the Department of Defense.
Bill later worked on distributed systems, which led him to work with scientists around the world. He points to Moore’s Law as the driving factor behind the explosion of data that is making new experiments like CMS or Atlas possible. “Scientific instruments are built with solid-state sensors, the same semiconductor technology use in computers chips. As the gate dimension gets smaller, the resolution of the instruments increases and the amount of data grows exponentially.”
Looking back, Arpanet was the first multi-service wide area network, from which the Internet that we know today grew from. When they moved from NP to IP in the early eighties –- and IP actually became the Internet protocol -– they had to stop the entire network. It was the last time it happened. “LBNL has a long history in the development of the Internet and its association with Unix as the original operating system” says Bill Johnston.
The energy science community is still connected to the Internet but they have developped since 1986 a parallel network to handle demanding experiments (picture left). That is ESnet now in its fitth generation. It connects 40 Department of Energy sites as well as international exchange points. Bill Johnston led its transformation from 2003 to 2008. He took it it from a data pipe “plumbing ” approach to a service-driven architecture.
The frontiers of science breaking exponential data records
Eli echoes Bill’s sentiment that we are yet again at the verge of a new era of science. “Imagine that you are a scientist and you have a difficult problem to solve” Eli tells me with passion. “I am going to throw all the tools I have at my disposal. The most effective way to get the largest amount of information into a brain is through the eyes. Visualization is a very effective scientific tool.
Bill recounts the first experiment that used the concept of distributed computing to assist visualization. It was for a brain scan experiment in 1991 in Albuquerque, New Mexico. Two computers were connected with the imaging instrument via a T3 connection (45 Mpbs) because one computer alone could not handle all the data in real-time. A doctor could remotely steer the scan and see the image in front of him.
Data for the LHC experiment are now transported and shared around the world via fiber optic cables that support multiple channels of light, each at incredible speed. And data are processed by IP routers at each node. Eli takes me to the networking room at LBNL where they installed new equipment (picture right) that can transport and process data at the speed of 100 Gbps on each channel.
That is more than a factor 1,000 improvement in twenty years, between the brain scan demonstration and this year LHC experiment. ESnet has recently broken the barrier of exchanging 10 petabytes of data every month, and this without having to travel with magnetic tapes as scientists used to. Eli and Bill tell me with a smile that one of the data service ESnet supports is called "PHEDEx".
Why society does need to keep doing it
I can see how progress in science translates later with better data services at home. Big Data is everywhere now. I ask however how we can justify spending billions of dollars on new research tools when the economy is hitting a big crunch. “The reason we have this modern society at all is because western civilization has been investing in science for the past several centuries”, Eli notes. “It is hard to look at any discovery any given day and say which one will change the world. In retrospect you can. If we stop science, everything stops.”
Bill jumps in to clarify that science advances in two ways: a scientific problem that pushes capabilities like massive distributed data computation, or a new capability that opens new research. For example, another group at Berkeley is using the wakefield effect to look at building particle accelerators in the relatively miniature space of an office.
“This is an example of a tool that was developed without a problem to solve in sight. Now the scientific community is looking at it and saying: ‘wow, see everything that we can do with this!’”. Eli adds to Bill’s point that “there is a synergy between the two”. Discoveries and capabilities feed each other.
Berkeley does not have a large-scale accelerator on-site although Ernest Larwence started the field of Big Science, mostly due to real-estate constraints. “There was only going to be one super collider.” Bill Johnston acknowledges. The LHC spans miles across the Swiss and the French border (picture of CMS sensor at LHC above- courtesy of CERN). Researchers at LBNL have to work every day with other researchers around the world. “Why is large collaboration happening everywhere? This is because the ‘easy science’ has been done.” explains Bill.
The realm of the unseen is not limited to particles
Data intensive collaboration is also what we are seeing in astrophysics, meteorology, genomics, etc. “Genomics has actually a super Moore’s law”, corrects Eli, “It will be interesting to see whether they can sustain that rate.” At a smaller scale, we can see it with new services on the Internet that makes our life easier. We live in a connected society, and we virtually travel the world every day. Scientific research had a lot to do with making it happen.
Working in a university or a national lab is not always easy. It is political. Two other research networks in the US, Internet 2 and the National Lambda Rail, were embattled in politics for years. People may not see it, like the boson that we can only infer (picture of simulation left - courtesy of SSPL), but it is there. Politics is part of the process. The nomination of Bill Johnston at the head of ESnet was no exemption.
When I talk to him about network virtualization as the new hot subject in the tech business news, and mention that Stanford and Berkeley Universities for once collaborated to found the Open Networking Foundation, he can't avoid a smile: "Did we collaborate, really?" Sometimes it is easier to work with researchers in another continent than across the Bay.
Bill remains discrete about the political struggles that are part of advanced research and collaboration. He'd rather focuss on serving the scientific community, and that way human kind as a whole. He is part of the lesser known heroes. ESnet supports open networking protocols like Open Flow. For each Mark Zuckerberg playing on a new website in a University dorm, there is another Bill Johnston who works in the shadow of the media years before to make distributed collaboration possible.
Hats off to Bill and Eli, and a future generation of dazzling scientific plumbers.