The Irresistible Forces Behind Flow Computing A FiRe Conversation with Larry Smarr Hosted by  Mark Anderson

 

[Please open the attached .pdf for best viewing.] ______ *** RESCHEDULED *** SNS 3rd Annual Predictions : West Thursday, August 10, 5pm-8pm Pullman Hotel - Redwood City Register here.

In This Issue Week of 5/15/2017    Vol. 22 Issue 18 FEATURE: The Irresistible Forces Behind Flow Computing: A FiRe Conversation with Larry Smarr o   About Larry Smarr Inside SNS Upcoming SNS Events o   FiRe 2017 o   Where's Mark?

Publisher's Note: This week's issue represents the culmination of two major trends in the SNS world. First, in our continuing work to create concept (and real) designs in future computing, through our last FiRe conference we were able to move from our previous designs for the Pattern Recognition Processor (PRP) and the Pattern Computer into a new design that we've labeled "Flow Computing."

Second, when we asked global experts at FiRe 2016 to examine our proposal that Flow and Interaction drive, as Murray Cantor put it onstage, "everything in the universe," we all benefited as these brilliant minds affirmed our beliefs and moved the whole concept forward into each of their areas of expertise.

In this interview, Larry Smarr sets out the reasons and drives behind the need for, and ultimate architecture of, flow computing, which he and I believe is the inevitable compute architecture needed in order to handle the massive data loads of the next 10 years and beyond. Anyone involved in the future of computing will want to read Larry's thoughts, and get ahead of this curve. - mra.

Photos 2016 Kris Krg

Mark Anderson: Dr. Smarr?

Larry Smarr: Mark!

Anderson: Good morning.

Smarr: Good to see you again.

Anderson: Good to see you. Have a seat.

Smarr: Here we are again.

Anderson: I doubt there's anybody here who doesn't know Larry Smarr. I think it's the Smarr and Anderson show, these days.

Smarr: Smarr family and Anderson family.

Anderson: Yeah, we had Benjamin Smarr, we've had Joe Smarr ...

We're going to have a conversation this morning about the whole idea of flows in computing, and who needs it and why. This started with - Larry and I have this no-longer-secret favorite thing that we do, which is the Smarr-Anderson Road Trip. I have this little silver convertible, and we get into it and we go places, every year. And all we do is just share ideas for about 48 hours. It's the most fun thing I think I do all year long, other than FiRe itself.

And this year ... You can picture the little silver car, and there's Larry, there's Mark, and we're driving along, it's very beautiful, it's on an island up in the Northwest, and almost from the very beginning - I think we were in the parking lot, still, getting onto this ferry - and we start talking about flows, and it just builds.

Smarr: Mm-hmm.

Anderson: And Larry's talking about stories about the increased amount of data in the world, and we're talking about streams of data that are flowing, and what's going to happen, and who's going to do what about it, and it just immediately comes clear that we don't have - today - we don't have the proper ways of dealing with all that.

So I think what we're going to try and do this morning is essentially re-create that conversation, and I'm going to ask Larry to give us some examples of the problem and what we have to do to meet it. So, would you like to tell a few stories about flows of data?

Smarr: Well, as you all know, I'm the principal investigator for an NSF grant called the Pacific Research Platform. And what that is, is realizing that with this exponential increase in big data and big data flows, the ordinary commodity internet isn't gonna cut it. And so we really need, like the interstate highway system was needed, as well as Route 66. So we've been working on this for about 15 years, using optical fibers, which have the ability to carry - you know, a trillion bits per second, actually, in the glass. And to be able to establish between scientific instruments, investigators, supercomputers, over the whole West Coast - and we take a broad view of the West Coast, that goes to Australia and to Chicago, and actually to the Netherlands [laughter]. It's like that New Yorker [cartoon] view of the world.

Essentially, we picked scientific teams - and I'm going to give you an example of two of them - that typify where we're going over the next few years. And this is hooking up at about 10 to 100 gigabits a second, which is about 1,000 times the speed through the normal internet.

As a person who's spent 25 years doing generalistic astrophysics and observation and astronomy, that's close to my heart. So, one of them is the Large Synoptic Survey Telescope, the LSST, which the NSF is funding. It's a giant telescope being built in Chile, up in the mountains, the very top [of the] mountains, and it is looking at the sky. Every night, it reads out about one-and-a-half billion pixels a second, and it has two 40-gigabit-per-second - for those of you who have home cable modems at, say, 10 megabits? this is 40,000 megabits a second - to the National Center for Supercomputing Applications in Illinois, that I founded 30 years ago.

It follows most of the observable universe. It in particular follows 30 billion objects in the sky and looks at their changes. So it essentially turns the universe into a video stream. And then it's computing anything that moves, gets brighter, dimmer, changes color ... How many of those do you think it's going to see a night? Because it has to send an alert out to all the telescopes on Earth and in space. One to 10 million. And, by the way, those alerts go out from the telescope through the fiberoptic supercomputer one minute after the observation in Chile.

So that's a level of flow that is literally turning the whole universe into a movie. That's never been done before.