Powers Unfiltered

Dan and I arrived in Scotland on Monday after a long but trouble-free journey (thanks, British Airways and BellaTerra Travel). After attempting to catch up on sleep Monday night (I can’t sleep on planes, day or night, no matter how long the trip…), we attended Day 1 of High Throughput Computing Week at the UK National eScience Center facility at the University of Edinburgh.

The mix of attendees was interesting — but not as diverse as I had hoped. One idea of this event was to bring together business and academic users of high-throughput computing solutions. There are a handful of business speakers (including me and Dan), but I did not see many other business attendees. The attendee population seemed primarily UK academic.

Day 1 presentations included the incomparable Miron Livny, father of Condor and the leading academic in HTC. While he’s given similar presentations before, I learn more each time I hear him speak. You can hear the frustration creeping into his voice as he describes the current grid computing movement — because there is so much re-discovery and re-invention and re-defining of concepts Miron has worked on for nearly three decades. His insights are excellent, and anyone serious about HTC needs to listen — carefully.

One benefit of attending events like this is that I get to meet people with whom I’ve only corresponded via email before. For example, Professor Antonio Mungioli of Universidade de São Paulo, Brazil, came to our Web site and asked many good questions last year.  He also gave an excellent presentation on Day 1, describing his experiences with grid computing in the state of San Paulo.  Like Miron, he emphasized social and organizational issues over technology issues — in his view, cooperation and collaboration are more important to a project’s success than any specific technology choice.

We were also fortunate to have Akash Chopra of Barrie & Hibbart present on Day 1.  Akash described some of the compute-intensive calculations his insurance customers must perform, and how he used the Digipede Network to grid-enable their Economic Scenario Generator (ESG) software.  He also discussed some of the real-world problems associated with bringing grid computing to his customers, who in most cases have very limited and specific requirements for application performance.  He encouraged the audience to think from a customer perspective, and to focus on short-term value delivered to those customers, rather than getting caught up in the dream of “the grid” as a worldwide plug-in computing resource.  His perspective was a refreshing dose of reality.

There were also several other interesting presentations by academic HTC users, by the Condor team, and by Jason Stowe of Cycle Computing.  But if I wrote about everything, I’d never finish this!

We went out for some authentic Scottish food with about half the participants (it’s better if you don’t read the definition of haggis until after you’ve eaten it), and had a fine evening.

Day 2 was entirely devoted to two hands-on workshops — the morning one by Digipede, and the afternoon one by the Condor team.  As improbable as it sounds, Dan and I were both pretty coherent considering the Digipede half of the day ran from 1:00 AM to 4:30 AM Pacific time.  I gave a short introduction to Digipede’s space in the HTC world, and then we moved downstairs to a lab / classroom with 22 user workstations (several participants had to double up — we had good attendence).  Dan gave a more technical introduction to Digipede, then led the participants through a “hello world” exercise.  After that, we let them figure out a more realistic eScience application mostly on their own (code used in high-energy plama physics experiments, courtesy of conference organizer David Wallom).  All the participants successfully completed this second exercise in less than an hour, so we had plenty of time left for code demos — and even in this crowd of (nearly) no .NET developers, the audience immediately saw the benefits of the Digipede programming model.

(The Condor guys did a workshop too — it also went very well, but I was unable to attend much of the afternoon.)

I missed the morning on Day 3 (out visiting partners / clients, and seeing Edinburgh Castle along the way), but the afternoon was all about requirements gathering for HTC in the commercial sector.  This was interesting, and I participated enthusiastically (not surprisingly, I have some opinions on this!) but I’ll leave it to conference organizer David Wallom to  synthesize the results.

It’s clear to me that the area of High Throughput Computing is not exactly the same as grid computing, high-performance computing, cluster computing, or any other area.  When it comes to mapping specific products to these areas, however, there are limits to how useful these distinctions are.  There are some workloads that will run about equally well on HTC, HPC, grid, and/or cluster products.  The current version of the Digipede Network falls closer to HTC and grid computing than to HPC and cluster computing, but like many other vendors we’ve been known to creep across boundaries as required to broaden the problem-space that we address.

I’ll also miss the last day of HTC Week — I’m going to London for more partner and client meetings — but it’s been quite worthwhile.  I will try to distill a few conclusions on the way home this weekend — watch this space.