Powers Unfiltered

An entrepreneur’s journey into grid computing and partnering with Microsoft, by John Powers

Powers Unfiltered header image 2

HTC Week, Days 1-3

November 29th, 2007 · 4 Comments

Dan and I arrived in Scotland on Monday after a long but trouble-free journey (thanks, British Airways and BellaTerra Travel). After attempting to catch up on sleep Monday night (I can’t sleep on planes, day or night, no matter how long the trip…), we attended Day 1 of High Throughput Computing Week at the UK National eScience Center facility at the University of Edinburgh.

The mix of attendees was interesting — but not as diverse as I had hoped. One idea of this event was to bring together business and academic users of high-throughput computing solutions. There are a handful of business speakers (including me and Dan), but I did not see many other business attendees. The attendee population seemed primarily UK academic.

Day 1 presentations included the incomparable Miron Livny, father of Condor and the leading academic in HTC. While he’s given similar presentations before, I learn more each time I hear him speak. You can hear the frustration creeping into his voice as he describes the current grid computing movement — because there is so much re-discovery and re-invention and re-defining of concepts Miron has worked on for nearly three decades. His insights are excellent, and anyone serious about HTC needs to listen — carefully.

One benefit of attending events like this is that I get to meet people with whom I’ve only corresponded via email before. For example, Professor Antonio Mungioli of Universidade de São Paulo, Brazil, came to our Web site and asked many good questions last year.  He also gave an excellent presentation on Day 1, describing his experiences with grid computing in the state of San Paulo.  Like Miron, he emphasized social and organizational issues over technology issues — in his view, cooperation and collaboration are more important to a project’s success than any specific technology choice.

We were also fortunate to have Akash Chopra of Barrie & Hibbart present on Day 1.  Akash described some of the compute-intensive calculations his insurance customers must perform, and how he used the Digipede Network to grid-enable their Economic Scenario Generator (ESG) software.  He also discussed some of the real-world problems associated with bringing grid computing to his customers, who in most cases have very limited and specific requirements for application performance.  He encouraged the audience to think from a customer perspective, and to focus on short-term value delivered to those customers, rather than getting caught up in the dream of “the grid” as a worldwide plug-in computing resource.  His perspective was a refreshing dose of reality.

There were also several other interesting presentations by academic HTC users, by the Condor team, and by Jason Stowe of Cycle Computing.  But if I wrote about everything, I’d never finish this!

We went out for some authentic Scottish food with about half the participants (it’s better if you don’t read the definition of haggis until after you’ve eaten it), and had a fine evening.

Day 2 was entirely devoted to two hands-on workshops — the morning one by Digipede, and the afternoon one by the Condor team.  As improbable as it sounds, Dan and I were both pretty coherent considering the Digipede half of the day ran from 1:00 AM to 4:30 AM Pacific time.  I gave a short introduction to Digipede’s space in the HTC world, and then we moved downstairs to a lab / classroom with 22 user workstations (several participants had to double up — we had good attendence).  Dan gave a more technical introduction to Digipede, then led the participants through a “hello world” exercise.  After that, we let them figure out a more realistic eScience application mostly on their own (code used in high-energy plama physics experiments, courtesy of conference organizer David Wallom).  All the participants successfully completed this second exercise in less than an hour, so we had plenty of time left for code demos — and even in this crowd of (nearly) no .NET developers, the audience immediately saw the benefits of the Digipede programming model.

(The Condor guys did a workshop too — it also went very well, but I was unable to attend much of the afternoon.)

I missed the morning on Day 3 (out visiting partners / clients, and seeing Edinburgh Castle along the way), but the afternoon was all about requirements gathering for HTC in the commercial sector.  This was interesting, and I participated enthusiastically (not surprisingly, I have some opinions on this!) but I’ll leave it to conference organizer David Wallom to  synthesize the results.

It’s clear to me that the area of High Throughput Computing is not exactly the same as grid computing, high-performance computing, cluster computing, or any other area.  When it comes to mapping specific products to these areas, however, there are limits to how useful these distinctions are.  There are some workloads that will run about equally well on HTC, HPC, grid, and/or cluster products.  The current version of the Digipede Network falls closer to HTC and grid computing than to HPC and cluster computing, but like many other vendors we’ve been known to creep across boundaries as required to broaden the problem-space that we address.

I’ll also miss the last day of HTC Week — I’m going to London for more partner and client meetings — but it’s been quite worthwhile.  I will try to distill a few conclusions on the way home this weekend — watch this space.

Tags: Events · Grid applications · Presentations

4 responses so far ↓

  • 1 Dave Berry // Nov 30, 2007 at 12:55 pm


    Thank you very much for coming to NeSC and participating in the High Throughput Computing week. We certainly learnt a lot from your presence and I’m glad that you found it useful and enjoyable.

    We have run events with the Condor guys before but this is the first time that we’ve broadened this to include HTC in general. There is talk of repeating this next year and we would certainly like to broaden participation. If you (or any of your readers) have any suggestions, please let us know.

    I think the participants learnt a lot from the hands-on day in particular. Being able to compare Digipede and Condor and learn their relative strengths was very useful. I know people were impressed with the ease of writing application for .NET in Digipede. Also, our training team is keen to pursue the idea of an HTC patterns library.

    In conclusion, it was good to meet you and we valued your participation highly. Please keep in touch.

    Dave Berry.

  • 2 Chris Vaughan // Dec 10, 2007 at 1:03 pm

    There was also the Daresbury MEW going on that week down in Runcorn, and SC was the week before. I think the amount of events crammed into that 3 week block really tax everyone out. Possibly better timing of events next year would help things a long.


  • 3 Ross Hamilton // Jan 3, 2008 at 4:23 pm

    John – Sounds interesting. You did not specify if you were in the half that ate the haggis, and if you were, did you enjoy it ? Speak soon.

  • 4 john // Jan 3, 2008 at 11:54 pm

    I was in the half that SAMPLED the haggis — did not order my own plate, but I enjoyed the bite or two I had. And the sausages and mash were excellent. And the single-malt sampling — better still, to the best of my recollection.

Leave a Comment