Dan and I arrived in Scotland on Monday after a long but trouble-free journey (thanks, British Airways and BellaTerra Travel). After attempting to catch up on sleep Monday night (I can’t sleep on planes, day or night, no matter how long the trip…), we attended Day 1 of High Throughput Computing Week at the UK National eScience Center facility at the University of Edinburgh.

The mix of attendees was interesting — but not as diverse as I had hoped. One idea of this event was to bring together business and academic users of high-throughput computing solutions. There are a handful of business speakers (including me and Dan), but I did not see many other business attendees. The attendee population seemed primarily UK academic.

Day 1 presentations included the incomparable Miron Livny, father of Condor and the leading academic in HTC. While he’s given similar presentations before, I learn more each time I hear him speak. You can hear the frustration creeping into his voice as he describes the current grid computing movement — because there is so much re-discovery and re-invention and re-defining of concepts Miron has worked on for nearly three decades. His insights are excellent, and anyone serious about HTC needs to listen — carefully.

One benefit of attending events like this is that I get to meet people with whom I’ve only corresponded via email before. For example, Professor Antonio Mungioli of Universidade de São Paulo, Brazil, came to our Web site and asked many good questions last year.  He also gave an excellent presentation on Day 1, describing his experiences with grid computing in the state of San Paulo.  Like Miron, he emphasized social and organizational issues over technology issues — in his view, cooperation and collaboration are more important to a project’s success than any specific technology choice.

We were also fortunate to have Akash Chopra of Barrie & Hibbart present on Day 1.  Akash described some of the compute-intensive calculations his insurance customers must perform, and how he used the Digipede Network to grid-enable their Economic Scenario Generator (ESG) software.  He also discussed some of the real-world problems associated with bringing grid computing to his customers, who in most cases have very limited and specific requirements for application performance.  He encouraged the audience to think from a customer perspective, and to focus on short-term value delivered to those customers, rather than getting caught up in the dream of “the grid” as a worldwide plug-in computing resource.  His perspective was a refreshing dose of reality.

There were also several other interesting presentations by academic HTC users, by the Condor team, and by Jason Stowe of Cycle Computing.  But if I wrote about everything, I’d never finish this!

We went out for some authentic Scottish food with about half the participants (it’s better if you don’t read the definition of haggis until after you’ve eaten it), and had a fine evening.

Day 2 was entirely devoted to two hands-on workshops — the morning one by Digipede, and the afternoon one by the Condor team.  As improbable as it sounds, Dan and I were both pretty coherent considering the Digipede half of the day ran from 1:00 AM to 4:30 AM Pacific time.  I gave a short introduction to Digipede’s space in the HTC world, and then we moved downstairs to a lab / classroom with 22 user workstations (several participants had to double up — we had good attendence).  Dan gave a more technical introduction to Digipede, then led the participants through a “hello world” exercise.  After that, we let them figure out a more realistic eScience application mostly on their own (code used in high-energy plama physics experiments, courtesy of conference organizer David Wallom).  All the participants successfully completed this second exercise in less than an hour, so we had plenty of time left for code demos — and even in this crowd of (nearly) no .NET developers, the audience immediately saw the benefits of the Digipede programming model.

(The Condor guys did a workshop too — it also went very well, but I was unable to attend much of the afternoon.)

I missed the morning on Day 3 (out visiting partners / clients, and seeing Edinburgh Castle along the way), but the afternoon was all about requirements gathering for HTC in the commercial sector.  This was interesting, and I participated enthusiastically (not surprisingly, I have some opinions on this!) but I’ll leave it to conference organizer David Wallom to  synthesize the results.

It’s clear to me that the area of High Throughput Computing is not exactly the same as grid computing, high-performance computing, cluster computing, or any other area.  When it comes to mapping specific products to these areas, however, there are limits to how useful these distinctions are.  There are some workloads that will run about equally well on HTC, HPC, grid, and/or cluster products.  The current version of the Digipede Network falls closer to HTC and grid computing than to HPC and cluster computing, but like many other vendors we’ve been known to creep across boundaries as required to broaden the problem-space that we address.

I’ll also miss the last day of HTC Week — I’m going to London for more partner and client meetings — but it’s been quite worthwhile.  I will try to distill a few conclusions on the way home this weekend — watch this space.

Tags: Digipede, eScience, grid-computing, high-throughput-computing, HTC

I have been invited to speak at a conference on High Throughput Computing at the UK National e-Science Center in Edinburgh next week.  This is extraordinary on a number of levels.

• First, the event is called “High Throughput Computing Week.”  If someone had told you five years ago that there was a “High Throughput Computing Day,” you might well have asked “how will they fill the afternoon?”  But there will be four days packed with great content at this event.
• Second, I am an economist, not a computer scientist.  I am likely to be among the least knowledgeable attendees along a number of important dimensions, yet I will be expected to provide some useful information.
• Finally, I work with very few scientists; most of Digipede’s customers are mainstream enterprises running relatively mundane production applications.  The field of e-Science is of interest, but definitely not the primary focus of our firm.

Yet I accepted the invitation enthusiastically, and went the extra mile to commit to a three-hour workshop (which meant “volunteering” Dan Ciruli for the trip as well).  To be sure, part of the value of this trip will be seeing customers — we have two current Digipede customers and one partner in Edinburgh.  But I also think we’ll have something interesting to contribute to the event; many of our customers use the Digipede Network to solve high-throughput computing problems, and I think attendees will be interested in hearing about that experience.  We’re also pretty excited about the opportunity to learn more about what leading academic and business HTC users are doing. 

I’ll report more from the show.

Tags: Digipede, grid-computing, high-throughput-computing, HTC

I am back home from SC ’07, my fifth Supercomputing conference. I saw some really impressive new technology from market leaders old and new. I heard remarkable claims and forecasts from analysts, pundits, and marketing flacks alike. I learned a lot, and contributed what I could.

Yet at this conference, full of the so-called thought leaders in high-performance computing, I once again ran into many instances of unthinking knee-jerk Microsoft bashing. While much of the IT world has come to grips with the fact that Microsoft (like gravity) is likely to be around a while, the Supercomputing crowd still has some holdouts. Literally, I heard people claim that “nobody” would use Windows for high-performance computing (provably incorrect), and that positive coverage of Microsoft’s HPC offerings was “bought, not earned” (unsubstantiated rubbish). Another blogger in this field (John West of insideHPC.com) told me that he’s had “…readers take the time to send me an email saying they would never read my stuff again if I kept covering MS’s CCS products and their entry into HPC positively.” What?!?

It is amazing to me the level of religious ferver that Microsoft still inspires. The bashers out there can be perfectly calm and reasonable about a wide range of topics – but say the word “Microsoft,” and they turn bright red and irrational. I have watched this phenomenon for years, and still find it inexplicable. Microsoft is a company. That company makes software. Some of their software is very, very good. Some of it is remarkably bad. I don’t understand why some people find it so hard to remain objective (or even civil) when discussing their products and market presence.

Many Microsoft bashers think that all of us at Digipede are mouthpieces for the Evil Empire, and that we are just pawns of the Microsoft machine. On the other hand, while we have plenty of fans within Microsoft, there are also some Microsoft employees who think we are difficult annoying troublemakers (especially me).

In fact, none of us at Digipede love or hate Microsoft – we work with Microsoft. We do so for real-world business reasons that help us change the world for the better while building a great company. We work with other companies too, but Microsoft occupies a special place in the technology landscape, and we work very, very hard to understand how to work with them to our mutual benefit. There are some great people there doing great things, and the bashers only hurt themselves by blinding themselves to these very real contributions.

Microsoft’s HPC initiatives in the past three years have greatly increased their presence in the HPC market at a time when that market is expanding rapidly. Their HPC offerings have some advantages and disadvantages compared to other products in the market, and should be evaluated on those terms. Microsoft bashing lowers the level of discourse to a useless level, at a time when we can all benefit from a more objective and reasoned discussion of how they affect our market.

I’ll stop there for now, and climb into my asbestos suit…

Tags: HPC, insideHPC, Microsoft, SC07, Supercomputing

I am at the Supercomputing conference in Reno Nevada today and tomorrow.

If you’re here and want to meet up, please call my cell — 510-326-1761.

Much of the time I’ll be in the booth of our partner AMD (thanks guys!) showing off the latest version of the Digipede Network running on Windows Compute Cluster Server (CCS) on some very cool hardware from Scalable Servers. Come and see!

Tags: Digipede, grid-computing, HPC, Reno, SC07, Supercomputing