And as he reports, thanks to improvements in both Mono and the Digipede Network, the answer is — not much. We’ve got a working prototype of a Digipede Agent running under Mono on Linux that runs a Digipede job.

Digipede on Linux? Has the world turned upside down?

Hardly.

Since the beginning, Digipede has been focussed on adding value to the Microsoft platform. And customers know that. Customers also understand that Microsoft is getting better and better at making sure its products interoperate with others, even on other platforms, and Microsoft’s partners have to facilitate that. We get questions from customers pretty frequently about Mono, and lately those questions have gotten more specific, so it seems prudent to investigate any technical blockers from time to time.

So let me re-state what Robert said, and what I said above — this is an initial assessment, a technical experiment only, not a shipping product.  Rob’s post (and mine) are not a product announcement — this is a blatant “trial balloon.”  We want to hear what the market thinks of Digipede on Mono.

Why might this be interesting? Let’s back up a step and take a look at enterprise grid and HPC deployments.

Most enterprise customers have what is often called a “mixed” IT environment. That’s a euphemism for an unplanned and chaotic assortment of technologies that have piled up over the years into some type of barely-managed infrastructure. In almost every enterprise, Windows runs on most or all of the desktops. In almost every enterprise, there is some mixture of Windows and Linux servers, with maybe some Solaris and/or other UNIX flavor(s) thrown in. In almost every enterprise with an HPC infrastructure, most or all HPC nodes run Linux.

This is just reality — Windows is miles ahead in 2008 desktop market share, and Linux is miles ahead in 2008 HPC market share. Do I wish it were different? Sure — if Microsoft had a bigger share of the HPC market (and we’ve been working diligently to help make that happen), we’d have an even bigger market into which we could sell our software. And that will happen, I have no doubt. We tell all our customers “Windows HPC Server is the best option for adding power to a Digipede grid,” and that’s the truth. Go buy some now.

But the fact remains, there’s a lot of existing infrastructure — desktops, 32-bit Windows servers, Linux servers and cluster nodes, Solaris servers, and more — that enterprises are not going to throw away.  All this infrastructure represents potential grid computing power.  The Digipede Network has always run on heterogeneous Windows networks — with Agents running on 32- and 64-bit Windows desktops, 32- and 64-bit Windows servers, and cluster nodes running Windows HPC Server (formerly Compute Cluster Server). Our reluctance to include boxes that don’t run Windows has always been mostly about applications — it’s still relatively rare to find applications that are actually deployed across multiple operating systems simultaneously.

But as Mono gets better and better, we hear from enterprise customers and prospects who are getting more interested in it. They like the idea of being able to use more of their existing infrastructure more efficiently. They want to take advantage of Digipede’s great developer experience to deploy more applications — with minimal changes to that infrastructure.

So let’s get back to Robert’s closing question: “Now that we can do it, what should we do with it?” What do you think? Is the market crying out for a multi-OS .NET grid? Or is what we’re hearing just idle curiosity?  Let’s hear from all sides.

Yet at this conference, full of the so-called thought leaders in high-performance computing, I once again ran into many instances of unthinking knee-jerk Microsoft bashing. While much of the IT world has come to grips with the fact that Microsoft (like gravity) is likely to be around a while, the Supercomputing crowd still has some holdouts. Literally, I heard people claim that “nobody” would use Windows for high-performance computing (provably incorrect), and that positive coverage of Microsoft’s HPC offerings was “bought, not earned” (unsubstantiated rubbish). Another blogger in this field (John West of insideHPC.com) told me that he’s had “…readers take the time to send me an email saying they would never read my stuff again if I kept covering MS’s CCS products and their entry into HPC positively.” What?!?

It is amazing to me the level of religious ferver that Microsoft still inspires. The bashers out there can be perfectly calm and reasonable about a wide range of topics – but say the word “Microsoft,” and they turn bright red and irrational. I have watched this phenomenon for years, and still find it inexplicable. Microsoft is a company. That company makes software. Some of their software is very, very good. Some of it is remarkably bad. I don’t understand why some people find it so hard to remain objective (or even civil) when discussing their products and market presence.

Many Microsoft bashers think that all of us at Digipede are mouthpieces for the Evil Empire, and that we are just pawns of the Microsoft machine. On the other hand, while we have plenty of fans within Microsoft, there are also some Microsoft employees who think we are difficult annoying troublemakers (especially me).

In fact, none of us at Digipede love or hate Microsoft – we work with Microsoft. We do so for real-world business reasons that help us change the world for the better while building a great company. We work with other companies too, but Microsoft occupies a special place in the technology landscape, and we work very, very hard to understand how to work with them to our mutual benefit. There are some great people there doing great things, and the bashers only hurt themselves by blinding themselves to these very real contributions.

Microsoft’s HPC initiatives in the past three years have greatly increased their presence in the HPC market at a time when that market is expanding rapidly. Their HPC offerings have some advantages and disadvantages compared to other products in the market, and should be evaluated on those terms. Microsoft bashing lowers the level of discourse to a useless level, at a time when we can all benefit from a more objective and reasoned discussion of how they affect our market.

I’ll stop there for now, and climb into my asbestos suit…

If you’re here and want to meet up, please call my cell — 510-326-1761.

Much of the time I’ll be in the booth of our partner AMD (thanks guys!) showing off the latest version of the Digipede Network running on Windows Compute Cluster Server (CCS) on some very cool hardware from Scalable Servers. Come and see!

It’s time to leak some information about the Digipede Network Version 2.0, coming this summer to a grid near you.
Never mind that we’re plagued by the same version-number inconsistency that affects every other software company (and yes, our most recent release really is called 1.3.2f; want to make something of it?) — this is the Summer of 2.0.

We’ve been chewing away on suggestions (ranging from obvious to brilliant) brought forth by our customers, partners, prospects, and fans over the past several months, and the new release will have something for everybody.

Shortly after we brought out 1.3, I wrote here that this release was about “developers, developers, developers.” And it was. And it is. And it shall be to come. It’s no secret now that the secret sauce of the Digipede Network is in the way we make it drop-dead simple for developers to adapt real-world applications to a grid — without changing programming paradigms or development tools.

So first and foremost — you’ll see more of the same. Developers get more ways to manipulate pools of compute resources, more development patterns for definition of jobs and tasks, more code samples to jump-start their own applications — more of the best SDK in the business.

Second, you’ll see the evolutionary new Digipede Control. Yes, it’s familiar — but it’s more powerful, with new ways for administrators to work with larger groups of Digipede Agents, users, and applications. As we work with customers with larger and larger grids, we’re constantly improving overall manageability; 2.0 will continue this trend, and so will 2.1, and 2.3.2f.

Third, you’ll see your results even faster.  You’ll see lower latency throughout the system, improving performance on short tasks significantly. And with more clever options for taking advantage of multiple cores, broad classes of applications can now see even greater performance increases with very little effort.

Fourth, you’ll see even more integration with Microsoft products.  You’ll be able to schedule jobs directly from the Digipede Network onto a Windows CCS cluster via the CCS Job Scheduler (nice).  You’ll have a Visual Studio plug-in that allows debugging of a master application and a distributed application all on a single machine (sweet).  And — well actually, there’s lots more, but why give it all away now?  I’ll have more to say as we get closer to a release date.  

Marc writes insightfully on a variety of issues related to grid and high-performance computing, and he’s recently finished his best and most comprehensive series to date.  It’s a seven-part series of posts based on a speech he gave on the state of high-performance computing in finance, at a Microsoft HPC event in New York earlier this year. 

I was fortunate to be at the HPC event, to attend Marc’s presentation, and to watch as the attendees learned a lot about the real needs of customers in this market.  Read, and see what I mean:

• Part One
• Part Two
• Part Three
• Part Four
• Part Five
• Part Six
• Part Seven

I will not attribute causality, but I will note that in the six months since Marc gave this presentation, Digipede sales are up more than 600% over sales in the year-earlier period, and that the majority of that increase is in financial services. 

Thanks to our friends at HP, our decision not to have our own booth at SIFMA worked out great.  We camped in the front corner of their booth for three days — vastly more productive real estate than any independent small booth would have been for us. 

And thanks to our friends at Microsoft, we had plenty of other customer-facing opportunities away from the show floor (their two receptions in their super-crowded 4th-floor suite had acceptable drinks, mediocre food, and really, really good content — exactly the right combination to bring in the right audience!). 

Things I saw that were interesting:

• HP blades.  I know, I mostly write about software, and if you’re not into hardware, skip ahead.  But I have to say — running our software on the HP cluster in their booth was just a pleasure.  Yes, they use the same Intel and/or AMD chips everybody is using, but that’s where the similarity ends.  HP continues to extend its lead in manageability, convenience, cooling, and more.  We were running demonstrations of the Digipede Network on six dual-core, dual-processor C-class blades with Microsoft Windows Server 2003 Compute Cluster Edition (CCE), and the results were impressive.  Even a small system like this (a “grid-in-a-box” configuration) can provide order-of-magnitude increases in application performance for many computational finance applications.  And the real beauty of this setup is how simple it is to expand once you get started — plug in more hardware, and expand your grid.  Link it to your existing desktops and servers throughout your business, and expand your grid further.  All just as smooth and slick as you please.  You will not find a simpler or smoother introduction to the grid experience than an HP cluster running CCE and the Digipede Network Professional Edition.  Period. 
• The Microsoft / GigaSpaces demonstration of a high-volume market data application.  It looked slick, and showed off low-latency functionality from our friends at GigaSpaces.  Quite cool — and another indication of Stevan Vidich’s impressive breadth of expertise in creating important solutions using technology from Microsoft and its partners.  The Microsoft partner ecosystem is among Microsoft’s most valuable assets — but an underappreciated asset among many at Microsoft.  Kudos to Stevan for continuing to show the way!
• The Microsoft / Linedata / Lab49 demonstration of a trading application for buy-side customers.  While the underlying functionality was interesting, the presentation layer stole the show.  New apps that really take advantage of Windows Presentation Foundation are going to make a real impact faster than most people realize — and will drive a lot of business for Microsoft and its partners.  In financial services, our friends at Lab49 provide a rare combination of top-notch development expertise and deep domain expertise — and that combination showed through in this demonstration.  Great job, guys.

And the other thing I saw that was interesting was — customers.  While many vendors complained that this year had few real customer prospects, that was not our experience.  We saw customers from small hedge funds to big investment banks that were interested in scheduling follow-up appointments immediately.  Financial services remains the hottest market for grid computing, and it’s moving our way.  So now I’m off to follow up on a thick stack of business cards from prospective customers, partners, and more.  See you all there next year.

If you have not been to this event, it is unlike any other trade show I’ve ever attended. Located entirely within the New York Hilton at 1335 Avenue of the Americas (between 53rd and 54th Streets), there are 7000+ attendees, and apparently an equal number of booths. There are booths in the ballrooms, booths in the exhibit halls, booths in the hallways, booths in the pretty much every location but the restrooms (and they crowd right up against those, too). In the exhibit halls, the feel is pretty much “big trade show;” in the hallways, the feel is more “Middle-eastern street market,” or, during peak hours, “mosh pit.”

This year, the name may be changed, but the song remains the same — happy chaos.  This year, there’s no Digipede booth.  We’ll be camped in the HP booth, running demos of our software running on Windows Compute Cluster and some nice HP cluster hardware — and running between the many customer and partner meetings already scheduled.  I go to New York a lot — I grew up there, and it’s always fun working at New York speed – but next week promises to crank that up another couple of notches. 

If you will be at the SIFMA event, just email me (john at digipede dot net), and we’ll find a time to get together.

Microsoft Research has been doing amazing work in the field of vaccine design for AIDS.  Here is the Microsoft press release about the importance of this work.  

Here is Dan Fay’s most recent post about this subject.  Look who’s in his first sentence.  

Also public is the contribution that Digipede CTO Robert Anderson has made to this effort, including posting some code on CodePlex to get important parts of this analysis running on the Digipede Network. 

No more dots today.

Actually, no one from Digipede is physically at TechEd — but if you’re there, look in your bag!  See that VSIP Partner Resource DVD?  We’re on there!  You now have your own free copy of the Digipede Network Developer Edition — install it, try it, and see what all the excitement is about.

(I’ve said it before and I’ll say it again — VSIP rocks.  The program, the people, the exposure, the focus — it’s all good.  Pound for pound, it’s the best partner program at Microsoft, period.  If you make software for the Microsoft platform, find a way to make it touch Visual Studio, and join VSIP now.  Even their Web site sucks less than it used to.)

And if you’re NOT at TechEd — find one of those DVDs!  Microsoft makes zillions of them.  Or just get the Digipede Network Developer Edition freshly brewed on the Digipede Web site here.

Enterprise architects in financial services are looking to service-oriented architectures (SOA) to address many real-world problems – brittle systems with tight interdependencies, data stuck in single-purpose silos, and applications that don’t scale to meet growing demand, to name a few.  But implementing an SOA can also expose new scalability issues.  New high-performance computing (HPC) offerings from Microsoft and its partners are ideally suited for scaling out compute-intensive components of an SOA.  Using real-world examples from financial services companies, this presentation will describe how to grid-enable compute-intensive analytic services for use in an SOA.  

You can sign up for the event here.  (True, somehow all references to Digipede, John Powers, and Dan Ciruli are omitted from this description, but it’s us nevertheless.)  Hope to see you there!