Digipede CTO Robert Anderson is blogging about a recent experiment we’ve conducted in our lab, assessing what it would take to get the Digipede Agent running on Mono. (For those of you who don’t know, Mono is a cross-platform implementation of .NET, developed as an open-source project led by Miguel de Icaza, and sponsored by Novell.)

And as he reports, thanks to improvements in both Mono and the Digipede Network, the answer is — not much. We’ve got a working prototype of a Digipede Agent running under Mono on Linux that runs a Digipede job.

Digipede on Linux? Has the world turned upside down?

Hardly.

Since the beginning, Digipede has been focussed on adding value to the Microsoft platform. And customers know that. Customers also understand that Microsoft is getting better and better at making sure its products interoperate with others, even on other platforms, and Microsoft’s partners have to facilitate that. We get questions from customers pretty frequently about Mono, and lately those questions have gotten more specific, so it seems prudent to investigate any technical blockers from time to time.

So let me re-state what Robert said, and what I said above — this is an initial assessment, a technical experiment only, not a shipping product.  Rob’s post (and mine) are not a product announcement — this is a blatant “trial balloon.”  We want to hear what the market thinks of Digipede on Mono.

Why might this be interesting? Let’s back up a step and take a look at enterprise grid and HPC deployments.

Most enterprise customers have what is often called a “mixed” IT environment. That’s a euphemism for an unplanned and chaotic assortment of technologies that have piled up over the years into some type of barely-managed infrastructure. In almost every enterprise, Windows runs on most or all of the desktops. In almost every enterprise, there is some mixture of Windows and Linux servers, with maybe some Solaris and/or other UNIX flavor(s) thrown in. In almost every enterprise with an HPC infrastructure, most or all HPC nodes run Linux.

This is just reality — Windows is miles ahead in 2008 desktop market share, and Linux is miles ahead in 2008 HPC market share. Do I wish it were different? Sure — if Microsoft had a bigger share of the HPC market (and we’ve been working diligently to help make that happen), we’d have an even bigger market into which we could sell our software. And that will happen, I have no doubt. We tell all our customers “Windows HPC Server is the best option for adding power to a Digipede grid,” and that’s the truth. Go buy some now.

But the fact remains, there’s a lot of existing infrastructure — desktops, 32-bit Windows servers, Linux servers and cluster nodes, Solaris servers, and more — that enterprises are not going to throw away.  All this infrastructure represents potential grid computing power.  The Digipede Network has always run on heterogeneous Windows networks — with Agents running on 32- and 64-bit Windows desktops, 32- and 64-bit Windows servers, and cluster nodes running Windows HPC Server (formerly Compute Cluster Server). Our reluctance to include boxes that don’t run Windows has always been mostly about applications — it’s still relatively rare to find applications that are actually deployed across multiple operating systems simultaneously.

But as Mono gets better and better, we hear from enterprise customers and prospects who are getting more interested in it. They like the idea of being able to use more of their existing infrastructure more efficiently. They want to take advantage of Digipede’s great developer experience to deploy more applications — with minimal changes to that infrastructure.

So let’s get back to Robert’s closing question: “Now that we can do it, what should we do with it?” What do you think? Is the market crying out for a multi-OS .NET grid? Or is what we’re hearing just idle curiosity?  Let’s hear from all sides.

Tags: Digipede, grid-computing, HPC, HPC Server, Linux, Microsoft, Mono, Solaris

In his most recent Dr. Dobb’s article, Matt Davey has some good commentary on Parallel Extensions to the .NET Framework, including PLINQ — as well as some very practical ideas about how developers can work with LINQ and the Digipede Network to build high-performance grid applications in .NET today.

Matt illustrates his ideas with some sample code that can “price trades in parallel on a Digipede grid,” with the results returned directly to his client LINQ application — simple, elegant, and very useful.

Matt has many more ideas for financial developers — his blog, Tales from a Trading Desk, is a must-read.

Never one to leave well enough alone, my colleague Dan Ciruli’s recent commentary on Matt’s workmentions Digipede’s concurrency patterns for taking advantage of multi-core processors. We’ll have lots more on that shortly.

Tags: .NET, Digipede, HPC, LINQ, Matt Davey, Parallel Extensions, PLINQ

I am back home from SC ’07, my fifth Supercomputing conference. I saw some really impressive new technology from market leaders old and new. I heard remarkable claims and forecasts from analysts, pundits, and marketing flacks alike. I learned a lot, and contributed what I could.

Yet at this conference, full of the so-called thought leaders in high-performance computing, I once again ran into many instances of unthinking knee-jerk Microsoft bashing. While much of the IT world has come to grips with the fact that Microsoft (like gravity) is likely to be around a while, the Supercomputing crowd still has some holdouts. Literally, I heard people claim that “nobody” would use Windows for high-performance computing (provably incorrect), and that positive coverage of Microsoft’s HPC offerings was “bought, not earned” (unsubstantiated rubbish). Another blogger in this field (John West of insideHPC.com) told me that he’s had “…readers take the time to send me an email saying they would never read my stuff again if I kept covering MS’s CCS products and their entry into HPC positively.” What?!?

It is amazing to me the level of religious ferver that Microsoft still inspires. The bashers out there can be perfectly calm and reasonable about a wide range of topics – but say the word “Microsoft,” and they turn bright red and irrational. I have watched this phenomenon for years, and still find it inexplicable. Microsoft is a company. That company makes software. Some of their software is very, very good. Some of it is remarkably bad. I don’t understand why some people find it so hard to remain objective (or even civil) when discussing their products and market presence.

Many Microsoft bashers think that all of us at Digipede are mouthpieces for the Evil Empire, and that we are just pawns of the Microsoft machine. On the other hand, while we have plenty of fans within Microsoft, there are also some Microsoft employees who think we are difficult annoying troublemakers (especially me).

In fact, none of us at Digipede love or hate Microsoft – we work with Microsoft. We do so for real-world business reasons that help us change the world for the better while building a great company. We work with other companies too, but Microsoft occupies a special place in the technology landscape, and we work very, very hard to understand how to work with them to our mutual benefit. There are some great people there doing great things, and the bashers only hurt themselves by blinding themselves to these very real contributions.

Microsoft’s HPC initiatives in the past three years have greatly increased their presence in the HPC market at a time when that market is expanding rapidly. Their HPC offerings have some advantages and disadvantages compared to other products in the market, and should be evaluated on those terms. Microsoft bashing lowers the level of discourse to a useless level, at a time when we can all benefit from a more objective and reasoned discussion of how they affect our market.

I’ll stop there for now, and climb into my asbestos suit…

Tags: HPC, insideHPC, Microsoft, SC07, Supercomputing

I am at the Supercomputing conference in Reno Nevada today and tomorrow.

If you’re here and want to meet up, please call my cell — 510-326-1761.

Much of the time I’ll be in the booth of our partner AMD (thanks guys!) showing off the latest version of the Digipede Network running on Windows Compute Cluster Server (CCS) on some very cool hardware from Scalable Servers. Come and see!

Tags: Digipede, grid-computing, HPC, Reno, SC07, Supercomputing

John West at insideHPC.com has come up with a great idea.  He’s doing profiles of interesting companies in or around the High-Performance Computing (HPC) market, with the intent of distilling the marketing language and press coverage and white papers and Webcasts and conference events into something unique and important about each firm.

As I read him, John’s not trying to set himself up as a new HPC market analyst a la Gartner or Forrester or the451 or the rest — but he’s trying to make sense of the market for himself and his readers.  Frankly, I can’t imagine how customers and potential customers sort through the noise in this market without some sort of truly expert and independent guidance.
We’re fortunate enough to be the second entry in his new series of profiles, filed under “the 411″ on insideHPC.com.  He captured the importance of Digipede quite well in this recent profile, at least for the HPC market (only part of our market can really be called HPC — many of our customers would not identify themselves as part of the HPC market).  Go read it!

Tags: HPC, insideHPC, John-West

Dan Ciruli and I will be at HPC on Wall Street at the Roosevelt Hotel in New York City next Monday (September 17).

We’ll be releasing Version 2.0 of the Digipede Network, and meeting with many analysts, press, customers, prospects, partners, and bartenders.  Since some of that could be pretty time consuming, we’ll also be in town September 18-19 for Microsoft meetings and more of the above.

If you’ll be there, let me know and we’ll find a way to meet up.

Tags: Digipede, grid-computing, HPC

I recently met with the folks at Windows in Financial Services — a good source of information about the progress Microsoft and its partners are making in the financial services market. 

They interviewed me about my thoughts on the current market for distributed computing in finance — I ended up talking a lot about the changing economics in this market, especially with respect to the relative costs of computing power and skilled developers.  Let me know how you think it turned out!

Tags: Digipede, distributed-computing, economics, finance, grid-computing, HPC, Windows, windows-in-financial-services

OK, if you’re reading about distributed computing, you’re probably already reading Serial to Parallel to Distributed by Marc Jacobs, but if you’re not, it’s time to start.  Marc is a former Digipede client (at a big secretive asset manager) who has since moved on to the elite financial technology consulting firm Lab49. 

Marc writes insightfully on a variety of issues related to grid and high-performance computing, and he’s recently finished his best and most comprehensive series to date.  It’s a seven-part series of posts based on a speech he gave on the state of high-performance computing in finance, at a Microsoft HPC event in New York earlier this year. 

I was fortunate to be at the HPC event, to attend Marc’s presentation, and to watch as the attendees learned a lot about the real needs of customers in this market.  Read, and see what I mean:

• Part One
• Part Two
• Part Three
• Part Four
• Part Five
• Part Six
• Part Seven

I will not attribute causality, but I will note that in the six months since Marc gave this presentation, Digipede sales are up more than 600% over sales in the year-earlier period, and that the majority of that increase is in financial services. 

Tags: Digipede, grid-computing, high-performance-computing, HPC, Lab49, Marc-Jacobs, Microsoft

After a little extra travel drama (sitting the standard three hours on the runway at JFK), I’m back in Oakland.  SIFMA was interesting, productive, and exhausting, as usual. 

Thanks to our friends at HP, our decision not to have our own booth at SIFMA worked out great.  We camped in the front corner of their booth for three days — vastly more productive real estate than any independent small booth would have been for us. 

And thanks to our friends at Microsoft, we had plenty of other customer-facing opportunities away from the show floor (their two receptions in their super-crowded 4th-floor suite had acceptable drinks, mediocre food, and really, really good content — exactly the right combination to bring in the right audience!). 

Things I saw that were interesting:

• HP blades.  I know, I mostly write about software, and if you’re not into hardware, skip ahead.  But I have to say — running our software on the HP cluster in their booth was just a pleasure.  Yes, they use the same Intel and/or AMD chips everybody is using, but that’s where the similarity ends.  HP continues to extend its lead in manageability, convenience, cooling, and more.  We were running demonstrations of the Digipede Network on six dual-core, dual-processor C-class blades with Microsoft Windows Server 2003 Compute Cluster Edition (CCE), and the results were impressive.  Even a small system like this (a “grid-in-a-box” configuration) can provide order-of-magnitude increases in application performance for many computational finance applications.  And the real beauty of this setup is how simple it is to expand once you get started — plug in more hardware, and expand your grid.  Link it to your existing desktops and servers throughout your business, and expand your grid further.  All just as smooth and slick as you please.  You will not find a simpler or smoother introduction to the grid experience than an HP cluster running CCE and the Digipede Network Professional Edition.  Period. 
• The Microsoft / GigaSpaces demonstration of a high-volume market data application.  It looked slick, and showed off low-latency functionality from our friends at GigaSpaces.  Quite cool — and another indication of Stevan Vidich’s impressive breadth of expertise in creating important solutions using technology from Microsoft and its partners.  The Microsoft partner ecosystem is among Microsoft’s most valuable assets — but an underappreciated asset among many at Microsoft.  Kudos to Stevan for continuing to show the way!
• The Microsoft / Linedata / Lab49 demonstration of a trading application for buy-side customers.  While the underlying functionality was interesting, the presentation layer stole the show.  New apps that really take advantage of Windows Presentation Foundation are going to make a real impact faster than most people realize — and will drive a lot of business for Microsoft and its partners.  In financial services, our friends at Lab49 provide a rare combination of top-notch development expertise and deep domain expertise — and that combination showed through in this demonstration.  Great job, guys.

And the other thing I saw that was interesting was — customers.  While many vendors complained that this year had few real customer prospects, that was not our experience.  We saw customers from small hedge funds to big investment banks that were interested in scheduling follow-up appointments immediately.  Financial services remains the hottest market for grid computing, and it’s moving our way.  So now I’m off to follow up on a thick stack of business cards from prospective customers, partners, and more.  See you all there next year.

Tags: c-class-blades, Digipede, finance, grid-computing, HP, HPC, Microsoft, SIFMA

Tags: ccs, Compute Cluster Server, Compute-Cluster, Digipede, HPC, Microsoft