In a single sentence containing at least three significant understatements, Ms. Gage writes:

Microsoft still gets criticized sometimes for being slow to the Internet or hard to do business with, but Lewin has won praise over the years for his courtesy, efficiency and ability to connect outsiders to the right people inside Microsoft, which is not an easy task.

Whew.  Let’s parse that.

“Microsoft still gets criticized sometimes for being . . . hard to do business with . . .”  There is no question that doing business with ANY huge company is hard.  Building a close relationship with Microsoft (or any tech giant) is not for the faint of heart.  Microsoft presents some special challenges that I could go on about at length (oh wait, I’ve done that multiple times…), but let’s just stipulate that some of these criticisms are justified while some are not.

“…but Lewin has won praise over the years for his courtesy, efficiency and ability to connect outsiders to the right people inside Microsoft…”   Bingo.  Dan’l Lewin has done more to expose the helpful side of Microsoft to startups, entrepreneurs, and VCs than anyone would have thought possible just a few years ago.  His Emerging Business Team is the API for startups that want hooks into Microsoft.  Digipede has received numerous tangible and intangible benefits from working with the EBT; the group brings the attitude that they can’t wait to help interesting startups, and it’s Dan’l who sets the tone and agenda for that critical group.

“…which is not an easy task.”  No kidding.  I’m back to my API analogy.  If you would rather to try to reverse-engineer the Microsoft org chart from the outside, good luck — but a single call to the EBT can get you to the right person within Microsoft faster than any other method I know.

The Bay Area is teaming with “experts” who would have us believe that Microsoft has become irrelevant.  In my experience, entrepreneurs ignore Microsoft at their peril.  Far better to understand what they’re doing and why than to pretend they aren’t there.  Dan’l and his team are great resources for entrepreneurs who want to understand and work with Microsoft.  So — good job, Ms. Gage, for profiling Microsoft’s local champion of innovation.  Well worth reading.

In exactly one week of long, sunny days, the system of 18 panels produced 154 kWh — or 22 kWh per day.  In that same period (which included some nice days and some very hot days — it was over 100 degrees F at our house yesterday), the new meter tells me that our house used all that plus 151 kWh more from the grid.  So even with the pool filter running and above-normal air conditioner usage, the solar panels produced almost exactly half the electricity used by our large-ish suburban California home.  Not bad, given that we could only put solar panels on a relatively small fraction of the roof (too much shade on the rest of it).

More economic analysis to follow after I get a bill or two.

Case in point: Marc Adler’s recent post about Velocity, in which he says:

. . . It is no secret that the most prevelant use of object caches on Wall Street is with Grid Computing. How will Velocity interface with Compute Cluster? How about Digipede (if I know John Powers, he probably has support for Velocity already)?

Well, Marc, if you’re going to post on Sunday mornings, you may not get instant confirmation of your clever guesses, but now it’s Monday afternoon, so I can confirm — yes, we have a working PoC in the lab at Digipede, proving (to ourselves at least) that the Digipede Network and Velocity work great together. And yes, it provides significant performance improvements for certain types of activities important to Wall Street folk.

If anyone is still wondering why we chose .NET as the basis for our grid computing software, this is just the latest example — Microsoft just keeps giving us great free stuff on which to build. The Velocity CTP came out last week; this week, we have working code that provides real benefits.

Rob just posted his initial comments; there will be more.  And we’ll have feedback for the Velocity team.  Watch this space (and Rob’s, and Dan’s…).

In any case, Derrick Harris of Grid Today has done a great job of connecting the dots for us in his excellent article today.  Read the whole article, because he offers good insight on how important this announcement really is, but here’s his analysis of how it affects Digipede:

Whatever emerges from Velocity also should be good news to Microsoft’s technology partners — in particular Digipede, which has been delivering distributed computing to .NET apps and now might get the add-on technology it needs to compete with the big boys. Digipede has received no shortage of praise from customers and commentators alike about its relatively inexpensive and very user-friendly solution, but one of the drawbacks has been its limitation in terms of what types of jobs the Digipede Network can handle, namely CPU-intensive jobs benefitting from parallel processing. If Microsoft and Digipede can make Velocity and the Digipede Network function as a unit and keep the price down, Digipede could find itself selling to a whole new, real-time-data-loving audience. That this integration will occur is pure speculation on my part, but it seems to make sense on the surface.

I have no comment on specifics at the moment, but let’s just say — Derrick, you nailed it.

And as he reports, thanks to improvements in both Mono and the Digipede Network, the answer is — not much. We’ve got a working prototype of a Digipede Agent running under Mono on Linux that runs a Digipede job.

Digipede on Linux? Has the world turned upside down?

Hardly.

Since the beginning, Digipede has been focussed on adding value to the Microsoft platform. And customers know that. Customers also understand that Microsoft is getting better and better at making sure its products interoperate with others, even on other platforms, and Microsoft’s partners have to facilitate that. We get questions from customers pretty frequently about Mono, and lately those questions have gotten more specific, so it seems prudent to investigate any technical blockers from time to time.

So let me re-state what Robert said, and what I said above — this is an initial assessment, a technical experiment only, not a shipping product.  Rob’s post (and mine) are not a product announcement — this is a blatant “trial balloon.”  We want to hear what the market thinks of Digipede on Mono.

Why might this be interesting? Let’s back up a step and take a look at enterprise grid and HPC deployments.

Most enterprise customers have what is often called a “mixed” IT environment. That’s a euphemism for an unplanned and chaotic assortment of technologies that have piled up over the years into some type of barely-managed infrastructure. In almost every enterprise, Windows runs on most or all of the desktops. In almost every enterprise, there is some mixture of Windows and Linux servers, with maybe some Solaris and/or other UNIX flavor(s) thrown in. In almost every enterprise with an HPC infrastructure, most or all HPC nodes run Linux.

This is just reality — Windows is miles ahead in 2008 desktop market share, and Linux is miles ahead in 2008 HPC market share. Do I wish it were different? Sure — if Microsoft had a bigger share of the HPC market (and we’ve been working diligently to help make that happen), we’d have an even bigger market into which we could sell our software. And that will happen, I have no doubt. We tell all our customers “Windows HPC Server is the best option for adding power to a Digipede grid,” and that’s the truth. Go buy some now.

But the fact remains, there’s a lot of existing infrastructure — desktops, 32-bit Windows servers, Linux servers and cluster nodes, Solaris servers, and more — that enterprises are not going to throw away.  All this infrastructure represents potential grid computing power.  The Digipede Network has always run on heterogeneous Windows networks — with Agents running on 32- and 64-bit Windows desktops, 32- and 64-bit Windows servers, and cluster nodes running Windows HPC Server (formerly Compute Cluster Server). Our reluctance to include boxes that don’t run Windows has always been mostly about applications — it’s still relatively rare to find applications that are actually deployed across multiple operating systems simultaneously.

But as Mono gets better and better, we hear from enterprise customers and prospects who are getting more interested in it. They like the idea of being able to use more of their existing infrastructure more efficiently. They want to take advantage of Digipede’s great developer experience to deploy more applications — with minimal changes to that infrastructure.

So let’s get back to Robert’s closing question: “Now that we can do it, what should we do with it?” What do you think? Is the market crying out for a multi-OS .NET grid? Or is what we’re hearing just idle curiosity?  Let’s hear from all sides.

Sure, the individual steps like specifying, developing, testing, documenting, and planning support for new software features are difficult enough — but the discipline of knowing when to STOP adding features, and to focus instead on finishing a complete, polished, release-ready product is tougher than it sounds to those outside the industry.

In any software organization worthy of the name, there are more good ideas than can possibly be put into any specific product release. There are also just a stunning number of bad ideas competing for inclusion in shipping products (I am notorious within Digipede for proposing needlessly specific bad ideas. Mercifully, my partners of 20 years have honed their skills in talking me out of the worst of them.)

We decided early on at Digipede that our feature set would be guided by three principles: Performance, simplicity, and a focus on adding value to the Microsoft platform. Over the past five years, these principles have helped us make decisions on what to include (and as importantly, what to exclude) from our software.

Last month, we reached general availability of the latest release of the Digipede Network, Version 2.1. You can see what’s new in this release on our Web site, but now that our customers have had an opportunity to upgrade, let’s look at a few of the specific new features to see how we did in sticking to those principles:

Job concurrency: The improved Digipede Agent software can manage different applications running simultaneously on multiple cores of a single compute node, maximizing utilization of compute nodes on the grid. Users can set Job Concurrency values to allow the Digipede Agents to work on multiple jobs simultaneously: designate which applications can safely run with other applications, which applications can run side-by-side with themselves, and which applications are not compatible for concurrent jobs.

Performance! This one is just amazing. As new machines ship with more and more cores inside, I am continually baffled at the lack of attention from ALL the major vendors out there about how to take advantage of those cores. Sure, Intel talks about compilers and Microsoft talks about Parallel Extensions and so on — but in shipping products in 2008, there’s just incredibly little help for users and developers who want to take advantage of multi-core processors. What we shipped in Version 2.0 last September is still miles ahead of other software options in terms of both development patterns and execution modes for multi-core processing. With Version 2.1, we’ve extended that lead significantly — if you want to take advantage of dual-CPU quad-core servers and desktops TODAY, you need to take a look at how the Digipede Network handles concurrency. Watch the 4-minute video that shows how, then get an evaluation copy of the software and try it yourself!

Management APIs: New management APIs give developers programmatic ability to create, modify, and delete resource pools. (Available in Professional Edition only)

Performance (specifically, scalability), and Simplicity (of grid management). A browser-based UI for grid management is great — for small grids. As our customers deploy larger and larger grids, they need both the browser-based UI of Digipede Control and a wider range of tools for the programmatic manipulation of grid resources. It is vastly simpler to take advantage of thousands of grid nodes through simple extensions to our management API.

Risk-free sharing: “Pool Rank” permits risk-free sharing of resources: you can add your servers to the enterprise grid and ensure that they always work on your jobs first. That means that by joining the grid, you can only improve your application performance. You can donate your cycles when you are not using them without worrying that your application performance will degrade, because you are always guaranteed that your machines will work for you.

Performance and Simplicity. We’ve also referred to this feature as “Selfish sharing.” We hear from other grid vendors about how users “must” get over the practice of “server hugging.” We try not to be so arrogant; we’ve never found that scolding our customers is good business practice. If customers want to preserve unconditional priority on their own servers, we say “good for them.” So we’ve built a straighforward way to preserve absolute priority for the resource owner, even when they offer to share surplus resources. From what our customers tell us, we think this approach encourages efficient resource sharing far more than lecturing ever would.

First Grid Computing Solution Certified for Windows Server 2008: We followed the long and winding road of the Early Adopter program to become the first grid solution to obtain this important certification, so that customers can be confident that our software works not only with the Microsoft products they use today, but with all the latest improvements Microsoft is bringing to market now.

Performance, Simplicity, and Microsoft focus. By aligning with Microsoft’s technology and strategy, we help our customers create a truly dynamic IT infrastructure. Server 2008 brings many benefits in performance and manageability, and we’re confident that our customers will be upgrading quickly (more quickly than, say, to Vista); we want to be sure they can use our latest capabilities on Microsoft’s best OS platform.

Let me be candid here; these benefits do not come free to ISVs. I have considerable anxiety over extending yet further the number of versions of Microsoft products we support — for example, while I think Server 2008 is great, and Visual Studio 2008 is great, and the new SQL Server 2008 will be great, staying current means we’ll have to start enforcing our requirements by turning away requests for support of Windows 2000 and SQL Server 2000. The combinatorics for testing on multiple OS versions, .NET versions, SQL Server versions, IIS versions, and upgrade paths for our own software versions get out of hand quickly. I’ll have more on this issue another day. For now I’ll just say I’m happy with our decision to stay current — mostly.

Automatic Failover Package and Integration with NLB: Failover has long been a feature of the Digipede Network Professional Edition but with the optional Automatic Failover Package, organizations can now have complete out-of-the-box integration with Windows Server 2008 load balancing, giving “hands-free” failover to mission-critical applications.

Performance, Simplicity, and Microsoft focus — yes, even this advanced capability was guided by our goal of simplicity. While automatic failover is often considered a complex requirement, we made some basic decisions to keep it as simple as possible. First, we made automatic failover it’s own SKU, so customers without the need for high-availability configuration don’t even have to think about it. Second, we did away with a lot of the manual scripting that often slows implementation of failover solutions — you can have it running very quickly. Finally, we left as much as possible to popular existing technologies — SQL clustering and NLB — so the implementation steps will be as familiar as possible.

Reports Package: Assembles critical information about the use and optimization of the grid, with easy-to-understand charts and graphs, flagging of critical information, and drill-down capability, giving enterprises fully integrated optimization of grid performance, with tracking of who contributes to and who benefits from grid resources.

Performance, Simplicity, and Microsoft focus — In larger systems, simple and informative visual tools are essential for wringing the most performance possible from a grid. Users and administrators become far more productive in their routine monitoring functions and troubleshooting activities with this new package, which plugs directly into Digipede Control (our admin UI). And by building on SQL Reporting, we’ve created a framework for future extensions.

Overall, I’m pleased with the extent to which we have driven the improvement of our product by staying focused on the three principles described above. To be a little less self-congratulatory, I wish we had stopped adding features at least two months earlier and brought most of these capabilities to market sooner, rather than piling quite so much into a single release (and there’s certainly more than I’ve had a chance to discuss here).  Perhaps another day, I’ll have a chance to discuss some of the things we (purposely) left out!  Now that V2.1 is in the market (and getting rave reviews from our customers), I’m eager to see what great new applications our imaginative customers create and deploy on our latest platform.

This wonderful opportunity led to the “while-we’re-at-it” syndrome that plagues so many home repair / improvement projects. So while we’re replacing the roof — why not investigate installing a photovoltaic system at the same time?

And after a bit of analysis (more on that another day), we decided to go for it. The process of collecting bids and testimonials went smoothly (hint — the housing market really is as soft as people say; contractors are hungry!), and after a bit of permitting and paperwork — BAM, new roof and new solar panels. From two signed contracts (roofer and solar guys) to completed project took less than 10 days, including Memorial Day weekend. PG&E will be out “soon”with a new meter, and we’ll be getting about half our electricity from the sun.

The whole system will pay for itself in about 8 years (given California’s outrageous electric rates), and we’ll take a few toes off our carbon footprint.

Panels like these were quite rare even here in California even 2-3 years ago. Now, they’re popping up all over. The California Healthcare Foundation (where Marian works) just put up a 100 KW system, which is 30 times the capacity of ours. Google put in a 1.6 MW system a year or two ago. Walmart has started putting them on stores here. It’s not enough to account for much power yet, but it’s a big change in a short time.

If you want to find out how solar might work for you, call Jason at Borrego Solar, and tell him I sent you — they did a great job (and no, I don’t have any financial interest in Borrego).

His article is in two parts, and this first part provides a good overview of the complex problem he’s facing — he confronts issues in financial modeling, data sources, genetic models and grid computing. As a result, Part One does not dig too deeply into coding details. But it’s worth a read — you’ll understand the architectural decisions he’s facing, and how he’s planning to address them. Plus, from what I’ve heard about Part Two (which will be out in June), there’s plenty of detail (and code) coming.

This is the second time in two months that we’ve seen influential financial modelers implement their public examples using the Digipede Network (see also Matt Davey’s recent Dr. Dobb’s article).

This is consistent with what we’re seeing from customers. While there are many grid offerings in the market, there seems to be a growing consensus that if you use .NET, there are significant advantages to working with a grid solution built on .NET. Or conversely, there’s no point trying to fit a square peg into a round hole — i.e., there’s no point trying to graft a .NET application onto a grid built for other technologies when a better option exists.

This is the “application centric” view — grids should follow applications, making it easier for developers to adapt applications to a grid, even if that means limiting the options for running those applications to a particular set of resources (in Digipede’s case, Windows machines running .NET).

The other view is “infrastructure centric” — that OS should not matter, that a grid should allow applications to be deployed across all resources, even if that means restricting the application technologies and development patterns allowed for such deployment.

Digipede has been unapologetically in the “application centric” camp for five years now, but what do others think? Has Derek made a wise choice by trading off ease of development for deployment limited to a single OS? We think so, but let’s hear from you!

Matt illustrates his ideas with some sample code that can “price trades in parallel on a Digipede grid,” with the results returned directly to his client LINQ application — simple, elegant, and very useful.

Matt has many more ideas for financial developers — his blog, Tales from a Trading Desk, is a must-read.

Never one to leave well enough alone, my colleague Dan Ciruli’s recent commentary on Matt’s workmentions Digipede’s concurrency patterns for taking advantage of multi-core processors. We’ll have lots more on that shortly.

As reported here last week, Digipede had a station in the Partner Pavillion at the Windows Server 2008 launch in LA last Wednesday.  I watched Ballmer (good), some Microsoft’s demos (pretty good), and learned yet more about the latest huge wave of innovation coming out of Redmond (excellent). 

But mostly, we were at the event to show and tell, not just listen and learn.  Any investment in a booth or station at an industry event has to be weighed at least in part by the exposure gained and leads gathered — and in my judgment this one was a success, thanks to some innovative preparation by the Digipede team.

First of all, we decided months ago to time our press release announcing Version 2.1 of the Digipede Network to coincide with the Server 2008 launch.  That involved some careful product decisions (hard, but essential anyway), doing a press release (easy), getting Microsoft’s approval and participation in that press release (time-consuming, but feasible), and earning the Certified for Windows Server 2008 designation (hard, and quite time-consuming, but worthwhile in the love ). 

We have been talking about making stickers out of the Digipede logo for some time — our logo looks cool, and just seems to belong on a sticker.  Robert Anderson sent around an innocent note about a month ago, which said, in its entirety, “Should we get some Deatle stickers made for the launch?    They can’t be too expensive.”  (Deatle is the name of the creature on our logo.)  The idea of seeing lots of Digipede stickers at the Server 2008 launch suddenly became compelling to all of us, and we set about making it happen.

There followed several rounds of innovative sticker design (by our own Nathan Trueblood and our wonderfully patient and creative graphic artist Norma McDonough), which resulted in the blue-and-silver oval seen here. 

But even stickers as lovely as this don’t just display themselves, and my suggestion of sneaking in early and plastering them on every wall and post was wisely overruled (as uncool, quite possibly illegal, and certainly easily traced).  So — Where could we reasonably put hundreds of stickers without incurring the wrath of the LA Convention Center?

Dan Ciruli’s elegant solution was to get the attendees to wear them at the event, using the time-tested technique of bribery.  “Wear a Digipede sticker for a chance to win a Zune” was the original idea.  We thought about an XBox, but those are quite a bit more expensive and harder to ship…  But as luck would have it, I attended another Microsoft partner event up in Bellevue in mid-February, and actually WON an XBox in a drawing.  I re-gifted XBox to Digipede for use at the launch, on the condition that none of my partners could tell my 12-year-old twin boys that I had owned an XBox for half an hour and not brought it home.  (They don’t read this — so everyone shhhhh.)

It turns out that in a room full of Windows developers and IT geeks, you definitely want to be giving away an XBox.  At first, traffic was a little light, and nobody was taking the stickers.  But within the first half hour of the event, as soon as a few people were wearing the stickers, other attendees got wind of the contest, and traffic picked up.  The rules were simple — if a Digipede spotter sees you wearing a Deatle sticker at an undisclosed time and place in the LA Convention Center, you’ll win the XBox. 

These few rules got people thinking — I have to wear this sticker somewhere visible, and I can’t just restrict my movements to hanging around the Digipede station (or even the Partner Pavillion).  At the Digipede station, people were putting stickers on their jackets, on their backs, on their backpacks, on their laptops, and more creative spots.   Soon, Deatles were crawling everywhere.  By early afternoon, even when I was out of the Partner Pavillion at lunch or in the halls or in the Microsoft Pavillion, people were walking up to me and asking — “I see those stickers everywhere; what the hell is Digipede?”  This was definitely working.

All the while, at Station 56 in the Partner Pavillion, Dan, Nathan, and I were keeping reasonably disciplined about striking up a brief conversation with each and every visitor.  Anyone who ONLY wanted a sticker for a chance to win the XBox was more than welcome to it; but many others stopped long enough to engage about the amazing grid computing software we were demonstrating as well.  (Some of the demos we did were similar to this.)  We gathered many leads this way, all of whom were at least partially qualified in person before scanning their badges (we did not scan those who just wanted a chance at the XBox – they don’t need to receive more emails from us, and we don’t need to fill our CRM system with them). 

There were LOTS of other giveaways going on throughout the Partner Pavillion – but they were all of the type “dump your card or scan your badge here and you might win a something.”  This is a popular approach, and no doubt the folks running these will think they got “more” leads than we did — but in my experience, a card dropped in a bowl (or a badge scanned for a prize) is not nearly as useful as a badge scanned after a qualifying conversation and/or demonstration.

(Note to the Microsoft organizers of this event – you did a PERFECT job with the partner lead collection system.  There was none of the insane doublespeak about who gave permission to share what leads with which partner — everything worked exactly as the partners and attendees would expect.  No doubt Microsoft’s legal legions are wringing their hands over the idea that Microsoft partners collected leads from Microsoft customers at a Microsoft event using a lead collection system featuring badges created by Microsoft — but unlike prior events, this was very well done!)

About 10 minutes before the Partner Pavillion closed, Dan and I went out to the main escalator, picked a random number between 1 and 60 (using a handy random number generator I wear on my left wrist — the number was 16), and started counting Deatles.  I would estimate that OUTSIDE the Partner Pavillion, about one in 10 attendees we saw was wearing a Digipede sticker.  (This means that no attendees with their eyes open could have missed seeing quite a few of these stickers.)  When we reached the 16th sighting, we awarded the XBox to a lucky Glendale-based IT professional, who was appropriately excited and appreciative.

So what did we get out of all this? 
1.  A large number of partially qualified leads with whom to follow up.
2.  A lot of brand awareness among Server 2008 users, buyers, and technical influencers.
3.  Additional visibility with the Microsoft Server team.

Worth the trip?  We think so, but as always, we won’t know for sure until we’ve worked the leads some more.  John’s One and Only Rule of Trade Shows — “Work it, or it’s not worth it.”