The world is indeed getting smaller — and quicker, and better connected. And you might as well talk about what you’re doing, because smart well-connected people will figure everything out immediately anyway.

Case in point: Marc Adler’s recent post about Velocity, in which he says:

. . . It is no secret that the most prevelant use of object caches on Wall Street is with Grid Computing. How will Velocity interface with Compute Cluster? How about Digipede (if I know John Powers, he probably has support for Velocity already)?

Well, Marc, if you’re going to post on Sunday mornings, you may not get instant confirmation of your clever guesses, but now it’s Monday afternoon, so I can confirm — yes, we have a working PoC in the lab at Digipede, proving (to ourselves at least) that the Digipede Network and Velocity work great together. And yes, it provides significant performance improvements for certain types of activities important to Wall Street folk.

If anyone is still wondering why we chose .NET as the basis for our grid computing software, this is just the latest example — Microsoft just keeps giving us great free stuff on which to build. The Velocity CTP came out last week; this week, we have working code that provides real benefits.

Rob just posted his initial comments; there will be more.  And we’ll have feedback for the Velocity team.  Watch this space (and Rob’s, and Dan’s…).

Tags: .NET, Digipede, grid-computing, marc adler, Microsoft

Derek Furguson of Bear Stearns (now JPMorgan Chase) has a good article in .NET Developer Journal about how to apply genetic algorithms and grid computing to the problem of market timing in stock trading. I was pleased to see that he chose to implement his algorithms using the Digipede Network.

His article is in two parts, and this first part provides a good overview of the complex problem he’s facing — he confronts issues in financial modeling, data sources, genetic models and grid computing. As a result, Part One does not dig too deeply into coding details. But it’s worth a read — you’ll understand the architectural decisions he’s facing, and how he’s planning to address them. Plus, from what I’ve heard about Part Two (which will be out in June), there’s plenty of detail (and code) coming.

This is the second time in two months that we’ve seen influential financial modelers implement their public examples using the Digipede Network (see also Matt Davey’s recent Dr. Dobb’s article).

This is consistent with what we’re seeing from customers. While there are many grid offerings in the market, there seems to be a growing consensus that if you use .NET, there are significant advantages to working with a grid solution built on .NET. Or conversely, there’s no point trying to fit a square peg into a round hole — i.e., there’s no point trying to graft a .NET application onto a grid built for other technologies when a better option exists.

This is the “application centric” view — grids should follow applications, making it easier for developers to adapt applications to a grid, even if that means limiting the options for running those applications to a particular set of resources (in Digipede’s case, Windows machines running .NET).

The other view is “infrastructure centric” — that OS should not matter, that a grid should allow applications to be deployed across all resources, even if that means restricting the application technologies and development patterns allowed for such deployment.

Digipede has been unapologetically in the “application centric” camp for five years now, but what do others think? Has Derek made a wise choice by trading off ease of development for deployment limited to a single OS? We think so, but let’s hear from you!

Tags: .NET, Bear Stearns, Derek Furguson, Digipede, dotnet, genetic algorithms, grid-computing, JPMorgan Chase, market timing, Matt Davey

In his most recent Dr. Dobb’s article, Matt Davey has some good commentary on Parallel Extensions to the .NET Framework, including PLINQ — as well as some very practical ideas about how developers can work with LINQ and the Digipede Network to build high-performance grid applications in .NET today.

Matt illustrates his ideas with some sample code that can “price trades in parallel on a Digipede grid,” with the results returned directly to his client LINQ application — simple, elegant, and very useful.

Matt has many more ideas for financial developers — his blog, Tales from a Trading Desk, is a must-read.

Never one to leave well enough alone, my colleague Dan Ciruli’s recent commentary on Matt’s workmentions Digipede’s concurrency patterns for taking advantage of multi-core processors. We’ll have lots more on that shortly.

Tags: .NET, Digipede, HPC, LINQ, Matt Davey, Parallel Extensions, PLINQ

Marc at Magmasystems relates his findings from a recent trip to London, where he says:

Got together with some ex-colleagues, who were marvelling at the Lodnon consulting market.
The hot areas are Grid Computing, with the prevelant stack being DataSynapse and Tangasol. Also demand is picking up for WPF, with Morgan Stanley leading the way. The daily rates for qualified individuals are about 1000 pounds per day, which at the current exchange rates, is about $2000.

And

It will be interesting to see if Microsoft’s Compute Cluster Server and Digipede can make any inroads intot his market. There seems to be a very strong bias against using .NET for a grid infrastructure, something which I hope to see turned around in 2007.

It will be interesting to see, indeed.

The “strong bias” Marc reports is real — in some places.  But the financial services market is large, and surprisingly diverse.  Most of the bias we encounter seems to melt away when customers experience real benefits. 

.NET penetration is large and growing in financial services companies, and .NET workloads are (quite) difficult to adapt to a grid based primarily on Linux and Java.  We don’t have to win the hearts and minds of every Linux-centric grid user to make a big impact in this market. 

In our experience, the bias Marc describes is strongest in IT, which has been taught for years that grid computing means Linux and UNIX almost by definition.  But the developer community is different, and often more in touch with the scalability requirements of specific applications.  These are the hearts and minds Digipede and Microsoft are winning — because adapting applications to the grid needs to be easier, and that’s our strength.  Developers who use Microsoft Visual Studio to develop their applications (.NET, COM, or anything else) find the Digipede Framework SDK provides the most natural approach available for adapting their applications to a grid.

And it’s free, as part of the Digipede Network Developer Edition.  Check it out, .NET developers — it might just be your ticket to 1000 pounds a day!   Here you go. 

Digipede and Microsoft are also working together to win over the IT guys.  With the new Windows Server 2003 Compute Cluster Edition (CCE), Microsoft has made the deployment and administration of many servers as easy as one (and dropped the price for compute-grid deployments by about 80% too — you need to check this out).   There is no question that for grid computing deployments in financial services, CCE represents the most cost-effective way to add computing power to a Digipede-based grid.

So - the change Marc is hoping for in 2007 is exactly what we’re working to make happen!

Tags: .NET, DataSynapse, Digipede, dotnet, financial-services, grid-computing, Magmasystems, Microsoft

Attention .NET Developers of the greater Seattle area (I hear there are some up there…).

Digipede Evangelist Kim Greenlee will give a presentation on Concurrent Software Development at the .NET Developers Association meeting on Monday, December 11.  The meeting is on the Microsoft campus – in Building 40, the Steptoe Room (#1450).  She blogs about it here; be sure to bring this meeting announcement with you, or Microsoft security may stop you.

Tags: .NET, concurrency, Digipede, Evangelism, Kim-Greenlee, Microsoft, presentation, Software-development

Congratulations to Mike Gunderloy on the one thousandth issue of The Daily Grind.  There is no better place for your daily dose of .NET developer news.  Great work, Mike — Keep it up!

Tags: .NET, developer, Larkware, Mike-Gunderloy, the-daily-grind

Digipede is sponsoring Silicon Valley Code Camp this weekend (October 7-8).  There are many good sessions, including (so far) three by Team Digipede.  Our Evangelista Kim Greenlee has a session on concurrency, and another one on debugging in VS2005.  Our Director of Products Dan Ciruli is talking about using .NET behind Excel Services. Go see ‘em!

Tags: .NET, Code-Camp, concurrency, Dan-Ciruil, Digipede, dotnet, Excel-Services, Kim-Greenlee, Silicon-Valley-Code-Camp

OK, here’s where I get to tell the real story about the new release the Digipede Network that I could not fit into the somewhat restrictive form of a press release earlier this month.  For the facts, you can go see “what’s new.”  But for the STORY, well, read on.  First, let’s go deep into the history of Digipede, and into what the Digipede Network is all about.  Hang on — this will be fun.

When we started Digipede back in 2003, we could see the market for distributed computing getting ready to take off, and we could see where we would fit into that market.  Before we started writing code, I went around and interviewed current and former CEOs and CTOs of successful and not-so-successful software companies in distributed computing.  We read, we went to conferences, we shopped the competition.  We went to see customers and prospective customers with real distributed computing needs.  And everywhere we looked we saw — layer after layer of needless complexity.  Customers were clearly being frozen out of the market by what they perceived as an insurmountable threshhold of technical and commercial complexity.

We set three criteria for ourselves and our product very early in the process:

1. The Digipede Network must provide dramatic improvements in application scalability and performance.  This is what grid computing is all about.  Sure, it’s also about asset utilization and virtualization and automation and provisioning and federation and flexible policies and IT agility and a side of fries, but we had to start somewhere.  The Digipede Network v1.0 would be all about improved application scalability and performance.
2. The Digipede Network must be radically easier to buy, install, learn, and use than any other grid solution.  I realize this sounds like marketing language.  Before it ever became marketing language, though, it was a battle cry in our office.  From the moment we started specifying our product until the moment we released it, we kept focussed on simplifying grid computing.  Before this was marketing language, this was hundreds of individual design, development, licensing, and pricing decisions.  Simple is better.
3. The Digipede Network must be the slam-dunk choice for anyone wanting a grid computing solution on the Microsoft platform.  We saw lots of grid work being done on the Linux platform (based in part on grid computing’s academic roots, and in part on IBM’s early recognition of grid’s potential), and very little commercially interesting grid activity on Windows.  An ecosystem of grid startups had formed around IBM; we declared ourselves charter members of the Microsoft grid ecosystem, and set out to make that mean something.

With those principles in hand, we went to work.  Version 1.0 was the product of more than two years of hard-core startup work — long days, long nights, passionate arguments, inadequate resources, testing and more testing, benchmarking, documenting and more documenting, beta feedback, head scratching, and more testing.

Released June 28, 2005, the Digipede Network Version 1.0 soon won critical acclaim and a critical mass of customers, We felt like the market had validated the path we were on — so we went back to work.

Version 1.2 (don’t ask where 1.1 went) made us compatible with Microsoft .NET 2.0; we also added some cool features like easier distribution of .NET objects, integration with Visual Studio 2005, job dependency, and better — you guessed it — scalability and performance.

And then we did something else — while Version 1.2 was out there doing well, we decided to release our SDK to all developers, in a special Digipede Network Developer Edition, for free.  After all, we’d just spent all that time on some pretty developer-friendly features, and we wanted to see what clever things developers could do with it.

And developers just went nuts.

They started to try out the Digipede Network as a way to increase the scalability and performance of their applications.  Word got around (I still frankly don’t entirely understand how), and developers from Bangor to Bangalore started coming at us with every scale-out problem under the sun.  Can we use the Digipede Network to process millions of large image files?  Can we scale out tax return processing?  Can we put a bioinformatic search algorithm behind a Web site and maintain quality of service?  Can we price fixed income assets faster?  Can we predict storm damage more accurately by increasing the number of scenarios analyzed?  Can we embed the Digipede Network inside our genetic algorithms for finding new asset trading opportunities?  Can we create visual If we have SharePoint, and we publish compute-intensive spreadsheets using Excel Services, can the Digipede Network scale out the calculations in the Excel user-defined functions?  And to their surprise and delight, the answer usually came up — yes.

And so we came to a decision for Version 1.3 — time to double down.  Time to focus even more on developers.  We took our already-great APIs and opened them up further, documented them better, wrote up more examples, baked in finer-grained control of jobs and tasks, and took the suggestions of some of our smartest developer-customers to make the grid computing system of developers’ dreams.  (Face it — developers dream about some weird stuff.)  All for the simple reason — the more applications are adapted to the grid, the more everybody needs the grid.

So that’s it — Version 1.3 is about developers, developers, developers.

And it couldn’t come at a better time.  Microsoft announced general availability of Windows Compute Cluster Server 2003 (CCS) last month (earth’s best way to deploy and administer many high-performance Windows servers together), and we’ve got earth’s best way to adapt .NET and COM applications to a grid of Windows machines (running CCS and every other Windows OS since 2000).  It’s never been a better time to develop scalable applications on the Windows platform.  So what are you waiting for — come and get it!

Tags: .NET, ccs, Compute Cluster Server, developer, developers, Digipede, Digipede-Network, distributed-computing, dotnet, grid-computing, Microsoft, scalability, Windows

Digipede Evangelista Kim Greenlee will present at the Bay Area .NET User Group on Thursday, September 28.  The show starts at 6:30, at 1 Market Street (the “Landmark Building”) in San Francisco.  (Yes, that’s Microsoft’s office — go to the second floor, and score some free pizza.)  It’s an easy BART ride from wherever you  are (in the Bay Area), and it’s well worth the trip.  She’s talking about concurrency, a topic near and dear to the hearts of architects and developers everywhere.  Multi-threading, grid computing, clusters, grid objects — if you’re thinking about scalability and concurrency, this talk’s for you.  Details and registration — go here.

Tags: .NET, Bay-Area-.NET-User-Group, concurrency, Digipede, dotnet, grid-computing, grid-objects, Kim-Greenlee, scalability

I spent portions of the last three days at the Visual Studio Industry Partners (VSIP) summit in Redmond.  Thanks and congratulations to the whole VSIP team for a worthwhile and fun event!

VSIP is another of the Microsoft partner programs that can deliver great value if you’re patient and willing to work a bit.  As a continuing public service, I will now distill the information about the program you SHOULD be able to get from the VSIP Web site, but you can’t, because their Web site is almost as bad as the main Microsoft partner program site. 

(Without breaching the fearsome Microsoft NDA, here’s some feedback from a Microsoft manager at yesterday’s morning session, who shall remain anonymous:  “I would rather swim in a vat of squid than use the current VSIP Web site.”  Believe me, I can’t make this stuff up.  But at least THIS team is aware of the problems and is working to fix them; is anyone listenting at partner.microsoft.com?) 

VSIP is for partners that extend the capabilities of Microsoft’s Visual Studio and related development products.  There are various levels of program participation, and you get more benefits by moving up from Affiliate to Alliance to Premier.  There are a variety of ways to “extend the capabilities” of VS, from integrating new programming languages, to providing macros and add-ins, to delivering UI components, to writing libraries with new capabilities.  The Digipede Framework SDK, for example, extends Visual Studio through libraries that enable developers to write grid-enabled applications using the same tools and programming paradigms they already use.  (It’s part of the free Digipede Network Developer Edition — ask for one here today!)  You can read more about VSIP here, or at least that link works as of this moment, but navigation beyond that is at your own risk.

Now it’s certainly true that Digipede fits as closely with other Microsoft products as with Visual Studio.  So why do we spend our precious limited bandwidth with VSIP? 

Because these guys get it.  They listen.  They work hard to provide value for partners.  They understand that Visual Studio wins more market share by having great extensibility and great partners helping to extend it.  They are building a great ecosystem where partners really matter.  They have access to millions of developers on the Microsoft platform, many of whom we’d like to reach, and the VSIP team is both responsive and proactive in helping us reach those developers.  When developers have more choices and better tools, everybody wins. 

And by the way — Note to Allison Watson — THESE GUYS know how to feed partners!  Allison, see if you can recruit Laura Templeton and Amy Bang to help manage the next Worldwide Partner Conference.  (If you’ve missed the reference here, the definitive description of the packs of hungry partners prowling the halls of the last WWPC can be found in Dan Ciruli’s now-classic post, “Allison Watson Owes Me Lunch.”) 

When I say “these guys listen,” that was reflected in the way they approached this entire summit.  At some (ok, many) Microsoft meetings, a typical one-hour session is 59 minutes of PowerPoint and fast talk, concluding with a URL (that might or might not point to a valid address) where you can get more information.  While there was a bit of that in some of the VSIP presentations, mostly there was a fair bit of give-and-take with the attending partners.

My favorite presenter at this event, Luca Bolognese, is apparently well aware of this phenomenon.  He began the last 45-minute section of the day as follows, absolutely deadpan:

“I have four hours of material to present, and I will try not to go over four hours.  My presentation is in three parts, any one of which I may easily screw up, so please hold your questions until the end so I do not have to start over.” 

His presentation was actually excellent, he took questions as he went along, and he finished on time.  I can’t tell you what he’s working on, but he can, and it’s slick.  Check out his blog (although it’s not updated recently), or check out the LINQ Project, and if you have a chance to hear him speak, do it.

Other slick things I can’t talk about are talked about by Microsoft bloggers like Paul Andrew , who is overseeing the wonderful new Windows Workflow Foundation, and Richard Turner, who manages the invaluable Product Formerly Known as Indigo (Windows Communication Foundation).  These are two core pieces of .NET 3.0.  Let me second Robert W. Anderson’s request:  Free .NET 3!  Paul and Andrew — somehow, please find a way to  release .NET 3 independent of (and before) Vista! 

I’ll wrap this up by saying – if your product touches Visual Studio in any way, this is a program worth considering, and a team worth getting to know.  
 

Tags: .NET, .NET-3.0, ISV, Microsoft, partner, Partnering with Microsoft, Visual-Studio, VSIP, WCF, Windows-Workflow-Foundation, Workflow