Monday, 28 January 2008

Why we lost WSE and didn't migrate to WCF

I have the dubious privilege of being a Microsoft specialist in a largely open source / Java team. Once in a while on my blog therefore you may see posts which have C# code in them, or otherwise talk about Microsoft technologies. I like C#, it's an elegant language. You can see in it's evolution the Microsoft strategy of watching others, adopting later, and improving. It's not like them to invent something, but very like them to learn from others solutions to problems and try and do something that bit better. Learning from other peoples mistakes can save you a load of hassle in general.

Another good example, as well as C#, is the ASP.Net MVC framework, which a colleague who's been banging on to me about Rails for some time said was actually very good.

(Saying that, we both eventually agreed that server side code should not emit HTML to the browser directly, but that sensible web architectures these days would have scriptaculous, or dojo or somesuch doing all the mark up work, and a web server should just expose resources with data in JSON, YAML, or whatever other flavour of data markup you like; a web application then is Just An API. The ASP.Net MVC framework partially fails on that; it has a lot of convenience features for client side development, and while I understand the rationale for this, particularly for an MSFT toolkit, the purist in me doesn't like it).

I digress.

Where I work we have this bunch of APIs for doing funky stuff with the phone network, sending text messages etc, which we've implemented as Soap Messages, with a bit of WS-Security and PKI for good measure. There's detailed instructions on the protocols we use just here.

For those of you who think WS-* sucks, we listened, we heard, and we are releasing Mojo in all it's REST and Webby goodness. Thank the Lord.

PKI, certificates and WS-Security are hard, no matter what the tools vendors tell you, and most developers struggle a bit with these things unless the client and server are in a very tight relationship (e.g. same version of .Net on both sides). This is the practice our security team wanted us to follow through with though.

All this friction led us to release SDKs to encapsulate all that toughness away from our developer community, and also to reduce our support costs. These SDKs hide away all that stuff to do with building proxies to our services, implementing the security policy, signing messages with certificates etc.
We support the SDKs, but our response if you ask us how to directly invoke the services on the wire is "good luck to you". This has not caused us much of a problem to date, and if you can't use our SDKs for Java, .Net, PHP, or Python, let us know and if you have a strong case we may build one for just your language. Probably not though, we've covered quite a few bases.

The first SDK we produced for our services was ready for Microsoft's TechEd conference in Barcelona in 2006. At that time, we only supported the .Net environment, largely because we chose to launch at that event. We built that first SDK on the WSE3 framework, which handled databinding, cryptography etc, we just put in a custom task or two and tweaked the processing pipeline. We followed a similar approach for the first Java SDK we produced later, based on Xfire.

However when we tackled the PHP and Python SDKs we took a step back and decided, rather than a databinding approach, to go for templating instead. Most of the messages that are exchanged by the SDK are very small, and there aren't that many. The complexity of SOAP message transfer is all about what I call header gunk. And that's why we ship SDKs, to hide that gunk that no developer ever wants to see, and to make sure they write code in a paradigm they are used to; object rather than message oriented. Most of the time this approach is deemed good, and Service SDKs are considered to be a good sign of SOA maturity. FWIW.

We're now aiming to get to the point where all our APIs are generated from a highly controlled model Schema -> WSDL -> Messages -> Documentation -> Object mapping -> SDK generation. So that the team who works on exposure has as little work to do as possible, and follows the DRY principle. And of course so teams who work on the services themselves have encoded guidance on how to expose themselves, as it's very easy to create bad APIs!

There were a couple of pain points around using the WSE stack.

Firstly, having this 11.4MB download required, which we had no control over.

Secondly, the dependency on the Windows Certificate Store. Because WSE wouldn't just pick up certificates from the file system, folk who wanted to run their web applications on shared servers had to beg their sysadmins to add certs to the cert store, so that IIS had access to them. What a pain. And that pain has gone nowhere with WCF; the security credential used by the running application must have appropriate access to the cert store. Microsoft WHY!!!??? Why can't you just allow certs to be picked up and read from the file system like other peoples stacks?

So we've changed things ourselves, and thanks to some sterling and brain-aching work by Piotr and Josh, after an original proof of concept from Robbie, we have removed the dependency on the Windows Certificate store, and also removed additional dependencies for our .Net SDK, other than the .Net 2.0 runtime itself. Though if you want to use the cert store, that option is still available, we still support it.

So now the amount of config you have to handle for our SDK is small and simple:

<configuration>
<
appSettings>
<
add key="Web21c_Environment" value="sandbox"/>
<
add key="certFile" value="yourcert_Sandbox.pfx"/>
<
add key="certPassword" value="certpassword"/>
<
add key="serverCertFile" value="btsdkservercert-acorn.cer"/>
</
appSettings>
<
system.web>
<
webServices>
<
soapExtensionTypes>
<
add type="BT.Sdk.Core.Web21cSoapExtension, BT.Sdk.Core" priority="1" group="high" />
</
soapExtensionTypes>
</
webServices>
</
system.web>
</
configuration>



There. That's better now, isn't it?



if there's anyone from MSFT reading this blog who has an opinion (Shy, Don, as friends of mine you spring to mind), and thinks they could've persuaded us to use the heavy WCF stack rather than the templating and custom pipeline we've developed, I'd love to hear your responses.

4 comments:

Adrian Smith said...

1. Poor old Robbie should get some credit for this!

2. It's a real shame that the <system.web> bit in the config couldn't be hidden away somewhere, as it seems like just some noise that users don't need to see.

Tim Stevens said...

Adrian, you're right! I have edited the post to give credit where it's due. Robbie, sorry I forgot you mate. It's not personal, or if it is, I'm not gonna tell you, that would be mean.

Agree about <system.web> piece, but at least we don't spread config across two files, and have reduced it by about 80%, and removed most of the IIS/Security account/Certificate Permission problem. It's a vast improvement IMO.

Robbie said...

1. =p

2. The system.web part gets picked up, part of the filter pattern applied to how .Net handles SOAP services. All the magic happens behind the scenes and that system.web tag points to the part of the code where you tell it what/how to sign.

It's may be possible to reduce further, but that may require going down to templating the SOAP and using HTTP POST instead of letting the underlaying .Net framework do all that jazz.

Lakshmi said...

Do we get an SDK which supports Fiery Controllers with Dotnet

Tim Stevens

Tim Stevens
Work
Consume
Obey
Be Silent
Die