Late last year when we were developing FireAnt after some time of thinking through the plumbing, the programming model and the development paradigm we came to an interesting dilemma: what would the over the wire format be? Our basic goal was to send only data (or as close to only data as possible) over the wire to the end user and make the development model easy (e.g. no hand written spaghetti code). While that sounds simple enough, the requirement leads to a bunch of possibilities on how to serialize the data for transport via XmlHttp.
And we knew from lots of performance work we’d done over the course of 2004 what kind of connections, speeds, networks, and bandwidth were available to our customers all over the world. We’d done field tests in China, India, and other parts of Asia and Europe, so we had a pretty good idea of how bad internet connections out there are (1s+ latency, 25%+ packet loss and worse). We also knew that our current (called Wave 10) product didn’t fare too well in those scenarios due to the raw amount of data we were sending across the wire so a goal of Kahuna and FireAnt was to reduce it.
The first, and most obvious, was to send the data in XML. It’s called XmlHttp after all, right? The second was to just use SOAP. ASP.NET makes SOAP services easy to create. The third and last was to create our own over the wire protocol. If you look under the hood, you’ll see we’ve done the latter and I’ll try to explain why here.
Before getting in to a discussion of each format, one distinction that I want to make is the difference in the upstream (client to server) and downstream (server to client) formats. They both have very different requirements. You have a (fairly) limited programming model on the client that’s cooperatively multithreaded, so doing a lot of processing on it can get expensive. You also have a sophisticated server with a sophisticated programming language and (for all intents and purposes) unlimited processing resources.
All of the above possible on the wire formats will work perfectly well, but in the context of the goals that we had, building out own worked the best.
As you can see if we had chosen XML and SOAP we would have had to do a lot of work to make it work seamlessly with our development model. Further with SOAP, it’s a pretty heavy protocol when dealing with lightweight data (e.g. a GUID, or a command acknowledgement). In building FPP, we basically decided to split the representation of data on the way up versus the way down to hit the goal of reducing the amount of data over the wire. We then generate all the glue that held the client and the server together to make the development model straightforward.