coffee, black   no sugar


20060627 Tuesday June 27, 2006
HTTP in .Net v2.0 - experiences

When I was young, every programmer was supposed to do three things in life: plant a tree, father a child and write a text editor. Not so any more! Real Programmers of today implement HTTP stacks.

Well, I wrote a HTTP client in Java for a customer of mine some time ago. Since then HTTP client APIs have become a little hobby of mine. And when I delved into .NET and C# on some other project, I was curious to see how the company that invented XmlHttpRequest has done it. If you have to use that piece of code, my experiences so far might be a good read for you - there are some pits to fall into.

Please be aware that this is not a thorough review, just a collection of experiences. I probably have missed some nice features or overseen some things. You are more than welcome to tell me about them.

Creating/Sending a Request

The object model of the .Net HTTP stack works with Request and Response objects. The request gives the repsonse. And the request materializes out of thin air:

  WebRequest request = WebRequest.Create(url);
WebRequest? Wasn't this about HTTP? Well there is a HttpWebRequest as subclass of WebRequest - this abstraction thingie, right? Turns out the cousins of HTTP are FTP and File. I have not looked at what FileRequest really does.

Back to HTTP. Since WebRequest is too abstract for most purposes, I need the real thing. Turns out, once cannot create HttpWebRequests - one has to create the former and hope for the best. So our code looks like this now:

  HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
Does someone else than yours truly thinks this is unnecessarily ugly?

Ok, lets get our first response:

  request.Method = "GET";
  HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Urghs, casting again. Now lets look at the HTTP response status:
  if (res.StatusCode == 200) {
    ...
This does not compile. StatusCode returns a HttpStatusCode which is an enum! Needless to mention that 207, 423 or 507 have no assigned constants. Interestingly enough, when the response is 207 you get a HttpStatusCode instance - but it is not among the declared ones...but let's not get distracted too much. Here is how one checks the response:
  if ((int)res.StatusCode == 200) {
    ...
Ok, simply another cast. I sometimes wonder what I have overlooked.

WTF?

Were the words when I made the first request to a URL which did not respond with a 200. Instead of a returned response I got an Exception! Yikes! A WebException! Yes, it is exceptional that one gets a 404. At least in the experience of the guy who designed this API - what a lucky man.

Since I am usually not so lucky, I have to wear some protection. My code now looks like this:

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
HttpWebResponse response = null;
try {
  request.Method = "GET";
  response = (HttpWebResponse)request.GetResponse();
}
catch (WebExcpetion ex) {
  response = (HttpWebResponse)ex.Response;
  if (response == null) {
     //seems like something really went wrong
     throw ex;
  }
}
// here we have a response...
switch ((int)response.StatusCode) {
  ...

Be Gone, Thou Response!

One nastiness about HTTP responses and languages with automatic memory management is that the library code is uncertain of when to discard a possible response body. The garbage collector might run next in an hour for all we know, so the destruction of the response object is too late. .Net works around this by requiring you to call response.Close(). And you better do this or no connection will be reused. The code, ladies and gentlemen:

  HttpWebResponse response = null;
  try {
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
    try {
      ...
    }
    catch (WebExcpetion ex) {
      ...
    }
    ...
  }
  finally {
    if (response != null) {
      response.Close();
    }
  }
This is a workable solution. In my own stack, the API user had to announce that he is interested in a response body. Otherwise the stack cleaned up the connection immediately. I still prefer that, but this is really a matter of taste.

\\r as in required

The .Net implementation has been RFC 2616 compliant in the - admittedly shallow - use cases of mine. A little too strict perhaps. I would wish to at least configure a bit more relaxed error checking. For example the lib throws an exception when response header lines are delimited by linefeed only. And right enough, the spec says that header lines must be seperated by carriage return + linefeed pairs. So the lib's behavior is Ok.

On the other hand, it would be nice to be able to talk to such a server. Even RFC 2616 recommends to clients to accept since linefeeds as well (RFC 2616 Appendix C). Difficult things should be possible.

Wrapping it Up

The .Net HTTP stack works ok. It's design is far form elegant. When an API requires 3 casts in the 99% of all use cases, something is wrong. That non-200 responses throw exceptions is ridiculous. It's tight adherence to the letters of 2616 is a bit worrying to me. Most products lack control over the servers they operate against, so I feel a bit uncomfortable with it.

Open Ends

It would be interesting to test the stack with Mark's cacheability and XHR tests in mind and look a bit more under the hood.

Also the cookie/session management is not covered here. That request objects materialize out of thin air means that there is some automagic managment behind the curtains...

Technorati Tags: , , ,