tollkuci.com

As defined in the “HTTP RFC“, HTTP (Hypertext Transfer Protocol) is “an application-level protocol for distributed, collaborative, hypermedia information systems. It is a generic, stateless, object-oriented protocol which can be used for any tasks, such as name servers and distributed object management systems, through extension of its request methods. A feature of HTTP is the typing and negotiation of data representation, allowing systems to be built independently of the data being transferred.” That’s the long official definition, while the practical most important aspect is that it is a stateless protocol. What this means is that each request/response is totally independent from any other request. Basically we always have a server, a client and the HTTP protocol that transmit the data between them. Since the protocol is stateless if we want to store state we need to pass it with each request/response. In order to understand how this can be done we need to look in more details of how the HTTP protocol works, but to give you an idea, we have three options:

1. Use cookies - That’s how the magic of “session” works in some platforms.
2. Use URL parameters (GET request) - An alternative way to support “session”.
3. Use form hidden fields - That’s how the ASP.NET web forms and ViewState concept work.

Each HTTP request/response is compromised of two parts: the header and the body. While the header is required, the body is optional.They are separated by an empty line.
Now let’s have a more closer look on how a HTTP request/response looks like.

HTTP Request

One of the best ways to understand the HTTP protocol is to use a tool that captures HTTP request/response headers. If you’re using Firefox, there’s an add-on called Live HTTP Headers that does a nice job. A request for this blog results in the following HTTP request header:

GET / HTTP/1.1
Host: blog.tollkuci.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 (.NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,sq;q=0.7,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: __utma=192489849.890738818.1223636002.1223636002.1223636002.1; __utmb=192489849;
X-lori-time-1: 1223637168885
If-Modified-Since: Fri, 10 Oct 2008 11:09:44 GMT

As you can see a HTTP request header is just a list of name/value pairs separated but new line. While the HTTP header has a big list of possible fields, a typical request such as the one above contains the most useful ones and a good understanding of these fields it’s more that enough for most practical reasons. Now let’s analyze each field of the above request in detail:

• The first line of the HTTP header indicates the type of the request (in this case GET), the path of the request (in this case /, meaning the default page) and the version of the protocol to be used (in this case HTTP 1.1).
• The Host field indicates the server when the request should be made.
• User-Agent: Informative fields with details about the user agent (that is the browser) that is making the request. In this case it includes details about browser (Firefox), Windows and .NET. The server receiving the request may make decisions based on this field, for example render different versions for different browsers.
• Accept: Specify the media type that the response should have. In this case it is specified that text/html and application/xhtml+xml are the preferred response types. However if that’s not possible application/xml can be used and if that’s not possible any media type can be returned. The order above is specified by the quality parameter (q). Since text/html and application/xhtml+xml do not have a quality parameter it default to 1.
• Accept-Encoding: Similar to Accept, but it restricts the encoding of the response. In this case it means that the browser can accept gzip and deflate encodings. If the server also supports gzip or deflate encodings it can compress the response before sending it to the browser, thus saving bandwidth.
• Accept-Charset: Again, similar to Accept, but it defines the characters sets that are allowed for the response. In this case “ISO-8859-1″ is the preferred character set, then UTF-8 and then any other character set.
• Keep-Alive and Connection specify underlying TCP connection details which are mostly irrelevant. Basically they are used to specify if the underlying connection should be closed for each request/response or should be alive for a specified interval.
• Cookie: This is one of the most important fields. It include the list of valid cookies that the user agent (browser) has for this domain. Cookies are at the heart of higher-level concepts such as “Session”. The basic idea is that the browser keeps track of valid cookies for a domain and send them with each request. I’ll talk with much more details for cookies in a future article.
• X-lori-time-1: This is a custom header sent by one of the add-ons I use in Firefox. I included it here to show that it is possible to include customer headers in a HTTP request.
• If-Modified-Since: It is used for caching purposes and tells the server to not send the data, but to send a special response (304, Not Modified) if the resource is not modified since the specified date. In this case the user agent (browser) will use the cached copy of the resource.

If the above case, since the request is a GET request there’s no message body. In a POST request, the request body will include the name and values of the fields in the form.

HTTP Response

HTTP response also includes a header and a body. The response header for the above request is as below:

HTTP/1.x 200 OK
Cache-Control: no-cache, must-revalidate, max-age=0
Connection: close
Date: Fri, 10 Oct 2008 11:12:50 GMT
Pragma: no-cache
Content-Type: text/html; charset=UTF-8
Content-Encoding: gzip
Expires: Wed, 11 Jan 1984 05:00:00 GMT
Last-Modified: Fri, 10 Oct 2008 11:12:49 GMT
Vary: Accept-Encoding
Server: WWW Server/1.1
X-Powered-By: ASP.NET, PHP/5.2.0
X-Pingback: http://blog.tollkuci.com/xmlrpc.php
Status: 200 OK

Let’s examine each field in detail:

• The first line include details of the protocol used and the status of the response (200 OK means that the request is fulfilled successfully).
• Cache-Control: Used to define the behavior of caching for the returned resource. In this case, no-cache means that the resource should not be cached anywhere, that the client must re-validate the request and that max-age of caching is 0. End result of all this is that the page should not be cached. There are several other options that can be used to fine tune caching. We’ll have the chance to talk more of caching in a future article.
• Connection: As explained for the Request header specify details for the underlying TCP connection.
• Date: Date and time when the request was processed.
• Pragma: Generic field that can be used for specific implementations. In this case it specifies that the page should not be cached.
• Content-Type: Specified the media type of the response body.
• Content-Encoding: Encoding of the response body. Since the browser supported gzip encoding and the server can generate gzip response it has compressed the response using gzip.
• Expires: Date and time when the resource should expire from the cache. In this case a past date is used to indicate that the page should not be cached.
• Last-Modified: Date and time when the request was last modified.
• Vary: Specify the mechanism that was used to generate the response. In this case it means that Accept-Encoding was used to generate the response as gzip.
• Server: Details about the server that is sending the response.
• X-Powered-By: Customer header used the return details about the server that is sending the response.
• X-Pingback: Again customer header used to specify the pingback url.
• Status: Must important field that includes the response status code and name. 200 OK means that the request was fulfilled successfully. Other common response statuses are:
• 301 Moved Permanently - indicates that the resource was moved permanently to a new URL.
• 302 Moved Temporarily - indicates that the resources was moved temporarily to a new URL.
• 401 Unauthorized - indicates that the request requires user authentication.
• 403 Forbidden - indicates that the credentials specified with the request are not valid and the request is forbidden.
• 404 Not Found - the most well know status. Indicates that the specified resource cannot be found.
• 500 Internal Server Error - the server is unable to fulfill the request.

For a list of all header fields and response codes, you can always refer to the HTTP RFC.

Conclusions

This was just an overview of the basics of the HTTP protocol. There are lots of resource on the web that go down on details for each of the above mentioned topics, by my idea was to give you only the most important concepts. The most important thing to remember from this article is the stateless nature of HTTP and that each request/response is independent from any other. Since in most application it is necessary to store the state somehow, mechanism like url parameters, cookies or hidden form fields are used to maintain this state. We’ll look at them in more details in this series. As a final note, use a tool (such as the Live HTTP Headers add-on) to experiment with HTTP headers. Also don’t be afraid to “play” with the HTTP protocol by just using telnet to connect with any host (telnet tollkuci.com 80) and to start typing the request header manually.