Understanding the HTTP Protocol

Pages on the Web are transferred using the HTTP protocol. This protocol specifies how users (or, in some cases, systems) make requests of servers over the World Wide Web, and how these servers respond to these requests. Understanding the basics of this protocol will help you understand how dynamic pages work.

At its core, the HTTP protocol is a transactional system. A client sends a request to a server, and the server sends a response back to the client. Most developers at this point understand the basics of the system. One part of the request is the URL, or Uniform Resource Locator. When you click a link in a browser, a request is sent to the server that contains the desired file.

What most people don't realize is that the client's computer sends a lot more to the server than a URL request. It also sends quite a bit of information about itself, including the browser (referred to as the user agent), username, IP address, file types it can accept (such as GIF and JPEG), and several sources of data. The request contains a header and a body; most of the information just specified is sent in the header. The reason people don't realize this is happening, of course, is that it is not visible to the human user.

Once the server receives the request, it responds if it can. It looks at the requested document, and if that document has any server-side code on it (such as ASP or ColdFusion), the server processes those instructions. When it is finished, it removes the server-side code and combines the output from the server-side code along with the XHTML in the document, and sends it all back to the client in the body of the response. The response, like the request, has a header, which contains information for the client system (such as the document size and type and the date and time of the response). About all a user can see of this transaction are the URL and the output XHTML page.

The following figure represents an HTTP transaction. Solid rectangles show documents visible to the user, while dashed rectangles represent documents hidden from the user.

If you wondered at the end of Lesson 4 exactly how test_form_processor.asp gained access to the data that the user entered in the form on that page, the request is your answer. The firstName variable and its value were sent in the body of the request message when you clicked the Submit button. In other words, the hidden portions of the HTTP request and response messages make it possible to send data values back and forth between the client and server, which makes them available to ASP or ColdFusion scripts on the server. It also means, as you'll see in this lesson, that the server can send data back to the client with directions to store the data in a cookie.

Before we start looking at how to take advantage of the HTTP protocol to facilitate dynamic Web development, there is one more vital behavior you need to know about the protocol: Once the server sends its response to the client, it forgets about the entire transaction. That is, if the client makes a second request of the server, the server has no idea that this is the same client who made a request a moment ago. For this reason, HTTP is known as a stateless protocol.

The statelessness of the HTTP protocol is a significant problem for those developing Web applications. If the Web forgets who you are in between every page, how can you get through the successive screens of an online shopping cart? If you have a multipart survey, how can you ensure that the data entered on page 1 is inserted into the database along with the data entered on page 4? You have seen online shopping carts and probably taken multipage surveys, so you know there are solutions to these problems.

It is hard to develop dynamic Web sites without an awareness of the request-and-response transaction and the statelessness of HTTP. Binding data to pages takes place within this context, and the differences between querystring and form variables, POST and GET, and setting and retrieving cookiesall of which you are about to become familiar withare considerably less obscure when you understand how they relate to the HTTP protocol.