HTTP Proxy Servers and Proxying

HTTP Proxy Servers and Proxying

(Page 1 of 3)
In my overview of the HTTP operational model,
I described how HTTP was designed to support not just communication between a client and server, but also the inclusion of intermediaries that may sit in the communication path between them. One of the most important types of intermediary is a device called
a proxy server, or more simply, just a proxy.
A proxy is a “middleman” that acts as both a client and a server. It accepts requests from a client as if it were a server, then forwards them (possibly modifying them) to the real server, which sees the proxy
as a client. The server responds back to the proxy, which forwards the reply back to the client. Proxies can be eithertransparent, meaning that they do not modify requests and responses, or
non-transparent, if they do so in order to provide a particular service.

Note:The term “transparent proxy” can also be used to refer to a proxy that is interposed automatically between a client and server—such as an organization-wide firewall—as opposed to one that a user manually configures.

Benefits of Proxies
Since proxies have the ability to fully process all client requests and server responses, they can be extremely useful in a number of circumstances. They can be used to implement or enhance many important capabilities.
Security
Proxies can be set up to examine both outgoing requests and incoming responses, to address various security concerns. For example, filtering can be set up to prevent users from requesting “objectionable” content,
or to screen out harmful replies such as files containing hidden viruses.
Caching
As I mentioned in the previous topic,
it can be advantageous to set up a “shared cache” that is implemented on an intermediary, so resources requested by one client can be made available to another. This can be done within a proxy server.
Performance
In some circumstances, the existence of a proxy server can significantly improve performance, particularly by reducing latency. An excellent example of this is the proxy server that is used by my own satellite
Internet connection.
Due to the distance from the earth to the satellite, it takes over 500 milliseconds for a round trip request/response cycle between my PC and an Internet server. If I load a Web page containing images, I would
have to wait 500+ milliseconds to get the HTML page, at which point my browser would then have to generate new requests for each graphical element, meaning another 500+ millisecond delay for each.
Instead, my ISP has a proxy server to which I send my requests for Web pages. It looks through the HTML of these pages and automatically requests any elements such as graphics for me. It then sends them straight
back to my machine, cutting the time required to display a full Web page drastically.


Key Concept:One of the most important types of intermediary devices in HTTP is a
proxy server, which acts as a middleman between the client and server, handling both requests and responses. A proxy server may either transport messages unchanged or may modify them to implement certain features and capabilities. Proxies are often
used to increase the security and/or performance of Web access.

Comparing Proxies and Caches
Proxying and caching are concepts that have a number of similarities, especially in terms of the impact that they have on basic HTTP operation. Like caching, proxying has become more important in recent years,
and also complicates HTTP in a number of ways. The HTTP/1.1 standard includes a number of specific features to support proxies, and also addresses a number of concerns related to proxying.
The fact that both proxying and caching represent ways in which basic HTTP client/server communication is changed, combined with the ability of proxies to perform caching, sometimes leads people to think caches
and proxies are the same, when they are not. A proxy is actually a separate element that resides in the HTTP request/response chain, where caches can be implemented within any device in that chain, including a proxy.
Another key way that caches and proxies differ is that caches are used “automatically” when they are enabled, where proxies are not. To use a proxy, client software must be told to use the proxy, and supplied
with its IP address or domain name. The client then sends all requests to the proxy rather than to the actual server that the user specifies.
mportant Proxying Issues
As I mentioned above, there are a few different issues that come into play when proxies are used in HTTP. Here are some of the more important ones, and how HTTP deals with them. Again, for much more information
on proxying, please refer to your trusty copy of RFC 2616.
Capability Inconsistencies:
Issues arise when a client and server don’t use the same version of HTTP, or don’t support the same features; for example, some servers may not support all of the methods that a client may try to use. This is
made all the more complex when a proxy enters the picture. Of particular concern is the situation where a client and server may agree on a particular feature that the proxy does not. The proxy must make sure that it passes along headers or other elements that
it may not comprehend.
Authentication Issues
The use of proxy servers often introduces new authentication or security requirements. In addition to authenticating with an end server, the proxy may specify that the client needs to present separate authentication
credentials to it as well. This is done using the HTTP Proxy-Authorization and
Proxy-Authenticate headers; see the topic on HTTP security for details.
Caching Interaction
Not only do both caching and proxying both complicate HTTP, they can complicate each other. Many of
the issues in handling caching, such as header caching, expiration and validation, become
more complex when proxies are involved. Some of the Cache-Control
general header directives are specific to proxying.
Another issue is that the use of proxying and caching together can lead to distortions in the apparent number of times that a Web resource is accessed. This is important in situations where Web pages are supported
by advertising, based on the number of times the page is accessed. Sometimes in this situation, special codes are placed in URLs called “cache busters” are used to force pages not to be stored in shared caches.
Encodings
Content encodings are applied end-to-end and so should
not be affected by proxies. Transfer encoding is done hop-by-hop, so a proxy may use different encodings in handling different transfers of a single request or response.
Tracing Proxy Handling
It is useful in some circumstances, especially when multiple proxies may be in the request/response chain, to be able to trace what proxies have processed a particular message. To this end, HTTP/1.1 requires that
each proxy that handles a message identify itself in the Via header.