Apache HttpClient (理解Http Client)

摘要: 对Apache HttpClient 项目做一个基本的介绍,基于版本4.5.x

The Apache HttpComponents™ project is responsible for creating and maintaining a toolset of low level Java components focused on HTTP and associated protocols. Include HttpClient, HttpCore, HttpAsyncClient three sub projects.

HttpClient

HTTP 协议（1.0 ，1.1）的实现，更多的是面向开发者使用的API，用于发起Http请求。请求的执行（excute()），拦截（interceptor），请求路由（route），请求代理（proxy），请求认证（发起Https验证，Oauth认证等）, 请求结果的处理（ResponseHandler）, 异常处理等基本操作封装和配置，同时允许使用者高度配置。

基本包含以下内容：

怎么发起http 和 https 请求

怎么通过代理发起请求

怎么接收返回值

怎么配置连接池，连接和执行线程的关系

怎么设置cookie，和一些认证信息

怎么配置长连接

设置超时，以及为什么要设置超时

Tips: HttpClient实例是可以线程共享的。

就以Apache HttpClient 提供的config example 来看一下。

/**
* This example demonstrates how to customize and configure the most common aspects
* of HTTP request execution and connection management.
*/
public class ClientConfiguration {

public final static void main(String[] args) throws Exception {

// Use custom message parser / writer to customize the way HTTP
// messages are parsed from and written out to the data stream.
HttpMessageParserFactory<HttpResponse> responseParserFactory = new DefaultHttpResponseParserFactory() {

@Override
public HttpMessageParser<HttpResponse> create(
SessionInputBuffer buffer, MessageConstraints constraints) {
LineParser lineParser = new BasicLineParser() {

@Override
public Header parseHeader(final CharArrayBuffer buffer) {
try {
return super.parseHeader(buffer);
} catch (ParseException ex) {
return new BasicHeader(buffer.toString(), null);
}
}

};
return new DefaultHttpResponseParser(
buffer, lineParser, DefaultHttpResponseFactory.INSTANCE, constraints) {

@Override
protected boolean reject(final CharArrayBuffer line, int count) {
// try to ignore all garbage preceding a status line infinitely
return false;
}

};
}

};
HttpMessageWriterFactory<HttpRequest> requestWriterFactory = new DefaultHttpRequestWriterFactory();

// Use a custom connection factory to customize the process of
// initialization of outgoing HTTP connections. Beside standard connection
// configuration parameters HTTP connection factory can define message
// parser / writer routines to be employed by individual connections.
HttpConnectionFactory<HttpRoute, ManagedHttpClientConnection> connFactory = new ManagedHttpClientConnectionFactory(
requestWriterFactory, responseParserFactory);

// Client HTTP connection objects when fully initialized can be bound to
// an arbitrary network socket. The process of network socket initialization,
// its connection to a remote address and binding to a local one is controlled
// by a connection socket factory.

// SSL context for secure connections can be created either based on
// system or application specific properties.
SSLContext sslcontext = SSLContexts.createSystemDefault();

// Create a registry of custom connection socket factories for supported
// protocol schemes.
Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create()
.register("http", PlainConnectionSocketFactory.INSTANCE)
.register("https", new SSLConnectionSocketFactory(sslcontext))
.build();

// Use custom DNS resolver to override the system DNS resolution.
DnsResolver dnsResolver = new SystemDefaultDnsResolver() {

@Override
public InetAddress[] resolve(final String host) throws UnknownHostException {
if (host.equalsIgnoreCase("myhost")) {
return new InetAddress[] { InetAddress.getByAddress(new byte[] {127, 0, 0, 1}) };
} else {
return super.resolve(host);
}
}

};

// Create a connection manager with custom configuration.
PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager(
socketFactoryRegistry, connFactory, dnsResolver);

// Create socket configuration
SocketConfig socketConfig = SocketConfig.custom()
.setTcpNoDelay(true)
.build();
// Configure the connection manager to use socket configuration either
// by default or for a specific host.
connManager.setDefaultSocketConfig(socketConfig);
connManager.setSocketConfig(new HttpHost("somehost", 80), socketConfig);
// Validate connections after 1 sec of inactivity
connManager.setValidateAfterInactivity(1000);

// Create message constraints
MessageConstraints messageConstraints = MessageConstraints.custom()
.setMaxHeaderCount(200)
.setMaxLineLength(2000)
.build();
// Create connection configuration
ConnectionConfig connectionConfig = ConnectionConfig.custom()
.setMalformedInputAction(CodingErrorAction.IGNORE)
.setUnmappableInputAction(CodingErrorAction.IGNORE)
.setCharset(Consts.UTF_8)
.setMessageConstraints(messageConstraints)
.build();
// Configure the connection manager to use connection configuration either
// by default or for a specific host.
connManager.setDefaultConnectionConfig(connectionConfig);
connManager.setConnectionConfig(new HttpHost("somehost", 80), ConnectionConfig.DEFAULT);

// Configure total max or per route limits for persistent connections
// that can be kept in the pool or leased by the connection manager.
connManager.setMaxTotal(100);
connManager.setDefaultMaxPerRoute(10);
connManager.setMaxPerRoute(new HttpRoute(new HttpHost("somehost", 80)), 20);

// Use custom cookie store if necessary.
CookieStore cookieStore = new BasicCookieStore();
// Use custom credentials provider if necessary.
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
// Create global request configuration
RequestConfig defaultRequestConfig = RequestConfig.custom()
.setCookieSpec(CookieSpecs.DEFAULT)
.setExpectContinueEnabled(true)
.setTargetPreferredAuthSchemes(Arrays.asList(AuthSchemes.NTLM, AuthSchemes.DIGEST))
.setProxyPreferredAuthSchemes(Arrays.asList(AuthSchemes.BASIC))
.build();

// Create an HttpClient with the given custom dependencies and configuration.
CloseableHttpClient httpclient = HttpClients.custom()
.setConnectionManager(connManager)
.setDefaultCookieStore(cookieStore)
.setDefaultCredentialsProvider(credentialsProvider)
.setProxy(new HttpHost("myproxy", 8080))
.setDefaultRequestConfig(defaultRequestConfig)
.build();

try {
HttpGet httpget = new HttpGet("http://httpbin.org/get");
// Request configuration can be overridden at the request level.
// They will take precedence over the one set at the client level.
RequestConfig requestConfig = RequestConfig.copy(defaultRequestConfig)
.setSocketTimeout(5000)
.setConnectTimeout(5000)
.setConnectionRequestTimeout(5000)
.setProxy(new HttpHost("myotherproxy", 8080))
.build();
httpget.setConfig(requestConfig);

// Execution context can be customized locally.
HttpClientContext context = HttpClientContext.create();
// Contextual attributes set the local context level will take
// precedence over those set at the client level.
context.setCookieStore(cookieStore);
context.setCredentialsProvider(credentialsProvider);

System.out.println("executing request " + httpget.getURI());
CloseableHttpResponse response = httpclient.execute(httpget, context);
try {
System.out.println("----------------------------------------");
System.out.println(response.getStatusLine());
System.out.println(EntityUtils.toString(response.getEntity()));
System.out.println("----------------------------------------");

// Once the request has been executed the local context can
// be used to examine updated state and various objects affected
// by the request execution.

// Last executed request
context.getRequest();
// Execution route
context.getHttpRoute();
// Target auth state
context.getTargetAuthState();
// Proxy auth state
context.getTargetAuthState();
// Cookie origin
context.getCookieOrigin();
// Cookie spec used
context.getCookieSpec();
// User security token
context.getUserToken();

} finally {
response.close();
}
} finally {
httpclient.close();
}
}

}

HttpConnection, Socket and Thread

HttpConnection, Http连接是复杂的，有状态的，非线程安全的对象，它需要恰当的管理以便正确地执行功
能。 “HTTP 连接”一次仅只能由一个执行线程来使用。 HttpClient 采用一个特殊实体来
管理访问 HTTP 连接，这被称为 HTTP 连接管理器，ClientConnectionManager接口就是代表。HTTP 连接管理器的目的是作为工厂创建新的 HTTP 连接，管理持久连接的生命周期和同步访问持久连接（确保同一时间仅有一个线程可以访问一个连接）。内部的 HTTP 连接管理器使用 OperatedClientConnection 实例，这个实例为真正的连接扮演了一个代理，来管理连接状态和控制I/O操作的执行。如果一个管理的连接被释放或者被使用者明确地关闭，潜在的连接就会从它的代理分离，退回到管理器中。
尽管“服务消费者”仍然保存一个代理实例的引用。它也不能执行任何I/O操作、有意或无意地改变真实连接的状态。

first : HttpConnection 由ClientConnectionManager 管理，确保同一时间仅有一个线程可以访问一个连接，

Socket(java.net.Socket), This class implements client sockets (also called just "sockets"). A socket is an endpoint for communication between two machines.
The actual work of the socket is performed by an instance of the SocketImpl class. An application, by changing the socket factory that creates the socket implementation, can configure itself to create sockets appropriate to the local firewall.

second : 然后通过SocketFactory创建Socket来建立连接，Thread block until response return or timeout.

HttpClientContext clientContext = HttpClientContext.create();
PlainConnectionSocketFactory sf = PlainConnectionSocketFactory.getSocketFactory();
Socket socket = sf.createSocket(clientContext);
int timeout = 1000; //ms
HttpHost target = new HttpHost("localhost");
InetSocketAddress remoteAddress = new InetSocketAddress(
InetAddress.getByAddress(new byte[] {127,0,0,1}), 80);
sf.connectSocket(timeout, socket, target, remoteAddress, null, clientContext);

###Conclusion
关于HttpClient 的详细内容你应该读这篇文章 Http Client，这里介绍了Spring RestTemplate 中使用HttpClient,以及为什么很多人喜欢使用HttpClient (因为它的高度可配置化)。

###Reference

HttpClient4.5中文翻译

Apache HttpClient Example