httpclient超时总结

Httpclient超时

背景：
网站这边多次因为httpclient调用超时时间没设置好导致关掉，影响非常不好，而且问题重复出现，查看网络，没有比较明确介绍httpclient所有超时相关的设置（大部分只提到连接超时（connectintimeout）,读超时（sockettimeout），对连接池超时提到的比较少），因此本文对超时这块最介绍，其他功能性内容，apache官方提供了比较详细的解析，这里不做讨论。具体可见：http://hc.apache.org/httpcomponents-client-ga/tutorial/html/index.html
网站这边用的是:
<dependency>
<groupId>commons-httpclient</groupId>
<artifactId>commons-httpclient</artifactId>
<version>3.1</version>
</dependency>

配置：
privatefinalstaticMultiThreadedHttpConnectionManagermanager=newMultiThreadedHttpConnectionManager();
privatefinalstaticHttpClienthttpclient=newHttpClient(manager);
httpclient.setConnectionTimeout(1000);
httpclient.setTimeout(1000);

现象：
页面无法打开，堆栈信息如下：
Name: trhead-142
State: WAITING onorg.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ConnectionPool@69a4cb
Total blocked: 0 Total waited: 1

Stack trace:
java.lang.Object.wait(Native Method)
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.doGetConnection(MultiThreadedHttpConnectionManager.java:518)
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.getConnectionWithTimeout(MultiThreadedHttpConnectionManager.java:416)
org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:153)
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
com.madding.test.MyRunnable.run(MyTest1.java:53)
java.lang.Thread.run(Thread.java:619)

大部分线程等待在：org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.doGetConnection

分析：
问题：很明显连接池超时没设置，导致请求在线程池中等待，进而引起dubb无法处理其他请求。
解决：加httpclient.setHttpConnectionFactoryTimeout(1000);，设置连接池超时。

说明：废弃方法尽量不要用

进一步分析：
为什么没设置这个会导致请求等待：

MultiThreadedHttpConnectionManager代码在获取连接时去连接池取，而连接池在没设置超时timeToWait为0，即一直处于等待状态，如果没有notify不会结束等待。
试用
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.2</version>
</dependency>

发现代码中即使不设置线程池超时，在高并发下也能正常访问，查看代码，发现是因为在默认没设置线程池超时时把连接超时的时间作为线程超时时间，代码如下：

而且httpclient4的代码结构相对3做了很大重构，且实现比较优雅，可以考虑在本地私服把4.x版本添加进来。

完整的超时可参考如下：
staticPoolingClientConnectionManagerconnectionManager=null;
staticHttpClienthttpclient
=null;

static{
connectionManager=newPoolingClientConnectionManager();
connectionManager.setMaxTotal(1);
httpclient=newDefaultHttpClient(connectionManager);
httpclient.getParams().setParameter("http.socket.timeout",1000);
httpclient.getParams().setParameter("http.connection.timeout",1000);
httpclient.getParams().setParameter("http.connection-manager.timeout",100000000L);
}

具体测试代码如下：
3.1:
packagecom.madding.test;

importorg.apache.commons.httpclient.DefaultMethodRetryHandler;
importorg.apache.commons.httpclient.HttpClient;
importorg.apache.commons.httpclient.HttpStatus;
importorg.apache.commons.httpclient.MultiThreadedHttpConnectionManager;
importorg.apache.commons.httpclient.methods.GetMethod;

/**
*@author madding.lip
*/
publicclassMyHttpClientTest3_1
{

privatefinalstaticMultiThreadedHttpConnectionManagermanager=newMultiThreadedHttpConnectionManager();
privatefinalstaticHttpClienthttpclient=newHttpClient(manager);

publicstaticvoidmain(String[]
args) {
httpclient.getParams().setParameter("http.socket.timeout",1000);
httpclient.getParams().setParameter("http.connection.timeout",1000);
httpclient.getParams().setParameter("http.connection-manager.timeout",60*60L);
// httpclient.setConnectionTimeout(1000);
// httpclient.setTimeout(1000);
// httpclient.setHttpConnectionFactoryTimeout(1000);

for(inti
= 0; true;i++) {
newThread(newMyRunnable(httpclient),"trhead-"+
i).start();
try{
Thread.sleep(100);
}catch(InterruptedException e) {
}
}
}

}

classMyRunnableimplementsRunnable
{

HttpClienthttpclient=null;

MyRunnable(HttpClient client){
httpclient= client;
}

publicvoidrun()
{
GetMethod getMethod =newGetMethod("http://www.apache.org/");

try{
DefaultMethodRetryHandlerretryHandler
= newDefaultMethodRetryHandler();
retryHandler.setRetryCount(0);
getMethod.setMethodRetryHandler(retryHandler);

intstatusCode =httpclient.executeMethod(getMethod);
if(statusCode == HttpStatus.SC_OK){
getMethod.getResponseBodyAsString();
}
}catch(Exception e) {
System.err.println(e);
}finally{
getMethod.releaseConnection();
}
}
}

4.2:
packagecom.madding.test;

importjava.io.IOException;

importorg.apache.http.HttpEntity;
importorg.apache.http.HttpResponse;
importorg.apache.http.client.ClientProtocolException;
importorg.apache.http.client.HttpClient;
importorg.apache.http.client.methods.HttpGet;
importorg.apache.http.client.methods.HttpUriRequest;
importorg.apache.http.impl.client.DefaultHttpClient;
importorg.apache.http.impl.conn.PoolingClientConnectionManager;
importorg.apache.http.util.EntityUtils;

/**
*@author madding.lip
*/
publicclassMyHttpClientTest4_2
{
staticPoolingClientConnectionManagerconnectionManager=null;
staticHttpClienthttpclient
=null;

static{
connectionManager=newPoolingClientConnectionManager();
connectionManager.setMaxTotal(1);
httpclient=newDefaultHttpClient(connectionManager);
httpclient.getParams().setParameter("http.socket.timeout",1000);
httpclient.getParams().setParameter("http.connection.timeout",1000);
httpclient.getParams().setParameter("http.connection-manager.timeout",100000000L);
}

publicstaticvoidmain(String[]
args) {

for(inti
= 0; true;i++) {
newThread(newMyTest(httpclient),"trhead-"+
i).start();
try{
Thread.sleep(100);
}catch(InterruptedException e) {
}
}
}
}

classMyTestimplementsRunnable
{
staticHttpClienthttpclient
=null;

publicMyTest(HttpClient hc){
httpclient= hc;
}

publicvoidrun()
{
HttpUriRequest httpget =newHttpGet("http://www.apache.org/");
try{
HttpResponse response =httpclient.execute(httpget);

HttpEntity entity =response.getEntity();

System.out.println("----------------------------------------");
System.out.println(response.getStatusLine());
if(entity !=null){
System.out.println("Responsecontent
length: " +entity.getContentLength());
EntityUtils.toString(entity);
//System.out.println(EntityUtils.toString(entity));
}
System.out.println("----------------------------------------");

}catch(ClientProtocolException e) {
System.err.println(e);
}catch(IOException e) {
System.err.println(e);
}finally{
if(httpget !=null){
httpget.abort();
}
}
}

}