Could not connect to ZooKeeper slave01:2181,slave02:2181,slave03:2181

这是在通过写MapReduce读取HBase的数据，给指定的列加上索引到solr发现的

zookeeper.out日志：

2015-11-20 16:59:20,603 [myid:3] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too many connections from /10.21.3.77 - max is 60

2015-11-20 16:59:20,619 [myid:3] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too many connections from /10.21.3.76 - max is 60

2015-11-20 16:59:20,762 [myid:3] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too many connections from /10.21.3.76 - max is 60

2015-11-20 16:59:20,774 [myid:3] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too many connections from /10.21.3.77 - max is 60

2015-11-20 16:59:20,915 [myid:3] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too many connections from /10.21.3.76 - max is 60

2015-11-20 16:59:20,973 [myid:3] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too many connections from /10.21.3.77 - max is 60

2015-11-20 16:59:21,219 [myid:3] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too many connections from /10.21.3.77 - max is 60

2015-11-20 16:59:22,058 [myid:3] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too many connections from /10.21.3.77 - max is 60

MapReduce执行报错：

15/11/20 16:59:41 INFO mapreduce.Job: Task Id : attempt_1448000031899_0003_m_000012_2, Status : FAILED

Error: org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper slave01:2181,slave02:2181,slave03:2181 within 10000 ms

at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:164)

at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:115)

at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:105)

at org.apache.solr.common.cloud.ZkStateReader.<init>(ZkStateReader.java:212)

at org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:241)

at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:524)

at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:124)

at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)

at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)

at com.yiban.mapreduce.HBaseToSolrIndex$HBaseToSolrIndexMapper.map(HBaseToSolrIndex.java:60)

at com.yiban.mapreduce.HBaseToSolrIndex$HBaseToSolrIndexMapper.map(HBaseToSolrIndex.java:29)

at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)

at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)

at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)

at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper slave01:2181,slave02:2181,slave03:2181 within 10000 ms

at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:223)

at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:156)

... 18 more

查看zookeeper 2181端口的连接数：使用netstat -nap | grep 2181 | wc -l查看

正常连接数

[root@slave01 hbase-1.0.2]# netstat -nap | grep 2181 | wc -l

11

[root@slave02 lib]# netstat -nap | grep 2181 | wc -l

11

[root@slave03 lib]# netstat -nap | grep 2181 | wc -l

15

运行MapReduce连接数，太多不正常

[root@slave01 hbase-1.0.2]# netstat -nap | grep 2181 | wc -l

801

[root@slave01 hbase-1.0.2]# netstat -nap | grep 2181 | wc -l

826

[root@slave02 lib]# netstat -nap | grep 2181 | wc -l

566

[root@slave02 lib]# netstat -nap | grep 2181 | wc -l

641

[root@slave03 lib]# netstat -nap | grep 2181 | wc -l

196

[root@slave03 lib]# netstat -nap | grep 2181 | wc -l

529

[root@slave03 lib]# netstat -nap | grep 2181 | wc -l

563

[root@slave03 lib]# ^C

[root@slave03 lib]# netstat -nap | grep 2181 | wc -l

546

有问题代码：

写在MapReduce里的map方法里面(这里是局部变量，每调用一次map方法就会创建一个zookeeper连接，导致连接数直线上升)

CloudSolrServer solrServer = new CloudSolrServer("slave01:2181,slave02:2181,slave03:2181");//获取zookeeper地址

正常代码：

写在加载时的静态块里面(整个MapReduce只创建一个zookeeper连接)

private static CloudSolrServer solrServer;

static{

solrServer = new CloudSolrServer("slave01:2181,slave02:2181,slave03:2181");//获取zookeeper地址

}

总结：不管是什么连接，能用全局创建尽量用全局创建，这样能避免连接数过多问题