Mycat源码篇 : MyCat线程模型分析

参考MyCat权威指南，对MyCat-Server里面的线程模型做简要分析:

1. 线程模型图

根据MyCat权威指南，做出以下线程模型图:



MyCat的线程模型主要分为三部分（: 网络通讯线程、业务线程和定时任务线程，下面分别介绍这些线程的使用:

【温馨提示】

这里排除JVM自身使用的线程，只关注MyCat服务所使用的线程，如果需要详细了解MyCat里面使用的所有线程，请参考《MyCat权威指南》-> 开发篇 -> MyCat线程模型分析

2. 网络通讯线程

MyCat-Server使用了原生Java NIO和AIO来构建其最重要的网络通讯层，在MyCat-Server启动之前可以通过配置文件来选择使用NIO网络模型或者AIO网络模型。MyCat官方推荐使用NIO模型。所以，这里仅针对NIO网络通讯模型来分析，网络通讯层线程的使用体现在NIOAcceptor、NIOConnector和NIOReactor。

2.1 NIOAcceptor

NIOAcceptor线程负责处理客户端连接MyCat的事件（在NIO网络编程中，对应于NIO服务端），包括默认的8066服务端口（Server）和9066管理端口（Manager）。

NIOAcceptor继承Thread类，在run方法里面处理客户端连接MyCat的连接请求，代码如下所示 :

@Override
public void run() {
final Selector tSelector = this.selector;
for (;;) {
++acceptCount;
try {
tSelector.select(1000L);
Set<SelectionKey> keys = tSelector.selectedKeys();
try {
for (SelectionKey key : keys) {
if (key.isValid() && key.isAcceptable()) {
accept();
} else {
key.cancel();
}
}
} finally {
keys.clear();
}
} catch (Exception e) {
LOGGER.warn(getName(), e);
}
}
}

对于来自客户端的连接，将被Selector事件选择器捕获，对于成功的连接，将调用accept方法做进一步处理，下面是accept方法的代码 :

private void accept() {
SocketChannel channel = null;
try {
channel = serverChannel.accept();
channel.configureBlocking(false);
FrontendConnection c = factory.make(channel);
c.setAccepted(true);
c.setId(ID_GENERATOR.getId());
NIOProcessor processor = (NIOProcessor) MycatServer.getInstance()
.nextProcessor();
c.setProcessor(processor);

NIOReactor reactor = reactorPool.getNextReactor();
reactor.postRegister(c);

} catch (Exception e) {
LOGGER.warn(getName(), e);
closeChannel(channel);
}
}

accept方法将客户端连接封装成FrontendConnection对象，分配相关资源，然后将客户端连接注册到NIOReactor上。

【温馨提示】

这里注意到，NIOProcessor（持有处理业务逻辑的线程池）和NIOReactor（负责处理NIO网络通讯的读写事件）都进行了池化，目的是让对象得以共享这些稀缺的资源。

2.2 NIOConnector

NIOConnector线程负责处理MyCat连接后端MySQL实例的事件（在NIO网络编程中，对应于NIO Client）。与NIOAcceptor类似，该类继承Thread类，在run方法里面处理连接事件，代码如下所示 :

@Override
public void run() {
final Selector tSelector = this.selector;
for (;;) {
++connectCount;
try {
//查看有无连接就绪
tSelector.select(1000L);
connect(tSelector);
Set<SelectionKey> keys = tSelector.selectedKeys();
try {
for (SelectionKey key : keys) {
Object att = key.attachment();
if (att != null && key.isValid() && key.isConnectable()) {
finishConnect(key, att);
} else {
key.cancel();
}
}
} finally {
keys.clear();
}
} catch (Exception e) {
LOGGER.warn(name, e);
}
}
}

在connect方法里面会判断连接队列（元素入队列体现在postConnect方法方法里）是否有连接需要进行处理，如果队列有值，那么取得连接对象主动去跟后端的MySQL实例建立连接。代码如下所示 :

private void connect(Selector selector) {
AbstractConnection c = null;
while ((c = connectQueue.poll()) != null) {
try {
SocketChannel channel = (SocketChannel) c.getChannel();
//注册OP_CONNECT监听与后端连接是否真正建立
channel.register(selector, SelectionKey.OP_CONNECT, c);
//主动连接
channel.connect(new InetSocketAddress(c.host, c.port));
} catch (Exception e) {
c.close(e.toString());
}
}
}

连接建立成功后调用finishConnect方法注册对应的读写事件，然后与NIOAcceptor一样，将读写事件的处理交由NIOReactor处理。代码如下所示 :

private void finishConnect(SelectionKey key, Object att) {
BackendAIOConnection c = (BackendAIOConnection) att;
try {
if (finishConnect(c, (SocketChannel) c.channel)) { //做原生NIO连接是否完成的判断和操作
clearSelectionKey(key);
c.setId(ID_GENERATOR.getId());
//绑定特定的NIOProcessor以作idle清理
NIOProcessor processor = MycatServer.getInstance()
.nextProcessor();
c.setProcessor(processor);
//与特定NIOReactor绑定监听读写
NIOReactor reactor = reactorPool.getNextReactor();
reactor.postRegister(c);
}
} catch (Exception e) {
//如有异常，将key清空
clearSelectionKey(key);
c.close(e.toString());
c.onConnectFailed(e);
}
}

2.3 NIOReactor

一般的NIO网络编程示例，在Server accept客户端连接之后，会直接在该线程里注册读写事件，最后再分发给逻辑处理线程池去处理逻辑，示例代码如下所示 :

for (;;) {
try {
tSelector.select();
Set<SelectionKey> keys = tSelector.selectedKeys();
try {
for (SelectionKey key : keys) {
if (key.isValid() && key.isAcceptable()) {
accept(); // accept register OP_READ and OP_WRITE
} else if(key.isReadable) {
read(); // read data and do business using thread pool
} else if(key.isWritable) {
write(); // write data using thread pool
}
}
} finally {
keys.clear();
}
} catch (Exception e) {
//
}
}

在高性能IO框架实现中，大多采用Reactor模型，将处理连接和处理读写事件分离，模型图如下所示 :



MyCat中采用了reactor模式，将读写事件的注册和接收分离到另外的线程去处理。这个另外的线程指的就是NIOReactor，并且，为了优化并发下频繁进行读写，NIOReactor进行了池化（NIOReactorPool）。

3. 业务线程

BusinessExecutor采用Java线程池来实现，线程池大小默认计算方式如下所示：

this.processorExecutor = (DEFAULT_PROCESSORS != 1) ? DEFAULT_PROCESSORS * 2 : 4;

其中DEFAULT_PROCESSORS为系统CPU可用核心数。单核心的线程池大小为4，非单核心线程池大小为系统CPU可用核心数的2倍。

线程池的任务队列使用LinkedTransferQueue。

业务线程主要用在以下方面：

负责对读取到的网络字节数据做进一步处理

创建后台mysql连接

多分片节点结果集包返回汇聚后处理

4. 定时任务线程

TimerExecutor 定时任务线程池，默认线程大小为2。TimerExecutor结合Timer类进行定时任务调度，主要功能为：

定时检查前端连接和后端空闲连接，回收不可用的连接。

定期发送心跳包以及心跳检测

【温馨提示】 定时逻辑代码入口在MyCatServer.java类里面

5. jstack dump线程

我们可以使用jdk自带工具jstack来查看MyCat-Server线程的使用情况:

首先使用jps查看MyCat-Server的进程号:



然后使用jstack [mycat-server-pid]打印出线程使用情况:

C:\WINDOWS\system32>jstack 8648
2016-08-11 22:34:47
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.66-b18 mixed mode):

"Timer1" #31 daemon prio=5 os_prio=0 tid=0x000000001a75a800 nid=0x21dc waiting on condition [0x000000001e0df000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000000d7c00730> (a java.util.concurrent.LinkedTransferQueue)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

"Timer0" #30 daemon prio=5 os_prio=0 tid=0x000000001a759000 nid=0x1ce0 waiting on condition [0x000000001dfde000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000000d7c00730> (a java.util.concurrent.LinkedTransferQueue)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

"BusinessExecutor7" #29 daemon prio=5 os_prio=0 tid=0x000000001a759800 nid=0x13b4 waiting on condition [0x000000001dbde000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000000d7c18178> (a java.util.concurrent.LinkedTransferQueue)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

"BusinessExecutor6" #28 daemon prio=5 os_prio=0 tid=0x000000001a75b000 nid=0x1090 waiting on condition [0x000000001dade000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000000d7c18178> (a java.util.concurrent.LinkedTransferQueue)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

"BusinessExecutor5" #27 daemon prio=5 os_prio=0 tid=0x000000001a753800 nid=0xf08 waiting on condition [0x000000001d9de000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000000d7c18178> (a java.util.concurrent.LinkedTransferQueue)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

"BusinessExecutor4" #26 daemon prio=5 os_prio=0 tid=0x000000001a758000 nid=0x14a0 waiting on condition [0x000000001d8df000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000000d7c18178> (a java.util.concurrent.LinkedTransferQueue)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

"BusinessExecutor3" #25 daemon prio=5 os_prio=0 tid=0x000000001a756800 nid=0x21c4 waiting on condition [0x000000001d7de000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000000d7c18178> (a java.util.concurrent.LinkedTransferQueue)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

"BusinessExecutor2" #24 daemon prio=5 os_prio=0 tid=0x000000001a757800 nid=0x21c0 waiting on condition [0x000000001d6de000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000000d7c18178> (a java.util.concurrent.LinkedTransferQueue)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

"BusinessExecutor1" #23 daemon prio=5 os_prio=0 tid=0x000000001a755000 nid=0x21b4 waiting on condition [0x000000001d5de000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000000d7c18178> (a java.util.concurrent.LinkedTransferQueue)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

"BusinessExecutor0" #22 daemon prio=5 os_prio=0 tid=0x000000001a4e5000 nid=0x21b0 waiting on condition [0x000000001d4de000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000000d7c18178> (a java.util.concurrent.LinkedTransferQueue)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737)
at java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647)
at java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

"$_MyCatServer" #21 prio=5 os_prio=0 tid=0x000000001a4ec000 nid=0x21b8 runnable [0x000000001d3de000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000000d7c0bb68> (a sun.nio.ch.Util$2)
- locked <0x00000000d7c0bb78> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000000d7c0bae8> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.opencloudb.net.NIOAcceptor.run(NIOAcceptor.java:86)

"$_MyCatManager" #20 prio=5 os_prio=0 tid=0x000000001a4ea800 nid=0x21bc runnable [0x000000001d2df000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000000d7c00de8> (a sun.nio.ch.Util$2)
- locked <0x00000000d7c00df8> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000000d7c00d68> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.opencloudb.net.NIOAcceptor.run(NIOAcceptor.java:86)

"$_NIOConnector" #19 prio=5 os_prio=0 tid=0x000000001a4e9000 nid=0x20f0 runnable [0x000000001d1de000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000000d7c1b600> (a sun.nio.ch.Util$2)
- locked <0x00000000d7c1b610> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000000d7c1b580> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.opencloudb.net.NIOConnector.run(NIOConnector.java:76)

"$_NIOREACTOR-3-RW" #18 prio=5 os_prio=0 tid=0x000000001a4e8000 nid=0x2314 runnable [0x000000001d0de000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000000d7c13868> (a sun.nio.ch.Util$2)
- locked <0x00000000d7c13878> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000000d7c137e8> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.opencloudb.net.NIOReactor$RW.run(NIOReactor.java:89)
at java.lang.Thread.run(Thread.java:745)

"$_NIOREACTOR-2-RW" #17 prio=5 os_prio=0 tid=0x000000001a4e6800 nid=0x1774 runnable [0x000000001cfdf000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000000d7c1e880> (a sun.nio.ch.Util$2)
- locked <0x00000000d7c1e890> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000000d7c1e800> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.opencloudb.net.NIOReactor$RW.run(NIOReactor.java:89)
at java.lang.Thread.run(Thread.java:745)

"$_NIOREACTOR-1-RW" #16 prio=5 os_prio=0 tid=0x000000001a4e7800 nid=0x2320 runnable [0x000000001cede000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000000d7c07a68> (a sun.nio.ch.Util$2)
- locked <0x00000000d7c07a78> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000000d7c079e8> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.opencloudb.net.NIOReactor$RW.run(NIOReactor.java:89)
at java.lang.Thread.run(Thread.java:745)

"$_NIOREACTOR-0-RW" #15 prio=5 os_prio=0 tid=0x000000001a4e6000 nid=0x20ac runnable [0x000000001cdde000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000000d7c0f0c0> (a sun.nio.ch.Util$2)
- locked <0x00000000d7c0f0d0> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000000d7c0f040> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.opencloudb.net.NIOReactor$RW.run(NIOReactor.java:89)
at java.lang.Thread.run(Thread.java:745)

"Log4jWatchdog" #14 daemon prio=5 os_prio=0 tid=0x000000001a4e9800 nid=0x16d0 waiting on condition [0x000000001bd0f000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.log4j.helpers.FileWatchdog.run(FileWatchdog.java:104)

"net.sf.ehcache.CacheManager@3abfe836" #12 daemon prio=5 os_prio=0 tid=0x000000001a4eb000 nid=0x2298 in Object.wait() [0x000000001bb0f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000d7c454a8> (a java.util.TaskQueue)
at java.lang.Object.wait(Object.java:502)
at java.util.TimerThread.mainLoop(Timer.java:526)
- locked <0x00000000d7c454a8> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:505)

"MyCatTimer" #11 daemon prio=5 os_prio=0 tid=0x000000001a372000 nid=0x1f48 in Object.wait() [0x000000001b7bf000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.util.TimerThread.mainLoop(Timer.java:552)
- locked <0x00000000d7c46428> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:505)

"Thread-0" #10 prio=5 os_prio=0 tid=0x0000000019e9b000 nid=0x2304 waiting on condition [0x000000001a6ee000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000000d7cda8b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
at org.opencloudb.route.MyCATSequnceProcessor$ExecuteThread.run(MyCATSequnceProcessor.java:102)

"Service Thread" #9 daemon prio=9 os_prio=0 tid=0x000000001967d000 nid=0x1f5c runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C1 CompilerThread2" #8 daemon prio=9 os_prio=2 tid=0x00000000195f7800 nid=0x213c waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C2 CompilerThread1" #7 daemon prio=9 os_prio=2 tid=0x00000000195f1000 nid=0x20fc waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #6 daemon prio=9 os_prio=2 tid=0x0000000017f9a800 nid=0x1cdc waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Attach Listener" #5 daemon prio=5 os_prio=2 tid=0x0000000017f4f000 nid=0x72c waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" #4 daemon prio=9 os_prio=2 tid=0x0000000017f4e000 nid=0x214c runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Finalizer" #3 daemon prio=8 os_prio=1 tid=0x00000000033b7000 nid=0x2150 in Object.wait() [0x000000001928f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
- locked <0x00000000d7d301b0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)

"Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x0000000017f09000 nid=0x1328 in Object.wait() [0x000000001918f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)
- locked <0x00000000d7d303c8> (a java.lang.ref.Reference$Lock)

"main" #1 prio=5 os_prio=0 tid=0x00000000030be000 nid=0x22c8 waiting on condition [0x00000000032bf000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.opencloudb.MycatStartup.main(MycatStartup.java:57)

"VM Thread" os_prio=2 tid=0x0000000017f07000 nid=0x1218 runnable

"GC task thread#0 (ParallelGC)" os_prio=0 tid=0x00000000032d7800 nid=0x1d54 runnable

"GC task thread#1 (ParallelGC)" os_prio=0 tid=0x00000000032d9000 nid=0x23f4 runnable

"GC task thread#2 (ParallelGC)" os_prio=0 tid=0x00000000032da800 nid=0x1600 runnable

"GC task thread#3 (ParallelGC)" os_prio=0 tid=0x00000000032dc000 nid=0x1aac runnable

"VM Periodic Task Thread" os_prio=2 tid=0x00000000196b9800 nid=0x2308 waiting on condition

JNI global references: 253