Canal-Server的HA功能验证
2016-10-20 18:59
453 查看
测试项目:canal的HA功能验证
测试平台:CentOS Linux release 7.1.1503 (Core) Mysql测试版本:5.6.24-72.2-log Canal版本:1.0.22
测试方案:
在测试机部署两台canal-server,配置为:172.17.1.181:11121和172.17.1.181:11120
部署一台mysql:172.17.1.181:3306
测试步骤:
(1) 启动canal-server-1:bin/start.sh,查看zookeeper中canal-server信息:
zk: localhost:2181(CONNECTED) 47] get /otter/canal/destinations/example/running
{"active":true,"address":"172.17.1.181:11121","cid":1}
cZxid = 0x50015c322
ctime = Thu Oct 20 13:49:11 CST 2016
mZxid = 0x50015c322
mtime = Thu Oct 20 13:49:11 CST 2016
pZxid = 0x50015c322
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x357e032d65a0027
dataLength = 54
numChildren = 0
(2) 启动canal-server-2: bin/start.sh
(3) 启动canal-client: ./start.sh >log.log 2>&1 &
(4) 执行数据库数据出入脚本(默认插入数据9999条记录):./insert-data-182.sh,查看client端日志输出
================> binlog[mysql-bin.000003:8224261] , executeTime : 1476943586000 , delay : 17313ms
BEGIN ----> Thread id: 30032
----------------> binlog[mysql-bin.000003:8224389] , name[test,canal_test] , eventType : INSERT , executeTime : 1476943586000 , delay
: 17315ms
id : 1 type=int(10) update=true
name : joe type=varchar(20) update=true
age : 20 type=int(4) update=true
department : AE type=varchar(20) update=true
----------------
END ----> transaction id: 90100
================> binlog[mysql-bin.000003:8224440] , executeTime : 1476943586000 , delay : 17322ms
(5) 等待一会,停止canal-server-1:bin/stop.sh:
1)查看zookeeper中canal-server信息[已切换到canal-server-2]:
[zk: localhost:2181(CONNECTED) 57] get /otter/canal/destinations/example/running
{"active":true,"address":"172.17.1.181:11120","cid":1}
cZxid = 0x50015ec19
ctime = Thu Oct 20 14:03:13 CST 2016
mZxid = 0x50015ec19
mtime = Thu Oct 20 14:03:13 CST 2016
pZxid = 0x50015ec19
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x157dae781b501f7
dataLength = 54
numChildren = 0
2)查看client端日志输出:
================> binlog[mysql-bin.000003:9197437] , executeTime : 1476943610000 , delay : 597ms
BEGIN ----> Thread id: 33804
----------------> binlog[mysql-bin.000003:9197565] , name[test,canal_test] , eventType : INSERT , executeTime : 1476943610000 , delay
: 597ms
id : 3773 type=int(10) update=true
name : joe type=varchar(20) update=true
age : 20 type=int(4) update=true
department : AE type=varchar(20) update=true
----------------
END ----> transaction id: 101416
================> binlog[mysql-bin.000003:9197616] , executeTime : 1476943610000 , delay : 597ms
2016-10-20 14:06:50.648 [Thread-2] WARN c.alibaba.otter.canal.client.impl.ClusterCanalConnector - something goes wrong when getWithoutAck
data from server:null
com.alibaba.otter.canal.protocol.exception.CanalClientException: java.io.IOException: Connection reset by peer
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.getWithoutAck(SimpleCanalConnector.java:281)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.getWithoutAck(SimpleCanalConnector.java:252)
at com.alibaba.otter.canal.client.impl.ClusterCanalConnector.getWithoutAck(ClusterCanalConnector.java:180)
at com.alibaba.otter.canal.example.AbstractCanalClientTest.process(AbstractCanalClientTest.java:113)
at com.alibaba.otter.canal.example.AbstractCanalClientTest$2.run(AbstractCanalClientTest.java:80)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:197)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.read(SimpleCanalConnector.java:376)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.readNextPacket(SimpleCanalConnector.java:366)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.receiveMessages(SimpleCanalConnector.java:286)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.getWithoutAck(SimpleCanalConnector.java:279)
... 5 more
2016-10-20 14:06:56.080 [Thread-2] INFO c.alibaba.otter.canal.client.impl.ClusterCanalConnector - restart the connector for next round
retry.
================> binlog[mysql-bin.000003:9155641] , executeTime : 1476943609000 , delay : 7201ms
BEGIN ----> Thread id: 33642
----------------> binlog[mysql-bin.000003:9155769] , name[test,canal_test] , eventType : INSERT , executeTime : 1476943609000 , delay
: 7201ms
id : 3611 type=int(10) update=true
name : joe type=varchar(20) update=true
age : 20 type=int(4) update=true
department : AE type=varchar(20) update=true
----------------
END ----> transaction id: 100930
================> binlog[mysql-bin.000003:9155820] , executeTime : 1476943609000 , delay : 7201ms
注意:【切换canal-server后,client端存在重复消费,切换前入库数据记录ID:3773,切换后数据记录却从ID:3611重新消费】
(5) 等待一会,启动canal-server-1:bin/start.sh,停止canal-server-2:bin/stop.sh
1)查看zookeeper中canal-server信息[已切换到canal-server-1]:
[zk: localhost:2181(CONNECTED) 58] get /otter/canal/destinations/example/running
{"active":true,"address":"172.17.1.181:11121","cid":1}
cZxid = 0x50015ec54
ctime = Thu Oct 20 14:03:38 CST 2016
mZxid = 0x50015ec54
mtime = Thu Oct 20 14:03:38 CST 2016
pZxid = 0x50015ec54
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x157dae781b501f8
dataLength = 54
numChildren = 0
2)查看client端日志输出
================> binlog[mysql-bin.000003:10076959] , executeTime : 1476943632000 , delay : 469ms
BEGIN ----> Thread id: 37217
----------------> binlog[mysql-bin.000003:10077087] , name[test,canal_test] , eventType : INSERT , executeTime : 1476943632000 , delay
: 469ms
id : 7182 type=int(10) update=true
name : joe type=varchar(20) update=true
age : 20 type=int(4) update=true
department : AE type=varchar(20) update=true
----------------
END ----> transaction id: 111654
================> binlog[mysql-bin.000003:10077138] , executeTime : 1476943632000 , delay : 469ms
2016-10-20 14:07:12.510 [Thread-2] WARN c.alibaba.otter.canal.client.impl.ClusterCanalConnector - something goes wrong when getWithoutAck
data from server:null
com.alibaba.otter.canal.protocol.exception.CanalClientException: java.io.IOException: Connection reset by peer
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.getWithoutAck(SimpleCanalConnector.java:281)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.getWithoutAck(SimpleCanalConnector.java:252)
at com.alibaba.otter.canal.client.impl.ClusterCanalConnector.getWithoutAck(ClusterCanalConnector.java:180)
at com.alibaba.otter.canal.example.AbstractCanalClientTest.process(AbstractCanalClientTest.java:113)
at com.alibaba.otter.canal.example.AbstractCanalClientTest$2.run(AbstractCanalClientTest.java:80)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:197)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.read(SimpleCanalConnector.java:376)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.readNextPacket(SimpleCanalConnector.java:366)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.receiveMessages(SimpleCanalConnector.java:286)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.getWithoutAck(SimpleCanalConnector.java:279)
... 5 more
2016-10-20 14:07:17.924 [Thread-2] INFO c.alibaba.otter.canal.client.impl.ClusterCanalConnector - restart the connector for next round
retry.
================> binlog[mysql-bin.000003:10056061] , executeTime : 1476943631000 , delay : 7092ms
BEGIN ----> Thread id: 37136
----------------> binlog[mysql-bin.000003:10056189] , name[test,canal_test] , eventType : INSERT , executeTime : 1476943631000 , delay
: 7092ms
id : 7101 type=int(10) update=true
name : joe type=varchar(20) update=true
age : 20 type=int(4) update=true
department : AE type=varchar(20) update=true
----------------
END ----> transaction id: 111411
================> binlog[mysql-bin.000003:10056240] , executeTime : 1476943631000 , delay : 7092ms
注意:【切换canal-server后,client端存在重复消费,切换前入库数据记录ID:7182,切换后数据记录却从ID:7101重新消费】
d
测试平台:CentOS Linux release 7.1.1503 (Core) Mysql测试版本:5.6.24-72.2-log Canal版本:1.0.22
测试方案:
在测试机部署两台canal-server,配置为:172.17.1.181:11121和172.17.1.181:11120
部署一台mysql:172.17.1.181:3306
测试步骤:
(1) 启动canal-server-1:bin/start.sh,查看zookeeper中canal-server信息:
zk: localhost:2181(CONNECTED) 47] get /otter/canal/destinations/example/running
{"active":true,"address":"172.17.1.181:11121","cid":1}
cZxid = 0x50015c322
ctime = Thu Oct 20 13:49:11 CST 2016
mZxid = 0x50015c322
mtime = Thu Oct 20 13:49:11 CST 2016
pZxid = 0x50015c322
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x357e032d65a0027
dataLength = 54
numChildren = 0
(2) 启动canal-server-2: bin/start.sh
(3) 启动canal-client: ./start.sh >log.log 2>&1 &
(4) 执行数据库数据出入脚本(默认插入数据9999条记录):./insert-data-182.sh,查看client端日志输出
================> binlog[mysql-bin.000003:8224261] , executeTime : 1476943586000 , delay : 17313ms
BEGIN ----> Thread id: 30032
----------------> binlog[mysql-bin.000003:8224389] , name[test,canal_test] , eventType : INSERT , executeTime : 1476943586000 , delay
: 17315ms
id : 1 type=int(10) update=true
name : joe type=varchar(20) update=true
age : 20 type=int(4) update=true
department : AE type=varchar(20) update=true
----------------
END ----> transaction id: 90100
================> binlog[mysql-bin.000003:8224440] , executeTime : 1476943586000 , delay : 17322ms
(5) 等待一会,停止canal-server-1:bin/stop.sh:
1)查看zookeeper中canal-server信息[已切换到canal-server-2]:
[zk: localhost:2181(CONNECTED) 57] get /otter/canal/destinations/example/running
{"active":true,"address":"172.17.1.181:11120","cid":1}
cZxid = 0x50015ec19
ctime = Thu Oct 20 14:03:13 CST 2016
mZxid = 0x50015ec19
mtime = Thu Oct 20 14:03:13 CST 2016
pZxid = 0x50015ec19
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x157dae781b501f7
dataLength = 54
numChildren = 0
2)查看client端日志输出:
================> binlog[mysql-bin.000003:9197437] , executeTime : 1476943610000 , delay : 597ms
BEGIN ----> Thread id: 33804
----------------> binlog[mysql-bin.000003:9197565] , name[test,canal_test] , eventType : INSERT , executeTime : 1476943610000 , delay
: 597ms
id : 3773 type=int(10) update=true
name : joe type=varchar(20) update=true
age : 20 type=int(4) update=true
department : AE type=varchar(20) update=true
----------------
END ----> transaction id: 101416
================> binlog[mysql-bin.000003:9197616] , executeTime : 1476943610000 , delay : 597ms
2016-10-20 14:06:50.648 [Thread-2] WARN c.alibaba.otter.canal.client.impl.ClusterCanalConnector - something goes wrong when getWithoutAck
data from server:null
com.alibaba.otter.canal.protocol.exception.CanalClientException: java.io.IOException: Connection reset by peer
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.getWithoutAck(SimpleCanalConnector.java:281)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.getWithoutAck(SimpleCanalConnector.java:252)
at com.alibaba.otter.canal.client.impl.ClusterCanalConnector.getWithoutAck(ClusterCanalConnector.java:180)
at com.alibaba.otter.canal.example.AbstractCanalClientTest.process(AbstractCanalClientTest.java:113)
at com.alibaba.otter.canal.example.AbstractCanalClientTest$2.run(AbstractCanalClientTest.java:80)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:197)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.read(SimpleCanalConnector.java:376)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.readNextPacket(SimpleCanalConnector.java:366)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.receiveMessages(SimpleCanalConnector.java:286)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.getWithoutAck(SimpleCanalConnector.java:279)
... 5 more
2016-10-20 14:06:56.080 [Thread-2] INFO c.alibaba.otter.canal.client.impl.ClusterCanalConnector - restart the connector for next round
retry.
================> binlog[mysql-bin.000003:9155641] , executeTime : 1476943609000 , delay : 7201ms
BEGIN ----> Thread id: 33642
----------------> binlog[mysql-bin.000003:9155769] , name[test,canal_test] , eventType : INSERT , executeTime : 1476943609000 , delay
: 7201ms
id : 3611 type=int(10) update=true
name : joe type=varchar(20) update=true
age : 20 type=int(4) update=true
department : AE type=varchar(20) update=true
----------------
END ----> transaction id: 100930
================> binlog[mysql-bin.000003:9155820] , executeTime : 1476943609000 , delay : 7201ms
注意:【切换canal-server后,client端存在重复消费,切换前入库数据记录ID:3773,切换后数据记录却从ID:3611重新消费】
(5) 等待一会,启动canal-server-1:bin/start.sh,停止canal-server-2:bin/stop.sh
1)查看zookeeper中canal-server信息[已切换到canal-server-1]:
[zk: localhost:2181(CONNECTED) 58] get /otter/canal/destinations/example/running
{"active":true,"address":"172.17.1.181:11121","cid":1}
cZxid = 0x50015ec54
ctime = Thu Oct 20 14:03:38 CST 2016
mZxid = 0x50015ec54
mtime = Thu Oct 20 14:03:38 CST 2016
pZxid = 0x50015ec54
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x157dae781b501f8
dataLength = 54
numChildren = 0
2)查看client端日志输出
================> binlog[mysql-bin.000003:10076959] , executeTime : 1476943632000 , delay : 469ms
BEGIN ----> Thread id: 37217
----------------> binlog[mysql-bin.000003:10077087] , name[test,canal_test] , eventType : INSERT , executeTime : 1476943632000 , delay
: 469ms
id : 7182 type=int(10) update=true
name : joe type=varchar(20) update=true
age : 20 type=int(4) update=true
department : AE type=varchar(20) update=true
----------------
END ----> transaction id: 111654
================> binlog[mysql-bin.000003:10077138] , executeTime : 1476943632000 , delay : 469ms
2016-10-20 14:07:12.510 [Thread-2] WARN c.alibaba.otter.canal.client.impl.ClusterCanalConnector - something goes wrong when getWithoutAck
data from server:null
com.alibaba.otter.canal.protocol.exception.CanalClientException: java.io.IOException: Connection reset by peer
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.getWithoutAck(SimpleCanalConnector.java:281)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.getWithoutAck(SimpleCanalConnector.java:252)
at com.alibaba.otter.canal.client.impl.ClusterCanalConnector.getWithoutAck(ClusterCanalConnector.java:180)
at com.alibaba.otter.canal.example.AbstractCanalClientTest.process(AbstractCanalClientTest.java:113)
at com.alibaba.otter.canal.example.AbstractCanalClientTest$2.run(AbstractCanalClientTest.java:80)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:197)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.read(SimpleCanalConnector.java:376)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.readNextPacket(SimpleCanalConnector.java:366)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.receiveMessages(SimpleCanalConnector.java:286)
at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.getWithoutAck(SimpleCanalConnector.java:279)
... 5 more
2016-10-20 14:07:17.924 [Thread-2] INFO c.alibaba.otter.canal.client.impl.ClusterCanalConnector - restart the connector for next round
retry.
================> binlog[mysql-bin.000003:10056061] , executeTime : 1476943631000 , delay : 7092ms
BEGIN ----> Thread id: 37136
----------------> binlog[mysql-bin.000003:10056189] , name[test,canal_test] , eventType : INSERT , executeTime : 1476943631000 , delay
: 7092ms
id : 7101 type=int(10) update=true
name : joe type=varchar(20) update=true
age : 20 type=int(4) update=true
department : AE type=varchar(20) update=true
----------------
END ----> transaction id: 111411
================> binlog[mysql-bin.000003:10056240] , executeTime : 1476943631000 , delay : 7092ms
注意:【切换canal-server后,client端存在重复消费,切换前入库数据记录ID:7182,切换后数据记录却从ID:7101重新消费】
d
相关文章推荐
- canalserver1.0.22基于mariadb10.1.14的HA功能bug
- canal-client的HA功能验证
- [RM HA 1] Cloudera CDH5 RM HA功能验证
- 用issnode+IIS来托管NodeJs Server之三:加入Windows集成验证功能
- Cloudera CDH5 RM HA功能验证
- 用issnode+IIS来托管NodeJs Server之三:加入Windows集成验证功能
- 在WebService中实现window身份验证的功能
- WebLogic Server 7中强大的EJB QL子查询功能
- 【实战】Cyrus IMAP Server IMAPMAGICPLUS预验证远程缓冲区溢出漏洞分析
- SQL SERVER 2005 EXPRESS将在明年六月份实现REPORT SERVER功能
- 使用 Microsoft SQL Server 2000 的全文搜索功能构建 Web 搜索应用程序
- 创建具有自验证功能的textbox控件
- ASP.NET中常用功能代码总结(2)——生成验证图片
- .net 下验证控件[validation server control]的一些吐血使用经验
- Windows 2003 Server磁盘配额功能揭密
- Java Server Faces(JSF)历险(二),验证和转换
- 今天到Microsoft SQL Server网站看了一下, 觉得SQL2005的功能好强大约
- [博客园公告]登录时增加了验证码验证功能
- 一个功能强大的表单验证程序