高可用消息队列服务构建-RABBITMQ
2012-06-25 10:58
603 查看
openstack采用了一个松耦合的方式来进行各个组建之间的通信,具体来讲组建之间的通信是通过rabbitmq的消息队列进行的。
当openstack系统规模变大时,消息队列就成为了性能发挥的瓶颈。
网上有人给出了集群化的消息队列配置方案,如下:
原文:http://www.sebastien-han.fr/blog/2012/05/21/openstack-high-availability-rabbitmq/
Rabbitmq has his own buildin cluster management system. Here, we don’t need Pacemaker, everything is managed by RabbitMQ itself.
RabbitMQ or more generally the management queues layer is a critical component of OpenStack because every requests/queries use this layer to communicate.
RabbitMQ generates a cookie for each server instance. This cookie must be the same on each member of the cluster:
Check your cluster status, on the node 01 or 02, whatever:
Eventually check your queues on both nodes, they should be identical:
Cluster nodes can be of two types: disk or ram. Disk nodes replicate data in ram and on disk, thus providing redundancy in the event of node failure and recovery from global events such as power failure across all nodes. Ram nodes replicate data in ram only
and are mainly used for scalability. A cluster must always have at least one disk node.
You can also verify that the connection is well established between the node:
If you changed you IP address or your hostname, this is pretty nasty and harsh but it works:
The IP address and/or will be refresh in the rabbitmq database.
Clustering doesn’t mean high-availability, this is why I put a load-balancer on top. Here HAProxy will balance the request only on one node, if this node fails the request will be route to the other node. It’s simple as that. The native port of HAproxy and
the OpenStack queues ports are configured.
Et voilà!
当openstack系统规模变大时,消息队列就成为了性能发挥的瓶颈。
网上有人给出了集群化的消息队列配置方案,如下:
原文:http://www.sebastien-han.fr/blog/2012/05/21/openstack-high-availability-rabbitmq/
Rabbitmq has his own buildin cluster management system. Here, we don’t need Pacemaker, everything is managed by RabbitMQ itself.
RabbitMQ or more generally the management queues layer is a critical component of OpenStack because every requests/queries use this layer to communicate.
CLUSTERING SETUP
1 | $ sudo apt-get install rabbitmq-server |
12 | rabbitmq-01:~$ $ sudo cat /var/lib/rabbitmq/.erlang.cookie |
12 | rabbitmq-02:~$ sudo rabbitmqctl cluster_status Cluster status of node 'rabbit@rabbitmq-02' ... [{nodes,[{disc,['rabbit@rabbitmq-01']},{ram,['rabbitmq-02']}]}, {running_nodes,['rabbit@rabbitmq-01','rabbit@rabbitmq-02']}] ...done. |
12 | $ sudo rabbitmqctl list_queues name slave_pids synchronised_slave_pids Listing queues ... network_fanout_3a9877aa044c47b7846bbe01bf7faa4a volume.server-03 amq.gen-gXDebEl-MNze8Nv83Ox3wD volume_fanout_2df3d830f3ac40a98fe924bb032f6512 cert_fanout_4b1d7951b629470db0d13880b5814a00 amq.gen-waleDspYQ1IP5lKkpmkc9P volume scheduler.server-05 consoleauth_fanout_a0c57c1bca3645eda5b25788de9dd484 amq.gen-gxnO_iZvkdB9BaLHX6g9LT amq.gen-AsSmN5K6zqJ3-xDlJeP16t amq.gen-A9B2HzA2dZmHjP-bNv2rwb compute scheduler network cert.server-05 consoleauth.server-05 network.server2 compute.server2 amq.gen-wN5T2Ylyqbu41-_RzJ4vf8 amq.gen-wHYGWuERrgzEHbBPrmTfwS scheduler_fanout_d57ca4552c9542ed860a0ef8a9d48534 cert compute_fanout_b9c1dd93cb1a4282bb32c066e7d621d2 amq.gen-QtDR90yqOLXuaNEPHozmt_ consoleauth ...done. |
and are mainly used for scalability. A cluster must always have at least one disk node.
You can also verify that the connection is well established between the node:
12 | $ sudo netstat -plantu | grep 10.0. tcp 0 0 10.0.0.1:39958 10.0.0.2:46117 ESTABLISHED 5294/beam.smp |
TIPS
CHANGE THE IP OR THE HOSTNAME OF A NODE
If you changed you IP address or your hostname, this is pretty nasty and harsh but it works:12 | $ sudo rabbitmqctl stop_app $ sudo dpkg-reconfigure rabbitmq-server Stopping rabbitmq-server: RabbitMQ is not running rabbitmq-server. Starting rabbitmq-server: SUCCESS rabbitmq-server. |
CONVERT RAM NODE TO DISK NODE
12 | rabbitmq-02:~$ sudo rabbitmqctl cluster_status Cluster status of node 'rabbit@server-02' ... [{nodes,[{disc,['rabbit@server-01']},{ram,['rabbit@server-02']}]}, {running_nodes,['rabbit@server-01','rabbit@server-02']}] ...done. rabbitmq-02:~$ sudo rabbitmqctl stop_app Stopping node 'rabbit@server-02' ... ...done. rabbitmq-02:~$ sudo rabbitmqctl cluster rabbit@server-01 rabbit@server-02 Clustering node 'rabbit@server-02' with ['rabbit@server-01', 'rabbit@server-02'] ... ...done. rabbitmq-02:~$ sudo rabbitmqctl start_app Starting node 'rabbit@server-02' ... ...done. rabbitmq-02:~$ sudo rabbitmqctl cluster_status Cluster status of node 'rabbit@server-02' ... [{nodes,[{disc,['rabbit@server-02','rabbit@server-01']}]}, {running_nodes,['rabbit@server-01','rabbit@server-02']}] ...done. |
HAPROXY CONFIGURATION
Clustering doesn’t mean high-availability, this is why I put a load-balancer on top. Here HAProxy will balance the request only on one node, if this node fails the request will be route to the other node. It’s simple as that. The native port of HAproxy andthe OpenStack queues ports are configured.
global log 127.0.0.1 local0 #log loghost local0 info maxconn 1024 #chroot /usr/share/haproxy user haproxy group haproxy daemon #debug #quiet defaults log global #log 127.0.0.1:514 local0 debug log 127.0.0.1 local1 debug mode tcp option tcplog option dontlognull retries 3 option redispatch maxconn 1024 # Default! timeout connect 5000ms timeout client 50000ms timeout server 50000ms listen rabbitmq_cluster 0.0.0.0:4369 mode tcp balance roundrobin server server-07_active 172.17.1.8:4369 check inter 5000 rise 2 fall 3 server server-08_backup 172.17.1.9:4369 backup check inter 5000 rise 2 fall 3 listen rabbitmq_cluster_openstack 0.0.0.0:5672 mode tcp balance roundrobin server server-07_active 172.17.1.8:5672 check inter 5000 rise 2 fall 3 server server-08_backup 172.17.1.9:5672 backup check inter 5000 rise 2 fall 3
Et voilà!
相关文章推荐
- 高可用消息队列服务构建-RABBITMQ
- (十一)RabbitMQ消息队列-如何实现高可用
- RabbitMQ服务主机名更改导致消息队列无法连接
- RabbitMQ+HAProxy构建高可用消息队列
- 消息队列基础 RabbitMQ与AMQP协议详解——超大规模高可用OpenStack核心技术深入解析系列(二)
- RabbitMQ消息队列服务
- RabbitMQ 3.0.3 发布,高级消息队列服务
- (四)RabbitMQ消息队列-服务详细配置与日常监控管理
- [导入]构建WCF面向服务的应用程序系列课程(14):消息队列.zip(17.56 MB)
- 跨数据库分布式实时事务 - 基于RabbitMQ实时消息队列服务实现
- RabbitMQ系列二(构建消息队列机制)
- 消息队列基础 RabbitMQ与AMQP协议详解——超大规模高可用OpenStack核心技术深入解析系列(二)
- RabbitMQ系列二(构建消息队列)
- (四)RabbitMQ消息队列-服务详细配置与日常监控管理
- (四)RabbitMQ消息队列-服务详细配置与日常监控管理
- (十二)RabbitMQ消息队列-性能测试
- 上传图文消息内的图片获取URL【订阅号与服务号认证后均可用】
- RabbitMQ五种消息队列学习(六)--通配符模式(路由类型:Topic)
- 使用NBear.MQ分布式服务消息队列模块开发分布式系统
- SpringBoot整合RabbitMQ实现微服务间的异步消息沟通