随笔
2013-06-17 00:01
423 查看
IP地址规划：
LVS：
10.1.1.23 主LVS+keepalived 10.1.1.22 备LVS+keepalived
主mysql：
10.1.1.5 DRBD+heartbeat 10.1.1.6 DRBD+heartbeat 10.1.1.7 VIP（heartbeat） 10.1.1.10 VIP（lvs）
从mysql+nginx：
10.1.1.102 Ng+php 10.1.1.102 Ng+php VIP1：10.1.1.8 （nginx） VIP2: 10.1.1.9 （mysql）安装DRBD+mysql+heartbeat
安装DRBD
1.1.规划
10.1.1.5（主） /data 10.1.1.6（备） /data
2.安装（主备安装一致）：
2.1 安装依赖:
yum install kernel-devel kernel-headers
2.2.编译安装
tar -zxf drbd-8.3.16.tar.gz ./configure --prefix=/usr/local/drbd --with-km make && make install2.3 拷贝配置文件模板
mv /usr/local/drbd/etc/drbd.conf /usr/local/drbd/drbd.conf.old cp /tmp/soft/drbd-8.3.16/scripts/drbd.conf.exampl /usr/local/drbd/etc/drbd.conf
2.4 设置链接
ln -s /usr/local/drbd/etc/drbd.conf /etc/drbd.conf ln -s /usr/local/drbd/etc/rc.d/init.d/drbd /etc/init.d/drbd3.配置文件（主备一致）：
global {
usage-count no;
}
common {
syncer { rate 200M; }
}
resource r0 {
protocol C;
handlers {
pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";
pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";
local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f";
fence-peer "/usr/lib/heartbeat/drbd-peer-outdater -t 5";
pri-lost "/usr/lib/drbd/notify-pri-lost.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";
split-brain "/usr/lib/drbd/notify-split-brain.sh root";
out-of-sync "/usr/lib/drbd/notify-out-of-sync.sh root";
}
startup {
degr-wfc-timeout 120;
outdated-wfc-timeout 2;
}
disk {
on-io-error detach;
fencing resource-only;
}
net {
cram-hmac-alg "sha1";
shared-secret "ckl893";
}
on ckl-bd1 {
device /dev/drbd0;
disk /dev/sda5;
address 10.1.1.5:7788;
flexible-meta-disk internal;
}
on ckl-bd2 {
device /dev/drbd0;
disk /dev/sda5;
address 10.1.1.6:7788;
meta-disk internal;
}
}4.创建数据块：
（主备一致）
#dd if=/dev/zero bs=1M count=1 of=/dev/sda5;sync #drbdadm create-md r0
5.启动DRBD服务
#umount /data #/etc/init.d/drdb start6.配置主服务，格式化DRBD（10.1.1.5）：
#drbdsetup /dev/drbd1 primary –o # mkfs.ext4 /dev/drbd0
7.主挂载DRBD，创建文件（10.1.1.5）：
#mount /dev/drbd0 /mnt #cd /mnt #touch nihao.txt
8.测试同步：
停掉主(10.1.1.5):
#/etc/init.d/drbd stop
从设置为主（10.1.1.6）:
#drbdsetup /dev/drbd1 primary –o # mount /dev/drbd0 /mnt
#cd /mnt # ll nihao.txt -rw-r--r-- 1 root root 0 May 7 10:42 nihao.txt9.切换回来：
10.1.1.6上执行：
#umount /mnt #drbdadm secondary all10.1.1.5 上执行：
#drbdadm primary all # mount /dev/drbd0 /mnt
安装mysql
1.mysql安装规划：
10.1.1.5（主）：/mnt DRBD挂载的分区 10.1.1.6（备）
2.首先启动主DRBD（10.1.1.5）挂载DRBD到/mnt
安装mysql到/mnt 下
3.备（10.1.1.6）切换到主，挂载DRBD 到/mnt
安装mysql到/mnt 下
4.切换回
10.1.1.5主 10.1.1.6备
安装heartbeat
规划：10.1.1.5 （主）
10.1.1.6 （备） 10.1.1.7 （heartbeat VIP）主备安装一致：
1.安装heartbeat 下载需要的源：
wget ftp://mirror.switch.ch/pool/1/mirror/scientificlinux/6rolling/x86_64/os/Packages/epel-release-6-5.noarch.rpm[/code]2.安装heartbeat：yum -y install heartbeat*
3.配置heartbeat
3.1.拷贝模板配置文件：cp /usr/share/doc/heartbeat-3.0.4/ha.cf /etc/ha.d/ cp /usr/share/doc/heartbeat-3.0.4/authkeys /etc/ha.d/ cp /usr/share/doc/heartbeat-3.0.4/haresources /etc/ha.d/4.heartbeat主配置文件（主备一致）：cat /etc/ha.d/ha.cf logfile /var/log/ha-log logfacility local0 keepalive 2 deadtime 30 warntime 10 initdead 120 udpport 694 baud 19200 bcast eth0 auto_failback on watchdog /dev/watchdog node ckl-bd1 node ckl-bd2 ping 10.1.1.1 respawn hacluster /usr/lib64/heartbeat/ipfail apiauth ipfail gid=haclient uid=hacluster5.heartbeat认证文件(主备一致）：cat /etc/ha.d/authkeys auth 1 1 sha1 ckl893!6.资源文件
10.1.1.5（主）：# cat /etc/ha.d/haresources ckl-bd1 drbddisk::r0 Filesystem::/dev/drbd0::/mnt mysql IPaddr::10.1.1.7/24/eth0:1 10.1.1.6（备）： # cat /etc/ha.d/haresources ckl-bd2 drbddisk::r0 Filesystem::/dev/drbd0::/mnt mysql IPaddr::10.1.1.7/24/eth0:1
drbdisk脚本需要添加到/etc/ha.d/resource.d 下
7.测试heartbeat+mysql+DRBD
主，停掉mysql，停掉heartbeat
从监测到后，挂载DRBD，启动mysql
配置mysql主从同步10.1.1.107 主 10.1.1.102 从 10.1.1.103 从配置简单略过
安装LVS
1.安装lvs10.1.1.23 10.1.1.22
执行同样操作：
1.1.安装需要包：yum -y install popt popt-devel popt-static libnl libnl-devel1.2.lvs在2.4之后内核自带，查看是否已经存在：# cat /boot/config-2.6.32-573.el6.x86_64 | grep -i -A 15 "ipvs" # IPVS transport protocol load balancing support # CONFIG_IP_VS_PROTO_TCP=y CONFIG_IP_VS_PROTO_UDP=y CONFIG_IP_VS_PROTO_AH_ESP=y CONFIG_IP_VS_PROTO_ESP=y CONFIG_IP_VS_PROTO_AH=y CONFIG_IP_VS_PROTO_SCTP=y # # IPVS scheduler # CONFIG_IP_VS_RR=m CONFIG_IP_VS_WRR=m CONFIG_IP_VS_LC=m CONFIG_IP_VS_WLC=m CONFIG_IP_VS_LBLC=m CONFIG_IP_VS_LBLCR=m CONFIG_IP_VS_DH=m CONFIG_IP_VS_SH=m CONFIG_IP_VS_SED=m CONFIG_IP_VS_NQ=m # # IPVS application helper # CONFIG_IP_VS_FTP=m CONFIG_IP_VS_PE_SIP=m # # IP: Netfilter Configuration # CONFIG_NF_DEFRAG_IPV4=m CONFIG_NF_CONNTRACK_IPV4=m # CONFIG_NF_CONNTRACK_PROC_COMPAT is not set CONFIG_IP_NF_QUEUE=m CONFIG_IP_NF_IPTABLES=m CONFIG_IP_NF_MATCH_ADDRTYPE=m CONFIG_IP_NF_MATCH_AH=m CONFIG_IP_NF_MATCH_ECN=m1.3.说明ipvs：IPVS transport protocol：ipvs支持的协议，tcp、udp、ah、esp、sctp ipvs scheduler：ipvs调度模式，RR、WRR、LC、WLC、LBLC、LBLCR、DH、SH、SED、NQ1.4.安装lvsadmin(lvs管理工具）：tar -zxvf ipvsadm-1.26.tar.gz cd ipvsadm-1.26 make && make install
2.安装keepalived
2.1.安装需要包：yum -y install openssl-devel popt-devel libnl* libpopt* popt-static ln -s /usr/src/kernels/2.6.32-279.el6.x86_64/ /usr/src/linux2.2.安装lvstar -zxvf keepalived-1.2.4.tar.gz cd keepalived-1.2.4 ./configure --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --mandir=/usr/share/man --infodir=/usr/share/info --sharedstatedir=/usr/com --with-kernel-dir=/usr/src/linux make && make install3.配置keepalived
3.1.在10.1.1.23 上配置：# vi /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { kangly@cosyjoy.com } notification_email_from Alexandre.Cassen@firewall.loc smtp_server 10.1.1.23 smtp_connect_timeout 30 router_id LVS_MASTER } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 77 priority 100 advert_int 1 authentication { auth_type PASS auth_pass ckl893 } virtual_ipaddress { 10.1.1.8/24 #nginx VIP 10.1.1.9/24 #mysql slave VIP 10.1.1.10/24 #mysql 主VIP } } virtual_server 10.1.1.8 80 { delay_loop 6 lb_algo rr lb_kind DR persistence_timeout 50 protocol TCP real_server 10.1.1.102 80 { weight 1 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 } } real_server 10.1.1.103 80 { weight 1 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } } virtual_server 10.1.1.9 3306 { delay_loop 30 lb_algo rr lb_kind DR persistence_timeout 120 protocol TCP real_server 10.1.1.102 3306 { weight 1 MISC_CHECK { misc_path "/etc/keepalived/check_slave.pl 10.1.1.102" misc_dynamic } } real_server 10.1.1.103 3306 { weight 1 MISC_CHECK { misc_path "/etc/keepalived/check_slave.pl 10.1.1.103" misc_dynamic } } }3.2.配置备lvs
在10.1.1.22 上：
主从配置文件不一致
state BACKUP 状态为备份
priority 90 优先级小于主
4.本地添加VIP脚本：
10.1.1.5
10.1.1.6# cat /etc/init.d/lvsrs #!/bin/bash #description : start realserver VIP=10.1.1.10 . /etc/rc.d/init.d/functions case "$1" in start) echo " start LVS of REALServer" /sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce ;; stop) /sbin/ifconfig lo:0 down echo "close LVS REALserver" echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce ;; *) echo "Usage: $0 {start|stop}" exit 1 esac5.本地添加ip脚本：
10.1.1.102
10.1.1.103#!/bin/bash #description : start realserver VIP1=10.1.1.8 VIP2=10.1.1.9 . /etc/rc.d/init.d/functions case "$1" in start) echo " start LVS of REALServer" /sbin/ifconfig lo:0 $VIP1 broadcast $VIP1 netmask 255.255.255.255 up /sbin/ifconfig lo:1 $VIP2 broadcast $VIP2 netmask 255.255.255.255 up echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce ;; stop) /sbin/ifconfig lo:0 down echo "close LVS REALserver" echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce ;; *) echo "Usage: $0 {start|stop}" exit 1 esacarp相关参数说明：arp_annonuce（是否将本机地址网络通过arp通告给其它网络）： 0：通告本机所有地址接口的网络。 1：尽量只通告本接口地址的网络。 2：只通告连接本地借口地址网络。 arg_ignore（是否响应其它的arp请求）： 0：在接受到请求后，将本地所有地址响应出去。 1：请求报文从哪个接口进入，就必须使用此接口配置的地址才能响应。
6.mysql 同步检测脚本：
注意事项：执行用户必须有show slave status 权限# cat check_slave.pl #!/usr/bin/perl -w use DBI; use DBD::mysql; $SBM = 120; $db = "cookbook"; $host = $ARGV[0]; $port = 3306; $user = "root"; $pw = 'yiqu@123'; #这里是单引号，因为有@ $query = "show slave status"; $dbh = DBI->connect("DBI:mysql:$db:$host:$port", $user, $pw, { RaiseError => 0,PrintError => 0 }); if (!defined($dbh)) { exit 1; } $sqlQuery = $dbh->prepare($query); $sqlQuery->execute; $Slave_IO_Running = ""; $Slave_SQL_Running = ""; $Seconds_Behind_Master = ""; while (my $ref = $sqlQuery->fetchrow_hashref()) { $Slave_IO_Running = $ref->{'Slave_IO_Running'}; $Slave_SQL_Running = $ref->{'Slave_SQL_Running'}; $Seconds_Behind_Master = $ref->{'Seconds_Behind_Master'}; } $sqlQuery->finish; $dbh->disconnect(); if ( $Slave_IO_Running eq "No" || $Slave_SQL_Running eq "No" ) { exit 1; } else { if ( $Seconds_Behind_Master gt $SBM ) { exit 1; } else { exit 0; } }7.启动LVS# /etc/init.d/keepalived start查看LVS路由表信息：#ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.1.1.8:80 rr persistent 50 -> 10.1.1.102:80 Route 1 0 0 -> 10.1.1.103:80 Route 1 0 0 TCP 10.1.1.9:3306 rr persistent 120 -> 10.1.1.102:3306 Route 124 0 0 -> 10.1.1.103:3306 Route 124 0 0 TCP 10.1.1.10:3306 rr persistent 120 -> 10.1.1.7:3306 Route 1 0 0
8.测试web
停掉：10.1.1.102 nginx# /etc/init.d/nginx stop查看LVS路由表信息：# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.1.1.8:80 rr persistent 50 -> 10.1.1.103:80 Route 1 0 0 TCP 10.1.1.9:3306 rr persistent 120 -> 10.1.1.102:3306 Route 124 0 0 -> 10.1.1.103:3306 Route 124 0 0 TCP 10.1.1.10:3306 rr persistent 120 -> 10.1.1.7:3306 Route 1 0 0路由表删除了10.1.1.102 的信息;
启动10.1.1.102 nginx# /etc/init.d/nginx start
查看LVS路由表信息：# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.1.1.8:80 rr persistent 50 -> 10.1.1.102:80 Route 1 0 0 -> 10.1.1.103:80 Route 1 0 0 TCP 10.1.1.9:3306 rr persistent 120 -> 10.1.1.102:3306 Route 124 0 0 -> 10.1.1.103:3306 Route 124 0 0 TCP 10.1.1.10:3306 rr persistent 120 -> 10.1.1.7:3306 Route 1 0 09.测试mysql读
停止10.1.1.102 mysql
# /etc/init.d/mysqld stop
查看路由表信息：# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.1.1.8:80 rr persistent 50 -> 10.1.1.102:80 Route 1 0 0 -> 10.1.1.103:80 Route 1 0 0 TCP 10.1.1.9:3306 rr persistent 120 -> 10.1.1.103:3306 Route 124 0 0 TCP 10.1.1.10:3306 rr persistent 120 -> 10.1.1.7:3306 Route 1 0 0已经剔除了10.1.1.102 mysql
启动10.1.1.102 mysql# /etc/init.d/mysqld start查看路由表信息：# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.1.1.8:80 rr persistent 50 -> 10.1.1.102:80 Route 1 0 0 -> 10.1.1.103:80 Route 1 0 0 TCP 10.1.1.9:3306 rr persistent 120 -> 10.1.1.103:3306 Route 124 0 0 TCP 10.1.1.10:3306 rr persistent 120 -> 10.1.1.7:3306 Route 1 0 0测试检测同步脚本：
停掉10.1.1.102 同步：>slave stop；
查看路由表信息：# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.1.1.8:80 rr persistent 50 -> 10.1.1.102:80 Route 1 0 0 -> 10.1.1.103:80 Route 1 0 0 TCP 10.1.1.9:3306 rr persistent 120 -> 10.1.1.103:3306 Route 124 0 0 TCP 10.1.1.10:3306 rr persistent 120 -> 10.1.1.7:3306 Route 1 0 0启动同步>slave start；查看路由表信息：# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.1.1.8:80 rr persistent 50 -> 10.1.1.102:80 Route 1 0 0 -> 10.1.1.103:80 Route 1 0 0 TCP 10.1.1.9:3306 rr persistent 120 -> 10.1.1.103:3306 Route 124 0 0 TCP 10.1.1.10:3306 rr persistent 120 -> 10.1.1.7:3306 Route 1 0 0错误信息：[root@localhost keepalived]# tail -f /var/log/messages May 6 11:00:03 localhost Keepalived_vrrp[4233]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.1.1.8 May 6 11:00:10 localhost Keepalived_vrrp[4233]: receive an invalid ip number count associated with VRID! May 6 11:00:10 localhost Keepalived_vrrp[4233]: bogus VRRP packet received on eth0 !!! May 6 11:00:10 localhost Keepalived_vrrp[4233]: VRRP_Instance(VI_1) Dropping received VRRP packet... May 6 11:00:11 localhost Keepalived_vrrp[4233]: receive an invalid ip number count associated with VRID!
解决办法：修改virtual_router_id 51 为另一个值，如：virtual_router_id 77change master to master_host='10.1.1.7',master_port=3306,master_user='repl',master_password='slavepass',master_log_file='mysql-bin.000017',master_log_pos=881; perl Can't locate DBI.pm yum -y install perl perl-DBI perl-DBD-MySQL
本文出自 “深呼吸再出击” 博客，请务必保留此出处http://ckl893.blog.51cto.com/8827818/1682356
内容来自用户分享和网络整理，不保证内容的准确性，如有侵权内容，可联系管理员处理 
相关文章推荐
- 随笔之POSIX cond和Windows同步对象Event的讨论
- SQL语法随笔
- 随笔，LINUX基础
- JS工作随笔04
- 随笔-运幄
- Spring的四种事务管理（一种编程式事务+三种声明事务）<学习随笔>
- 设计模式随笔系列：来杯咖啡-装饰者模式（Decorator）[转]
- [随笔] 项目性能追踪 之 structs & spring 的拦截器
- sl学习随笔
- easyui 使用随笔
- SoLoMo 随笔
- Page Cache的优化（随笔）
- 2012-02-01随笔
- Java 基础入门随笔（11） JavaSE版——继承、覆盖、抽象类
- SNS游戏中的数据特征随笔
- 【android随笔】工具类整理--两次点击BACK键，退出应用的实现
- 【随笔】协作、分享、竞争和尊重--团队性格塑造！
- 随笔—邀请赛前训— Codeforces Round #330 (Div. 2) Vitaly and Night
- 随笔
- 如何提高码农产量，基于ASP.NET MVC的敏捷开发框架开发随笔一