failover 故障检测、报警、自动切换 程序 monitor_nginx.sh 用于对web server (Nginx) 监测与故障处理

 

前言

前几天，群里有人说用shell程序实现failover,所以今天自己写一个shell程序 monitor_nginx.sh

failover,fault detection and switching ,故障转移与切换程序(shell)

一.程序功能

用于检测web server (Nginx)是否宕掉，如果是用短信报警(sendmail139邮箱报警)并自动故障处理，浮动改热备机IP，实现故障转移与自动切换

二.系统架构



如图所示

172.29.141.112 上部署 Nginx 与 fastcgi(对外开放)

172.29.141.113 上部署 Nginx 与 fastcgi (作为热备)

172.29.141.115 上部署 MySQL

三. 程序源码

cat monitor_nginx.sh

[c-sharp]
view plaincopyprint?

#!/bin/bash
#author longxibendi
#blog http://blog.csdn.net/longxibendi #function failover the web server and send message to SA

# send email to longxibendi@139.com
function_sendEmail ()
{

/usr/local/bin/sendEmail -f monitor_sys@163.com -t longxibendi@139.com -s smtp.163.com -u "web sever down" -xu monitor_sys -xp 123456789 -m "` date ` "

}

# change ip address
function_change_ip ()
{
/sbin/ifconfig eth0 172.29.141.112 broadcast 172.29.141.255 netmask 255.255.255.0 up

/sbin/route add -host 172.29.141.112 dev eth0
/sbin/arping -I eth0 -c 3 -s 172.29.141.112 172.29.141.115
### /sbin/arping is to add arp list in 172.29.141.115
}

# to start local nginx web server
function_start_nginx_server ()
{
ulimit -SHn 65535
/usr/local/webserver/php/sbin/php-fpm start
/usr/local/webserver/nginx/sbin/nginx

}

# main function
function_main ()
{

while true
do
http_status_code=`curl -o /dev/null -s -w %{http_code} http://172.29.141.112/wordpress/`
# echo "$http_status_code";
if [ ${http_status_code} != 200 ] ;then
function_sendEmail ;
function_change_ip ;
function_start_nginx_server ;
fi

# execute curl per 3 seconds

sleep 3

done

}

function_main ;

#!/bin/bash#author longxibendi#blog http://blog.csdn.net/longxibendi#function failover the web server and send message to SA # send email to longxibendi@139.comfunction_sendEmail (){ /usr/local/bin/sendEmail -f monitor_sys@163.com -t longxibendi@139.com -s smtp.163.com -u "web sever down" -xu monitor_sys -xp 123456789 -m "` date ` " } # change ip address function_change_ip (){ /sbin/ifconfig eth0 172.29.141.112 broadcast 172.29.141.255 netmask 255.255.255.0 up /sbin/route add -host 172.29.141.112 dev eth0 /sbin/arping -I eth0 -c 3 -s 172.29.141.112 172.29.141.115 ### /sbin/arping is to add arp list in 172.29.141.115 } # to start local nginx web server function_start_nginx_server (){ ulimit -SHn 65535 /usr/local/webserver/php/sbin/php-fpm start /usr/local/webserver/nginx/sbin/nginx } # main functionfunction_main (){ while truedo http_status_code=`curl -o /dev/null -s -w %{http_code} http://172.29.141.112/wordpress/` # echo "$http_status_code"; if [ ${http_status_code} != 200 ] ;then function_sendEmail ; function_change_ip ; function_start_nginx_server ; fi # execute curl per 3 seconds sleep 3 done } function_main ;

四.程序说明

function_sendEmail 发送email到139邮箱，报警

function_change_ip 浮动更改ip，添加路由，修改db的arp表

function_start_nginx_server 启动本机nginx服务器，这一步也可以调整，如果本机的nginx已启动的话

function_main 主函数，通过curl 网址得到状态码

基本思想有两条

<!--[if !supportLists]-->1．<!--[endif]-->通过curl检测故障

<!--[if !supportLists]-->2．<!--[endif]-->通过浮动修改(ifconfig) ip 实现故障切换(failover)

五.如何部署

在172.29.141.113上部署。如果在112上部署，不需要function_change_ip这个函数。这是一种简单的解决方案，也可以用keepalive，如果PV很大，可能需要用lvs+keepalive或者硬件负载均衡器。

六.参考资料

1. man curl 这里给出重要的参数

-w/--write-out <format>

Defines what to display on stdout after a completed and success-

ful operation. The format is a string that may contain plain

text mixed with any number of variables. The string can be spec-

ified as "string", to get read from a particular file you spec-

ify it "@filename" and to tell curl to read the format from

stdin you write "@-".

The variables present in the output format will be substituted

by the value or text that curl thinks fit, as described below.

All variables are specified like %{variable_name} and to output

a normal % you just write them like %%. You can output a newline

by using /n, a carriage return with /r and a tab space with /t.

NOTE: The %-letter is a special letter in the win32-environment,

where all occurrences of % must be doubled when using this

option.

Available variables are at this point:

url_effective The URL that was fetched last. This is mostly

meaningful if you’ve told curl to follow loca-

tion: headers.

http_code The numerical code that was found in the last

retrieved HTTP(S) page.

http_connect The numerical code that was found in the last

response (from a proxy) to a curl CONNECT

request. (Added in 7.12.4)

time_total The total time, in seconds, that the full opera-

tion lasted. The time will be displayed with mil-

lisecond resolution.

time_namelookup

The time, in seconds, it took from the start

until the name resolving was completed.

time_connect The time, in seconds, it took from the start

until the connect to the remote host (or proxy)

was completed.

time_pretransfer

The time, in seconds, it took from the start

until the file transfer is just about to begin.

This includes all pre-transfer commands and nego-

tiations that are specific to the particular pro-

tocol(s) involved.

time_redirect The time, in seconds, it took for all redirection

steps include name lookup, connect, pretransfer

and transfer before final transaction was

started. time_redirect shows the complete execu-

tion time for multiple redirections. (Added in

7.12.3)

time_starttransfer

The time, in seconds, it took from the start

until the first byte is just about to be trans-

ferred. This includes time_pretransfer and also

the time the server needs to calculate the

result.

size_download The total amount of bytes that were downloaded.

size_upload The total amount of bytes that were uploaded.

size_header The total amount of bytes of the downloaded head-

ers.

size_request The total amount of bytes that were sent in the

HTTP request.

speed_download The average download speed that curl measured for

the complete download.

speed_upload The average upload speed that curl measured for

the complete upload.

content_type The Content-Type of the requested document, if

there was any.

num_connects Number of new connects made in the recent trans-

fer. (Added in 7.12.3)

num_redirects Number of redirects that were followed in the

request. (Added in 7.12.3)

ftp_entry_path The initial path libcurl ended up in when logging

on to the remote FTP server. (Added in 7.15.4)

2.HTTP状态码参考

http://baike.baidu.com/view/1790469.htm