LVS realserver配置机制

在配置LVS的real server的时候，总是会有以下几句:

ifconfig lo:0 $VIP netmask 255.255.255.255broadcast $VIP

 /sbin/route add -host $VIP dev lo:0

 echo "1" > /proc/sys/net/ipv4/conf/lo/arp_ignore

 echo "2" > /proc/sys/net/ipv4/conf/lo/arp_announce

 echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore

 echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce

 sysctl -p > /dev/null 2>&1

实验配置为：

VIP:192.168.1.99

realserver: 192.168.1.41

client: 192.168.1.182

当前realserver机器的路由如下：

[root@internal41 ~]# route

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

192.168.1.0   *                   255.255.255.0      U            0      0        0 eth1

192.168.2.0     *                  255.255.255.0      U            0      0        0 eth0

162.251.0.0     *                  255.255.0.0           U            0      0        0 eth1

default         billrouter.cn. 0.0.0.0                     UG          0      0        0 eth1

如果不设置arp_ignore和arp_announce，那么通过一台其他机器（192.168.1.182）ping VIP(192.168.1.98)的时候，将会有响应如下：

[root@internal41 ~]# tcpdump -i eth1 -nn icmp

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode

listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes

10:03:58.128381 IP 192.168.1.182 > 192.168.1.99: ICMP echo request, id 512, seq 61952, length 40

10:03:58.128447 IP 192.168.1.99 > 192.168.1.182: ICMP echo reply, id 512, seq 61952, length 40

10:03:59.129950 IP 192.168.1.182 > 192.168.1.99: ICMP echo request, id 512, seq 62208, length 40

10:03:59.129961 IP 192.168.1.99 > 192.168.1.182: ICMP echo reply, id 512, seq 62208, length 40

10:04:00.130915 IP 192.168.1.182 > 192.168.1.99: ICMP echo request, id 512, seq 62464, length 40

10:04:00.130926 IP 192.168.1.99 > 192.168.1.182: ICMP echo reply, id 512, seq 62464, length 40

10:04:01.131884 IP 192.168.1.182 > 192.168.1.99: ICMP echo request, id 512, seq 62720, length 40

10:04:01.131897 IP 192.168.1.99 > 192.168.1.182: ICMP echo reply, id 512, seq 62720, length 40

可见响应数据包通过eth1返回。

而LVS就是要禁止这样的事情发生，使得realserver不能响应对VIP的ARP请求。

先看一下arp_ignore与arp_announce的定义：

arp_ignore - INTEGER

        Define different modes for sending replies in response to received ARP requests that resolve local target IP addresses:

        0 - (default): reply for any local target IP address, configured on any interface

        1 - reply only if the target IP address is local address configured on the incoming interface

        2 - reply only if the target IP address is local address configured on the incoming interface and both with the sender's IP address are part from same subnet on this interface

        3 - do not reply for local addresses configured with scope host, only resolutions for global and link addresses are replied

        4-7 - reserved

        8 - do not reply for all local addresses

        The max value from conf/{all,interface}/arp_ignore is used when ARP request is received on the {interface}

arp_announce - INTEGER

        Define different restriction levels for announcing the local source IP address from IP packets in ARP requests sent on interface:

        0 - (default) Use any local address, configured on any interface

        1 - Try to avoid local addresses that are not in the target's subnet for this interface. This mode is useful when target hosts reachable via this interface require the source IP address in ARP requests to be part of their logical network configured
on the receiving interface. When we generate the request we will check all our subnets that include the target IP and will preserve the source address if it is from such subnet. If there is no such subnet we select source address according to the rules for
level 2.

        2 - Always use the best local address for this target.       In this mode we ignore the source address in the IP packet and try to select local address that we prefer for talks with the target host. Such local address is selected by looking for primary
IP addresses on all our subnets on the outgoing interface that include the target IP address. If no suitable local address is found we select the first local address we have on the outgoing interface or on all other interfaces, with the hope we will receive
reply for our request and even sometimes no matter the source IP address we announce. The max value from conf/{all,interface}/arp_announce is used. Increasing the restriction level gives more chance for receiving answer from the resolved target while decreasing
the level announces more valid sender's information.

对一个网络interface比如eth0来说，如果不设置arp_ignore （默认值为0）,那么它将对发往其他interface的ARP请求（该interface无法响应的条件下）进行响应。

对于本实验环境而言，realserver的lo:0上绑定了VIP，如果不设置arp_ignore，那么发往lo:0的ARP请求将会通过eth1返回响应，

将arp_ignore设置为1以后，eth1不会帮lo:0返回响应。其实对本实验环境来说，抑制ARP请求通过echo "1" > /proc/sys/net/ipv4/conf/eth1/arp_ignore就可以实现，最开始的做法应该是针对所有情况的通用做法。

再来看arp_announce，正好与arp_ignore相对应，arp_announce是针对网络interface往外发送ARP请求的时候，设置源IP地址，默认为0，比如lo:0通过eth1往外发ARP请求，默认情况下源IP地址是lo:0上绑定的VIP地址，这样接收方就会confuse，因为realserver不止一个，这样会出现一个IP对应多个MAC地址的情况，实际应该只是最后一次的生效。
设置为2就保证lo:0通过eth1发ARP请求的时候，源IP地址是eth1本身的IP地址。对本实验环境来说，设置echo "2" > /proc/sys/net/ipv4/conf/eth1/arp_announce就可以满足条件，最开始的做法是通用做法。

当设置好以后，在client上先通过arp -d清除arp表，然后去ping VIP,发现已经ping不通了。