linux高可用性Web服务器群集之 ISCSI+GFS+RHCS

一，搭建实验环境

Vmware8.0.11
node1.liuwei.com： eth0：192.168.101.100 host-only
添加 sdb1 1G 心跳线: eth1：192.168.2.1 VMnet2
node2.liuwei.com: eth0: 192.168.101.200 host-only
添加 sdb1 1G 心跳线：eth1:192.168.2.2 VMnet2
target.liuwei.com eth0： 192.168.101.1 host-only
创建yum 仓库

二， [/b]设置target服务器的网卡信息、hosts[/b]、主机名，时钟同步及sdb[/b]硬盘初始化并使相互ping通[/b]
1.setup
2.vim /etc/hosts

127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
192.168.101.100 node1.liuwei.com
192.168.101.200 node2.liuwei.com
192.168.101.1 target.liuwei.com

3.vim /etc/sysconfig/network

NETWORKING=yes
NETWORKING_IPV6=yes
HOSTNAME=target.liuwei.com
4.service network restart
5. hwclock -s

新建磁盘：（这里使用磁盘分区）[/b]
6.fdisk /dev/sdb

n p 3 +1000M w
重新识别分区：（此处不需要格式化挂载）[/b]
7. partprobe /dev/sdb

[root@target ~]# fdisk -l
Disk /dev/sda: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 25 200781 83 Linux
/dev/sda2 26 2192 17406427+ 83 Linux
/dev/sda3 2193 2453 2096482+ 82 Linux swap / Solaris
Disk /dev/sdb: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 * 1 13 104391 83 Linux
/dev/sdb2 14 38 200812+ 83 Linux
/dev/sdb3 39 161 987997+ 83 Linux
8.安装[/b]scsi-target[/b]软件包[/b]
[root@target~]# yum install scsi-target-utils[/b]
[/b]9.设置[/b]target[/b]开机启动，并启动[/b]tgtd[/b]服务[/b]
[root@target-server ~]# chkconfig tgtd on[/b]
[root@target ~]# chkconfig --list tgtd [/b]
tgtd 0:off 1:off 2:on 3:on 4:on 5:on 6:off[/b]
[root@target ~]# service tgtd start[/b]
Starting SCSI target daemon: [ OK ][/b]
10.配置[/b]target[/b]，新建[/b]target[/b]设备和逻辑单元设备[/b]
[root@[/b]target ~]# tgtadm --lld iscsi --op new --mode target --tid 1 --targetname iqn.2012-05-10.com.liuwei.node1:target[/b]
[root@target ~]# tgtadm --lld iscsi --op show --mode target[/b]
[root@target~]# tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 1 --backing-store /dev/sdb3[/b]
[root@target~]# tgtadm --lld iscsi --op show --mode target[/b]
配置身份验证机制：[/b]
[root@target~]# tgtadm --lld iscsi --op bind --mode target --tid 1 --initiator-address 192.168.101.0/24[/b]
[root@target ~]# tgtadm --lld iscsi --op show --mode target[/b]

Target 1: iqn.2012-05-10.com.liuwei.target:target

System information:

Driver: iscsi

State: ready

I_T nexus information:

I_T nexus: 9

Initiator: iqn.2012-05-10.com.liuwei.nodel

Connection: 0

IP Address: 192.168.101.100

LUN information:

LUN: 0

此时的所有配置为临时配置，开机重启会丢失的，若要永久保持，又如下：[/b]2[/b]中方法[/b]

方法[/b]1[/b]：将以上脚本写到开机启动脚本中：[/b]/etc/rc.d/rc.local[/b]
方法[/b]2[/b]：开机启动[/b]
[root@target ~]# vim /etc/tgt/targets.conf[/b]

<target iqn.2012-05-10.com.liuwei.target:target>

backing-store /dev/sdb3

initiator-address 192.168.101.0/24

</target>

安装luci安装包，以便一会儿登录该图形界面：
[root@target ~]# yum install luci –y
[root@target ~]# luci_admin init //该界面的用户名是admin ，密码自己设置，但至少6个字符。
[root@target ~]# service luci start

三 node1.liuwei.com（存储的客户端）配置[/b]
1.setup

2.vim /etc/hosts

127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
192.168.101.100 node1.liuwei.com
192.168.101.200 node2.liuwei.com
192.168.101.1 target.liuwei.com

3.vim /etc/sysconfig/network

NETWORKING=yes
NETWORKING_IPV6=yes
HOSTNAME=node1.liuwei.com
4.service network restart
5. hwclock -s

安装配置[/b]
[root@node1~]# yum install -y iscsi-initiator-utils[/b]
[root@node1~]# vim /etc/iscsi/initiatorname.iscsi[/b]
InitiatorName=iqn.2012-05-10.com.liuwei.node1[/b]
[root@node1~]# chkconfig iscsi on[/b]
[root@node1 ~]# service iscsi start[/b]
iscsid is stopped[/b]
Turning off network shutdown. Starting iSCSI daemon: [ OK ][/b]
[ OK ][/b]
Setting up iSCSI targets: iscsiadm: No records found![/b]
[ OK ][/b]
[root@node1 ~]# iscsiadm --mode discovery --type sendtargets --portal 192.168.101.1 [/b]
//寻找发现服务器

192.168.101.254:3260,1 iqn.2012-05-10.com.liuwei.target:target
// 发现后登录服务器

[root@node1 ~]# iscsiadm --mode node --targetname iqn.2012-05-10.com.liuwei.node1:target --portal 192.168.101.1:3260 --login[/b]
Login to [iface: default, target: iqn.2012-05-10.com.liuwei.node1:target, portal: 192.168.101.254,3260]: successful[/b]

[align=left][root@node1~]# fdisk -l[/b][/align]
[align=left] [/b]/查看磁盘分区情况 会发现有一块共享的存储磁盘，正好是target共享出来的[/align]
[align=left]Disk /dev/sdb: 1011 MB, 1011709440 bytes[/b][/align]
[align=left]32 heads, 61 sectors/track, 1012 cylinders[/b][/align]
[align=left]Units = cylinders of 1952 * 512 = 999424 bytes[/b][/align]

[/b]此时target的[/b]/dev/sdb3将自动排序作为本地磁盘 /dev/sdb[/b]

[align=left]Disk /dev/sdb doesn't contain a valid partition table[/b][/align]
此时/dev/sdb就相当于本地磁盘（可以随意使用，可以格式化，挂载等）

[align=left] 新建磁盘：（这里使用磁盘分区）[/b][/align]
6.fdisk /dev/sdb

n p 1+1000M w

由于ext2和ext3文件系统没有锁机制和推送机制，下面我们换用gfs来解决锁机制和推送机制问题。 [root@node1 ~]# yum install gfs-utils kmod-gfs

Loaded plugins: rhnplugin, security

This system is not registered with RHN.

RHN support will be disabled.

rhel-cluster | 1.3 kB 00:00

rhel-clusterstorage | 1.3 kB 00:00

rhel-server | 1.3 kB 00:00

rhel-vt | 1.3 kB 00:00

Setting up Install Process

Resolving Dependencies

--> Running transaction check

---> Package gfs-utils.i386 0:0.1.20-1.el5 set to be updated

---> Package kmod-gfs.i686 0:0.1.34-2.el5 set to be installed

--> Finished Dependency Resolution

Dependencies Resolved

================================================================================

Package Arch Version Repository Size

================================================================================

Installing:

gfs-utils i386 0.1.20-1.el5 rhel-clusterstorage 234 k

kmod-gfs i686 0.1.34-2.el5 rhel-clusterstorage 147 k

Transaction Summary

================================================================================

Install 2 Package(s)

Update 0 Package(s)

Remove 0 Package(s)

Total download size: 381 k

Is this ok [y/N]: y

Downloading Packages:

--------------------------------------------------------------------------------

Total 26 MB/s | 381 kB 00:00

Running rpm_check_debug

Running Transaction Test

Finished Transaction Test

Transaction Test Succeeded

Running Transaction

Installing : kmod-gfs 1/2

Installing : gfs-utils 2/2

Installed:

gfs-utils.i386 0:0.1.20-1.el5 kmod-gfs.i686 0:0.1.34-2.el5

Complete!

[root@ node1]# pvcreate /dev/sdb //创建物理卷
[root@node1]# vgcreate vg0 /dev/sdb //创建物理卷卷组
[root@node1]# lvcreate -L 800M -n lv0 vg0 //创建逻辑卷
若出现情况不一致的情况,启动clvmd服务
service clvmd restart
[root@node1]# gfs_mkfs -p lock_dlm -t cluster1:lv0 -j 3 /dev/vg0/lv0 //格式化成gfs文件系统
[root@node1]# mkdir /mnt/1
[root@node1]# mount /dev/vg0/lv0 /mnt/1 //将逻辑卷挂载到新建的挂载点
以上步骤即可解决锁机制和推送机制的问题。

[root@node1 ~]# cd /mnt/1

[root@node1]# echo "node1.liuwei.com">>index.html // 为新磁盘建立一个web页面，以便以后访问该页面
为了实现在luci图形界面上启动www服务是可以自动挂载，我们在这里把/mnt/1挂载点卸载掉。

[root@node1 ~]# umount /mnt/1

4.下面登录luci图形界面进行配置：

[root@node2 ~]# yum install luci

Loaded plugins: rhnplugin, security

This system is not registered with RHN.

RHN support will be disabled.

Setting up Install Process

Resolving Dependencies

--> Running transaction check

---> Package luci.i386 0:0.12.2-6.el5 set to be updated

--> Processing Dependency: python-imaging for package: luci

--> Running transaction check

---> Package python-imaging.i386 0:1.1.5-5.el5 set to be updated

--> Processing Dependency: tkinter for package: python-imaging

--> Running transaction check

---> Package tkinter.i386 0:2.4.3-27.el5 set to be updated

--> Processing Dependency: libTix8.4.so for package: tkinter

--> Running transaction check

---> Package tix.i386 1:8.4.0-11.fc6 set to be updated

--> Finished Dependency Resolution

Dependencies Resolved

================================================================================

Package Arch Version Repository Size

================================================================================

Installing:

luci i386 0.12.2-6.el5 rhel-cluster 26 M

Installing for dependencies:

python-imaging i386 1.1.5-5.el5 rhel-server 410 k

tix i386 1:8.4.0-11.fc6 rhel-server 329 k

tkinter i386 2.4.3-27.el5 rhel-server 275 k

Transaction Summary

================================================================================

Install 4 Package(s)

Update 0 Package(s)

Remove 0 Package(s)

Total download size: 27 M

Is this ok [y/N]: y

Downloading Packages:

--------------------------------------------------------------------------------

Total 84 MB/s | 27 MB 00:00

Running rpm_check_debug

Running Transaction Test

Finished Transaction Test

Transaction Test Succeeded

Running Transaction

Installing : tix 1/4

Installing : tkinter 2/4

Installing : python-imaging 3/4

Installing : luci 4/4

Installed:

luci.i386 0:0.12.2-6.el5

Dependency Installed:

python-imaging.i386 0:1.1.5-5.el5 tix.i386 1:8.4.0-11.fc6

tkinter.i386 0:2.4.3-27.el5

Complete!

[root@node2 ~]# luci_admin init

Initializing the luci server

Creating the 'admin' user

Enter password:

Confirm password:

Please wait...

The admin password has been successfully set.

Generating SSL certificates...

The luci server has been successfully initialized

You must restart the luci server for changes to take effect.

Run "service luci restart" to do so

[root@node2 ~]# service luci restart

Shutting down luci: [确定]

Starting luci: Generating https SSL certificates... done

[确定]

Point your web browser to https://node2.liuwei.com:8084 to access luci

[root@node2 ~]#

https://target.liuwei.com:8084

未完待续
http://zdcgood.blog.51cto.com/4449060/856270 http://xjzhujunjie.blog.51cto.com/3582724/855117
安装ricci软件包：
yum install –y ricci
安装后启动该服务 service ricci start
3.node2客户端的配置情况跟node1一样，在这里就不做说明.

Type: controller
SCSI ID: deadbeaf1:0
SCSI SN: beaf10
Size: 0 MB
Online: Yes
Removable media: No
Backing store: No backing store
LUN: 1
Type: disk
SCSI ID: deadbeaf1:1
SCSI SN: beaf11
Size: 1012 MB
Online: Yes
Removable media: No
Backing store: /dev/sdb3
Account information:
ACL information:
192.168.101.0/24