mdadm命令详解及实验过程

一.概念
mdadm是multiple devices admin的简称，它是Linux下的一款标准的软件 RAID 管理工具，作者是Neil Brown
二.特点

mdadm能够诊断、监控和收集详细的阵列信息 　　
mdadm是一个单独集成化的程序而不是一些分散程序的集合，因此对不同RAID管理命令有共通的语法
　　
mdadm能够执行几乎所有的功能而不需要配置文件(也没有默认的配置文件）

三.作用 （引用）

在linux系统中目前以MD(Multiple Devices)虚拟块设备的方式实现软件RAID,利用多个底层的块设备虚拟出一个新的虚拟设备,并且利用条带化(stripping)技术将数据块均匀分布到多个磁盘上来提高虚拟设备的读写性能,利用不同的数据冗祭算法来保护用户数据不会因为某个块设备的故障而完全丢失,而且还能在设备被替换后将丢失的数据恢复到新的设备上.
目前MD支持linear,multipath,raid0(stripping),raid1(mirror),raid4,raid5,raid6,raid10等不同的冗余级别和级成方式,当然也能支持多个RAID陈列的层叠组成raid1 0,raid5 1等类型的陈列

四.实验
试题：建立4个大小为1G的磁盘，并将其中3个创建为raid5的阵列磁盘，1个为热备份磁盘。测试热备份磁盘替换阵列中的磁盘并同步数据。移除损坏的磁盘，添加一个新磁盘作为热备份磁盘。最后要求开机自动挂载。
4.1创建磁盘
[root@xiao ~]# fdisk /dev/sda

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').

Command (m for help): n
First cylinder (10486-13054, default 10486):
Using default value 10486
Last cylinder, +cylinders or +size{K,M,G} (10486-13054, default 13054): +1G

Command (m for help): n
First cylinder (10618-13054, default 10618):
Using default value 10618
Last cylinder, +cylinders or +size{K,M,G} (10618-13054, default 13054): +1G

Command (m for help): n
First cylinder (10750-13054, default 10750):
Using default value 10750
Last cylinder, +cylinders or +size{K,M,G} (10750-13054, default 13054): +1G

Command (m for help): n
First cylinder (10882-13054, default 10882):
Using default value 10882
Last cylinder, +cylinders or +size{K,M,G} (10882-13054, default 13054): +1G

Command (m for help): t
Partition number (1-8): 8
Hex code (type L to list codes): fd
Changed system type of partition 8 to fd (Linux raid autodetect)

Command (m for help): t
Partition number (1-8): 7
Hex code (type L to list codes): fd
Changed system type of partition 7 to fd (Linux raid autodetect)

Command (m for help): t
Partition number (1-8): 6
Hex code (type L to list codes): fd
Changed system type of partition 6 to fd (Linux raid autodetect)

Command (m for help): t
Partition number (1-8): 5
Hex code (type L to list codes): fd
Changed system type of partition 5 to fd (Linux raid autodetect)

Command (m for help): p

Disk /dev/sda: 107.4 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0008ed57

Device Boot Start End Blocks Id System
/dev/sda1 * 1 26 204800 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2 26 10225 81920000 83 Linux
/dev/sda3 10225 10486 2097152 82 Linux swap / Solaris
/dev/sda4 10486 13054 20633279 5 Extended
/dev/sda5 10486 10617 1058045 fd Linux raid autodetect
/dev/sda6 10618 10749 1060258+ fd Linux raid autodetect
/dev/sda7 10750 10881 1060258+ fd Linux raid autodetect
/dev/sda8 10882 11013 1060258+ fd Linux raid autodetect

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: 设备或资源忙.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.

4.2加载内核

[root@xiao ~]# partx -a /dev/sda5 /dev/sda
[root@xiao ~]# partx -a /dev/sda6 /dev/sda
[root@xiao ~]# partx -a /dev/sda7 /dev/sda
[root@xiao ~]# partx -a /dev/sda8 /dev/sda

4.3创建raid5及其热备份盘

[root@xiao ~]# mdadm -C /dev/md0 -l 5 -n 3 -x 1 /dev/sda{5,6,7,8}
mdadm: /dev/sda5 appears to be part of a raid array:
level=raid5 devices=3 ctime=Wed Dec 17 00:58:24 2014
mdadm: /dev/sda6 appears to be part of a raid array:
level=raid5 devices=3 ctime=Wed Dec 17 00:58:24 2014
mdadm: /dev/sda7 appears to be part of a raid array:
level=raid5 devices=3 ctime=Wed Dec 17 00:58:24 2014
mdadm: /dev/sda8 appears to be part of a raid array:
level=raid5 devices=3 ctime=Wed Dec 17 00:58:24 2014
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.

4.4初始化时间和磁盘阵列的读写的应用相关，使用cat /proc/mdstat信息查询RAID阵列当前重构的速度和预期的完成时间。

[root@xiao ~]# cat /proc/mdstat
Personalities : [raid1] [raid0] [raid6] [raid5] [raid4]
md0 : active raid5 sda7[4] sda8[3](S) sda6[1] sda5[0]
2113536 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [UU_]
[=========>...........] recovery = 45.5% (482048/1056768) finish=0.3min speed=30128K/sec

unused devices: <none>
[root@xiao ~]# cat /proc/mdstat
Personalities : [raid1] [raid0] [raid6] [raid5] [raid4]
md0 : active raid5 sda7[4] sda8[3](S) sda6[1] sda5[0]
2113536 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]

unused devices: <none>

[root@xiao ~]# mke2fs -t ext3 /dev/md0 //格式化raid
4.5挂载raid到/mnt目录下，并查看是否正常（显示lost+found为正常）

[root@xiao ~]# mount /dev/md0 /mnt
[root@xiao ~]# ls /mnt
lost+found

4.6查看raid阵列的详细信息

[root@xiao ~]# mdadm -D /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Wed Dec 17 03:38:08 2014
Raid Level : raid5
Array Size : 2113536 (2.02 GiB 2.16 GB)
Used Dev Size : 1056768 (1032.17 MiB 1082.13 MB)
Raid Devices : 3
Total Devices : 4
Persistence : Superblock is persistent

Update Time : Wed Dec 17 03:55:11 2014
State : clean
Active Devices : 3
Working Devices : 4
Failed Devices : 0
Spare Devices : 1

Layout : left-symmetric
Chunk Size : 512K

Name : xiao:0 (local to host xiao)
UUID : bce110f2:34f3fbf1:8de472ed:633a374f
Events : 18

Number Major Minor RaidDevice State
0 8 5 0 active sync /dev/sda5
1 8 6 1 active sync /dev/sda6
4 8 7 2 active sync /dev/sda7

3 8 8 - spare /dev/sda8
4.7模拟损坏其中的一个磁盘，这里我选择 /dev/sda6磁盘

[root@xiao ~]# mdadm /dev/md0 --fail /dev/sda6
mdadm: set /dev/sda6 faulty in /dev/md0

4.7查看raid阵列详细信息，发现/dev/sda8自动替换了损坏的/dev/sda6磁盘。
[root@xiao ~]# mdadm -D /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Wed Dec 17 03:38:08 2014
Raid Level : raid5
Array Size : 2113536 (2.02 GiB 2.16 GB)
Used Dev Size : 1056768 (1032.17 MiB 1082.13 MB)
Raid Devices : 3
Total Devices : 4
Persistence : Superblock is persistent

Update Time : Wed Dec 17 04:13:59 2014
State : clean, degraded, recovering
Active Devices : 2
Working Devices : 3
Failed Devices : 1
Spare Devices : 1

Layout : left-symmetric
Chunk Size : 512K

Rebuild Status : 43% complete

Name : xiao:0 (local to host xiao)
UUID : bce110f2:34f3fbf1:8de472ed:633a374f
Events : 26

Number Major Minor RaidDevice State
0 8 5 0 active sync /dev/sda5
3 8 8 1 spare rebuilding /dev/sda8
4 8 7 2 active sync /dev/sda7

1 8 6 - faulty /dev/sda6

[root@xiao ~]# cat /proc/mdstat
Personalities : [raid1] [raid0] [raid6] [raid5] [raid4]
md0 : active raid5 sda7[4] sda8[3] sda6[1](F) sda5[0]
2113536 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] #正常的情况会是[UUU],若第一个磁盘损坏则显示[ _UU ].

4.8 移除损坏的硬盘

[root@xiao ~]# mdadm /dev/md0 -r /dev/sda6
mdadm: hot removed /dev/sda6 from /dev/md0

4.9添加一个新硬盘作为热备份盘
[root@xiao ~]# fdisk /dev/sda

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').

Command (m for help): n
First cylinder (11014-13054, default 11014):
Using default value 11014
Last cylinder, +cylinders or +size{K,M,G} (11014-13054, default 13054): +1G

Command (m for help): t
Partition number (1-9): 9
Hex code (type L to list codes): fd
Changed system type of partition 9 to fd (Linux raid autodetect)

Command (m for help): p

Disk /dev/sda: 107.4 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0008ed57

Device Boot Start End Blocks Id System
/dev/sda1 * 1 26 204800 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2 26 10225 81920000 83 Linux
/dev/sda3 10225 10486 2097152 82 Linux swap / Solaris
/dev/sda4 10486 13054 20633279 5 Extended
/dev/sda5 10486 10617 1058045 fd Linux raid autodetect
/dev/sda6 10618 10749 1060258+ fd Linux raid autodetect
/dev/sda7 10750 10881 1060258+ fd Linux raid autodetect
/dev/sda8 10882 11013 1060258+ fd Linux raid autodetect
/dev/sda9 11014 11145 1060258+ fd Linux raid autodetect

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: 设备或资源忙.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
[root@xiao ~]# partx -a /dev/sda9 /dev/sda

[root@xiao ~]# mdadm /dev/md0 --add /dev/sda9
mdadm: added /dev/sda9

[root@xiao ~]# mdadm -D /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Wed Dec 17 03:38:08 2014
Raid Level : raid5
Array Size : 2113536 (2.02 GiB 2.16 GB)
Used Dev Size : 1056768 (1032.17 MiB 1082.13 MB)
Raid Devices : 3
Total Devices : 4
Persistence : Superblock is persistent

Update Time : Wed Dec 17 04:39:35 2014
State : clean
Active Devices : 3
Working Devices : 4
Failed Devices : 0
Spare Devices : 1

Layout : left-symmetric
Chunk Size : 512K

Name : xiao:0 (local to host xiao)
UUID : bce110f2:34f3fbf1:8de472ed:633a374f
Events : 41

Number Major Minor RaidDevice State
0 8 5 0 active sync /dev/sda5
3 8 8 1 active sync /dev/sda8
4 8 7 2 active sync /dev/sda7

5 8 9 - spare /dev/sda9
5.开机自动挂载
编辑/etc/fsab文件
/dev/md0 /mnt ext3 defaults 0 0
:wq

6.mdadm中文man(引用)
基本语法 ： mdadm [mode] [options]
[mode] 有7种：
Assemble：将以前定义的某个阵列加入当前在用阵列。
Build：Build a legacy array ，每个device 没有 superblocks
Create：创建一个新的阵列，每个device 具有 superblocks
Manage： 管理阵列，比如 add 或 remove
Misc：允许单独对阵列中的某个 device 做操作，比如抹去superblocks 或 终止在用的阵列。
Follow or Monitor:监控 raid 1,4,5,6 和 multipath 的状态
Grow：改变raid 容量或 阵列中的 device 数目
可用的 [options]:
-A, --assemble：加入一个以前定义的阵列
-B, --build：Build a legacy array without superblocks.
-C, --create：创建一个新的阵列
-Q, --query：查看一个device，判断它为一个 md device 或是 一个 md 阵列的一部分
-D, --detail：打印一个或多个 md device 的详细信息
-E, --examine：打印 device 上的 md superblock 的内容
-F, --follow, --monitor：选择 Monitor 模式
-G, --grow：改变在用阵列的大小或形态
-h, --help：帮助信息，用在以上选项后，则显示该选项信息
--help-options
-V, --version
-v, --verbose：显示细节
-b, --brief：较少的细节。用于 --detail 和 --examine 选项
-f, --force
-c, --config= ：指定配置文件，缺省为 /etc/mdadm/mdadm.conf
-s, --scan：扫描配置文件或 /proc/mdstat以搜寻丢失的信息。配置文件/etc/mdadm/mdadm.conf
create 或 build 使用的选项:
-c, --chunk=:Specify chunk size of kibibytes. 缺省为 64.
--rounding=: Specify rounding factor for linear array (==chunk size)
-l, --level=:设定 raid level.
--create可用:linear, raid0, 0, stripe, raid1,1, mirror, raid4, 4, raid5, 5, raid6, 6, multipath, mp.
--build可用：linear, raid0, 0, stripe.
-p, --parity=：设定 raid5 的奇偶校验规则:eft-asymmetric, left-symmetric, right-asymmetric, right-symmetric, la, ra, ls, rs.缺省为left-symmetric
--layout=:类似于--parity
-n, --raid-devices=:指定阵列中可用 device 数目，这个数目只能由 --grow 修改
-x, --spare-devices=：指定初始阵列的富余device 数目
-z, --size=：组建RAID1/4/5/6后从每个device获取的空间总数
--assume-clean:目前仅用于 --build 选项
-R, --run:阵列中的某一部分出现在其他阵列或文件系统中时，mdadm会确认该阵列。此选项将不作确认。
-f, --force:通常mdadm不允许只用一个device 创建阵列，而且创建raid5时会使用一个device作为missing drive。此选项正相反。
-a, --auto{=no,yes,md,mdp,part,p}{NN}：