vsphere虚拟机下的Linux在线移除硬盘后的报错处理（基于LVM）

1）故障描述（ vmware虚拟机，在线移除一块被设置为pv格式的硬盘后，提示unknown device错误）：说明：/dev/sdb1 /dev/sdc1 /dev/sdd1是组成/dev/vg_test/lvm_test的PV，这里我们模拟/dev/sdd1硬盘被直接断电了

[root@docker mnt]# pvdisplay
/dev/sdd: read failed after 0 of 4096 at 0: Input/output error
/dev/sdd: read failed after 0 of 4096 at 21474770944: Input/output error
/dev/sdd: read failed after 0 of 4096 at 21474828288: Input/output error
/dev/sdd: read failed after 0 of 4096 at 4096: Input/output error
/dev/sdd1: read failed after 0 of 512 at 21467824128: Input/output error
/dev/sdd1: read failed after 0 of 512 at 21467938816: Input/output error
/dev/sdd1: read failed after 0 of 512 at 0: Input/output error
/dev/sdd1: read failed after 0 of 512 at 4096: Input/output error
/dev/sdd1: read failed after 0 of 2048 at 0: Input/output error
Couldn't find device with uuid 166hLZ-2POl-GIsv-ELGh-8YVT-Zcoa-ZcLXzp.
--- Physical volume ---
PV Name               /dev/sdb1
VG Name               vg_test
PV Size               19.99 GiB / not usable 1.43 MiB
Allocatable           yes (but full)
PE Size               4.00 MiB
Total PE              5118
Free PE               0
Allocated PE          5118
PV UUID               fuQdIY-qkJw-fZJB-AauO-Zqpj-91RI-S1GG0X

--- Physical volume ---
PV Name               /dev/sdc1
VG Name               vg_test
PV Size               19.99 GiB / not usable 1.43 MiB
Allocatable           yes
PE Size               4.00 MiB
Total PE              5118
Free PE               5116
Allocated PE          2
PV UUID               TRlpMQ-snM6-u24f-cMHa-oiVt-XJ3w-6fjAbp

--- Physical volume ---
PV Name               unknown device
VG Name               vg_test
PV Size               19.99 GiB / not usable 1.43 MiB
Allocatable           yes
PE Size               4.00 MiB
Total PE              5118
Free PE               5118
Allocated PE          0
PV UUID               166hLZ-2POl-GIsv-ELGh-8YVT-Zcoa-ZcLXzp

2）处理步骤：

[root@docker mnt]# vgreduce --removemissing vg_test
/dev/sdd: read failed after 0 of 4096 at 0: Input/output error
/dev/sdd1: read failed after 0 of 2048 at 0: Input/output error
/dev/sdd: read failed after 0 of 4096 at 21474770944: Input/output error
/dev/sdd: read failed after 0 of 4096 at 21474828288: Input/output error
/dev/sdd: read failed after 0 of 4096 at 4096: Input/output error
/dev/sdd1: read failed after 0 of 512 at 21467824128: Input/output error
/dev/sdd1: read failed after 0 of 512 at 21467938816: Input/output error
/dev/sdd1: read failed after 0 of 512 at 0: Input/output error
/dev/sdd1: read failed after 0 of 512 at 4096: Input/output error
Couldn't find device with uuid 166hLZ-2POl-GIsv-ELGh-8YVT-Zcoa-ZcLXzp.
Wrote out consistent volume group vg_test
[root@docker mnt]# vgchange -a y
/dev/sdd: read failed after 0 of 4096 at 0: Input/output error
/dev/sdd1: read failed after 0 of 2048 at 0: Input/output error
1 logical volume(s) in volume group "vg_test" now active
[root@docker mnt]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3             193G   14G  170G   8% /
tmpfs                 1.9G  228K  1.9G   1% /dev/shm
/dev/sda1             190M   80M  100M  45% /boot
/dev/mapper/vg_test-lvm_test
20G   28M   19G   1% /mnt

测试表明：整个过程都不会影响/mnt中的数据运行

[root@docker mnt]# pvdisplay
/dev/sdd: read failed after 0 of 4096 at 0: Input/output error
/dev/sdd: read failed after 0 of 4096 at 21474770944: Input/output error
/dev/sdd: read failed after 0 of 4096 at 21474828288: Input/output error
/dev/sdd: read failed after 0 of 4096 at 4096: Input/output error
/dev/sdd1: read failed after 0 of 512 at 21467824128: Input/output error
/dev/sdd1: read failed after 0 of 512 at 21467938816: Input/output error
/dev/sdd1: read failed after 0 of 512 at 0: Input/output error
/dev/sdd1: read failed after 0 of 512 at 4096: Input/output error
/dev/sdd1: read failed after 0 of 2048 at 0: Input/output error
--- Physical volume ---
PV Name               /dev/sdb1
VG Name               vg_test
PV Size               19.99 GiB / not usable 1.43 MiB
Allocatable           yes (but full)
PE Size               4.00 MiB
Total PE              5118
Free PE               0
Allocated PE          5118
PV UUID               fuQdIY-qkJw-fZJB-AauO-Zqpj-91RI-S1GG0X

--- Physical volume ---
PV Name               /dev/sdc1
VG Name               vg_test
PV Size               19.99 GiB / not usable 1.43 MiB
Allocatable           yes
PE Size               4.00 MiB
Total PE              5118
Free PE               5116
Allocated PE          2
PV UUID               TRlpMQ-snM6-u24f-cMHa-oiVt-XJ3w-6fjAbp

发现仍旧有I/O的错误出现，找到合适停机窗口，重启了系统，故障得到解决(这里注意重启前检查fstab信息，看下mount -a是否有报错）。

[root@docker ~]# pvdisplay
--- Physical volume ---
PV Name               /dev/sdb1
VG Name               vg_test
PV Size               19.99 GiB / not usable 1.43 MiB
Allocatable           yes (but full)
PE Size               4.00 MiB
Total PE              5118
Free PE               0
Allocated PE          5118
PV UUID               fuQdIY-qkJw-fZJB-AauO-Zqpj-91RI-S1GG0X

--- Physical volume ---
PV Name               /dev/sdc1
VG Name               vg_test
PV Size               19.99 GiB / not usable 1.43 MiB
Allocatable           yes
PE Size               4.00 MiB
Total PE              5118
Free PE               5116
Allocated PE          2
PV UUID               TRlpMQ-snM6-u24f-cMHa-oiVt-XJ3w-6fjAbp

这样，系统就已经没有报错信息了。