AIX 系统故障之--扩展文件系统故障

AIX 系统故障之--扩展文件系统故障

系统环境： 操作系统： AIX5300-08 Oracle： Oracle 10gR2（64）故障现象：
smit chfs 扩展 “ /" 时，提示找不到”device “，不能扩展；在rootvg 扩展pv 后，后来pv 被破坏，导致ODM出错

1、查看rootvg
[root@aix216 /]#lsvg -p rootvg
rootvg:
PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION
hdisk0 active 546 161 107..00..00..12..42
0516-304 : Unable to find device id 0000c908116d1325 in the Device
Configuration Database.
0000c908116d1325 missing 320 320 64..64..64..64..64
[root@aix216 /]#

----尝试解决
[root@aix216 /]#redefinevg -d hdisk0 rootvg
[root@aix216 /]#lsvg -p rootvg
rootvg:
PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION
hdisk0 active 546 161 107..00..00..12..42
0516-304 : Unable to find device id 0000c908116d1325 in the Device
Configuration Database.
0000c908116d1325 missing 320 320 64..64..64..64..64

[root@aix216 /]#lspv
hdisk0 0027713e1b5646e5 rootvg active

[root@aix216 /]#reducevg
0516-606 reducevg: Volume group name not entered.
Usage: reducevg [-d] [-f] VGname PVname...
Reduces volume group size by removing a physical volume.

2、从rootvg 中删除hdisk0
------ reducevg 删除pv 后，hd5 被删除
[root@aix216 /]#reducevg rootvg 0027713e1b5646e5
0516-016 ldeletepv: Cannot delete physical volume with allocated
partitions. Use either migratepv to move the partitions or
reducevg with the -d option to delete the partitions.
0516-884 reducevg: Unable to remove physical volume 0027713e1b5646e5.
[root@aix216 /]#reducevg rootvg 0027713e1b5646e5 -d
0516-306 getlvodm: Unable to find physical volume -d in the Device
Configuration Database.
0516-016 ldeletepv: Cannot delete physical volume with allocated
partitions. Use either migratepv to move the partitions or
reducevg with the -d option to delete the partitions.
0516-884 reducevg: Unable to remove physical volume 0027713e1b5646e5.
0516-306 getlvodm: Unable to find physical volume -d in the Device
Configuration Database.
0516-884 reducevg: Unable to remove physical volume -d.
[root@aix216 /]#reducevg -d rootvg 0027713e1b5646e5
0516-914 rmlv: Warning, all data belonging to logical volume
hd5 on physical volume 0027713e1b5646e5 will be destroyed.
rmlv: Do you wish to continue? y(es) n(o)?
y
0516-1246 rmlv: If hd5 is the boot logical volume, please run 'chpv -c <diskname>'
as root user to clear the boot record and avoid a potential boot
off an old boot image that may reside on the disk from which this
logical volume is moved/removed.
rmlv: Logical volume hd5 is removed.
0516-1734 rmlv: Warning, savebase failed. Please manually run 'savebase' before rebooting.
0516-914 rmlv: Warning, all data belonging to logical volume
fwdump on physical volume 0027713e1b5646e5 will be destroyed.
rmlv: Do you wish to continue? y(es) n(o)?
yes
0516-1008 rmlv: Logical volume fwdump must be closed. If the logical volume
contains a filesystem, the umount command will close the LV device.
0516-1734 rmlv: Warning, savebase failed. Please manually run 'savebase' before rebooting.
0516-884 reducevg: Unable to remove physical volume 0027713e1b5646e5.
0516-1734 reducevg: Warning, savebase failed. Please manually run 'savebase' before rebooting.
[root@aix216 /]#savebase
[root@aix216 /]#

[root@aix216 /tmp]#lsvg -l rootvg
rootvg:
LV NAME TYPE LPs PPs PVs LV STATE MOUNT POINT
hd6 paging 32 32 1 open/syncd N/A
hd8 jfs2log 1 1 1 open/syncd N/A
hd4 jfs2 136 136 1 open/syncd /
hd2 jfs2 24 24 1 open/syncd /usr
hd9var jfs2 12 12 1 open/syncd /var
hd3 jfs2 16 16 1 open/syncd /tmp
hd1 jfs2 16 16 1 open/syncd /home
hd10opt jfs2 24 24 1 open/syncd /opt
fwdump jfs2 1 1 1 open/syncd /var/adm/ras/platform
fs001 jfs2 120 120 1 open/syncd /u01
loglv00 jfslog 1 1 1 open/syncd N/A
lv00 jfs 1 1 1 open/syncd /var/adm/csd

3、重新建立 hd5
重建hd5 ，出错
[root@aix216 /tmp]#mklv -y hd5 -t boot -a e rootvg 1
mklv -y hd5 -t boot -a e rootvg 1
0516-304 lquerypv: Unable to find device id 0000c908116d13250000000000000000 in the Device
Configuration Database.
0516-634 mklv: /tmp directory does not have enough space,
delete some files and try again.
0516-1734 mklv: Warning, savebase failed. Please manually run 'savebase' before rebooting.

[root@aix216 /tmp]#bosboot -ad /dev/ipldevice
0516-306 lslv: Unable to find hd5 in the Device
Configuration Database.

0301-168 bosboot: The current boot logical volume, /dev/hd5,
does not exist on /dev/hdisk0.
----- 提示：/tmp空间不足，实际/tmp空间很大，尝试删除tmp 下所有的文件

[root@aix216 /tmp]#df -m
Filesystem MB blocks Free %Used Iused %Iused Mounted on
/dev/hd4 7168.00 6032.57 16% 4899 1% /
/dev/hd2 3072.00 1207.03 61% 46571 15% /usr
/dev/hd9var 1536.00 696.15 55% 1170 1% /var
/dev/hd3 2048.00 2039.82 1% 25 1% /tmp
/dev/fwdump 128.00 127.66 1% 4 1% /var/adm/ras/platform
/dev/hd1 2048.00 2003.45 3% 233 1% /home
/proc - - - - - /proc
/dev/hd10opt 1024.00 749.93 27% 4799 3% /opt
/dev/lv00 128.00 123.94 4% 18 1% /var/adm/csd
/dev/fs001 15360.00 15357.19 1% 7 1% /u01

[root@aix216 /tmp]# rm -rf /tmp/*

4、再次删除rootvg 下的 hdisk0

[root@aix216 /tmp]#lsvg -p rootvg
rootvg:
PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION
hdisk0 active 546 162 108..00..00..12..42
0516-304 : Unable to find device id 0000c908116d1325 in the Device
Configuration Database.
0000c908116d1325 missing 320 320 64..64..64..64..64

[root@aix216 /tmp]#reducevg -d rootvg "0000c908116d1325"
0516-304 putlvodm: Unable to find device id 0000c908116d13250000000000000000 in the Device
Configuration Database.
0516-896 reducevg: Warning, cannot remove physical volume 0000c908116d1325 from
Device Configuration Database.
0516-1734 reducevg: Warning, savebase failed. Please manually run 'savebase' before rebooting.
[root@aix216 /tmp]#synclvodm -v rootvg
synclvodm: Physical volume data updated.
synclvodm: Logical volume hd6 updated.
synclvodm: Logical volume hd8 updated.
synclvodm: Logical volume hd4 updated.
synclvodm: Logical volume hd2 updated.
synclvodm: Logical volume hd9var updated.
synclvodm: Logical volume hd3 updated.
synclvodm: Logical volume hd1 updated.
synclvodm: Logical volume hd10opt updated.
synclvodm: Logical volume fwdump updated.
synclvodm: Logical volume fs001 updated.
synclvodm: Logical volume loglv00 updated.
synclvodm: Logical volume lv00 updated.

[root@aix216 /tmp]#lsvg -l rootvg
rootvg:
LV NAME TYPE LPs PPs PVs LV STATE MOUNT POINT
hd6 paging 32 32 1 open/syncd N/A
hd8 jfs2log 1 1 1 open/syncd N/A
hd4 jfs2 136 136 1 open/syncd /
hd2 jfs2 24 24 1 open/syncd /usr
hd9var jfs2 12 12 1 open/syncd /var
hd3 jfs2 16 16 1 open/syncd /tmp
hd1 jfs2 16 16 1 open/syncd /home
hd10opt jfs2 24 24 1 open/syncd /opt
fwdump jfs2 1 1 1 open/syncd /var/adm/ras/platform
fs001 jfs2 120 120 1 open/syncd /u01
loglv00 jfslog 1 1 1 open/syncd N/A
lv00 jfs 1 1 1 open/syncd /var/adm/csd

5、重新建立 hd5
继续重建hd5 ，重建成功

[root@aix216 /tmp]#mklv -y hd5 -t boot -a e rootvg 1
mklv -y hd5 -t boot -a e rootvg 1
hd5
[root@aix216 /tmp]#bosboot -ad /dev/ipldevice

bosboot: Boot image is 37908 512 byte blocks.
[root@aix216 /tmp]#

6、故障解决

查看rootvg pv ，问题解决

[root@aix216 /tmp]#lsvg -p rootvg
rootvg:
PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION
hdisk0 active 546 161 107..00..00..12..42
[root@aix216 /tmp]#

[root@aix216 /tmp]#lsvg -l rootvg
rootvg:
LV NAME TYPE LPs PPs PVs LV STATE MOUNT POINT
hd5 boot 1 1 1 closed/syncd N/A
hd6 paging 32 32 1 open/syncd N/A
hd8 jfs2log 1 1 1 open/syncd N/A
hd4 jfs2 136 136 1 open/syncd /
hd2 jfs2 24 24 1 open/syncd /usr
hd9var jfs2 12 12 1 open/syncd /var
hd3 jfs2 16 16 1 open/syncd /tmp
hd1 jfs2 16 16 1 open/syncd /home
hd10opt jfs2 24 24 1 open/syncd /opt
fwdump jfs2 1 1 1 open/syncd /var/adm/ras/platform
fs001 jfs2 120 120 1 open/syncd /u01
loglv00 jfslog 1 1 1 open/syncd N/A
lv00 jfs 1 1 1 open/syncd /var/adm/csd
[root@aix216 /tmp]#

本文出自 “天涯客的blog” 博客，请务必保留此出处http://tiany.blog.51cto.com/513694/1389549