Friday, December 18, 2015

ASM DISK HEADER CORRUPTION & REPAIR





ASM DISK HEADER CORRUPTION & REPAIR

In this article I will demonstrate how to corrupt the ASM DISK header and to repair it using kfed utility.
ASM metadata is stored in disk groups and each ASM disk has ASM metadata. ASM metadata block size is 4 KB (kfdhdb.blksize=4096).Typically first Allocation unit on every ASM disk will always have the disk header(block0).

Below are the detailed steps –

Step 1 – Backup the ASM metadata (first 2MB of disk – VOL1 )

  $ dd if=/dev/oracleasm/disks/VOL1 of=/tmp/asmvol_bkp.dd bs=1M count=2

Step 2 – Corrupt the Disk header Metadata
·         dd zero out the disk header metadata

$ dd if=/dev/zero of=/dev/oracleasm/disks/VOL1 bs=4096 count=1
   1+0 records in
   1+0 records out
   4096 bytes (4.1 kB) copied, 3.8e-05 seconds, 108 MB/s

Step 3 - Check for damage to ASM disk header metadata ( Allocation unit # 0 , block # 0)

SQL> select DISK_NUMBER,HEADER_STATUS,substr(PATH,1,20),label from v$asm_disk;

DISK_NUMBER   HEADER_STATUS      PATH              LABEL
----------- --------------------------------------------------------------------------
          0    CANDIDATE                          ORCL:VOL1   VOL1
          1    MEMBER                                 ORCL:VOL2     VOL2
          0    MEMBER                                 ORCL:VOL3     VOL3

$ asmcmd lsdsk –p

Group_Num  Disk_Num    Incarn  Mount_Stat  Header_Stat  Mode_Stat  State     Path
    1         0  3607148357  CACHED   CANDIDATE ONLINE     NORMAL  ORCL:VOL1
    1         1  3607148358  CACHED    MEMBER       ONLINE     NORMAL  ORCL:VOL2
    2         0  3607148368  CACHED    MEMBER       ONLINE     NORMAL  ORCL:VOL3

$ kfed read /dev/oracleasm/disks/VOL1 aun=0 blkn=0

kfbh.endian:                          0 ; 0x000: 0x00
kfbh.hard:                            0 ; 0x001: 0x00
kfbh.type:                            0 ; 0x002: KFBTYP_INVALID
kfbh.datfmt:                          0 ; 0x003: 0x00
kfbh.block.blk:                       0 ; 0x004: T=0 NUMB=0x0
kfbh.block.obj:                       0 ; 0x008: TYPE=0x0 NUMB=0x0
kfbh.check:                           0 ; 0x00c: 0x00000000
kfbh.fcn.base:                        0 ; 0x010: 0x00000000
kfbh.fcn.wrap:                        0 ; 0x014: 0x00000000
kfbh.spare1:                          0 ; 0x018: 0x00000000
kfbh.spare2:                          0 ; 0x01c: 0x00000000
2B6BCD150400 00000000 00000000 00000000 00000000  [................]
  Repeat 255 times
KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0]

Note-  Here If you see kfbh.type=KFBTYP_INVALID , then it indicates that ASM disk header is corrupted.Typically Valid ASM disk should have kfbh.type=KFBTYP_DISKHEAD (ASM disk header).

Step 4 - Check available Copy Of ASM DISK HEADER in disk.
·          ASM contains the copy of ASM disk header in (Allocation unit# 1, Block# 254) of every ASM disk.

$ kfed read /dev/oracleasm/disks/VOL1 aun=1 blkn=254 |grep KFBTYP
  kfbh.type:                            1 ; 0x002: KFBTYP_DISKHEAD

 Note - Here we can see that ASM disk “VOL1” has valid copy of ASM disk header.

Step 5 -  Repair the ASM Disk Header

   $ kfed  repair /dev/oracleasm/disks/VOL1

Step 6 –  Verify the ASM Disk Header Metadata

$ kfed read /dev/oracleasm/disks/VOL1 aun=0 blkn=0 | grep KFBTYP
  kfbh.type:                            1 ; 0x002: KFBTYP_DISKHEAD

SQL> select DISK_NUMBER,HEADER_STATUS,substr(PATH,1,20),label from v$asm_disk;

DISK_NUMBER   HEADER_STATUS     PATH                LABEL
----------- ----------------------------------------------------------------------------
          0                    MEMBER                ORCL:VOL1  VOL1
          1                    MEMBER                  ORCL:VOL2    VOL2
          0                    MEMBER                  ORCL:VOL3    VOL3

$ asmcmd lsdsk -p
Group_Num  Disk_Num      Incarn  Mount_Stat  Header_Stat  Mode_Stat  State   Path
  1     0  3607148357  CACHED      MEMBER    ONLINE    NORMAL  ORCL:VOL1
  1     1  3607148358  CACHED      MEMBER       ONLINE     NORMAL  ORCL:VOL2
  2     0  3607148368  CACHED      MEMBER       ONLINE     NORMAL  ORCL:VOL3

1 comment: