Oracle ASM disks Demystfied

Oracle ASM disks Demystfied

When asm disks are created using

oracleasm create disk

oracleasm create disk 'DISK1' /dev/sdb --> This info is stored in header of raw/block device.
ASM Label information is stored in each disk header and can be viewed with kfed read

oracleasm-discover gives the listing of disks along with label names

oracleasm driver scans the devices as mentioned in below files in ORACLEASM_SCANORDER
"dm" --> devices starting with /dev

[oracle@DBRAC01 sysconfig]$ file oracleasm-_dev_oracleasm
oracleasm-_dev_oracleasm: ASCII English text
[oracle@DBRAC01 sysconfig]$ cat oracleasm-_dev_oracleasm
#
# This is a configuration file for automatic loading of the Oracle
# Automatic Storage Management library kernel driver.  It is generated
# By running /etc/init.d/oracleasm configure.  Please use that method
# to modify this file
#

# ORACLEASM_ENABELED: 'true' means to load the driver on boot.
ORACLEASM_ENABLED=true

# ORACLEASM_UID: Default user owning the /dev/oracleasm mount point.
ORACLEASM_UID=oracle

# ORACLEASM_GID: Default group owning the /dev/oracleasm mount point.
ORACLEASM_GID=dba

# ORACLEASM_SCANBOOT: 'true' means scan for ASM disks on boot.
ORACLEASM_SCANBOOT=true

# ORACLEASM_SCANORDER: Matching patterns to order disk scanning
ORACLEASM_SCANORDER="dm"

# ORACLEASM_SCANEXCLUDE: Matching patterns to exclude disks from scan
ORACLEASM_SCANEXCLUDE="sd"

[oracle@DBRAC01 sysconfig]$ cd /dev/oracleasm/disks
[oracle@DBRAC01 disks]$ ls -ltr
total 0
brw-rw---- 1 oracle dba 253, 11 Jul 23 14:09 DISK4
brw-rw---- 1 oracle dba 253, 12 Jul 23 14:09 DISK3
brw-rw---- 1 oracle dba 253, 13 Jul 23 14:09 DISK2
brw-rw---- 1 oracle dba 253, 14 Jul 23 14:09 DISK1
brw-rw---- 1 oracle dba 253, 15 Jul 23 14:09 DISK5
[oracle@DBRAC01 disks]$ df -ha
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                       97G   66G   27G  72% /
proc                     0     0     0   -  /proc
sysfs                    0     0     0   -  /sys
devpts                   0     0     0   -  /dev/pts
/dev/mapper/VolGroup00-LogVol02
                      9.7G  927M  8.3G  10% /var
/dev/mapper/VolGroup00-LogVol03
                      4.9G  1.5G  3.2G  32% /home
/dev/mapper/VolGroup00-LogVol04
                       40G   31G  9.3G  77% /u01
/dev/cciss/c0d0p1     244M   37M  195M  16% /boot
tmpfs                  19G  324M   19G   2% /dev/shm
/dev/mapper/VolGroup00-var--crash_lv
                       40G  177M   40G   1% /var/crash
none                     0     0     0   -  /proc/sys/fs/binfmt_misc
sunrpc                   0     0     0   -  /var/lib/nfs/rpc_pipefs
oracleasmfs              0     0     0   -  /dev/oracleasm
nfsd                     0     0     0   -  /proc/fs/nfsd

[oracle@DBRAC01 disks]$ /usr/sbin/oracleasm-discover
Using ASMLib from /opt/oracle/extapi/64/asm/orcl/1/libasm.so
[ASM Library - Generic Linux, version 2.0.4 (KABI_V2)]
Discovered disk: ORCL:DISK1 [419424957 blocks (214745577984 bytes), maxio 512]
Discovered disk: ORCL:DISK2 [419424957 blocks (214745577984 bytes), maxio 512]
Discovered disk: ORCL:DISK3 [419424957 blocks (214745577984 bytes), maxio 512]
Discovered disk: ORCL:DISK4 [419424957 blocks (214745577984 bytes), maxio 512]
Discovered disk: ORCL:DISK5 [419424957 blocks (214745577984 bytes), maxio 512]
[oracle@DBRAC01 disks]$ kfed read /dev/oracleasm/disks/DISK1
kfbh.endian:                          1 ; 0x000: 0x01
kfbh.hard:                          130 ; 0x001: 0x82
kfbh.type:                            1 ; 0x002: KFBTYP_DISKHEAD
kfbh.datfmt:                          1 ; 0x003: 0x01
kfbh.block.blk:                       0 ; 0x004: blk=0
kfbh.block.obj:              2147483648 ; 0x008: disk=0
kfbh.check:                  3179708272 ; 0x00c: 0xbd867f70
kfbh.fcn.base:                        0 ; 0x010: 0x00000000
kfbh.fcn.wrap:                        0 ; 0x014: 0x00000000
kfbh.spare1:                          0 ; 0x018: 0x00000000
kfbh.spare2:                          0 ; 0x01c: 0x00000000
kfdhdb.driver.provstr:    ORCLDISKDISK1 ; 0x000: length=13
kfdhdb.driver.reserved[0]:   1263749444 ; 0x008: 0x4b534944
kfdhdb.driver.reserved[1]:           49 ; 0x00c: 0x00000031
kfdhdb.driver.reserved[2]:            0 ; 0x010: 0x00000000
kfdhdb.driver.reserved[3]:            0 ; 0x014: 0x00000000
kfdhdb.driver.reserved[4]:            0 ; 0x018: 0x00000000
kfdhdb.driver.reserved[5]:            0 ; 0x01c: 0x00000000
kfdhdb.compat:                186646528 ; 0x020: 0x0b200000
kfdhdb.dsknum:                        0 ; 0x024: 0x0000
kfdhdb.grptyp:                        1 ; 0x026: KFDGTP_EXTERNAL
kfdhdb.hdrsts:                        3 ; 0x027: KFDHDR_MEMBER
kfdhdb.dskname:                   DISK1 ; 0x028: length=5
kfdhdb.grpname:                    DATA ; 0x048: length=4
kfdhdb.fgname:                    DISK1 ; 0x068: length=5
kfdhdb.capname:                         ; 0x088: length=0
kfdhdb.crestmp.hi:             32983095 ; 0x0a8: HOUR=0x17 DAYS=0x1 MNTH=0x2 YEAR=0x7dd
kfdhdb.crestmp.lo:           3869179904 ; 0x0ac: USEC=0x0 MSEC=0x3c0 SECS=0x29 MINS=0x39
kfdhdb.mntstmp.hi:             33001236 ; 0x0b0: HOUR=0x14 DAYS=0x18 MNTH=0x3 YEAR=0x7de
kfdhdb.mntstmp.lo:            575480832 ; 0x0b4: USEC=0x0 MSEC=0x349 SECS=0x24 MINS=0x8
kfdhdb.secsize:                     512 ; 0x0b8: 0x0200
kfdhdb.blksize:                    4096 ; 0x0ba: 0x1000
kfdhdb.ausize:                  1048576 ; 0x0bc: 0x00100000
kfdhdb.mfact:                    113792 ; 0x0c0: 0x0001bc80
kfdhdb.dsksize:                  204797 ; 0x0c4: 0x00031ffd
kfdhdb.pmcnt:                         3 ; 0x0c8: 0x00000003
kfdhdb.fstlocn:                       1 ; 0x0cc: 0x00000001
kfdhdb.altlocn:                       2 ; 0x0d0: 0x00000002
kfdhdb.f1b1locn:                      2 ; 0x0d4: 0x00000002
kfdhdb.redomirrors[0]:                0 ; 0x0d8: 0x0000
kfdhdb.redomirrors[1]:                0 ; 0x0da: 0x0000
kfdhdb.redomirrors[2]:                0 ; 0x0dc: 0x0000
kfdhdb.redomirrors[3]:                0 ; 0x0de: 0x0000
kfdhdb.dbcompat:              168820736 ; 0x0e0: 0x0a100000
kfdhdb.grpstmp.hi:             32983095 ; 0x0e4: HOUR=0x17 DAYS=0x1 MNTH=0x2 YEAR=0x7dd
kfdhdb.grpstmp.lo:           3868648448 ; 0x0e8: USEC=0x0 MSEC=0x1b9 SECS=0x29 MINS=0x39
kfdhdb.vfstart:                      64 ; 0x0ec: 0x00000040
kfdhdb.vfend:                        96 ; 0x0f0: 0x00000060
kfdhdb.spfile:                        0 ; 0x0f4: 0x00000000
kfdhdb.spfflg:                        0 ; 0x0f8: 0x00000000
kfdhdb.ub4spare[0]:                   0 ; 0x0fc: 0x00000000
kfdhdb.ub4spare[1]:                   0 ; 0x100: 0x00000000

Snippet....

If there's any issue with oralceasm driver, the disks are not visible in /dev/oracleasm/disks/..
this can happen when the server/OS kernel is upgraded and oracleasm driver is not updated to the same kernel.

 For this we need to download the correct asm driver for the kernel and update.
Post OS patching, it's recommended to relink binaries...

Troublshooting any errors with asm or any other... look for /var/log/messages

Before making any changes to asm configurations... take a backup of current configuration once the asm 

services are down from /etc/sysconfig/oracleasm*

[oracle@DBRAC01 sysconfig]$ ls -ltr oracle*
lrwxrwxrwx 1 root root  24 Jan 31  2013 oracleasm -> oracleasm-_dev_oracleasm
-rw-r--r-- 1 root root 260 Jan 31  2013 oracleasm-update
-rw-r--r-- 1 root root 778 Feb  4  2013 oracleasm-_dev_oracleasm