we have two node RAC ol6-112-rac1 / ol6-112-rac2. Which is currently having +DATA DG of 20G. We want to add more disk to this DG. For that we need to first provision the disk which can be later added.
shutdown both the RAC VM ol6-112-rac1 /ol6-112-rac2
create disk asm11,asm12,asm13,asm14 from CDM prompt
C:\Users\SHALABH>cd C:\Users\SHALABH
C:\Users\SHALABH>"c:\Program Files\Oracle\VirtualBox\VBoxManage" createhd --filename asm11.vdi --size 5120 --format VDI --variant Fixed
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Medium created. UUID: 11e82b6f-dc0c-4eb8-af80-0f8c247bf8fb
C:\Users\SHALABH>"c:\Program Files\Oracle\VirtualBox\VBoxManage" createhd --filename asm12.vdi --size 5120 --format VDI --variant Fixed
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Medium created. UUID: 7ed7391d-733f-4788-90b6-f53893090fcc
C:\Users\SHALABH>"c:\Program Files\Oracle\VirtualBox\VBoxManage" createhd --filename asm13.vdi --size 5120 --format VDI --variant Fixed
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Medium created. UUID: 5671ce4a-1a0e-4ee8-bb5f-127b4890ca2d
C:\Users\SHALABH>"c:\Program Files\Oracle\VirtualBox\VBoxManage" createhd --filename asm14.vdi --size 5120 --format VDI --variant Fixed
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Medium created. UUID: 5485dadd-19cf-41db-9a97-b2d7973ce925attach these disks to node 1. notice the port used is 5,6,7,8 because we already have asm1,asm2,asm3,asm4 disk on port 1,2,3,4 respectively
C:\Users\SHALABH>"c:\Program Files\Oracle\VirtualBox\VBoxManage" storageattach ol6-112-rac1 --storagectl "SATA" --port 5 --device 0 --type hdd --medium asm11.vdi --mtype shareable
C:\Users\SHALABH>"c:\Program Files\Oracle\VirtualBox\VBoxManage" storageattach ol6-112-rac1 --storagectl "SATA" --port 6 --device 0 --type hdd --medium asm12.vdi --mtype shareable
C:\Users\SHALABH>"c:\Program Files\Oracle\VirtualBox\VBoxManage" storageattach ol6-112-rac1 --storagectl "SATA" --port 7 --device 0 --type hdd --medium asm13.vdi --mtype shareable
C:\Users\SHALABH>"c:\Program Files\Oracle\VirtualBox\VBoxManage" storageattach ol6-112-rac1 --storagectl "SATA" --port 8 --device 0 --type hdd --medium asm14.vdi --mtype shareable
make them shareable
C:\Users\SHALABH>"c:\Program Files\Oracle\VirtualBox\VBoxManage" modifyhd asm11.vdi --type shareable
C:\Users\SHALABH>"c:\Program Files\Oracle\VirtualBox\VBoxManage" modifyhd asm12.vdi --type shareable
C:\Users\SHALABH>"c:\Program Files\Oracle\VirtualBox\VBoxManage" modifyhd asm13.vdi --type shareable
C:\Users\SHALABH>"c:\Program Files\Oracle\VirtualBox\VBoxManage" modifyhd asm14.vdi --type shareablestart the first node ol6-112-rac1. you can see sdf, sdg, sdh, sdi under /dev. These are the newly created disks
[root@ol6-112-rac1 dev]# cd /dev
[root@ol6-112-rac1 dev]# ls sd*
sda sda1 sda2 sdb sdc sdd sde sdf sdg sdh sdi
partition them
[root@ol6-112-rac1 dev]# fdisk /dev/sdf
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x23662a7f.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-652, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-652, default 652):
Using default value 652
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@ol6-112-rac1 dev]# fdisk /dev/sdg
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x4246e6e1.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-652, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-652, default 652):
Using default value 652
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@ol6-112-rac1 dev]# fdisk /dev/sdh
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x6d6e0d3f.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-652, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-652, default 652):
Using default value 652
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@ol6-112-rac1 dev]# fdisk /dev/sdi
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xf6af1166.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-652, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-652, default 652):
Using default value 652
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.verify the newly created partitions
[root@ol6-112-rac1 dev]# ls sd*
sda sda1 sda2 sdb sdc sdd sde sdf sdf1 sdg sdg1 sdh sdh1 sdi sdi1Make a note of SCSI id
[root@ol6-112-rac1 dev]# cat /etc/scsi_id.config
options=-g
[root@ol6-112-rac1 dev]# /sbin/scsi_id -g -u -d /dev/sdf
1ATA_VBOX_HARDDISK_VB11e82b6f-fbf87b24
[root@ol6-112-rac1 dev]# /sbin/scsi_id -g -u -d /dev/sdg
1ATA_VBOX_HARDDISK_VB7ed7391d-cc0f0993
[root@ol6-112-rac1 dev]# /sbin/scsi_id -g -u -d /dev/sdh
1ATA_VBOX_HARDDISK_VB5671ce4a-2dca9048
[root@ol6-112-rac1 dev]# /sbin/scsi_id -g -u -d /dev/sdi
1ATA_VBOX_HARDDISK_VB5485dadd-25e93c97Using these values, edit the “/etc/udev/rules.d/99-oracle-asmdevices.rules” file adding the new 4 entries. All parameters for a single entry must be on the same line.
current values
[root@ol6-112-rac1 dev]# cat /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB8ca20b54-f2f0437e", NAME="asm-disk1", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB8fcd94fe-4aa5fe87", NAME="asm-disk2", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB399e289c-23d1bbd2", NAME="asm-disk3", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB62be42c3-7e28d1d6", NAME="asm-disk4", OWNER="oracle", GROUP="dba", MODE="0660"
after adding new entries
[root@ol6-112-rac1 dev]# vi /etc/udev/rules.d/99-oracle-asmdevices.rules
[root@ol6-112-rac1 dev]# cat /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB8ca20b54-f2f0437e", NAME="asm-disk1", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB8fcd94fe-4aa5fe87", NAME="asm-disk2", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB399e289c-23d1bbd2", NAME="asm-disk3", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB62be42c3-7e28d1d6", NAME="asm-disk4", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB11e82b6f-fbf87b24", NAME="asm-disk11", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB7ed7391d-cc0f0993", NAME="asm-disk12", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB5671ce4a-2dca9048", NAME="asm-disk13", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB5485dadd-25e93c97", NAME="asm-disk14", OWNER="oracle", GROUP="dba", MODE="0660"
Load updated block device partition tables.
[root@ol6-112-rac1 dev]# /sbin/partprobe /dev/sdf1
[root@ol6-112-rac1 dev]# /sbin/partprobe /dev/sdg1
[root@ol6-112-rac1 dev]# /sbin/partprobe /dev/sdh1
[root@ol6-112-rac1 dev]# /sbin/partprobe /dev/sdi1Test the rules are working as expected.
[root@ol6-112-rac1 dev]# /sbin/udevadm test /block/sdf/sdf1
[root@ol6-112-rac1 dev]# /sbin/udevadm test /block/sdg/sdg1
[root@ol6-112-rac1 dev]# /sbin/udevadm test /block/sdh/sdh1
[root@ol6-112-rac1 dev]# /sbin/udevadm test /block/sdi/sdi1Reload the UDEV rules and start UDEV.
[root@ol6-112-rac1 dev]# /sbin/udevadm control --reload-rules
[root@ol6-112-rac1 dev]# /sbin/start_udev
Starting udev: [ OK ]The disks should now be visible and have the correct ownership using the following command.
[root@ol6-112-rac1 dev]# ls -al /dev/asm*
brw-rw----. 1 oracle dba 8, 17 Jan 5 11:11 /dev/asm-disk1
brw-rw----. 1 oracle dba 8, 81 Jan 5 11:11 /dev/asm-disk11
brw-rw----. 1 oracle dba 8, 97 Jan 5 11:11 /dev/asm-disk12
brw-rw----. 1 oracle dba 8, 113 Jan 5 11:11 /dev/asm-disk13
brw-rw----. 1 oracle dba 8, 129 Jan 5 11:11 /dev/asm-disk14
brw-rw----. 1 oracle dba 8, 33 Jan 5 11:11 /dev/asm-disk2
brw-rw----. 1 oracle dba 8, 49 Jan 5 11:11 /dev/asm-disk3
brw-rw----. 1 oracle dba 8, 65 Jan 5 11:11 /dev/asm-disk4
[root@ol6-112-rac1 dev]# ls sd*
sda sda1 sda2 sdb sdc sdd sde sdf sdg sdh sdi
attach the disk to second node ol6-112-rac2
C:\Users\SHALABH>"c:\Program Files\Oracle\VirtualBox\VBoxManage" storageattach ol6-112-rac2 --storagectl "SATA" --port 5 --device 0 --type hdd --medium asm11.vdi --mtype shareable
C:\Users\SHALABH>"c:\Program Files\Oracle\VirtualBox\VBoxManage" storageattach ol6-112-rac2 --storagectl "SATA" --port 6 --device 0 --type hdd --medium asm12.vdi --mtype shareable
C:\Users\SHALABH>"c:\Program Files\Oracle\VirtualBox\VBoxManage" storageattach ol6-112-rac2 --storagectl "SATA" --port 7 --device 0 --type hdd --medium asm13.vdi --mtype shareable
C:\Users\SHALABH>"c:\Program Files\Oracle\VirtualBox\VBoxManage" storageattach ol6-112-rac2 --storagectl "SATA" --port 8 --device 0 --type hdd --medium asm14.vdi --mtype shareablestart ol6-112-rac2. New partitions are visible
[root@ol6-112-rac2 ~]# cd /dev
[root@ol6-112-rac2 dev]# ls sd*
sda sda1 sda2 sdb sdc sdd sde sdf sdf1 sdg sdg1 sdh sdh1 sdi sdi1but disk are not yet visible
[root@ol6-112-rac2 dev]# ls -al /dev/asm*
brw-rw----. 1 oracle dba 8, 17 Jan 5 11:18 /dev/asm-disk1
brw-rw----. 1 oracle dba 8, 33 Jan 5 11:18 /dev/asm-disk2
brw-rw----. 1 oracle dba 8, 49 Jan 5 11:18 /dev/asm-disk3
brw-rw----. 1 oracle dba 8, 65 Jan 5 11:18 /dev/asm-disk4modify /etc/udev/rules.d/99-oracle-asmdevices.rules and make it same as on ol6-112-rac1
[root@ol6-112-rac2 dev]# cat /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB8ca20b54-f2f0437e", NAME="asm-disk1", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB8fcd94fe-4aa5fe87", NAME="asm-disk2", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB399e289c-23d1bbd2", NAME="asm-disk3", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB62be42c3-7e28d1d6", NAME="asm-disk4", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB11e82b6f-fbf87b24", NAME="asm-disk11", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB7ed7391d-cc0f0993", NAME="asm-disk12", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB5671ce4a-2dca9048", NAME="asm-disk13", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB5485dadd-25e93c97", NAME="asm-disk14", OWNER="oracle", GROUP="dba", MODE="0660"
Load updated block device partition tables.
[root@ol6-112-rac2 dev]# /sbin/partprobe /dev/sdf1
[root@ol6-112-rac2 dev]# /sbin/partprobe /dev/sdg1
[root@ol6-112-rac2 dev]# /sbin/partprobe /dev/sdh1
[root@ol6-112-rac2 dev]# /sbin/partprobe /dev/sdi1
[root@ol6-112-rac2 dev]# ls sd*
sda sda1 sda2 sdb sdc sdd sde sdf sdf1 sdg sdg1 sdh sdh1 sdi sdi1
test the rules working as expected
[root@ol6-112-rac2 dev]# /sbin/udevadm test /block/sdf/sdf1
[root@ol6-112-rac2 dev]# /sbin/udevadm test /block/sdg/sdg1
[root@ol6-112-rac2 dev]# /sbin/udevadm test /block/sdh/sdh1
[root@ol6-112-rac2 dev]# /sbin/udevadm test /block/sdi/sdi1Reload the UDEV rules and start UDEV.
[root@ol6-112-rac2 dev]# /sbin/udevadm control --reload-rules
[root@ol6-112-rac2 dev]# /sbin/start_udev
Starting udev: [ OK ]The disks should now be visible and have the correct ownership using the following command.
[root@ol6-112-rac2 dev]# ls -al /dev/asm*
brw-rw----. 1 oracle dba 8, 17 Jan 5 11:21 /dev/asm-disk1
brw-rw----. 1 oracle dba 8, 81 Jan 5 11:21 /dev/asm-disk11
brw-rw----. 1 oracle dba 8, 97 Jan 5 11:21 /dev/asm-disk12
brw-rw----. 1 oracle dba 8, 113 Jan 5 11:21 /dev/asm-disk13
brw-rw----. 1 oracle dba 8, 129 Jan 5 11:21 /dev/asm-disk14
brw-rw----. 1 oracle dba 8, 33 Jan 5 11:21 /dev/asm-disk2
brw-rw----. 1 oracle dba 8, 49 Jan 5 11:21 /dev/asm-disk3
brw-rw----. 1 oracle dba 8, 65 Jan 5 11:21 /dev/asm-disk4veify asmca is able to see them
we can see disk are visible
DBATracker 