This is ninth part of my RAC lab series. As we have first rac node even with assigned storage, I will show you how to clone it to build second RAC node. This step by step procedure can also be used for creation of even more nodes 3,4,5 whatever – this is completely your choice.
RAC lab Part 1 – Installing the Ubuntu 16.04 desktop
RAC lab Part 2 – Virtualbox installation and configuration
RAC lab Part 3 – VMs configuration
RAC lab Part 4 – Installing the Openfiler software
RAC lab Part 5 – Installing OEL 7.x on VM
RAC lab Part 6 – OEL 7.x configuration for Oracle 12c GI & RAC DB installation
RAC lab Part 7 – Openfilers configuration and cloning
RAC lab Part 8 – OEL 7.x prepare storage
RAC lab Part 9 – Clone first RAC node as 2nd node and prepare config.
RAC lab Part 10 – Installing Grid Infrastructure
RAC lab Part 11 – Installing Database Software
RAC lab Part 12 – Creating a Container Database
Clone oel7rac1n1 as oel7rac1n2
- First of all stop oel7rac1n1 before using it as the clone source.
- Clone oel7rac1n1
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage clonevm oel7rac1n1 --name oel7rac1n2 --groups "/RacLab1" --basefolder /vbox-repo1/metadata --register 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Machine has been successfully cloned as "oel7rac1n2"
- Modify oel7rac1n2’s disk path and name. Modify snapshots directory
# modify disk path vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifymedium /vbox-repo1/metadata/RacLab1/oel7rac1n2/oel7rac1n2-disk1.vdi --move /vbox-repo1/disks/oel7rac1n2_localOSdisk1.vdi 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Move medium with UUID 4bd7e50f-0824-49b9-bf32-3b8f1fde9888 finished # check result vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage showvminfo oel7rac1n2|grep SATA Storage Controller Name (1): SATA Controller SATA Controller (1, 0): /vbox-repo1/disks/oel7rac1n2_localOSdisk1.vdi (UUID: 4bd7e50f-0824-49b9-bf32-3b8f1fde9888) # modify snapshot directory vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel7rac1n2" --snapshotfolder /vbox-repo1/snapshots/
- Start new cloned VM
vboxuser1@macieksrv.dba24.pl /opt/vboxshared/Oracle12.1_install $ vboxmanage startvm oel7rac1n2 Waiting for VM "oel7rac1n2" to power on... VM "oel7rac1n2" has been successfully started.
Network configuration on oel7rac1n2
- Check macs of the machine for bridged interface in order to set fixed IP on dhcp server
vboxuser1@macieksrv.dba24.pl /opt/vboxshared/Oracle12.1_install $ vboxmanage showvminfo oel7rac1n2|grep MAC|grep Bridge NIC 1: MAC: 08002706F6B1, Attachment: Bridged Interface 'wlx98ded00b5b05', Cable connected: on, Trace: off (file: none), Type: 82540EM, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: deny, Bandwidth group: none
- Login to the server via the VM console and bounce bridged interface to get IP from dhcp (you should have already configured dhcp for its mac)
maciek@oel7rac1n1.dba24.pl ~ $ sudo nmcli con down bridged
maciek@oel7rac1n1.dba24.pl ~ $ sudo nmcli con up bridged
# check if IP arrived
maciek@oel7rac1n1.dba24.pl ~ $ sudo ip addr
1: lo: <loopback,up,lower_up> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
2: enp0s3: <broadcast,multicast,up,lower_up> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:06:f6:b1 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.23/24 brd 192.168.1.255 scope global enp0s3
- Now we can login to the server’s IP 192.168.1.23 using ssh.
- Check configured networks list
maciek@oel7rac1n1.dba24.pl ~ $ sudo nmcli con show NAME UUID TYPE DEVICE apriv1-internal c23f2a48-d4e7-4c4d-8c67-170ef5ae9118 802-3-ethernet enp0s17 apriv2-internal 87d2f222-d768-4fd6-be6d-96a10d198922 802-3-ethernet enp0s16 bridged 5851f1af-fb84-4bc1-bc3f-394ab7524714 802-3-ethernet enp0s3 priv1-internal 0acddf7c-be94-49ec-8d05-2fc6bf10d25a 802-3-ethernet enp0s9 priv2-internal 518cd49c-51fb-482f-9bf1-19af3c462a6d 802-3-ethernet enp0s10 storage-internal 4d92bf55-a505-405d-b4a2-fd36cfbef270 802-3-ethernet enp0s8
- Modify IPs for each network accordingly
maciek@oel7rac1n1.dba24.pl ~ $ sudo nmcli con mod storage-internal ipv4.addresses "192.168.10.23/24" maciek@oel7rac1n1.dba24.pl ~ $ sudo nmcli con mod priv1-internal ipv4.addresses "192.168.20.23/24" maciek@oel7rac1n1.dba24.pl ~ $ sudo nmcli con mod priv2-internal ipv4.addresses "192.168.21.23/24" maciek@oel7rac1n1.dba24.pl ~ $ sudo nmcli con mod apriv1-internal ipv4.addresses "192.168.30.23/24" maciek@oel7rac1n1.dba24.pl ~ $ sudo nmcli con mod apriv2-internal ipv4.addresses "192.168.31.23/24"
- Restart network and check IPs
maciek@oel7rac1n1.dba24.pl ~ $ sudo systemctl restart network
maciek@oel7rac1n1.dba24.pl ~ $ ip addr
1: lo: <loopback,up,lower_up> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
2: enp0s3: <broadcast,multicast,up,lower_up> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:06:f6:b1 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.23/24 brd 192.168.1.255 scope global enp0s3
3: enp0s8: <broadcast,multicast,up,lower_up> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:68:13:a5 brd ff:ff:ff:ff:ff:ff
inet 192.168.10.23/24 brd 192.168.10.255 scope global enp0s8
4: enp0s9: <broadcast,multicast,up,lower_up> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:15:01:c3 brd ff:ff:ff:ff:ff:ff
inet 192.168.20.23/24 brd 192.168.20.255 scope global enp0s9
5: enp0s10: <broadcast,multicast,up,lower_up> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:12:17:c5 brd ff:ff:ff:ff:ff:ff
inet 192.168.21.23/24 brd 192.168.21.255 scope global enp0s10
6: enp0s16: <broadcast,multicast,up,lower_up> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:7e:95:1c brd ff:ff:ff:ff:ff:ff
inet 192.168.30.23/24 brd 192.168.30.255 scope global enp0s16
7: enp0s17: <broadcast,multicast,up,lower_up> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:65:c7:36 brd ff:ff:ff:ff:ff:ff
inet 192.168.31.23/24 brd 192.168.31.255 scope global enp0s17
- Change hostname
maciek@oel7rac1n1.dba24.pl ~ $ sudo hostnamectl set-hostname oel7rac1n2
- Reboot this vm to check if all is ok after it is up again
- If all is ok start first node as it is safe now (all IPs have been changed properly)
vboxuser1@macieksrv.dba24.pl / $ vboxmanage startvm oel7rac1n1 Waiting for VM "oel7rac1n1" to power on... VM "oel7rac1n1" has been successfully started.
Configure storage on oel7rac1n2
- Because oel7rac1n2 is an exact clone of oel7rac1n1 its iscsi initiator name is the same. We have to change it in order to be able to login to iscsci target together with oel7rac1n1.
# change maciek@oel7rac1n2.dba24.pl ~ $ cat /etc/iscsi/initiatorname.iscsi InitiatorName=iqn.1988-12.com.oracle:e942b9ba36f # into maciek@oel7rac1n2.dba24.pl ~ $ cat /etc/iscsi/initiatorname.iscsi InitiatorName=iqn.1988-12.com.oracle:e942b9ba36a
- Reboot VM
- Check if oel7rac1n2 is able to access all the disks we need
maciek@oel7rac1n2.dba24.pl ~ $ lsscsi [1:0:0:0] cd/dvd VBOX CD-ROM 1.0 /dev/sr0 [3:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sda [7:0:0:0] disk OPNFILER VIRTUAL-DISK 0 /dev/sdb [7:0:0:1] disk OPNFILER VIRTUAL-DISK 0 /dev/sdc [7:0:0:2] disk OPNFILER VIRTUAL-DISK 0 /dev/sdd [7:0:0:3] disk OPNFILER VIRTUAL-DISK 0 /dev/sde [7:0:0:4] disk OPNFILER VIRTUAL-DISK 0 /dev/sdf [7:0:0:5] disk OPNFILER VIRTUAL-DISK 0 /dev/sdg [7:0:0:6] disk OPNFILER VIRTUAL-DISK 0 /dev/sdh [7:0:0:7] disk OPNFILER VIRTUAL-DISK 0 /dev/sdi [7:0:0:8] disk OPNFILER VIRTUAL-DISK 0 /dev/sdj [9:0:0:0] disk OPNFILER VIRTUAL-DISK 0 /dev/sdk [9:0:0:1] disk OPNFILER VIRTUAL-DISK 0 /dev/sdl [9:0:0:2] disk OPNFILER VIRTUAL-DISK 0 /dev/sdn [9:0:0:3] disk OPNFILER VIRTUAL-DISK 0 /dev/sdo [9:0:0:4] disk OPNFILER VIRTUAL-DISK 0 /dev/sdp [9:0:0:5] disk OPNFILER VIRTUAL-DISK 0 /dev/sdq [9:0:0:6] disk OPNFILER VIRTUAL-DISK 0 /dev/sdr [9:0:0:7] disk OPNFILER VIRTUAL-DISK 0 /dev/sds [9:0:0:8] disk OPNFILER VIRTUAL-DISK 0 /dev/sdt [11:0:0:0] disk OPNFILER VIRTUAL-DISK 0 /dev/sdm maciek@oel7rac1n2.dba24.pl ~ $ ls -l /dev/asm/* lrwxrwxrwx 1 root root 7 12-01 17:19 /dev/asm/op1-rac1-asm-crs-01 -> ../sdb1 lrwxrwxrwx 1 root root 7 12-01 17:19 /dev/asm/op1-rac1-asm-data-01 -> ../sdc1 lrwxrwxrwx 1 root root 7 12-01 17:19 /dev/asm/op1-rac1-asm-data-02 -> ../sdd1 lrwxrwxrwx 1 root root 7 12-01 17:19 /dev/asm/op1-rac1-asm-data-03 -> ../sde1 lrwxrwxrwx 1 root root 7 12-01 17:19 /dev/asm/op1-rac1-asm-data-04 -> ../sdf1 lrwxrwxrwx 1 root root 7 12-01 17:19 /dev/asm/op1-rac1-asm-fra-01 -> ../sdg1 lrwxrwxrwx 1 root root 7 12-01 17:19 /dev/asm/op1-rac1-asm-fra-02 -> ../sdh1 lrwxrwxrwx 1 root root 7 12-01 17:19 /dev/asm/op1-rac1-asm-fra-03 -> ../sdi1 lrwxrwxrwx 1 root root 7 12-01 17:19 /dev/asm/op1-rac1-asm-fra-04 -> ../sdj1 lrwxrwxrwx 1 root root 7 12-01 17:19 /dev/asm/op2-rac1-asm-crs-02 -> ../sdk1 lrwxrwxrwx 1 root root 7 12-01 17:19 /dev/asm/op2-rac1-asm-data-01 -> ../sdl1 lrwxrwxrwx 1 root root 7 12-01 17:19 /dev/asm/op2-rac1-asm-data-02 -> ../sdn1 lrwxrwxrwx 1 root root 7 12-01 17:19 /dev/asm/op2-rac1-asm-data-03 -> ../sdo1 lrwxrwxrwx 1 root root 7 12-01 17:19 /dev/asm/op2-rac1-asm-data-04 -> ../sdp1 lrwxrwxrwx 1 root root 7 12-01 17:19 /dev/asm/op2-rac1-asm-fra-01 -> ../sdq1 lrwxrwxrwx 1 root root 7 12-01 17:19 /dev/asm/op2-rac1-asm-fra-02 -> ../sdr1 lrwxrwxrwx 1 root root 7 12-01 17:19 /dev/asm/op2-rac1-asm-fra-03 -> ../sds1 lrwxrwxrwx 1 root root 7 12-01 17:19 /dev/asm/op2-rac1-asm-fra-04 -> ../sdt1 lrwxrwxrwx 1 root root 7 12-01 17:19 /dev/asm/op3-rac1-asm-crs-03 -> ../sdm1
Yup, yup, yup. Luns appeared where needed
We can now proceed to the next part of the series describing GI installation.
What do you think?? Please post your comments 
Source: My experience and WorlWideWeb
About the author
Maciej Tokar
An Oracle technology geek and crazy long distance runner, DBA24 Owner
Senior Oracle DBA / Consultant / [OCP10g, OCP12c, OCE RAC 10g] / [experience: 9y+]
Currently working for Bluegarden (Oslo Norway) by Miratech Group
Past: Mastercard / Trevica by Britenet, Citi International PLC, PZU
