This is third Part of my RAC lab series guiding me and you step by step in the process of building of a fully usable RAC environment for tests and learning. This third part will show you how to create virtual machines accordingly to the specification required for our laboratory.
I will show you how to create new template VM that we will later use as base for new nodes or openfiler servers. Of course you can make them all one by one, but it’s simpler to put one complete config together and then just clone it when required. This way you can build clusters with more than 2 nodes in RAC, or more than 2 storage servers very quickly.
RAC lab Part 1 – Installing the Ubuntu 16.04 desktop
RAC lab Part 2 – Virtualbox installation and configuration
RAC lab Part 3 – VMs configuration
RAC lab Part 4 – Installing the Openfiler software
RAC lab Part 5 – Installing OEL 7.x on VM
RAC lab Part 6 – OEL 7.x configuration for Oracle 12c GI & RAC DB installation
RAC lab Part 7 – Openfilers configuration and cloning
RAC lab Part 8 – OEL 7.x prepare storage
RAC lab Part 9 – Clone first RAC node as 2nd node and prepare config.
RAC lab Part 10 – Installing Grid Infrastructure
RAC lab Part 11 – Installing Database Software
RAC lab Part 12 – Creating a Container Database
Virtual Machines configuration in the Virtualbox environment
First let’s take a look at the plan of the network layout – this is very important to plan it in advance. Many bad things can happen later if you skip it 
The Internal network is a kind of a network logically defined between Virtual Machines inside the Virtualbox. This networks are accessible just between the VMs, you can’t reach that network from the host’s operating system or from the outside world.
Read more about internal networks here: https://www.virtualbox.org/manual/ch06.html#network_internal
Bridged network allows your VMs’ interfaces to be plugged in into the physical network of your host. You choose the interface your VMs will be bridged to. In my case I am using wlan interface, because I want to have access to the environment not only from my Virtuablox host (my ubuntu workstation), but also from other computers in my home network.
Read more about bridged network here: https://www.virtualbox.org/manual/ch06.html#network_bridged
- Following table provides information on the Virtualbox networks. I am gonna use that list to build IP addresses for my servers
| VM network | Network Address | Description |
| bridged | 192.168.1.0 | public network bridged to the outer world |
| storage-internal | 192.168.10.0 | storage private network between VMs |
| priv1-internal | 192.168.20.0 | private interconnect for RAC HAIP VLAN1 |
| priv2-internal | 192.168.21.0 | private interconnect for RAC HAIP VLAN2 |
| apriv1-internal | 192.168.30.0 | private network for ASM VLAN1 |
| apriv2-internal | 192.168.31.0 | private network for ASM VLAN2 |
- Below the IP addresses and networks that will be used by the openfiler VMs
| VM | DNS Name | Type | IP Address | DHCP/static IP | VB network type | VB network name |
| openfiler1 | openfiler1 | public | 192.168.1.10 | DHCP-fixed IP | bridged | bridged |
| openfiler1 | openfiler1-spriv | storage private | 192.168.10.10 | static | internal | storage-internal |
| openfiler2 | openfiler2 | public | 192.168.1.11 | DHCP-fixed IP | bridged | bridged |
| openfiler2 | openfiler2-spriv | storage private | 192.168.10.11 | static | internal | storage-internal |
| openfiler3 | openfiler3 | public | 192.168.1.12 | DHCP-fixed IP | bridged | bridged |
| openfiler3 | openfiler3-spriv | storage private | 192.168.10.12 | static | internal | storage-internal |
- Finally here come the RAC nodes’ addresses:
| VM | DNS Name | Type | IP Address | DHCP/static IP | VB network type | VB network name |
| oel7rac1n1 | oel7rac1n1 | public | 192.168.1.21 | DHCP-fixed IP | bridged | bridged |
| oel7rac1n1 | oel7rac1n1-vip | virtual | 192.168.1.22 | static | bridged | bridged |
| oel7rac1n1 | oel7rac1n1-spriv | storage private | 192.168.10.21 | static | internal | storage-internal |
| oel7rac1n1 | oel7rac1n1-priv1 | interconnect | 192.168.20.21 | static | internal | priv1-internal |
| oel7rac1n1 | oel7rac1n1-priv2 | interconnect | 192.168.21.21 | static | internal | priv2-internal |
| oel7rac1n1 | oel7rac1n1-apriv1 | asm priv vlan1 | 192.168.30.21 | static | internal | apriv1-internal |
| oel7rac1n1 | oel7rac1n1-apriv2 | asm priv vlan2 | 192.168.31.21 | static | internal | apriv2-internal |
| oel7rac1n2 | oel7rac1n2 | public | 192.168.1.23 | DHCP-fixed IP | bridged | bridged |
| oel7rac1n2 | oel7rac1n2-vip | virtual | 192.168.1.24 | static | bridged | bridged |
| oel7rac1n2 | oel7rac1n2-spriv | storage private | 192.168.10.23 | static | internal | storage-internal |
| oel7rac1n2 | oel7rac1n2-priv1 | interconnect | 92.168.20.23 | static | internal | priv1-internal |
| oel7rac1n2 | oel7rac1n2-priv2 | interconnect | 192.168.21.23 | static | internal | priv2-internal |
| oel7rac1n2 | oel7rac1n2-apriv1 | asm priv vlan1 | 192.168.30.23 | static | internal | apriv1-internal |
| oel7rac1n2 | oel7rac1n2-apriv2 | asm priv vlan2 | 192.168.31.23 | static | internal | apriv2-internal |
- and SCAN virtual addresses:
| VM | DNS Name | Type | IP Address | DHCP/static IP | VB network type | VB network name |
| not VM fixed | rac1-scan | virtual | 192.168.1.101 | cluster managed | bridged | bridged |
| not VM fixed | rac1-scan | virtual | 192.168.1.24 | cluster managed | bridged | bridged |
| not VM fixed | rac1-scan | virtual | 192.168.10.23 | cluster managed | bridged | bridged |
- Since we got now network requirements, please take a look at the VMs needs for Openfiler here:
http://openfiler.com/products/system-requirements
I have tweaked them a little to fit my needs and workstation capacity. Take a look at the host information below. I got 32GB RAM 1CPU 4 core + HT (8 virt cpus)
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage list hostinfo Host Information: Host time: 2016-11-23T10:21:24.668000000Z Processor online count: 8 Processor count: 8 Processor online core count: 4 Processor core count: 4 Processor supports HW virtualization: yes Processor supports PAE: yes Processor supports long mode: yes Processor supports nested paging: yes Processor#0 speed: 3800 MHz Processor#0 description: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz Processor#1 speed: 3800 MHz Processor#1 description: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz Processor#2 speed: 3800 MHz Processor#2 description: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz Processor#3 speed: 3800 MHz Processor#3 description: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz Processor#4 speed: 3800 MHz Processor#4 description: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz Processor#5 speed: 3800 MHz Processor#5 description: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz Processor#6 speed: 3800 MHz Processor#6 description: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz Processor#7 speed: 3800 MHz Processor#7 description: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz Memory size: 32130 MByte Memory available: 29587 MByte Operating system: Linux Operating system version: 4.4.0-47-generic
What I am going to do is to assign resources accordingly to the server type as in the following table:
| vm type | memory | swap | local storage | additional storage*1 | network ifaces | virtual cpus** | max virtual cpus*** |
| openfiler | 2GB | 2GB | 8GB (thin provis.) | 50GB (thin provis.) | 2 | 1 | 4 |
| oracle RAC node | 6GB | 16.5GB | 30GB (thin provis.) | none | 6 | 2 | 4 |
* Additional storage – storage not related to OS requirements but required in order to be shared to other VMs. In our case Openfiler shares disks for ASM at RAC nodes.
** Virtual CPUs – number of CPUs VM is started with
*** Max Virtual CPUs – maximum number of CPUs that can be hotplugged into the VM
** Virtual CPUs – number of CPUs VM is started with
*** Max Virtual CPUs – maximum number of CPUs that can be hotplugged into the VM
Configure VM for openfiler
- Check available operating system types in Virtualbox to find the best match for Openfiler server
vboxuser1@macieksrv.dba24.pl ~ $ VBoxManage list ostypes |grep -Ev 'Family|64 bit'|egrep -v '(Wind|Solaris|BSD|Netware|DOS|MacOS|Mac OS)' ID: Other Description: Other/Unknown ID: Other_64 Description: Other/Unknown (64-bit) ID: Linux22 Description: Linux 2.2 ID: Linux24 Description: Linux 2.4 (32-bit) ID: Linux24_64 Description: Linux 2.4 (64-bit) ID: Linux26 Description: Linux 2.6 / 3.x / 4.x (32-bit) ID: Linux26_64 Description: Linux 2.6 / 3.x / 4.x (64-bit) ID: ArchLinux Description: Arch Linux (32-bit) ID: ArchLinux_64 Description: Arch Linux (64-bit) ID: Debian Description: Debian (32-bit) ID: Debian_64 Description: Debian (64-bit) ID: OpenSUSE Description: openSUSE (32-bit) ID: OpenSUSE_64 Description: openSUSE (64-bit) ID: Fedora Description: Fedora (32-bit) ID: Fedora_64 Description: Fedora (64-bit) ID: Gentoo Description: Gentoo (32-bit) ID: Gentoo_64 Description: Gentoo (64-bit) ID: Mandriva Description: Mandriva (32-bit) ID: Mandriva_64 Description: Mandriva (64-bit) ID: RedHat Description: Red Hat (32-bit) ID: RedHat_64 Description: Red Hat (64-bit) ID: Turbolinux Description: Turbolinux (32-bit) ID: Turbolinux_64 Description: Turbolinux (64-bit) ID: Ubuntu Description: Ubuntu (32-bit) ID: Ubuntu_64 Description: Ubuntu (64-bit) ID: Xandros Description: Xandros (32-bit) ID: Xandros_64 Description: Xandros (64-bit) ID: Oracle Description: Oracle (32-bit) ID: Oracle_64 Description: Oracle (64-bit) ID: Linux Description: Other Linux (32-bit) ID: Linux_64 Description: Other Linux (64-bit) ID: OS2Warp3 Description: OS/2 Warp 3 ID: OS2Warp4 Description: OS/2 Warp 4 ID: OS2Warp45 Description: OS/2 Warp 4.5 ID: OS2eCS Description: eComStation ID: OS21x Description: OS/2 1.x ID: OS2 Description: Other OS/2 ID: L4 Description: L4 ID: QNX Description: QNX ID: JRockitVE Description: JRockitVE ID: VBoxBS_64 Description: VirtualBox Bootsector Test (64-bit)
I think the one fits best here
ID: Linux_64
Description: Other Linux (64-bit)
- Create openfiler virtual machines
vboxuser1@macieksrv:~$ vboxmanage createvm --name openfiler1 --groups "/Openfilers" --ostype Linux_64 --basefolder /vbox-repo1/metadata --register Virtual machine 'openfiler1' is created and registered. UUID: 002ac87b-755b-46e5-897d-35ebbd5f70f2 Settings file: '/vbox-repo1/metadata/Openfilers/openfiler1/openfiler1.vbox' vboxuser1@macieksrv:~$ vboxmanage createvm --name openfiler2 --groups "/Openfilers" --ostype Linux_64 --basefolder /vbox-repo1/metadata --register Virtual machine 'openfiler2' is created and registered. UUID: fabddd53-40f7-4c5c-85a1-348c3aadc259 Settings file: '/vbox-repo1/metadata/Openfilers/openfiler2/openfiler2.vbox'
- List VMs already created
vboxuser1@macieksrv:~$ VBoxManage list vms
"openfiler1" {002ac87b-755b-46e5-897d-35ebbd5f70f2}
"openfiler2" {fabddd53-40f7-4c5c-85a1-348c3aadc259}
But wait, maybe it could be better to create one machine per type and then just clone (less effort). Let’s delete second openfiler machine
vboxuser1@macieksrv:~$ vboxmanage unregistervm openfiler2 --delete
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
vboxuser1@macieksrv:~$ VBoxManage list vms
"openfiler1" {002ac87b-755b-46e5-897d-35ebbd5f70f2}
VM has been sucessfully deleted.
- Download openfiler ISO from
https://downloads.sourceforge.net/project/openfiler/openfiler-distribution-iso-2.99-x64/openfileresa-2.99.1-x86_64-disc1.iso
and place it into the directory “/vbox-repo1/isofiles/”. Full path to the file should be “/vbox-repo1/isofiles/openfileresa-2.99.1-x86_64-disc1.iso”
- Prepare VM for Openfiler using modifyvm option of vboxmanage tool
Read more on how create VM and modify its settings here:
https://www.virtualbox.org/manual/ch08.html#vboxmanage-createvm
You can also find out what each option used in next code boxes means.
At the beginning let’s set some basic settings for VM
# allow VM to hotplug CPUs vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --cpuhotplug on # set maximum number of cpus allowed for this VM (4xCPU), VM initially # starts with one cpu number 0 assigned to it vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --cpus 4 # set RAM and videoram vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --vram 256 --memory 2048 # set modern chipset type vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --chipset ich9 --acpi on --pagefusion on --ioapic on
- Prepare storage for openfiler.
Start with creating disks, one for local OS and the other one for iSCSI disks sharing
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage createmedium disk --filename /vbox-repo1/disks/openfiler1_localOSdisk1.vdi --size 10240 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Medium created. UUID: fb8e0303-83c8-4712-a487-e7e50b7fc1d5 vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage createmedium disk --filename /vbox-repo1/disks/openfiler1_functionaldisk1.vdi --size 51200 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Medium created. UUID: 633fffa6-6812-485f-8e30-84d4082a7e81 # let's check them vboxuser1@macieksrv.dba24.pl ~ $ ls -l /vbox-repo1/disks/openfiler1_functionaldisk1.vdi -rw------- 1 vboxuser1 vboxusers 2097152 lis 10 12:58 /vbox-repo1/disks/openfiler1_functionaldisk1.vdi vboxuser1@macieksrv.dba24.pl ~ $ ls -l /vbox-repo1/disks/openfiler1_localOSdisk1.vdi -rw------- 1 vboxuser1 vboxusers 2097152 lis 10 12:56 /vbox-repo1/disks/openfiler1_localOSdisk1.vdi
As you can see above the disks are not of the fixed size, they are dynamic and will grow accordingly to the usage, their performance is lower but it saves space.
Attach storage IDE controller to the VM and stick installation iso of Openfiler to it as Slave Master (port 1 device 0)
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage storagectl "openfiler1" --name "IDE Controller" --add ide --portcount 2 vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage storageattach "openfiler1" --storagectl "IDE Controller" --port 1 --device 0 --type dvddrive --medium /vbox-repo1/isofiles/openfileresa-2.99.1-x86_64-disc1.iso
Now attach created VDI image disks to the SATA controller (first create SATA controller)
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage storagectl "openfiler1" --name "SATA Controller" --add sata --portcount 5 vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage storageattach "openfiler1" --storagectl "SATA Controller" --port 1 --type hdd --medium /vbox-repo1/disks/openfiler1_localOSdisk1.vdi vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage storageattach "openfiler1" --storagectl "SATA Controller" --port 2 --type hdd --medium /vbox-repo1/disks/openfiler1_functionaldisk1.vdi
Finally check the disks
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage showmediuminfo disk /vbox-repo1/disks/openfiler1_functionaldisk1.vdi UUID: 633fffa6-6812-485f-8e30-84d4082a7e81 Parent UUID: base State: created Type: normal (base) Location: /vbox-repo1/disks/openfiler1_functionaldisk1.vdi Storage format: VDI Format variant: dynamic default Capacity: 51200 MBytes Size on disk: 2 MBytes Encryption: disabled In use by VMs: openfiler1 (UUID: 002ac87b-755b-46e5-897d-35ebbd5f70f2) vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage showmediuminfo disk /vbox-repo1/disks/openfiler1_localOSdisk1.vdi UUID: fb8e0303-83c8-4712-a487-e7e50b7fc1d5 Parent UUID: base State: created Type: normal (base) Location: /vbox-repo1/disks/openfiler1_localOSdisk1.vdi Storage format: VDI Format variant: dynamic default qCapacity: 10240 MBytes Size on disk: 2 MBytes Encryption: disabled In use by VMs: openfiler1 (UUID: 002ac87b-755b-46e5-897d-35ebbd5f70f2)
Disks have been properly assigned to the openfiler1 VM
- Set boot device to dvd in order to boot from openfiler's iso during first startup
# set DVD as first boot device, disk as secon vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --boot1 dvd --boot2 disk
- Configure network accordingly to the table with the network spec for openfiler.
Create first network interface as bridged and second as internal (for storage network)
# Add first bridged network card and fix it to physical wlan adapter on host (wlx98ded00b5b05) vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --nic1 bridged --bridgeadapter1 wlx98ded00b5b05 # Add second network card connecting to storage-internal internal network vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --nic2 intnet --intnet2 storage-internal
- Configure snapshots for VM
# Specify folder for snapshots - first create it maciek@macieksrv.dba24.pl ~ $ sudo -i mkdir /vbox-repo1/snapshots maciek@macieksrv.dba24.pl ~ $ sudo chown vboxuser1 /vbox-repo1/snapshots vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --snapshotfolder /vbox-repo1/snapshots/
- Configure bidirectional clipboard, drag and drop and Virtualbox Remote Desktop
# set clipboard and drag and drop to be bidirectional between host and VM vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --clipboard bidirectional --draganddrop bidirectional # Enable Virtual Remote Desktop on this VM vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --vrde on --vrdeport 5010-5020
- Let's check settings of this VM after configuration has been completed
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage showvminfo "openfiler1"|more Name: openfiler1 Groups: /Openfilers Guest OS: Other Linux (64-bit) UUID: 002ac87b-755b-46e5-897d-35ebbd5f70f2 Config file: /vbox-repo1/metadata/Openfilers/openfiler1/openfiler1.vbox Snapshot folder: /vbox-repo1/snapshots Log folder: /vbox-repo1/metadata/Openfilers/openfiler1/Logs Hardware UUID: 002ac87b-755b-46e5-897d-35ebbd5f70f2 Memory size: 2048MB Page Fusion: on VRAM size: 256MB CPU exec cap: 100% HPET: off Chipset: ich9 Firmware: BIOS Number of CPUs: 4 PAE: on Long Mode: on Triple Fault Reset: off APIC: on X2APIC: on CPUID Portability Level: 0 CPUID overrides: None Boot menu mode: message and menu Boot Device (1): DVD Boot Device (2): HardDisk Boot Device (3): HardDisk Boot Device (4): Not Assigned ACPI: on IOAPIC: on BIOS APIC mode: APIC Time offset: 0ms RTC: local time Hardw. virt.ext: on Nested Paging: on Large Pages: off VT-x VPID: on VT-x unr. exec.: on Paravirt. Provider: Default Effective Paravirt. Provider: KVM State: powered off (since 2016-11-09T19:48:36.000000000) Monitor count: 1 3D Acceleration: off 2D Video Acceleration: off Teleporter Enabled: off Teleporter Port: 0 Teleporter Address: Teleporter Password: Tracing Enabled: off Allow Tracing to Access VM: off Tracing Configuration: Autostart Enabled: off Autostart Delay: 0 Default Frontend: Storage Controller Name (0): IDE Controller Storage Controller Type (0): PIIX4 Storage Controller Instance Number (0): 0 Storage Controller Max Port Count (0): 2 Storage Controller Port Count (0): 2 Storage Controller Bootable (0): on Storage Controller Name (1): SATA Controller Storage Controller Type (1): IntelAhci Storage Controller Instance Number (1): 0 Storage Controller Max Port Count (1): 30 Storage Controller Port Count (1): 5 Storage Controller Bootable (1): on IDE Controller (1, 0): /vbox-repo1/isofiles/openfileresa-2.99.1-x86_64-disc1.iso (UUID: c7402463-c2c5-40fe-924b-3a02a183fab0) SATA Controller (1, 0): /vbox-repo1/disks/openfiler1_localOSdisk1.vdi (UUID: fb8e0303-83c8-4712-a487-e7e50b7fc1d5) SATA Controller (2, 0): /vbox-repo1/disks/openfiler1_functionaldisk1.vdi (UUID: 633fffa6-6812-485f-8e30-84d4082a7e81) NIC 1: MAC: 08002780201C, Attachment: Bridged Interface 'wlx98ded00b5b05', Cable connected: on, Trace: off (file: none), Type: 82540EM, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: den y, Bandwidth group: none NIC 2: MAC: 080027782E66, Attachment: Internal Network 'storage-internal', Cable connected: on, Trace: off (file: none), Type: 82540EM, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: den y, Bandwidth group: none NIC 3: disabled ........ lines removed for better clarity ........ NIC 36: disabled Pointing Device: PS/2 Mouse Keyboard Device: PS/2 Keyboard UART 1: disabled UART 2: disabled UART 3: disabled UART 4: disabled LPT 1: disabled LPT 2: disabled Audio: enabled (Driver: PulseAudio, Controller: AC97, Codec: STAC9700) Clipboard Mode: Bidirectional Drag and drop Mode: Bidirectional VRDE: enabled (Address 0.0.0.0, Ports 5010-5020, MultiConn: off, ReuseSingleConn: off, Authentication type: null) Video redirection: disabled VRDE property: TCP/Ports = "5010-5020" VRDE property: TCP/Address = <not set=""> VRDE property: VideoChannel/Enabled = <not set=""> VRDE property: VideoChannel/Quality = <not set=""> VRDE property: VideoChannel/DownscaleProtection = <not set=""> VRDE property: Client/DisableDisplay = <not set=""> VRDE property: Client/DisableInput = <not set=""> VRDE property: Client/DisableAudio = <not set=""> VRDE property: Client/DisableUSB = <not set=""> VRDE property: Client/DisableClipboard = <not set=""> VRDE property: Client/DisableUpstreamAudio = <not set=""> VRDE property: Client/DisableRDPDR = <not set=""> VRDE property: H3DRedirect/Enabled = <not set=""> VRDE property: Security/Method = <not set=""> VRDE property: Security/ServerCertificate = <not set=""> VRDE property: Security/ServerPrivateKey = <not set=""> VRDE property: Security/CACertificate = <not set=""> VRDE property: Audio/RateCorrectionMode = <not set=""> VRDE property: Audio/LogPath = <not set=""> USB: disabled EHCI: disabled XHCI: disabled USB Device Filters: <none> Bandwidth groups: <none> Shared folders: <none> Video capturing: not active Capture screens: 0 Capture file: /vbox-repo1/metadata/Openfilers/openfiler1/openfiler1.webm Capture dimensions: 1024x768 Capture rate: 512 kbps Capture FPS: 25 Guest: Configured memory balloon size: 0 MB
You can also view VM details in Virtualbox Manager GUI
VM for openfiler is ready now
We can treat this openfiler installation as a template. In the next part of the series I am going to install openfiler software on this VM and then there is a chance that it can be cloned easily as openfiler2 - instead of instaling openfiler2 from scratch.
Configure VM for oracle RAC node
- Create oel72rac1n1 virtual machine. Notice the name is different than the one in the specification tables. oel72rac1n1 was an initial name, but as you will see this is not a good name so eventually it will have been changed by the end of the installation.
vboxuser1@macieksrv:~$ vboxmanage createvm --name oel72rac1n1 --groups "/RacLab1" --ostype Oracle_64 --basefolder /vbox-repo1/metadata --register Virtual machine 'oel72rac1n1' is created and registered. UUID: 9d9f28c6-023b-438b-b429-d70373246640 Settings file: '/vbox-repo1/metadata/RacLab1/oel72rac1n1/oel72rac1n1.vbox'
- List VMs already created
vboxuser1@macieksrv:~$ VBoxManage list vms
"openfiler1" {002ac87b-755b-46e5-897d-35ebbd5f70f2}
"oel72rac1n1" {9d9f28c6-023b-438b-b429-d70373246640}
- Download OEL 7.2 ISO from edelivery.oracle.com
Download and save it into the directory "/vbox-repo1/isofiles/". Full path to the file should be "/vbox-repo1/isofiles/V100082-01.iso"
- Prepare VM for oel72racn1 using modifyvm option of vboxmanage tool
At the beginning let's set some basic settings for VM
# allow VM to hotplug CPUs vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --cpuhotplug on # set maximum number of cpus allowed for this VM (4xCPU) vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --cpus 4 # set starting value for this vm (2xCPU), so pin cpu 1 (0 by default) #vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --plugcpu 1 # set RAM and videoram vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --vram 256 --memory 6144 # set modern chipset type and ACPI and IOAPIC vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --chipset ich9 --acpi on --pagefusion on --ioapic on
- Prepare storage for oracle rac node.
Start with creating disks, one for local OS and the other one for iSCSI disks sharing
# create local disk image vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage createmedium disk --filename /vbox-repo1/disks/oel72rac1n1_localOSdisk1.vdi --size 51200 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Medium created. UUID: be004bfa-0069-452c-b9b5-2448a988cd3b # let's check the disk vboxuser1@macieksrv.dba24.pl ~ $ ls -l /vbox-repo1/disks/oel72rac1n1_localOSdisk1.vdi -rw------- 1 vboxuser1 vboxusers 2097152 lis 10 21:16 /vbox-repo1/disks/oel72rac1n1_localOSdisk1.vdi
As you can see above the disks is not of the fixed size, it is dynamic and will grow accordingly to the usage, its performance is lower but it saves space.
Attach storage IDE controller to the VM and stick installation iso of OEL 7.2 to it as Slave Master (port 1 device 0)
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage storagectl "oel72rac1n1" --name "IDE Controller" --add ide --portcount 2 vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage storageattach "oel72rac1n1" --storagectl "IDE Controller" --port 1 --device 0 --type dvddrive --medium /vbox-repo1/isofiles/V100082-01.iso
Now attach created VDI image disk to the SATA controller (first create SATA controller)
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage storagectl "oel72rac1n1" --name "SATA Controller" --add sata --portcount 5 vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage storageattach "oel72rac1n1" --storagectl "SATA Controller" --port 1 --type hdd --medium /vbox-repo1/disks/oel72rac1n1_localOSdisk1.vdi
Finally check the disk
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage showmediuminfo disk /vbox-repo1/disks/oel72rac1n1_localOSdisk1.vdi UUID: be004bfa-0069-452c-b9b5-2448a988cd3b Parent UUID: base State: created Type: normal (base) Location: /vbox-repo1/disks/oel72rac1n1_localOSdisk1.vdi Storage format: VDI Format variant: dynamic default Capacity: 51200 MBytes Size on disk: 2 MBytes Encryption: disabled In use by VMs: oel72rac1n1 (UUID: 9d9f28c6-023b-438b-b429-d70373246640)
Disk has been properly assigned to the oel72rac1n1 VM
- Set boot device to dvd in order to boot from openfiler's iso during first startup
# set DVD as first boot device, disk as second vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --boot1 dvd --boot2 disk
- Configure network accordingly to the table with the network spec for oracle rac nodes.
You can find the specification at the very beginning of this article
# Add first bridged network card and fix it to physical wlan adapter on host (wlx98ded00b5b05) vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --nic1 bridged --bridgeadapter1 wlx98ded00b5b05 # Add network cards and additional internal networks for priv (interconnect) and apriv (asm network) and also for storage network vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --nic2 intnet --intnet2 storage-internal vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --nic3 intnet --intnet3 priv1-internal vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --nic4 intnet --intnet4 priv2-internal vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --nic5 intnet --intnet5 apriv1-internal vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --nic6 intnet --intnet6 apriv2-internal
- Configure snapshots for VM
# Specify folder for snapshots - first create it vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --snapshotfolder /vbox-repo1/snapshots/
- Configure bidirectional clipboard, drag and drop and Virtualbox Remote Desktop
# set clipboard and drag and drop to be bidirectional between host and VM vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --clipboard bidirectional --draganddrop bidirectional # Enable Virtual Remote Desktop on this VM vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --vrde on --vrdeport 5010-5020
- Let's check settings of this VM after configuration has been completed
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage showvminfo "oel72rac1n1"|more Name: oel72rac1n1 Groups: /RacLab1 Guest OS: Oracle (64-bit) UUID: 9d9f28c6-023b-438b-b429-d70373246640 Config file: /vbox-repo1/metadata/RacLab1/oel72rac1n1/oel72rac1n1.vbox Snapshot folder: /vbox-repo1/snapshots Log folder: /vbox-repo1/metadata/RacLab1/oel72rac1n1/Logs Hardware UUID: 9d9f28c6-023b-438b-b429-d70373246640 Memory size: 6144MB Page Fusion: on VRAM size: 256MB CPU exec cap: 100% HPET: off Chipset: ich9 Firmware: BIOS Number of CPUs: 4 PAE: on Long Mode: on Triple Fault Reset: off APIC: on X2APIC: on CPUID Portability Level: 0 CPUID overrides: None Boot menu mode: message and menu Boot Device (1): DVD Boot Device (2): HardDisk Boot Device (3): HardDisk Boot Device (4): Not Assigned ACPI: on IOAPIC: on BIOS APIC mode: APIC Time offset: 0ms RTC: local time Hardw. virt.ext: on Nested Paging: on Large Pages: off VT-x VPID: on VT-x unr. exec.: on Paravirt. Provider: Default Effective Paravirt. Provider: KVM State: powered off (since 2016-11-09T19:49:01.000000000) Monitor count: 1 3D Acceleration: off 2D Video Acceleration: off Teleporter Enabled: off Teleporter Port: 0 Teleporter Address: Teleporter Password: Tracing Enabled: off Allow Tracing to Access VM: off Tracing Configuration: Autostart Enabled: off Autostart Delay: 0 Default Frontend: Storage Controller Name (0): IDE Controller Storage Controller Type (0): PIIX4 Storage Controller Instance Number (0): 0 Storage Controller Max Port Count (0): 2 Storage Controller Port Count (0): 2 Storage Controller Bootable (0): on Storage Controller Name (1): SATA Controller Storage Controller Type (1): IntelAhci Storage Controller Instance Number (1): 0 Storage Controller Max Port Count (1): 30 Storage Controller Port Count (1): 5 Storage Controller Bootable (1): on IDE Controller (1, 0): /vbox-repo1/isofiles/V100082-01.iso (UUID: 48419f91-ace0-4e22-afda-1ba7a2b70b77) SATA Controller (1, 0): /vbox-repo1/disks/oel72rac1n1_localOSdisk1.vdi (UUID: be004bfa-0069-452c-b9b5-2448a988cd3b) NIC 1: MAC: 0800276A189E, Attachment: Bridged Interface 'wlx98ded00b5b05', Cable connected: on, Trace: off (file: none), Type: 82540EM, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: den y, Bandwidth group: none NIC 2: MAC: 080027DB1740, Attachment: Internal Network 'storage-internal', Cable connected: on, Trace: off (file: none), Type: 82540EM, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: den y, Bandwidth group: none NIC 3: MAC: 080027D59C97, Attachment: Internal Network 'priv1-internal', Cable connected: on, Trace: off (file: none), Type: 82540EM, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: deny, Bandwidth group: none NIC 4: MAC: 080027AE9147, Attachment: Internal Network 'priv2-internal', Cable connected: on, Trace: off (file: none), Type: 82540EM, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: deny, Bandwidth group: none NIC 5: MAC: 08002748771B, Attachment: Internal Network 'apriv1-internal', Cable connected: on, Trace: off (file: none), Type: 82540EM, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: deny , Bandwidth group: none NIC 6: MAC: 0800271A1EE2, Attachment: Internal Network 'apriv2-internal', Cable connected: on, Trace: off (file: none), Type: 82540EM, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: deny , Bandwidth group: none Pointing Device: PS/2 Mouse Keyboard Device: PS/2 Keyboard UART 1: disabled UART 2: disabled UART 3: disabled UART 4: disabled LPT 1: disabled LPT 2: disabled Audio: enabled (Driver: PulseAudio, Controller: AC97, Codec: STAC9700) Clipboard Mode: Bidirectional Drag and drop Mode: Bidirectional VRDE: enabled (Address 0.0.0.0, Ports 5010-5020, MultiConn: off, ReuseSingleConn: off, Authentication type: null) Video redirection: disabled VRDE property: TCP/Ports = "5010-5020" VRDE property: TCP/Address = <not set=""> VRDE property: VideoChannel/Enabled = <not set=""> VRDE property: VideoChannel/Quality = <not set=""> VRDE property: VideoChannel/DownscaleProtection = <not set=""> VRDE property: Client/DisableDisplay = <not set=""> VRDE property: Client/DisableInput = <not set=""> VRDE property: Client/DisableAudio = <not set=""> VRDE property: Client/DisableUSB = <not set=""> VRDE property: Client/DisableClipboard = <not set=""> VRDE property: Client/DisableUpstreamAudio = <not set=""> VRDE property: Client/DisableRDPDR = <not set=""> VRDE property: H3DRedirect/Enabled = <not set=""> VRDE property: Security/Method = <not set=""> VRDE property: Security/ServerCertificate = <not set=""> VRDE property: Security/ServerPrivateKey = <not set=""> VRDE property: Security/CACertificate = <not set=""> VRDE property: Audio/RateCorrectionMode = <not set=""> VRDE property: Audio/LogPath = <not set=""> USB: disabled EHCI: disabled XHCI: disabled USB Device Filters: <none> Bandwidth groups: <none> Shared folders: <none> Video capturing: not active Capture screens: 0 Capture file: /vbox-repo1/metadata/RacLab1/oel72rac1n1/oel72rac1n1.webm Capture dimensions: 1024x768 Capture rate: 512 kbps Capture FPS: 25 Guest: Configured memory balloon size: 0 MB
You can also view VM details in Virtualbox Manager GUI
VM for oracle RAC node is ready now
We can treat this VM installation as a template of a pure VM without OS, but in this tutorial I will install OEL and prepare OS for RAC installation and then will use this VM as complete ready-to-clone template for new nodes. In the next parts of the series I am going to install OEL 7,2 software on this VM and set OS accordingly to Oracle requirements for 12c GI and RAC databases.
What do you think?? Please post your comments
Source: My experience and WorlWideWeb
About the author
Maciej Tokar
An Oracle technology geek and crazy long distance runner, DBA24 Owner
Senior Oracle DBA / Consultant / [OCP10g, OCP12c, OCE RAC 10g] / [experience: 9y+]
Currently working for Bluegarden (Oslo Norway) by Miratech Group
Past: Mastercard / Trevica by Britenet, Citi International PLC, PZU
