This is third Part of my RAC lab series guiding me and you step by step in the process of building of a fully usable RAC environment for tests and learning. This third part will show you how to create virtual machines accordingly to the specification required for our laboratory.

I will show you how to create new template VM that we will later use as base for new nodes or openfiler servers. Of course you can make them all one by one, but it’s simpler to put one complete config together and then just clone it when required. This way you can build clusters with more than 2 nodes in RAC, or more than 2 storage servers very quickly.

RAC lab Part 1 – Installing the Ubuntu 16.04 desktop
RAC lab Part 2 – Virtualbox installation and configuration
RAC lab Part 3 – VMs configuration
RAC lab Part 4 – Installing the Openfiler software
RAC lab Part 5 – Installing OEL 7.x on VM
RAC lab Part 6 – OEL 7.x configuration for Oracle 12c GI & RAC DB installation
RAC lab Part 7 – Openfilers configuration and cloning
RAC lab Part 8 – OEL 7.x prepare storage
RAC lab Part 9 – Clone first RAC node as 2nd node and prepare config.
RAC lab Part 10 – Installing Grid Infrastructure
RAC lab Part 11 – Installing Database Software
RAC lab Part 12 – Creating a Container Database

 
 
Virtual Machines configuration in the Virtualbox environment

First let’s take a look at the plan of the network layout – this is very important to plan it in advance. Many bad things can happen later if you skip it :)

networksinvirtualbox_3rdvote

The Internal network is a kind of a network logically defined between Virtual Machines inside the Virtualbox. This networks are accessible just between the VMs, you can’t reach that network from the host’s operating system or from the outside world.

Read more about internal networks here: https://www.virtualbox.org/manual/ch06.html#network_internal

Bridged network allows your VMs’ interfaces to be plugged in into the physical network of your host. You choose the interface your VMs will be bridged to. In my case I am using wlan interface, because I want to have access to the environment not only from my Virtuablox host (my ubuntu workstation), but also from other computers in my home network.

Read more about bridged network here: https://www.virtualbox.org/manual/ch06.html#network_bridged

  • Following table provides information on the Virtualbox networks. I am gonna use that list to build IP addresses for my servers
VM network Network Address Description
bridged 192.168.1.0 public network bridged to the outer world
storage-internal 192.168.10.0 storage private network between VMs
priv1-internal 192.168.20.0 private interconnect for RAC HAIP VLAN1
priv2-internal 192.168.21.0 private interconnect for RAC HAIP VLAN2
apriv1-internal 192.168.30.0 private network for ASM VLAN1
apriv2-internal 192.168.31.0 private network for ASM VLAN2
  • Below the IP addresses and networks that will be used by the openfiler VMs
VM DNS Name Type IP Address DHCP/static IP VB network type VB network name
openfiler1 openfiler1 public 192.168.1.10 DHCP-fixed IP bridged bridged
openfiler1 openfiler1-spriv storage private 192.168.10.10 static internal storage-internal
openfiler2 openfiler2 public 192.168.1.11 DHCP-fixed IP bridged bridged
openfiler2 openfiler2-spriv storage private 192.168.10.11 static internal storage-internal
openfiler3 openfiler3 public 192.168.1.12 DHCP-fixed IP bridged bridged
openfiler3 openfiler3-spriv storage private 192.168.10.12 static internal storage-internal
  • Finally here come the RAC nodes’ addresses:
VM DNS Name Type IP Address DHCP/static IP VB network type VB network name
oel7rac1n1 oel7rac1n1 public 192.168.1.21 DHCP-fixed IP bridged bridged
oel7rac1n1 oel7rac1n1-vip virtual 192.168.1.22 static bridged bridged
oel7rac1n1 oel7rac1n1-spriv storage private 192.168.10.21 static internal storage-internal
oel7rac1n1 oel7rac1n1-priv1 interconnect 192.168.20.21 static internal priv1-internal
oel7rac1n1 oel7rac1n1-priv2 interconnect 192.168.21.21 static internal priv2-internal
oel7rac1n1 oel7rac1n1-apriv1 asm priv vlan1 192.168.30.21 static internal apriv1-internal
oel7rac1n1 oel7rac1n1-apriv2 asm priv vlan2 192.168.31.21 static internal apriv2-internal
oel7rac1n2 oel7rac1n2 public 192.168.1.23 DHCP-fixed IP bridged bridged
oel7rac1n2 oel7rac1n2-vip virtual 192.168.1.24 static bridged bridged
oel7rac1n2 oel7rac1n2-spriv storage private 192.168.10.23 static internal storage-internal
oel7rac1n2 oel7rac1n2-priv1 interconnect 92.168.20.23 static internal priv1-internal
oel7rac1n2 oel7rac1n2-priv2 interconnect 192.168.21.23 static internal priv2-internal
oel7rac1n2 oel7rac1n2-apriv1 asm priv vlan1 192.168.30.23 static internal apriv1-internal
oel7rac1n2 oel7rac1n2-apriv2 asm priv vlan2 192.168.31.23 static internal apriv2-internal
  • and SCAN virtual addresses:
VM DNS Name Type IP Address DHCP/static IP VB network type VB network name
not VM fixed rac1-scan virtual 192.168.1.101 cluster managed bridged bridged
not VM fixed rac1-scan virtual 192.168.1.24 cluster managed bridged bridged
not VM fixed rac1-scan virtual 192.168.10.23 cluster managed bridged bridged

I have tweaked them a little to fit my needs and workstation capacity. Take a look at the host information below. I got 32GB RAM 1CPU 4 core + HT (8 virt cpus)

vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage list hostinfo
Host Information:

Host time: 2016-11-23T10:21:24.668000000Z
Processor online count: 8
Processor count: 8
Processor online core count: 4
Processor core count: 4
Processor supports HW virtualization: yes
Processor supports PAE: yes
Processor supports long mode: yes
Processor supports nested paging: yes
Processor#0 speed: 3800 MHz
Processor#0 description: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz
Processor#1 speed: 3800 MHz
Processor#1 description: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz
Processor#2 speed: 3800 MHz
Processor#2 description: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz
Processor#3 speed: 3800 MHz
Processor#3 description: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz
Processor#4 speed: 3800 MHz
Processor#4 description: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz
Processor#5 speed: 3800 MHz
Processor#5 description: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz
Processor#6 speed: 3800 MHz
Processor#6 description: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz
Processor#7 speed: 3800 MHz
Processor#7 description: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz
Memory size: 32130 MByte
Memory available: 29587 MByte
Operating system: Linux
Operating system version: 4.4.0-47-generic

What I am going to do is to assign resources accordingly to the server type as in the following table:

vm type memory swap local storage additional storage*1 network ifaces virtual cpus** max virtual cpus***
openfiler 2GB 2GB 8GB (thin provis.) 50GB (thin provis.) 2 1 4
oracle RAC node 6GB 16.5GB 30GB (thin provis.) none 6 2 4
* Additional storage – storage not related to OS requirements but required in order to be shared to other VMs. In our case Openfiler shares disks for ASM at RAC nodes.
** Virtual CPUs – number of CPUs VM is started with
*** Max Virtual CPUs – maximum number of CPUs that can be hotplugged into the VM
 
 
Configure VM for openfiler
 
  • Check available operating system types in Virtualbox to find the best match for Openfiler server
vboxuser1@macieksrv.dba24.pl ~ $ VBoxManage list ostypes |grep -Ev 'Family|64 bit'|egrep -v '(Wind|Solaris|BSD|Netware|DOS|MacOS|Mac OS)'
ID:          Other
Description: Other/Unknown

ID:          Other_64
Description: Other/Unknown (64-bit)

ID:          Linux22
Description: Linux 2.2

ID:          Linux24
Description: Linux 2.4 (32-bit)

ID:          Linux24_64
Description: Linux 2.4 (64-bit)

ID:          Linux26
Description: Linux 2.6 / 3.x / 4.x (32-bit)

ID:          Linux26_64
Description: Linux 2.6 / 3.x / 4.x (64-bit)

ID:          ArchLinux
Description: Arch Linux (32-bit)

ID:          ArchLinux_64
Description: Arch Linux (64-bit)

ID:          Debian
Description: Debian (32-bit)

ID:          Debian_64
Description: Debian (64-bit)

ID:          OpenSUSE
Description: openSUSE (32-bit)

ID:          OpenSUSE_64
Description: openSUSE (64-bit)

ID:          Fedora
Description: Fedora (32-bit)

ID:          Fedora_64
Description: Fedora (64-bit)

ID:          Gentoo
Description: Gentoo (32-bit)

ID:          Gentoo_64
Description: Gentoo (64-bit)

ID:          Mandriva
Description: Mandriva (32-bit)

ID:          Mandriva_64
Description: Mandriva (64-bit)

ID:          RedHat
Description: Red Hat (32-bit)

ID:          RedHat_64
Description: Red Hat (64-bit)

ID:          Turbolinux
Description: Turbolinux (32-bit)

ID:          Turbolinux_64
Description: Turbolinux (64-bit)

ID:          Ubuntu
Description: Ubuntu (32-bit)

ID:          Ubuntu_64
Description: Ubuntu (64-bit)

ID:          Xandros
Description: Xandros (32-bit)

ID:          Xandros_64
Description: Xandros (64-bit)

ID:          Oracle
Description: Oracle (32-bit)

ID:          Oracle_64
Description: Oracle (64-bit)

ID:          Linux
Description: Other Linux (32-bit)

ID:          Linux_64
Description: Other Linux (64-bit)

ID:          OS2Warp3
Description: OS/2 Warp 3

ID:          OS2Warp4
Description: OS/2 Warp 4

ID:          OS2Warp45
Description: OS/2 Warp 4.5

ID:          OS2eCS
Description: eComStation

ID:          OS21x
Description: OS/2 1.x

ID:          OS2
Description: Other OS/2

ID:          L4
Description: L4

ID:          QNX
Description: QNX

ID:          JRockitVE
Description: JRockitVE

ID:          VBoxBS_64
Description: VirtualBox Bootsector Test (64-bit)

I think the one fits best here

ID: Linux_64
Description: Other Linux (64-bit)

  • Create openfiler virtual machines
vboxuser1@macieksrv:~$ vboxmanage createvm --name openfiler1 --groups "/Openfilers" --ostype Linux_64 --basefolder /vbox-repo1/metadata --register
Virtual machine 'openfiler1' is created and registered.
UUID: 002ac87b-755b-46e5-897d-35ebbd5f70f2
Settings file: '/vbox-repo1/metadata/Openfilers/openfiler1/openfiler1.vbox'

vboxuser1@macieksrv:~$ vboxmanage createvm --name openfiler2 --groups "/Openfilers" --ostype Linux_64 --basefolder /vbox-repo1/metadata --register
Virtual machine 'openfiler2' is created and registered.
UUID: fabddd53-40f7-4c5c-85a1-348c3aadc259
Settings file: '/vbox-repo1/metadata/Openfilers/openfiler2/openfiler2.vbox'
  • List VMs already created
vboxuser1@macieksrv:~$ VBoxManage list vms
"openfiler1" {002ac87b-755b-46e5-897d-35ebbd5f70f2}
"openfiler2" {fabddd53-40f7-4c5c-85a1-348c3aadc259}

But wait, maybe it could be better to create one machine per type and then just clone (less effort). Let’s delete second openfiler machine

vboxuser1@macieksrv:~$ vboxmanage unregistervm openfiler2 --delete
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%

vboxuser1@macieksrv:~$ VBoxManage list vms
"openfiler1" {002ac87b-755b-46e5-897d-35ebbd5f70f2}

VM has been sucessfully deleted.

  • Download openfiler ISO from

https://downloads.sourceforge.net/project/openfiler/openfiler-distribution-iso-2.99-x64/openfileresa-2.99.1-x86_64-disc1.iso
and place it into the directory “/vbox-repo1/isofiles/”. Full path to the file should be “/vbox-repo1/isofiles/openfileresa-2.99.1-x86_64-disc1.iso”

  • Prepare VM for Openfiler using modifyvm option of vboxmanage tool

Read more on how create VM and modify its settings here:
https://www.virtualbox.org/manual/ch08.html#vboxmanage-createvm
You can also find out what each option used in next code boxes means.

At the beginning let’s set some basic settings for VM

# allow VM to hotplug CPUs 
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --cpuhotplug on

# set maximum number of cpus allowed for this VM (4xCPU), VM initially
# starts with one cpu number 0 assigned to it
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --cpus 4

# set RAM and videoram
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --vram 256 --memory 2048

# set modern chipset type
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --chipset ich9 --acpi on  --pagefusion on --ioapic on

  • Prepare storage for openfiler.

Start with creating disks, one for local OS and the other one for iSCSI disks sharing

vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage createmedium disk --filename /vbox-repo1/disks/openfiler1_localOSdisk1.vdi --size 10240
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Medium created. UUID: fb8e0303-83c8-4712-a487-e7e50b7fc1d5

vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage createmedium disk --filename /vbox-repo1/disks/openfiler1_functionaldisk1.vdi --size 51200
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Medium created. UUID: 633fffa6-6812-485f-8e30-84d4082a7e81

# let's check them
vboxuser1@macieksrv.dba24.pl ~ $ ls -l /vbox-repo1/disks/openfiler1_functionaldisk1.vdi
-rw------- 1 vboxuser1 vboxusers 2097152 lis 10 12:58 /vbox-repo1/disks/openfiler1_functionaldisk1.vdi

vboxuser1@macieksrv.dba24.pl ~ $ ls -l /vbox-repo1/disks/openfiler1_localOSdisk1.vdi
-rw------- 1 vboxuser1 vboxusers 2097152 lis 10 12:56 /vbox-repo1/disks/openfiler1_localOSdisk1.vdi

As you can see above the disks are not of the fixed size, they are dynamic and will grow accordingly to the usage, their performance is lower but it saves space.

Attach storage IDE controller to the VM and stick installation iso of Openfiler to it as Slave Master (port 1 device 0)

vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage storagectl "openfiler1" --name "IDE Controller" --add ide --portcount 2
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage storageattach "openfiler1" --storagectl "IDE Controller"  --port 1 --device 0 --type dvddrive --medium /vbox-repo1/isofiles/openfileresa-2.99.1-x86_64-disc1.iso

Now attach created VDI image disks to the SATA controller (first create SATA controller)

vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage storagectl "openfiler1" --name "SATA Controller" 
--add sata --portcount 5
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage storageattach "openfiler1" --storagectl "SATA Controller"  --port 1 --type hdd --medium /vbox-repo1/disks/openfiler1_localOSdisk1.vdi
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage storageattach "openfiler1" --storagectl "SATA Controller"  --port 2 --type hdd --medium /vbox-repo1/disks/openfiler1_functionaldisk1.vdi

Finally check the disks

vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage showmediuminfo disk /vbox-repo1/disks/openfiler1_functionaldisk1.vdi
UUID:           633fffa6-6812-485f-8e30-84d4082a7e81
Parent UUID:    base
State:          created
Type:           normal (base)
Location:       /vbox-repo1/disks/openfiler1_functionaldisk1.vdi
Storage format: VDI
Format variant: dynamic default
Capacity:       51200 MBytes
Size on disk:   2 MBytes
Encryption:     disabled
In use by VMs:  openfiler1 (UUID: 002ac87b-755b-46e5-897d-35ebbd5f70f2)

vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage showmediuminfo disk /vbox-repo1/disks/openfiler1_localOSdisk1.vdi
UUID:           fb8e0303-83c8-4712-a487-e7e50b7fc1d5
Parent UUID:    base
State:          created
Type:           normal (base)
Location:       /vbox-repo1/disks/openfiler1_localOSdisk1.vdi
Storage format: VDI
Format variant: dynamic default
qCapacity:       10240 MBytes
Size on disk:   2 MBytes
Encryption:     disabled
In use by VMs:  openfiler1 (UUID: 002ac87b-755b-46e5-897d-35ebbd5f70f2)

Disks have been properly assigned to the openfiler1 VM

  • Set boot device to dvd in order to boot from openfiler's iso during first startup
# set DVD as first boot device, disk as secon
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --boot1 dvd --boot2 disk
  • Configure network accordingly to the table with the network spec for openfiler.

Create first network interface as bridged and second as internal (for storage network)


# Add first bridged network card and fix it to physical wlan adapter on host (wlx98ded00b5b05)
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --nic1 bridged --bridgeadapter1 wlx98ded00b5b05

# Add second network card connecting to storage-internal internal network
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --nic2 intnet --intnet2 storage-internal
  • Configure snapshots for VM
# Specify folder for snapshots - first create it
maciek@macieksrv.dba24.pl ~ $ sudo -i mkdir /vbox-repo1/snapshots
maciek@macieksrv.dba24.pl ~ $ sudo chown vboxuser1 /vbox-repo1/snapshots

vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --snapshotfolder /vbox-repo1/snapshots/
  • Configure bidirectional clipboard, drag and drop and Virtualbox Remote Desktop
# set clipboard and drag and drop to be bidirectional between host and VM
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --clipboard bidirectional --draganddrop bidirectional

# Enable Virtual Remote Desktop on this VM
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "openfiler1" --vrde on --vrdeport 5010-5020

  • Let's check settings of this VM after configuration has been completed
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage showvminfo "openfiler1"|more
Name:            openfiler1
Groups:          /Openfilers
Guest OS:        Other Linux (64-bit)
UUID:            002ac87b-755b-46e5-897d-35ebbd5f70f2
Config file:     /vbox-repo1/metadata/Openfilers/openfiler1/openfiler1.vbox
Snapshot folder: /vbox-repo1/snapshots
Log folder:      /vbox-repo1/metadata/Openfilers/openfiler1/Logs
Hardware UUID:   002ac87b-755b-46e5-897d-35ebbd5f70f2
Memory size:     2048MB
Page Fusion:     on
VRAM size:       256MB
CPU exec cap:    100%
HPET:            off
Chipset:         ich9
Firmware:        BIOS
Number of CPUs:  4
PAE:             on
Long Mode:       on
Triple Fault Reset: off
APIC:            on
X2APIC:          on
CPUID Portability Level: 0
CPUID overrides: None
Boot menu mode:  message and menu
Boot Device (1): DVD
Boot Device (2): HardDisk
Boot Device (3): HardDisk
Boot Device (4): Not Assigned
ACPI:            on
IOAPIC:          on
BIOS APIC mode:  APIC
Time offset:     0ms
RTC:             local time
Hardw. virt.ext: on
Nested Paging:   on
Large Pages:     off
VT-x VPID:       on
VT-x unr. exec.: on
Paravirt. Provider: Default
Effective Paravirt. Provider: KVM
State:           powered off (since 2016-11-09T19:48:36.000000000)
Monitor count:   1
3D Acceleration: off
2D Video Acceleration: off
Teleporter Enabled: off
Teleporter Port: 0
Teleporter Address:
Teleporter Password:
Tracing Enabled: off
Allow Tracing to Access VM: off
Tracing Configuration:
Autostart Enabled: off
Autostart Delay: 0
Default Frontend:
Storage Controller Name (0):            IDE Controller
Storage Controller Type (0):            PIIX4
Storage Controller Instance Number (0): 0
Storage Controller Max Port Count (0):  2
Storage Controller Port Count (0):      2
Storage Controller Bootable (0):        on
Storage Controller Name (1):            SATA Controller
Storage Controller Type (1):            IntelAhci
Storage Controller Instance Number (1): 0
Storage Controller Max Port Count (1):  30
Storage Controller Port Count (1):      5
Storage Controller Bootable (1):        on
IDE Controller (1, 0): /vbox-repo1/isofiles/openfileresa-2.99.1-x86_64-disc1.iso (UUID: c7402463-c2c5-40fe-924b-3a02a183fab0)
SATA Controller (1, 0): /vbox-repo1/disks/openfiler1_localOSdisk1.vdi (UUID: fb8e0303-83c8-4712-a487-e7e50b7fc1d5)
SATA Controller (2, 0): /vbox-repo1/disks/openfiler1_functionaldisk1.vdi (UUID: 633fffa6-6812-485f-8e30-84d4082a7e81)
NIC 1:           MAC: 08002780201C, Attachment: Bridged Interface 'wlx98ded00b5b05', Cable connected: on, Trace: off (file: none), Type: 82540EM, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: den
y, Bandwidth group: none
NIC 2:           MAC: 080027782E66, Attachment: Internal Network 'storage-internal', Cable connected: on, Trace: off (file: none), Type: 82540EM, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: den
y, Bandwidth group: none

NIC 3:           disabled

........ lines removed for better clarity ........

NIC 36:           disabled
Pointing Device: PS/2 Mouse
Keyboard Device: PS/2 Keyboard
UART 1:          disabled
UART 2:          disabled
UART 3:          disabled
UART 4:          disabled
LPT 1:           disabled
LPT 2:           disabled
Audio:           enabled (Driver: PulseAudio, Controller: AC97, Codec: STAC9700)
Clipboard Mode:  Bidirectional
Drag and drop Mode: Bidirectional
VRDE:            enabled (Address 0.0.0.0, Ports 5010-5020, MultiConn: off, ReuseSingleConn: off, Authentication type: null)
Video redirection: disabled
VRDE property: TCP/Ports  = "5010-5020"
VRDE property: TCP/Address = <not set="">
VRDE property: VideoChannel/Enabled = <not set="">
VRDE property: VideoChannel/Quality = <not set="">
VRDE property: VideoChannel/DownscaleProtection = <not set="">
VRDE property: Client/DisableDisplay = <not set="">
VRDE property: Client/DisableInput = <not set="">
VRDE property: Client/DisableAudio = <not set="">
VRDE property: Client/DisableUSB = <not set="">
VRDE property: Client/DisableClipboard = <not set="">
VRDE property: Client/DisableUpstreamAudio = <not set="">
VRDE property: Client/DisableRDPDR = <not set="">
VRDE property: H3DRedirect/Enabled = <not set="">
VRDE property: Security/Method = <not set="">
VRDE property: Security/ServerCertificate = <not set="">
VRDE property: Security/ServerPrivateKey = <not set="">
VRDE property: Security/CACertificate = <not set="">
VRDE property: Audio/RateCorrectionMode = <not set="">
VRDE property: Audio/LogPath = <not set="">
USB:             disabled
EHCI:            disabled
XHCI:            disabled

USB Device Filters:

<none>

Bandwidth groups:  <none>

Shared folders:  <none>

Video capturing:    not active
Capture screens:    0
Capture file:       /vbox-repo1/metadata/Openfilers/openfiler1/openfiler1.webm
Capture dimensions: 1024x768
Capture rate:       512 kbps
Capture FPS:        25

Guest:

Configured memory balloon size:      0 MB

You can also view VM details in Virtualbox Manager GUI

virtualmachineopenfiler1correctmemory

VM for openfiler is ready now

We can treat this openfiler installation as a template. In the next part of the series I am going to install openfiler software on this VM and then there is a chance that it can be cloned easily as openfiler2 - instead of instaling openfiler2 from scratch.

 
 
Configure VM for oracle RAC node
 
  • Create oel72rac1n1 virtual machine. Notice the name is different than the one in the specification tables. oel72rac1n1 was an initial name, but as you will see this is not a good name so eventually it will have been changed by the end of the installation.
vboxuser1@macieksrv:~$ vboxmanage createvm --name oel72rac1n1 --groups "/RacLab1" --ostype Oracle_64 --basefolder /vbox-repo1/metadata --register
Virtual machine 'oel72rac1n1' is created and registered.
UUID: 9d9f28c6-023b-438b-b429-d70373246640
Settings file: '/vbox-repo1/metadata/RacLab1/oel72rac1n1/oel72rac1n1.vbox'
  • List VMs already created
vboxuser1@macieksrv:~$ VBoxManage list vms
"openfiler1" {002ac87b-755b-46e5-897d-35ebbd5f70f2}
"oel72rac1n1" {9d9f28c6-023b-438b-b429-d70373246640}
  • Download OEL 7.2 ISO from edelivery.oracle.com

Download and save it into the directory "/vbox-repo1/isofiles/". Full path to the file should be "/vbox-repo1/isofiles/V100082-01.iso"

  • Prepare VM for oel72racn1 using modifyvm option of vboxmanage tool

At the beginning let's set some basic settings for VM

# allow VM to hotplug CPUs 
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --cpuhotplug on

# set maximum number of cpus allowed for this VM (4xCPU)
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --cpus 4

# set starting value for this vm (2xCPU), so pin cpu 1 (0 by default)
#vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --plugcpu 1

# set RAM and videoram
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --vram 256 --memory 6144

# set modern chipset type and ACPI and IOAPIC
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --chipset ich9 --acpi on  --pagefusion on --ioapic on
  • Prepare storage for oracle rac node.

Start with creating disks, one for local OS and the other one for iSCSI disks sharing

# create local disk image
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage createmedium disk --filename /vbox-repo1/disks/oel72rac1n1_localOSdisk1.vdi --size 51200
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Medium created. UUID: be004bfa-0069-452c-b9b5-2448a988cd3b

# let's check the disk
vboxuser1@macieksrv.dba24.pl ~ $ ls -l /vbox-repo1/disks/oel72rac1n1_localOSdisk1.vdi
-rw------- 1 vboxuser1 vboxusers 2097152 lis 10 21:16 /vbox-repo1/disks/oel72rac1n1_localOSdisk1.vdi

As you can see above the disks is not of the fixed size, it is dynamic and will grow accordingly to the usage, its performance is lower but it saves space.

Attach storage IDE controller to the VM and stick installation iso of OEL 7.2 to it as Slave Master (port 1 device 0)

vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage storagectl "oel72rac1n1" --name "IDE Controller" --add ide --portcount 2

vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage storageattach "oel72rac1n1" --storagectl "IDE Controller"  --port 1 --device 0 --type dvddrive --medium /vbox-repo1/isofiles/V100082-01.iso

Now attach created VDI image disk to the SATA controller (first create SATA controller)

vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage storagectl "oel72rac1n1" --name "SATA Controller" --add sata --portcount 5

vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage storageattach "oel72rac1n1" --storagectl "SATA Controller"  --port 1 --type hdd --medium /vbox-repo1/disks/oel72rac1n1_localOSdisk1.vdi

Finally check the disk

vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage showmediuminfo disk /vbox-repo1/disks/oel72rac1n1_localOSdisk1.vdi
UUID:           be004bfa-0069-452c-b9b5-2448a988cd3b
Parent UUID:    base
State:          created
Type:           normal (base)
Location:       /vbox-repo1/disks/oel72rac1n1_localOSdisk1.vdi
Storage format: VDI
Format variant: dynamic default
Capacity:       51200 MBytes
Size on disk:   2 MBytes
Encryption:     disabled
In use by VMs:  oel72rac1n1 (UUID: 9d9f28c6-023b-438b-b429-d70373246640)

Disk has been properly assigned to the oel72rac1n1 VM

  • Set boot device to dvd in order to boot from openfiler's iso during first startup
# set DVD as first boot device, disk as second
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --boot1 dvd --boot2 disk
  • Configure network accordingly to the table with the network spec for oracle rac nodes.
    You can find the specification at the very beginning of this article
# Add first bridged network card and fix it to physical wlan adapter on host (wlx98ded00b5b05)
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --nic1 bridged --bridgeadapter1 wlx98ded00b5b05

# Add network cards and additional internal networks for priv (interconnect) and apriv (asm network) and also for storage network
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --nic2 intnet --intnet2 storage-internal
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --nic3 intnet --intnet3 priv1-internal
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --nic4 intnet --intnet4 priv2-internal
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --nic5 intnet --intnet5 apriv1-internal
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --nic6 intnet --intnet6 apriv2-internal

  • Configure snapshots for VM
# Specify folder for snapshots - first create it
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --snapshotfolder /vbox-repo1/snapshots/
  • Configure bidirectional clipboard, drag and drop and Virtualbox Remote Desktop
# set clipboard and drag and drop to be bidirectional between host and VM
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --clipboard bidirectional --draganddrop bidirectional

# Enable Virtual Remote Desktop on this VM
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage modifyvm "oel72rac1n1" --vrde on --vrdeport 5010-5020
  • Let's check settings of this VM after configuration has been completed
vboxuser1@macieksrv.dba24.pl ~ $ vboxmanage showvminfo "oel72rac1n1"|more
Name:            oel72rac1n1
Groups:          /RacLab1
Guest OS:        Oracle (64-bit)
UUID:            9d9f28c6-023b-438b-b429-d70373246640
Config file:     /vbox-repo1/metadata/RacLab1/oel72rac1n1/oel72rac1n1.vbox
Snapshot folder: /vbox-repo1/snapshots
Log folder:      /vbox-repo1/metadata/RacLab1/oel72rac1n1/Logs
Hardware UUID:   9d9f28c6-023b-438b-b429-d70373246640
Memory size:     6144MB
Page Fusion:     on
VRAM size:       256MB
CPU exec cap:    100%
HPET:            off
Chipset:         ich9
Firmware:        BIOS
Number of CPUs:  4
PAE:             on
Long Mode:       on
Triple Fault Reset: off
APIC:            on
X2APIC:          on
CPUID Portability Level: 0
CPUID overrides: None
Boot menu mode:  message and menu
Boot Device (1): DVD
Boot Device (2): HardDisk
Boot Device (3): HardDisk
Boot Device (4): Not Assigned
ACPI:            on
IOAPIC:          on
BIOS APIC mode:  APIC
Time offset:     0ms
RTC:             local time
Hardw. virt.ext: on
Nested Paging:   on
Large Pages:     off
VT-x VPID:       on
VT-x unr. exec.: on
Paravirt. Provider: Default
Effective Paravirt. Provider: KVM
State:           powered off (since 2016-11-09T19:49:01.000000000)
Monitor count:   1
3D Acceleration: off
2D Video Acceleration: off
Teleporter Enabled: off
Teleporter Port: 0
Teleporter Address:
Teleporter Password:
Tracing Enabled: off
Allow Tracing to Access VM: off
Tracing Configuration:
Autostart Enabled: off
Autostart Delay: 0
Default Frontend:
Storage Controller Name (0):            IDE Controller
Storage Controller Type (0):            PIIX4
Storage Controller Instance Number (0): 0
Storage Controller Max Port Count (0):  2
Storage Controller Port Count (0):      2
Storage Controller Bootable (0):        on
Storage Controller Name (1):            SATA Controller
Storage Controller Type (1):            IntelAhci
Storage Controller Instance Number (1): 0
Storage Controller Max Port Count (1):  30
Storage Controller Port Count (1):      5
Storage Controller Bootable (1):        on
IDE Controller (1, 0): /vbox-repo1/isofiles/V100082-01.iso (UUID: 48419f91-ace0-4e22-afda-1ba7a2b70b77)
SATA Controller (1, 0): /vbox-repo1/disks/oel72rac1n1_localOSdisk1.vdi (UUID: be004bfa-0069-452c-b9b5-2448a988cd3b)
NIC 1:           MAC: 0800276A189E, Attachment: Bridged Interface 'wlx98ded00b5b05', Cable connected: on, Trace: off (file: none), Type: 82540EM, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: den
y, Bandwidth group: none
NIC 2:           MAC: 080027DB1740, Attachment: Internal Network 'storage-internal', Cable connected: on, Trace: off (file: none), Type: 82540EM, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: den
y, Bandwidth group: none
NIC 3:           MAC: 080027D59C97, Attachment: Internal Network 'priv1-internal', Cable connected: on, Trace: off (file: none), Type: 82540EM, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: deny,
 Bandwidth group: none
NIC 4:           MAC: 080027AE9147, Attachment: Internal Network 'priv2-internal', Cable connected: on, Trace: off (file: none), Type: 82540EM, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: deny,
 Bandwidth group: none
NIC 5:           MAC: 08002748771B, Attachment: Internal Network 'apriv1-internal', Cable connected: on, Trace: off (file: none), Type: 82540EM, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: deny
, Bandwidth group: none
NIC 6:           MAC: 0800271A1EE2, Attachment: Internal Network 'apriv2-internal', Cable connected: on, Trace: off (file: none), Type: 82540EM, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: deny
, Bandwidth group: none
Pointing Device: PS/2 Mouse
Keyboard Device: PS/2 Keyboard
UART 1:          disabled
UART 2:          disabled
UART 3:          disabled
UART 4:          disabled
LPT 1:           disabled
LPT 2:           disabled
Audio:           enabled (Driver: PulseAudio, Controller: AC97, Codec: STAC9700)
Clipboard Mode:  Bidirectional
Drag and drop Mode: Bidirectional
VRDE:            enabled (Address 0.0.0.0, Ports 5010-5020, MultiConn: off, ReuseSingleConn: off, Authentication type: null)
Video redirection: disabled
VRDE property: TCP/Ports  = "5010-5020"
VRDE property: TCP/Address = <not set="">
VRDE property: VideoChannel/Enabled = <not set="">
VRDE property: VideoChannel/Quality = <not set="">
VRDE property: VideoChannel/DownscaleProtection = <not set="">
VRDE property: Client/DisableDisplay = <not set="">
VRDE property: Client/DisableInput = <not set="">
VRDE property: Client/DisableAudio = <not set="">
VRDE property: Client/DisableUSB = <not set="">
VRDE property: Client/DisableClipboard = <not set="">
VRDE property: Client/DisableUpstreamAudio = <not set="">
VRDE property: Client/DisableRDPDR = <not set="">
VRDE property: H3DRedirect/Enabled = <not set="">
VRDE property: Security/Method = <not set="">
VRDE property: Security/ServerCertificate = <not set="">
VRDE property: Security/ServerPrivateKey = <not set="">
VRDE property: Security/CACertificate = <not set="">
VRDE property: Audio/RateCorrectionMode = <not set="">
VRDE property: Audio/LogPath = <not set="">
USB:             disabled
EHCI:            disabled
XHCI:            disabled

USB Device Filters:

<none>

Bandwidth groups:  <none>

Shared folders:  <none>

Video capturing:    not active
Capture screens:    0
Capture file:       /vbox-repo1/metadata/RacLab1/oel72rac1n1/oel72rac1n1.webm
Capture dimensions: 1024x768
Capture rate:       512 kbps
Capture FPS:        25

Guest:

Configured memory balloon size:      0 MB

You can also view VM details in Virtualbox Manager GUI

oracleracnode1afterconfiguration

VM for oracle RAC node is ready now

We can treat this VM installation as a template of a pure VM without OS, but in this tutorial I will install OEL and prepare OS for RAC installation and then will use this VM as complete ready-to-clone template for new nodes. In the next parts of the series I am going to install OEL 7,2 software on this VM and set OS accordingly to Oracle requirements for 12c GI and RAC databases.

What do you think?? Please post your comments :)
Source: My experience and WorlWideWeb

About the author

 
maciej tokar
Maciej Tokar

An Oracle technology geek and crazy long distance runner, DBA24 Owner
Senior Oracle DBA / Consultant / [OCP10g, OCP12c, OCE RAC 10g] / [experience: 9y+]
Currently working for Bluegarden (Oslo Norway) by Miratech Group
Past: Mastercard / Trevica by Britenet, Citi International PLC, PZU

 
View Maciej Tokar's profile on LinkedIn         logoDB4
LinkedIn Auto Publish Powered By : XYZScripts.com