This is eleventh part of my RAC lab series. I will show you step by step how to install Oracle Database 12c RAC software

RAC lab Part 1 – Installing the Ubuntu 16.04 desktop
RAC lab Part 2 – Virtualbox installation and configuration
RAC lab Part 3 – VMs configuration
RAC lab Part 4 – Installing the Openfiler software
RAC lab Part 5 – Installing OEL 7.x on VM
RAC lab Part 6 – OEL 7.x configuration for Oracle 12c GI & RAC DB installation
RAC lab Part 7 – Openfilers configuration and cloning
RAC lab Part 8 – OEL 7.x prepare storage
RAC lab Part 9 – Clone first RAC node as 2nd node and prepare config.
RAC lab Part 10 – Installing Grid Infrastructure
RAC lab Part 11 – Installing Database Software
RAC lab Part 12 – Creating a Container Database

 
 
Oracle 12c official documentation
 

Please refer to the official 12.1 documentation and CLUVFY documentation:

 
 
 
 
Cluster verification before database software installation – CLUVFY
 
  • Verify if system is ready for database software installation: cluvfy stage -pre dbinst

Use the cluvfy stage -pre dbinst command to check the specified nodes before installing or creating an Oracle RAC database to verify that your system meets all of the criteria for installing or creating an Oracle RAC database.

[oracle@oel7rac1n1 bin]$ ./cluvfy stage -pre dbinst -n oel7rac1n1,oel7rac1n2 -r 12.1

Performing pre-checks for database installation

Checking node reachability...
Node reachability check passed from node "oel7rac1n1"


Checking user equivalence...
User equivalence check passed for user "oracle"

Checking node connectivity...

Checking hosts config file...

Verification of the hosts config file successful

Check: Node connectivity using interfaces on subnet "192.168.30.0"
Node connectivity passed for subnet "192.168.30.0" with node(s) oel7rac1n1,oel7rac1n2
TCP connectivity check passed for subnet "192.168.30.0"


Check: Node connectivity using interfaces on subnet "192.168.1.0"
Node connectivity passed for subnet "192.168.1.0" with node(s) oel7rac1n2,oel7rac1n1
TCP connectivity check passed for subnet "192.168.1.0"


Check: Node connectivity using interfaces on subnet "192.168.20.0"
Node connectivity passed for subnet "192.168.20.0" with node(s) oel7rac1n2,oel7rac1n1
TCP connectivity check passed for subnet "192.168.20.0"


Check: Node connectivity using interfaces on subnet "192.168.21.0"
Node connectivity passed for subnet "192.168.21.0" with node(s) oel7rac1n2,oel7rac1n1
TCP connectivity check passed for subnet "192.168.21.0"


Check: Node connectivity using interfaces on subnet "192.168.31.0"
Node connectivity passed for subnet "192.168.31.0" with node(s) oel7rac1n2,oel7rac1n1
TCP connectivity check passed for subnet "192.168.31.0"

Checking subnet mask consistency...
Subnet mask consistency check passed for subnet "192.168.1.0".
Subnet mask consistency check passed for subnet "192.168.20.0".
Subnet mask consistency check passed for subnet "192.168.21.0".
Subnet mask consistency check passed for subnet "192.168.30.0".
Subnet mask consistency check passed for subnet "192.168.31.0".
Subnet mask consistency check passed.
Node connectivity check passed

Checking multicast communication...

Checking subnet "192.168.20.0" for multicast communication with multicast group "224.0.0.251"...
Check of subnet "192.168.20.0" for multicast communication with multicast group "224.0.0.251" passed.

Checking subnet "192.168.21.0" for multicast communication with multicast group "224.0.0.251"...
Check of subnet "192.168.21.0" for multicast communication with multicast group "224.0.0.251" passed.

Checking subnet "192.168.30.0" for multicast communication with multicast group "224.0.0.251"...
Check of subnet "192.168.30.0" for multicast communication with multicast group "224.0.0.251" passed.

Checking subnet "192.168.31.0" for multicast communication with multicast group "224.0.0.251"...
Check of subnet "192.168.31.0" for multicast communication with multicast group "224.0.0.251" passed.

Check of multicast communication passed.
Total memory check passed
Available memory check passed
Swap space check passed
Free disk space check passed for "oel7rac1n2:/tmp"
Free disk space check passed for "oel7rac1n1:/tmp"
Check for multiple users with UID value 2001 passed
User existence check passed for "oracle"
Group existence check passed for "oinstall"
Group existence check passed for "dba"
Group existence check passed for "asmdba"
Membership check for user "oracle" in group "oinstall" [as Primary] passed
Membership check for user "oracle" in group "dba" passed
Membership check for user "oracle" in group "asmdba" passed
Run level check passed
Hard limits check passed for "maximum open file descriptors"
Soft limits check passed for "maximum open file descriptors"
Hard limits check passed for "maximum user processes"
Soft limits check passed for "maximum user processes"
System architecture check passed
Kernel version check passed
Kernel parameter check passed for "semmsl"
Kernel parameter check passed for "semmns"
Kernel parameter check passed for "semopm"
Kernel parameter check passed for "semmni"
Kernel parameter check passed for "shmmax"
Kernel parameter check passed for "shmmni"
Kernel parameter check passed for "shmall"
Kernel parameter check passed for "file-max"
Kernel parameter check passed for "ip_local_port_range"
Kernel parameter check passed for "rmem_default"
Kernel parameter check passed for "rmem_max"
Kernel parameter check passed for "wmem_default"
Kernel parameter check passed for "wmem_max"
Kernel parameter check passed for "aio-max-nr"
Package existence check passed for "binutils"
Package existence check passed for "compat-libcap1"
Package existence check passed for "compat-libstdc++-33(x86_64)"
Package existence check passed for "libgcc(x86_64)"
Package existence check passed for "libstdc++(x86_64)"
Package existence check passed for "libstdc++-devel(x86_64)"
Package existence check passed for "sysstat"
Package existence check passed for "gcc"
Package existence check passed for "gcc-c++"
Package existence check passed for "ksh"
Package existence check passed for "make"
Package existence check passed for "glibc(x86_64)"
Package existence check passed for "glibc-devel(x86_64)"
Package existence check passed for "libaio(x86_64)"
Package existence check passed for "libaio-devel(x86_64)"
Check for multiple users with UID value 0 passed
Current group ID check passed

Starting check for consistency of primary group of root user

Check for consistency of root user's primary group passed
Default user file creation mask check passed

Checking CRS integrity...

Clusterware version consistency passed.

CRS integrity check passed

Checking Cluster manager integrity... 


Checking CSS daemon...
Oracle Cluster Synchronization Services appear to be online.

Cluster manager integrity check passed


Checking node application existence...

Checking existence of VIP node application (required)
VIP node application check passed

Checking existence of NETWORK node application (required)
NETWORK node application check passed

Checking existence of ONS node application (optional)
ONS node application check passed

Oracle Clusterware is installed on all nodes.
CTSS resource check passed
Query of CTSS for time offset passed

CTSS is in Active state. Proceeding with check of clock time offsets on all nodes...
Check of clock time offsets passed
Oracle Cluster Time Synchronization Services check passed
Checking integrity of file "/etc/resolv.conf" across nodes

"domain" and "search" entries do not coexist in any "/etc/resolv.conf" file
The DNS response time for an unreachable node is within acceptable limit on all nodes
checking DNS response from all servers in "/etc/resolv.conf"

Check for integrity of file "/etc/resolv.conf" passed

Time zone consistency check passed

Checking Single Client Access Name (SCAN)...

Checking TCP connectivity to SCAN listeners...
TCP connectivity to SCAN listeners exists on all cluster nodes

Checking name resolution setup for "rac1-scan.dba24.pl"...

Checking integrity of name service switch configuration file "/etc/nsswitch.conf" ...
All nodes have same "hosts" entry defined in file "/etc/nsswitch.conf"
Check for integrity of name service switch configuration file "/etc/nsswitch.conf" passed


Checking SCAN IP addresses...
Check of SCAN IP addresses passed

Verification of SCAN VIP and listener setup passed
Checking VIP configuration.
Checking VIP Subnet configuration.
Check for VIP Subnet configuration passed.
Checking VIP reachability
Check for VIP reachability passed.


ASM and CRS versions are compatible
Database Clusterware version compatibility passed.
Group existence check passed for "asmadmin"
Membership check for user "grid" in group "asmadmin" passed
Group existence check passed for "asmoper"
Membership check for user "grid" in group "asmoper" passed
Group existence check passed for "asmdba"
Membership check for user "grid" in group "asmdba" passed
Membership check for user "grid" in group "asmdba" passed

Starting check for /dev/shm mounted as temporary file system ...

Check for /dev/shm mounted as temporary file system passed

Starting check for Maximum locked memory check ...

ERROR: 

PRVE-0044 : No entry in configuration file when checking locked memory limit on node "oel7rac1n2"
PRVE-0044 : No entry in configuration file when checking locked memory limit on node "oel7rac1n1"

Check for Maximum locked memory check failed

Pre-check for database installation was unsuccessful on all the nodes. 

# All ok except of

ERROR: 

PRVE-0044 : No entry in configuration file when checking locked memory limit on node "oel7rac1n2"
PRVE-0044 : No entry in configuration file when checking locked memory limit on node "oel7rac1n1"

Check for Maximum locked memory check failed

Check for Maximum locked memory check failed clufy is not aware of the new directories layout
It searches for parameters in file /etc/security/limits.conf but does not look inside files in /etc/security/limits.d/ directory.

And we got there file with required lines :

[root@oel7rac1n1 ~]# cat /etc/security/limits.d/99-oracle.conf
@oinstall        soft     memlock  5662310
@oinstall        hard     memlock  5662310

We are ready to install db software.

 
 
 
OUI Database Software Installation
 
  • Run OUI installer


 	 	maciek@macieksrv.dba24.pl ~ $ ssh -X -Y oracle@192.168.1.21

 	 	[oracle@oel7rac1n1 bin]$ cd /u01/app/oracle/oracle-software/database/

 	 	[oracle@oel7rac1n1 database]$ ./runInstaller

 	 	Starting Oracle Universal Installer...

 	 	Checking Temp space: must be greater than 500 MB. Actual 14378 MB Passed

 	 	Checking swap space: must be greater than 150 MB. Actual 16868 MB Passed

 	 	Checking monitor: must be configured to display at least 256 colors. Actual 16777216 Passed

 	 	Preparing to launch Oracle Universal Installer from /tmp/OraInstall2016-12-05_03-14-01PM. Please wait ...

 	 	
 
  • I usually skip this part

1-step-1-screenshot-from-2016-12-05-15-24-42

  • We want to install software only without creating a new database in this step

2-step-2-screenshot-from-2016-12-05-15-24-59

  • Choose RAC database

3-step-3-screenshot-from-2016-12-05-15-25-04

  • Verify if ssh user equivalency works properly

4-step-4-screenshot-from-2016-12-05-15-25-17

  • Select languages to install

5-step-5-screenshot-from-2016-12-05-15-25-30

  • Enterprise edition

6-step-6-screenshot-from-2016-12-05-15-25-36

  • Specify installation directories

7-step-7a-screenshot-from-2016-12-05-15-25-59

  • Dirs being checked

8-step-7b-screenshot-from-2016-12-05-15-26-04

  • Select os groups for different roles (leave default)

9-step-8-screenshot-from-2016-12-05-15-26-19

  • Prerequists being checked

10-step-9a-screenshot-from-2016-12-05-15-26-26

  • Verify and skip if not applicable

11-step-9b-screenshot-from-2016-12-05-15-27-30

  • Memorylocked has been set properly – ignore

12-step-9c-screenshot-from-2016-12-05-15-27-51

  • Check ignore all after you verified OUI complains

13-step-9d-screenshot-from-2016-12-05-15-28-00

  • Confirm you want to ignore some warnings

14-step-9e-screenshot-from-2016-12-05-15-28-08

  • Installation summary screen

15-step-10-screenshot-from-2016-12-05-15-28-25

  • Run root scripts when asked

16-step-11-screenshot-from-2016-12-05-16-05-09

 

 	 	[root@oel7rac1n1 ~]# /u01/app/oracle/product/12.1.0/dbhome_1/root.sh

 	 	Performing root user operation.

 	 	The following environment variables are set as:

 	 	 ORACLE_OWNER= oracle

 	 	 ORACLE_HOME= /u01/app/oracle/product/12.1.0/dbhome_1

 	 	Enter the full pathname of the local bin directory: [/usr/local/bin]:

 	 	The contents of "dbhome" have not changed. No need to overwrite.

 	 	The contents of "oraenv" have not changed. No need to overwrite.

 	 	The contents of "coraenv" have not changed. No need to overwrite.

 	 	Entries will be added to the /etc/oratab file as needed by

 	 	Database Configuration Assistant when a database is created

 	 	Finished running generic part of root script.

 	 	Now product-specific root actions will be performed.

 	 	[root@oel7rac1n2 ~]# /u01/app/oracle/product/12.1.0/dbhome_1/root.sh

 	 	Performing root user operation.

 	 	The following environment variables are set as:

 	 	 ORACLE_OWNER= oracle

 	 	 ORACLE_HOME= /u01/app/oracle/product/12.1.0/dbhome_1

 	 	Enter the full pathname of the local bin directory: [/usr/local/bin]:

 	 	The contents of "dbhome" have not changed. No need to overwrite.

 	 	The contents of "oraenv" have not changed. No need to overwrite.

 	 	The contents of "coraenv" have not changed. No need to overwrite.

 	 	Entries will be added to the /etc/oratab file as needed by

 	 	Database Configuration Assistant when a database is created

 	 	Finished running generic part of root script.

 	 	Now product-specific root actions will be performed.

 	 	
 
  • Installation completed.

17-step-12-screenshot-from-2016-12-05-16-06-45

 

Oracle Database RAC software successfully installed!!

 
 
 
What do you think?? Please post your comments :)
Source: My experience and WorlWideWeb

About the author

 
maciej tokar
Maciej Tokar

An Oracle technology geek and crazy long distance runner, DBA24 Owner
Senior Oracle DBA / Consultant / [OCP10g, OCP12c, OCE RAC 10g] / [experience: 9y+]
Currently working for Bluegarden (Oslo Norway) by Miratech Group
Past: Mastercard / Trevica by Britenet, Citi International PLC, PZU

 
View Maciej Tokar's profile on LinkedIn         logoDB4
LinkedIn Auto Publish Powered By : XYZScripts.com