Running cluvfy (Cluster Verification Utility) during 12c R2 RAC installation

Following is the output of cluster verification utility that I executed during 12c R2 RAC installation. You can see 2 examples in the following. Example 1 demonstrated if you don’t have passwordless SSH connectivity configured for “grid” and “oracle” users, so you will need to execute this from each node one by one. Following output is from one node only.

Example 1

Change directory to the location where GI software was extracted, and invoke “runcluvfy.sh” script.

[grid@salman11 grid]$ ./runcluvfy.sh stage -pre crsinst -n salman11 -fixup -verbose Verifying Physical Memory ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      3.6103GB (3785700.0KB)    8GB (8388608.0KB)         failed Verifying Physical Memory ...FAILED (PRVF-7530) Verifying Available Physical Memory ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      2.2215GB (2329424.0KB)    50MB (51200.0KB)          passed Verifying Available Physical Memory ...PASSED Verifying Swap Size ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      4GB (4194300.0KB)         3.6103GB (3785700.0KB)    passed Verifying Swap Size ...PASSED Verifying Free Space: salman11:/usr,salman11:/var,salman11:/etc,salman11:/sbin,salman11:/tmp ...   Path              Node Name     Mount point   Available     Required      Status   ----------------  ------------  ------------  ------------  ------------  ------------   /usr              salman11      /             39.9765GB     25MB          passed   /var              salman11      /             39.9765GB     5MB           passed   /etc              salman11      /             39.9765GB     25MB          passed   /sbin             salman11      /             39.9765GB     10MB          passed   /tmp              salman11      /             39.9765GB     1GB           passed Verifying Free Space: salman11:/usr,salman11:/var,salman11:/etc,salman11:/sbin,salman11:/tmp ...PASSED Verifying User Existence: grid ...   Node Name     Status                    Comment   ------------  ------------------------  ------------------------   salman11      passed                    exists(54322)   Verifying Users With Same UID: 54322 ...PASSED Verifying User Existence: grid ...PASSED Verifying Group Existence: asmadmin ...   Node Name     Status                    Comment   ------------  ------------------------  ------------------------   salman11      passed                    exists Verifying Group Existence: asmadmin ...PASSED Verifying Group Existence: asmdba ...   Node Name     Status                    Comment   ------------  ------------------------  ------------------------   salman11      passed                    exists Verifying Group Existence: asmdba ...PASSED Verifying Group Existence: oinstall ...   Node Name     Status                    Comment   ------------  ------------------------  ------------------------   salman11      passed                    exists Verifying Group Existence: oinstall ...PASSED Verifying Group Membership: asmdba ...   Node Name         User Exists   Group Exists  User in Group  Status   ----------------  ------------  ------------  ------------  ----------------   salman11          yes           yes           yes           passed Verifying Group Membership: asmdba ...PASSED Verifying Group Membership: asmadmin ...   Node Name         User Exists   Group Exists  User in Group  Status   ----------------  ------------  ------------  ------------  ----------------   salman11          yes           yes           yes           passed Verifying Group Membership: asmadmin ...PASSED Verifying Group Membership: oinstall(Primary) ...   Node Name         User Exists   Group Exists  User in Group  Primary       Status   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          yes           yes           yes           yes           passed Verifying Group Membership: oinstall(Primary) ...PASSED Verifying Run Level ...   Node Name     run level                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      5                         3,5                       passed Verifying Run Level ...PASSED Verifying Hard Limit: maximum open file descriptors ...   Node Name         Type          Available     Required      Status   ----------------  ------------  ------------  ------------  ----------------   salman11          hard          65536         65536         passed Verifying Hard Limit: maximum open file descriptors ...PASSED Verifying Soft Limit: maximum open file descriptors ...   Node Name         Type          Available     Required      Status   ----------------  ------------  ------------  ------------  ----------------   salman11          soft          65536         1024          passed Verifying Soft Limit: maximum open file descriptors ...PASSED Verifying Hard Limit: maximum user processes ...   Node Name         Type          Available     Required      Status   ----------------  ------------  ------------  ------------  ----------------   salman11          hard          16384         16384         passed Verifying Hard Limit: maximum user processes ...PASSED Verifying Soft Limit: maximum user processes ...   Node Name         Type          Available     Required      Status   ----------------  ------------  ------------  ------------  ----------------   salman11          soft          16384         2047          passed Verifying Soft Limit: maximum user processes ...PASSED Verifying Soft Limit: maximum stack size ...   Node Name         Type          Available     Required      Status   ----------------  ------------  ------------  ------------  ----------------   salman11          soft          10240         10240         passed Verifying Soft Limit: maximum stack size ...PASSED Verifying Architecture ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      x86_64                    x86_64                    passed Verifying Architecture ...PASSED Verifying OS Kernel Version ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      3.8.13-35.3.1.el7uek.x86_64  3.8.13                    passed Verifying OS Kernel Version ...PASSED Verifying OS Kernel Parameter: semmsl ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          250           250           250           passed Verifying OS Kernel Parameter: semmsl ...PASSED Verifying OS Kernel Parameter: semmns ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          32000         32000         32000         passed Verifying OS Kernel Parameter: semmns ...PASSED Verifying OS Kernel Parameter: semopm ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          100           100           100           passed Verifying OS Kernel Parameter: semopm ...PASSED Verifying OS Kernel Parameter: semmni ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          128           128           128           passed Verifying OS Kernel Parameter: semmni ...PASSED Verifying OS Kernel Parameter: shmmax ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          4398046511104  4398046511104  1938278400    passed Verifying OS Kernel Parameter: shmmax ...PASSED Verifying OS Kernel Parameter: shmmni ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          4096          4096          4096          passed Verifying OS Kernel Parameter: shmmni ...PASSED Verifying OS Kernel Parameter: shmall ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          1073741824    1073741824    378570        passed Verifying OS Kernel Parameter: shmall ...PASSED Verifying OS Kernel Parameter: file-max ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          6815744       6815744       6815744       passed Verifying OS Kernel Parameter: file-max ...PASSED Verifying OS Kernel Parameter: ip_local_port_range ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          between 9000 & 65500  between 9000 & 65500  between 9000 & 65535  passed Verifying OS Kernel Parameter: ip_local_port_range ...PASSED Verifying OS Kernel Parameter: rmem_default ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          262144        262144        262144        passed Verifying OS Kernel Parameter: rmem_default ...PASSED Verifying OS Kernel Parameter: rmem_max ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          4194304       4194304       4194304       passed Verifying OS Kernel Parameter: rmem_max ...PASSED Verifying OS Kernel Parameter: wmem_default ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          262144        262144        262144        passed Verifying OS Kernel Parameter: wmem_default ...PASSED Verifying OS Kernel Parameter: wmem_max ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          1048576       1048576       1048576       passed Verifying OS Kernel Parameter: wmem_max ...PASSED Verifying OS Kernel Parameter: aio-max-nr ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          1048576       1048576       1048576       passed Verifying OS Kernel Parameter: aio-max-nr ...PASSED Verifying OS Kernel Parameter: panic_on_oops ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          1             1             1             passed Verifying OS Kernel Parameter: panic_on_oops ...PASSED Verifying Package: binutils-2.23.52.0.1 ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      binutils-2.23.52.0.1-16.el7  binutils-2.23.52.0.1      passed Verifying Package: binutils-2.23.52.0.1 ...PASSED Verifying Package: compat-libcap1-1.10 ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      compat-libcap1-1.10-7.el7  compat-libcap1-1.10       passed Verifying Package: compat-libcap1-1.10 ...PASSED Verifying Package: libgcc-4.8.2 (x86_64) ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      libgcc(x86_64)-4.8.5-11.el7  libgcc(x86_64)-4.8.2      passed Verifying Package: libgcc-4.8.2 (x86_64) ...PASSED Verifying Package: libstdc++-4.8.2 (x86_64) ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      libstdc++(x86_64)-4.8.2-16.el7  libstdc++(x86_64)-4.8.2   passed Verifying Package: libstdc++-4.8.2 (x86_64) ...PASSED Verifying Package: libstdc++-devel-4.8.2 (x86_64) ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      libstdc++-devel(x86_64)-4.8.2-16.el7  libstdc++-devel(x86_64)-4.8.2  passed Verifying Package: libstdc++-devel-4.8.2 (x86_64) ...PASSED Verifying Package: sysstat-10.1.5 ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      sysstat-10.1.5-4.el7      sysstat-10.1.5            passed Verifying Package: sysstat-10.1.5 ...PASSED Verifying Package: ksh ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      ksh                       ksh                       passed Verifying Package: ksh ...PASSED Verifying Package: make-3.82 ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      make-3.82-21.el7          make-3.82                 passed Verifying Package: make-3.82 ...PASSED Verifying Package: glibc-2.17 (x86_64) ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      glibc(x86_64)-2.17-157.el7_3.1  glibc(x86_64)-2.17        passed Verifying Package: glibc-2.17 (x86_64) ...PASSED Verifying Package: glibc-devel-2.17 (x86_64) ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      glibc-devel(x86_64)-2.17-157.el7_3.1  glibc-devel(x86_64)-2.17  passed Verifying Package: glibc-devel-2.17 (x86_64) ...PASSED Verifying Package: libaio-0.3.109 (x86_64) ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      libaio(x86_64)-0.3.109-13.el7  libaio(x86_64)-0.3.109    passed Verifying Package: libaio-0.3.109 (x86_64) ...PASSED Verifying Package: libaio-devel-0.3.109 (x86_64) ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      libaio-devel(x86_64)-0.3.109-13.el7  libaio-devel(x86_64)-0.3.109  passed Verifying Package: libaio-devel-0.3.109 (x86_64) ...PASSED Verifying Package: nfs-utils-1.2.3-15 ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      nfs-utils-1.3.0-0.el7     nfs-utils-1.2.3-15        passed Verifying Package: nfs-utils-1.2.3-15 ...PASSED Verifying Package: smartmontools-6.2-4 ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      smartmontools-6.2-4.el7   smartmontools-6.2-4       passed Verifying Package: smartmontools-6.2-4 ...PASSED Verifying Package: net-tools-2.0-0.17 ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      net-tools-2.0-0.17.20131004git.el7  net-tools-2.0-0.17        passed Verifying Package: net-tools-2.0-0.17 ...PASSED Verifying Port Availability for component "Oracle Notification Service (ONS)" ...   Node Name         Port Number   Protocol      Available     Status   ----------------  ------------  ------------  ------------  ----------------   salman11          6200          TCP           yes           successful   salman11          6100          TCP           yes           successful Verifying Port Availability for component "Oracle Notification Service (ONS)" ...PASSED Verifying Port Availability for component "Oracle Cluster Synchronization Services (CSSD)" ...   Node Name         Port Number   Protocol      Available     Status   ----------------  ------------  ------------  ------------  ----------------   salman11          42424         TCP           yes           successful Verifying Port Availability for component "Oracle Cluster Synchronization Services (CSSD)" ...PASSED Verifying Users With Same UID: 0 ...PASSED Verifying Current Group ID ...PASSED Verifying Root user consistency ...   Node Name                             Status   ------------------------------------  ------------------------   salman11                              passed Verifying Root user consistency ...PASSED Verifying Package: cvuqdisk-1.0.10-1 ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman11      cvuqdisk-1.0.10-1         cvuqdisk-1.0.10-1         passed Verifying Package: cvuqdisk-1.0.10-1 ...PASSED Verifying Node Connectivity ...   Verifying Hosts File ...   Node Name                             Status   ------------------------------------  ------------------------   salman11                              passed   Verifying Hosts File ...PASSED Interface information for node "salman11"  Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU  ------ --------------- --------------- --------------- --------------- ----------------- ------  enp0s8 192.231.231.50  192.231.231.0   0.0.0.0         192.231.231.1   08:00:27:A4:2E:3B 1500  enp0s9 10.10.10.10     10.10.10.0      0.0.0.0         192.231.231.1   08:00:27:AC:CD:5E 1500 Check: MTU consistency of the subnet "192.231.231.0".   Node              Name          IP Address    Subnet        MTU   ----------------  ------------  ------------  ------------  ----------------   salman11          enp0s8        192.231.231.50  192.231.231.0  1500 Check: MTU consistency of the subnet "10.10.10.0".   Node              Name          IP Address    Subnet        MTU   ----------------  ------------  ------------  ------------  ----------------   salman11          enp0s9        10.10.10.10   10.10.10.0    1500   Verifying Check that maximum (MTU) size packet goes through subnet ...PASSED Verifying Node Connectivity ...PASSED Verifying Multicast check ... Checking subnet "192.231.231.0" for multicast communication with multicast group "224.0.0.251" Verifying Multicast check ...PASSED Verifying ASMLib installation and configuration verification. ...   Verifying '/etc/init.d/oracleasm' ...PASSED   Verifying '/dev/oracleasm' ...PASSED   Verifying '/etc/sysconfig/oracleasm' ...PASSED   Node Name                             Status   ------------------------------------  ------------------------   salman11                              passed Verifying ASMLib installation and configuration verification. ...PASSED Verifying Network Time Protocol (NTP) ...   Verifying '/etc/ntp.conf' ...   Node Name                             File exists?   ------------------------------------  ------------------------   salman11                              no   Verifying '/etc/ntp.conf' ...PASSED   Verifying '/etc/chrony.conf' ...   Node Name                             File exists?   ------------------------------------  ------------------------   salman11                              no   Verifying '/etc/chrony.conf' ...PASSED   Verifying '/var/run/ntpd.pid' ...   Node Name                             File exists?   ------------------------------------  ------------------------   salman11                              no   Verifying '/var/run/ntpd.pid' ...PASSED   Verifying '/var/run/chronyd.pid' ...   Node Name                             File exists?   ------------------------------------  ------------------------   salman11                              no   Verifying '/var/run/chronyd.pid' ...PASSED Verifying Network Time Protocol (NTP) ...PASSED Verifying Same core file name pattern ...PASSED Verifying User Mask ...   Node Name     Available                 Required                  Comment   ------------  ------------------------  ------------------------  ----------   salman11      0022                      0022                      passed Verifying User Mask ...PASSED Verifying User Not In Group "root": grid ...   Node Name     Status                    Comment   ------------  ------------------------  ------------------------   salman11      passed                    does not exist Verifying User Not In Group "root": grid ...PASSED Verifying Time zone consistency ...PASSED Verifying resolv.conf Integrity ...   Verifying (Linux) resolv.conf Integrity ...   Node Name                             Status   ------------------------------------  ------------------------   salman11                              failed   checking response for name "salman11" from each of the name servers specified   in "/etc/resolv.conf"   Node Name     Source                    Comment                   Status   ------------  ------------------------  ------------------------  ----------   salman11      192.231.231.1             IPv4                      failed   Verifying (Linux) resolv.conf Integrity ...FAILED (PRVF-5636, PRVG-10048) Verifying resolv.conf Integrity ...FAILED (PRVF-5636, PRVG-10048) Verifying DNS/NIS name service ...PASSED Verifying Domain Sockets ...PASSED Verifying /boot mount ...PASSED Verifying Daemon "avahi-daemon" not configured and running ...   Node Name     Configured                Status   ------------  ------------------------  ------------------------   salman11      no                        passed   Node Name     Running?                  Status   ------------  ------------------------  ------------------------   salman11      no                        passed Verifying Daemon "avahi-daemon" not configured and running ...PASSED Verifying Daemon "proxyt" not configured and running ...   Node Name     Configured                Status   ------------  ------------------------  ------------------------   salman11      no                        passed   Node Name     Running?                  Status   ------------  ------------------------  ------------------------   salman11      no                        passed Verifying Daemon "proxyt" not configured and running ...PASSED Verifying User Equivalence ...PASSED Verifying File system mount options for path /var ...PASSED Verifying zeroconf check ...PASSED Verifying ASM Filter Driver configuration ...PASSED Pre-check for cluster services setup was unsuccessful on all the nodes. Failures were encountered during execution of CVU verification request "stage -pre crsinst". Verifying Physical Memory ...FAILED salman11: PRVF-7530 : Sufficient physical memory is not available on node           "salman11" [Required physical memory = 8GB (8388608.0KB)] Verifying resolv.conf Integrity ...FAILED salman11: PRVF-5636 : The DNS response time for an unreachable node exceeded           "15000" ms on following nodes: salman11 salman11: PRVG-10048 : Name "salman11" was not resolved to an address of the           specified type by name servers o"192.231.231.1". salman11: Check for integrity of file "/etc/resolv.conf" failed   Verifying (Linux) resolv.conf Integrity ...FAILED   salman11: PRVF-5636 : The DNS response time for an unreachable node exceeded             "15000" ms on following nodes: salman11   salman11: PRVG-10048 : Name "salman11" was not resolved to an address of the             specified type by name servers o"192.231.231.1". CVU operation performed:      stage -pre crsinst Date:                         Apr 22, 2017 1:12:53 PM CVU home:                     /u01/app/12.2.0/grid/ User:                         grid

First “failure” detected by cluvfy because I have not sufficient amount of memory. Since I am on Virtualbox without ample amount of resources, so I ignored it.
Rest of the failures are because I don’t have a DNS in my virtual environment, and a valid DNS server entry in resolve.conf.
All of above can be ignored (for my case at least)

Example 2   

If we have password less SSH connectivity enabled, we can run cluvfy from one node to verify all RAC nodes. See following output.
Change directory to the location where GI software was extracted and invoke “runcluvfy.sh” script.

[grid@salman11 grid]$ ./runcluvfy.sh stage -pre crsinst -n salman11,salman12 -fixup -verbose Verifying Physical Memory ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      3.6104GB (3785812.0KB)    8GB (8388608.0KB)         failed   salman11      3.6104GB (3785812.0KB)    8GB (8388608.0KB)         failed Verifying Physical Memory ...FAILED (PRVF-7530) Verifying Available Physical Memory ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      2.7644GB (2898664.0KB)    50MB (51200.0KB)          passed   salman11      2.3254GB (2438344.0KB)    50MB (51200.0KB)          passed Verifying Available Physical Memory ...PASSED Verifying Swap Size ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      4GB (4194300.0KB)         3.6104GB (3785812.0KB)    passed   salman11      4GB (4194300.0KB)         3.6104GB (3785812.0KB)    passed Verifying Swap Size ...PASSED Verifying Free Space: salman12:/usr,salman12:/var,salman12:/etc,salman12:/sbin,salman12:/tmp ...   Path              Node Name     Mount point   Available     Required      Status   ----------------  ------------  ------------  ------------  ------------  ------------   /usr              salman12      /             53.5537GB     25MB          passed   /var              salman12      /             53.5537GB     5MB           passed   /etc              salman12      /             53.5537GB     25MB          passed   /sbin             salman12      /             53.5537GB     10MB          passed   /tmp              salman12      /             53.5537GB     1GB           passed Verifying Free Space: salman12:/usr,salman12:/var,salman12:/etc,salman12:/sbin,salman12:/tmp ...PASSED Verifying Free Space: salman11:/usr,salman11:/var,salman11:/etc,salman11:/sbin,salman11:/tmp ...   Path              Node Name     Mount point   Available     Required      Status   ----------------  ------------  ------------  ------------  ------------  ------------   /usr              salman11      /             39.9785GB     25MB          passed   /var              salman11      /             39.9785GB     5MB           passed   /etc              salman11      /             39.9785GB     25MB          passed   /sbin             salman11      /             39.9785GB     10MB          passed   /tmp              salman11      /             39.9785GB     1GB           passed Verifying Free Space: salman11:/usr,salman11:/var,salman11:/etc,salman11:/sbin,salman11:/tmp ...PASSED Verifying User Existence: grid ...   Node Name     Status                    Comment   ------------  ------------------------  ------------------------   salman12      passed                    exists(54322)   salman11      passed                    exists(54322)   Verifying Users With Same UID: 54322 ...PASSED Verifying User Existence: grid ...PASSED Verifying Group Existence: asmadmin ...   Node Name     Status                    Comment   ------------  ------------------------  ------------------------   salman12      passed                    exists   salman11      passed                    exists Verifying Group Existence: asmadmin ...PASSED Verifying Group Existence: asmdba ...   Node Name     Status                    Comment   ------------  ------------------------  ------------------------   salman12      passed                    exists   salman11      passed                    exists Verifying Group Existence: asmdba ...PASSED Verifying Group Existence: oinstall ...   Node Name     Status                    Comment   ------------  ------------------------  ------------------------   salman12      passed                    exists   salman11      passed                    exists Verifying Group Existence: oinstall ...PASSED Verifying Group Membership: asmdba ...   Node Name         User Exists   Group Exists  User in Group  Status   ----------------  ------------  ------------  ------------  ----------------   salman12          yes           yes           yes           passed   salman11          yes           yes           yes           passed Verifying Group Membership: asmdba ...PASSED Verifying Group Membership: asmadmin ...   Node Name         User Exists   Group Exists  User in Group  Status   ----------------  ------------  ------------  ------------  ----------------   salman12          yes           yes           yes           passed   salman11          yes           yes           yes           passed Verifying Group Membership: asmadmin ...PASSED Verifying Group Membership: oinstall(Primary) ...   Node Name         User Exists   Group Exists  User in Group  Primary       Status   ----------------  ------------  ------------  ------------  ------------  ------------   salman12          yes           yes           yes           yes           passed   salman11          yes           yes           yes           yes           passed Verifying Group Membership: oinstall(Primary) ...PASSED Verifying Run Level ...   Node Name     run level                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      5                         3,5                       passed   salman11      5                         3,5                       passed Verifying Run Level ...PASSED Verifying Hard Limit: maximum open file descriptors ...   Node Name         Type          Available     Required      Status   ----------------  ------------  ------------  ------------  ----------------   salman12          hard          65536         65536         passed   salman11          hard          65536         65536         passed Verifying Hard Limit: maximum open file descriptors ...PASSED Verifying Soft Limit: maximum open file descriptors ...   Node Name         Type          Available     Required      Status   ----------------  ------------  ------------  ------------  ----------------   salman12          soft          1024          1024          passed   salman11          soft          65536         1024          passed Verifying Soft Limit: maximum open file descriptors ...PASSED Verifying Hard Limit: maximum user processes ...   Node Name         Type          Available     Required      Status   ----------------  ------------  ------------  ------------  ----------------   salman12          hard          16384         16384         passed   salman11          hard          16384         16384         passed Verifying Hard Limit: maximum user processes ...PASSED Verifying Soft Limit: maximum user processes ...   Node Name         Type          Available     Required      Status   ----------------  ------------  ------------  ------------  ----------------   salman12          soft          16384         2047          passed   salman11          soft          16384         2047          passed Verifying Soft Limit: maximum user processes ...PASSED Verifying Soft Limit: maximum stack size ...   Node Name         Type          Available     Required      Status   ----------------  ------------  ------------  ------------  ----------------   salman12          soft          10240         10240         passed   salman11          soft          10240         10240         passed Verifying Soft Limit: maximum stack size ...PASSED Verifying Architecture ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      x86_64                    x86_64                    passed   salman11      x86_64                    x86_64                    passed Verifying Architecture ...PASSED Verifying OS Kernel Version ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      3.8.13-35.3.1.el7uek.x86_64  3.8.13                    passed   salman11      3.8.13-35.3.1.el7uek.x86_64  3.8.13                    passed Verifying OS Kernel Version ...PASSED Verifying OS Kernel Parameter: semmsl ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          250           250           250           passed   salman12          250           250           250           passed Verifying OS Kernel Parameter: semmsl ...PASSED Verifying OS Kernel Parameter: semmns ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          32000         32000         32000         passed   salman12          32000         32000         32000         passed Verifying OS Kernel Parameter: semmns ...PASSED Verifying OS Kernel Parameter: semopm ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          100           100           100           passed   salman12          100           100           100           passed Verifying OS Kernel Parameter: semopm ...PASSED Verifying OS Kernel Parameter: semmni ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          128           128           128           passed   salman12          128           128           128           passed Verifying OS Kernel Parameter: semmni ...PASSED Verifying OS Kernel Parameter: shmmax ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          4398046511104  4398046511104  1938335744    passed   salman12          4398046511104  4398046511104  1938335744    passed Verifying OS Kernel Parameter: shmmax ...PASSED Verifying OS Kernel Parameter: shmmni ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          4096          4096          4096          passed   salman12          4096          4096          4096          passed Verifying OS Kernel Parameter: shmmni ...PASSED Verifying OS Kernel Parameter: shmall ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          1073741824    1073741824    378581        passed   salman12          1073741824    1073741824    378581        passed Verifying OS Kernel Parameter: shmall ...PASSED Verifying OS Kernel Parameter: file-max ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          6815744       6815744       6815744       passed   salman12          6815744       6815744       6815744       passed Verifying OS Kernel Parameter: file-max ...PASSED Verifying OS Kernel Parameter: ip_local_port_range ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          between 9000 & 65500  between 9000 & 65500  between 9000 & 65535  passed   salman12          between 9000 & 65500  between 9000 & 65500  between 9000 & 65535  passed Verifying OS Kernel Parameter: ip_local_port_range ...PASSED Verifying OS Kernel Parameter: rmem_default ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          262144        262144        262144        passed   salman12          262144        262144        262144        passed Verifying OS Kernel Parameter: rmem_default ...PASSED Verifying OS Kernel Parameter: rmem_max ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          4194304       4194304       4194304       passed   salman12          4194304       4194304       4194304       passed Verifying OS Kernel Parameter: rmem_max ...PASSED Verifying OS Kernel Parameter: wmem_default ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          262144        262144        262144        passed   salman12          262144        262144        262144        passed Verifying OS Kernel Parameter: wmem_default ...PASSED Verifying OS Kernel Parameter: wmem_max ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          1048576       1048576       1048576       passed   salman12          1048576       1048576       1048576       passed Verifying OS Kernel Parameter: wmem_max ...PASSED Verifying OS Kernel Parameter: aio-max-nr ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          1048576       1048576       1048576       passed   salman12          1048576       1048576       1048576       passed Verifying OS Kernel Parameter: aio-max-nr ...PASSED Verifying OS Kernel Parameter: panic_on_oops ...   Node Name         Current       Configured    Required      Status        Comment   ----------------  ------------  ------------  ------------  ------------  ------------   salman11          1             1             1             passed   salman12          1             1             1             passed Verifying OS Kernel Parameter: panic_on_oops ...PASSED Verifying Package: binutils-2.23.52.0.1 ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      binutils-2.23.52.0.1-16.el7  binutils-2.23.52.0.1      passed   salman11      binutils-2.23.52.0.1-16.el7  binutils-2.23.52.0.1      passed Verifying Package: binutils-2.23.52.0.1 ...PASSED Verifying Package: compat-libcap1-1.10 ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      compat-libcap1-1.10-7.el7  compat-libcap1-1.10       passed   salman11      compat-libcap1-1.10-7.el7  compat-libcap1-1.10       passed Verifying Package: compat-libcap1-1.10 ...PASSED Verifying Package: libgcc-4.8.2 (x86_64) ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      libgcc(x86_64)-4.8.5-11.el7  libgcc(x86_64)-4.8.2      passed   salman11      libgcc(x86_64)-4.8.5-11.el7  libgcc(x86_64)-4.8.2      passed Verifying Package: libgcc-4.8.2 (x86_64) ...PASSED Verifying Package: libstdc++-4.8.2 (x86_64) ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      libstdc++(x86_64)-4.8.2-16.el7  libstdc++(x86_64)-4.8.2   passed   salman11      libstdc++(x86_64)-4.8.2-16.el7  libstdc++(x86_64)-4.8.2   passed Verifying Package: libstdc++-4.8.2 (x86_64) ...PASSED Verifying Package: libstdc++-devel-4.8.2 (x86_64) ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      libstdc++-devel(x86_64)-4.8.2-16.el7  libstdc++-devel(x86_64)-4.8.2  passed   salman11      libstdc++-devel(x86_64)-4.8.2-16.el7  libstdc++-devel(x86_64)-4.8.2  passed Verifying Package: libstdc++-devel-4.8.2 (x86_64) ...PASSED Verifying Package: sysstat-10.1.5 ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      sysstat-10.1.5-4.el7      sysstat-10.1.5            passed   salman11      sysstat-10.1.5-4.el7      sysstat-10.1.5            passed Verifying Package: sysstat-10.1.5 ...PASSED Verifying Package: ksh ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      ksh                       ksh                       passed   salman11      ksh                       ksh                       passed Verifying Package: ksh ...PASSED Verifying Package: make-3.82 ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      make-3.82-21.el7          make-3.82                 passed   salman11      make-3.82-21.el7          make-3.82                 passed Verifying Package: make-3.82 ...PASSED Verifying Package: glibc-2.17 (x86_64) ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      glibc(x86_64)-2.17-157.el7_3.1  glibc(x86_64)-2.17        passed   salman11      glibc(x86_64)-2.17-157.el7_3.1  glibc(x86_64)-2.17        passed Verifying Package: glibc-2.17 (x86_64) ...PASSED Verifying Package: glibc-devel-2.17 (x86_64) ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      glibc-devel(x86_64)-2.17-157.el7_3.1  glibc-devel(x86_64)-2.17  passed   salman11      glibc-devel(x86_64)-2.17-157.el7_3.1  glibc-devel(x86_64)-2.17  passed Verifying Package: glibc-devel-2.17 (x86_64) ...PASSED Verifying Package: libaio-0.3.109 (x86_64) ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      libaio(x86_64)-0.3.109-13.el7  libaio(x86_64)-0.3.109    passed   salman11      libaio(x86_64)-0.3.109-13.el7  libaio(x86_64)-0.3.109    passed Verifying Package: libaio-0.3.109 (x86_64) ...PASSED Verifying Package: libaio-devel-0.3.109 (x86_64) ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      libaio-devel(x86_64)-0.3.109-13.el7  libaio-devel(x86_64)-0.3.109  passed   salman11      libaio-devel(x86_64)-0.3.109-13.el7  libaio-devel(x86_64)-0.3.109  passed Verifying Package: libaio-devel-0.3.109 (x86_64) ...PASSED Verifying Package: nfs-utils-1.2.3-15 ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      nfs-utils-1.3.0-0.el7     nfs-utils-1.2.3-15        passed   salman11      nfs-utils-1.3.0-0.el7     nfs-utils-1.2.3-15        passed Verifying Package: nfs-utils-1.2.3-15 ...PASSED Verifying Package: smartmontools-6.2-4 ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      smartmontools-6.2-4.el7   smartmontools-6.2-4       passed   salman11      smartmontools-6.2-4.el7   smartmontools-6.2-4       passed Verifying Package: smartmontools-6.2-4 ...PASSED Verifying Package: net-tools-2.0-0.17 ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      net-tools-2.0-0.17.20131004git.el7  net-tools-2.0-0.17        passed   salman11      net-tools-2.0-0.17.20131004git.el7  net-tools-2.0-0.17        passed Verifying Package: net-tools-2.0-0.17 ...PASSED Verifying Port Availability for component "Oracle Notification Service (ONS)" ...   Node Name         Port Number   Protocol      Available     Status   ----------------  ------------  ------------  ------------  ----------------   salman12          6200          TCP           yes           successful   salman11          6200          TCP           yes           successful   salman12          6100          TCP           yes           successful   salman11          6100          TCP           yes           successful Verifying Port Availability for component "Oracle Notification Service (ONS)" ...PASSED Verifying Port Availability for component "Oracle Cluster Synchronization Services (CSSD)" ...   Node Name         Port Number   Protocol      Available     Status   ----------------  ------------  ------------  ------------  ----------------   salman12          42424         TCP           yes           successful   salman11          42424         TCP           yes           successful Verifying Port Availability for component "Oracle Cluster Synchronization Services (CSSD)" ...PASSED Verifying Users With Same UID: 0 ...PASSED Verifying Current Group ID ...PASSED Verifying Root user consistency ...   Node Name                             Status   ------------------------------------  ------------------------   salman12                              passed   salman11                              passed Verifying Root user consistency ...PASSED Verifying Package: cvuqdisk-1.0.10-1 ...   Node Name     Available                 Required                  Status   ------------  ------------------------  ------------------------  ----------   salman12      cvuqdisk-1.0.10-1         cvuqdisk-1.0.10-1         passed   salman11      cvuqdisk-1.0.10-1         cvuqdisk-1.0.10-1         passed Verifying Package: cvuqdisk-1.0.10-1 ...PASSED Verifying Node Connectivity ...   Verifying Hosts File ...   Node Name                             Status   ------------------------------------  ------------------------   salman11                              passed   salman12                              passed   Verifying Hosts File ...PASSED Interface information for node "salman11"  Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU  ------ --------------- --------------- --------------- --------------- ----------------- ------  enp0s8 192.231.231.50  192.231.231.0   0.0.0.0         192.231.231.1   08:00:27:A4:2E:3B 1500  enp0s9 10.10.10.10     10.10.10.0      0.0.0.0         192.231.231.1   08:00:27:AC:CD:5E 1500 Interface information for node "salman12"  Name   IP Address      Subnet          Gateway         Def. Gateway    HW Address        MTU  ------ --------------- --------------- --------------- --------------- ----------------- ------  enp0s8 192.231.231.51  192.231.231.0   0.0.0.0         192.231.231.1   08:00:27:1D:DB:E5 1500  enp0s9 10.10.10.11     10.10.10.0      0.0.0.0         192.231.231.1   08:00:27:BA:DB:5F 1500 Check: MTU consistency of the subnet "192.231.231.0".   Node              Name          IP Address    Subnet        MTU   ----------------  ------------  ------------  ------------  ----------------   salman11          enp0s8        192.231.231.50  192.231.231.0  1500   salman12          enp0s8        192.231.231.51  192.231.231.0  1500 Check: MTU consistency of the subnet "10.10.10.0".   Node              Name          IP Address    Subnet        MTU   ----------------  ------------  ------------  ------------  ----------------   salman11          enp0s9        10.10.10.10   10.10.10.0    1500   salman12          enp0s9        10.10.10.11   10.10.10.0    1500   Verifying Check that maximum (MTU) size packet goes through subnet ...PASSED   Source                          Destination                     Connected?   ------------------------------  ------------------------------  ----------------   salman11[enp0s8:192.231.231.50]  salman12[enp0s8:192.231.231.51]  yes   Source                          Destination                     Connected?   ------------------------------  ------------------------------  ----------------   salman11[enp0s9:10.10.10.10]    salman12[enp0s9:10.10.10.11]    yes   Verifying subnet mask consistency for subnet "192.231.231.0" ...PASSED   Verifying subnet mask consistency for subnet "10.10.10.0" ...PASSED Verifying Node Connectivity ...PASSED Verifying Multicast check ... Checking subnet "192.231.231.0" for multicast communication with multicast group "224.0.0.251" Verifying Multicast check ...PASSED Verifying ASMLib installation and configuration verification. ...   Verifying '/etc/init.d/oracleasm' ...PASSED   Verifying '/dev/oracleasm' ...PASSED   Verifying '/etc/sysconfig/oracleasm' ...PASSED   Node Name                             Status   ------------------------------------  ------------------------   salman11                              passed   salman12                              passed Verifying ASMLib installation and configuration verification. ...PASSED Verifying Network Time Protocol (NTP) ...   Verifying '/etc/ntp.conf' ...   Node Name                             File exists?   ------------------------------------  ------------------------   salman12                              no   salman11                              no   Verifying '/etc/ntp.conf' ...PASSED   Verifying '/etc/chrony.conf' ...   Node Name                             File exists?   ------------------------------------  ------------------------   salman12                              no   salman11                              no   Verifying '/etc/chrony.conf' ...PASSED   Verifying '/var/run/ntpd.pid' ...   Node Name                             File exists?   ------------------------------------  ------------------------   salman12                              no   salman11                              no   Verifying '/var/run/ntpd.pid' ...PASSED   Verifying '/var/run/chronyd.pid' ...   Node Name                             File exists?   ------------------------------------  ------------------------   salman12                              no   salman11                              no Verifying '/var/run/chronyd.pid' ...PASSED Verifying Network Time Protocol (NTP) ...PASSED Verifying Same core file name pattern ...PASSEDVerifying User Mask ...   Node Name     Available                 Required                  Comment   ------------  ------------------------  ------------------------  ----------   salman12      0022                      0022                      passed   salman11      0022                      0022                      passed Verifying User Mask ...PASSED Verifying User Not In Group "root": grid ...   Node Name     Status                    Comment   ------------  ------------------------  ------------------------   salman12      passed                    does not exist   salman11      passed                    does not exist Verifying User Not In Group "root": grid ...PASSED Verifying Time zone consistency ...PASSED Verifying resolv.conf Integrity ...   Verifying (Linux) resolv.conf Integrity ...   Node Name                             Status   ------------------------------------  ------------------------   salman11                              failed   salman12                              failed   checking response for name "salman12" from each of the name servers specified   in "/etc/resolv.conf"   Node Name     Source                    Comment                   Status   ------------  ------------------------  ------------------------  ----------   salman12      192.231.231.1             IPv4                      failed   checking response for name "salman11" from each of the name servers specified   in "/etc/resolv.conf"   Node Name     Source                    Comment                   Status   ------------  ------------------------  ------------------------  ----------   salman11      192.231.231.1             IPv4                      failed   Verifying (Linux) resolv.conf Integrity ...FAILED (PRVF-5636, PRVG-10048) Verifying resolv.conf Integrity ...FAILED (PRVF-5636, PRVG-10048) Verifying DNS/NIS name service ...PASSED Verifying Domain Sockets ...PASSED Verifying /boot mount ...PASSED Verifying Daemon "avahi-daemon" not configured and running ...   Node Name     Configured                Status   ------------  ------------------------  ------------------------   salman12      no                        passed   salman11      no                        passed   Node Name     Running?                  Status   ------------  ------------------------  ------------------------   salman12      no                        passed   salman11      no                        passed Verifying Daemon "avahi-daemon" not configured and running ...PASSED Verifying Daemon "proxyt" not configured and running ...   Node Name     Configured                Status   ------------  ------------------------  ------------------------   salman12      no                        passed   salman11      no                        passed   Node Name     Running?                  Status   ------------  ------------------------  ------------------------   salman12      no                        passed   salman11      no                        passed Verifying Daemon "proxyt" not configured and running ...PASSED Verifying User Equivalence ...PASSED Verifying File system mount options for path /var ...PASSED Verifying zeroconf check ...PASSED Verifying ASM Filter Driver configuration ...PASSED Pre-check for cluster services setup was unsuccessful on all the nodes. Failures were encountered during execution of CVU verification request "stage -pre crsinst". Verifying Physical Memory ...FAILED salman12: PRVF-7530 : Sufficient physical memory is not available on node           "salman12" [Required physical memory = 8GB (8388608.0KB)] salman11: PRVF-7530 : Sufficient physical memory is not available on node           "salman11" [Required physical memory = 8GB (8388608.0KB)] Verifying resolv.conf Integrity ...FAILED salman12: PRVF-5636 : The DNS response time for an unreachable node exceeded           "15000" ms on following nodes: salman11,salman12 salman12: PRVG-10048 : Name "salman12" was not resolved to an address of the           specified type by name servers o"192.231.231.1". salman12: Check for integrity of file "/etc/resolv.conf" failed salman11: PRVF-5636 : The DNS response time for an unreachable node exceeded           "15000" ms on following nodes: salman11,salman12 salman11: PRVG-10048 : Name "salman11" was not resolved to an address of the           specified type by name servers o"192.231.231.1". salman11: Check for integrity of file "/etc/resolv.conf" failed   Verifying (Linux) resolv.conf Integrity ...FAILED   salman12: PRVF-5636 : The DNS response time for an unreachable node exceeded             "15000" ms on following nodes: salman11,salman12   salman12: PRVG-10048 : Name "salman12" was not resolved to an address of the             specified type by name servers o"192.231.231.1".   salman11: PRVF-5636 : The DNS response time for an unreachable node exceeded             "15000" ms on following nodes: salman11,salman12   salman11: PRVG-10048 : Name "salman11" was not resolved to an address of the             specified type by name servers o"192.231.231.1". CVU operation performed:      stage -pre crsinst Date:                         Apr 10, 2017 3:03:29 PM CVU home:                     /u01/app/12.2.0/grid/ User:                         grid

Again failures are same as explained above in example 1 that can be ignored (for my case). If you see any other issue here, resolve that before proceeding to the RAC installation.