Oracle Database 18c: RAC Upgrade 12c to 18c (12.2.0.1 to 18.3) - Part 1/2

In this blog series of 2 blogs, I am going to upgrade my 12c cluster with one Database to 18c Cluster. 

If you want to jump to next part click here for Part 2. 

If you want to see how to upgrade 12cr1 to 12cr2 you can check one of my previous blogs.
There will be few repeat points for your information from my last blog. 

Here is a brief of my cluster. 

1. Version
/u01/app/12201/grid/bin/crsctl query crs activeversion
Oracle Clusterware active version on the cluster is [12.2.0.1.0]

2. Disk Groups

[As grid]
/u01/app/12201/grid/bin/asmcmd lsdg
State Type Rebal Sector Logical_Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED NORMAL N 512 512 4096 4194304 307188 238296 102396 67950 0 Y DATA/
MOUNTED NORMAL N 512 512 4096 4194304 307188 304148 102396 100876 0 N RECO/

3. Databases 
/u01/app/12201/grid/bin/srvctl config database

PRBRM

4. Patch Versions
Grid
./opatch lspatches
28163235;ACFS JUL 2018 RELEASE UPDATE 12.2.0.1.180717 (28163235)
28163190;OCW JUL 2018 RELEASE UPDATE 12.2.0.1.180717 (28163190)
28163133;Database Jul 2018 Release Update : 12.2.0.1.180717 (28163133)
27144050;Tomcat Release Update 12.2.0.1.0(ID:171023.0830) (27144050)

26839277;DBWLM RELEASE UPDATE 12.2.0.1.0(ID:170913) (26839277)

Database

./opatch lspatches

27923353;OJVM RELEASE UPDATE: 12.2.0.1.180717 (27923353)

28163133;Database Jul 2018 Release Update : 12.2.0.1.180717 (28163133)

In my cluster upgrade, I always keep backup of cluster registry, cluster resources running and listeners & instances running on each node for finaly comparison (just in case)

If configuration is correct, all things will come up automatically, however few things might not - for example a listener running from DB home, not registered with Clusterware.

In general below is the advise for18c upgrade.

1. Backup OCR (Local & Global, Logical too)
2. Take Snapshot of resources (just in case you want to compare)
3. Have 33G+ free space in DG with OCR as this is mandatory requirement for upgrade else you will get an error -  INS-43100 asking for space
4. Have /etc/resolv.conf files matching across all nodes
5. Have the mandatory patch installed for GI Upgrade - I generally always patch for upgrades.
6. Have atleast 15GB free space on your installation mount point.
7. And lastly make sure you have on all servers clusterware up and running (There is an option in the installer to skip upgrade on unreachable nodes, which I doubt you want it)


8. Finally run the Upgrade

On  Node 1,2 (and more if there)
[As grid:]
mkdir -p /home/grid/org/Upgrade18cGI

On  Node 1
[As root:]
Set the environment to 12.2.0.1.x Grid Home / (+ASM1)
#ocrconfig  -export /home/grid/org/Upgrade18cGI/OCRLogicalBackupCluster.bak

#ocrconfig -showbackuploc
Note down your backup loc here. 
#ocrconfig -backuploc  #ocrconfig  -manualbackup 

#ocrconfig -local -export /home/grid/org/Upgrade18cGI/OCRLogicalBackup_Local.bak

[As root:]
Set the environment to 12.2.0.1.x Grid Home / (+ASM1)
#crsctl stat res -t > /home/grid/org/Upgrade18cGI/crsctl_stat_res_t.log

On  Node 2, 3 and others (if more) 
[As root:]
Set the environment to 12.2.0.1.x Grid Home / (+ASM2) (+ASM3)
#ocrconfig -local -export /home/grid/org/Upgrade18cGI/OCRLogicalBackup_Local.bak

On Each Node 
#ps -ef | grep pmon > /home/grid/org/Upgrade18cGI/pmon_snapshot.log

# ps -ef | grep tns > /home/grid/org/Upgrade18cGI/tns_snapshot.log

Create a blackout in OEM
Disable any jobs / cron (better disable cron daemon, if only oracle server is running)

Create Grid Home Directory on each node 

[As root:]
#mkdir -p /u01/app/180/grid

Unzip Oracle Media on first node. 

[As root on Node 1]
cd /u01/app/180/grid
#unzip -qq LINUX.X64_180000_grid_home.zip 
[As root on all Nodes]
#chown -R grid:oinstall /u01/app/180/grid
(Make a note of image based installation here) 
[As Grid]  - Run Cluster Verification in pre - crs install stage

#cd /u01/app/180/grid
#./runcluvfy.sh stage -pre crsinst -upgrade -rolling -src_crshome  /u01/app/12201/grid -dest_crshome /u01/app/180/grid -dest_version 18.3.0.0 -fixup -verbose  | tee /tmp/cluvfy_upgd.out

Pre-check for cluster services setup was successful.

CVU operation performed:      stage -pre crsinst
Date:                         Aug 29, 2018 4:48:11 AM
CVU home:                     /u01/app/180/grid/
User:                         grid

I generally redirect output using tee command to see it incoming and also have a copy

Fix anything which is shown as failed.

Finally Unset any environment variables related to Oracle
# unset ORACLE_HOME ORACLE_SID ORACLE_BASE

Follow the next blog which provides screenshots and the upgrade process.
[grid@rac1 grid]$ ./gridSetup.sh