Change Public/SCAN/Virtual IP/Name in 11g/12c RAC

When working with Real Application Cluster DB, changing the infrastructure properties is bit tricky if not difficult. There is dependency chain with several network & name properties and dependent component from Oracle perspective are also needed to be modified. 

I recently undertook the exercise to do so for one of our RAC Cluster which resulted in this post. There are several use cases which I have covered as follows. 

Case I.   Changing Public Host-name

Public hostname is recorded in OCR, it is entered during installation phase. It can not be modified after the installation. The only way to modify public hostname is by deleting the node, then add the node back with a new hostname, or reinstall the clusterware.

Case II.  Changing Public IP Only Without Changing Interface, Subnet or Netmask

If the change is only public IP address and the new ones are still in the same subnet, nothing needs to be done on clusterware layer, all changes need to be done at OS layer to reflect the change.

1. Shutdown Oracle Clusterware stack
2. Modify the IP address at network layer, DNS and /etc/hosts file to reflect the change
3. Restart Oracle Clusterware stack

Above change can be done in rolling fashion, eg: one node at a time.

Case II.  Changing SCAN / SCAN IP

SCAN is used to access cluster as whole from oracle database clients and can redirect your connection request to any available node on the cluster where the requested service is running. This resource is cluster resource and can fail over to any other node if the node where it is running should fail. The entry of SCAN is in OCR and IP is configured at DNS level. 
So to change to SCAN IP / Name one has to first populate the changes on DNS to take it into effect. Once the changes are in effect, one can modify the SCAN resource in OCR as follows. Remember SCAN acts as cluster entry point and load balancing process , the restart to SCAN will require a brief outage. However the existing connection will not have any impact. 

[oracle@dbrac2 ~]$ srvctl  status scan_listener
SCAN Listener LISTENER_SCAN1 is enabled
SCAN listener LISTENER_SCAN1 is running on node dbrac1
[oracle@dbrac2 ~]$ srvctl  status scan
SCAN VIP scan1 is enabled
SCAN VIP scan1 is running on node dbrac1

[oracle@dbrac2 ~]$ srvctl config  scan
SCAN name: dbrac-scan.localdomain, Network: 1
Subnet IPv4: 192.168.2.0/255.255.255.0/eth1, static
Subnet IPv6:
SCAN 0 IPv4 VIP: 192.168.2.110
SCAN VIP is enabled.
SCAN VIP is individually enabled on nodes:
SCAN VIP is individually disabled on nodes:

[oracle@dbrac2 ~]$ srvctl stop scan_listener
[oracle@dbrac2 ~]$ srvctl stop scan
[oracle@dbrac2 ~]$ srvctl  status scan_listener
SCAN Listener LISTENER_SCAN1 is enabled
SCAN listener LISTENER_SCAN1 is not running
[oracle@dbrac2 ~]$ srvctl  status scan
SCAN VIP scan1 is enabled
SCAN VIP scan1 is not running

-- MODIFY THE SCAN IP AT OS LEVEL

[root@dbrac2 ~]$ srvctl  modify scan -scanname dbrac-scan.localdomain

[oracle@dbrac2 ~]$ srvctl  config scan
SCAN name: dbrac-scan.localdomain, Network: 1
Subnet IPv4: 192.168.2.0/255.255.255.0/eth1, static
Subnet IPv6:
SCAN 0 IPv4 VIP: 192.168.2.120
SCAN VIP is enabled.

[oracle@dbrac2 ~]$ srvctl start scan_listener
[oracle@dbrac2 ~]$ srvctl start scan

Since this is my test cluster, I have configured only one SCAN, but regardless of it, the process remains the same for SCAN with 3 IPs.

Case II.  Changing Virtual IP / Virtual Host Name

-- CHANGING NODE VIP FROM 192.168.2.103 TO 192.168.2.203 ON DBRAC1
-- SINCE NODE VIP IS PART OF NODE APPS ONE NEEDS TO MODIFY THE IP ADDRESS ON OS LEVEL AND THEN USE SRVCTL TO MODIFY NODEAPPS

[oracle@dbrac1 automation]$ srvctl  config vip -node dbrac1
VIP exists: network number 1, hosting node dbrac1
VIP Name: dbrac1-vip.localdomain
VIP IPv4 Address: 192.168.2.103
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:

[oracle@dbrac1 automation]$ srvctl  stop vip -node dbrac1
PRCR-1065 : Failed to stop resource ora.dbrac1.vip
CRS-2529: Unable to act on 'ora.dbrac1.vip' because that would require stopping or relocating 'ora.LISTENER.lsnr', but the force option was not specified

[oracle@dbrac1 automation]$ srvctl  stop vip -node dbrac1 -force
[oracle@dbrac1 automation]$ srvctl  status  vip -node dbrac1
VIP dbrac1-vip.localdomain is enabled
VIP dbrac1-vip.localdomain is not running

-- NOW MODIFY THE ADDRESS OF NODE VIP ON OS LEVEL USING EITHER /etc/hosts OR DNS.
-- Once done, use SRVCTL to modify OCR resource. 
-- Here I am not changing the name, but only IP 

[oracle@dbrac1 automation]$ srvctl modify nodeapps -node dbrac1 -address dbrac1-vip.localdomain/255.255.255.0/eth1
[oracle@dbrac1 automation]$ srvctl config   vip -node dbrac1
VIP exists: network number 1, hosting node dbrac1
VIP Name: dbrac1-vip.localdomain
VIP IPv4 Address: 192.168.2.203
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:

[root@dbrac2 ~]# srvctl  config nodeapps
Network 1 exists
Subnet IPv4: 192.168.2.0/255.255.255.0/eth1, static
Subnet IPv6:
Ping Targets:
Network is enabled
Network is individually enabled on nodes:
Network is individually disabled on nodes:
VIP exists: network number 1, hosting node dbrac1
VIP Name: dbrac1-vip.localdomain
VIP IPv4 Address: 192.168.2.203
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
VIP exists: network number 1, hosting node dbrac2
VIP Name: dbrac2-vip.localdomain
VIP IPv4 Address: 192.168.2.104
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
ONS exists: Local port 6100, remote port 6200, EM port 2016, Uses SSL false
ONS is enabled
ONS is individually enabled on nodes:
ONS is individually disabled on nodes:

[oracle@dbrac1 automation]$ srvctl  start vip -node dbrac1
[root@dbrac2 ~]# srvctl  status  nodeapps
VIP dbrac1-vip.localdomain is enabled
VIP dbrac1-vip.localdomain is running on node: dbrac1
VIP dbrac2-vip.localdomain is enabled
VIP dbrac2-vip.localdomain is running on node: dbrac2
Network is enabled
Network is running on node: dbrac1
Network is running on node: dbrac2
ONS is enabled
ONS daemon is running on node: dbrac1
ONS daemon is running on node: dbrac2

-- ON NODE2

[oracle@dbrac2 ~]$ srvctl  stop vip -node dbrac2 -force
[oracle@dbrac2 ~]$ srvctl  status  vip -node dbrac2
VIP dbrac2-vip.localdomain is enabled
VIP dbrac2-vip.localdomain is not running

[oracle@dbrac2 ~]$ srvctl modify nodeapps -node dbrac2 -address dbrac2-vip.localdomain/255.255.255.0/eth1

[oracle@dbrac2 ~]$ srvctl  config vip  -n dbrac2
VIP exists: network number 1, hosting node dbrac2
VIP Name: dbrac2-vip.localdomain
VIP IPv4 Address: 192.168.2.204
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:

[oracle@dbrac2 ~]$ srvctl  start vip -node dbrac2

[oracle@dbrac2]$ srvctl status  nodeapps
VIP dbrac1-vip.localdomain is enabled
VIP dbrac1-vip.localdomain is running on node: dbrac1
VIP dbrac2-vip.localdomain is enabled
VIP dbrac2-vip.localdomain is running on node: dbrac2
Network is enabled
Network is running on node: dbrac1
Network is running on node: dbrac2
ONS is enabled
ONS daemon is running on node: dbrac1
ONS daemon is running on node: dbrac2

[oracle@dbrac1 automation]$ crs_stat -t  | grep vip

ora.dbrac1.vip ora....t1.type ONLINE    ONLINE    dbrac1     
ora.dbrac2.vip ora....t1.type ONLINE    ONLINE    dbrac2     
ora.scan1.vip  ora....ip.type ONLINE    ONLINE    dbrac1


A special case for 11gR2  VIP Name Change - 
modifying the VIP hostname only without changing the IP address.

For example: only VIP hostname changes from dbrac1-vip to dbrac1-nvip, IP and other attributes remain the same.

If IP address is not changed, above modify command will not change the USR_ORA_VIP value in 'crsctl stat res ora.dbrac1.vip -p' output. Please use the following command:

# crsctl modify res ora.dbrac1.vip -attr USR_ORA_VIP=ora.dbrac1.nvip

Verify the changes for USR_ORA_VIP field:
# crsctl stat res ora.dbrac1.vip -p |grep USR_ORA_VIP

Three important flag for crsctl stat res command are as follows. 
   -p                Print static configuration
   -v                Print runtime configuration
   -f                Print full configuration