Sunday, June 10, 2012

Upgrade Oracle IAS/HTTP Server from 10.1.3.4 to 10.1.3.5

In this post I am going to show how to upgrade Oracle IAS (Application Server) 10g from 10.1.3.4 to 12.1.3.5. IAS 10.1.3.4 comes default with Apps/EBS R12.1.1.  Hence when you try to upgrade it with R12.1.3, as one of the pre-requisite, one has to upgrade IAS to 10.1.3.5.

Note – This upgrade installer is riddled with few bugs so pls refer to Troubleshooting section, when you come across any issues. I will really appreciate it you hit some new issue, please leave a comment about the issue and I will try to answer that.

Background –
Oracle Application Server 10g Release 3 (10.1.3) Patch Set 5 (10.1.3.5.0) is not a complete software distribution. You must install this patch set over an existing installation. Specifically, this patch set can be applied only to the following Oracle Application Server releases:
Oracle Application Server 10g Release 3 (10.1.3.0.0)
Oracle Application Server 10g Release 3 (10.1.3.1.0), which included the Oracle SOA Suite
Oracle Application Server 10g Release 3 (10.1.3.2.0), which included the Oracle WebCenter Suite
Oracle Application Server 10g Release 3 Patch Set 3 (10.1.3.3.0)
Oracle Application Server 10g Release 3 (10.1.3.4.0)
Any Oracle Application Server 10g Release 3 (10.1.3.n) standalone installation

Patch Set Application

Back up Your Existing Environment
There is no automatic procedure for removing 10g Release 3 (10.1.3) Patch Set 5 (10.1.3.5.0). Oracle strongly recommends that you make a backup of your existing environment before you install Oracle Application Server 10gRelease 3 (10.1.3) Patch Set 5 (10.1.3.5.0). If you choose to remove Oracle Application Server 10g Release 3 (10.1.3) Patch Set 5 (10.1.3.5.0), or if you encounter a problem with application of the patch, then you can restore your original environment.
You must make a back up of the database schema that is being used so that the database can be reverted back to the original state.

Imp Note -
As part of the pre-requisite patch I first decided to apply latest OPatch. The reason being is that at the end of patch, the OUI uses OPatch to apply one off patches. The OPatch, one needs to apply is 6880880 for 10g. Be aware that you don’t apply 11g OPatch, as it is the latest one, you might confronted with Java invalid or unsupported class errors. The reason is that 11g OPatch uses higher version of Java.

Pre-requisite Patch (OPatch Upgrade) –

Download p6880880_101000_LINUX.zip patch for your suitable Linux distribution
Make sure you are pointing to correct ORACLE HOME i.e. 10.1.3 IAS Home

[oracle@appsnode 10.1.3]$ unzip /home/oracle/1213Patches/pre-req-patches/p6880880_101000_LINUX.zip -d /u01/app/oracle/visr12/apps/tech_st/10.1.3/

Archive:  /home/oracle/1213Patches/pre-req-patches/p6880880_101000_LINUX.zip
   creating: /u01/app/oracle/visr12/apps/tech_st/10.1.3/OPatch/ocm/
 extracting: /u01/app/oracle/visr12/apps/tech_st/10.1.3/OPatch/ocm/ocm.zip
   creating: /u01/app/oracle/visr12/apps/tech_st/10.1.3/OPatch/ocm/lib/
................................
  inflating: /u01/app/oracle/visr12/apps/tech_st/10.1.3/OPatch/perl_modules/XML.pm
[oracle@appsnode 10.1.3]$ cd OPatch
[oracle@appsnode OPatch]$ ./opatch version
Oracle Interim Patch Installer version 1.0.0.0.64
Copyright (c) 2011 Oracle Corporation. All Rights Reserved..

Oracle recommends you to use the latest OPatch version
and read the OPatch documentation available in the OPatch/docs
directory for usage. For information about the latest OPatch and
other support-related issues, refer to document ID 293369.1
available on My Oracle Support (https://myoraclesupport.oracle.com)

OPatch Version: 1.0.0.0.64

[oracle@appsnode 10.1.2]$ unzip /home/oracle/1213Patches/pre-req-patches/p6880880_101000_LINUX.zip -d /u01/app/oracle/visr12/apps/tech_st/10.1.2/
Archive:  /home/oracle/1213Patches/pre-req-patches/p6880880_101000_LINUX.zip
.............................................
 inflating: /u01/app/oracle/visr12/apps/tech_st/10.1.2/OPatch/perl_modules/XML.pm
[oracle@appsnode 10.1.2]$ cd OPatch
[oracle@appsnode OPatch]$ ./opatch version
Oracle Interim Patch Installer version 1.0.0.0.64
Copyright (c) 2011 Oracle Corporation. All Rights Reserved..

Oracle recommends you to use the latest OPatch version
and read the OPatch documentation available in the OPatch/docs
directory for usage. For information about the latest OPatch and
other support-related issues, refer to document ID 293369.1
available on My Oracle Support (https://myoraclesupport.oracle.com)

OPatch Version: 1.0.0.0.64

Once the OPatch is successfully upgraded, one needs to start the upgrade Installer as follows
Download patch 8626084 PLACEHOLDER BUG TO UPLOAD AS 10.1.3.5  PATCH-SET ON ARU (Patchset)
Unzip the patch in the stage location

[oracle@appsnode Disk1]$ ./runInstaller
Starting Oracle Universal Installer...



Select the existing IAS 10.1.3 home to upgrade

Ignore the options to download patches and updates if you wish 


During pre-checks the installer will say, it cant detect the OPMN processes running even though they are up and healthy. Check Troubleshooting Section for workaround/fix
If the OPMN stack is down, start it up as follows..
[oracle@appsnode tech_st]$ cd $ADMIN_SCRIPTS_HOME
[oracle@appsnode scripts]$ ./adstrtal.sh  apps/apps


Next screen will prompt you to enter oc4jadmin user password. If you don't remember it you can choose to change it. 
Pls check troubleshooting section for further info.

Once the authentication is successful, the installer will start. Review the progress...

At the end you will be asked to execute root.sh as follows

[oracle@appsnode config]$ su -
Password:
[root@appsnode ~]# cd /u01/app/oracle/visr12/apps/tech_st/10.1.3
[root@appsnode 10.1.3]# ./root.sh


Check version post installation - 
[oracle@appsnode config]$ pwd
/u01/app/oracle/visr12/apps/tech_st/10.1.3/config
[oracle@appsnode config]$ vi ias.properties
Version=10.1.3.5.0



Troubleshooting -

Issue -1
Please make sure you navigate to the $INST_TOP/ora/10.1.3 and source the <sid_machine>.env file to set your 10.1.3 ORACLE_HOME before running the installer. This is already mentioned in (Upgrading to the Latest OracleAS 10g 10.1.3.x Patch Set in Oracle E-Business Suite Release 12 [ID 454811.1]).

Issue -2
OPMN is not start while applying patch 10.1.3.5 stops at 83%

Problem:
OPMN is not start while applying patch 10.1.3.5 stops at 83%
While Upgrading to the Latest OracleAS 10g 10.1.3.x Patch Set in Oracle E-Business Suite Release 12 [ID 454811.1]
 Installer gave an error as show below exactly when reaching 83%

#########
OPMN is failing to start. OPMN must be started in order to correctly install and configure the patchset. Please start OPMN manually in order to continue with the patchset installation.
#########

$IAS_ORACLE_HOME/opmn/bin/opmnctl is trying to read opmn.xml from path pointed by $ORACLE_CONFIG_HOME which points to 10.1.2 home instead opmnctl should need 10.1.3 home

Do below:
Point ORACLE_CONFIG_HOME to 10.1.3 home and test by running the below command before running the installer again.
$IAS_ORACLE_HOME/opmn/bin/opmnctl status (or) startall
Export the environment variable before invoking runinstaller and this should be able to bypass the issue.

Solution:

$ echo $ORACLE_CONFIG_HOME
$ export ORACLE_CONFIG_HOME=/erpapp/aderp/inst/apps/adtmp_erptest-temp/ora/10.1.3
$ echo $ORACLE_CONFIG_HOME
/erpapp/aderp/inst/apps/adtmp_erptest-temp/ora/10.1.3

Issue 3 –
If the 'oc4jadmin' password for the OAFM oc4j instance is not known, then reset the password in the files as bellow. before running the script. Take the backup of system-jazn-data.xml before resetting the password.
Replace the modified system-jazn-data.xml with backup of system-jazn-data.xml after running the script.

File 1  - $INST_TOP/ora/10.1.3/j2ee/oafm/config/system-jazn-data.xml
file 2 - /u01/app/oracle/visr12/apps/tech_st/10.1.3/j2ee/oafm/config

        For example, if you want to reset the password to 'welcome12', below should be the entry:

              <user>
                  <name>oc4jadmin</name>
                  <display-name>OC4J Administrator</display-name>
                  <guid>23C8E4F0BDDE11DCBFB8AF3B7E0DDB2D</guid>
                  <description>OC4J Administrator</description>
                  <credentials>!welcome12</credentials>
              </user>

Issue 4 –

OCM patch required even though you're not using OCM
While installing OAS 10gR3 patch 8626084 to upgrade OAS to 10.1.3.5, I received an error that step "Run One-off OPatches" had failed.
Looking at the log file, I found the following:

This is a OCM patch.
Home has OCM installed but not configured.

To run in silent mode, OPatch requires a response file for Oracle Configuration Manager (OCM).
Run /u01/app/oracle/product/oas10gr3/OPatch/ocm/bin/emocmrsp to generate an OCM response file. The generated response file can be reused on different platforms and in multiple OPatch silent installs.

To regenerate an OCM response file, rerun /u01/app/oracle/product/oas10gr3/OPatch/ocm/bin/emocmrsp.
 ERROR: OPatch failed because of cmd. args. problem.

The following did the trick and I was able to complete the patch installation.
/u01/app/oracle/product/oas10gr3/ccr/bin/setupCCR -s -d