Migrating  AutoConfig on the Database Tier/

Re-Build the Context file

During one of our DB Migration activity from 11gR1 to 11gR2 DB, when we tried to run Auto Config on DB node post upgrade, it failed consistently. I can't fathom any reason for failure. However to fix the situation, we need to migrate auto config to DB tier. 

There was also another problem. When I try to start the listener from new 11gR2 home it couldn't start. After lots of head banging, I decided to recreate the listener as well.

Following is the process which fixed my issue.

-- Create appsutl.zip on appsnode

[oracle@appsnode 7651166]$ cd /u01/app/oracle/visr12/apps/apps_st/appl/ad/12.0.0/bin

[oracle@appsnode bin]$ perl admkappsutil.pl
Starting the generation of appsutil.zip
Log file located at /u01/app/oracle/visr12/inst/apps/visr12_appsnode/admin/log/MakeAppsUtil_07090645.log
output located at /u01/app/oracle/visr12/inst/apps/visr12_appsnode/admin/out/appsutil.zip
MakeAppsUtil completed successfully.
[oracle@appsnode bin]$ cd /u01/app/oracle/visr12/inst/apps/visr12_appsnode/admin/out/
[oracle@appsnode out]$ ll -ltr
total 2868
drwxr-xr-x 2 oracle oinstall    4096 May 14 05:50 05140550
drwxr-xr-x 2 oracle oinstall    4096 May 14 06:05 05140605
drwxr-xr-x 2 oracle oinstall    4096 May 14 07:05 05140701
drwxr-xr-x 2 oracle oinstall   12288 May 14 07:11 05140705
drwxr-xr-x 2 oracle oinstall    4096 May 15 06:23 05150623
drwxr-xr-x 2 oracle oinstall    4096 May 15 06:25 05150624
drwxr-xr-x 2 oracle oinstall    4096 May 15 06:25 05150625
drwxr-xr-x 2 oracle oinstall   12288 May 25 07:03 05250658
drwxr-xr-x 2 oracle oinstall    4096 Jun  5 04:47 06050447
drwxr-xr-x 2 oracle oinstall   12288 Jun  5 05:00 06050457
drwxr-xr-x 2 oracle oinstall   12288 Jun  5 05:13 06050512
drwxr-xr-x 2 oracle oinstall   12288 Jun  6 03:16 06060314
drwxr-xr-x 2 oracle oinstall   12288 Jun  6 05:42 06060541
drwxr-xr-x 2 oracle oinstall   12288 Jun  6 06:21 06060620
drwxr-xr-x 2 oracle oinstall   12288 Jul  3 05:13 07030512
drwxr-xr-x 3 oracle oinstall    4096 Jul  9 06:45 appsutil
-rw-r--r-- 1 oracle oinstall 2799287 Jul  9 06:45 appsutil.zip


-- Copy it to DB node
[oracle@appsnode out]$ scp appsutil.zip  appsdbnode:/home/oracle/
appsutil.zip                                                  100% 2734KB   2.7MB/s   00:00

-- On DB Node re-generate the context file 

[oracle@appsdbnode appsutil]$ cd bin
[oracle@appsdbnode  bin]$ perl ./adbldxml.pl appsuser=apps appspass=apps

-- Running the above script will also create listner.ora file under your ORACLE_HOME
--  $ORACLE_HOME/network/admin/<CONTEXT>/listener.ora file



--Check the context file 
[oracle@appsdbnode appsutil]$ ./bin/adchkcfg.sh  contextfile=./visr12_appsdbnode.xml
Enter the APPS password:
The log file for this session is located at: /u01/app/oracle/product/11.2.0/db_1/appsutil/log/visr12_appsdbnode/07090710/adconfig.log
AutoConfig is running in test mode and building diffs...
AutoConfig will consider the custom templates if present.
        Using ORACLE_HOME location : /u01/app/oracle/product/11.2.0/db_1
        Classpath                   : :/u01/app/oracle/product/11.2.0/db_1/jdbc/lib/ojdbc5.jar:/u01/app/oracle/product/11.2.0/db_1/appsutil/java/xmlparserv2.jar:/u01/app/oracle/product/11.2.0/db_1/appsutil/java:/u01/app/oracle/product/11.2.0/db_1/jlib/netcfg.jar:/u01/app/oracle/product/11.2.0/db_1/jlib/ldapjclnt11.jar
        Using Context file          : /u01/app/oracle/product/11.2.0/db_1/appsutil/out/visr12_appsdbnode/07090710/visr12_appsdbnode.xml
Context Value Management will now update the test Context file
        Updating test Context file...COMPLETED


        [ Test mode ]
        No uploading of Context File and its templates to database.
Updating rdbms version in Context file to db112
Updating rdbms type in Context file to 32 bits
Testing templates from ORACLE_HOME ...
Differences text report is located at: /u01/app/oracle/product/11.2.0/db_1/appsutil/out/visr12_appsdbnode/07090710/cfgcheck.txt
        Generating Profile Option differences report...COMPLETED
Differences text report for the Database is located at: /u01/app/oracle/product/11.2.0/db_1/appsutil/out/visr12_appsdbnode/07090710/ProfileReport.txt
        Generating File System differences report......COMPLETED
Differences html report is located at: /u01/app/oracle/product/11.2.0/db_1/appsutil/out/visr12_appsdbnode/07090710/cfgcheck.html
Differences Zip report is located at: /u01/app/oracle/product/11.2.0/db_1/appsutil/out/visr12_appsdbnode/07090710/ADXcfgcheck.zip
AutoConfig completed successfully.


[oracle@appsdbnode appsutil]$ vi /u01/app/oracle/product/11.2.0/db_1/appsutil/out/visr12_appsdbnode/07090710/cfgcheck.txt


-- Run the auto config again
[oracle@appsdbnode appsutil]$ ./bin/adconfig.sh  contextfile=visr12_appsdbnode.xml
Enter the APPS user password:
The log file for this session is located at: /u01/app/oracle/product/11.2.0/db_1/appsutil/log/visr12_appsdbnode/07090712/adconfig.log
AutoConfig is configuring the Database environment...
AutoConfig will consider the custom templates if present.
        Using ORACLE_HOME location : /u01/app/oracle/product/11.2.0/db_1
        Classpath                   : :/u01/app/oracle/product/11.2.0/db_1/jdbc/lib/ojdbc5.jar:/u01/app/oracle/product/11.2.0/db_1/appsutil/java/xmlparserv2.jar:/u01/app/oracle/product/11.2.0/db_1/appsutil/java:/u01/app/oracle/product/11.2.0/db_1/jlib/netcfg.jar:/u01/app/oracle/product/11.2.0/db_1/jlib/ldapjclnt11.jar
        Using Context file          : /u01/app/oracle/product/11.2.0/db_1/appsutil/visr12_appsdbnode.xml
Context Value Management will now update the Context file
        Updating Context file...COMPLETED
        Attempting upload of Context file and templates to database...COMPLETED
Updating rdbms version in Context file to db112
Updating rdbms type in Context file to 32 bits
Configuring templates from ORACLE_HOME ...
AutoConfig completed successfully.

Patch Download Doesn't Connect to Metalink/My Oracle Support from OAM Console

During configuring Auto patch download process from OAM console in Apps R12.x, we hit following error in application logs...

Error - 

Cannot connect to http://updates.oracle.com. Please ensure Metalink settings have been configured.

or

Connection Refused.

With Patch Manager, customers using an older version might get one of the following errors when using the Add Oracle Patch. List of Values on the third page of the Patch Manager interview

The fix is to download and apply the patch 10629956_R12.ADO.B_R12_GENERIC.

[oracle@appsnode 1213Patches]$ unzip p10629956_R12.ADO.B_R12_GENERIC.zip

Archive:  p10629956_R12.ADO.B_R12_GENERIC.zip

  inflating: 10629956/README.html

  inflating: 10629956/u10629956.drv

  inflating: 10629956/f10629956.ldt

 extracting: 10629956/j10629956_ad.zip

 extracting: 10629956/j10629956_fnd.zip

  inflating: 10629956/marker1.txt

  inflating: 10629956/b10629956.ldt

  inflating: 10629956/README.txt

  inflating: 10629956/patch_metadata.xml

  inflating: 10629956/ad/patch/115/manualsteps/ad_apply_patch.xml

[oracle@appsnode 1213Patches]$ cd 10629956/

[oracle@appsnode 10629956]$ adpatch defaultsfile=/u01/app/oracle/visr12/apps/apps_st/appl/admin/visr12/adalldefaults.txt  workers=4 driver=u10629956.drv  logfile=patch_10629956.log

Saving product information.

AD code level : [B.3]

Trying to obtain a lock...

The default directory is [/home/oracle/1213Patches/10629956] :

You have specified u10629956.drv

as the name of your patchdriver file.

AutoPatch is complete.

-- Disable the Maintenance mode and start the apps stack

[oracle@appsnode 10629956]$ adstrtal.sh apps/apps

[oracle@appsnode 10629956]$ adopmnctl.sh  status

---------------------------------+--------------------+---------+---------

ias-component                            | process-type       |     pid | status

---------------------------------+--------------------+---------+---------

OC4JGroup:default_group          | OC4J:oafm          |    6223 | Alive

OC4JGroup:default_group          | OC4J:forms         |    6158 | Alive

OC4JGroup:default_group          | OC4J:oacore        |    6029 | Alive

HTTP_Server                               | HTTP_Server        |    5977 | Alive

Now if one tries to save the credentials it works fine....

Ref : New Required Patches for Patch Wizard, Patch Manager, and Oracle Application Change Management Pack for Oracle E-Business Suite Releases 11i, 12.0, and 12.1 [ID 1267768.1]

The Concurrent requests are in Pending status appears / No Concurrent Manager defined to execute the request

The Concurrent request are in Pending status appears / No Concurrent Manager defines to execute the request

The submitted concurrent request are in pending state. The logs states, No Concurrent Manager is running to attend the job. When checked the Admin console for concurrent manager, we found that Internal Concurrent Manager and standard manager were not active. 

Error from $APPLCSF - 

Shutting down Internal Concurrent Manager : 10-JUL-2012 06:14:04
The visr12_0709@visr12 internal concurrent manager has terminated successfully - exiting.

When I checked the app node for FNDLIBR  process, the process was running but OAM console was showing it as terminated. So, obviously something was not right. So I decided to bounce the concurrent manager 

Possible cause for Inactive OR No Manager

i) No manager is defined to run the request

or

ii) All managers are locked by run-alone requests

or

iii) Concurrent Manager Service is down

or

iv) No concurrent manager process because of workshift

To check Work Shift for any Concurrent Manager From System Administrator responsibility > Concurrent > Manager > Define > Work Shifts > Processes

Fix 

[oracle@appsnode log]$ adcmctl.sh  stop apps/apps

You are running adcmctl.sh version 120.17.12010000.5

Shutting down concurrent managers for visr12 ...

ORACLE Password:

Submitted request 400227 for CONCURRENT FND SHUTDOWN

adcmctl.sh: exiting with status 0

adcmctl.sh: check the logfile /u01/app/oracle/visr12/inst/apps/visr12_appsnode/logs/appl/admin/log/adcmctl.txt for more information ...

-- starting the Concurrent Manager with diag=Y for detailed info
[oracle@appsnode log]$ adcmctl.sh  start apps/apps diag=Y

You are running adcmctl.sh version 120.17.12010000.5

Starting concurrent manager for visr12 ...

Starting visr12_0710@visr12 Internal Concurrent Manager

Default printer is noprint

adcmctl.sh: exiting with status 0

adcmctl.sh: check the logfile /u01/app/oracle/visr12/inst/apps/visr12_appsnode/logs/appl/admin/log/adcmctl.txt for more information ...

[oracle@appsnode log]$ adcmctl.sh  status apps/apps

You are running adcmctl.sh version 120.17.12010000.5

Internal Concurrent Manager is Active.

adcmctl.sh: exiting with status 0

[oracle@appsnode log]$ ps -ef | grep FNDLIBR

oracle    7650  7644  0 06:39 pts/1    00:00:03 FNDLIBR                                                    

oracle    7924  7759  0 06:55 ?        00:00:02 FNDLIBR                                                                                                                                                               

oracle    7925  7759  0 06:55 ?        00:00:02 FNDLIBR                                                                                                                                                               

oracle    7926  7759  0 06:55 ?        00:00:02 FNDLIBR                                                                                                                                                               

oracle    7931  7759  0 06:55 ?        00:00:00 FNDLIBR              

Once started I opened Concurrent Request form, from OAM > system administrator responsibility > concurrent > request option. 

Once the form is opened Navigate to +Concurrent > Manager > Administrator and double click on it to open another form. Here you will see Administer Concurrent Manager screen, from which you can identify running or pending requests.

for more detailed logs, pls check $LOG_HOME/appl/conc OR $APPLCSF

This summary is not available. Please click here to view the post.

 "FRM-92120 Registry.dat is Missing" Error Message When Launching a Forms Based Function

After an upgrade of Oracle e-Business Suite, when launching a forms based function from a client browser, the user encounters the following error:

FRM-92120: Registry file  https://appsnode:8000/OA_JAVA/oracle/forms/registry/Registry.dat 

is missing

Details
Java Exception
oracle.forms.engine.RunformsException:FRM-92100 :Registry file https://appsnode:8000/OA_JAVA/oracle/forms/registry/Registry.dat is missing

at java.lang.Throwable<init>(Compiled Code)
at java.lang.Exception.<init>Compiled Code)
at oracle.forms.engine.RunformsException.<init>(Unknown Source)

This error only occurs for a small number of user PCs and is due to the Jinitiator installation not being completed successfully, or has been corrupted

Solution - 

1. From Control Panel click on the Add /Remove program and remove Oracle Jinitiator  on the user's machine.
2. Relogin to the Oracle Ebusiness Suite which will prompt for a clean install of Oracle Jinitiator.

After re-Install of Jinitiator the forms function opens up fine. 

Note - If you are using any non MS browser, you may still face the issue. However the issue could be intermittent so its advisable to use it with IE

Login Page not Coming Up / Guest Log In Fails / LogIn Issue in EBS/Apps R12

The issue occurred post R12.1.3 Patch application. We started the DB Tier and Apps Tier successfully and it came up nice. But when one tried to load log-in page for Apps, we got 500 - Internal Server Error. 

So we checked the log files under $LOG_HOME and found out that one jtf logfile was showing Invalid username/password Error. 

Guest is the very important user in EBS/Apps as all the authentication for Applications users go thru  Guest user, hence if the guest credentials are improper the login page will never appear.

Few Important points about Guest User - 

 Stored in DBC file on Apps node-  $FND_TOP/secure/

 Remember that, Guest is an application user not a DB user while APPS/APPLSYS/APPLSYSPUB is a DB user.

 It should be consistent on both the places, DBC File and in FND Profile. Otherwise application will not work.

 Guest user password cannot be changed through FNDCPASS.Most appropiate way to change it through user define form. However there is other way to accomplish this. it can be done through FND_WEB_SEC package stored in apps schema

 It should never be end dated. Otherwise after End Date your application will falter

So it was the time to check the integrity of the Guest user password. Following is the way to do it.

1. Check Value in DBC File

[oracle@appsdbnode appl]$ grep -i GUEST_USER_PWD $FND_SECURE/visr12.dbc

GUEST_USER_PWD=GUEST/ORACLE

2. Check profile option value in DB

[oracle@appsdbnode appl]$ sqlplus apps/apps@visr12

SQL> select fnd_profile.value('GUEST_USER_PWD') from dual;

FND_PROFILE.VALUE(’GUEST_USER_PWD’)

——————————————————————————–

GUEST/ORACLE

Value for step 1 and 2 must be sync.

Note - However in my case R12 DB the profile option value for not showing up. I tried a lot to find some info on it but couldn't find much. This was surprising, I will tell you later, WHY?!?

3. Guest user connectivity check

SQL> select FND_WEB_SEC.VALIDATE_LOGIN('GUEST','ORACLE') from dual;

FND_WEB_SEC.VALIDATE_LOGIN('GUEST','ORACLE')

——————————————————————————–-----

Y

If you are getting above mentioned values everything is file in your Instance.

However, my story is not normal (I wouldn't be here, if everything had been normal!!)

As I mentioned earlier the fnd_profile value was not appearing. So as per MOS I decided to alter it. 

1. Reset GUEST password:

-- delete fnd vault value and change the password

SQL> exec fnd_vault.del('FND');

SQL> commit;

SQL> select FND_WEB_SEC.CHANGE_GUEST_PASSWORD ('ORACLE','<applsyspassword>') from dual;

SQL> commit;

-- check for profile exist or not. This check should give you the output with fnd option name. In my case it was showing no rows.

SQL> select profile_option_name from fnd_profile_options_tl where user_profile_option_name like 'Guest%';

no rows selected

2. Test if Profile Updated or not?

DECLARE

     stat boolean;

BEGIN

     stat := FND_PROFILE.SAVE('GUEST_USER_PWD','GUEST/ORACLE','SITE');

     IF stat THEN

          dbms_output.put_line( 'Stat = TRUE - profile updated' );

     ELSE

          dbms_output.put_line( 'Stat = FALSE - profile NOT updated' );

     END IF;

end;

/

Remember - If one tries to change GUEST user password using FNDCPASS, It will result in error. Instead Use following API:

[oracle@appsdbnode appl]$ java oracle.apps.fnd.security.AdminAppServer APPS/"APPS Password" UPDATE GUEST_USER_PWD=GUEST/"Guest User Password" DB_HOST="Host_Name" DB_PORT="PortNumber" DB_NAME="SID"

4. Check the log of FNDCPASS for any error for GUEST user if there is no error for GUEST user then do the next step

5. Compile JSP

[oracle@appsdbnode appl]$ cd $FND_TOP/patch/115/bin

[oracle@appsdbnode bin]$  ojspCompile.pl --compile --flush

6. Set GUEST_USER_PWD password in $FND_SECURE/<sid>.dbc to GUEST/ORACLE (in my case. could be different in your case)

7. Bounce Middle Tier Services and Retest the issue.

The problem was still not solved and I was back where I was started. Meanwhile I came across one forum which suggested the re-generation of the jar files  from adadmin with force could help so I tried it. But again in Vain.

So after all this pain the issue was still not resolved. And I was left with no option after virtually trying everything. I thought may be something not right in config files and its worth trying Autoconfig. May be that can fix the Issue or No Issue!!! I know it was fluke but believe me, it worked !!!!!!

I ran AUTOCONFIG on DB Tier and on Apps Tier and started the Apps. Tried loading the log-in page and it came up nicely this time.

And came back my Login page. Can take sigh of relief now. Hope this will help you in trouble-shooting your Log-in Issues with EBS/Apps

Final Frontier - 

Access the following URL and complete all tests (Results from this test are required by MOS, if you have raised an SR/TAR with Oracle)

URL - http://host.domain:port/OA_HTML/jsp/fnd/aoljtest.jsp 

Disable Oracle Database Vault

Recently I needed to upgrade one of our 11gR1 DB (11.1.0.7) to 11gR2(11.2.0.1). I have my binaries for R2 in place and backup of the DB was also done. 
So I fired up DBUA from R2 Oracle Home, but hit the error as follows

database vault option is enabled in oracle home. This option must be disabled for upgrade


So I was left with choice to disable the DB Vault option as follows
[oracle@appsdbnode db_1]$ sqlplus "/ as sysdba"
SQL> shut immediate;


-- For Oracle RAC installations, run these commands on all nodes.

[oracle@appsdbnode db_1]$ pwd

/u01/app/oracle/product/11.2.0/db_1
[oracle@appsdbnode db_1]$ cd rdbms/lib/
[oracle@appsdbnode lib]$ make -f ins_rdbms.mk dv_off
/usr/bin/ar cr /u01/app/oracle/visr12/db/tech_st/11.1.0/rdbms/lib/libknlopt.a /u01/app/oracle/visr12/db/tech_st/11.1.0/rdbms/lib/kzvndv.o
[oracle@appsdbnode lib]$ cd ../../bin/
[oracle@appsdbnode bin]$ ./relink all
writing relink log to: /u01/app/oracle/visr12/db/tech_st/11.1.0/install/relink.log


-- If you have DB in place with DV option enabled you need to disable it as follows.



[oracle@appsdbnode bin]$dvca -action disable 
  -oh /u01/app/oracle/product/11.2.0/db_1
  -service visr12
  -instance visr12
  -dbname visr12 
  -owner_account dvowner 
  -logfile dvcalog.txt 
  
Enter SYS password: sys_password
Enter owner password: dvowner_password


Once done you can now connect to your DB to check if the Vault option is disabled or not. 
SQL> SELECT * FROM V$OPTION WHERE PARAMETER = 'Oracle Database Vault';
If successfully done, the value should be FALSE