Search This Blog

Saturday, November 25, 2017

How to attach ebs 10.1.2 oracle home to central inventory

Hello,

This post is to define "How to attach ebs 10.1.2 oracle home to central inventory"

bash-3.2$ opatch apply

Oracle Interim Patch Installer version 1.0.0.0.64
Copyright (c) 2011 Oracle Corporation. All Rights Reserved..

Oracle recommends you to use the latest OPatch version
and read the OPatch documentation available in the OPatch/docs
directory for usage. For information about the latest OPatch and
other support-related issues, refer to document ID 293369.1
available on My Oracle Support (https://myoraclesupport.oracle.com)

Oracle Home           : /a01/appstier/UAT/fs2/EBSapps/10.1.2
Oracle Home Inventory : /a01/appstier/UAT/fs2/EBSapps/10.1.2/inventory
Central Inventory     : /var/opt/oracle/oraInventory
   from               : /var/opt/oracle/oraInst.loc
OUI location          : /a01/appstier/UAT/fs2/EBSapps/10.1.2/oui
OUI shared library    : /a01/appstier/UAT/fs2/EBSapps/10.1.2/oui/lib/solaris/liboraInstaller.so
Java location         : /a01/appstier/UAT/fs2/EBSapps/10.1.2/jdk/jre/bin/java
Log file location     : /a01/appstier/UAT/fs2/EBSapps/10.1.2/.patch_storage/<patch ID>/*.log

Creating log file "/a01/appstier/UAT/fs2/EBSapps/10.1.2/.patch_storage/18752577/Apply_18752577_11-25-2017_09-57-56.log"

The Oracle Home /a01/appstier/UAT/fs2/EBSapps/10.1.2 is not registered with the Central Inventory.  OPatch was not able to get details of the home from the inventory.

ERROR: OPatch failed because of Inventory problem.

When we apply patch in oracle home, OPatch checks for the inventory to understand the patch detail which already present. It gives the error above if it does not find any patch details. Since oracle home is not registered with central inventory, opatch could nt get any of the patch details thus it threw an error. We need to attach this particular oracle home to central inventory in order to overcome this error. This is one of the method to overcome this error. 

bash-3.2$ cat /var/opt/oracle/oraInst.loc
inventory_loc=/var/opt/oracle/oraInventory
inst_group=dba

Source env file

bash-3.2$ cd $ORACLE_HOME/oui/bin/

Command to be executed :

./runInstaller -silent -attachHome -invPtrLoc /var/opt/oracle/oraInst.loc ORACLE_HOME="/a01/appstier/UAT/fs2/EBSapps/10.1.2" ORACLE_HOME_NAME="UAT_TOOLS_fs2_EBSapps_10_1_2"

bash-3.2$ ./runInstaller -silent -attachHome -invPtrLoc /var/opt/oracle/oraInst.loc ORACLE_HOME="/a01/appstier/UAT/fs2/EBSapps/10.1.2" ORACLE_HOME_NAME="UAT_TOOLS_fs2_EBSapps_10_1_2"

Starting Oracle Universal Installer...
No pre-requisite checks found in oraparam.ini, no system pre-requisite checks will be executed.
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2017-11-25_10-27-20AM. Please wait ...bash-3.2$

bash-3.2$ 'AttachHome' was successful.

Thursday, October 12, 2017

RMAN error : ORA-17628: Oracle error 19505 returned by remote Oracle server

Hello DBA's,

While i was doing active replication from production to test i got the below error in RMAN. RMAN triggered from dev server as usual. When i saw remote server i doubted the prod server then realized i was absolutely wrong. I looked into alert long in prod and everything was clean. I did tnsping and checked listener (lsnrctl status), everything was perfect from production side.

Since we connect DEV as auxiliary instance in RMAN, remote server actually means dev server. So if you ever get this error it should be caused by DEV server.

Read  my blog to know how to duplicate database with backup.

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of Duplicate Db command at 10/11/2017 15:54:56
RMAN-05501: aborting duplication of target database
RMAN-03015: error occurred in stored script Memory Script
RMAN-03009: failure of backup command on c3 channel at 10/11/2017 15:54:56
ORA-17628: Oracle error 19505 returned by remote Oracle server

Error from alert log in DEV :

Create Relation IPS_PACKAGE_UNPACK_HISTORY
Completed: alter database mount
Wed Oct 11 15:54:45 2017
Errors in file /d02/DEV/oratest/11204/admin/DEV_******/diag/rdbms/dev/DEV/trace/DEV_ora_8082.trc:
ORA-19505: failed to identify file "/a02/oraclone/DEV/data/undo01.dbf"
ORA-27040: file create error, unable to create file
SVR4 Error: 2: No such file or directory
Additional information: 1
Wed Oct 11 15:54:45 2017
Errors in file /d02/DEV/oratest/11204/admin/DEV_******/diag/rdbms/dev/DEV/trace/DEV_ora_8084.trc:

I can see the instance is mounted without any issue. Even i opened the trace file too and found about the same error as alert. 

ORA-19505: failed to identify file "/a02/oraclone/DEV/data/undo01.dbf"
ORA-27040: file create error, unable to create file
SVR4 Error: 2: No such file or directory
Additional information: 1


From the error i decided to check the below,

- Is the dev os user has proper permission?
- Is the directory exist?
- Checked the path for data files and log files.

Finally i found out,

db_file_name_convert='/u01/oracle/PROD/data','/a02/oraclone/DEV/data'
log_file_name_convert='/u01/oracle/PROD/data','/a02/oraclone/DEV/data'

As i checked in the pfile i provided the wrong path for the parameters above. path is actually '/d02/DEV/oratest/data', since i provided the wrong one RMAN was trying to look into the path above and returned into error. Error disappeared after i update right path in pfile. 

Its purely processing issue and this post is enough for Larry to promote Oracle 18c self-autonomous database. :)

Happy learning! 
Hope DBA never dies!

Thursday, September 14, 2017

ADOP issues : FS_CLONE error in 12.2.4

Hello Friends,

This issue have been encountered  while running fs_clone. Handling adop is pretty comfortable if we know where to find the error logs exactly. You can read this post once to know where adop error logs should be checked. 

I am running fs_clone since we apply patch as hotpatch. Its not necessary to run fs_clone if you apply a patch through adop full cycle(Means prepare, apply, finalize, cutover, cleanup). If we apply as hot patch, its recommended to run fs_clone once the patch successful to make sure both run and patch file systems are in sync. Make sure no code movement to patch file system is in progress when you run fs_clone since any code copied to patch file system will be overwritten by fs_clone. 

Command to trigger fs_clone :  adop phase=fs_clone

Error  in adop main session:

Program : txkValidateSetup.pl started @ Thu Sep 14 11:02:48 2017

*** Log File = /u01/oracle/PROD/fs_ne/EBSapps/log/adop/28/fs_clone_20170914_110147/PROD_*********/ADOPValidations_Thu_Sep_14_11_02_48_2017.log

SETUP VALIDATION is in progress. This may take few minutes to complete.
        [UNEXPECTED]Error occurred while executing "perl /u01/oracle/PROD/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPValidations.pl  -contextfile=/u01/oracle/PROD/fs2/inst/apps/PROD_******/appl/admin/PROD_******.xml -patchctxfile=/u01/oracle/PROD/fs1/inst/apps/PROD_******/appl/admin/PROD_******.xml -phase=fs_clone -logloc=/u01/oracle/PROD/fs_ne/EBSapps/log/adop/28/fs_clone_20170914_110147/PROD_****** -promptmsg=hide"
        [UNEXPECTED]Error 1 occurred while Executing txkADOPValidation script on ******
        Log file: /u01/oracle/PROD/fs_ne/EBSapps/log/adop/28/adop_20170914_110147.log


[STATEMENT] Please run adopscanlog utility, using the command

"adopscanlog -latest=yes"

to get the list of the log files along with snippet of the error message corresponding to each log file.
  
adop exiting with status = 1 (Fail)

 Nothing in detail in the adop session. So i had to dig into adop logs folder. navigate $ADOP_LOG_HOME where adop writes all of its log. we can find the logs for each session id. 

-bash-3.2$ cd $ADOP_LOG_HOME 

-bash-3.2$ ls -ltr --> for latest session. 


drwxr-xr-x   5 applmgr  oinstall     512 May 11 10:30 26 --> for session id 26
drwxr-xr-x   3 applmgr  oinstall     512 May 30 09:50 27 --> for session id 27
-rw-r--r--   1 applmgr  oinstall    1122 Sep  4 16:54 adop_20170904_165405.log --> adop  main session log
-rw-r--r--   1 applmgr  oinstall    1122 Sep  5 09:23 adop_20170905_092239.log
-rw-r--r--   1 applmgr  oinstall   10133 Sep 14 11:02 CheckWLSAdminPwd.log
drwxr-xr-x   5 applmgr  oinstall     512 Sep 14 11:02 28

As my latest adop session was 28 i navigated into folder named '28' and look for the logs. If you are not sure about the session id, look for the adop screen where it will show the session id while it begins. Alternatively you can query ad_adop_sessions table for more session detail.  

-bash-3.2$ cd 28
  
ls -ltr 

drwxr-xr-x   3 applmgr  oinstall     512 Sep  5 10:18 fs_clone_20170905_101716
-rw-r--r--   1 applmgr  oinstall    4878 Sep  5 10:19 adop_20170905_101716.log
drwxr-xr-x   3 applmgr  oinstall     512 Sep 14 10:59 fs_clone_20170914_105815
-rw-r--r--   1 applmgr  oinstall    6910 Sep 14 10:59 adop_20170914_105815.log
drwxr-xr-x   3 applmgr  oinstall     512 Sep 14 11:02 fs_clone_20170914_110147
-rw-r--r--   1 applmgr  oinstall    5040 Sep 14 11:03 adop_20170914_110147.log

Look for the latest directory.

-bash-3.2$ cd fs_clone_20170914_110147
  
-bash-3.2$ ls
ADOPValidations_Thu_Sep_14_11_02_48_2017.log

There were only one log file since this issue was encountered at the beginning stage itself.   
-bash-3.2$ vi ADOPValidations_Thu_Sep_14_11_02_48_2017.log
  
Nodes with context files in the FND_OAM_CONTEXT_FILES table on both run and patch file systems: NONE
Nodes without context files in the FND_OAM_CONTEXT_FILES table on either/or run and patch file systems: ******
Corrective Action:
- If the run file system context file for a node is missing, run AutoConfig on the run file system of that node to sync with the value with the database.
- If the patch file system context file of a node is missing, run AutoConfig on the patch file system of that node with the -syncctx option as follows to sync with the
value with the database.

On UNIX:
 sh <AD_TOP>/bin/adconfig.sh contextfile=<CONTEXT_FILE> -syncctx



Above is the error logged in the log. As soon as i see this i queried the table to see whether both xml files (run and patch) are present in db. 

select * from apps.FND_OAM_CONTEXT_FILES;






 Results clearly showed me there is no xml file for patch fs in the db. So i ran the below to generate xml file for patch file system. 


-bash-3.2$ sh /u01/oracle/PROD/fs1/EBSapps/appl/ad/12.0.0/bin/adconfig.sh contextfile=/u01/oracle/PROD/fs1/inst/apps/PROD_******/appl/admin/PROD_******.xml -syncctx

Enter the APPS user password:
The log file for this session is located at: /u01/oracle/PROD/fs1/inst/apps/PROD_******/admin/log/09141113/adconfig.log

Option specified      : Synchronize context file
Only context file synchronization will be performed

        Classpath                   : /u01/oracle/PROD/fs1/FMW_Home/Oracle_EBS-app1/shared-libs/ebs-appsborg/WEB-INF/lib/ebsAppsborgManifest.jar:/u01/oracle/PROD/fs1/EBSapps/comn/java/classes

        Using ContextFile     : /u01/oracle/PROD/fs1/inst/apps/PROD_******/appl/admin/PROD_******.xml

        Synchronizing the context file ......COMPLETED

AutoConfig completed successfully.

I again queried to see the changes. Look into #8 row, we can see xml file row has been populated for patch file system now. 




fs_clone completed successfully after this change. 


Find FMW version in EBS R12.2.4


Please follow these steps to find out FMW version of your oracle application.

- Login as application owner (ex : applmgr)
- Source run fs env file

Then do the below ,

-bash-3.2$ cd $FMW_HOME

-bash-3.2$ cd Oracle_EBS-app1/

-bash-3.2$ pwd
/u01/oracle/PROD/fs2/FMW_Home/Oracle_EBS-app1

-bash-3.2$ export ORACLE_HOME=/u01/oracle/PROD/fs2/FMW_Home/Oracle_EBS-app1

-bash-3.2$ ./opatch lsinv
-bash: ./opatch: No such file or directory


-bash-3.2$ ls
META-INF            ccr                 common              install             oraInst.loc         shared-libs
OPatch              cfgtoollogs         deployment_plans    inventory           oui                 srvm
applications        clone               diagnostics         jdk                 setEBSDomainEnv.sh  sysman

-bash-3.2$ cd OPatch/

-bash-3.2$ ./opatch lsinv
Invoking OPatch 11.2.0.1.2

Oracle Interim Patch Installer version 11.2.0.1.2
Copyright (c) 2010, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/oracle/PROD/fs2/FMW_Home/Oracle_EBS-app1
Central Inventory : /u01/oracle/PROD/oraInventory
   from           : /var/opt/oracle/oraInst.loc
OPatch version    : 11.2.0.1.2
OUI version       : 11.2.0.3.0
OUI location      : /u01/oracle/PROD/fs2/FMW_Home/Oracle_EBS-app1/oui
Log file location : /u01/oracle/PROD/fs2/FMW_Home/Oracle_EBS-app1/cfgtoollogs/opatch/opatch2017-09-14_10-48-56AM.log

Patch history file: /u01/oracle/PROD/fs2/FMW_Home/Oracle_EBS-app1/cfgtoollogs/opatch/opatch_history.txt


OPatch detects the Middleware Home as "/u01/oracle/PROD/fs2/FMW_Home"

Lsinventory Output file location : /u01/oracle/PROD/fs2/FMW_Home/Oracle_EBS-app1/cfgtoollogs/opatch/lsinv/lsinventory2017-09-14_10-48-56AM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1):

Oracle E-Business Suite Component                                    11.1.1.6.0 --> FMW version
There are 1 products installed in this Oracle Home.


There are no Interim patches installed in this Oracle Home.


--------------------------------------------------------------------------------

OPatch succeeded.

Monday, September 4, 2017

How to change weblogic password if lost or forgotten : java s_nm_jvm_startup_properties with example


If the Admin Password of an EBS WebLogic Domain is lost or forgotten :

1. Shut down all running services. Since the AdminServer password is not known, the
servers cannot be stopped from the console and so must be killed as follows.

--1. Connect to the Oracle E-Business Suite instance and source the application tier
environment file.

--2. Identify the PIDs of Node Manager, AdminServer, and all running Managed
Servers:
$ ps -ef | grep "NodeManager"
$ ps -ef | grep "weblogic.Name=AdminServer"
$ ps -ef | grep "weblogic.Name=forms-c4ws_server"
$ ps -ef | grep "weblogic.Name=forms_server"
$ ps -ef | grep "weblogic.Name=oafm_server"
$ ps -ef | grep "weblogic.Name=oacore_server"

3. Kill all these processes, starting with Node Manager and followed by the
Managed Servers.

2. Back up these folders, and then delete them:
<EBS_DOMAIN_HOME>/security/ DefaultAuthenticatorInit.ldift
<EBS_DOMAIN_HOME>/servers/<server_name>/data/ldap
<EBS_DOMAIN_HOME>/servers/<server_name>/security/boot.properties
<EBS_DOMAIN_HOME>/servers/<server_name>/data/nodemanager/boot.
properties
Where:
• <EBS_DOMAIN_HOME> is the absolute path of the EBS WebLogic domain
• <server_name> is the name of the server directory under
<EBS_DOMAIN_HOME>.

If the password is not reset correctly, the backed up files and folders can be
restored.
Note: For certain servers, the boot.properties file may be present in
only one location of the two specified above. In such a case, back it
up and then delete it.

--3. Set up a new environment to change the WLS AdminServer password.
1. Start a new session and connect to the Oracle E-Business Suite instance.
2. Do not source the application tier environment file.
3. Run the following command to source the WebLogic Server domain
environment:
$ cd <EBS_DOMAIN_HOME>/bin
$ source setDomainEnv.sh
4. Run the following commands:
$ cd <EBS_DOMAIN_HOME>/security
$ java weblogic.security.utils.AdminAccount <wls_adminuser>
<wls_admin_new_password> .
Where:
• <wls_adminuser> is the same as the value of context variable
s_wls_admin_user
• <wls_admin_new_password> is the new WLS AdminServer password you
wish to set.
Note: Do not omit the trailing period ('.') in the above
command: it is needed to specify the current domain directory.


--4. Start AdminServer from the command line. You will be prompted for the WebLogic
Server username and password, so that the AdminServer boot.properties file
can be generated.
1. Go to the EBS Domain Home:
$ cd <EBS_DOMAIN_HOME>
2. Start AdminServer:
$ java <s_nm_jvm_startup_properties> -Dweblogic.system.
StoreBootIdentity=true -Dweblogic.Name=AdminServer weblogic.
Server

• <s_nm_jvm_startup_properties> is the same as the value of context variable
ss_nm_jvm_startup_properties

Example:

MAke sure to remove trailing space and line separators.

$ java -d64 -Xms1024m -Xmx1024m -XX:CompileThreshold=8000 -XX:PermSize=128m -XX:MaxPermSize=256m -Djava.security.policy=/u01/oracle/PROD/fs2/FMW_Home/wlserver_10.3/server/lib/weblogic.policy -Xverify:none -da -Dplatform.home=/u01/oracle/PROD/fs2/FMW_Home/wlserver_10.3 -Dwls.home=/u01/oracle/PROD/fs2/FMW_Home/wlserver_10.3/server -Dweblogic.home=/u01/oracle/PROD/fs2/FMW_Home/wlserver_10.3/server -Dcommon.components.home=/u01/oracle/PROD/fs2/FMW_Home/oracle_common -Djrf.version=11.1.1 -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger -Ddomain.home=/u01/oracle/PROD/fs2/FMW_Home/user_projects/domains/EBS_domain_PROD -Doracle.server.config.dir=/u01/oracle/PROD/fs2/FMW_Home/user_projects/domains/EBS_domain_PROD/config/fmwconfig/servers/AdminServer -Doracle.domain.config.dir=/u01/oracle/PROD/fs2/FMW_Home/user_projects/domains/EBS_domain_PROD/config/fmwconfig -Digf.arisidbeans.carmlloc=/u01/oracle/PROD/fs2/FMW_Home/user_projects/domains/EBS_domain_PROD/config/fmwconfig/carml -Digf.arisidstack.home=/u01/oracle/PROD/fs2/FMW_Home/user_projects/domains/EBS_domain_PROD/config/fmwconfig/arisidprovider -Doracle.security.jps.config=/u01/oracle/PROD/fs2/FMW_Home/user_projects/domains/EBS_domain_PROD/config/fmwconfig/jps-config.xml -Doracle.deployed.app.dir=/u01/oracle/PROD/fs2/FMW_Home/user_projects/domains/EBS_domain_PROD/servers/AdminServer/tmp/_WL_user -Doracle.deployed.app.ext=/- -Dweblogic.alternateTypesDirectory=/u01/oracle/PROD/fs2/FMW_Home/oracle_common/modules/oracle.ossoiap_11.1.1,/u01/oracle/PROD/fs2/FMW_Home/oracle_common/modules/oracle.oamprovider_11.1.1 -Djava.protocol.handler.pkgs=oracle.mds.net.protocol -Dweblogic.jdbc.remoteEnabled=false -Dportlet.oracle.home=/u01/oracle/PROD/fs2/FMW_Home/oracle_common -Dem.oracle.home=/u01/oracle/PROD/fs2/FMW_Home/oracle_common -Dweblogic.management.discover=true -Dwlw.iterativeDev= -Dwlw.testConsole= -Dwlw.logErrorsToConsole= -Dweblogic.ext.dirs=/u01/oracle/PROD/fs2/FMW_Home/patch_wls1036/profiles/default/sysext_manifest_classpath -Dweblogic.system.StoreBootIdentity=true -Dweblogic.Name=AdminServer weblogic.Server

The above command prompts for the WebLogic Server username and password:
Enter username to boot WebLogic server:
Enter password to boot WebLogic server:

Provide the same credentials as you provided in Step 3.

--5. Change Node Manager password
1. Log in to the WebLogic Administration console.
2. Click the 'Lock & Edit' button.
3. In the left panel, click on the EBS Domain link.
4. Select the 'Security' tab.
5. Click on the 'Advanced' link.
6. Edit the 'Node Manager password' field and set it to the new WebLogic Server
password. The password should be same as set in Step 3.
7. Edit the 'Confirm Node Manager Password' field and set it to the new
WebLogic Server password. The password should be same as set in Step 3.
8. Save and activate the changes.

--6. The first time, AdminServer has to be stopped from the Admin console. Follow
these steps:
1. Log in to the WebLogic Administration console.
2. Shut down AdminServer.

--7. Set up your environment to start AdminServer again. AdminServer should now be
started using the normal AD script, which will also start Node Manager using the
new password.
1. Launch a new session and connect to the Oracle E-Business Suite instance.
2. Source the application tier environment file.
3. Start AdminServer with the following command:
$ $ADMIN_SCRIPTS_HOME/adadminsrvctl.sh

--8. Start the Managed Servers. For the first time, all Managed Servers should be started
from the WebLogic Server Admin console. This step will create boot.properties
files for the respective Managed Servers. Follow these steps:
1. Log in to the WebLogic Server Administration Console
2. Start all Managed Servers, one at a time

--9. Shut down all the Managed Servers. This is so the new credentials will be picked up
at the next startup. Follow these steps:
1. Log in to the WebLogic AdminServer console.
2. Shut down all Managed Servers.
3. Shut down AdminServer.

--10. Shut down Node Manager using the normal AD script.
$ $ADMIN_SCRIPTS_HOME/adnodemgrctl.sh stop

--11. Copy the boot.properties file for each Managed Server.
WebLogic Server native scripts use the boot.properties file. The above steps
have created the boot.properties file under
<EBS_DOMAIN_HOME>/servers/<Managed Server
name>/data/nodemanager, which is used by Node Manager. For each Managed
Server, copy the newly-generated boot.properties file from
<EBS_DOMAIN_HOME>/servers/<Managed Servername>/data/nodemanager to <EBS_DOMAIN_HOME>/servers/<ManagedServer name>/security.
The EBS WebLogic Server domain password has now been changed, and all servers
can now be started using the normal AD scripts.
To start AdminServer:
$ADMIN_SCRIPTS_HOME/adadminsrvctl.sh start
To start the Managed Servers:
$ $ADMIN_SCRIPTS_HOME/admanagedsrvctl.sh start <managed_server_name>

--12. The above steps have changed the Oracle WebLogic AdminServer password on the
run file system. You now need to perform an fs_clone operation, to change the
WebLogic EBS Domain password on the patch file system:
1. Launch a new session and connect to the Oracle E-Business Suite instance.
2. Source the application tier environment file.
3. Run the command:
3-42 Oracle E-Business Suite Setup Guide
$ adop phase=fs_clone

Monday, August 21, 2017

12.2 apps tier cloning fails - EBSapps/comn/clone/FMW/pasteBinary.sh ERROR: Script failed, exit code 255

Adcfgclone is referring to production inventory in test server due to which cloning is returning into the error. I have provided the error and action plan in this post.

Error :

Started ApplyAppsTier at Thu Aug 17 08:20:01 AST 2017
Version:
        ApplyAppsTier.java      :       120.11.12020000.6
#############################################################

START: Creating the FMW Home from ApplyAppsTier
FMW JDK version in use: 1.7.0_25

START: Creating FMW Home.
Running /d02/DEV/appltest/fs2/EBSapps/comn/clone/FMW/pasteBinary.sh -javaHome /d02/DEV/appltest/fs2/EBSapps/comn/util/jdk -al /d02/DEV/appltest/fs2/EBSapps/comn/clone/FMW/FMW_Home.jar -tl /d02/DEV/appltest/fs2/FMW_Home -invPtrLoc /var/opt/oracle/oraInst.loc -ldl /d02/DEV/appltest/fs2/inst/apps/DEV_ahqtest/admin/log/clone/fmwT2PApply -silent true -debug true -executeSysPrereqs false
Script Executed in 655369 milliseconds, returning status 255
ERROR: Script failed, exit code 255


I found the error from the logs below. Error is while creating fmw and i was able to figure out the rror in the folder below.

/d02/DEV/appltest/fs2/inst/apps/DEV_ahqtest/admin/log/clone/fmwT2PApply

Please look out for *.error file where errors will be logged. 

bash-3.2$ ls -ltr

CLONE2017-08-20_03-47-24PM.error  --> Error will be logged here. 
CLONE2017-08-20_03-47-24PM.log

Error logged :

"CLONE2017-08-17_08-20-03AM.error" 19 lines, 1981 characters
SEVERE : Aug 17, 2017 8:30:57 AM - ERROR - CLONE-20218   Cloning is not successful.
SEVERE : Aug 17, 2017 8:30:57 AM - CAUSE - CLONE-20218   An internal operation failed.
SEVERE : Aug 17, 2017 8:30:57 AM - ACTION - CLONE-20218   Provide the clone log and error file for investigation.
java.lang.Exception: Invalid Oracle home as its inventory pointer file /d02/DEV/appltest/fs2/FMW_Home/oracle_common/oraInst.loc has invalid entry for Oracle inventory.
        at oracle.as.clone.util.OracleHomeDepRefUtil.setInvPtrLocInInventoryManager(OracleHomeDepRefUtil.java:327)
        at oracle.as.clone.util.OracleHomeDepRefUtil.backupAndSetInvPtrLocBeforeInstallEngineInvoke(OracleHomeDepRefUtil.java:269)
        at oracle.as.clone.util.OracleHomeDepRefUtil.preSetupBeforeInstallEngineInvoke(OracleHomeDepRefUtil.java:207)
        at oracle.as.clone.util.OracleHomeDepRefUtil.setOHProperties(OracleHomeDepRefUtil.java:465)
        at oracle.as.clone.util.OracleHomeDepRefUtil.setOracleHomeProperties(OracleHomeDepRefUtil.java:424)
        at oracle.as.clone.mwhome.cloner.OracleHomeApplyCloner.updateOracleHomeInfo(OracleHomeApplyCloner.java:1904)
        at oracle.as.clone.mwhome.cloner.NormalOracleHomeApplyCloner.doPostApplyClone(NormalOracleHomeApplyCloner.java:127)
        at oracle.as.clone.request.ApplyCloneRequest.performFirstPostoperation(ApplyCloneRequest.java:248)
        at oracle.as.clone.request.ApplyCloneRequest.applyArchive(ApplyCloneRequest.java:219)
        at oracle.as.clone.request.ApplyCloneRequest._clone(ApplyCloneRequest.java:69)
        at oracle.as.clone.process.CloningExecutionProcess.execute(CloningExecutionProcess.java:131)
        at oracle.as.clone.process.CloningExecutionProcess.execute(CloningExecutionProcess.java:114)
        at oracle.as.clone.client.CloningClient.executeT2PCommand(CloningClient.java:238)
        at oracle.as.clone.client.CloningClient.main(CloningClient.java:115)
SEVERE : Invalid Oracle home as its inventory pointer file /d02/DEV/appltest/fs2/FMW_Home/oracle_common/oraInst.loc has invalid entry for Oracle inventory.

I found the production inventory location in the file above. Corrected the inventory location in the test server in the file below.

bash-3.2$ find ./ -name '*ora*.loc'
$RUN_FS/EBSapps/10.1.2/oraInst.loc

You can also do the below,

Change All orinst.loc
mv /var/opt/oracle/oraInventory  /var/opt/oracle/oraInventory.org
Remove all files under /tmp
Re-run adcfclone




  


Wednesday, May 10, 2017

Oracle EBS login page is not working

Error :

The system has encountered an error when servicing the request, Please try again. If the error persists, please contact the system administrator.

Just i wanted to share the issue i encountered today. We are unable to login to application due to the error above. We got the error instead of login page.

What i did?

- As soon as i get the error in login page, i checked the apache logs. I can see some chain failed error but it was tripping all the time (Not only today), So i decided it should nt be cause of the today error. 
- I logged into admin console and checked all managed servers since we are in 12.2 (Everything was runnig fine). 
-I bounced apache once but no use
- I went to db tier and connected to sqlplus where i found the actual issue. When i connected to sqlplus i got the error below :

ERROR:

ORA-00020: maximum number of processes (200) exceeded

I found the cause of the issue. Application was unable to create db connection since limit is overloaded. 

How i narrowed down the issue?

I did how many processed connected to db.

ps -ef|grep oratest (Make sure you have only one instance running under OS user, I had only one instance running under oratest)

I checked how many connections coming from outside,

ps -ef|grep -i local=no|grep oratest

I found out totally 170 sessions were created from outside. 

How i fixed?

I decided to kill all local=no processes to reduce number of processes.  My environment was test and i knew the impact. 

****Please double check and check with your manager or your peers before you kill any such process in your environment.****

ps -ef|grep -i local=no|grep -i oratest|grep -v grep|awk '{print $2}'|xargs kill -9

Above command will get all the process id's of local=no processes and kill them. 

Within a minute, limit is freed up and i was able to connect to sqlplus. As soon as application were able to connect to db, we got the login page.

Thanks!
Sikky


Sunday, May 7, 2017

11.2.0.3 to 12.1.0.2 upgrade for E-biz R12.2

Hello Friends,

I have  tested the below in my environment and i was able to upgrade the database to 12.1.0.2. This upgrade done for oracle E-biz R12.2 so it covers lot additional steps for application tier as well. Please refer these steps only if you are upgrading for E-business suite database.

a) Take a backup using RMAN
             (or)
b)  Perform Cold Backup  ( if  your database is in NOARCHIVELOG mode)

--> db_backup_directory >> Location of the Database backup.
--> controlfile_backup_directory >> Location of the Controlfile backup.

Ensure that all database components/objects provided by Oracle are VALID in the source database prior to starting the upgrade.
Ensure that you do not have duplicate objects in the SYS and SYSTEM schema.

SQL> SELECT dbms_stats.get_prefs('CONCURRENT') from dual;
When 'concurrent statistics gathering' is not not set to 'FALSE', change the value to 'FALSE before the upgrade.
BEGIN
DBMS_STATS.SET_GLOBAL_PREFS('CONCURRENT','FALSE');
END;
/


Scripts placed at /a02/upgrade_files_12c
cd <location of the script>
$ sqlplus / as sysdba
sql> alter session set nls_language='American';
sql> @dbupgdiag.sql
sql> exit

Review the report
$ORACLE_HOME/rdbms/admin/utlrp.sql

Please run health check scripts as recommended by oracle. Note 136697.1 hcheck.sql" script to check for known problems in Oracle8i, Oracle9i, Oracle10g and Oracle 11g

Script - /a02/upgrade_files_12c

Note 884522.1 How to Download and Run Oracle's Database Pre-Upgrade Utility
@$ORACLE_HOME/rdbms/admin/preupgrd.sql
Review the report and fix the errors.

Take a backup of db links. Copy the output and keep it safer.

SQL> SELECT 'CREATE '||DECODE(U.NAME,'PUBLIC','public ')||'DATABASE LINK '||CHR(10)
||DECODE(U.NAME,'PUBLIC',Null, 'SYS','',U.NAME||'.')|| L.NAME||chr(10)
  2    3  ||'CONNECT TO ' || L.USERID || ' IDENTIFIED BY "'||L.PASSWORD||'" USING
'''||L.HOST||''''
  4    5  ||chr(10)||';' TEXT
  6  FROM SYS.LINK$ L, SYS.USER$ U
  7  WHERE L.OWNER# = U.USER#;



TIMESTAMP WITH TIME ZONE Data Type –Conditional

DST 18 upgrade. This Can be done as part of post upgrade.

Ref : 1665676.1

Optimizer Statistics
EXEC DBMS_STATS.GATHER_DICTIONARY_STATS;

Before upgrading Oracle Database, you must wait until all materialized views have completed refreshing.

Run the following query to determine if there are any materialized view refreshes still in progress:

SQL> select s.obj#,o.obj#,s.containerobj#,lastrefreshdate,pflags,xpflags,o.name,o.owner#, bitand(s.mflags, 8) from obj$ o, sum$ s
where o.obj# = s.obj# and o.type# = 42 AND bitand(s.mflags, 8) = 8;

Ensure That No Files Need Media Recovery Before Upgrading

SELECT * FROM v$recover_file;

Ensure That No Files Are in Backup Mode Before Upgrading, make sure no files are in backup mode.

SQL> SELECT * FROM v$backup WHERE status != 'NOT ACTIVE';


Purge the Database Recycle Bin Before Upgrading :

SQL> PURGE DBA_RECYCLEBIN

NOTE : The database recycle bin must be empty during the upgrade process to avoid possible ORA-00600 errors and to minimize the upgrade time.

Verify SYS and SYSTEM Default tablespace
Ensure the users SYS and SYSTEM have 'SYSTEM' as their default tablespace.
You must have sufficient space in the 'SYSTEM' tablespace or set extents to unlimited.
SQL> SELECT username, default_tablespace
     FROM dba_users
     WHERE username in ('SYS','SYSTEM');


Check whether database has any externally authenticated SSL users

select name from sys.user$ where ext_username is not null and password='GLOBAL';

Location of datafiles, redo logs and control files
Note the location of datafiles, redo logs and control files. Also take a backup of all configuration files like listener.ora, tnsnames.ora, etc. from $ORACLE_HOME.

SQL> SELECT name FROM v$controlfile;
SQL> SELECT file_name FROM dba_data_files;
SQL> SELECT group#, member FROM v$logfile;

Review and Remove any unnecessary hidden/underscore parameters

SQL> SELECT name, value from SYS.V$PARAMETER WHERE name LIKE '\_%' ESCAPE '\' order by name;

Check the XDB ACLs has start_date and end_date ACE attributes
Before upgrading the database to 12c, please run the below query as SYS:

SQL> select aclid, start_date, end_date from xds_ace where start_date is not null;

Download and Install Oracle 12c Release 1 in a new Oracle Home and make sure there are no relinking errors.
Install database examples which is mandatory step for ebs. You wont have oracle database examples with 12c stage, you need to download examples through patch p21419221_121020_platform_9of10.zip.
Install the latest available Patchset from My Oracle Support (if available).
Install the latest opatch available for your platform and database version (if available).
Install the latest available Critical Patch Update (if available).

Copy following  configuration files from the $ORACLE_HOME of the database being upgraded to the new Oracle Home for Oracle Database 12c
          - Parameter file (spfile or pfile)
          - Password file  (orapwsid)

Remove or comment out obsolete and deprecated initialization parameters.

 Stop the listener for the database.

$ lsnrctl stop

After 12c oracle home installation

Execute the below in source:
@?/rdbms/admin/utluppkg.sql
    SET SERVEROUTPUT ON;
    exec dbms_preup.run_fixup_and_report('INVALID_SYS_TABLEDATA')
    SET SERVEROUTPUT OFF;
Save Database Control Files and Data with the emdwgrd Utility
$ emdwgrd -save -sid old_SID -path save_directory


if upgrading database from 11.2.0.3
It is recommended to apply the patch 21550777 on 12.1.0.2 ORACLE_HOME before starting the upgrade to Materialized views related issues.
/d02/DEV/oratest/12.1.0/OPatch/opatch apply

21550777
18966843
19291380
19393542
19472320
19627012
19649152
19779059
19835133
19896336
20093776
20181016
20204035
20294666
20798891
20830911
20887355
21153266
22223463
18793246
19591608
25643931 (19908836)
20766180
21321429
21864513
21904072
21967332
Footnote 2 - If a conflict is reported with Patch 18893947 roll back the conflicting patch before applying Patch 21967332.
22098146
22338374
Footnote 3 - If a conflict is reported with Patch 21188532 roll back the conflicting patch before applying Patch 22338374.
22496904
22731026
23089357
23595848
24481723
24560906
FOotnote 4 - If a conflict is reported with Patch 21286665 roll back the conflicting patch before applying Patch 24560906.
24911308
Footnote 5 - If a conflict is reported with Patch 23745950 roll back the conflicting patch before applying Patch 24911308.
20369415
20880215

Apply Jan cpu 2015

Rollback the patches below

20369415
20766180
21321429
21967332
22731026 (Not in oh)
24684434 (not in oh)
25154882 (Not in oh)
24326444 (Not in oh)
25203714 (Not in oh)
23595848
24481723
24732088
24911308
25643931

Applied Jan 2017 cpu

24732088
cd /a02/patches/12c_upg_files/24917987/24968615/24732088
opatch apply

25101514
24917972 JVM

Applied rolled back patches


20369415
20766180
21321429
21967332
22731026
24684434
25154882
24326444
25203714

$ perl $ORACLE_HOME/nls/data/old/cr9idata.pl


Install JRE 6 or 7 (Copy appsutil directory from source to target oracle home)
Ensure Applications patching cycle is complete
Shut down Applications server processes and database listener
Drop SYS.ENABLED$INDEXES (conditional)
Remove the MGDSYS schema (conditional)

Create a new listener for Oracle Database 12c Release 1 (12.1) database
Shutdown the database.

$ sqlplus "/as sysdba"
SQL> shutdown immediate;

bash-3.2$ export ORACLE_HOME=/d02/DEV/oratest/12.1.0/
bash-3.2$ export PATH=$ORACLE_HOME/bin:$PATH
bash-3.2$ export ORACLE_BASE=/d02/DEV/oratest

Update the oratab entry to set the new ORACLE_HOME pointing to ORCL and disable automatic startup.

Sample : cat /etc/oratab

            #orcl:/opt/oracle/product/11.2/db_1:N
            orcl:/opt/oracle/product/12.2/db_1:N


$ cd $ORACLE_HOME/rdbms/admin
$ sqlplus "/ as sysdba"
SQL> startup UPGRADE
SQL> exit

Run the catctl.pl script from the new Oracle home.
In this release, the new Upgrade Utility, catctl.pl, replaces catupgrd.sql.
To run catctl.pl on Linux:
Example: Where parallelism is 6 ( n=6)

cd $ORACLE_HOME/rdbms/admin
$ORACLE_HOME/perl/bin/perl catctl.pl -n  6 -l $ORACLE_HOME/diagnostics catupgrd.sql



SQL> select count(*) from dba_objects where status='INVALID';

  COUNT(*)
----------
    103861


Post upgrade


Please create stats on fixed objects two weeks after the upgrade using the command:
   EXECUTE DBMS_STATS.GATHER_FIXED_OBJECTS_STATS;


 You should follow that with the execution of rdbms/admin/utlrp.sql, and a
 comparison of invalid objects before and after the upgrade using
 rdbms/admin/utluiobj.sql



DST 18 upgrade

After the upgrade to 12.1.0.2 you can:
* (recommended) update the 12.1.0.2 database(s) to DSTv18 (standard DST version of 12.1.0.2) by :
or using the upg_tzv_check.sql and upg_tzv_apply.sql scripts in note 1585343.1 Scripts to automatically update the RDBMS DST (timezone) version in an 11gR2 or 12cR1 database .
or following note 1509653.1 Updating the RDBMS DST version in 12c Release 1 (12.1.0.1 and up) using DBMS_DST from step 3a) using "18" as (<the new DST version number>) in that note.
* (optional) update to a higher DST than 18 version if this is needed.
The latest DST patch and related note on how to apply this is found in Note 412160.1 Updated Time Zones in Oracle Time Zone File patches under "C) Notes covering the current DST available updates".
Simply follow the note for the latest DST update


Conn / as sysdba
-- this gives the current RDBMS DST version
SELECT version FROM v$timezone_file;
-- check also
SELECT PROPERTY_NAME, SUBSTR(property_value, 1, 30) value
FROM DATABASE_PROPERTIES
WHERE PROPERTY_NAME LIKE 'DST_%'
ORDER BY PROPERTY_NAME;

Conn / as sysdba
spool countstatsTSTZ.log
@countstatsTSTZ.sql
spool off
For most databases the biggest amount of data that is affected by DST updates will be in DBMS_SCHEDULER tables.
Conn / as sysdba
exec dbms_scheduler.purge_log;
Conn / as sysdba
-- check current nr of rows in HISTHEAD / HISTGRM
select count(*) from SYS.WRI$_OPTSTAT_HISTGRM_HISTORY;
select count(*) from SYS.WRI$_OPTSTAT_HISTHEAD_HISTORY;
-- check the current retention of stats
-- the default value is 31
select systimestamp - dbms_stats.get_stats_history_availability from dual;
-- now disable stats retention
exec dbms_stats.alter_stats_history_retention(0);
-- remove all stats
exec DBMS_STATS.PURGE_STATS(systimestamp);
-- check result of purge
select count(*) from SYS.WRI$_OPTSTAT_HISTGRM_HISTORY;
select count(*) from SYS.WRI$_OPTSTAT_HISTHEAD_HISTORY;
-- AFTER the DST update you can set the retention back to the or
Conn / as sysdba
spool upg_tzv_check.log
@upg_tzv_check.sql
spool off
A succesfull run will show at the end:
 INFO: A newer RDBMS DST version than the one currently used is found.
 INFO: Note that NO DST update was yet done.
 INFO: Now run upg_tzv_apply.sql to do the actual RDBMS DST update.
 INFO: Note that the upg_tzv_apply.sql script will
 INFO: restart the database 2 times WITHOUT any confirmation or prompt.

Conn / as sysdba
spool upg_tzv_apply.log
@upg_tzv_apply.sql
spool off
A succesfull run will show at the end:
 INFO: The RDBMS DST update is successfully finished.
 INFO: Make sure to exit this sqlplus session.
 INFO: Do not use it for timezone related selects.
To see what's happening during upg_tzv_check.sql or upg_tzv_apply.sql one can use:
CONN / as sysdba

set PAGES 1000
select TARGET, TO_CHAR(START_TIME,'HH24:MI:SS - DD-MM-YY'), TIME_REMAINING, SOFAR,
TOTALWORK, SID, SERIAL#, OPNAME from V$SESSION_LONGOPS
where sid in
(select SID from V$SESSION where CLIENT_INFO = 'upg_tzv')
and SOFAR < TOTALWORK
order by START_TIME;

select S.SID, S.SERIAL#, S.SQL_ID, S.PREV_SQL_ID,
S.EVENT#, S.EVENT, S.P1TEXT, S.P1, S.P2TEXT, S.P2, S.P3TEXT, S.P3, S.TIME_REMAINING_MICRO,
S.SEQ#, S.BLOCKING_SESSION, BS.PROGRAM "Blocking Program",
Q1.SQL_TEXT "Current SQL", Q2.SQL_TEXT "Previous SQL"
from V$SESSION S, V$SQLAREA Q1, V$SQLAREA Q2, V$SESSION BS
where S.SQL_ID = Q1.SQL_ID(+)
and S.PREV_SQL_ID = Q2.SQL_ID(+)
and S.BLOCKING_SESSION = BS.SID(+)
and S.CLIENT_INFO = 'upg_tzv';


Fix for ORA-01804: failure to initialize timezone information:

This is resolved by editing the template ad8idbux.env in location .../oracle/db/tech_st/11.2.0/appsutil/template changing:

From:
set ORA_TZFILE=%s_db_oh%\oracore\zoneinfo\timezlrg.dat

To:
set ORA_TZFILE=%s_db_oh%\oracore\zoneinfo\timezlrg_17.dat



exec dbms_stats.alter_stats_history_retention(31);
After the upgrade, run utluiobj.sql from $ORACLE_HOME/rdbms/admin/ to identify/compare any new invalid objects due to the upgrade.
@$ORACLE_HOME/rdbms/admin/utluiobj.sql

20880215
AND perform one of the following as the table is already logically corrupted
a. Reinitialize the columns in the table:
alter table <owner>.<table_name> modify ( <column_name> null);
alter table <owner>.<table_name> modify ( <column_name> not null);
OR
b. Drop and recreate the affected table(s) ("schema"."table")

JAN CPU 2017 - Patch post installation instructions.

Is it enough if you run datapatch once for all patches. Though catctl.pl will cover this step. We can execute datapatch once if needed.

% ./datapatch –verbose


1. If the OJVM PSU is also installed, you may see invalid objects after execution of datapatch in the previous step. If this is the case, run utlrp.sql to revalidate these objects.
cd $ORACLE_HOME/rdbms/admin
    sqlplus /nolog
2. SQL> CONNECT / AS SYSDBA
3. SQL> @utlrp.sql
4. This patch now includes the OJVM Mitigation patch (Patch:19721304). If an OJVM PSU is installed or planned to be installed, no further actions are necessary. Otherwise, the workaround of using the OJVM Mitigation patch can be activated. As SYSDBA do the following from the admin directory:
5. SQL > @dbmsjdev.sql
6. SQL > exec dbms_java_dev.disable
For more information on the OJVM mitigation patch, see Document 1929745.1 Oracle Recommended Patches -- "Oracle JavaVM Component Database PSU" (OJVM PSU) Patches.

2.6.3 Upgrade Oracle Recovery Manager Catalog
If you are using the Oracle Recovery Manager, the catalog needs to be upgraded. Enter the following command to upgrade it:
$ rman catalog username/password@alias
RMAN> UPGRADE CATALOG;
RMAN> UPGRADE CATALOG;

SELECT TO_CHAR(action_time, 'DD-MON-YYYY HH24:MI:SS') AS action_time,
       action,
             version,
status,
      bundle_series from sys.dba_registry_sqlpatch order by action_time;

Copy $APPL_TOP/admin/adgrants.sql (adgrants_nt.sql for Windows) from the administration server node to the database server node.

sqlplus "/ as sysdba" @adgrants.sql (or adgrants_nt.sql)
    [APPS schema name]


SQL> conn / as sysdba
Connected.
SQL> @?/rdbms/admin/adgrants.sql APPS
Connected.

Set CTXSYS parameter

$ sqlplus "/ as sysdba"
SQL> exec ctxsys.ctx_adm.set_parameter('file_access_role', 'public');

Validate Workflow ruleset
On the administration server node, use SQL*Plus to connect to the database as APPS and run the $FND_TOP/patch/115/sql/wfaqupfix.sql script using the following command:
$ sqlplus [APPS user]/[APPS password] @wfaqupfix.sql\
  [APPLSYS user] [APPS user]

Deregister the current database server (conditional)
If you plan to change the database port, host, or database name parameter on the database server, you must also update AutoConfig on the database tier and deregister the current database server node.

$ sqlplus apps/[APPS password]
SQL> exec fnd_conc_clone.setup_clean;

Implement and run AutoConfig
Enabling AutoConfig on a New Oracle Home

 Install Java Runtime Environment (JRE) on the Database tier
** copy entire appsutil directory from source to target oracle home
The JRE resides in the <ORACLE_HOME>/appsutil/jre directory on the database
tier.

3. Generate the Database Context File
$ perl <RDBMS_ORACLE_HOME>/appsutil/bin/adbldxml.pl

4. Run AutoConfig on the Database tier
Run AutoConfig on the database tier by executing one of the following commands:
On UNIX:
$ <RDBMS_ORACLE_HOME>/appsutil/bin/adconfig.sh \
contextfile=<context_file>


References :
Complete Checklist for Manual Upgrades to Oracle Database 12c Release 1 (12.1) (Doc ID 1503653.1)
Interoperability Notes Oracle EBS 12.2 with Oracle Database 12c Release 1 (Doc ID 1926201.1)

Saturday, May 6, 2017

ORA-01804: failure to initialize timezone information

This error encountered during 12c upgrade. My source database was running in 11.2.0.3 with DSTv17 and i was upgrading database to 12.1.0.2. I had to updgrade DST to v18 as part of the upgrade. AFter DST upgrade i was getting the error below while connecting to sqlplus and while running autoconfig as well.

Error :

ORA-01804: failure to initialize timezone information

SP2-0152: ORACLE may not be functioning properly

The first thing to do when you hit ORA-1804 is to check which timezone file Oracle attempts to use:
If the ORA_TZFILE environment variable is set then that points to the file that will be used.
If ORA_TZFILE is not set then until Oracle 9.2.0.5 $ORACLE_HOME/oracore/zoneinfo/timezone.dat is the default file that is used, since the 9.2.0.5 patchset (and all higher versions up to 11.1.0.7) the default file is $ORACLE_HOME/oracore/zoneinfo/timezlrg.dat
Since 11.2 the .dat files have a prefix indicating the DST version. Hence there are NO timezone.dat or timezlrg.dat in 11.2 and higher . 

For example timezlrg_4.dat is the DSTv4 "large" file, timezlrg_11.dat is the DSTv11 "large" file.
The first thing to do is to check if ORA_TZFILE is set and if it is, remove it and restart the database and listener. Oracle will then try to load $ORACLE_HOME/oracore/zoneinfo/timezlrg.dat or, from 11.2 onwards, the $ORACLE_HOME/oracore/zoneinfo/timezlrg_XX.dat file with XX being the higher number available in the $ORACLE_HOME/oracore/zoneinfo/ directory.

If for some reason the file that Oracle wants to use is not available then this is the cause of the problem.
If the file is present then an alternative reason for the error could be that Oracle cannot load the file because of permission problems.

If all above is correct then it might be a .dat file that is corrupt.

In my scenario, timezlrg.dat file which is set to ORA_TZFILE was not present. From 11g onwards we have version introduced in file name. So i updated the file name as below in ad8idbux.env as mentioned below :

bash-3.2$ echo $ORA_TZFILE
/d02/DEV/oratest/12.1.0/oracore/zoneinfo/timezlrg.dat
bash-3.2$ ls -ltr /d02/DEV/oratest/12.1.0/oracore/zoneinfo/timezlrg.dat
/d02/DEV/oratest/12.1.0/oracore/zoneinfo/timezlrg.dat: No such file or directory

You can find timezone files present under the location /d02/DEV/oratest/12.1.0/oracore/zoneinfo. Update the highest version available. I updated timezlrg_17.dat

---------------------In ad8idbux.env file---------------------

if test "%s_database%" = "db112" -o "%s_database%" = "db121" ; then
     ORA_TZFILE=""
else
     ORA_TZFILE="$ORACLE_HOME/oracore/zoneinfo/timezlrg.dat"
fi
export ORA_TZFILE


1. This is resolved by editing the template ad8idbux.env in location .../oracle/db/tech_st/11.2.0/appsutil/template changing:

From:
set ORA_TZFILE=%s_db_oh%\oracore\zoneinfo\timezlrg.dat

To:
set ORA_TZFILE=%s_db_oh%\oracore\zoneinfo\timezlrg_17.dat

2. Running AutoConfig after this change means the environment file is created with the ORA_TZFILE value as timezlrg_14.dat and AutoConfig completes successfully


References :
Autoconfig Failed On 12c Database With "ORA-01804: failure to initialize timezone information" (Doc ID 1908836.1)
AutoConfig Encounters A ORA-1804 Error: Failure To Initialize Timezone Information (Doc ID 1917923.1)
AutoConfig Encounters A ORA-1804 Error: Failure To Initialize Timezone Information (Doc ID 1917923.1)




Note : these .dat files are only read by Oracle, never written to, so if these files become corrupt then the reason for they becoming corrupt is at OS level or I/O subsystem.
So while copying / reinstalling those files might solve the symptom (the ORA-1804) , the root cause is non-Oracle and a deep diagnostic of the filesystem/ I/O subsystem is strongly advised.
Also there is a big chance these .dat files are not the only files who became corrupt.

Because these timezone files are static and can only be updated by Oracle development the solution is the same in both cases:
Find a different database (same RDBMS and (!) DST version & platform) that does not have these problems (check for example by selecting from V$TIMEZONE_NAMES)
Copy the *.dat files from that system over to the failing system
Restart all Oracle processes
or Reinstall the Oracle software.

Tuesday, May 2, 2017

Steps to setup forms in linux

EBS r12 Forms are not opening in linux?

Below issue can be easily resolve in windows just by installing jre but in linux it is not an easier one.

If you get this error in linux :

In order to access this application, you must install the J2SE Plugin version 1.6.0_07 on your client and NPX_PLUGIN_PATH environment variable is set before starting Netscape. To install this plugin, click here to download the oaj2se.exe executable. Once the download is complete, double-click the oaj2se.exe file to install the plugin. You will be prompted to restart your browser when the installation is complete.

First do the below :

http://onlineappsdba.com/index.php/2007/10/12/access-oracle-apps-r12-11i-from-linux-client-mozilla-firefox/

But the above one dint work out for me. My mozilla and oel were 64-bit so i did the below :

When you done with the plugin installation, issue about:plugins and check whether your installed jre plugin is showing in mozilla. if it does you can access the forms.


To know the recommended jre plugin version issue the below :

[oracle@dev186 scripts]$ grep plugin $CONTEXT_FILE
         <!-- JDK plugins -->
         <sun_plugin_ver oa_var="s_sun_plugin_ver">1.6.0_07</sun_plugin_ver>
         <sun_plugin_type oa_var="s_sun_plugin_type">jdk</sun_plugin_type>


Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.18) Gecko/20110621 Oracle/3.6-1.0.1.el5_6 Firefox/3.6.18

Download and install the below : (Make sure you os bit version when you download)

install ---> jre-6u33-linux-x64.bin

Since my os were 64 bit we must create soft link to /usr/lib64 folder instead of /usr/lib
go to /usr/lib64/mozilla/plugins

Create softlink as below :
ln -s /oracle/jre1.6.0_33/lib/amd64/libnpjp2.so  .

libnpjp2.so -> /oracle/jre1.6.0_33/lib/amd64/libnpjp2.so

Once you are done, make sure you see your installed plugin in mozilla. to check issue about:plugins in mozilla URL.

If you see it you should be able to access forms now.

Enjoy EBS :)

How to start or stop ebs R12 services?

How to start or stop ebs R12 services?


This exercise will be really helpful for beginners who start their career into oracle APPS. I have just explained on how to start and stop oracle apps R12.. R12 has come up with lot of automation scripts and it reduces human intervention a lot. Before we start services, we must understand in which order they should be started..


Starting services :


1) Source your db environment file

2)Start the DB and DB listener

3)Start the applications with adstrtal script. 


(Note : Nothing can be done from application tier when your db or db listener is down)


Stopping services :


1) Source apps environment file

2) Stop applications with adstpall script

3) shutdown the db


Starting the application services :


Source db env file




Connecting through SQL* plus utility in order to start db


To start db


To source env file :

Go to $ORACLE_HOME 

[oracle@dev186 ~]$ cd /oracle/PROD/db/tech_st/11.1.0/ 
[oracle@dev186 11.1.0]$ . PROD_dev186.env 
[oracle@dev186 11.1.0]$ echo $ORACLE_HOME 
/oracle/PROD/db/tech_st/11.1.0

To make sure you have source correct db, issue the below :

[oracle@dev186 11.1.0]$ echo $ORACLE_SID
PROD


[oracle@dev186 11.1.0]$ sqlplus / as sysdba

SQL*Plus: Release 11.1.0.7.0 - Production on Mon Jun 23 09:21:48 2014

Copyright (c) 1982, 2008, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup
ORACLE instance started.

Total System Global Area 1068937216 bytes
Fixed Size                  2166536 bytes
Variable Size             427819256 bytes
Database Buffers          624951296 bytes
Redo Buffers               14000128 bytes
Database mounted.
Database opened.
SQL> exit

Its a best practice for a DBA to read logs.. In order to check db alert log go to the directory below and open alert log using tail command.

Alert log --> /oracle/PROD/db/tech_st/11.1.0/admin/PROD_dev186/diag/rdbms/prod/PROD/trace

Starting the DB Listener 

[oracle@dev186 11.1.0]$ lsnrctl start PROD

LSNRCTL for Linux: Version 11.1.0.7.0 - Production on 23-JUN-2014 09:24:39

Copyright (c) 1991, 2008, Oracle.  All rights reserved.

Starting /oracle/PROD/db/tech_st/11.1.0/bin/tnslsnr: please wait...

TNSLSNR for Linux: Version 11.1.0.7.0 - Production
System parameter file is /oracle/PROD/db/tech_st/11.1.0/network/admin/PROD_dev186/listener.ora
Log messages written to /oracle/PROD/db/tech_st/11.1.0/log/diag/tnslsnr/dev186/prod/alert/log.xml
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=dev186.chainsys.com)(PORT=1523)))

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=dev186.chainsys.com)(PORT=1523)))
STATUS of the LISTENER
------------------------
Alias                     PROD
Version                   TNSLSNR for Linux: Version 11.1.0.7.0 - Production
Start Date                23-JUN-2014 09:24:40
Uptime                    0 days 0 hr. 0 min. 0 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /oracle/PROD/db/tech_st/11.1.0/network/admin/PROD_dev186/listener.ora
Listener Log File         /oracle/PROD/db/tech_st/11.1.0/log/diag/tnslsnr/dev186/prod/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=dev186.chainsys.com)(PORT=1523)))
Services Summary...
Service "PROD" has 1 instance(s).
  Instance "PROD", status UNKNOWN, has 1 handler(s) for this service...
The command completed successfully
[oracle@dev186 11.1.0]$


So DB part is done, next we need to start the applications. Open an another terminal and go to $APPL_TOP to source apps env file. 

[oracle@dev186 ~]$ cd /oracle/PROD/apps/apps_st/appl/
[oracle@dev186 appl]$ . APPSPROD_dev186.env


Go to $ADMIN_SCRIPTS_HOME directory where you see lot scripts we use in day to day activities.

[oracle@dev186 appl]$ cd $ADMIN_SCRIPTS_HOME
[oracle@dev186 scripts]$ pwd
/oracle/PROD/inst/apps/PROD_dev186/admin/scripts
[oracle@dev186 scripts]$ ls
adalnctl.sh   adexecsql.pl     adoafmctl.sh   adstrtal.sh  ieo          L383162.log  L383167.log    sqlnet.log
adapcctl.sh   adformsctl.sh    adopmnctl.sh   afiedt.buf   java.sh      L383163.log  msc
adautocfg.sh  adformsrvctl.sh  adpreclone.pl  cmclean.sql  jtffmctl.sh  L383165.log  mwactl.sh
adcmctl.sh    adoacorectl.sh   adstpall.sh    gsmstart.sh  L383161.log  L383166.log  mwactlwrpr.sh



adalnctl.sh --> TO start apps listener alone

adoafmctl.sh --> To start oafm services alone

adapcctl.sh--> To start apache web services alone

adformsctl.sh --> To start forms services alone

adopmnctl.sh--> TO start opmn services altogether... (Apache, oacore, opmn and forms)

adcmctl.sh --> to start concurrent services alone

All these services can be started alone or can be started in a single script adstrtal.sh script.

if you execute it it will execute all the scripts above.




[oracle@dev186 scripts]$ ./adstrtal.sh apps/apps

You are running adstrtal.sh version 120.15

The logfile for this session is located at /oracle/PROD/inst/apps/PROD_dev186/logs/appl/admin/log/adstrtal.log
Executing service control script:
/oracle/PROD/inst/apps/PROD_dev186/admin/scripts/adopmnctl.sh start
script returned:
****************************************************

You are running adopmnctl.sh version 120.6

Starting Oracle Process Manager (OPMN) ...
opmnctl: opmn started.

adopmnctl.sh: exiting with status 0

adopmnctl.sh: check the logfile /oracle/PROD/inst/apps/PROD_dev186/logs/appl/admin/log/adopmnctl.txt for more information ...  


.end std out.

.end err out.

****************************************************


Executing service control script:
/oracle/PROD/inst/apps/PROD_dev186/admin/scripts/adalnctl.sh start
script returned:
****************************************************

adalnctl.sh version 120.3

Checking for FNDFS executable.
Starting listener process APPS_PROD.

adalnctl.sh: exiting with status 0


adalnctl.sh: check the logfile /oracle/PROD/inst/apps/PROD_dev186/logs/appl/admin/log/adalnctl.txt for more information ...  


.end std out.

.end err out.

****************************************************


Executing service control script:
/oracle/PROD/inst/apps/PROD_dev186/admin/scripts/adapcctl.sh start
script returned:
****************************************************

You are running adapcctl.sh version 120.7.12010000.2

Starting OPMN managed Oracle HTTP Server (OHS) instance ...
opmnctl: opmn is already running.
opmnctl: starting opmn managed processes...

adapcctl.sh: exiting with status 0

adapcctl.sh: check the logfile /oracle/PROD/inst/apps/PROD_dev186/logs/appl/admin/log/adapcctl.txt for more information ...  


.end std out.

.end err out.

****************************************************


Executing service control script:
/oracle/PROD/inst/apps/PROD_dev186/admin/scripts/adoacorectl.sh start
script returned:
****************************************************

You are running adoacorectl.sh version 120.13

Starting OPMN managed OACORE OC4J instance  ...

adoacorectl.sh: exiting with status 0

adoacorectl.sh: check the logfile /oracle/PROD/inst/apps/PROD_dev186/logs/appl/admin/log/adoacorectl.txt for more information ...  


.end std out.

.end err out.

****************************************************


Executing service control script:
/oracle/PROD/inst/apps/PROD_dev186/admin/scripts/adformsctl.sh start
script returned:
****************************************************

You are running adformsctl.sh  version 120.16

Starting OPMN managed FORMS OC4J instance  ...

adformsctl.sh: exiting with status 0

adformsctl.sh: check the logfile /oracle/PROD/inst/apps/PROD_dev186/logs/appl/admin/log/adformsctl.txt for more information ...  


.end std out.

.end err out.

****************************************************


Executing service control script:
/oracle/PROD/inst/apps/PROD_dev186/admin/scripts/adoafmctl.sh start
script returned:
****************************************************

You are running adoafmctl.sh version 120.8

Starting OPMN managed OAFM OC4J instance  ...

adoafmctl.sh: exiting with status 0

adoafmctl.sh: check the logfile /oracle/PROD/inst/apps/PROD_dev186/logs/appl/admin/log/adoafmctl.txt for more information ...  


.end std out.

.end err out.

****************************************************


Executing service control script:
/oracle/PROD/inst/apps/PROD_dev186/admin/scripts/adcmctl.sh start
script returned:
****************************************************

You are running adcmctl.sh version 120.17.12010000.3

Starting concurrent manager for PROD ...
Starting PROD_0623@PROD Internal Concurrent Manager
Default printer is noprint

adcmctl.sh: exiting with status 0


adcmctl.sh: check the logfile /oracle/PROD/inst/apps/PROD_dev186/logs/appl/admin/log/adcmctl.txt for more information ...  


.end std out.

.end err out.

****************************************************


Executing service control script:
/oracle/PROD/inst/apps/PROD_dev186/admin/scripts/jtffmctl.sh start
script returned:
****************************************************

You are running jtffmctl.sh version 120.3

Validating Fulfillment patch level via /oracle/PROD/apps/apps_st/comn/java/classes
Fulfillment patch level validated.
Starting Fulfillment Server for PROD on port 9302 ...

jtffmctl.sh: exiting with status 0


.end std out.

.end err out.

****************************************************


All enabled services for this node are started.

adstrtal.sh: Exiting with status 0

adstrtal.sh: check the logfile /oracle/PROD/inst/apps/PROD_dev186/logs/appl/admin/log/adstrtal.log for more information ...  

Note :  MAke sure all the services are exiting with status 0 which means there are no issues. If any of the services are ending with numbers we need to investigate depends on the issue.

To Make sure opmn is up and running issue the below,

[oracle@dev186 scripts]$ ./adopmnctl.sh status

You are running adopmnctl.sh version 120.6

Checking status of OPMN managed processes...

Processes in Instance: PROD_dev186.dev186.chainsys.com
---------------------------------+--------------------+---------+---------
ias-component                    | process-type       |     pid | status  
---------------------------------+--------------------+---------+---------
OC4JGroup:default_group          | OC4J:oafm          |   16277 | Alive   
OC4JGroup:default_group          | OC4J:forms         |   16203 | Alive   
OC4JGroup:default_group          | OC4J:oacore        |   16117 | Alive   
HTTP_Server                      | HTTP_Server        |   16060 | Alive   


adopmnctl.sh: exiting with status 0

adopmnctl.sh: check the logfile /oracle/PROD/inst/apps/PROD_dev186/logs/appl/admin/log/adopmnctl.txt for more information ...  

You have new mail in /var/spool/mail/oracle


All the four services should be in Alive status after you execute adstral.sh. if they are in down or init status there are some issues.. Users will be able to open forms only when all these services are alive.

To MAke sure listener is up and running :

[oracle@dev186 scripts]$ ps -ef|grep tns 
root        21     2  0 09:13 ?        00:00:00 [netns] 
oracle    5403     1  0 09:24 ?        00:00:00 /oracle/PROD/db/tech_st/11.1.0/bin/tnslsnr PROD -inherit 
oracle   16022     1  0 09:43 ?        00:00:00 /oracle/PROD/apps/tech_st/10.1.2/bin/tnslsnr APPS_PROD -inherit 
oracle   16782  4537  0 09:45 pts/1    00:00:00 grep tns 

DB listener should be up and running in order to function application...

LEts see how to stop the application now :


Go to $ADMIN_SCRIPTS_HOME and execute adstpall.sh script to stop all the applications. You can stop all the applications in a single script or one by one with the each script. For ex: if there is any issue with concurrent and you need to down concurrent manager is it enough if you just issue

[oracle@dev186 scripts]$ ./adcmctl.sh stop apps/apps

Note : Among all the scripts concurrent service is the only script which will you ask apps user password.

To start again,

[oracle@dev186 scripts]$ ./adcmctl.sh start apps/apps

To down all the services in a single script,

[oracle@dev186 scripts]$ ./adstpall.sh apps/apps 

You are running adstpall.sh version 120.10 

The logfile for this session is located at /oracle/PROD/inst/apps/PROD_dev186/logs/appl/admin/log/adstpall.log 
Executing service control script: 
/oracle/PROD/inst/apps/PROD_dev186/admin/scripts/jtffmctl.sh stop 
script returned: 
**************************************************** 

You are running jtffmctl.sh version 120.3 

Shutting down Fulfillment Server for PROD on port 9302 ... 

jtffmctl.sh: exiting with status 0 


.end std out. 

.end err out. 

**************************************************** 


Executing service control script: 
/oracle/PROD/inst/apps/PROD_dev186/admin/scripts/adcmctl.sh stop 
script returned: 
**************************************************** 

You are running adcmctl.sh version 120.17.12010000.3 

Shutting down concurrent managers for PROD ... 
ORACLE Password: 
Submitted request 390585 for CONCURRENT FND SHUTDOWN 


adcmctl.sh: exiting with status 0 


adcmctl.sh: check the logfile /oracle/PROD/inst/apps/PROD_dev186/logs/appl/admin/log/adcmctl.txt for more information ...   


.end std out. 

.end err out. 

**************************************************** 


Executing service control script: 
/oracle/PROD/inst/apps/PROD_dev186/admin/scripts/adoafmctl.sh stop 
script returned: 
**************************************************** 

You are running adoafmctl.sh version 120.8 

Stopping OPMN managed OAFM OC4J instance ... 

adoafmctl.sh: exiting with status 0 

adoafmctl.sh: check the logfile /oracle/PROD/inst/apps/PROD_dev186/logs/appl/admin/log/adoafmctl.txt for more information ...   


.end std out. 

.end err out. 

**************************************************** 


Executing service control script: 
/oracle/PROD/inst/apps/PROD_dev186/admin/scripts/adformsctl.sh stop 
script returned: 
**************************************************** 

You are running adformsctl.sh  version 120.16 

Stopping OPMN managed FORMS OC4J instance  ... 

adformsctl.sh: exiting with status 0 

adformsctl.sh: check the logfile /oracle/PROD/inst/apps/PROD_dev186/logs/appl/admin/log/adformsctl.txt for more information ...   


.end std out. 

.end err out. 

**************************************************** 


Executing service control script: 
/oracle/PROD/inst/apps/PROD_dev186/admin/scripts/adoacorectl.sh stop 
script returned: 
**************************************************** 

You are running adoacorectl.sh version 120.13 

Stopping OPMN managed OACORE OC4J instance ... 

adoacorectl.sh: exiting with status 0 

adoacorectl.sh: check the logfile /oracle/PROD/inst/apps/PROD_dev186/logs/appl/admin/log/adoacorectl.txt for more information ...   


.end std out. 

.end err out. 

**************************************************** 


Executing service control script: 
/oracle/PROD/inst/apps/PROD_dev186/admin/scripts/adapcctl.sh stop 
script returned: 
**************************************************** 

You are running adapcctl.sh version 120.7.12010000.2 

Stopping OPMN managed Oracle HTTP Server (OHS) instance ... 
opmnctl: stopping opmn managed processes... 

adapcctl.sh: exiting with status 0 

adapcctl.sh: check the logfile /oracle/PROD/inst/apps/PROD_dev186/logs/appl/admin/log/adapcctl.txt for more information ...   


.end std out. 

.end err out. 

**************************************************** 


Executing service control script: 
/oracle/PROD/inst/apps/PROD_dev186/admin/scripts/adalnctl.sh stop 
script returned: 
**************************************************** 

adalnctl.sh version 120.3 

Shutting down listener process APPS_PROD. 

adalnctl.sh: exiting with status 0 


adalnctl.sh: check the logfile /oracle/PROD/inst/apps/PROD_dev186/logs/appl/admin/log/adalnctl.txt for more information ...   


.end std out. 

.end err out. 

**************************************************** 


Executing service control script: 
/oracle/PROD/inst/apps/PROD_dev186/admin/scripts/adopmnctl.sh stop 
script returned: 
**************************************************** 

You are running adopmnctl.sh version 120.6 

Stopping Oracle Process Manager (OPMN)  and the managed processes ... 
opmnctl: stopping opmn and all managed processes... 

adopmnctl.sh: exiting with status 0 

adopmnctl.sh: check the logfile /oracle/PROD/inst/apps/PROD_dev186/logs/appl/admin/log/adopmnctl.txt for more information ...   


.end std out. 

.end err out. 

**************************************************** 


All enabled services on this node are stopped. 

adstpall.sh:Exiting with status 0 

adstpall.sh: check the logfile /oracle/PROD/inst/apps/PROD_dev186/logs/appl/admin/log/adstpall.log for more information ...   

Make sure opmn is down with the below ,

oracle@dev186 scripts]$ ./adopmnctl.sh status

It should show all the services are down.. also issue 

ps -ef|grep FNDLIBR --> to check conc manager is down
ps -ef|grep tns --> listener is down

Once everything down in apps side, go to db side and shut down the db and listener.

Happy learning.. I hope this blog is helpful for beginners..



You cannot complete this task because one of the following events caused a loss of page data: A system failure has occurred "Front end is locked after 12.2 upgrade"

  If you have upgraded to Oracle E-Business Suite 12.2.10 Release Update Pack from Oracle E-Business Suite 12.2.6 Release Update Pack (or ea...