How to attach ebs 10.1.2 oracle home to central inventory

Hello,

This post is to define "How to attach ebs 10.1.2 oracle home to central inventory"

bash-3.2$ opatch apply

Oracle Interim Patch Installer version 1.0.0.0.64
Copyright (c) 2011 Oracle Corporation. All Rights Reserved..

Oracle recommends you to use the latest OPatch version
and read the OPatch documentation available in the OPatch/docs
directory for usage. For information about the latest OPatch and
other support-related issues, refer to document ID 293369.1
available on My Oracle Support (https://myoraclesupport.oracle.com)

Oracle Home           : /a01/appstier/UAT/fs2/EBSapps/10.1.2
Oracle Home Inventory : /a01/appstier/UAT/fs2/EBSapps/10.1.2/inventory
Central Inventory     : /var/opt/oracle/oraInventory
   from               : /var/opt/oracle/oraInst.loc
OUI location          : /a01/appstier/UAT/fs2/EBSapps/10.1.2/oui
OUI shared library    : /a01/appstier/UAT/fs2/EBSapps/10.1.2/oui/lib/solaris/liboraInstaller.so
Java location         : /a01/appstier/UAT/fs2/EBSapps/10.1.2/jdk/jre/bin/java
Log file location     : /a01/appstier/UAT/fs2/EBSapps/10.1.2/.patch_storage/<patch ID>/*.log

Creating log file "/a01/appstier/UAT/fs2/EBSapps/10.1.2/.patch_storage/18752577/Apply_18752577_11-25-2017_09-57-56.log"

The Oracle Home /a01/appstier/UAT/fs2/EBSapps/10.1.2 is not registered with the Central Inventory.  OPatch was not able to get details of the home from the inventory.

ERROR: OPatch failed because of Inventory problem.

When we apply patch in oracle home, OPatch checks for the inventory to understand the patch detail which already present. It gives the error above if it does not find any patch details. Since oracle home is not registered with central inventory, opatch could nt get any of the patch details thus it threw an error. We need to attach this particular oracle home to central inventory in order to overcome this error. This is one of the method to overcome this error. 

bash-3.2$ cat /var/opt/oracle/oraInst.loc
inventory_loc=/var/opt/oracle/oraInventory
inst_group=dba

Source env file

bash-3.2$ cd $ORACLE_HOME/oui/bin/

Command to be executed :

./runInstaller -silent -attachHome -invPtrLoc /var/opt/oracle/oraInst.loc ORACLE_HOME="/a01/appstier/UAT/fs2/EBSapps/10.1.2" ORACLE_HOME_NAME="UAT_TOOLS_fs2_EBSapps_10_1_2"

bash-3.2$ ./runInstaller -silent -attachHome -invPtrLoc /var/opt/oracle/oraInst.loc ORACLE_HOME="/a01/appstier/UAT/fs2/EBSapps/10.1.2" ORACLE_HOME_NAME="UAT_TOOLS_fs2_EBSapps_10_1_2"

Starting Oracle Universal Installer...
No pre-requisite checks found in oraparam.ini, no system pre-requisite checks will be executed.
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2017-11-25_10-27-20AM. Please wait ...bash-3.2$

bash-3.2$ 'AttachHome' was successful.

RMAN error : ORA-17628: Oracle error 19505 returned by remote Oracle server

Hello DBA's,

While i was doing active replication from production to test i got the below error in RMAN. RMAN triggered from dev server as usual. When i saw remote server i doubted the prod server then realized i was absolutely wrong. I looked into alert long in prod and everything was clean. I did tnsping and checked listener (lsnrctl status), everything was perfect from production side.

Since we connect DEV as auxiliary instance in RMAN, remote server actually means dev server. So if you ever get this error it should be caused by DEV server.

Read  my blog to know how to duplicate database with backup.

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of Duplicate Db command at 10/11/2017 15:54:56
RMAN-05501: aborting duplication of target database
RMAN-03015: error occurred in stored script Memory Script
RMAN-03009: failure of backup command on c3 channel at 10/11/2017 15:54:56
ORA-17628: Oracle error 19505 returned by remote Oracle server

Error from alert log in DEV :

Create Relation IPS_PACKAGE_UNPACK_HISTORY
Completed: alter database mount
Wed Oct 11 15:54:45 2017
Errors in file /d02/DEV/oratest/11204/admin/DEV_******/diag/rdbms/dev/DEV/trace/DEV_ora_8082.trc:
ORA-19505: failed to identify file "/a02/oraclone/DEV/data/undo01.dbf"
ORA-27040: file create error, unable to create file
SVR4 Error: 2: No such file or directory
Additional information: 1
Wed Oct 11 15:54:45 2017
Errors in file /d02/DEV/oratest/11204/admin/DEV_******/diag/rdbms/dev/DEV/trace/DEV_ora_8084.trc:

I can see the instance is mounted without any issue. Even i opened the trace file too and found about the same error as alert. 

ORA-19505: failed to identify file "/a02/oraclone/DEV/data/undo01.dbf"
ORA-27040: file create error, unable to create file
SVR4 Error: 2: No such file or directory
Additional information: 1


From the error i decided to check the below,

- Is the dev os user has proper permission?
- Is the directory exist?
- Checked the path for data files and log files.

Finally i found out,

db_file_name_convert='/u01/oracle/PROD/data','/a02/oraclone/DEV/data'
log_file_name_convert='/u01/oracle/PROD/data','/a02/oraclone/DEV/data'

As i checked in the pfile i provided the wrong path for the parameters above. path is actually '/d02/DEV/oratest/data', since i provided the wrong one RMAN was trying to look into the path above and returned into error. Error disappeared after i update right path in pfile. 

Its purely processing issue and this post is enough for Larry to promote Oracle 18c self-autonomous database. :)

Happy learning! 

Hope DBA never dies!

ADOP issues : FS_CLONE error in 12.2.4

Hello Friends,

This issue have been encountered  while running fs_clone. Handling adop is pretty comfortable if we know where to find the error logs exactly. You can read this post once to know where adop error logs should be checked. 

I am running fs_clone since we apply patch as hotpatch. Its not necessary to run fs_clone if you apply a patch through adop full cycle(Means prepare, apply, finalize, cutover, cleanup). If we apply as hot patch, its recommended to run fs_clone once the patch successful to make sure both run and patch file systems are in sync. Make sure no code movement to patch file system is in progress when you run fs_clone since any code copied to patch file system will be overwritten by fs_clone. 

Command to trigger fs_clone :  adop phase=fs_clone

Error  in adop main session:

Program : txkValidateSetup.pl started @ Thu Sep 14 11:02:48 2017

*** Log File = /u01/oracle/PROD/fs_ne/EBSapps/log/adop/28/fs_clone_20170914_110147/PROD_*********/ADOPValidations_Thu_Sep_14_11_02_48_2017.log

SETUP VALIDATION is in progress. This may take few minutes to complete.

        [UNEXPECTED]Error occurred while executing "perl /u01/oracle/PROD/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPValidations.pl  -contextfile=/u01/oracle/PROD/fs2/inst/apps/PROD_******/appl/admin/PROD_******.xml -patchctxfile=/u01/oracle/PROD/fs1/inst/apps/PROD_******/appl/admin/PROD_******.xml -phase=fs_clone -logloc=/u01/oracle/PROD/fs_ne/EBSapps/log/adop/28/fs_clone_20170914_110147/PROD_****** -promptmsg=hide"

        [UNEXPECTED]Error 1 occurred while Executing txkADOPValidation script on ******

        Log file: /u01/oracle/PROD/fs_ne/EBSapps/log/adop/28/adop_20170914_110147.log

[STATEMENT] Please run adopscanlog utility, using the command

"adopscanlog -latest=yes"

to get the list of the log files along with snippet of the error message corresponding to each log file.

  

adop exiting with status = 1 (Fail)

 Nothing in detail in the adop session. So i had to dig into adop logs folder. navigate $ADOP_LOG_HOME where adop writes all of its log. we can find the logs for each session id. 

-bash-3.2$ cd $ADOP_LOG_HOME 

-bash-3.2$ ls -ltr --> for latest session. 

drwxr-xr-x   5 applmgr  oinstall     512 May 11 10:30 26 --> for session id 26

drwxr-xr-x   3 applmgr  oinstall     512 May 30 09:50 27 --> for session id 27

-rw-r--r--   1 applmgr  oinstall    1122 Sep  4 16:54 adop_20170904_165405.log --> adop  main session log

-rw-r--r--   1 applmgr  oinstall    1122 Sep  5 09:23 adop_20170905_092239.log

-rw-r--r--   1 applmgr  oinstall   10133 Sep 14 11:02 CheckWLSAdminPwd.log

drwxr-xr-x   5 applmgr  oinstall     512 Sep 14 11:02 28

As my latest adop session was 28 i navigated into folder named '28' and look for the logs. If you are not sure about the session id, look for the adop screen where it will show the session id while it begins. Alternatively you can query ad_adop_sessions table for more session detail.  

-bash-3.2$ cd 28

  

ls -ltr 

drwxr-xr-x   3 applmgr  oinstall     512 Sep  5 10:18 fs_clone_20170905_101716

-rw-r--r--   1 applmgr  oinstall    4878 Sep  5 10:19 adop_20170905_101716.log

drwxr-xr-x   3 applmgr  oinstall     512 Sep 14 10:59 fs_clone_20170914_105815

-rw-r--r--   1 applmgr  oinstall    6910 Sep 14 10:59 adop_20170914_105815.log

drwxr-xr-x   3 applmgr  oinstall     512 Sep 14 11:02 fs_clone_20170914_110147

-rw-r--r--   1 applmgr  oinstall    5040 Sep 14 11:03 adop_20170914_110147.log

Look for the latest directory.

-bash-3.2$ cd fs_clone_20170914_110147

  

-bash-3.2$ ls

ADOPValidations_Thu_Sep_14_11_02_48_2017.log

There were only one log file since this issue was encountered at the beginning stage itself.   

-bash-3.2$ vi ADOPValidations_Thu_Sep_14_11_02_48_2017.log

  

Nodes with context files in the FND_OAM_CONTEXT_FILES table on both run and patch file systems: NONE

Nodes without context files in the FND_OAM_CONTEXT_FILES table on either/or run and patch file systems: ******

Corrective Action:

- If the run file system context file for a node is missing, run AutoConfig on the run file system of that node to sync with the value with the database.

- If the patch file system context file of a node is missing, run AutoConfig on the patch file system of that node with the -syncctx option as follows to sync with the

value with the database.

On UNIX:

 sh <AD_TOP>/bin/adconfig.sh contextfile=<CONTEXT_FILE> -syncctx

Above is the error logged in the log. As soon as i see this i queried the table to see whether both xml files (run and patch) are present in db. 

select * from apps.FND_OAM_CONTEXT_FILES;

 Results clearly showed me there is no xml file for patch fs in the db. So i ran the below to generate xml file for patch file system. 

-bash-3.2$ sh /u01/oracle/PROD/fs1/EBSapps/appl/ad/12.0.0/bin/adconfig.sh contextfile=/u01/oracle/PROD/fs1/inst/apps/PROD_******/appl/admin/PROD_******.xml -syncctx

Enter the APPS user password:

The log file for this session is located at: /u01/oracle/PROD/fs1/inst/apps/PROD_******/admin/log/09141113/adconfig.log

Option specified      : Synchronize context file

Only context file synchronization will be performed

        Classpath                   : /u01/oracle/PROD/fs1/FMW_Home/Oracle_EBS-app1/shared-libs/ebs-appsborg/WEB-INF/lib/ebsAppsborgManifest.jar:/u01/oracle/PROD/fs1/EBSapps/comn/java/classes

        Using ContextFile     : /u01/oracle/PROD/fs1/inst/apps/PROD_******/appl/admin/PROD_******.xml

        Synchronizing the context file ......COMPLETED

AutoConfig completed successfully.

I again queried to see the changes. Look into #8 row, we can see xml file row has been populated for patch file system now. 

fs_clone completed successfully after this change. 

Find FMW version in EBS R12.2.4

Please follow these steps to find out FMW version of your oracle application.

- Login as application owner (ex : applmgr)
- Source run fs env file

Then do the below ,

-bash-3.2$ cd $FMW_HOME

-bash-3.2$ cd Oracle_EBS-app1/

-bash-3.2$ pwd
/u01/oracle/PROD/fs2/FMW_Home/Oracle_EBS-app1

-bash-3.2$ export ORACLE_HOME=/u01/oracle/PROD/fs2/FMW_Home/Oracle_EBS-app1

-bash-3.2$ ./opatch lsinv
-bash: ./opatch: No such file or directory


-bash-3.2$ ls
META-INF            ccr                 common              install             oraInst.loc         shared-libs
OPatch              cfgtoollogs         deployment_plans    inventory           oui                 srvm
applications        clone               diagnostics         jdk                 setEBSDomainEnv.sh  sysman

-bash-3.2$ cd OPatch/

-bash-3.2$ ./opatch lsinv
Invoking OPatch 11.2.0.1.2

Oracle Interim Patch Installer version 11.2.0.1.2
Copyright (c) 2010, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/oracle/PROD/fs2/FMW_Home/Oracle_EBS-app1
Central Inventory : /u01/oracle/PROD/oraInventory
   from           : /var/opt/oracle/oraInst.loc
OPatch version    : 11.2.0.1.2
OUI version       : 11.2.0.3.0
OUI location      : /u01/oracle/PROD/fs2/FMW_Home/Oracle_EBS-app1/oui
Log file location : /u01/oracle/PROD/fs2/FMW_Home/Oracle_EBS-app1/cfgtoollogs/opatch/opatch2017-09-14_10-48-56AM.log

Patch history file: /u01/oracle/PROD/fs2/FMW_Home/Oracle_EBS-app1/cfgtoollogs/opatch/opatch_history.txt


OPatch detects the Middleware Home as "/u01/oracle/PROD/fs2/FMW_Home"

Lsinventory Output file location : /u01/oracle/PROD/fs2/FMW_Home/Oracle_EBS-app1/cfgtoollogs/opatch/lsinv/lsinventory2017-09-14_10-48-56AM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1):

Oracle E-Business Suite Component                                    11.1.1.6.0 --> FMW version
There are 1 products installed in this Oracle Home.


There are no Interim patches installed in this Oracle Home.


--------------------------------------------------------------------------------

OPatch succeeded.

How to change weblogic password if lost or forgotten : java s_nm_jvm_startup_properties with example

If the Admin Password of an EBS WebLogic Domain is lost or forgotten :

1. Shut down all running services. Since the AdminServer password is not known, the
servers cannot be stopped from the console and so must be killed as follows.

--1. Connect to the Oracle E-Business Suite instance and source the application tier
environment file.

--2. Identify the PIDs of Node Manager, AdminServer, and all running Managed
Servers:
$ ps -ef | grep "NodeManager"
$ ps -ef | grep "weblogic.Name=AdminServer"
$ ps -ef | grep "weblogic.Name=forms-c4ws_server"
$ ps -ef | grep "weblogic.Name=forms_server"
$ ps -ef | grep "weblogic.Name=oafm_server"
$ ps -ef | grep "weblogic.Name=oacore_server"

3. Kill all these processes, starting with Node Manager and followed by the
Managed Servers.

2. Back up these folders, and then delete them:
<EBS_DOMAIN_HOME>/security/ DefaultAuthenticatorInit.ldift
<EBS_DOMAIN_HOME>/servers/<server_name>/data/ldap
<EBS_DOMAIN_HOME>/servers/<server_name>/security/boot.properties
<EBS_DOMAIN_HOME>/servers/<server_name>/data/nodemanager/boot.
properties
Where:
• <EBS_DOMAIN_HOME> is the absolute path of the EBS WebLogic domain
• <server_name> is the name of the server directory under
<EBS_DOMAIN_HOME>.

If the password is not reset correctly, the backed up files and folders can be
restored.
Note: For certain servers, the boot.properties file may be present in
only one location of the two specified above. In such a case, back it
up and then delete it.

--3. Set up a new environment to change the WLS AdminServer password.
1. Start a new session and connect to the Oracle E-Business Suite instance.
2. Do not source the application tier environment file.
3. Run the following command to source the WebLogic Server domain
environment:
$ cd <EBS_DOMAIN_HOME>/bin
$ source setDomainEnv.sh
4. Run the following commands:
$ cd <EBS_DOMAIN_HOME>/security
$ java weblogic.security.utils.AdminAccount <wls_adminuser>
<wls_admin_new_password> .
Where:
• <wls_adminuser> is the same as the value of context variable
s_wls_admin_user
• <wls_admin_new_password> is the new WLS AdminServer password you
wish to set.
Note: Do not omit the trailing period ('.') in the above
command: it is needed to specify the current domain directory.


--4. Start AdminServer from the command line. You will be prompted for the WebLogic
Server username and password, so that the AdminServer boot.properties file
can be generated.
1. Go to the EBS Domain Home:
$ cd <EBS_DOMAIN_HOME>
2. Start AdminServer:
$ java <s_nm_jvm_startup_properties> -Dweblogic.system.
StoreBootIdentity=true -Dweblogic.Name=AdminServer weblogic.
Server

• <s_nm_jvm_startup_properties> is the same as the value of context variable
ss_nm_jvm_startup_properties

Example:

MAke sure to remove trailing space and line separators.

$ java -d64 -Xms1024m -Xmx1024m -XX:CompileThreshold=8000 -XX:PermSize=128m -XX:MaxPermSize=256m -Djava.security.policy=/u01/oracle/PROD/fs2/FMW_Home/wlserver_10.3/server/lib/weblogic.policy -Xverify:none -da -Dplatform.home=/u01/oracle/PROD/fs2/FMW_Home/wlserver_10.3 -Dwls.home=/u01/oracle/PROD/fs2/FMW_Home/wlserver_10.3/server -Dweblogic.home=/u01/oracle/PROD/fs2/FMW_Home/wlserver_10.3/server -Dcommon.components.home=/u01/oracle/PROD/fs2/FMW_Home/oracle_common -Djrf.version=11.1.1 -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger -Ddomain.home=/u01/oracle/PROD/fs2/FMW_Home/user_projects/domains/EBS_domain_PROD -Doracle.server.config.dir=/u01/oracle/PROD/fs2/FMW_Home/user_projects/domains/EBS_domain_PROD/config/fmwconfig/servers/AdminServer -Doracle.domain.config.dir=/u01/oracle/PROD/fs2/FMW_Home/user_projects/domains/EBS_domain_PROD/config/fmwconfig -Digf.arisidbeans.carmlloc=/u01/oracle/PROD/fs2/FMW_Home/user_projects/domains/EBS_domain_PROD/config/fmwconfig/carml -Digf.arisidstack.home=/u01/oracle/PROD/fs2/FMW_Home/user_projects/domains/EBS_domain_PROD/config/fmwconfig/arisidprovider -Doracle.security.jps.config=/u01/oracle/PROD/fs2/FMW_Home/user_projects/domains/EBS_domain_PROD/config/fmwconfig/jps-config.xml -Doracle.deployed.app.dir=/u01/oracle/PROD/fs2/FMW_Home/user_projects/domains/EBS_domain_PROD/servers/AdminServer/tmp/_WL_user -Doracle.deployed.app.ext=/- -Dweblogic.alternateTypesDirectory=/u01/oracle/PROD/fs2/FMW_Home/oracle_common/modules/oracle.ossoiap_11.1.1,/u01/oracle/PROD/fs2/FMW_Home/oracle_common/modules/oracle.oamprovider_11.1.1 -Djava.protocol.handler.pkgs=oracle.mds.net.protocol -Dweblogic.jdbc.remoteEnabled=false -Dportlet.oracle.home=/u01/oracle/PROD/fs2/FMW_Home/oracle_common -Dem.oracle.home=/u01/oracle/PROD/fs2/FMW_Home/oracle_common -Dweblogic.management.discover=true -Dwlw.iterativeDev= -Dwlw.testConsole= -Dwlw.logErrorsToConsole= -Dweblogic.ext.dirs=/u01/oracle/PROD/fs2/FMW_Home/patch_wls1036/profiles/default/sysext_manifest_classpath -Dweblogic.system.StoreBootIdentity=true -Dweblogic.Name=AdminServer weblogic.Server

The above command prompts for the WebLogic Server username and password:
Enter username to boot WebLogic server:
Enter password to boot WebLogic server:

Provide the same credentials as you provided in Step 3.

--5. Change Node Manager password
1. Log in to the WebLogic Administration console.
2. Click the 'Lock & Edit' button.
3. In the left panel, click on the EBS Domain link.
4. Select the 'Security' tab.
5. Click on the 'Advanced' link.
6. Edit the 'Node Manager password' field and set it to the new WebLogic Server
password. The password should be same as set in Step 3.
7. Edit the 'Confirm Node Manager Password' field and set it to the new
WebLogic Server password. The password should be same as set in Step 3.
8. Save and activate the changes.

--6. The first time, AdminServer has to be stopped from the Admin console. Follow
these steps:
1. Log in to the WebLogic Administration console.
2. Shut down AdminServer.

--7. Set up your environment to start AdminServer again. AdminServer should now be
started using the normal AD script, which will also start Node Manager using the
new password.
1. Launch a new session and connect to the Oracle E-Business Suite instance.
2. Source the application tier environment file.
3. Start AdminServer with the following command:
$ $ADMIN_SCRIPTS_HOME/adadminsrvctl.sh

--8. Start the Managed Servers. For the first time, all Managed Servers should be started
from the WebLogic Server Admin console. This step will create boot.properties
files for the respective Managed Servers. Follow these steps:
1. Log in to the WebLogic Server Administration Console
2. Start all Managed Servers, one at a time

--9. Shut down all the Managed Servers. This is so the new credentials will be picked up
at the next startup. Follow these steps:
1. Log in to the WebLogic AdminServer console.
2. Shut down all Managed Servers.
3. Shut down AdminServer.

--10. Shut down Node Manager using the normal AD script.
$ $ADMIN_SCRIPTS_HOME/adnodemgrctl.sh stop

--11. Copy the boot.properties file for each Managed Server.
WebLogic Server native scripts use the boot.properties file. The above steps
have created the boot.properties file under
<EBS_DOMAIN_HOME>/servers/<Managed Server
name>/data/nodemanager, which is used by Node Manager. For each Managed
Server, copy the newly-generated boot.properties file from
<EBS_DOMAIN_HOME>/servers/<Managed Servername>/data/nodemanager to <EBS_DOMAIN_HOME>/servers/<ManagedServer name>/security.
The EBS WebLogic Server domain password has now been changed, and all servers
can now be started using the normal AD scripts.
To start AdminServer:
$ADMIN_SCRIPTS_HOME/adadminsrvctl.sh start
To start the Managed Servers:
$ $ADMIN_SCRIPTS_HOME/admanagedsrvctl.sh start <managed_server_name>

--12. The above steps have changed the Oracle WebLogic AdminServer password on the
run file system. You now need to perform an fs_clone operation, to change the
WebLogic EBS Domain password on the patch file system:
1. Launch a new session and connect to the Oracle E-Business Suite instance.
2. Source the application tier environment file.
3. Run the command:
3-42 Oracle E-Business Suite Setup Guide
$ adop phase=fs_clone

12.2 apps tier cloning fails - EBSapps/comn/clone/FMW/pasteBinary.sh ERROR: Script failed, exit code 255

Adcfgclone is referring to production inventory in test server due to which cloning is returning into the error. I have provided the error and action plan in this post.

Error :

Started ApplyAppsTier at Thu Aug 17 08:20:01 AST 2017
Version:
        ApplyAppsTier.java      :       120.11.12020000.6
#############################################################

START: Creating the FMW Home from ApplyAppsTier
FMW JDK version in use: 1.7.0_25

START: Creating FMW Home.
Running /d02/DEV/appltest/fs2/EBSapps/comn/clone/FMW/pasteBinary.sh -javaHome /d02/DEV/appltest/fs2/EBSapps/comn/util/jdk -al /d02/DEV/appltest/fs2/EBSapps/comn/clone/FMW/FMW_Home.jar -tl /d02/DEV/appltest/fs2/FMW_Home -invPtrLoc /var/opt/oracle/oraInst.loc -ldl /d02/DEV/appltest/fs2/inst/apps/DEV_ahqtest/admin/log/clone/fmwT2PApply -silent true -debug true -executeSysPrereqs false
Script Executed in 655369 milliseconds, returning status 255
ERROR: Script failed, exit code 255


I found the error from the logs below. Error is while creating fmw and i was able to figure out the rror in the folder below.

/d02/DEV/appltest/fs2/inst/apps/DEV_ahqtest/admin/log/clone/fmwT2PApply

Please look out for *.error file where errors will be logged. 

bash-3.2$ ls -ltr

CLONE2017-08-20_03-47-24PM.error  --> Error will be logged here. 

CLONE2017-08-20_03-47-24PM.log

Error logged :

"CLONE2017-08-17_08-20-03AM.error" 19 lines, 1981 characters
SEVERE : Aug 17, 2017 8:30:57 AM - ERROR - CLONE-20218   Cloning is not successful.
SEVERE : Aug 17, 2017 8:30:57 AM - CAUSE - CLONE-20218   An internal operation failed.
SEVERE : Aug 17, 2017 8:30:57 AM - ACTION - CLONE-20218   Provide the clone log and error file for investigation.
java.lang.Exception: Invalid Oracle home as its inventory pointer file /d02/DEV/appltest/fs2/FMW_Home/oracle_common/oraInst.loc has invalid entry for Oracle inventory.
        at oracle.as.clone.util.OracleHomeDepRefUtil.setInvPtrLocInInventoryManager(OracleHomeDepRefUtil.java:327)
        at oracle.as.clone.util.OracleHomeDepRefUtil.backupAndSetInvPtrLocBeforeInstallEngineInvoke(OracleHomeDepRefUtil.java:269)
        at oracle.as.clone.util.OracleHomeDepRefUtil.preSetupBeforeInstallEngineInvoke(OracleHomeDepRefUtil.java:207)
        at oracle.as.clone.util.OracleHomeDepRefUtil.setOHProperties(OracleHomeDepRefUtil.java:465)
        at oracle.as.clone.util.OracleHomeDepRefUtil.setOracleHomeProperties(OracleHomeDepRefUtil.java:424)
        at oracle.as.clone.mwhome.cloner.OracleHomeApplyCloner.updateOracleHomeInfo(OracleHomeApplyCloner.java:1904)
        at oracle.as.clone.mwhome.cloner.NormalOracleHomeApplyCloner.doPostApplyClone(NormalOracleHomeApplyCloner.java:127)
        at oracle.as.clone.request.ApplyCloneRequest.performFirstPostoperation(ApplyCloneRequest.java:248)
        at oracle.as.clone.request.ApplyCloneRequest.applyArchive(ApplyCloneRequest.java:219)
        at oracle.as.clone.request.ApplyCloneRequest._clone(ApplyCloneRequest.java:69)
        at oracle.as.clone.process.CloningExecutionProcess.execute(CloningExecutionProcess.java:131)
        at oracle.as.clone.process.CloningExecutionProcess.execute(CloningExecutionProcess.java:114)
        at oracle.as.clone.client.CloningClient.executeT2PCommand(CloningClient.java:238)
        at oracle.as.clone.client.CloningClient.main(CloningClient.java:115)
SEVERE : Invalid Oracle home as its inventory pointer file /d02/DEV/appltest/fs2/FMW_Home/oracle_common/oraInst.loc has invalid entry for Oracle inventory.

I found the production inventory location in the file above. Corrected the inventory location in the test server in the file below.

bash-3.2$ find ./ -name '*ora*.loc'
$RUN_FS/EBSapps/10.1.2/oraInst.loc

You can also do the below,

Change All orinst.loc
mv /var/opt/oracle/oraInventory  /var/opt/oracle/oraInventory.org
Remove all files under /tmp
Re-run adcfclone

  

Oracle EBS login page is not working

Error :

The system has encountered an error when servicing the request, Please try again. If the error persists, please contact the system administrator.

Just i wanted to share the issue i encountered today. We are unable to login to application due to the error above. We got the error instead of login page.

What i did?

- As soon as i get the error in login page, i checked the apache logs. I can see some chain failed error but it was tripping all the time (Not only today), So i decided it should nt be cause of the today error. 
- I logged into admin console and checked all managed servers since we are in 12.2 (Everything was runnig fine). 
-I bounced apache once but no use
- I went to db tier and connected to sqlplus where i found the actual issue. When i connected to sqlplus i got the error below :

ERROR:

ORA-00020: maximum number of processes (200) exceeded

I found the cause of the issue. Application was unable to create db connection since limit is overloaded. 

How i narrowed down the issue?

I did how many processed connected to db.

ps -ef|grep oratest (Make sure you have only one instance running under OS user, I had only one instance running under oratest)

I checked how many connections coming from outside,

ps -ef|grep -i local=no|grep oratest

I found out totally 170 sessions were created from outside. 

How i fixed?

I decided to kill all local=no processes to reduce number of processes.  My environment was test and i knew the impact. 

****Please double check and check with your manager or your peers before you kill any such process in your environment.****

ps -ef|grep -i local=no|grep -i oratest|grep -v grep|awk '{print $2}'|xargs kill -9

Above command will get all the process id's of local=no processes and kill them. 

Within a minute, limit is freed up and i was able to connect to sqlplus. As soon as application were able to connect to db, we got the login page.

Thanks!

Sikky