R12.2 Apps DBA. Powered by Blogger.

Starting EBS Fails with Exception while starting server oacore_server1 Unable to obtain lock on oacore_server1.lok

No comments :
startup weblogic fails with:
Starting server oacore_server1 ...
Error Starting server oacore_server1: weblogic.nodemanager.NMException: Exception while starting server 'oacore_server1'
ERROR: Unable to start up the managed server oacore_server1
07/06/14-13:37:05 :: admanagedsrvctl.sh: exiting with status 1
The oacore_server1.out log file under $EBS_DOMAIN_DOMAIN\servers\oacore_server1\logs\ shows:
<Jul 6, 2014 1:33:46 PM EDT> <Info> <WebLogicServer> <BEA-000377> <Starting WebLogic Server with Java HotSpot(TM) 64-Bit Server VM Version 23.25-b01 from Oracle Corporation>
<Jul 6, 2014 1:37:04 PM EDT> <Critical> <WebLogicServer> <BEA-000362> <Server failed. Reason:
There are 1 nested errors:
weblogic.management.ManagementException: Unable to obtain lock on ..EBS_domain/servers/oacore_server1/tmp/oacore_server1.lok. Server may already be running
    at weblogic.management.internal.ServerLocks.getServerLock(ServerLocks.java:206)
    at weblogic.management.internal.ServerLocks.getServerLock(ServerLocks.java:67)
    at weblogic.management.internal.DomainDirectoryService.start(DomainDirectoryService.java:74)
    at weblogic.t3.srvr.ServerServicesManager.startService(ServerServicesManager.java:461)
    at weblogic.t3.srvr.ServerServicesManager.startInStandbyState(ServerServicesManager.java:166)
    at weblogic.t3.srvr.T3Srvr.initializeStandby(T3Srvr.java:881)
    at weblogic.t3.srvr.T3Srvr.startup(T3Srvr.java:568)
    at weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:469)
    at weblogic.Server.main(Server.java:71)
The previous shutdown was not clean and the .lok file from the previous startup was still there.
SOLUTION
1. Remove or rename the file: $EBS_DOMAIN_HOME/servers/oacore_server1/tmp/oacore_server1.lok.
2. Re-teststarting EBS again.

Error Number of servers are not in sync between Run and Patch Context While fs_clone of patching

No comments :
LogFileName = adop_20140311_152649.log
ERROR
[UNEXPECTED]Error occurred while CLONE Patch File System from Run File System using command: <perl /u01/EBS/VIS/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl -contextfile=/u01/EBS/VIS/fs2/inst/apps/UPG_apcappsx21/appl/admin/UPG_apcappsx21.xml -patchcontextfile=/u01/EBS/VIS/fs1/inst/apps/UPG_apcappsx21/appl/admin/UPG_apcappsx21.xml -promptmsg=hide -console=off -mode=create -sessionid=5 -timestamp=20140311_152649 -outdir=/u01/EBS/VIS/fs_ne/EBSapps/log/adop/5/fs_clone_20140311_152649/UPG_apcappsx21>.
LogFileName = fsclone_stage/FSCloneStageAppsTier_03111527.log
ERROR
AdminServer is in RUNNING mode.
# Set WLS password...
# Passing the apps password to apps Tier...
runMSList size6
patchMSList size5
ERROR: Number of servers are not in sync between Run and Patch Context.
ERROR: FSClone would exit...
CAUSE
runMSList size6 means there are 6 managed servers in Run environment, patchMSList size5 means there are 5 managed servers in Patch environment, which caused error :
ERROR: Number of servers are not in sync between Run and Patch Context.
Use following sql to confirm :
SELECT
extractValue(XMLType(TEXT),'//oa_service_name[@oa_var="s_adminservername"]'),
extractValue(XMLType(TEXT),'//oacore_server_ports'),
extractValue(XMLType(TEXT),'//forms_server_ports'),
extractValue(XMLType(TEXT),'//oafm_server_ports'),
extractValue(XMLType(TEXT),'//forms-c4ws_server_ports'),
extractValue(XMLType(TEXT),'//oaea_server_ports')
from fnd_oam_context_files
where name not in ('TEMPLATE','METADATA')
and (status is null or status !='H')
and EXTRACTVALUE(XMLType(TEXT),'//file_edition_type')='patch'
and CTX_TYPE = 'A';
-------------------------
AdminServer
oacore_server1:7201
forms_server1:7401
oafm_server1:7601
forms-c4ws_server1:7801

SELECT
extractValue(XMLType(TEXT),'//oa_service_name[@oa_var="s_adminservername"]'),
extractValue(XMLType(TEXT),'//oacore_server_ports'),
extractValue(XMLType(TEXT),'//forms_server_ports'),
extractValue(XMLType(TEXT),'//oafm_server_ports'),
extractValue(XMLType(TEXT),'//forms-c4ws_server_ports'),
extractValue(XMLType(TEXT),'//oaea_server_ports')
from fnd_oam_context_files
where name not in ('TEMPLATE','METADATA')
and (status is null or status !='H')
and EXTRACTVALUE(XMLType(TEXT),'//file_edition_type')='run'
and CTX_TYPE = 'A';
-------------------------
AdminServer
oacore_server1:7202,oacore_server2:7204
forms_server1:7402
oafm_server1:7602
forms-c4ws_server1:7802

Run environment has one more managed server "oacore_server2" than Patch environment.
SOLUTION
Solution A: Remove/delete the managed server "oacore_server2" from the RUN environment filesystem
1.  Managed server oacore_server2 should be shutdown before it can be deleted(in this case, i shut down apache service with adapcctl.sh)
2.  Execute the below command on the Application tier node where the managed server resides:
# perl $AD_TOP/patch/115/bin/adProvisionEBS.pl ebs-delete-managedserver -contextfile=$RUN_BASE/inst/apps/UPG_apcappsx21/appl/admin/UPG_apcappsx21.xml -managedsrvname=oacore_server2 -servicetype=oacore -logfile=/tmp/remove_manage_server.log
NOTE: check the run filesystem context file for confirmation of no "oacore_server2" information
3. On the node containing the Oracle HTTP Server, run autoconfig:
# sh $INST_TOP/admin/scripts/adautocfg.sh
NOTE: The database server and database listener must remain available during the AutoConfig run, but all other database tier services should be shut down
4.  Execute the following script to remove details of the deleted managed server from the Oracle HTTP Server configuration files:
# perl $FND_TOP/patch/115/bin/txkSetAppsConf.pl -ctxfile=$RUN_BASE/inst/apps/UPG_apcappsx21/appl/admin/UPG_apcappsx21.xml -outfile=/tmp/delete_http_config.log
5.  On the node that contains the Oracle HTTP Server, bounce the Oracle HTTP Server:
$ sh $ADMIN_SCRIPTS_HOME/adapcctl.sh stop
$ sh $ADMIN_SCRIPTS_HOME/adapcctl.sh start
6.  Re-issue fs_clone phase

adop fs_clone phase Fails In txkADOPPreparePhaseSynchronize.pl Script When Creating FMW Archive

No comments :
12.2.4 version, When attempting to run adop phase=fs_clone the following error occurs:
In adop log file:
[UNEXPECTED]Error occurred while CLONE Patch File System from Run File System using command: "perl /DATA/app/appldev/TEST1/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl -contextfile=/DATA/app/appldev/TEST1/fs1/inst/apps/TEST1_abc/appl/admin/TEST1_abc.xml -patchcontextfile=/DATA/app/appldev/TEST1/fs2/inst/apps/TEST1_abc/appl/admin/TEST1_abc.xml -promptmsg=hide -console=off -mode=create -sessionid=18 -timestamp=20150325_010624 -outdir=/DATA/app/appldev/TEST1/fs_ne/EBSapps/log/adop/18/fs_clone_20150325_010624/TEST1_abc".
In txkADOPPreparePhaseSynchronize log file
===============================
Inside runADOPCustomSyncUp()...
===============================
Executing SYSTEM command: perl /DATA/app/appldev/TEST1/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPCustomSyncUp.pl -contextfile=/DATA/app/appldev/TEST1/fs1/inst/apps/TEST1_abc/appl/admin/TEST1_abc.xml -logfile=/DATA/app/appldev/TEST1/fs_ne/EBSapps/log/adop/18/fs_clone_20150402_232202/TEST1_abc/TXK_SYNC_create_Thu_Apr_2_23_27_26_2015/txkADOPCustomSyncUp.log -promptmsg=hide
EXIT STATUS: 1
/DATA/app/appldev/TEST1/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPCustomSyncUp.pl did not go through successfully.
LOG FILE: /DATA/app/appldev/TEST1/fs_ne/EBSapps/log/adop/18/fs_clone_20150402_232202/TEST1_abc/TXK_SYNC_create_Thu_Apr_2_23_27_26_2015/txkADOPCustomSyncUp.log.
*******FATAL ERROR*******
PROGRAM : (/DATA/app/appldev/TEST1/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl)
TIME : Fri Apr 3 00:22:46 2015
FUNCTION: main::runADOPCustomSyncUp [ Level 1 ]
ERRORMSG: /DATA/app/appldev/TEST1/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPCustomSyncUp.pl did not go through successfully.
In FSCloneStageAppsTier log file:
Attempting to delete: /DATA/app/appldev/TEST1/fs1/EBSapps/comn/adopclone_abc/FMW/t2pjdk
Creating prereq/webtier directory
Attempting to delete: /DATA/app/appldev/TEST1/fs1/EBSapps/comn/adopclone_abc//prereq
Copying /DATA/app/appldev/TEST1/fs1/FMW_Home/webtier/oui to /DATA/app/appldev/TEST1/fs1/EBSapps/comn/adopclone_abc//prereq/webtier
Copying /DATA/app/appldev/TEST1/fs1/FMW_Home/webtier/inventory/Scripts to /DATA/app/appldev/TEST1/fs1/EBSapps/comn/adopclone_abc//prereq/webtier
START: Creating FMW archive.
Running /DATA/app/appldev/TEST1/fs1/FMW_Home/oracle_common/bin/copyBinary.sh -javaHome /DATA/app/appldev/TEST1/fs1/EBSapps/comn/adopclone_abc/FMW/t2pjdk -al /DATA/app/appldev/TEST1/fs1/EBSapps/comn/adopclone_abc/FMW/FMW_Home.jar -smw /DATA/app/appldev/TEST1/fs1/FMW_Home -ldl /DATA/app/appldev/TEST1/fs1/inst/apps/TEST1_abc/admin/log/clone/fmwT2PStage -invPtrLoc /etc/oraInst.loc -silent true -debug true
Script Executed in 200 milliseconds, returning status 1
ERROR: Script failed, exit code 1
CAUSE
The issue is caused during clone staging process t2pjdk directly couldn't deleted because the customer placed the SSL certificate file under /DATA/app/appldev/INNOV/fs2/EBSapps/comn/adopclone_abc/FMW/t2pjdk/jre/lib/security/ and the file couldn't deleted.
During WLS component staging copyConfig utility is failing. The reason is /FMW/t2pjdk directory didn't stage correctly. During t2pjdk directory staging ,if t2pjdk directory already exist the system are trying to delete it. But if for some reason some files within t2pjdk is in already used state then code couldn't able to delete the t2pjdk. The customer enabled SSL on EBS and they copied the custom certificate file to the directory /DATA/app/appldev/INNOV/fs2/EBSapps/comn/adopclone_abc/FMW/t2pjdk/jre/lib/security/ to import it to the cacerts, The file is not seeded file so the t2pjdk directly couldn't deleted as normal.
The workaround is to remove the t2pjdk directory manually. After that, fs_clone complete normally. But after the customer complete a full adop cycle and run fs_clone again, the issue is reproduced.
SOLUTION
To implement the solution, please execute the following steps:
1. Move the custom certificate file from $COMMON_TOP/util/jdk64/jre/lib/security directory and place it outside of $COMMON_TOP/util/jdk64/ directory. (for both RUN and PATCH file system)
2.. Remove /DATA/app/appldev/INNOV/fs1/EBSapps/comn/adopclone_abd/FMW/t2pjdk/directory. (as fs1 is RUN File system)
3. Run fs_clone again.

adop phase=prepare Failed on txkADOPPreparePhaseSynchronize.pl

No comments :
The actual error is from FSCloneStageAppsTier_<time stamp>.log
START: Creating WLS config archive.
Script Executed in 200 milliseconds, returning status 1
ERROR: Script failed, exit code 1
CAUSE
The t2pjdk directory is not created completely.
SOLUTION
1. First refer to FSCloneStageAppsTier_<time stamp>.log file to get the t2pjdk directory path.
Following content might be found:
Creating t2pjdk directory
Directory /DATA/app/appldev/ORACLE/fs1/EBSapps/comn/adopclone_test/FMW/t2pjdk already exists.
So the t2pjdk directory path is /DATA/app/appldev/ORACLE/fs1/EBSapps/comn/adopclone_test/FMW/t2pjdk
2. Move the t2pjdk directory to a temp location
For example:
mkdir /tmp/t2pjdk_backup
mv /DATA/app/appldev/ORACLE/fs1/EBSapps/comn/adopclone_test/FMW/t2pjdk /tmp/t2pjdk_backup/FMW/t2pjdk
3. Rerun "adop phase=prepare".
Please be aware :
The same solution can be used for the adpreclone steps.

fsclone fails while running txkADOPPreparePhaseSynchronize.pl

No comments :
Error can be seen in adop_<TIME_STAMP>.log
[UNEXPECTED]Error occurred running "perl /u01/oracle/OCBPRD/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl -contextfile=/u01/oracle/OCBPRD/fs1/inst/apps/OCBPRD_ebsapp01/appl/admin/OCBPRD_ebsapp01.xml -patchcontextfile=/u01/oracle/OCBPRD/fs2/inst/apps/OCBPRD_ebsapp01/appl/admin/OCBPRD_ebsapp01.xml -promptmsg=hide -console=off -mode=create -sessionid=3 -timestamp=20151223_192751 -outdir=/u01/oracle/OCBPRD/fs_ne/EBSapps/log/adop/3/fs_clone_20151223_192751/OCBPRD_ebsapp01"
[UNEXPECTED]occurred during CLONE Patch File System from Run File System, running command: "perl /u01/oracle/OCBPRD/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl -contextfile=/u01/oracle/OCBPRD/fs1/inst/apps/OCBPRD_ebsapp01/appl/admin/OCBPRD_ebsapp01.xml -patchcontextfile=/u01/oracle/OCBPRD/fs2/inst/apps/OCBPRD_ebsapp01/appl/admin/OCBPRD_ebsapp01.xml -promptmsg=hide -console=off -mode=create -sessionid=3 -timestamp=20151223_192751 -outdir=/u01/oracle/OCBPRD/fs_ne/EBSapps/log/adop/3/fs_clone_20151223_192751/OCBPRD_ebsapp01"
CAUSE
The context variable s_invPtrLoc is not getting populated in patch context file. FSCloneApplyAppsTier log prints correct value for s_invPtrLoc but the value is missing in patch context file.
Context file customization is present.
From stage logs(FSCloneStage.log) below can be seen:
Copying /u01/oracle/OCBPRD/fs1/EBSapps/appl/ad/12.0.0/admin/template/custom/adxmlctx.tmp to /u01/oracle/OCBPRD/fs1/EBSapps/comn/adopclone_ebsapp01//context/apps/adxmlctx.tmp
The adxmlctx.tmp is being copied from custom folder.
Custom folder has old version of adxmlct.tmp (version 120.401.12020000.9.1202010.10)
While the actual version is 120.401.12020000.54
During clonecontext the template being used is from the stage which have the older version. Due that reason the patch context file is not getting created properly.
That's the reason the version of context files which can be seen in FND_OAM_CONTEXT_FILES are different between RUN and PATCH filesystems.
RUN fs version : 120.401.12020000.54
PATCH fs version : 120.401.12020000.9.1202010.10
SOLUTION
To solve the issue please do the below:
1.Remove the file $AD_TOP/admin/template/custom/adxmlctx.tmp
2.Run fs_clone with option force=yes

Error java.lang.IllegalStateException: Cipher not initialized R12.2

No comments :
On Oracle Applications 12.2.4,
when attempting to open the status diagram,
the following error occurs:
[1953]:STATEMENT:[fnd.wf.monitor.webui.GraphMonitorCO]:Exception in getHtml: java.lang.IllegalStateException: Cipher not initialized
[1954]:STATEMENT:[fnd.wf.monitor.webui.GraphMonitorCO]:Complete Exception stack is:
[1957]:STATEMENT:[fnd.wf.monitor.webui.GraphMonitorCO]:java.lang.IllegalStateException: Cipher not initialized at javax.crypto.Cipher.checkCipherState(Cipher.java:1672)
The issue can be reproduced at will with the following steps:
1. Workflow Administrator Web Applications
2. Administrator Workflow
3. Status Monitor
CAUSE
The Workflow code wasn't able to open the URL for the workflow diagram due to SSL setting was missing.
The debug information in NewWorkflowError.docx shows the following error when trying to connect to the url:
[1505]:STATEMENT:[fnd.wf.WFBaseUtil]:Done openConnection [1505]:STATEMENT:[fnd.wf.WFBaseUtil]:String representation of urlConn: weblogic.net.http.SOAPHttpsURLConnection:https://<serverName>:<portNumber>/forms/frmservlet?appletmode=nonforms&HTMLpageTitle=&HTMLpreApplet=&code=oracle.apps.fnd.wf.Monitor&width=800&height=400&archive=/OA_JAVA/oracle/apps/fnd/jar/wfmon.jar,/OA_JAVA/oracle/apps/fnd/jar/fndewt.jar,/OA_JAVA/oracle/apps/fnd/jar/fndswing.jar,/OA_JAVA/oracle/apps/fnd/jar/fndbalishare.jar,/OA_JAVA/oracle/apps/fnd/jar/fndctx.jar...
1505]:STATEMENT:[fnd.wf.WFBaseUtil]:Done setUseCaches
[1953]:STATEMENT:[fnd.wf.monitor.webui.GraphMonitorCO]:Exception in getHtml: java.lang.IllegalStateException: Cipher not initialized
[1954]:STATEMENT:[fnd.wf.monitor.webui.GraphMonitorCO]:Complete Exception stack is:
[1957]:STATEMENT:[fnd.wf.monitor.webui.GraphMonitorCO]:java.lang.IllegalStateException: Cipher not initialized at javax.crypto.Cipher.checkCipherState(Cipher.java:1672)
SOLUTION
To implement the solution, please execute the following steps:
1. Bring up the WLS Admin console for the EBS instance ->
http://host:port/console and login as the weblogic user.
2. Click on Lock & Edit.
3. Under the Domain Structure, navigate to EBS Domain -> Environment -> Servers.
4. Click on oacore_server1, and click on the SSL tab.
5. Click on Advanced to expand that section.
6. Look for the 'Use JSSE SSL' parameter and enable this by placing a check in the box next to it.
7. Click on Activate Changes.
8. Restart the managed server.
$ADMIN_SCRIPTS_HOME/admanagedsrvctl.sh stop oacore_server1
$ADMIN_SCRIPTS_HOME/admanagedsrvctl.sh start oacore_server1
9. Retest the issue.
10. Migrate the solution as appropriate to other environments.

Invalid Objects and Forms Generation Errors During R12.2

No comments :
When attempting to apply 12.2.4 upgrade patch(es) (adop phase=apply apply_mode=downtime patches=17919161,21900859 merge=yes)
the following error occurs.
The following Oracle Forms objects did not generate successfully:
pa forms/US PAXURVPS.fmx
The patch has FAILED,
Please check the adpatch logs for more details.
Object APPS.PA_STATUS_COMMITMENTS_V is invalid
SOLUTION
Please recompile APPS.PA_STATUS_COMMITMENTS_V and advise if there are errors, if not then use adadmin to compile form PAXURVPS.fmx
One can use the following to display errors if there are any errors when compiling the view
SQL>alter view APPS.PA_STATUS_COMMITMENTS_V compile;
show error;
OR
SQL> select owner, name, type, text from dba_errors where name = 'PA_STATUS_COMMITMENTS_V' order by 1,2,3,line;

Query to count the open cursors for the sessions.

No comments :
Customer faces 5000~8000 open cursor every day and due to this Java Listeners goes down.

SELECT 
  OC.SID, 
  OC.USER_NAME, 
  S.PROGRAM, 
  COUNT(*) COUNTER, 
  OC.SQL_ID, 
  OC.SQL_TEXT 
  FROM 
  V$OPEN_CURSOR OC, 
  V$SESSION S 
  WHERE 
  OC.SQL_TEXT NOT LIKE ''%obj#,%'' 
  AND OC.SQL_TEXT NOT LIKE ''%grantee#,%'' 
  AND OC.SQL_TEXT NOT LIKE ''%privilege#%'' 
  AND OC.SQL_TEXT NOT LIKE ''%/*+ rule */%'' 
  AND OC.SQL_TEXT NOT LIKE ''%col#%'' 
  AND OC.SQL_TEXT NOT LIKE ''%sys.mon_mods$%'' 
  AND OC.SQL_TEXT NOT LIKE ''%obj#=%'' 
  AND OC.SQL_TEXT NOT LIKE ''%update$,%'' 
  AND OC.SID=S.SID 
  AND OC.USER_NAME NOT IN 
 (''SYS'',''DBSNMP'',''SYSTEM'',''SYSMAN'',''RMAN'') 
  AND lower(SQL_TEXT) like ''%enqueue%'' 
  GROUP BY 
  OC.SID, 
  OC.USER_NAME, 
  S.PROGRAM, 
  OC.SQL_ID, 
  OC.SQL_TEXT 
  HAVING COUNT(*)>1 
  ORDER BY 
  COUNT(*) desc, 
  OC.USER_NAME, 
  OC.SID, 
  OC.SQL_TEXT;

How To Access Forms Directly In Oracle Applications R12

No comments :
Oracle Applications R12, users are not able to access Forms directly via the following url:
http://hostname.domain:port/forms/frmservlet
The following error would pop out:
APP-FND-01542: This Applications Server is not authorized to access this database.
The goal of this document is to allow users access to the Oracle Applications R12 directly via Forms especially in cases where access to the following url is not available:
http://hostname.domain:port/OA_HTML/AppsLogin
NOTE: Accessing Forms directly is not supported and should only be used for diagnostic and troubleshooting purposes.
SOLUTION
1. Edit $CONTEXT_FILE which is located in $INST_TOP/appl/admin
2. Change
<appserverid_authentication oa_var="s_appserverid_authentication">SECURE</appserverid_authentication>
to
<appserverid_authentication oa_var="s_appserverid_authentication">OFF</appserverid_authentication>
Application Server Security Authentication can take one of the following values {ON, OFF, SECURE}.
OFF - Server security is not checked. Any application server machine can access the database.
ON - Some level of trust is required to access the database. Either the Application Server is registered with the database or the module and version ID are known to be trusted.
SECURE - Full trust is required for access to the database. Only registered Application Server machines and trusted code modules may connect.
3. Run $INST_TOP/admin/scripts/adautocfg.sh to instantiate the changes
4. Users are now able to access Forms directly.
NOTE:
Remember to set the Application Server Security Authentication back to SECURE by settting the value of s_appserverid_authentication to SECURE and running autoconfig.

ORA-04045 errors during recompilation/revalidation of APPS.HXT_HOUR_DEDUCTION_RULES_WHO

No comments :
ADOP cleanup reports following error at ad.plsql.ad_zd_sys.drop_covered_object.
[EVENT]     Calling cleanup in STANDARD mode...
   [EVENT]     Log: @ADZDSHOWLOG.sql "2016/01/01 01:01:01"
   [ERROR]     ORA-04045: errors during recompilation/revalidation of APPS.HXT_HOUR_DEDUCTION_RULES_WHO
ORA-04023: Object HXT.HXT_HOUR_DEDUCTION_RULES
   [ERROR]     Failed to execute SQL statement :
                               declare
                              begin
                                 ad_zd_log.message('ad.bin.adop','WARNING','');
                              exception
                                 when others then
                                    raise_application_error(-20001,'Error while calling ad_zd_log.message(ad.bin.adop,WARNING,).' || sqlerrm);
                              end;
   [ERROR]     Error Message :
   [ERROR]     *** Error occurred while calling ad_zd_log. Please check log file for the details. ***
Steps to Reproduce:
The issue can be reproduced at will with the following steps:
 Run ADOP Cleanup phase.
 Error is printed in the log.
SOLUTION
1. Run the command below to compile the trigger.
alter trigger APPS.HXT_HOUR_DEDUCTION_RULES_WHO compile;
2. Run the query below and make sure the status is Enabled.
select owner, trigger_name, status
from dba_triggers
where trigger_name = 'HXT_HOUR_DEDUCTION_RULES_WHO';
  Expected result:
   OWNER      TRIGGER_NAME                   STATUS
   ---------- ------------------------------ ---------
   APPS       HXT_HOUR_DEDUCTION_RULES_WHO   ENABLED
   APPS       HXT_HOUR_DEDUCTION_RULES_WHO   ENABLED
3. Run the query below and make sure the status is valid.
select owner,object_name, object_type,status
from dba_objects
where object_name = 'HXT_HOUR_DEDUCTION_RULES_WHO';
   Expected result:
   OWNER OBJECT_NAME                    OBJECT_TYPE STATUS
   ----- ------------------------------ ----------- -------
   APPS  HXT_HOUR_DEDUCTION_RULES_WHO   TRIGGER     VALID