R12.2 ADOP ONLINE PATCHING.

The EBS Technology Codelevel Checker (ETCC) is downloaded via Patch 17537119 and instructions to run the EBS Technology Codelevel Checker (ETCC) (checkDBpatch.sh and checkMTpatch.sh) are available in th patch's readme. 
ADZDDBCC - database compliance checker, shows violations of the database object development standards documented in the Oracle E-Business Suite Developer's Guide, Part No. E22961. Warning: this script takes a long time to run.
ADZDSHOWED - Show database editions and current edition.
ADZDSHOWLOG - Show full diagnostic log for online patching infrastructure
ADZDSHOWLOGEVT - Show only event and error messages from online patching diagnostic log (a useful summary, without the detailed statement text).
ADZDSHOWLOGERR - Show only error messages from online patching diagnostic log.
ADZDSHOWEV TABLE_SYNONYM_NAME - Show editioning view column mapping for table.
ADZDSHOWTAB TABLE_SYNONYM_NAME - Show table information and related objects.
ADZDSHOWMV MVIEW_NAME - Show materialized view information and related objects.
ADZDSHOWTS - Show important tablespace status. Ensure that you have enough SYSTEM tablespace.
ADZDCMPED - Compare Patch Edition with Run Edition. Warning: this script may take a long time to run.
ADZDSHOWDDLS - Show stored DDL summary by phase.
ADZDALLDDLS - Show stored DDL statement text and status.
ADZDDDLERROR - Show stored DDL execution errors and messages.
adutlrcmp - Recompile all objects, with before/after status report. Warning: this script may take a long time to run.
The following scripts are for experts:
ADZDSHOWOBJS - Show Object Summary per edition. Counts of actual and stub (inherited) editioned object per edition.
ADZDSHOWAOBJS - Show Actual Objects in the current edition. These are the editioned objects that have been changed by the patch.
ADZDSHOWIOBJS - Show Inherited Objects in the current edition. These are the editioned objects that remain untouched in the Patch Edition.
ADZDSHOWCOBJS - Show Covered Object Summary per edition. Count of objects in old editions that have a replacement in the run edition.
ADZDSHOWCOBJX - Show Covered Object List. List of objects in old editions that have a replacement in the run edition.
ADZDSHOWSM - Show Seed Manager status.
ADZDSHOWTM - Show Table Manager status.
ADZDSHOWAD - AD (online patching) database object status
ADZDSHOWSES - Show sessions connected to the database (by edition).
ADZDSHOWDEP OBJECT_NAME - Show objects that OBJECT_NAME depends on.

ADZDSHOWDEPTREE OBJECT_NAME - Show full dependency tree of objects that OBJECT_NAME depends on.
==========================
How ADOP works ?
The online patching cycle consists of five phases which are executed in order. Example of a typical online patching cycle:
source <ebs_root>/EBSapps.env run
adop phase=prepare
adop phase=apply patches=123456
adop phase=finalize
adop phase=cutover
source <ebs_root>/EBSapps.env run
adop phase=cleanup
Note that after cutover the command line environment should be re-loaded as the run edition file system has changed.
In a multi-node deployment, adop commands are only executed from the primary node. The primary adop session uses remote execution to automatically perform required actions on any secondary node.
Multiple phases of adop can be executed in a single line command. Example of combined finalize/cutover/cleanup:
adop phase=finalize,cutover,cleanup
Prior to cutover, it is possible to execute additional “apply” and “finalize” phases as needed. Example of applying multiple patches using separate apply commands:
source <ebs_root>/EBSapps.env run
adop phase=prepare
adop phase=apply patches=123456
adop phase=apply patches=223456
adop phase=finalize
adop phase=apply patches=323456
adop phase=finalize
adop phase=cutover
source <ebs_root>/EBSapps.env run
adop phase=cleanup
Note that it is possible to apply additional patches after running the finalize phase, but if you do so then you will need to run the finalize phase again. Finalize must always be run immediately prior to cutover.
ADOP Common Parameters
workers=<number> [default: computed]
Number of parallel workers used to execute tasks. Default value is computed principally according to number of available CPU cores.
input_file=<file_name>
adop parameters can be specified in a text file, with one
<parameter>=<value>
on each line of the file. Command line parameters override input file parameters.
loglevel=(statement|procedure|event|warning|error|unexpected) [default: event]
Controls the level of diagnostic log detail displayed on the console output. Each log message is tagged with a level:
1) statement – is only used for debugging.
2) procedure – is only used for debugging high level procedures.
3) event – is used to display informational messages in normal processing. This is the default value.
4) warning – is used to indicate an internal error that is handled by the system and does not affect processing.
5) error – indicates an action failed and will need to be reviewed by the user, but the system was able to continue processing.
6) unexpected – indicates an unrecoverable error that halts processing and requires user intervention before processing can continue.
Setting loglevel will display messages at that level and higher.
prompt=(yes|no) [default: yes]
Specifies whether adop should prompt for user input on warnings. By default adop will ask user whether to continue or exit on some warning messages. If this parameter is set to “no” adop will remain fully non-interactive, and will continue past any warning messages without user confirmation.
Below is the list of Diagnostic Parameters. Normally these parameters are not used, until unless directed by Oracle Support:
allowcoredump=(yes|no) [default: no]
Specifies whether adop should create a core dump if it crashes. This option should only be used if directed by support.
analytics=(yes|no) [default: no]
Controls whether adop writes additional reports with information that might be helpful in some diagnostic situations. This option should not be used unless directed by Support.
defaultsfile=<file_name> [default: adalldefaults.txt]
Name of the response file providing default parameter values for non-interactive execution of adadmin and adop. The file must be in the $APPL_TOP/admin/$TWO_TASK directory in both run edition and patch edition file systems. The default file “adalldefaults.txt” is maintained by AutoConfig and normally you should not need to change any values.
ADOP Prepare Phase
Prepare phase will be create a new Online Patching Cycle ID and start with Syncronizing the File System of Run into Patch. This will be followed by creation of Patch Edition in database.
The phase has below specific parameter:
skipsyncerror=(yes|no) [default: no]
It specifies whether to ignore errors that may occur during incremental file system synchronization. This might happen if you applied a patch in the previous patching cycle that had errors but decided to continue with the cutover. When the patch is synchronized on the next patching cycle, the apply errors may occur again, but can be ignored.
After complition of Prepare Phase you can start with migration of customization to the Patch Edition File System and you can apply Application Technology Stack Patches i.e. Oracle Home (10.1.2) Patches and Weblogic Patches. This can be done until you are completed with Cutover Phase.
ADOP Apply Phase
This is the phase where in patches are actully applied.
The phase has below specific parameters:
apply=(yes|no) [default: yes]
Controls whether adop actually applies the patch. You can specify “apply=no” to run adop in test mode, where the patch will not actually be applied, and adop will record what it would have done in the log.
patches=<patch#>[,<patch#>…]
patches=<patch_directory>:<driver>[,<patch_directory>:<driver>…]
This parameter specifies a comma-separated list of patches to be applied. Patches can be specified either as the patch number or by the patch directory and driver file. All patches are expected to be in the $PATCH_TOP directory on all tiers. Patches are applied serially unless the merge=yes parameter is specified.
patchtop=<directory_name> [default: $PATCH_TOP]
Path to a user-specified directory where patches are unzipped. The default and recommend location is the $PATCH_TOP directory automatically created by the install. When using an alternate patchtop you must ensure that the location is not within the editioned file systems (fs1, fs2) and is accessible by the same path for all nodes of a multi-node deployment.
apply_mode=(online|downtime|hotpatch) [default: online]
It is used to specify how the patch will be applied. There 3 option can be explained as below:
online – It will apply a patch to the patch edition during an online patching cycle.
downtime – It will apply a patch to the run edition when application services are down. When using this mode, you only run the apply phase.
hotpatch – apply a patch to the run edition when application services are up. When using this mode, you only run the apply phase
In downtime mode, adop will validate that application services are shutdown before apply the patch. The patch will be applied to the run edition of the system. Downtime mode patching does not use an online patching cycle and hence if there is an online patching cycle in progress. The process of applying a patch in downtime mode completes more quickly than in online mode, but at the cost of increased system downtime.
In hotpatch mode, adop will apply the patch to the run edition of the system while application services are still running. Patches that can be safely applied in hotpatch mode (such as NLS and Online Help patches) will document this in the patch readme. Hotpatch mode cannot be used if there is an online patching cycle in progress.
merge=(yes|no) [default: no]
Indicates whether adop should merge a list of patches before applying. By default, adop will apply a list of patches serially in the order specified. You can also use AD Merge Patch to merge multiple patches ahead of the apply command.
restart=(yes|no) [default: no]
Use restart=yes to resume the previous failed apply command from where processing terminated. If an apply command fails, check the log files for further information. If the problem can be corrected, you can then restart the apply command where it left off using the restart parameter.
When restarting a failed apply it is important to use the same parameters as the failed command, with only the addition of the restart=yes parameter.
abandon=(yes|no) [default: no]
Use abandon=yes to abandon the previous failed apply command and start a new apply command. Note that any changes made to the system by the failed command will remain in effect. The abandon flag is most useful when applying a replacement patch for the failing patch. If a patch fails to apply and there is no replacement patch, you may also abort the online patching cycle. See abort phase later in this blog.
options=<patch_option>[,<patch_option>…]
Options can be specified in a comma-separated list to control advanced features when a patch is applied. These options are normally not needed unless specified by documentation or support. Note that these options can be prefixed with “no”, e.g. “nocheckfile”, to disable the behavior, and for some options “no” is the default.
checkfile [default: checkfile] – Skip running exec, SQL, and exectier commands if they are recorded as already run.
compiledb [default: compiledb] – Compile invalid objects in the database after running actions in the database driver.
compilejsp [default: compilejsp] – Compile out-of-date JSP files, if the patch has copy actions for at least one JSP file.
copyportion [default: copyportion] – Run commands found in a copy driver.
databaseportion [default: databaseportion] – Run commands found in a database driver.
generateportion [default: generateportion] – Run commands found in a generate driver.
integrity [default: nointegrity] – Perform patch integrity checking
autoconfig [default: autoconfig] – Run AutoConfig.
actiondetails [default: actiondetails] – Turn off display of action details.
parallel [default: parallel] – Run actions that update the database or actions that generate files in parallel.
prereq [default: noprereq] – Perform prerequisite patch checking prior to running patch driver files.
validate [default: novalidate] – Connect to all registered Oracle E-Business Suite schemas at the start of patch application.
phtofile [default: nophtofile] – Save patch history to file
forceapply [default: noforceapply] – Reapply a patch that has already been applied. Useful in combination with “nocheckfile” option to rerun files that have already been executed.
flags=<patch_flag>[,<patch_flag>…]
Flags can be specified in a comma-separated list to control advanced features when applying a patch. Note that these flags can be prefixed with “no”, e.g. “nologging”, to disable the behavior and for some flags “no” is the default.
hidepw [default: hidepw] – Omit the “HIDEPW:” comments in the log file.
trace [default: notrace] – Log all database operations to a trace file.
logging [default: nologging] – Create indexes in LOGGING or NOLOGGING mode.
autoskip [default: noautoskip] – To proceed with adpatch execution even if some driver actions failed. Failed actions are recorded in a log file.
preinstall=(yes|no) [default: no]
Allows a patch to be applied to the file system without connecting to the database. Do not use this parameter unless directed by Oracle.
wait_on_failed_job=(yes|no) [default: no]
Controls whether adop apply command exits when all workers have failed. Instead of exiting, you can force adop to wait, and use the “adctrl” to retry failed jobs.
printdebug=(yes|no) [default: no]
Controls whether to display additional debugging information.
uploadph=(yes|no) [default: yes]
Controls whether to upload patch history information to database after applying the patch.
ADOP Finalize Phase
Finalize Phase is performed to keep the system ready for Cutover phase. This phase perform various activities like:
1. Compiling Invalid Objects
2. Generating driverd objects
3. Pre-compute DDL to be run during Cutover
Finalize Phase have below specific parameters:
finalize_mode=(full|quick) [default: quick]
Quick mode will provide the shortest execution time, by skipping non-essential actions. Full mode performs additional actions such as gathering statistics that may improve performance after cutover.
ADOP Cutover Phase
Cutover phase perform below activities:
1. Bring down Application services
2. Promote Patch File System to the Run File System.
3. Promote Patch Database Edition to the Run Database Edition.
4. Perform Maintenance task
5. Bring up application services
Cutover Phase have below specific parameters:
mtrestart=(yes|no) [default: yes]
Specifies whether to restart application tier servers after cutover. Leave at default unless you need to perform any manual steps during downtime.
cm_wait=<minutes> [default: forever]
Specifies the number of minutes to wait for Concurrent Manager shutdown. Adop cutover starts by requesting a concurrent manager shutdown and then waits for in-progress requests to complete. If Concurrent Manager does not shutdown within the specified time limit, remaining concurrent requests will be killed and cutover will proceed.
Note that any concurrent requests killed during forced shutdown may need to be manually re-submitted after cutover. To avoid killing concurrent requests, schedule cutover at a time of minimal user activity or manually shutdown Concurrent Manager in advance of cutover.
ADOP Cleanup Phase
This phase will cleanup the Application and Database for the next Patching Cycle.
Cleanup phase specific parameters are:
cleanup_mode=(full|standard|quick) [default: standard]
Quick mode provides the shortest execution time, by skipping non-essential actions. Standard mode performs additional processing to drop obsolete code objects from old editions. Full mode performs additional processing to drop empty database editions and unused table columns.
Cloning the Patch Edition File System
The patch edition file system is normally synchronized with the run edition file system during the prepare phase. There are some cases where it is helpful or required to manually re-clone the patch edition file system from the run edition.
1) After aborting an online patching cycle.
2) After manually changing the run edition file system.
3) After patching middle-tier technology components.
4) After applying an EBS RUP.
By re-cloning the patch edition file system, you can be certain that it is correctly synchronized, and also minimize any synchronization delay that would normally occur on the next prepare command. This can be down by below command:
adop phase=fs_clone
If there is any error you must examine log files and correct the problem, then restart the fs_clone by running the command again. User below command if fs_clone does not restart correctly and you want to force the process to restart from the beginning.
adop phase=fs_clone force=yes
Aborting an online patching cycle
If an online patching cycle encounters problems that cannot be fixed immediately you can abort the patching cycle and return to normal runtime operation. Aborting an online patching cycle can be issue as below:
adop phase=abort
Note that once you are done with Cutover phase, you can abort ADOP Cycle.
The abort command drops the database patch edition and returns the system to normal runtime state. Immediately following abort, you must also run a full cleanup and fs_clone operation to fully remove effects of the failed online patching cycle.
Dropping old database editions
As online patching cycles are completed, the system will build up a number of old database editions. When the number of old database editions reaches about 25, you should consider running a special maintenance operation to drop old database editions. This can be down as below:
adop phase=prepare
adop phase=actualize_all
adop phase=finalize
adop phase=cutover
adop phase=cleanup cleanup_mode=full
This maintenance operation will take much longer than a typical online patching cycle, and should only be performed when there is no immediate need to start a new online patching cycle. The actualize all and full cleanup can be done separately as shown above, or can be executed in conjunction with an online patching cycle.
Log File Location
The adop log files are located on the non-editioned file system (fs_ne), under:
$NE_BASE/EBSapps/log/adop/<adop_session_id>/<phase>_<date>_<time>/<context_name>/log
Session

The adop utility maintains a session for each online patching cycle. A new session is created when you run the prepare phase. Each session is given a numeric ID number. The session is used to maintain the state of the online patching cycle across the various adop phases and commands. You can only run one adop session at a time on a particular Oracle E-Business Suite system.
======================================