Sunday, September 27, 2026

Oracle E-Business Suite Load Balancer Setup in OCI

Executive Summary

This white paper provides an end-to-end procedure for configuring an OCI public load balancer to route client traffic to an Oracle E-Business Suite (EBS) compute instance in a private subnet.

The document emphasizes practical troubleshooting considerations so that the guidance can be applied in a production environment. OCI Load Balancer distributes client traffic across backend servers, providing a highly available and scalable application entry point. The following sections describe the prerequisites, configuration steps, validation checks, and troubleshooting actions.

 

Prerequisites

Before configuring the load balancer, confirm that the following requirements are met:

1.    An Oracle EBS instance is installed and running in a private subnet. This example uses the Oracle E-Business Suite 12.2.13 Demo Install Image from OCI Marketplace. For instructions on deploying the demo instance and performing basic administration tasks such as start, stop, and status checks, refer to the link below.

      https://exploreora.blogspot.com/2026/08/how-to-use-ebs-marketplace-image-for.html

2.    Because the EBS instance is in a private subnet, a jump server (bastion host) must be available in a public subnet to provide administrative SSH access to the EBS backend server.

3.    Configure a security rule that permits the required traffic between the jump server subnet and the private subnet containing the EBS compute instance. Attach the rule to the EBS private subnet, and use the jump server subnet CIDR as the source.



4.        Store the private key for the EBS compute instance securely on the jump server, such as under /root/.ssh, and verify SSH access with: ssh -i <path-to-private-key> <IP-or-hostname>.

Load Balancer Deployment

Step 1: Identify the EBS Ports

First, identify the ports on which the EBS application services are listening.

Connect to the EBS VM through the public jump server, and run the following commands to review the listening ports:

netstat -tulpn | grep LISTEN

netstat -tulpn | grep httpd

(Not all processes could be identified, non-owned process info

 will not be shown, you would have to be root to see it all.)

tcp6       0      0 :::9999                 :::*                    LISTEN      204489/httpd.worker

tcp6       0      0 :::8000                 :::*                    LISTEN      204489/httpd.worker

tcp6       0      0 :::4443                 :::*                    LISTEN      204489/httpd.worker

Validate the HTTP endpoint locally:

curl -I http://localhost:8000

HTTP/1.1 200 OK

Date: Thu, 24 Sep 2026 17:33:25 GMT

Server:

Last-Modified: Wed, 23 Sep 2026 13:35:49 GMT

ETag: "519-65c268fa91740"

Accept-Ranges: bytes

Content-Length: 1305

X-Frame-Options: SAMEORIGIN

X-Content-Type-Options: nosniff

Content-Type: text/html

Content-Language: en

Alternatively, validate the HTTPS endpoint:

curl -k https://localhost:4443

<!-- $Header: index.html 120.6 2011/04/15 09:27:28 sbandla ship $ -->

<!--

  ###############################################################

 

  Do not edit settings in this file manually. They are managed

  automatically and will be overwritten when AutoConfig runs.

  For more information about AutoConfig, refer to the Oracle

  E-Business Suite Setup Guide.

 

  ###############################################################

 

  Template <AD_TOP>/admin/template/index.html

  To customize this page, please refer to Oracle MetaLink Note 387859.1.

-->

<!-- dbdrv: none -->

 

<!-- appdet.html - this is needed to make the page pass

                   the Rapid Install post install check -->

<html>

<HEAD>

<TITLE>E-Business Suite Home Page Redirect</TITLE>

<META http-equiv=REFRESH content="1; URL=http://myebs.regionalsub.regionalsub.oraclevcn.com:8000/OA_HTML/AppsLogin">

</HEAD>

<body>

<DIV ID="content">

The E-Business Home Page is located at <a href="http://myebs.regionalsub.regionalsub.oraclevcn.com:8000/OA_HTML/AppsLogin">http://myebs.regionalsub.regionalsub.oraclevcn.com:8000/OA_HTML/AppsLogin</a><br>

If your browser doesn't automatically redirect to its new location, click

<A HREF="http://myebs.regionalsub.regionalsub.oraclevcn.com:8000/OA_HTML/AppsLogin">here</A>.

</DIV>

</body>

</html>

Step 2: Prepare the Load Balancer Prerequisites

Before creating the load balancer, ensure that the following certificate-management components are available:

Ø An OCI Vault in which certificate-related keys will be protected

Ø A root certificate authority (CA)

Ø A certificate issued by the root CA

Create the Vault: The load balancer listener uses an SSL/TLS certificate and its private key to terminate encrypted client connections. It can then establish the configured connection to the authorized backend server.

In the OCI Console, navigate to Identity & Security and then Vault. Create a Vault and a master encryption key to protect the certificate key material. In this example, the master key uses the RSA algorithm with a 2,048-bit key length, as shown below.

 

Create the Root Certificate Authority

Under Identity & Security, select Certificate Authorities, as shown below.



Enter a meaningful common name for the certificate authority.


If Not Valid Before is left blank, the CA becomes valid immediately after creation. Select the Vault created in the preceding step. This example uses the ECDSA algorithm for the CA key.


For this test configuration, issuance rules and revocation can remain disabled. Review the settings, and then create the certificate authority. For production use, configure issuance and revocation controls in accordance with your organization’s security policy.


Create the Server Certificate

Use the root certificate authority created above to issue the certificate that will be assigned to the load balancer listener. Complete the fields shown in the OCI workflow, review the configuration, and create the certificate.

Step 3: Create the Load Balancer

From the Networking menu in the OCI Console, create a load balancer by following the workflow shown below.

 

 

In this example, websubnet is the public subnet used by the load balancer. Create this subnet before creating the load balancer.

The default bandwidth of 100 Mbps is used for this configuration.

Select the backend server from the appropriate compartment, and set the backend port to 4443 for HTTPS traffic.

               

Configure the health check with the protocol and port that match the endpoint being tested. The health check confirms that the load balancer can reach the backend server and receive a valid response.

Select the backend set. Although this example contains only one backend server, the network security rules must still allow traffic from the public load balancer subnet to the private subnet that hosts the backend.

On the Configure Listener page, select a Certificate Service-managed SSL certificate so that the listener can use the certificate created in the preceding steps. If you do not use an OCI-managed certificate, you must provide and manage your own certificate and private key.

Under Advanced SSL, select the certificate authority created earlier.

In the Manage Logging section, create a new log group if one does not already exist, and then complete the load balancer creation workflow.

Step 4: Configure Security Rules

Configure security rules for the following traffic flows:

1.    Allow inbound client traffic from the internet to the public load balancer subnet on the listener port, and allow the required outbound traffic from the load balancer subnet to the private backend subnet.

2.    Allow inbound traffic to the EBS backend from the load balancer subnet on the configured backend and health-check ports. Ensure that the corresponding outbound rules permit response traffic back to the load balancer subnet.

Attach the public-facing security rules to the load balancer subnet. In this example, the subnet is websubnet.

 

Next, attach the backend-facing security rules to the private subnet in which the EBS backend runs. In this example, the subnet is dbsubnet.

 

Validation and Troubleshooting

After completing the configuration, verify the backend health in the OCI Console. A healthy status confirms that the load balancer can reach the private EBS backend and that the configured health-check endpoint is responding.

 

When the load balancer’s public IP address was tested from a browser, the following timeout error occurred:myebs.regionalsub.regionalsub.oraclevcn.com took too long to respond.

ERR_CONNECTION_TIMED_OUT

The ERR_CONNECTION_TIMED_OUT message indicates that name resolution succeeded—for example, through the updated hosts file—but the browser could not establish a TCP connection to the public load balancer. This points to a routing, security-rule, firewall, or listener-access issue rather than a DNS-resolution failure.

In this case, the public load balancer subnet did not have a route to an Internet Gateway. Without that route, the subnet could not provide the required internet-facing return path.

Create or update the route table associated with the public load balancer subnet so that internet-bound traffic is routed through the Internet Gateway, as shown below. Also verify that the listener port is permitted by the applicable security rules.

After the Internet Gateway route was added and the network rules were verified, the public endpoint became reachable and the timeout error was resolved.

 

 



 

Tuesday, September 8, 2026

Accelerating ODA Cluster Upgrades Using Reimage and RMAN Recovery Automation

 

Introduction

Every Oracle DBA who manages Oracle Database Appliance (ODA) environments eventually encounters a familiar challenge: upgrading an ODA cluster that is running a significantly older release to a much newer target version.

In many cases, Oracle's supported upgrade path requires multiple intermediate upgrades. A cluster running an older version may need to be upgraded through several sequential releases before reaching the target version. While technically straightforward, this approach often becomes operationally expensive, time-consuming, and risky due to the number of maintenance activities involved.

Recently, I faced a situation where an ODA cluster required a significant version jump. Rather than spending hours, or sometimes days, performing multiple patch hops, I adopted a different strategy.

Reimage the ODA cluster directly to the target version and automate database restoration using RMAN backups.

To address this challenge, I developed a set of automation scripts that collect RMAN backup metadata, generate recovery command files, and streamline the database restoration process after an ODA reimage.

Disclaimer

Disclaimer: The scripts and procedures described in this article were developed to support a specific ODA upgrade and recovery scenario. While they have proven effective in my environment, every ODA deployment is unique.

Always test these scripts thoroughly in a non-production environment before using them in production.

Ensure that:

  • Valid RMAN backups are available.
  • Recovery procedures are tested end-to-end.
  • ODA reimage activities follow Oracle support recommendations.
  • Appropriate rollback plans are documented.
  • Recovery time objectives (RTO) and recovery point objectives (RPO) are validated.

The author assumes no responsibility for any data loss, downtime, or issues resulting from the use of these scripts without proper testing and validation.

The approach consisted of four major phases.

Assumption:

  •   RMAN backup is taken without recovery catalog
  •   Backup goes to a NFS file system so that reimage does not wipe out any backup 

Phase 1: Backup Preparation

One Day Before Maintenance

A Level 0 RMAN backup is scheduled.

This ensures that a complete recovery baseline exists before the maintenance window.

On Upgrade Day

A final archived log backup is taken.

This captures all recent transactions since the Level 0 backup and minimizes potential data loss.

Phase 2: Backup Metadata Collection 

Before reimaging the appliance, it is critical to preserve information required for restoration.

The first automation script was designed to collect:

  • Database Name
  • DBID
  • Latest RMAN Autobackup Piece
  • Backup Completion Timestamp
After reimaging SPFILE is unavailable,Controlfile is unavailable,Database metadata is gone.
Without this information, restoring databases becomes far more complicated.

Script 1: Backup Information Discovery

The script automatically:

  • Reads entries from /etc/oratab
  • Detects active databases
  • Switches Oracle environments dynamically
  • Connects as SYSDBA
  • Identifies the latest autobackup piece
  • Captures DBID information

Generated output resembles:

ORCL,4223490777,/mntbackup/c-4223490777-20260907-00,2026-09-07 18:01:00

This output becomes the foundation for automated recovery.

Sample script
##############################################################################
#!/bin/bash
# ==============================================================================
# Script: extract_autobackups.sh
# Description: Gathers DBNAME, DBID, and the LAST autobackup piece containing
#              both SPFILE and Controlfile for all active Oracle databases.
# Output format: ,DBNAME,DBID,LAST_AUTOBACKUP_PIECE
# ==============================================================================

# Ensure the script runs as the oracle user
if [ "$(whoami)" != "oracle" ]; then
    echo "Error: This script must be executed by the 'oracle' OS user."
    exit 1
fi

ORATAB=/etc/oratab

# Check if oratab exists
if [ ! -f "$ORATAB" ]; then
    echo "Error: /etc/oratab file not found on this server."
    exit 1
fi

#Remove output file if already exist for host
if [ -f ./`hostname -s`_output.log ];then
   rm ./`hostname -s`_output.log
fi
# Loop through all running, auto-start eligible entries in oratab
grep -v -e '^#' -e '^$' "$ORATAB" | cut -d: -f1,2 | while IFS=: read -r O_SID O_HOME; do

    # Check if the database instance process (pmon) is actually running
    if ps -ef | grep -v grep | grep -q "ora_pmon_${O_SID}"; then

        # Set up Oracle environment variables dynamically
        export ORACLE_SID=$O_SID
        export ORACLE_HOME=$O_HOME
        export PATH=$ORACLE_HOME/bin:$PATH
        export LD_LIBRARY_PATH=$ORACLE_HOME/lib

        # SQL query to extract the exact metadata matching your format requirement
        sqlplus -s / as sysdba <<-EOF >> ./`hostname -s`_output.log
        SET PAGESIZE 0
        SET FEEDBACK OFF
        SET VERIFY OFF
        SET HEADING OFF
        SET ECHO OFF
        SET LINESIZE 2000
        alter session set nls_date_format='YYYY-MM-DD HH24:MI';
        SELECT  d.name ||',' || d.dbid || ',' || b.handle||','||to_date(b.completion_time,'YYYY-MM-DD-HH24:MI')||':00'
        FROM v\$database d, v\$backup_piece b
        WHERE b.handle LIKE '%' || d.dbid || '%'
          AND b.completion_time = (
        SELECT MAX(completion_time)
        FROM v\$backup_piece
        WHERE handle LIKE '%' || (SELECT dbid FROM v\$database) || '%');
        EXIT;
        EOF
    fi
done
echo "Check output file `hostname -s`_output.log"
########################################################################

Phase 3: Automatic RMAN Recovery Script Generation

Once backup metadata is collected, a second script consumes the output and automatically generates RMAN recovery command files.

Instead of manually creating RMAN restore commands for every database, recovery scripts are generated dynamically.

Typical restore workflow:

Set DBID ->Restore SPFILE ->Startup nomount ->Restore controlfile ->Mount Database->Restore database->Recover database until time->Open resetlogs

The generated .rcv files provide a repeatable and standardized recovery process.
Sample script
###########################################################################
#!/bin/bash
# ==============================================================================
# Script: generate_rman_files.sh
# Description: Reads a CSV source file and generates a dedicated, cleanly formatted
#              RMAN run block command file (.rcv) for each database.
# ==============================================================================

# Input file containing your database metadata
read -p "Enter metadata input file: " INPUT_FILE

# Verify input file exists
if [ ! -f "$INPUT_FILE" ]; then
    echo "Error: Input file $INPUT_FILE not found."
    exit 1
fi

echo "======================================================================"
echo " Generating RMAN Command Files..."
echo "======================================================================"

# Read the file line by line using comma as the field separator
while IFS=, read -r DBNAME DBID BACKUP_PIECE COMP_TIME || [[ -n "$DBNAME" ]]; do
    DBNAME_LOWER=$(echo "$DBNAME" | tr '[:upper:]' '[:lower:]')
    # Skip empty lines or comment lines
    [[ -z "$DBNAME" || "$DBNAME" =~ ^# ]] && continue
    DBHOME=$(grep "^${DBNAME_LOWER}:" /etc/oratab | awk -F ":" '{print $2}')
    #echo $DBHOME
    # Define the output script name for this specific database
    OUTPUT_RCV="restore_${DBNAME,,}.rcv"

    # Build the clean, safely indented RMAN string block using regular spaces
    cat <<EOF > "${OUTPUT_RCV}"
    RUN
    {
    SET DBID $DBID;
    STARTUP NOMOUNT PFILE='\$ORACLE_HOME/dbs/init$DBNAME_LOWER.ora';
    RESTORE SPFILE FROM '$BACKUP_PIECE';
    SHUTDOWN IMMEDIATE;
    STARTUP NOMOUNT;
    RESTORE CONTROLFILE FROM '$BACKUP_PIECE';
    ALTER DATABASE MOUNT;
    ALLOCATE CHANNEL CH1 TYPE DISK;
    ALLOCATE CHANNEL CH2 TYPE DISK;
    ALLOCATE CHANNEL CH3 TYPE DISK;
    ALLOCATE CHANNEL CH4 TYPE DISK;
    RESTORE DATABASE;
    RECOVER DATABASE UNTIL TIME "to_date('$COMP_TIME','YYYY-MM-DD HH24:MI:SS')";
    }
EOF
    # Write the formatted string block directly out to the database's custom .rcv file
    echo "Created : $DBNAME_$OUTPUT_RCV"

done < "$INPUT_FILE"

echo "======================================================================"
echo " Generation complete!"
echo "======================================================================"
###########################################################################
Above script will create necessary .rcv files for all databases from metadata created in step1.

Benefits

Eliminates Human Error

No need to manually:

  • Identify DBID
  • Locate autobackups
  • Build RMAN command files

Improves Consistency

Every database follows the same recovery methodology.

Saves Time

Script generation is completed in minutes regardless of the number of databases.

Phase 4: ODA Reimage

With backups secured and recovery files generated we can proceed with below steps
  • Shutdown Databases
  • Shutdown Cluster
  • Reimage ODA
  • Deploy Target Version
  • Configure Infrastructure
If you have 30+ databases running in the cluster and you need to create databases with minimal error.Follow my previous blog where i have shown how to dynamically generate database creation command.
https://exploreora.blogspot.com/2026/08/oda-reimage-survival-guide.html


Post-Reimage Challenge: SPFILE Registration

One of the common challenges after restoring databases is Clusterware configuration consistency.

Even though the database is successfully restored,spfile restoration in ASM diskgroup will fail.To solve this problem we shall use script 3

Script 3: SPFILE Configuration Remediation

To solve this problem, another automation script was developed.

The script automatically:

  • Detects Grid Infrastructure version
  • Enumerates Clusterware-managed databases
  • Checks registered SPFILE location
  • Identifies missing SPFILE configurations
  • Copies SPFILE from ASM when required
  • Updates Clusterware metadata using SRVCTL
Sample script
#############################################################################
#!/bin/bash
#This script check whether SPFILE location is updated in cluster and changes accordingly
gi_version=`ps -ef | grep ohasd.bin | grep -v grep | awk -F " " '{print $8}' | awk -F "/" '{print $4}'`
for DBNAME in `/u01/app/$gi_version/grid/bin/srvctl config database`;do
ORACLE_HOME=`/u01/app/$gi_version/grid/bin/srvctl config database -d $DBNAME | grep -i 'Oracle home' | awk -F ":" '{print $2}'`
SPFILE=`/u01/app/$gi_version/grid/bin/srvctl config database -d $DBNAME | grep -i 'Spfile' | awk -F ": " '{print $2}'`
#echo $SPFILE
if [[ -z "$SPFILE" ]];then
#Check physical existence in ORACLE_HOME path
        SPFILE="${ORACLE_HOME}/dbs/spfile${DBNAME}.ora"
echo $SPFILE
        if [ -f $SPFILE ];then
                /u01/app/$gi_version/grid/bin/srvctl modify database -d $DBNAME -spfile $ORACLE_HOME/dbs/spfile${DBNAME}.ora
                /u01/app/$gi_version/grid/bin/srvctl config database -d $DBNAME | grep -i 'Spfile' | awk -F ": " '{print $2}'
                echo "SPFILE location for $DBNAME updated in cluster...."
        fi
else
        #SPFILE location starts with the string +DATA then copy it from ASM to local filesystem
        if [[ "$SPFILE" =~ ^\+DATA ]]; then
                echo "$SPFILE found in ASM and copying in ORACLE_HOME/dbs"
                su - grid -c "asmcmd cp ${SPFILE} /tmp/spfile${DBNAME}.ora"
                cp /tmp/spfile${DBNAME}.ora $ORACLE_HOME/dbs
                chown oracle:asmadmin  $ORACLE_HOME/dbs/spfile${DBNAME}.ora
                /u01/app/$gi_version/grid/bin/srvctl modify database -d $DBNAME -spfile $ORACLE_HOME/dbs/spfile${DBNAME}.ora
        fi
fi
done
##########################################################################
Now we can shutdown the database ,set ORACLE_SID,ORACLE_HOME using oraenv utility and run like  below
nohup rman target / cmdfile=restore_${DBNAME,,}.rcv log=restore_${DBNAME,,}.log &

Lessons Learned

  1. Always perform and validate RMAN backups before the maintenance window.
  2. Capture DBID and autobackup information ahead of the reimage.
  3. Automate RMAN command generation wherever possible.
  4. Validate Clusterware configuration after recovery.
  5. Keep recovery procedures standardized across all databases.
  6. Test the restore process in a non-production environment before implementation.
Hope you have learned something useful

Oracle E-Business Suite Load Balancer Setup in OCI

Executive Summary This white paper provides an end-to-end procedure for configuring an OCI public load balancer to route client traffic to a...