ODA Reimage Survival Guide: Automatically Recreating Dozens of Databases Before RMAN Restore

One of the biggest challenges during an Oracle Database Appliance (ODA) OS reimage is that the process wipes out all appliance metadata, database homes, and database registrations. While RMAN backups protect the database contents, they do not recreate the ODA infrastructure required to host those databases.

Recently, I had to plan a reimage of an ODA cluster running Oracle 19c with more than 20 databases spread across multiple database homes. The main challenge was:

After reimaging the appliance, how do we quickly recreate all databases using odacli create-database without manually collecting and typing parameters for every database?

This article describes a simple and repeatable approach.

The Challenge

When an ODA is reimaged:

• Linux OS is rebuilt
• Grid Infrastructure is reinstalled
• ODA repository metadata is wiped out
• Database homes disappear
• Databases are no longer registered with ODA

After the reimage, the typical workflow is:

1. Recreate DB homes
2. Create empty databases using odacli create-database
3. Restore RMAN backups
4. Recover databases

For a handful of databases this is manageable.

For 20, 30, or more databases, manually recreating commands becomes both time-consuming and error-prone.

 

Required Information

To recreate a database using ODA, we need information similar to:

odacli create-database \

--dbname cisprep6 \

-u cisprep6 \

-cs UTF8 \

-cl OLTP \

--dbtype SI \

--no-cdb \

--target-node oda-node1 \

--dbshape Odb4 \

--dbstorage ASM \

-dh <DBHOMEID>

 

The challenge is collecting these attributes for every database before the reimage.

Exploring the ODA Repository

Oracle stores database metadata inside the internal MySQL repository.

Connect to the repository:

/opt/oracle/dcs/mysql/bin/mysql --defaults-file=/opt/oracle/dcs/mysql/etc/mysqldb.cnf

List tables:

use dcsagentdb;

show tables;

The most useful tables were:

Db,DBNode

Understanding Database Metadata

The db table contains most of the information required for recreation.

Example query:

select

name,

databaseUniqueName,

characterSet,

dbClass,

dbType,

dbShape,

isCdb,

dbTargetNodeNumber,

dbVersion,

dbStorage

from db;

 

Sample output

+----------+--------------------+--------------+

| name | databaseUniqueName | characterSet |

+----------+--------------------+--------------+

| edadev | edadev | UTF8 |

| edatest | edatest | UTF8 |

| soaiq56 | soaiq56 | AL32UTF8 |

+----------+--------------------+--------------+

Resolving Target Node Names

The db table stores only the node number:0,1

To map that value to the actual node name:

select nodeNumber,nodeName from DBNode;

Example

+------------+----------------+

| nodeNumber | nodeName |

+------------+----------------+

| 0 | oda-node1 |

| 1 | oda-node2 |

+------------+----------------+

Generating ODA Commands Automatically

Instead of manually creating commands, we can make MySQL generate them.

Example query:

SELECT CONCAT(

'odacli create-database ',

'--dbname ',d.name,' ',

'-u ',d.databaseUniqueName,' ',

'-cs ',d.characterSet,' ',

'-cl ',UPPER(d.dbClass),' ',

'--dbtype ',UPPER(d.dbType),' ',

IF(TRIM(IFNULL(d.isCdb,''))='YES',

'--cdb ',

'--no-cdb '

),

'--target-node ',n.nodeName,' ',

'--dbshape ',d.dbShape,' ',

'--dbstorage ',UPPER(d.dbStorage),' '

)

FROM db d

JOIN DBNode n

ON n.nodeNumber=d.dbTargetNodeNumber;

This generates ready-to-run odacli create-database commands.

Complete Automation Script

The following script generates a database recreation script automatically.

#!/bin/bash

DBHOMEID="<DBHOMEID>"

SCRIPT_NAME="create_database_$(hostname -s).sh"

/opt/oracle/dcs/mysql/bin/mysql \

--defaults-file=/opt/oracle/dcs/mysql/etc/mysqldb.cnf <<EOF > ${SCRIPT_NAME}

USE dcsagentdb;

SELECT CONCAT(

'odacli create-database ',

'--dbname ',d.name,' ',

'-u ',d.databaseUniqueName,' ',

'-cs ',d.characterSet,' ',

'-cl ',UPPER(d.dbClass),' ',

'--dbtype ',UPPER(d.dbType),' ',

IF(TRIM(IFNULL(d.isCdb,''))='YES',

'--cdb ',

'--no-cdb '

),

'--target-node ',n.nodeName,' ',

'--dbshape ',d.dbShape,' ',

'--dbstorage ',UPPER(d.dbStorage),' ',

'-dh ${DBHOMEID}'

)

FROM db d

JOIN DBNode n

ON n.nodeNumber=d.dbTargetNodeNumber;

EOF

sed -i '/^odacli/!d' ${SCRIPT_NAME}

chmod u+x ${SCRIPT_NAME}

echo "Generated: ${SCRIPT_NAME}"

 

Why This Helps

Instead of manually documenting:

• Database name
• Unique name
• Character set
• Database shape
• Storage type
• Target node
• CDB/Non-CDB status

for every database, the script extracts everything directly from the ODA repository.

For environments with 20+ databases, this can save several hours of manual effort and significantly reduce human errors.

 

Recommended Pre-Reimage Checklist

Before starting the ODA reimage:

✅ RMAN backups validated

✅ Database home inventory captured

✅ odacli list-dbhomes output saved

✅ Database recreation script generated

✅ TDE wallets backed up (if applicable)

✅ Listener and application connection information documented

✅ Database services documented

✅ Post-reimage DB home creation plan prepared

 

Final Thoughts

RMAN backups are only part of an ODA recovery strategy. During a full appliance reimage, the biggest challenge is often recreating the appliance metadata accurately and consistently. Leveraging the ODA repository itself to generate odacli create-database commands provides a simple, reliable, and highly scalable approach for environments hosting dozens of databases.

 

Same dbhome we can group together and run like below

 

 cat create_databases_test.sh

/opt/oracle/dcs/bin/odacli list-dbhomes

echo -n "Enter DBHOME-ID from the above list: "

read DBHOMEID

odacli create-database --dbname db1 -u db1 -cs UTF8 -cl OLTP --dbtype SI --no-cdb --target-node tuslsoda01b --dbshape Odb4 --dbstorage ASM -dh $DBHOMEID

odacli create-database --dbname db2 -u db2 -cs UTF8 -cl OLTP --dbtype SI --no-cdb --target-node tuslsoda01a --dbshape Odb2 --dbstorage ASM -dh $DBHOMEID

 

Only catch is we have to type password 2 times for every database as odacli does not gives any option to provide password as an argument…

 

I hope you have learned something useful

 

 

 

 

How to use EBS marketplace image for quick deployment

 

Operating Oracle E-Business Suite on Oracle Cloud Infrastructure

Deployment, Listener Remediation, Lifecycle Management, and Secure Access for a Private-Subnet Environment

Technical White Paper | Oracle E-Business Suite Release 12.2 and Oracle Database 19c

 

Executive Summary

This white paper presents a repeatable operating model for an Oracle E-Business Suite environment provisioned on Oracle Cloud Infrastructure (OCI). It converts an observed deployment and troubleshooting exercise into a controlled procedure for environment discovery, database and application-tier startup, listener registration, orderly shutdown, service validation, and secure browser access. The reference implementation uses an Oracle Database 19c container database, an Oracle E-Business Suite Release 12.2 application tier, Enterprise Command Center (ECC) services, and a compute instance placed in a private subnet.

The central operational finding is that a successful database startup does not guarantee application availability. Name resolution, listener endpoint configuration, dynamic service registration, application environment ownership, middleware dependencies, and network reachability must all be validated as one service chain. The procedure described here emphasizes controlled sequencing, evidence-based validation, and configuration changes that remain supportable through AutoConfig.

1. Purpose and Scope

This document is intended for Oracle E-Business Suite administrators, database administrators, cloud engineers, and support teams responsible for a single-node or compact demonstration environment on OCI. It addresses the following operational outcomes:

·        Identify and source the correct Oracle E-Business Suite environment.

·        Start the database tier and resolve listener startup failures caused by hostname resolution.

·        Verify dynamic database-service registration with the listener.

·        Start the application tier and its middleware dependencies in a controlled sequence.

·        Stop the environment without creating avoidable recovery or consistency risks.

·        Provide secure access to an application hosted in a private subnet.

·        Establish validation checkpoints, log locations, and operational safeguards.

2. Reference Architecture and Environment

Table 1. Reference implementation
Component	Reference value	Operational role
Cloud platform	Oracle Cloud Infrastructure	Compute, networking, identity, and controlled access
Application	Oracle E-Business Suite Release 12.2	Application services, concurrent processing, Forms, OAF, and HTTP services
Database	Oracle Database 19c Enterprise Edition, CDB SID ebscdb	Persistent EBS data and service registration
Oracle home	/u01/install/APPS/19.0.0	Database binaries, network configuration, and diagnostics
Application root	/u01/install/APPS	EBS environment and lifecycle scripts
Network placement	Private subnet	Reduces direct internet exposure; requires an approved access path
Listener	ebscdb on TCP port 1521	Database connectivity and service discovery

 

The architecture should separate administrative access, application ingress, and database connectivity. For production, use distinct subnets and narrowly scoped security rules. A public load balancer can provide controlled HTTPS ingress to private application nodes, while OCI Bastion, corporate VPN, or FastConnect can provide administrative access without assigning public IP addresses to the EBS compute instance.

3. Deployment and Environment Discovery

Provision the EBS image through the approved OCI Marketplace or Oracle E-Business Suite Cloud Manager workflow. Confirm compartment, VCN, subnet, compute shape, storage, identity policies, and load balancer requirements before deployment.

 

Please follow below scrrenshot

Click on next and leave it default

 

Here I have pre-created a private subnet so that instance is not available from internet.Idea here is we will access through another compute instance which is in public subnet

Below step we are generating a private/public key pair for ssh login and transfer those file or copy the contents of those key to store the private/public key.

We can attach block volume later.For now just click next

Finally review and click on create.

Now login to your oci vm which can be accessed from your computer/laptop.Copy the public and private key in /root/.ssh folder and access like below

ssh -i /root /.ssh/<privatekey file> opc@<privateip of ebs compute instance>

 After the instance is available, sign in as the oracle operating-system account and locate the environment file:

find / -name "EBSapps.env" 2>/dev/null
/u01/install/APPS/EBSapps.env

Record the detected path, ownership, permissions, context name, database SID, hostname, and active run file system. Use the supplied wrapper scripts under /u01/install/APPS/scripts when available, because they encode environment-specific sequencing and may run AutoConfig or ECC lifecycle actions in addition to the standard EBS service scripts.

Under /u01/install/APPS/scripts we can see startdb.sh

Just execute ./startdb.sh

Database instance started successfully but listener failed to start with below error

Starting /u01/install/APPS/19.0.0/bin/tnslsnr: please wait...

 

TNSLSNR for Linux: Version 19.0.0.0.0 - Production

System parameter file is /u01/install/APPS/19.0.0/network/admin/listener.ora

Log messages written to /u01/install/APPS/19.0.0/log/diag/tnslsnr/ebsdemo1/ebscdb/alert/log.xml

Error listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=apps.example.com)(PORT=1521)))

TNS-12545: Connect failed because target host or object does not exist

 TNS-12560: TNS:protocol adapter error

  TNS-00515: Connect failed because target host or object does not exist

Listener failed to start. See the error message(s) above...

adcdblnctl.sh: exiting with status 1

adcdblnctl.sh: check the logfile /u01/install/APPS/19.0.0/appsutil/log/ebsdb_apps/adcdblnctl.txt for more information ... 

4. Listener Failure Analysis and Remediation

4.1 Observed symptom

The database wrapper started the CDB successfully, but the listener failed while binding to the configured host apps.example.com. The diagnostic chain was TNS-12545, TNS-12560, and TNS-00515. This indicates that the listener host value could not be resolved or mapped to a local interface; it is not evidence of a database startup failure.

4.2 Root cause

The provisioned listener configuration referenced a logical application hostname that was not resolvable from the host. Oracle Net therefore could not bind the listener endpoint. The database still opened because instance startup and listener startup are separate operations.

4.3 Corrective action

1.       Determine the private IP address and fully qualified domain name assigned to the OCI compute instance.

2.       Ensure forward resolution for the configured listener hostname. In a short-lived demonstration environment, add a controlled /etc/hosts entry; in a managed environment, use authoritative private DNS.

3.       Validate resolution with getent hosts and, where permitted, ping.

4.       Source the database environment and start the listener.

5.       Validate the listener endpoint and service registration.

6.       Make permanent hostname or listener changes through the relevant EBS context file and AutoConfig rather than relying on undocumented manual edits.

<private-ip> ******.dbsubnet.rootvcn.oraclevcn.com ***** apps.example.com apps
getent hosts apps.example.com
export ORACLE_HOME=/u01/install/APPS/19.0.0
export ORACLE_SID=ebscdb
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$LD_LIBRARY_PATH
$ORACLE_HOME/bin/lsnrctl start ebscdb

4.4 Dynamic service registration

A running listener may initially report The listener supports no services. This means the listener process is reachable, but the database has not registered its services against that endpoint. Validate the LOCAL_LISTENER value and force registration after making a supported change:

ALTER SYSTEM SET local_listener='
  (ADDRESS=(PROTOCOL=TCP)
  (HOST=apps.example.com)
  (PORT=1521))' SCOPE=BOTH;
ALTER SYSTEM REGISTER;

Then run lsnrctl status ebscdb and confirm that the expected database and pluggable-database services appear. Treat a manual ALTER SYSTEM statement as an immediate remediation only; reconcile the corresponding EBS context variable and run database-tier AutoConfig so that future configuration regeneration does not reverse the change.

5. Controlled Startup Procedure

The startup sequence establishes dependencies from the data layer upward. Perform each stage as the oracle account and stop if a mandatory validation fails.

Table 2. Startup sequence and checkpoints
Stage	Command or action	Required evidence
1. Environment	cd /u01/install/APPS/scripts	Correct host, owner, context, and filesystem confirmed
2. Database tier	./startdb.sh	Database opened; listener running; expected services registered
3. Application tier	./startapps.sh	AutoConfig, EBS application services, and enabled node services complete with status 0
4. Middleware	Review wrapper output	Node Manager, AdminServer, OHS, APPS listener, Forms, OAFM, OACORE, and concurrent manager are running
5. ECC	Review wrapper output	ECC AdminServer, ZooKeeper, and eccManaged start successfully if configured
6. Functional validation	Open approved application URL and submit a lightweight request	Login page responds, authentication succeeds, and background processing is operational

 

cd /u01/install/APPS/scripts
./startdb.sh
./startapps.sh

If the application wrapper reports that the environment must be sourced from apps, do not bypass the message by manually mixing database and application variables. Confirm the script's expected account and environment file, then rerun it with the correct application-tier context.

6. Controlled Shutdown Procedure

Shutdown reverses the dependency chain. Stop the application tier first so it no longer creates database sessions or submits work, then stop the database tier. Allow sufficient time for concurrent managers and WebLogic managed servers to exit cleanly. Use abort-style termination only when the standard shutdown path cannot complete and the operational risk has been accepted.

cd /u01/install/APPS/scripts
./stopapps.sh
./stopdb.sh

Note: The original notes showed a wrapper name resembling stopsapps.sh. Verify the exact installed filename with ls -l /u01/install/APPS/scripts and use the vendor-provided local wrapper. Do not create a substitute solely to match this paper.

Table 3. Shutdown acceptance criteria
Component group	Acceptance criterion
Application web and middleware	OHS, OPMN, managed servers, AdminServer, Node Manager, and APPS listener are stopped
Business processing	Concurrent managers and Fulfillment Server are stopped without unreviewed errors
ECC	eccManaged, ZooKeeper, ECC AdminServer, and Derby services are stopped when configured
Database	Database is closed and dismounted; instance and listener exit successfully
Logging	All wrapper and component logs are retained for the change record

 

7. Validation and Health Checks

·        Name resolution: The configured listener and application hostnames resolve to the intended private address.

·        Database: CDB and required PDBs are open in the expected mode.

·        Listener: The endpoint listens on the intended hostname and port, and required database services are visible.

·        Application services: All enabled services for the node report running.

·        HTTP path: The login URL is reachable only through the approved network path.

·        Functional test: An authorized user can authenticate and a lightweight concurrent request completes.

·        Diagnostics: Startup, shutdown, AutoConfig, listener, and managed-server logs contain no unexplained fatal errors.

8. Secure Access for a Private-Subnet Deployment

Adding an entry to a workstation hosts file changes name resolution only; it does not create a network route to a private OCI address. The preferred production pattern is an OCI load balancer that terminates TLS and forwards only the required application port to private application nodes. Administrative access should use OCI Bastion or a corporate connection such as IPSec VPN or FastConnect. A temporary SSH port-forwarding session may be appropriate for controlled testing, but it should not become the normal end-user access mechanism.

·        Use TLS for user-facing application traffic and avoid exposing native application or database ports to the internet.

·        Restrict security-list or network-security-group ingress to approved source CIDRs and required ports.

·        Use OCI IAM policies and time-limited bastion sessions for administrative access.

·        Retain the EBS compute and database tiers in private subnets wherever operationally feasible.

·        Use private DNS for stable service names instead of unmanaged host-file aliases.

·        Review the observed ECC WLST warning and migrate administrative connections from insecure T3 to an approved secure channel where supported.

9. Operational Recommendations

1.       Standardize configuration: Maintain host, port, and service values in the EBS context files and regenerate derived configuration through AutoConfig.

2.       Automate health checks: Build a non-destructive status script that checks the database, listener, OPMN/OHS, Node Manager, APPS listener, concurrent manager, AdminServer, and managed servers.

3.       Protect credentials: Do not place APPS, WebLogic, or database passwords in command histories, scripts, or shared logs.

4.       Formalize lifecycle automation: Add explicit prechecks, postchecks, timeouts, return-code handling, and change-record evidence to startup and shutdown wrappers.

5.       Monitor configuration drift: Compare listener, database, and application context values after patching, cloning, or hostname changes.

6.       Test recovery paths: Periodically validate backup, restore, and controlled restart procedures in a non-production environment.

7.       Separate availability from process state: A running process is not sufficient; monitor URL response, authentication, service registration, and representative business processing.

12. Conclusion

Operating Oracle E-Business Suite on OCI requires coordinated management across DNS, Oracle Net, database registration, AutoConfig, WebLogic, application services, and cloud networking. The incident examined in this paper demonstrates why lifecycle scripts must be paired with explicit validation: the database opened even though the listener failed, and the listener later ran before database services were registered. A disciplined dependency sequence, supported configuration management, secure private-subnet access, and evidence-driven health checks turn this type of deployment from an ad hoc installation into a maintainable operational platform.

DB system startup issue in ODA 19.30

 

Problem statement: Database creation in a ODA DB-system (version 19.30) was failing due to insufficient free memory.

DCS-10045:Validation error encountered: DB Shape validation on memory for DB creation/iRestore failed: DCS-10045:Validation error encountered: Available Memory is less than SGA Size { Available : 380MB and SGA Size : 7782MB } on <hostname>.

Inside the DBSystem only free Huge page was 212 which is approximately 400 MB and is exactly matching what we have received as an error.

cat /proc/meminfo | grep -i huge

AnonHugePages:         0 kB

ShmemHugePages:        0 kB

FileHugePages:         0 kB

HugePages_Total:   4083

HugePages_Free:    212

HugePages_Rsvd:        0

HugePages_Surp:        0

Hugepagesize:       2048 kB

Hugetlb:        8361984 kB

 

Action Taken:We attempted to increase huge page manually using the configuration file /etc/systcl.conf from vm.nr_hugepages=4083 to vm.nr_hugepages=8000.It was not taking into effect even after executing sysctl -p and still change was not reflected and HugePages_Total was showing 5929.DBSystem node was rebooted and after that ssh to node was failing with the below error

ssh: connect to host <hostname> port 22: No route to host

 

Troubleshooting: First of all,I must say above actions were taken harshly and never ever try that in a DB systems. Very first thing we should have done is to know DB-Systems detail using below command

odacli describe-dbsystem -n vmdb01

DB System details

--------------------------------------------------------------------------------

                       ID:  2b53b0dc-2d27-47e7-be09-85fdde8310fa

                     Name:  vmdb01

            Image version:  23.26.1.0.0

          Current version:  23.26.1.0.0

                    Shape:  dbs2

             Cluster name:  vmcl

             Grid version:  23.26.1.0.0

             NUMA enabled:  YES

                   Status:  CONFIGURED

                  Created:  2026-03-19 12:34:02 MST

                  Updated:  2026-03-19 13:21:32 MST

 

 User Information

--------------------------

        Multi User Status:  Multi User Access (MUA)

 

 CPU Pool

--------------------------

                     Name:  cpupool2c

          Number of cores:  2

 

                     Host:  oda02a

        Effective CPU set:  1,17,33,49

              Online CPUs:  1, 17, 33, 49

             Offline CPUs:  NONE

 

                     Host:  oda02b

        Effective CPU set:  1,17,33,49

              Online CPUs:  1, 17, 33, 49

             Offline CPUs:  NONE

 

 VM Storage

--------------------------

               Disk group:  DATA

              Volume name:  S8E0A8E79D

            Volume device:  /dev/asm/s8e0a8e79d-364

                     Size:  402.00 GB

                     Used:  132.15 GB

                     Free:  269.85 GB

                    Usage:  32.87%

              Mount Point:  /u05/app/sharedrepo/vmdb01

               Redundancy:  Mirror

          Acc Volume name:  AS8E0A8E79D

        Acc Volume device:  /dev/asm/as8e0a8e79d-163

          Acc Volume size:  1.61 GB

 

 VMs

--------------------------

                     Host:  oda02a

                  VM Name:  x3e0a3e89d

             VM Host Name:  db01.com

            VM image path:  /u05/app/sharedrepo/vmdb01/.ACFS/snaps/vm_x3e0a3e89d/x3e0a3e89d

          Number of cores:  2

                   Memory:  16.00 GB

             Target State:  ONLINE

            Current State:  ONLINE

 

                     Host:  oda02b

                  VM Name:  y3e0a3e89d

             VM Host Name:  db02.com

            VM image path:  /u05/app/sharedrepo/vmdb01/.ACFS/snaps/vm_y3e0a3e89d/y3e0a3e89d

          Number of cores:  2

                   Memory:  16.00 GB

             Target State:  ONLINE

            Current State:  ONLINE

 

 VNetworks

--------------------------

                     Host:  oda02a

                  VM Name:  x3e0a3e89d

                   Public:  10. / 255.255.254.0   / enp0s3 / BRIDGE(pubnet)

                      ASM:  192. / 255.255.255.128 / enp0s4 / BRIDGE(privasm) VLAN(icbond0.100)

             Interconnect:  192.**.**.**  / 255.255.255.252 / enp0s5 / BRIDGE(privnet1) VLAN(icbond0.101)

 

                     Host:  oda02b

                  VM Name:  y3e0a3e89d

                   Public:  10. / 255.255.254.0   / enp0s3 / BRIDGE(pubnet)

                      ASM:  192.**.**.** / 255.255.255.128 / enp0s4 / BRIDGE(privasm) VLAN(icbond0.100)

             Interconnect:  192.**.**.**/ 255.255.255.252 / enp0s5 / BRIDGE(privnet1) VLAN(icbond0.101)

 

 Extra VNetworks

--------------------------

                     Host:  oda02a

                  VM Name:  x3e0a3e89d

                   pubnet:  10. / 255.255.254.0   / PUBLIC

 

                     Host:  oda02b

                  VM Name:  y3e0a3e89d

                   pubnet:  10. / 255.255.254.0   / PUBLIC

 

 Databases

--------------------------

                     Name:  cdbtest

              Resource ID:  24883518-c164-40a7-970a-2f52f8a28e30

              Unique name:  cdbtest

              Database ID:  970363228

              Domain name:  .com

                   Status:  CONFIGURED

               DB Home ID:  a6c30871-762d-412e-939c-b6fbe7cdf7ff

                    Shape:  odb2

                  Version:  23.26.1.0.0

                  Edition:  EE

                     Type:  SI

                     Role:  PRIMARY

                    Class:  OLTP

                  Storage:  ASM

               Redundancy:

         Target node name:  db01

            Character set:  AL32UTF8

        NLS character set:  AL16UTF16

                 Language:  AMERICAN

                Territory:  AMERICA

          Console enabled:  false

        High Availability:  false

      Associated networks:  Public-network

         Backup config ID:

       Level 0 Backup Day:  sunday

       Autobackup enabled:  false

              TDE enabled:  false

                 CDB type:  true

                 PDB name:  pdbtest

           PDB admin user:  pdbadmin

 

 We can see there are 2 vm nodes under physical host oda02a and oda02b.Now we will take a xml dump file of each vm and compare the values

Every vm maintains its configuration file into a XML file under ACFS file storage where actual VM image is located.

Use below command to find running VM in the bare metal node like below

virsh list --all

 Id   Name         State

----------------------------

 7    x3e0a3e89d   running

Then dump contents of the XML file like below

virsh dumpxml x3e0a3e89d > /root/x3e0a3e89d.xml

Under html tag sourcefile we can see the location of the xml file

Note:- The virsh program is the main interface for managing virsh guest domains. The program can be used to create, pause, and shutdown domains. It can also be used to list current domains.virsh operations rely upon libvirtd library.

grep -i memory /root/x3e0a3e89d.xml

  <memory unit='KiB'>16777216</memory>

  <currentMemory unit='KiB'>16777216</currentMemory>

  <memoryBacking>

  </memoryBacking>

      <cell id='0' cpus='0-1' memory='8388608' unit='KiB'>

      <cell id='1' cpus='2-3' memory='8388608' unit='KiB'>

 

Now we will update actual xml configuration using below command where we are doubling memory in VM level also in VM node level

virsh edit x3e0a3e89d

Increase the memory value for the parameters per below

<memory unit='KiB'>33554432</memory>

<currentMemory unit='KiB'>33554432</currentMemory>

<cell id='0' cpus='0-1' memory='16777216' unit='KiB'>

<cell id='1' cpus='2-3' memory='16777216' unit='KiB'>

 

Now shutdown using below command

virsh destroy x3e0a3e89d

Domain 'x3e0a3e89d' destroyed

Now below command removes the VM definition from /etc/libvirt/qemu/x3e0a3e89d.xml but does not delete the virtual disks bydefault

virsh undefine x3e0a3e89d

Domain 'x3e0a3e89d' has been undefined

 

Done same from other node

virsh destroy y3e0a3e89d

Domain 'y3e0a3e89d' destroyed

 

virsh undefine y3e0a3e89d

Domain 'y3e0a3e89d' has been undefined

 

Now stop the db system and then start the db-system

odacli stop-dbsystem -n vmdb01

 

odacli start-dbsystem -n vmdb01

 

At this stage we were able to login in the DBsystem but dcs agent was down

odacli ping-agent

DCS-10033:Service DCS agent is down.

Restarted the agent using below command from root

systemctl stop initdcsagent

systemctl start initdcsagent

 

Now change what we made in vm level is not reflected in ODA repository.SO to sync that we executed below command to increase memory without changing the shape of the db system

odacli modify-dbsystem -m 32G -n vmdb01

So please mind that if we face low hugepage memory issue in dbsystem do not just change systcl.conf in vm nodes.

Stop dbsystem from both nodes

Use above command if you just want to increase memory without changing shape.That will take care and distribute equally new memory settings

You can also increase the shape of the dbsystem like below

odacli modify-dbsystem -n vmdb01-s dbs6

 

I hope you have learnt something useful…

  

Reference:- ODA ODACLI UPDATE-OSPARAMETERS or OAKCLI RECONFIGURE OSPARAMS to Calculate, Update and Set Kernel / Memory Values Including Hugepages

KB593630