OEM DBConsole startup issues on 10g R2 database for 11i/R12

On one of our cloned 11i Instance, I faced issues while trying to start Database Control (aka Enterprise Manager Console) using “emctl start dbconsole“.
This post covers issues and resolution (Issues mentioned here are specific to particular Database and Operating System Version)
.

Instance Details

Operating System : Solaris Sparc 64 Bit
Database : Oracle Database Version 10.2.0.2
Apps Version : 11.5.10.2

.
Things good to know about Enterprise Manager

Database Control : is Web-based tool to manage database. Using Database Control (OEM) you can perform administrative task like adding tablespace, table, index, managed space or view performance statistics…

Files/Directory for Database Control (DB Console) :
a) $ORACLE_HOME/sysman
b) $ORACLE_HOME/[hostname_sid]/sysman
c) $ORACLE_HOME/oc4j/j2ee/OC4J_DBConsole_[hostname]_[sid]

Files useful in troubleshooting : $ORACLE_HOME/[hostname_sid]/sysman/log/

How to use Database Control

a) ORACLE_HOME/bin/emctl start dbconsole  [To start DB Control] b) ORACLE_HOME/bin/emctl status dbconsole [To check status of DB Control] c) ORACLE_HOME/bin/emctl stop dbconsole   [To stop DB Control]

Issue 1 : On running “emctl start dbconsole“, I received error message

OC4J Configuration issue. $OH/oc4j/j2ee/OC4J_DBConsole_$hostname_$sid not found

This means enterprise manager was not configured and Fix was to configure enterprise manager using emca (Enterprise Manger Configuration Assistant).
Steps to create enterprise manager configuration files & repository are explained in “Oracle Enterprise Manager Advanced Configuration Guide  here

.

How to create Database Control(DB Console)

emca (Enterprise Manager Configuration Assistant) is utility to create Database Control Configuration files and Repository Objects

For 10.2.X Database I used
emca -config dbcontrol db -repos create

.

Issue 2 : While running emca, I got below error message

UnsatisfiedLinkError exception loading native library: njni10
Exception in thread “main” java.lang.UnsatisfiedLinkError: get
at oracle.net.common.NetGetEnv.get(Native Method)
at oracle.sysman. assistants. util.sqlEngine. SQLEngine. getEnvParams (SQLEngine.java:409)

as per Metalink note 553154.1, Fix was to set environment variable and relink executable

Fix 2 :
export LD_LIBRARY_PATH=$ORACLE_HOME/lib32
export LD_LIBRARY_PATH_64=$ORACLE_HOME/lib
cd $ORACLE_HOME/network/lib
make -f ins_net_client.mk client_sharedlib
relink all

Issue 3 : While running “relink all“, there was another error

ld: fatal: library -ljava: not found
ld: fatal: library -ljvm: not found
ld: fatal: File processing errors. No output written to $OH/10.2.0/sysman/lib/libnmemso.so
*** Error code 1
make: Fatal error: Command failed for target `$OH/10.2.0/sysman/lib/libnmemso.so’
Current working directory $OH/10.2.0/sysman/lib
*** Error code 1
make: Fatal error: Command failed for target `libnmemso’

Fix 3 : As per Metalink note 743833.1, issue was because there are references of source system in make file and fix was to change references from source to target instance. (DB was cloned as part of 11i Cloning steps)

cp $ORACLE_HOME/sysman/lib/env_sysman.mk $ORACLE_HOME/sysman/lib/env_sysman.mk.bak

change
JRE_LIB_DIR=$SOURCE_ORACLE_HOME/10.2.0/jdk/jre/lib/sparc  (It was pointing to source Directory)
to
JRE_LIB_DIR=$TARGET_ORACLE_HOME/10.2.0/jdk/jre/lib/sparc

and relink again using “relink all
.


Issue 4
: Just when I was expecting all issues to be resolved by now I got another (different) error message while  running “relink all

ld: warning: file $OH/10.2.0/lib/libclntsh.so.10.1: wrong ELF class: ELFCLASS64
Undefined                       first referenced
symbol                             in file
ztch                              $OH/10.2.0/lib32//libnmemso.so

ld: fatal: Symbol referencing errors. No output written to $OH/10.2.0/sysman/lib/emagent
*** Error code 1
make: Fatal error: Command failed for target `$OH/10.2.0/sysman/lib/emagent’
Current working directory $OH/10.2.0/sysman/lib
*** Error code 1
make: Fatal error: Command failed for target `emagent’
ld libclntsh.so.10.1 wrong ELF class ELFCLASS64

Fix 4 : another note from Metalink 395731.1, and fix was to create soft link for file “ldflags” like

ln -s $ORACLE_HOME/lib/ldflags $ORACLE_HOME/lib32/ldflags

After creating softlink, I did manage to relink all executable in ORACLE_HOME successfully and it was time to run emca for enterprise manager configuration

emca -config dbcontrol db -repos create

.

Issue 5 : While running above emca command, scripting was prompting for dbsnmp password again and again stating

Invalid username/password” (Even though I could connect to dbsnmp/ dbsnmp_passsword from sqlplus)

Error message 5 : On checking emca logs at  $OH/cfgtoollogs/emca/ emca_$timestamp.log

oracle.sysman.assistants.util.sqlEngine.SQLFatalErrorException: ORA-01034: ORACLE not available

.
Fix 6 : It was time to search in metalink again and this time I got note # 337260.1 according to this note issue was because of user profile (password limit and resource limit)

SQL> select u.username, u.profile, p.resource_name, p.limit
from dba_profiles p, dba_users u
where p.profile=u.profile
and u.username in (‘DBSNMP’)
and p.resource_type = ‘PASSWORD’
order by u.username, p.resource_name;

Profile for user dbsnmp was set according to metalink 337260.1 which means issue was something else, on doing some hit & trial I realized I was using SID in uppercase where as in database listener.ora SID was in lower case.

When I enetered SID name in lowercase, emca script accepted dbsnmp password and finally after 3 hours I managed to install/configure enterprise manager console on cloned target 11i Database Instance.

30-Oct-2008 16:51:05 oracle.sysman.emcp.EMDBPostConfig performConfiguration
INFO: >>>>>>>>>>> The Database Control URL is http://machinename:<port_number>/em <<<<<<<<<<<
Enterprise Manager configuration completed successfully
FINISHED EMCA at 30-Oct-2008 16:51:05


More on how to monitor database using database control (dbconsole) coming soon …

References

  • 278100.1  How To Drop, Create And Recreate DB Control In A 10g Database
  • 395162.1  How to manage DB Control 10.2 for RAC Database with emca
  • 743833.1  Rapid Clone & Em Dbconsole
  • 549079.1  Troubleshooting Database Control Startup Issues
  • 553154.1  “UnsatisfiedLinkError: njni10” When Tring to Create the dbconsole With EMCA
  • 337260.1  Dbsnmp Password Not Accepted
  • Enterprise Manager Advanced Configuration Guide

About the Author Masroof Ahmad

Leave a Comment:

16 comments
SALAU ADEBAYO says March 23, 2009

I need help.I want to know the solution for this error: \Jinitiator 1.3 Fatal Error. The Java Runtime Environment cannot be loaded from . This error occured from 2 workstations that asseses the server. This occured after the systems ware restored. The machine is a vista machine. Thanks

Reply
Atul Kumar says March 23, 2009

Jinitiaor/Java plugin is missing from these machines .

What is ur apps version ?
Are you using jinitiator or Java Plugin ?

Reply
Mr.Ramontic says May 12, 2009

Hi! I need help.
The question is: what http server does DB console (10g r2) use and how does this http server work?

Reply
Atul Kumar says May 12, 2009

DB console uses http listener of OC4J in ORACLE_HOME

OC4J acts as HTTP server for DBConsole

Reply
Mr.Ramontic says May 12, 2009

So, Apache is not used. Thank you.
Actually, i need to realize windows native authentication. I read that there is oc4j plug-in for IIS. What dod you think, can i use this plug-in to use IIS windows native before oc4j authentication? And is there any other way of using native auth with DBConsole (before basic auth of oc4j)?

Thanks for your answers.

Reply
» Error while cloning R12 - Database Tier : RC-00110: Fatal: Error occurred while relinking of ApplyDBTechStack : Error while running adlnkoh.sh Online Apps DBA: One Stop Shop for Apps DBA’s says April 19, 2010

[…] : As per Metalink note 743833.1 Rapid Clone & Em Dbconsole  also reported here; issue is because there are references of source system in make file and fix is to change […]

Reply
sani says December 27, 2010

Hi Atul

I am new in Oracle Apps R12, follow your instructions and successfully create OEM, but now problem is when I gave sys/password, to login in OEM, it says invalid password, but I know password is correct,

infact, db is not allowing me to login from any tool using RPC,

need your advice

thanks in advance
sani

Reply
Atul Kumar says December 27, 2010

@ Sani,
Use sysman account to login to OEM. If you wish to use sys account then create a password file on database using orapwd http://download.oracle.com/docs/cd/B28359_01/server.111/b28310/dba007.htm

Reply
sani says December 28, 2010

Thanks Atul and really impressed, I already done this;

you are right when, we do fresh installation of R12 on linux, db password file is not created by default, we have to create it manually;

thank you very much for your prompt reply 🙂

Thanks
Sani

Reply
manasa says April 18, 2011

For operational reason, I would like to extend the system jobs to be viewed / read only from the Operational staff userid. In short, Michael/sri can view jobs configured in the instance “abc”.

Do you think, Grid control will help? Or is there any method to assign read only access to EM for operational staff.

plz ans me soon!

Reply
Atul Kumar says April 18, 2011

@ manasa,
Yes this can be achieved in OEM and yours is very basic requirement .
OEMaccess (read only, only few tabs, or super user) is granted by OEM application role.

Check more at http://download.oracle.com/docs/cd/E11857_01/em.111/e16790/security3.htm#BABCCAFC

Reply
manasa says April 18, 2011

@atul:

Thank you so much …
since I am new to this i dont have much idea…

Reply
Amasun says November 10, 2011

$ emca

STARTED EMCA at Thu Nov 10 15:38:57 GMT-05:30 2011
Enter the following information about the database to be configured
Listener port number: 1521
Database SID: XXXX
Service name: XXXX
Email address for notification:
Email gateway for notification:
Password for dbsnmp:
Password for sysman:
Password for sys:

—————————————————————–

You have specified the following settings

Database ORACLE_HOME ……………. /home/oracle/OraHome_1
Enterprise Manager ORACLE_HOME ……………. /home/oracle/OraHome_1

Database host name ……………. XXXXX
Listener port number ……………. 1521
Database SID ……………. XXXXX
Service name ……………. XXXXX
Email address for notification ……………
Email gateway for notification ……………

—————————————————————–
Do you wish to continue? [yes/no]: yes
DBConsole is already configured for the database mdairy
Would you like to re-configure DBConsole for the specified database? [yes/no]: yes
Make sure that DBConsole has been shutdown
When you are ready to continue, press

UnsatisfiedLinkError exception loading native library: njni10
Exception in thread “main” java.lang.UnsatisfiedLinkError: get
at oracle.sysman.assistants.util.sqlEngine.SQLEngine.getEnvParams(SQLEngine.java:408)
at oracle.sysman.assistants.util.sqlEngine.SQLEngine.initialize(SQLEngine.java:227)
at oracle.sysman.assistants.util.sqlEngine.SQLEngine.(SQLEngine.java:212)
at oracle.sysman.assistants.util.sqlEngine.SQLPlusEngine.(SQLPlusEngine.java:104)
at oracle.sysman.assistants.util.sqlEngine.SQLEngine.getSQLEngine(SQLEngine.java:356)
at oracle.sysman.emcp.EMConfig.initSQLEngine(EMConfig.java:5352)
at oracle.sysman.emcp.EMConfig.checkConfiguration(EMConfig.java:948)
at oracle.sysman.emcp.EMConfig.perform(EMConfig.java:233)
at oracle.sysman.emcp.EMConfigAssistant.invokeEMCA(EMConfigAssistant.java:659)
at oracle.sysman.emcp.EMConfigAssistant.performSetup(EMConfigAssistant.java:608)
at oracle.sysman.emcp.EMConfigAssistant.statusMain(EMConfigAssistant.java:313)
at oracle.sysman.emcp.EMConfigAssistant.main(EMConfigAssistant.java:166)

Please help me out from this?

Reply
Amasun says November 10, 2011

OS:AIX 5.3
Oracle Version:Oracle 10g (10.1.0)

Please help me out.

Reply
Victor says January 17, 2013

found solution for my issue in “Issue 3 : While running “relink all“, there was another error”
Thank you.

Reply
jo says February 11, 2013

Hi, i have this error ORacle 10g (10.2.0.4) with patch 8350262, need your advice,thanks in advance, Jo

2013-02-08 14:28:52 Thread-5368 ERROR pingManager: nmepm_pingReposURL: Cannot connect to https://myhost:5500/em/upload/: retStatus=-1
2013-02-08 14:28:53 Thread-5368 WARN http: snmehl_connect: connect failed to (myhost:5500): No connection could be made because the target machine actively refused it.
(error = 10061)
2013-02-08 14:28:53 Thread-5368 ERROR pingManager: nmepm_pingReposURL: Cannot connect to https://myhost:5500/em/upload/: retStatus=-1
2013-02-08 14:28:59 Thread-5236 ERROR upload: Error in uploadXMLFiles. Trying again in 300.00 seconds.
# An unexpected error has been detected by HotSpot Virtual Machine:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000077b00895, pid=5804, tid=5808
#
# Java VM: Java HotSpot(TM) 64-Bit Server VM (1.5.0_11-b03 mixed mode)
# Problematic frame:
# C [ntdll.dll+0x50895]
#
# An error report file with more information is saved as hs_err_pid5804.log
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp

Reply
Add Your Reply