ORA-609 : TNS-12537: TNS:connect…

 

AlertLog中最近出现的错误：

 

 ***********************************************************************

Fatal NI connect error 12537, connecting to:

 (LOCAL=NO)

  VERSION INFORMATION:

 TNS for Solaris: Version 11.1.0.7.0 -
Production

 Oracle Bequeath NT Protocol Adapter for Solaris:
Version 11.1.0.7.0 - Production

 TCP/IP NT Protocol Adapter for Solaris: Version
11.1.0.7.0 - Production

  Time: 12-DEC-2011 08:52:38

  Tracing not turned on.

  Tns error struct:

    ns main err
code: 12537

   

TNS-12537: TNS:connection closed

    ns secondary
err code: 12560

    nt main err
code: 0

    nt secondary
err code: 0

    nt OS err
code: 0

ORA-609 : opiodr aborting process unknown ospid (8484_1)

Mon Dec 12 08:56:28 2011

Incremental checkpoint up to RBA [0x77a5.870bb.0], current log tail
at RBA [0x77a5.9ff51.0]

Mon Dec 12 09:00:53 2011

***********************************************************************

 

查询MetaLink:

ORA-609 TNS-12537 and TNS-12547 in 11g
Alert.log [ID 1116960.1] 

Cause

The ORA-609 error is thrown when a client connection of any kind
failed to complete or aborted the connection process before the
server process was completely spawned.

Very often, this connection abort is due to a
timeout.  Beginning with 10gR2, a default value
for inbound connect timeout has been set at 60
seconds.  This time limit is often inadequate for
the entire connection process to
complete.   

We have also discovered that the ORA-609 occurs frequently in
installations where the database is monitored by DB Console and the
Enterprise Manager agent
(emagent).   After the DB Console
is started and as a matter of routine, the emagent will repeatedly
try to connect to the target instances.  We can
see frequent emagent connections in the listener.log without
error.  However, on occasion it may have failed to
complete the conne
bfd4
ction process at the database so an ORA-609 is
thrown.  The emagent will simply retry the
connection and may be successful on the subsequent
try.  (Provided there is no real fault occurring
at the listener or database).  This temporary
failure to connect will not be reported back to DB Console and
there will be no indication, except for the ORA-609, that a fault
occurred.

 

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Solution

It can be somewhat challenging  to determine
the origin of the client that is causing the
error. 

For that reason, we often recommend increasing the values for
INBOUND_CONNECT_TIMEOUT at both listener and server side sqlnet.ora
file as a preventive measure.  If the
problem  is due to connection timeouts, an
increase in the following parameters should eliminate or reduce the
occurrence of the ORA-609s.

e.g.

Sqlnet.ora: SQLNET.INBOUND_CONNECT_TIMEOUT=180

Listener.ora:
INBOUND_CONNECT_TIMEOUT_listener_name=120

These settings are in seconds.  Again, the default
is 60.

If the issue persists and inbound connect does not have
any effect, the following steps are intended to help
locate  the client that may be causing the
errors.

1)  Suppress the TNS errors in
the alert.log by setting the following listener.ora file
parameter: 

DIAG_ADR_ENABLED_listener_name=OFF

This will cause the TNS errors to be posted to the
ORACLE_HOME/network/log/sqlnet.log file that is local to the
database and may yield useful information about the client's
address.

For example, here's a snippet from a server side sqlnet.log where
client address info was posted:

Production Time: 15-FEB-2010 07:15:01

Fatal NI connect error 12537, connecting to:

(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(Host=yourhost)(Port=1521))(CONNECT_DATA=(SID=PROD1DR)(CID=(PROGRAM=sqlplus)(HOST=client_host)(USER=client))))

Observe the PROGRAM and HOST fields on the last
line.  This is where the connection
originated.

Be sure to match timestamps in the sqlnet.log with the timestamps
of the alert.log errors.  Once you've located the
offending client, you can enable client tracing to try and
determine the cause:

TRACE_LEVEL_CLIENT=16

TRACE_DIRECTORY_CLIENT=<dir
location>

TRACE_TIMESTAMP_CLIENT=TRUE

DIAG_ADR_ENABLED=off  
<<<<<11g
or newer client requirement

If you need assistance with client or server tracing, please open
an SR with Global Customer Support.

2)  Check the listener.log for
client connections that were logged at timestamps that match the
ORA-609 timestamps as they appear in the
alert.log.  The client information is recorded in
each listener.log entry.  Since this error occurs
AFTER the listener has handled the connection, do not expect to see
errors in the listener.log.

Here's an example snippet of an incoming client connection that was
posted to the listener.log:

20-JAN-2009 17:08:45
(CONNECT_DATA=(SID=orcl)(CID=(PROGRAM=D:\oracle\product\10.1.0\Db_1\perl\5.6.1\bin\MSWin32-x86\perl.exe)(HOST=myclient)

Note that the exact timestamp, program name and client host will
often be recorded.  Again, once you've located the
offending client, enable tracing (see above) to try to capture the
connection failure.

3)  Enable server side Oracle Net
tracing and capture the TNS error along with the incoming
connection.

Match the PID that accompanies the ORA-609 to the server trace
label.  e.g. 

ORA-609 : opiodr aborting process unknown ospid
(4799_1)  *Note the PID

This PID would correspond to server trace
labeled:  svr_4799.trc.  Check
the server trace for either TNS error (the 609 will not appear) and
try to locate the originating client address.  If
assistance is needed for this investigation, please open an SR with
Oracle Support.

See below for instuctions on enabling Oracle Net server
tracing.

The
following details the discovery of the source of an ORA-609 for a
real case:

The alert.log reports the following messages intermittently but
frequently:

Mon
Nov 16 22:39:22 2009

ORA-609 : opiodr aborting process unknown ospid (nnnn)

Enabled Oracle Net server tracing:

TRACE_LEVEL_SERVER=16

TRACE_DIRECTORY_SERVER=<dir
location>

TRACE_TIMESTAMP_SERVER=TRUE

DIAG_ADR_ENABLED=off

Reloaded listener and wait for error to appear again.:

ORA-609 : opiodr aborting process unknown ospid
(5233_1)

Note that the server trace file set that corresponded to this event
was named svr_5233*.trc.

Of course the timestamps of the alert.log event and the server
trace creation matched as well.

A review of the server trace showed only an EOF failure and
the  TNS-12537 error:

Read
unexpected EOF ERROR

nserror: nsres: id=0, op=68, ns=12537

In this particular case, there was no information about the client
in the trace. This is atypical for a server trace.
  It may be that the client aborted before all the
client information was posted to the file. 
However, there was post in the listener.log for an emagent
connection that was established at the same point in time.

Here's an excerpt from a listener.log entry where an emagent
establishes a connection:

PROGRAM=D:\oracle\product\10.1.0\Db_1\bin\emagent.exe)

Checked the EM Agent traces and logs and discovered the following
entry:

Fatal
NI connect error 12547, connecting to:

(LOCAL=NO)

VERSION INFORMATION:

TNS for Solaris: Version 11.1.0.7.0 - Production

Oracle Bequeath NT Protocol Adapter for Solaris: Version 11.1.0.7.0
- Production

TCP/IP NT Protocol Adapter for Solaris: Version 11.1.0.7.0 -
Production

Time: 16-NOV-2009 22:39:22

****Tracing to file:
/backup/sid_traces/sqlnetlog/svr_5233.trc

Tns error struct:

ns main err code: 12547

TNS-12547: TNS:lost contact

ns secondary err code: 12560

nt main err code: 0

nt secondary err code: 0

nt OS err code: 0

****Note the name of the server trace which contains the
PID:  svr_5233.trc

Also, the timestamp of the agent event matches the timestamp of the
alert.log error.

 

Check the following locations for EM Agent
traces. If working with support on this issue and the EM Agent is
suspected, upload ALL files under:

$ORACLE_HOME/sysman/log/emagent.trc < Single node
agent trace location

$ORACLE_HOME/host/sysman/log/emagent.trc < RAC agent
trace location

It was determined that in this case, the emagent was aborting the
connection before it was complete and then simply reconnecting and
succeeding on the subsequent try.  No errors were
reported in the listener log or listener trace. No errors were
returned to the DB Console.  There was no apparent
outage of any kind.  No action
was taken to correct the ORA-609 in this case.  It
was decided that the message was informational and completely
benign.

Please review the following documents for more information about
timeouts and tracing:

Note 119706.1 Troubleshooting Guide TNS-12535 or ORA-12535 or
ORA-12170

Errors

Note 345197.1 Connections that Used to Work in Oracle 10.1
Now

Intermittently Fail with ORA-3113,ORA-3106 or ORA-3136 from

10.2 Onwards

Note 405755.1 Files Needed for Troubleshooting an EM 10G
Service Request

if an RDA is not Available

Note 395525.1 How to Enable Oracle SQLNet Client , Server ,
Listener ,

Kerberos and External procedure Tracing from Net Manager

Note 454927.1 Using and Disabling the Automatic Diagnostic
Repository

(ADR) with Oracle Net for 11g