生产系统RMAN备份失败ORA-00245

一、环境描述
AIX 6.1
Oracle 11.2.0.3 三节点RAC

二、问题描述

1.全库备份出错
Starting Control File and SPFILE Autobackup at 04-NOV-2017 10:54:04
released channel: d1
released channel: d2
released channel: d3
released channel: d4
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of Control File and SPFILE Autobackup command on d1 channel at 11/04/2017 10:54:05
ORA-00245: control file backup failed; target is likely on a local file system

2.RMAN参数配置
RMAN> show all;

RMAN configuration parameters for database with db_unique_name ORCL are:
CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 10 DAYS;
CONFIGURE BACKUP OPTIMIZATION ON;
CONFIGURE DEFAULT DEVICE TYPE TO DISK; # default
CONFIGURE CONTROLFILE AUTOBACKUP ON;
CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO '%F'; # default
CONFIGURE DEVICE TYPE DISK PARALLELISM 1 BACKUP TYPE TO BACKUPSET; # default
CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE MAXSETSIZE TO UNLIMITED; # default
CONFIGURE ENCRYPTION FOR DATABASE OFF; # default
CONFIGURE ENCRYPTION ALGORITHM 'AES128'; # default
CONFIGURE COMPRESSION ALGORITHM 'BASIC' AS OF RELEASE 'DEFAULT' OPTIMIZE FOR LOAD TRUE ; # default
CONFIGURE ARCHIVELOG DELETION POLICY TO NONE; # default
CONFIGURE SNAPSHOT CONTROLFILE NAME TO '/u01/app/oracle/product/11.2.0/dbhome_1/dbs/snapcf_orcl1.f'; # default

3.解决方法

RMAN> CONFIGURE SNAPSHOT CONTROLFILE NAME TO '+datadg/snapcf_orcl.f';

new RMAN configuration parameters:
CONFIGURE SNAPSHOT CONTROLFILE NAME TO '+datadg/snapcf_orcl.f';
new RMAN configuration parameters are successfully stored

三、官方说明
APPLIES TO:

Oracle Database - Enterprise Edition - Version 11.2.0.1.0 and later
Information in this document applies to any platform.
This issue is only applicable to RAC database.

From 11gR2 onwards, the controlfile backup happens without holding the controlfile enqueue. For non-RAC database, this doesn't change anything. But for RAC database, due to the changes made to the controlfile backup mechanism in 11gR2, any instance in the cluster may write to the snapshot controlfile. Due to this snapshot controlfile need to be visible to all instances.

The snapshot controlfile MUST be accessible by all nodes of a RAC database, if the snapshot controlfile does not reside on a shared device error will be raised at the time of RMAN backup while taking snapshot of controlfile.

This applies to backing up controlfile using sqlplus / having autobackup of controlfile configured on non
shared location.

ORA-245 error message description
----------------------------------------
00245, 00000, "control file backup operation failed"
*Cause: Failed to create a control file backup because some process
signaled an error during backup creation.
*Action: Check alert files for further information. This usually happens
because some process could not access the backup file during
backup creation. Any process of any instance that starts a
read/write control file transaction must have an access to the
backup control file during backup creation.
DESCRIPTION

1. In RAC environment controlfile autobackup fails with ora-0245

Autobackup of controlfile in RMAN is failing with error:
RMAN-571: ===========================================================
RMAN-569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-571: ===========================================================
RMAN-3009: failure of Control File and SPFILE Autobackup command on
ORA_DISK_1 channel at 10/27/2010 12:13:31
ORA-245: control file backup operation failed
2. In RAC environment, backup controlfile to non shared location fails

SQL> ALTER DATABASE BACKUP CONTROLFILE TO '/home/rac1122/test/control.bk' REUSE
*
ERROR at line 1:
ORA-245: control file backup operation failed
3. In RAC environment backing up standby controlfile to non shared location fails

SQL> alter database create standby controlfile as '/home/oracle/renostdbycntrl.ctl';

alter database create standby controlfile as
'/home/oracle/renostdbycntrl.ctl'
*
ERROR at line 1:
ORA-245: control file backup operation failed

4. In RAC environment copy current controlfile to '${DB_BACKUP_DIR}/rac_tnctv_control.bak';

channel ch1: starting datafile copy
copying current control file
RMAN-571: ===========================================================
RMAN-569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-571: ===========================================================
RMAN-3009: failure of backup command on 10/07/2011 11:36:42 channel at ch1
ORA-245: control file backup operation failed
5. In RAC environment, Rman backup fails if snapshot controlfile is not in shared location.

RMAN-00571: ========================================================
RMAN-00569: ============ ERROR MESSAGE STACK FOLLOWS =============
RMAN-00571: =========================================================
RMAN-03009: failure of resync command on default channel at 03/13/2012 10:19:41
ORA-00245: control file backup operation failed
6. In RAC environment, with NO RMAN activity, but controlfile autobackup is being created by database structure change. I.e., adding datafiles or a resize of datafiles with autoextend. Alert.log shows:

2016-12-30 23:46:10.244000 +00:00
Errors in file /ora01/app/oracle/diag/rdbms/persprop/persprop5/trace/persprop5_m000_24118.trc:
ORA-19624: operation failed, retry possible
ORA-19504: failed to create file "/orabck/uhilsfmt/persprop1/rman/c-3364950485-20161230-00.cf"
ORA-27040: file create error, unable to create file
SVR4 Error: 2: No such file or directory
Additional information: 1

OCCURRENCE

Only affect Real application Cluster (RAC), 11.2 onward.

SYMPTOMS

In RAC environment any form of controlfile backup may fail with ORA-0245 if the location of the Snapshot Controlfile is not a shared location. The backup of the controlfile actualy makes a backup of the SNAPSHOT controlfile. The Snapshot controlfile is created when the controlfile is about to be backed up. The Snapshot controlfile is a read-consistent copy of the controlfile.

NOTE: There is another issue in which this error occurs when the backup location is not shared, see RMAN BACKUP TO LOCAL DEVICE WITH SNAPSHOT ON SHARED FAILS ORA-00245 (Doc ID 1516654.1)

According to bug 18073805 (closed as not a bug), ORA-00227 can also be raised from "alter database backup controlfile to <file>" when the location of the file is not shared.

WORKAROUND

SOLUTION:

This is a RAC specific 'configuration' issue and the correct configuration is as described below

It is changed behaviour which requires that the snapshot controlfile in a RAC environment, is on a shared location.

1. Check the snapshot controlfile location:

RMAN> show snapshot controlfile name;

2. Configure the snapshot controlfile to a shared disk:

RMAN> CONFIGURE SNAPSHOT CONTROLFILE NAME TO '<shared_disk>/snapcf_<DBNAME>.f';
Or in case of ASM use

RMAN> CONFIGURE SNAPSHOT CONTROLFILE NAME TO '+<DiskGroup>/snapcf_<DBNAME>.f';
PATCHES

NA