This image has an empty alt attribute; its file name is image-304.png

when we perform switchover the new standby second instance doesn’t start automatically after switchover

our configuration

DGMGRL> show configuration

Configuration - broker_config

  Protection Mode: MaxPerformance
  Databases:
    TROY    - Primary database
    TROY_DR - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS

swichover

DGMGRL> switchover to 'TROY_DR'
Performing switchover NOW, please wait...
New primary database "TROY_DR" is opening...
Operation requires shutdown of instance "TROY1" on database "TROY"
Shutting down instance "TROY1"...
ORACLE instance shut down.
Operation requires startup of instance "TROY1" on database "TROY"
Starting instance "TROY1"...
ORACLE instance started.
Database mounted.
Database opened.
Switchover succeeded, new primary is "TROY_DR"

error message

DGMGRL>  show configuration

Configuration - broker_config

  Protection Mode: MaxPerformance
  Databases:
    TROY_DR - Primary database
    TROY    - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
ORA-16501: the Data Guard broker operation failed
ORA-16625: cannot reach database "TROY_DR"
DGM-17017: unable to determine configuration status

status of new standby

[root@ol6-112-rac1 ~]# /u01/app/11.2.0.3/grid/bin/crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
.......
.......
ora.troy.db
      1        ONLINE  ONLINE       ol6-112-rac1             Open,Readonly
      2        OFFLINE OFFLINE                               Instance Shutdown

immeditae fix

restart the new standby database

[oracle@ol6-112-rac1 ~]$ srvctl stop database -d TROY
srvctl start database -d TROY
[oracle@ol6-112-rac1 ~]$ srvctl start database -d TROY

now configuartion looks fine

[oracle@ol6-112-rac1 ~]$ dgmgrl
DGMGRL for Linux: Version 11.2.0.3.0 - 64bit Production

Copyright (c) 2000, 2009, Oracle. All rights reserved.

Welcome to DGMGRL, type "help" for information.
DGMGRL> connect sys/oracle@TROY
Connected.
DGMGRL> show configuration

Configuration - broker_config

  Protection Mode: MaxPerformance
  Databases:
    TROY_DR - Primary database
    TROY    - Physical standby database

Fast-Start Failover: DISABLED

Configuration Status:
SUCCESS

permanent fix

there could be multiple reasons for this error. apply one of these fix as per your case.

  • you have tnsnames.ora in your $ORACLE_HOME/network/admin but forget to keep the same in your $GRID_HOME/network/admin (my case)
  • you are using default port 1521 in local listener port or scan listener. So change it to non default port.
  • you are using scan in your tns connect identifier. you should instead use VIP. modifu your tns from scan to vip as mentioned below
TROY =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = ol6-112-scan)(PORT = 1525))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = TROY.localdomain)
      (UR=A)
    )
  )


TROY_DR =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = ol6-112-dr-scan)(PORT = 1525))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = TROY_DR.localdomain)
      (UR=A)
    )
  )


to ----->

TROY =
  (DESCRIPTION =
   (ADDRESS = (PROTOCOL = TCP)(HOST = ol6-112-rac1-vip)(PORT = 1525)) 
   (ADDRESS = (PROTOCOL = TCP)(HOST = ol6-112-rac2-vip)(PORT = 1525))
     (LOAD_BALANCE = yes)
   (CONNECT_DATA =
    (SERVICE_NAME = TROY.localdomain) 
    )
   )  

TROY_DR =
  (DESCRIPTION =
   (ADDRESS = (PROTOCOL = TCP)(HOST = ol6-112-dr-rac1-vip)(PORT = 1525)) 
   (ADDRESS = (PROTOCOL = TCP)(HOST = ol6-112-dr-rac2-vip)(PORT = 1525))
     (LOAD_BALANCE = yes)
   (CONNECT_DATA =
    (SERVICE_NAME = TROY_DR.localdomain) 
    )
   )
  • stabdby database is in active dataguard mode (read-only with apply). change it to mount mode before switchover
[oracle@ol6-112-dr-rac1 ~]$ srvctl stop database -d TROY_DR
[oracle@ol6-112-dr-rac1 ~]$ srvctl remove database -d TROY_DR
Remove the database TROY_DR? (y/[n]) y
[oracle@ol6-112-dr-rac1 ~]$ srvctl status database -d TROY_DR
PRCD-1120 : The resource for database TROY_DR could not be found.
PRCR-1001 : Resource ora.troy_dr.db does not exist
[oracle@ol6-112-dr-rac1 ~]$ srvctl add database -d TROY_DR -o /u01/app/oracle/product/11.2.0.3/db_1 -r physical_standby
[oracle@ol6-112-dr-rac1 ~]$ srvctl add instance -d TROY_DR -i TROY1 -n ol6-112-dr-rac1
[oracle@ol6-112-dr-rac1 ~]$ srvctl add instance -d TROY_DR -i TROY2 -n ol6-112-dr-rac2
[oracle@ol6-112-dr-rac1 ~]$  srvctl status database -d TROY_DR
Instance TROY1 is not running on node ol6-112-dr-rac1
Instance TROY2 is not running on node ol6-112-dr-rac2

[oracle@ol6-112-dr-rac1 ~]$ srvctl modify database -d TROY_DR -s mount

[oracle@ol6-112-dr-rac1 ~]$  srvctl start database -d TROY_DR
[oracle@ol6-112-dr-rac1 ~]$ srvctl status database -d TROY_DR
Instance TROY1 is running on node ol6-112-dr-rac1
Instance TROY2 is running on node ol6-112-dr-rac2
  • your databases are not properly added in cluster so remove and add them back using srvctl
[oracle@ol6-112-rac1 admin]$ srvctl stop database -d TROY
[oracle@ol6-112-rac1 admin]$ srvctl remove database -d TROY
Remove the database TROY? (y/[n]) y
[oracle@ol6-112-rac1 admin]$ srvctl status database -d TROY
PRCD-1120 : The resource for database TROY could not be found.
PRCR-1001 : Resource ora.troy.db does not exist
[oracle@ol6-112-rac1 admin]$ srvctl add database -d TROY -o /u01/app/oracle/product/11.2.0.3/db_1 -r primary
[oracle@ol6-112-rac1 admin]$ srvctl add instance -d TROY -i TROY1 -n ol6-112-rac1
[oracle@ol6-112-rac1 admin]$ srvctl add instance -d TROY -i TROY2 -n ol6-112-rac2
[oracle@ol6-112-rac1 admin]$ srvctl status database -d TROY
Instance TROY1 is not running on node ol6-112-rac1
Instance TROY2 is not running on node ol6-112-rac2
[oracle@ol6-112-rac1 admin]$ srvctl start database -d TROY

[oracle@ol6-112-dr-rac1 ~]$ srvctl stop database -d TROY_DR
[oracle@ol6-112-dr-rac1 ~]$ srvctl remove database -d TROY_DR
Remove the database TROY_DR? (y/[n]) y
[oracle@ol6-112-dr-rac1 ~]$ srvctl status database -d TROY_DR
PRCD-1120 : The resource for database TROY_DR could not be found.
PRCR-1001 : Resource ora.troy_dr.db does not exist
[oracle@ol6-112-dr-rac1 ~]$ srvctl add database -d TROY_DR -o /u01/app/oracle/product/11.2.0.3/db_1 -r physical_standby
[oracle@ol6-112-dr-rac1 ~]$ srvctl add instance -d TROY_DR -i TROY1 -n ol6-112-dr-rac1
[oracle@ol6-112-dr-rac1 ~]$ srvctl add instance -d TROY_DR -i TROY2 -n ol6-112-dr-rac2
[oracle@ol6-112-dr-rac1 ~]$  srvctl status database -d TROY_DR
Instance TROY1 is not running on node ol6-112-dr-rac1
Instance TROY2 is not running on node ol6-112-dr-rac2