-
Notifications
You must be signed in to change notification settings - Fork 60
Open
Description
My SIDB standby instance is stuck in Unhealthy even though the instance is running OK and Dataguard is running OK.
I tried /home/oracle/shutDown.sh immediate followed with /home/oracle/startUp.sh, worked as such but standby pod is never ready.
$ kubectl -n oracle-database get pods
NAME READY STATUS RESTARTS AGE
cnfdb-dgbroker-h881r 1/1 Running 0 22h
cnfdb11-czmxg 1/1 Running 1 (28h ago) 28h
cnfdb12-7e3mo 0/1 Running 0 9m43s
$ kubectl -n oracle-database get sidb
NAME EDITION STATUS ROLE VERSION CONNECT STR TCPS CONNECT STR OEM EXPRESS URL
cnfdb11 Enterprise Healthy PRIMARY 19.29.0.0.0 10.240.222.19:1521/NCNFDB11 Unavailable https://10.12.86.125:5500/em
cnfdb12 Enterprise Unhealthy PHYSICAL_STANDBY 19.29.0.0.0 10.12.86.125:1521/NCNFDB12 Unavailable https://10.12.86.125:5500/em
$ kubectl -n oracle-database get sidb cnfdb12 -o json | jq '.status'
{
"archiveLog": "true",
"clusterConnectString": "cnfdb12-ext.oracle-database:1521/NCNFDB12",
"conditions": [
{
"lastTransitionTime": "2026-01-15T09:45:27Z",
"message": "command terminated with exit code 1",
"observedGeneration": 1,
"reason": "LastReconcileCycleFailed",
"status": "True",
"type": "ReconcileError"
},
{
"lastTransitionTime": "2026-01-15T09:45:37Z",
"message": "no reconcile errors",
"observedGeneration": 1,
"reason": "LastReconcileCycleCompleted",
"status": "True",
"type": "ReconcileComplete"
},
{
"lastTransitionTime": "2026-01-27T13:17:21Z",
"message": "no reconcile errors",
"observedGeneration": 1,
"reason": "LastReconcileCycleQueued",
"status": "True",
"type": "ReconcileQueued"
}
],
"connectString": "10.12.86.125:1521/NCNFDB12",
"createdAs": "standby",
"datafilesCreated": "true",
"datafilesPatched": "false",
"dgBroker": "cnfdb-dgbroker",
"edition": "Enterprise",
"flashBack": "true",
"forceLog": "true",
"initParams": {
"cpuCount": 8,
"pgaAggregateTarget": 1599,
"processes": 640,
"sgaTarget": 4800
},
"isTcpsEnabled": false,
"oemExpressUrl": "https://10.12.86.125:5500/em",
"pdbConnectString": "10.12.86.125:1521/ORCLPDB1",
"pdbName": "ORCLPDB1",
"persistence": {
"accessMode": "ReadWriteOnce",
"size": "300Gi",
"storageClass": "csi-sc-cinderplugin"
},
"primaryDatabase": "cnfdb11",
"releaseUpdate": "19.29.0.0.0",
"replicas": 1,
"role": "PHYSICAL_STANDBY",
"sid": "NCNFDB12",
"status": "Unhealthy",
"tcpsConnectString": "Unavailable",
"tcpsPdbConnectString": "Unavailable",
"tcpsTlsSecret": ""
}
$ kubectl -n oracle-database get dgbroker
NAME PRIMARY STANDBYS PROTECTION MODE CONNECT STR STATUS FSFO
cnfdb-dgbroker NCNFDB11 NCNFDB12 MaxAvailability 10.12.86.110:1521/DATAGUARD Healthy true
$ kubectl -n oracle-database exec -it cnfdb11-czmxg -- /bin/bash
bash-5.1$ dgmgrl / as sysdba
DGMGRL for Linux: Release 19.0.0.0.0 - Production on Tue Jan 27 13:08:56 2026
Version 19.29.0.0.0
Copyright (c) 1982, 2019, Oracle and/or its affiliates. All rights reserved.
Welcome to DGMGRL, type "help" for information.
Connected to "NCNFDB11"
Connected as SYSDBA.
DGMGRL> show configuration
Configuration - dg_config
Protection Mode: MaxAvailability
Members:
ncnfdb11 - Primary database
ncnfdb12 - (*) Physical standby database
Fast-Start Failover: Enabled in Zero Data Loss Mode
Configuration Status:
SUCCESS (status updated 40 seconds ago)
The standby database seems however to be 100% healthy:
$ kubectl -n oracle-database exec -it cnfdb12-7e3mo -- /bin/bash
bash-5.1$ dgmgrl / as sysdba
SQL> SELECT RECOVERY_MODE FROM V$ARCHIVE_DEST_STATUS WHERE DEST_ID=1;
RECOVERY_MODE
----------------------------------
MANAGED REAL TIME APPLY
SQL> SELECT name, open_mode, database_role, switchover_status FROM v$database;
NAME OPEN_MODE DATABASE_ROLE SWITCHOVER_STATUS
--------- -------------------- ---------------- --------------------
NCNFDB11 READ ONLY WITH APPLY PHYSICAL STANDBY NOT ALLOWED
SQL> SELECT NAME, VALUE, DATUM_TIME FROM V$DATAGUARD_STATS WHERE NAME LIKE '%lag%';
NAME
--------------------------------
VALUE
----------------------------------------------------------------
DATUM_TIME
------------------------------
transport lag
+00 00:00:00
01/27/2026 13:06:03
apply lag
+00 00:00:00
01/27/2026 13:06:03
NAME
--------------------------------
VALUE
----------------------------------------------------------------
DATUM_TIME
------------------------------
SQL> SELECT PROCESS, STATUS, SEQUENCE# FROM V$MANAGED_STANDBY;
PROCESS STATUS SEQUENCE#
--------- ------------ ----------
ARCH CONNECTED 0
DGRD ALLOCATED 0
DGRD ALLOCATED 0
ARCH CLOSING 190
ARCH CLOSING 192
ARCH CLOSING 193
RFS IDLE 0
RFS IDLE 0
RFS IDLE 194
MRP0 APPLYING_LOG 194
RFS IDLE 0
11 rows selected.
SQL> SELECT THREAD#, LOW_SEQUENCE#, HIGH_SEQUENCE# FROM V$ARCHIVE_GAP;
no rows selected
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels