Description

This article describes the "Backup RE Active" chassis alarm on MX/PTX series devices and ways to Troubleshoot the alarm and mitigate it.

Symptoms

The following alarm will be seen on the device when mastership switched over from master to backup, Please check solution part of this KB for possible triggers.
 

user@router> show chassis alarms no-forwarding
1 alarms currently active
Alarm time              Class Description
2023-08-08 19:14:07 PDT Minor Backup RE Active

 

user@router> show chassis routing-engine no-forwarding

Routing Engine status:

 Slot 0:
Current state                 Backup
Election priority             Master

 
Routing Engine status:

 Slot 1:
Current state                 Master
Election priority             Backup

   
Configuration:

chassis {
    redundancy {
        routing-engine 0 master;
        routing-engine 1 backup;
        failover {
            on-loss-of-keepalives;
            on-disk-failure;
        }
        graceful-switchover;
    }

 

Config below will trigger mastership switchover once rpd of master routing engine crashes.

user@router> show configuration system processes routing | display inheritance 
failover other-routing-engine;


Logs:

Jun 23 00:00:25 device chassisd[9044]: CHASSISD_LOST_MASTERSHIP: Routing Engine lost mastership; exiting
Jun 23 00:00:43 device chassisd[35190]: CHASSISD_SNMP_TRAP7: SNMP trap generated: Fru Online (jnxFruContentsIndex 9, jnxFruL1Index 2, jnxFruL2Index 0, jnxFruL3Index 0, jnxFruName Routing Engine 1, jnxFruType 6, jnxFruSlot 1)

Jun 23 00:00:43 device snmpd[35192]: LIBJSNMP_NS_LOG_INFO: INFO: send_trap: RE currently not a global-master, discarding trap

Jun 23 00:00:43 device chassisd[35190]: CHASSISD_SNMP_TRAP3: ENTITY trap generated: entStateOperEnabled (entPhysicalIndex 143, entStateAdmin 4, entStateAlarm 0)

Jun 23 00:00:27 device kernel: mastership: sent other RE mastership loss signal

Jun 23 00:00:28 device mib2d[35194]: MIB2D_MASTERSHIP_UPDATE: Mastership switched new Chassis State MIB2D_SC, new RE State MCONTROL_STATE_INIT

Jun 23 00:00:28 device mib2d[35194]: MIB2D_MASTERSHIP_UPDATE: Mastership switched new Chassis State MIB2D_SC, new RE State MCONTROL_STATE_BACKUP

Oct 14 08:14:42 device alarmd[25330]: Alarm set: RE id=83886441, color=YELLOW, class=CHASSIS, reason=Backup RE Active
Oct 14 08:14:42 device craftd[11042]: Minor alarm set, Backup RE Active

Solution

The primary Routing Engine may fail and switch over to the backup Routing Engine (RE) due to following reasons, Please upload info requested by JTAC to figure out the root cause and run health check first, then follow suggestions from JTAC to move forward.
 

  • File system corruption in the RE, which results in the above alarm.
  • rpd(routing protocol daemon) crashes on master routing engine.
  • When there is internal hardware error
  • Hard disk failure, or hardware failure
  • When the primary RE is removed or reset and the backup RE takes over.

 

Please upload var/log and RSI(output of "request support info") of both REs(Routing Engine) and rpd coredump if rpd crashes to case.


When we see this alarm, it means that the state of the routing-engine is different with that of the configuration. We will find that RE0 is set as the primary and RE1 is set as the backup. It is different with the current state.We can see from the configuration that RE0 priority is set as master where as RE1 priority is set as back-up.


To clear the alarm,We have 2 solutions:[IN scheduled MW]
Please contact Juniper JTAC and figure out the root cause and run health check of both REs(Routing Engine) first before taking any actions.


1.If you require RE1 to be master and RE0 as back-up,
Then follow the below configuration:

set chassis redundancy routing-engine 0 backup
set chassis redundancy routing-engine 1 master
commit synchronize


2. we can proceed to switch the mastership of the device in a scheduled MW[May impact the traffic] .

Please check if mastership switchover is ready or not on both routing engine first.

 

show task replication
show nonstop-routing
show database-replication summary
show bgp replication
request chassis routing-engine master switch check 

show system switchover <<< This cmd is only available on backup RE(routing engine)

 

If everything of above output looks in-sync and healthy, then:


request chassis routing-engine master switch
 

Please perform the any 1 of the above steps mentioned in scheduled MW and this should clear the alarm.
 

To clear this alarm with the change of mastership, reference:
https://www.juniper.net/documentation/us/en/software/junos/high-availability/topics/ref/command/request-chassis-routing-engine-master.html

 

Note: that this must be done under a MW to avoid any problems that may arise.

Modification History

Initia Draft