Description

Graceful Routing Engine Switchover (GRES) is not enabled by default, and it is strongly recommended to be enabled on a EX Virtual Chassis (VC).  This article includes a FAQ on the why, what, and how of enabling GRES on a VC.

Symptoms

  • Enabling Graceful Routing Engine Switchover to minimize traffic disruption when Primary Routing Engine fails over to the Backup.

Solution

 

It is strongly recommended to configure GRES on a Virtual Chassis. It is not enabled by default.
 

Frequently Asked Questions (FAQ):

Below is a list of FAQ related to enabling GRES on your EX device. 
Click on the question to jump to the answer.

Q1.   What is GRES?
Q2.   Why should GRES be enabled?
Q3.   How do you configure Graceful Routing Engine Switchover (GRES)?
Q4.   What is the command to confirm GRES is enabled?
Q5.   What does the message, 'Standby Routing Engine is not ready for graceful switchover. Packet Forwarding Engines that are not ready for graceful switchover might be reset' mean?
Q6.   What is the dead interval in terms of GRES?
Q7.   What is the minimum Successive Routing Engine switchover events?
Q8.   What logs record the RE switchover process?
Q9.   What are the limitations of GRES?
Q10. How do you verify that GRES is working in the EX4200 or EX4500 Virtual Chassis?
 


_____________________________________________________________________________________

Q1.  What is GRES?
A1.  The following links in the Technical Documentation explain the concepts and behavior in detail:



Q2.  Why should GRES be enabled?
A2. The following paragraph from the GRES Description explains why GRES should be enabled:
When the backup Routing Engine assumes primary role in a redundant failover configuration (that is, when graceful Routing Engine switchover is not enabled), the Packet Forwarding Engines initialize their state to the boot state before they connect to the new primary Routing Engine. In contrast, in a graceful switchover configuration, the Packet Forwarding Engines do not reinitialize their state, but resynchronize their state to that of the new primary Routing Engine. The interruption to traffic is minimal.
The fact is that when GRES is enabled, both primary and backup are in SYNC with their kernel tables and PFE tables.  So that when failover happens, the backup can immediately start sending traffic instead of relearning all info. Relearning takes time and during this time there will be impact to traffic.

In other words, there is a time delay on RE switchover without GRES.  By default without any configuration the back-up RE is just waiting around to become Primary in case of an outage. The back-up RE does not run all process and does not pull information from the PFE, so information such as PFE interface & chassis components are not available on the back-up RE. Refer to the following output:
Back-up RE’s Interfaces details when GRES is not enabled 
{backup:1}[edit]
root@router-VC# run show interfaces terse | no-more                                   
Interface                     	Admin Link Proto    Local        Remote
vcp-0                           up    up  
vcp-0.32768                	up    up  
vcp-1                           up    up  
vcp-1.32768                	up    up  
bme0                           	up    up  
bme0.32768                	up    up   	    inet        128.0.0.17/2    
                                            			128.0.0.32/2    
                                    		    tnp      	0x11            
bme0.32770              	down  up   	    eth-switch
dsc                     	up    up  
gre                     	up    up  
ipip                    	up    up  
lo0                     	up    up  
lsi                    		up    up  
me0                     	up    up  
me0.0                   	up    up   	    eth-switch
mtun                    	up    up  
pimd                    	up    up  
pime                    	up    up  
tap                     	up    up
In a failure scenario, the new RE would have to connect to the PFE, causing the set & flush of all memory tables that are used for forwarding, which causes the fail-over time to be more than desired because the RE will have to connect to the PFE. This time increases if the RE has to restart any protocols such as OSPF and RSTP. Note that GRES by itself does not preserve routing protocol information across switchovers, the routing protocol adjacencies would go down and would have to be re-established on the new Primary Routing Engine once it is available. To preserve routing protocol states, GRES would have to be combined with a Graceful Restart mechanism for the routing protocol, if one is available. RFC3623 specifies a Graceful Restart mechanism for OSPF, which is supported on EX Series Switches.

When RSTP/VSTP is restarted on the new primary, it will go through the same RSTP states that initially block traffic for about three seconds. In the future, once the NSB/NSR feature is supported on the EX4200 Ethernet Switch, convergence time for a Virtual Chassis RE power outage for RSTP will be dramatically reduced. 
 


Q3. How do you configure Graceful Routing Engine Switchover (GRES)?

A3.   Enable GRES:

root@router-VC# set chassis redundancy graceful-switchover

Disable GRES:
root@router-VC# deactivate chassis redundancy graceful-switchover   
OR
root@router-VC# delete chassis redundancy graceful-switchover
 
 Additional reference: Configuring Graceful Routing Engine Switchover



Q4. What is the command to confirm GRES is enabled?
A4. Run the configuration command: show chassis .  In the following output, GRES is enabled:
root@router-VC#show chassis
redundancy {
    graceful-switchover;
}


Q5.  What does the message, 'Standby Routing Engine is not ready for graceful switchover. Packet Forwarding Engines that are not ready for graceful switchover might be reset' mean?
For example, when you run the RE failover command, you get the following error:
root@router-VC# run request chassis routing-engine master switch
Toggle mastership between routing engines ? [yes,no] (no) yes

Command aborted. Not ready for mastership switch, try after 230 secs.
A5.  As indicated in the note below from the Graceful Routing Engine Switchover Concepts , do not attempt switchover.  We recommend that you wait 240 seconds, and then reissue the command.
alt



Q6. What is the dead interval in terms of GRES ?  
A6. 2 Seconds in Juniper EX family (MX 4 seconds).   If the Backup Routing Engine does not hear a reply to the keepalives from the Primary in 2 seconds, it will take over the primary role. Even though this interval is configurable via the set chassis redundancy keepalive-time command when GRES is not configured, it is set to 2 seconds when GRES is configured and cannot be changed.



Q7. What is the minimum Successive Routing Engine switchover events?   
A7. Successive Routing Engine switchover events must be a minimum of 240 seconds (4 minutes) apart after both Routing Engines have come up.



Q8. What logs record the RE switchover process ?
A8.  The following log messages are automatically captured for you to review RE switchover activity.
root@router-VC# run show log messages

Aug 23 05:29:52 router-VC mgd[4496]: UI_MASTERSHIP_EVENT: Toggle routing engine mastership by 'root'
Aug 23 05:29:52 router-VC vccp[741]: auto_cnf_mastership_changed: new mastership = 1
Aug 23 05:29:53 router-VC chas[739]: cm_ipc_chasd_cleanup: CMCD: closed chassisd connection, try again
Aug 23 05:29:53 router-VC chas[739]: cm_pic_cleanup FPC 1 PIC 1 NULL
Aug 23 05:29:53 router-VC chas[739]: CM_CHANGE : state 0->1, slotid 1->1 mode 2->1 1M 0B gid_changed 0 master_changed 1 master_detected 0 members_changed 1 new_ineligible_member = 0
Aug 23 05:29:53 router-VC chas[739]: CM_CHANGE: 0B 1M 2L
Aug 23 05:29:53 router-VC chas[739]: CM_CHANGE: Mastership 0->1
Aug 23 05:29:53 router-VC /kernel: mastership: routing engine 1 becoming master
Aug 23 05:29:53 mcontrol_acquire_mastership Calling reconnect
Aug 23 05:29:53 CHASSISD_RELEASE_MASTERSHIP: Release mastership notification

Hint:  Look for a match on a string to find the timestamp that a switchover occurred. 
root@router-VC# run show log messages | grep mastership
 


Q9.   What are the limitations of GRES?
 
  • Limitation of GRES over Dot.1X -- We do not support GRES for dot1x yet. As a result, when GRES happens, we lose all authentication info (including the dynamic VLAN). After the switchover, the supplicant is expected to authenticate again. Upon successful re-authentication, the supplicant is expected to move to the correct VLAN.

 



Q10.  How do you verify that GRES is working in the EX4200 or EX4500 Virtual Chassis ?
A10.  Refer to the following: Verifying That Graceful Routing Engine Switchover Is Working

 

Modification History

2021-03-25: Updated the article terminology to align with Juniper's Inclusion & Diversity initiatives.
2021-12-09: Archive since it is already covered extensively in https://www.juniper.net/documentation/us/en/software/junos/high-availability/topics/concept/gres-overview.html