GRES functionality is not supported on qfx3008-I Interconnect devices. With or without GRES configuration on qfx3008-I interconnect devices, FPCs in the chassis will reboot during RE switchover. This article demonstrates that behavior and explains that this is expected.
Configuration:
user@router-1> show version Jul 27 14:31:39 Hostname: router-1 Model: qfx3008-i Junos: 14.1X53-D121.4 user@router-1> show version Jul 27 14:31:39 Hostname: router-1 Model: qfx3008-i Junos: 14.1X53-D121.4
Test results while GRES is configured.
Outputs prior to failover:
user@router-1> show chassis routing-engine Jul 27 14:03:44 Routing Engine status: Slot 0: Current state Master Election priority Master (default) Temperature 64 degrees C / 147 degrees F DRAM 3312 MB Installed Memory 4096 MB Memory utilization 86 percent CPU utilization: User 18 percent Background 0 percent Kernel 3 percent Interrupt 0 percent Idle 78 percent Model RE-QFXC08-CB4S Serial ID BUILTIN Start time 2017-07-26 15:45:29 UTC Uptime 22 hours, 18 minutes, 10 seconds Last reboot reason 0x200:normal shutdown Load averages: 1 minute 5 minute 15 minute 8.01 8.15 8.05 Routing Engine status: Slot 1: Current state Backup Election priority Backup (default) Temperature 44 degrees C / 111 degrees F DRAM 3312 MB Installed Memory 4096 MB Memory utilization 71 percent CPU utilization: User 3 percent Background 0 percent Kernel 2 percent Interrupt 0 percent Idle 95 percent Model RE-QFXC08-CB4S Serial ID BUILTIN Start time 2017-07-26 15:40:45 UTC Uptime 22 hours, 22 minutes, 43 seconds Last reboot reason 0x400:bios auto recovery reset Load averages: 1 minute 5 minute 15 minute 0.48 0.22 0.09
Please note that all the FPCs are 'online' for 22 hours.
user@router-1> show chassis fpc detail Jul 27 14:06:06 Slot 0 information: State Online Temperature 36 degrees C / 96 degrees F Start time 2017-07-26 16:04:12 UTC Uptime 22 hours, 1 minute, 49 seconds Slot 8 information: State Online Temperature 35 degrees C / 95 degrees F Start time 2017-07-26 16:04:18 UTC Uptime 22 hours, 1 minute, 43 seconds Slot 9 information: State Online Temperature 34 degrees C / 93 degrees F Start time 2017-07-26 16:04:25 UTC Uptime 22 hours, 1 minute, 37 seconds Slot 10 information: State Online Temperature 34 degrees C / 93 degrees F Start time 2017-07-26 16:04:32 UTC Uptime 22 hours, 1 minute, 30 seconds Slot 11 information: State Online Temperature 34 degrees C / 93 degrees F Start time 2017-07-26 16:04:38 UTC Uptime 22 hours, 1 minute, 23 seconds Slot 12 information: State Online Temperature 34 degrees C / 93 degrees F Start time 2017-07-26 16:04:45 UTC Uptime 22 hours, 1 minute, 17 seconds Slot 13 information: State Online Temperature 34 degrees C / 93 degrees F Start time 2017-07-26 16:04:51 UTC Uptime 22 hours, 1 minute, 11 seconds Slot 14 information: State Online Temperature 34 degrees C / 93 degrees F Start time 2017-07-26 16:04:57 UTC Uptime 22 hours, 1 minute, 4 seconds Slot 15 information: State Online Temperature 36 degrees C / 96 degrees F Start time 2017-07-26 16:05:04 UTC Uptime 22 hours, 57 seconds user@router-1-backup> show chassis fpc Jul 27 14:07:05 Aborted! This command can only be used on the master routing engine. user@router-1-backup> show system switchover Jul 27 14:07:25 Graceful switchover: Off <---- GRES is not enabled Peer state: Steady State
user@router-1# set chassis redundancy graceful-switchover Jul 27 14:07:54
Output below confirms that GRES is enabled:
user@router-1-backup> show system switchover Jul 27 14:08:56 Graceful switchover: On <---- GRACEFUL SWITCHOVER ON Configuration database: Ready Kernel database: Ready Peer state: Steady State
Failover performed. Backup RE becomes the primary RE.
user@router-1-backup> show chassis routing-engine <---- Slot-1 becomes the primary Jul 27 14:10:37 Routing Engine status: Slot 1: Current state Master Election priority Backup (default) Temperature 46 degrees C / 114 degrees F DRAM 3312 MB Installed Memory 4096 MB Memory utilization 71 percent CPU utilization: User 0 percent Background 0 percent Kernel 1 percent Interrupt 0 percent Idle 98 percent Model RE-QFXC08-CB4S Serial ID BUILTIN Start time 2017-07-26 15:40:45 UTC Uptime 22 hours, 29 minutes, 43 seconds Last reboot reason 0x400:bios auto recovery reset Load averages: 1 minute 5 minute 15 minute 0.09 0.14 0.08
FPC went offline for reboot.
user@router-1> show chassis fpc detail Jul 27 14:12:06 Slot 0 information: State Offline Reason Offlined due to major errors Slot 8 information: State Offline Reason Offlined due to major errors Slot 9 information: State Offline Reason Offlined due to major errors Slot 10 information: State Offline Reason Offlined due to major errors Slot 11 information: State Offline Reason Offlined due to major errors Slot 12 information: State Offline Reason Offlined due to major errors Slot 13 information: State Offline Reason Offlined due to major errors Slot 14 information: State Offline Reason Offlined due to major errors Slot 15 information: State Offline Reason Offlined due to major errors
FPCs came back online after 6 minutes 15 sec
user@router-1> show chassis fpc detail Jul 27 14:18:21 Slot 0 information: State Online Temperature 31 degrees C / 87 degrees F Start time 2017-07-27 14:17:32 UTC Uptime 0 seconds Slot 8 information: State Online Temperature 32 degrees C / 89 degrees F Start time 2017-07-27 14:17:38 UTC Uptime 0 seconds Slot 9 information: State Online Temperature 32 degrees C / 89 degrees F Start time 2017-07-27 14:17:44 UTC Uptime 0 seconds Slot 10 information: State Online Temperature 32 degrees C / 89 degrees F Start time 2017-07-27 14:17:51 UTC Uptime 0 seconds Slot 11 information: State Online Temperature 32 degrees C / 89 degrees F Start time 2017-07-27 14:17:57 UTC Uptime 0 seconds Slot 12 information: State Online Temperature 31 degrees C / 87 degrees F Start time 2017-07-27 14:18:03 UTC Uptime 0 seconds Slot 13 information: State Online Temperature 31 degrees C / 87 degrees F Start time 2017-07-27 14:18:10 UTC Uptime 0 seconds Slot 14 information: State Online Temperature 31 degrees C / 87 degrees F Start time 2017-07-27 14:18:17 UTC Uptime 0 seconds Slot 15 information: State Online Temperature 32 degrees C / 89 degrees F Start time 2017-07-27 14:18:23 UTC Uptime 0 seconds
GRES is not supported on qfx3008-I Interconnect device.
This is tested on code "14.1X53-D121.4". On this code, it accepts the configuration and the show output displays that GRES is enabled and kernel state is ready. However, once RE switchover happens or if RE fails over, it is expected that all the FPCs in the chassis will be rebooted.
This is an expected behavior in this chassis.
The qfx3008-i interconnect devices are implemented or used in the Qfabric system, which has redundant interconnects devices connecting to the node-group devices via fte-links. During Interconnect reboot or FPC restart, it is expected that the traffic will switch over to another Interconnect device with a possibility of losing a few packets. It is important to note that if all the other redundant interconnect devices are up and fte-links from the node-groups are UP, there will be no outage.
2021-04-07: Updated the article terminology to align with Juniper's Inclusion & Diversity initiatives