This article discusses how Flexible PIC Concentrators (FPCs) reconnect when a graceful Routing Engine (RE) switchover is performed on an M120 Series device.
In a dual RE scenario when RE switchover occurs either gracefully or ungracefully, the following may be observed:
Consider an M120 device with dual REs and empty FPC slots:
lab@M120-R1# run show chassis routing-engine | no-more Oct 19 10:59:59 Routing Engine status: Slot 0: Current state Backup Election priority Master (default) Temperature 44 degrees C / 111 degrees F CPU temperature 45 degrees C / 113 degrees F DRAM 2048 MB (2048 MB installed) Memory utilization 24 percent CPU utilization: User 0 percent Background 0 percent Kernel 1 percent Interrupt 0 percent Idle 99 percent Model RE-A-2000 Serial ID 1000737665 Start time 2018-10-15 10:49:34 IST Uptime 4 days, 10 minutes, 7 seconds Last reboot reason 0x11:power cycle/failure misc hardware reason Load averages: 1 minute 5 minute 15 minute 0.00 0.00 0.00 Routing Engine status: Slot 1: Current state Master Election priority Backup (default) Temperature 44 degrees C / 111 degrees F CPU temperature 48 degrees C / 118 degrees F DRAM 2048 MB (2048 MB installed) Memory utilization 29 percent CPU utilization: User 0 percent Background 0 percent Kernel 7 percent Interrupt 0 percent Idle 93 percent Model RE-A-2000 Serial ID 9012002390 Start time 2018-10-15 10:37:36 IST Uptime 4 days, 22 minutes, 20 seconds Last reboot reason Router rebooted after a normal shutdown. Load averages: 1 minute 5 minute 15 minute 0.11 0.09 0.04 [MASTER] lab@M120-R1> show chassis fpc Oct 19 11:02:05 Temp CPU Utilization (%) Memory Utilization (%) Slot State (C) Total Interrupt DRAM (MB) Heap Buffer 0 Online 50 0 0 128 7 51 1 Online 43 0 0 128 7 51 2 Online 35 0 0 128 9 51 3 Empty 4 Online 30 0 0 128 11 51 5 Empty
When RE switchover occurs, the following may be observed:
[MASTER] lab@M120-R1> request chassis routing-engine master switch Oct 19 11:03:20 Toggle mastership between routing engines ? [yes,no] (no) yes Oct 19 11:03:26 Resolving mastership... Complete. The other routing engine becomes the master. [BACKUP] lab@M120-R1> Oct 19 11:03:48 CHASSISD_FRU_OFFLINE_NOTICE: Taking FPC 3 offline: FRU did not reconnect Oct 19 11:03:48 fpc_down slot 3 reason Removal cargs 0x0 Oct 19 11:03:48 fpc_offline_now - slot 3, reason: FRU did not reconnect, error OK transition state 1 Oct 19 11:03:48 send: red alarm set, device FPC 4, reason FEB not online for FPC 4 Oct 19 11:03:48 CHASSISD_FRU_OFFLINE_NOTICE: Taking FPC 5 offline: FRU did not reconnect Oct 19 11:03:48 fpc_down slot 5 reason Removal cargs 0x0 Oct 19 11:03:48 fpc_offline_now - slot 5, reason: FRU did not reconnect, error OK transition state 1
When RE switchover occurs with GRES enabled, chassisd attempts to do a soft reconnect on all the slots of the chassis regardless of whether or not an FPC is present. The above messages indicate that due to slot 3 and slot 5 being empty, there was no reconnect. Hence, the respective FPC slots were taken offline.
The slots that have FPCs will be able to successfully establish a soft reconnect and will operate normally without any impact as observed below:
Oct 19 11:03:26 FPC 0 added Oct 19 11:03:26 reading FPC 0 initial state Oct 19 11:03:26 check_and_carp_on_i2cs_version I2CS version=0xf Oct 19 11:03:27 Done reading FPC 0 initial state Oct 19 11:03:27 FPC 1 added Oct 19 11:03:27 reading FPC 1 initial state Oct 19 11:03:27 check_and_carp_on_i2cs_version I2CS version=0xf Oct 19 11:03:28 Done reading FPC 1 initial state Oct 19 11:03:28 FPC 2 added Oct 19 11:03:28 reading FPC 2 initial state Oct 19 11:03:28 check_and_carp_on_i2cs_version I2CS version=0xf Oct 19 11:03:29 Done reading FPC 2 initial state Oct 19 11:03:29 FPC 4 added Oct 19 11:03:29 reading FPC 4 initial state Oct 19 11:03:30 check_and_carp_on_i2cs_version I2CS version=0xf Oct 19 11:03:30 Done reading FPC 4 initial state
It can also be noted from the uptime of the FPCs that the FPCs did not restart and only went through a soft restart with the chassisd process of the current primary RE.
lab@M120-R1> show chassis fpc detail Oct 19 11:38:07 Slot 0 information: State Online Temperature 50 degrees C / 122 degrees F Total CPU DRAM 128 MB Start time 2018-10-15 10:10:41 IST Uptime 4 days, 1 hour, 27 minutes, 26 seconds Slot 1 information: State Online Temperature 43 degrees C / 109 degrees F Total CPU DRAM 128 MB Start time 2018-10-15 10:10:43 IST Uptime 4 days, 1 hour, 27 minutes, 24 seconds Slot 2 information: State Online Temperature 35 degrees C / 95 degrees F Total CPU DRAM 128 MB Start time 2018-10-15 10:10:46 IST Uptime 4 days, 1 hour, 27 minutes, 21 seconds Slot 4 information: State Online Temperature 31 degrees C / 87 degrees F Total CPU DRAM 128 MB Start time 2018-10-15 10:10:54 IST Uptime 4 days, 1 hour, 27 minutes, 13 seconds
2021-04-05: Updated the article terminology to align with Juniper's Inclusion & Diversity initiatives