This article explains why some of the fabric planes on MX Series routers go offline after switchover and go online again, and clarifies whether any action must be taken.
Note: This article applies to single chassis and virtual chassis MX Series routers.
Customers may observe the following alarm on the new primary after a switchover:
labroot@jtac-mx480-r2016-re1> show chassis alarms | refresh 1 ---(refreshed at 2020-07-13 12:43:18 CST)--- 4 alarms currently active Alarm time Class Description 2020-07-13 12:43:18 CST Minor Loss of communication with Backup RE 2020-07-13 12:43:17 CST Minor CB 1 Fabric Chip 1 Not Online 2020-07-13 12:43:17 CST Minor CB 1 Fabric Chip 0 Not Online 2020-07-13 12:42:46 CST Minor Backup RE Active ---(refreshed at 2020-07-13 12:43:19 CST)--- 4 alarms currently active Alarm time Class Description 2020-07-13 12:43:18 CST Minor Loss of communication with Backup RE 2020-07-13 12:43:17 CST Minor CB 1 Fabric Chip 1 Not Online 2020-07-13 12:43:17 CST Minor CB 1 Fabric Chip 0 Not Online 2020-07-13 12:42:46 CST Minor Backup RE Active ---(refreshed at 2020-07-13 12:43:20 CST)--- 3 alarms currently active Alarm time Class Description 2020-07-13 12:43:18 CST Minor Loss of communication with Backup RE 2020-07-13 12:43:17 CST Minor CB 1 Fabric Chip 1 Not Online 2020-07-13 12:42:46 CST Minor Backup RE Active ---(refreshed at 2020-07-13 12:43:21 CST)--- 2 alarms currently active Alarm time Class Description 2020-07-13 12:43:18 CST Minor Loss of communication with Backup RE 2020-07-13 12:42:46 CST Minor Backup RE Active
Before switchover, all the fabric planes on CB1 are in Spare state. This is normal if we keep the fabric in redundancy mode.
labroot@jtac-mx480-r2016-re0> show chassis fabric summary Plane State Uptime 0 Online 3 days, 13 hours, 50 minutes, 31 seconds 1 Online 3 days, 13 hours, 50 minutes, 31 seconds 2 Online 3 days, 13 hours, 50 minutes, 31 seconds 3 Online 3 days, 13 hours, 50 minutes, 30 seconds 4 Spare 1 day, 19 hours, 46 minutes, 47 seconds 5 Spare 1 day, 19 hours, 46 minutes, 46 seconds 6 Spare 1 day, 19 hours, 46 minutes, 39 seconds 7 Spare 1 day, 19 hours, 46 minutes, 38 seconds
After switchover, the planes in Spare state go offline and come online quickly. When these planes are offline, the above alarm is generated. The alarms, however, are cleared after 1–2 seconds.
Jul 13 12:43:17 jtac-mx480-r2016-re1 chassisd[8601]: %DAEMON-5-CHASSISD_SNMP_TRAP10: SNMP trap generated: fabric plane offline (jnxFruContentsIndex 12, jnxFruL1Index 2, jnxFruL2Index 0, jnxFruL3Index 0, jnxFruName CB 1, jnxFruType 5, jnxFruSlot 1, jnxFruOfflineReason 2, jnxFruLastPowerOff 0, jnxFruLastPowerOn 183) Jul 13 12:43:17 jtac-mx480-r2016-re1 alarmd[65831]: %DAEMON-4: Alarm set: CB id=134218102, color=YELLOW, class=CHASSIS, reason=CB 1 Fabric Chip 0 Not Online Jul 13 12:43:17 jtac-mx480-r2016-re1 craftd[8605]: %DAEMON-4: Minor alarm set, CB 1 Fabric Chip 0 Not Online Jul 13 12:43:17 jtac-mx480-r2016-re1 chassisd[8601]: %DAEMON-5-CHASSISD_SNMP_TRAP10: SNMP trap generated: fabric plane offline (jnxFruContentsIndex 12, jnxFruL1Index 2, jnxFruL2Index 0, jnxFruL3Index 0, jnxFruName CB 1, jnxFruType 5, jnxFruSlot 1, jnxFruOfflineReason 2, jnxFruLastPowerOff 0, jnxFruLastPowerOn 183) Jul 13 12:43:17 jtac-mx480-r2016-re1 alarmd[65831]: %DAEMON-4: Alarm set: CB id=184549750, color=YELLOW, class=CHASSIS, reason=CB 1 Fabric Chip 1 Not Online Jul 13 12:43:17 jtac-mx480-r2016-re1 craftd[8605]: %DAEMON-4: Minor alarm set, CB 1 Fabric Chip 1 Not Online Jul 13 12:43:18 jtac-mx480-r2016-re1 alarmd[65831]: %DAEMON-4: Alarm set: RE id=218103913, color=YELLOW, class=CHASSIS, reason=Loss of communication with Backup RE Jul 13 12:43:18 jtac-mx480-r2016-re1 craftd[8605]: %DAEMON-4: Minor alarm set, Loss of communication with Backup RE Jul 13 12:43:19 jtac-mx480-r2016-re1 alarmd[65831]: %DAEMON-4: Alarm cleared: CB id=134218102, color=YELLOW, class=CHASSIS, reason=CB 1 Fabric Chip 0 Not Online Jul 13 12:43:19 jtac-mx480-r2016-re1 craftd[8605]: %DAEMON-4: Minor alarm cleared, CB 1 Fabric Chip 0 Not Online Jul 13 12:43:19 jtac-mx480-r2016-re1 chassisd[8601]: %DAEMON-5-CHASSISD_SNMP_TRAP7: SNMP trap generated: fabric plane online (jnxFruContentsIndex 12, jnxFruL1Index 2, jnxFruL2Index 0, jnxFruL3Index 0, jnxFruName CB 1, jnxFruType 5, jnxFruSlot 1) Jul 13 12:43:19 jtac-mx480-r2016-re1 chassisd[8601]: %DAEMON-6: fabric_check_update_coef not using dynamic serdes coefficients Jul 13 12:43:21 jtac-mx480-r2016-re1 alarmd[65831]: %DAEMON-4: Alarm cleared: CB id=184549750, color=YELLOW, class=CHASSIS, reason=CB 1 Fabric Chip 1 Not Online Jul 13 12:43:21 jtac-mx480-r2016-re1 craftd[8605]: %DAEMON-4: Minor alarm cleared, CB 1 Fabric Chip 1 Not Online Jul 13 12:43:21 jtac-mx480-r2016-re1 chassisd[8601]: %DAEMON-5-CHASSISD_SNMP_TRAP7: SNMP trap generated: fabric plane online (jnxFruContentsIndex 12, jnxFruL1Index 2, jnxFruL2Index 0, jnxFruL3Index 0, jnxFruName CB 1, jnxFruType 5, jnxFruSlot 1)
This is expected behavior when a switchover is performed since chassisd needs to go offline and come online, and spare fabric planes need to learn state after switchover. This affects only the spare planes, and hence does not have any service impact. No action is required.