Hot swapping M120 rear fan tray can cause a "Check CB" alarm. This article provides a workaround.
According to several test results, this seems to be a display issue, where we are seeing "check SIB" and "link error" on fabric link.
No specific configuration is needed. Here is the JTAC lab setup:
lab@M120-re0> show version Hostname: M120-re0 Model: m120 JUNOS Base OS boot [8.5R1.14] ... lab@M120-re0> show chassis hardware Hardware inventory: Item Version Part number Serial number Description Chassis JN108EC81AEA M120 Midplane REV 01 710-016198 RB8325 M120 Midplane FPM Board REV 06 710-011407 DA9342 M120 FPM Board FPM Display REV 02 710-011405 DA9116 M120 FPM Display FPM CIP REV 05 710-011410 DA9002 M120 FPM CIP PEM 0 Rev 04 740-011936 000223 AC Power Entry Module Routing Engine 0 REV 05 740-014080 1000677565 RE-A-1000 Routing Engine 1 REV 05 740-014080 1000678920 RE-A-1000 CB 0 REV 07 710-011403 DB4442 M120 Control Board CB 1 REV 07 710-011403 DB4473 M120 Control Board FPC 0 REV 05 710-015908 CZ2018 M120 CFPC 10GE PIC 0 BUILTIN BUILTIN 1x 10GE(LAN/WAN) XFP Xcvr 0 REV 01 740-014279 753019A00062 XFP-10G-LR FPC 2 REV 02 710-015837 DB6890 M120 FPC Type 2 Board B REV 04 710-015838 DB6952 M120 FPC Mezz Board FPC 3 REV 02 710-015835 DC5009 M120 FPC Type 1 PIC 0 REV 11 750-007643 JF4051 1x G/E IQ, 1000 BASE Xcvr 0 REV 01 740-011782 P6Q2GN0 SFP-SX PIC 1 REV 04 750-003033 HE2961 4x OC-3 SONET, MM PIC 2 REV 04 750-005728 JC9441 2x OC-3 ATM-II IQ, SMIR PIC 3 REV 06 750-010240 JC4017 1x G/E SFP, 1000 BASE Xcvr 0 REV 01 740-011782 P8Q23GR SFP-SX Board B REV 04 710-011390 DB4917 M120 FPC Mezz Board FEB 0 REV 04 710-015795 CZ4104 M120 FEB FEB 1 REV 04 710-015795 CZ4081 M120 FEB FEB 2 REV 04 710-015795 CZ7731 M120 FEB FEB 3 REV 04 710-015795 CZ7760 M120 FEB FEB 4 REV 04 710-015795 CZ7836 M120 FEB Fan Tray 0 Front Top Fan Tray Fan Tray 1 Front Bottom Fan Tray Fan Tray 2 Rear Top Fan Tray Fan Tray 3 Rear Bottom Fan Tray
Keep re-seating the rear-top fan tray. The issue is seen more frequently when pulling the fan tray slowly, (caused by loose contact of fan-tray?). Here are the results when pulling out the rear top fan tray.
<-- no alarm lab@M120-re0> show chassis alarms No alarms currently active
lab@M120-re0> show chassis alarms No alarms currently active
Got the following chassid log (during this time, the issue was first seen). lab@M120-re0> Jan 8 18:26:07 fpm_reno_rcb_intr rcb_ints_pending 0x00050004 Jan 8 18:26:07 re_kontron_gpio_intr: Button events pending 0x00000000 Jan 8 18:26:07 exit fpm_reno_rcb_intr ... Jan 8 18:26:07 fpm_reno_rcb_intr rcb_ints_pending 0x00050000 Jan 8 18:26:07 re_kontron_gpio_intr: Button events pending 0x00000000 Jan 8 18:26:07 exit fpm_reno_rcb_intr ... Jan 8 18:26:07 FM: Received plane state from PFE board 4 Jan 8 18:26:07 FM: plane state ... Jan 8 18:26:07 FM: msg->pb=4 Jan 8 18:26:07 FM: link status ... Jan 8 18:26:07 FM: 4 0 0 0 Jan 8 18:26:07 FM: 2 0 0 0 Jan 8 18:26:07 FM: 2 0 0 0 Jan 8 18:26:07 FM: 2 0 0 0 Jan 8 18:26:07 FM: dest status ... Jan 8 18:26:07 FR: Not all fabric planes in error for FEB 4 Jan 8 18:26:07 CHASSISD_SNMP_TRAP7: SNMP trap generated: Fru Check (jnxFruContentsIndex 12, jnxFruL1Index 1, jnxFruL2Index 0, jnxFruL3Index 0, jnxFruName CB 0, jnxFruType 5, jnxFruSlot 1) Jan 8 18:26:07 send: yellow alarm set, device CB 0, reason Check CB 0 FCHIP0 Jan 8 18:26:08 send: yellow alarm set, device Rear Top Fan Tray 2, reason Fan/Blower Absent Jan 8 18:26:08 CHASSISD_BLOWERS_SPEED_FULL: Fans and impellers being set to full speed [fan/blower missing/failed] Jan 8 18:26:08 Rear Top Fan Tray 2 removed Jan 8 18:26:08 CHASSISD_SNMP_TRAP7: SNMP trap generated: FRU removal (jnxFruContentsIndex 4, jnxFruL1Index 3, jnxFruL2Index 0, jnxFruL3Index 0, jnxFruName Rear Top Fan Tray, jnxFruType 13, jnxFruSlot 3) Jan 8 18:26:08 hwdb: entry for fan at slot 2 deleted lab@M120-re0> show chassis alarms 2 alarms currently active Alarm time Class Description 2008-01-08 18:26:08 JST Minor Fan/Blower Absent 2008-01-08 18:26:07 JST Minor Check CB 0 Fabric Chip 0 --> check CB alarms. syslog messages Jan 8 18:26:06.254 M120-re0 xntpd[4988]: %NTP-7: bind() fd 8, family 2, port 123, addr 10.0.0.1, in_classd=0 flags=0 fails: Can't assign requested address Jan 8 18:26:07.675 M120-re0 chassisd[4979]: %DAEMON-5-CHASSISD_SNMP_TRAP7: SNMP trap generated: Fru Check (jnxFruContentsIndex 12, jnxFruL1Index 1, jnxFruL2Index 0, jnxFruL3Index 0, jnxFruName CB 0, jnxFruType 5, jnxFruSlot 1) Jan 8 18:26:07.676 M120-re0 alarmd[4443]: %DAEMON-4: Alarm set: CB color=YELLOW, class=CHASSIS, reason=Check CB 0 Fabric Chip 0 Jan 8 18:26:07.676 M120-re0 craftd[4444]: %DAEMON-4: Minor alarm set, Check CB 0 Fabric Chip 0 Jan 8 18:26:08.355 M120-re0 chassisd[4979]: %DAEMON-5-CHASSISD_BLOWERS_SPEED_FULL: Fans and impellers being set to full speed [fan/blower missing/failed] Jan 8 18:26:08.354 M120-re0 alarmd[4443]: %DAEMON-4: Alarm set: Fan color=YELLOW, class=CHASSIS, reason=Fan/Blower Absent Jan 8 18:26:08.355 M120-re0 craftd[4444]: %DAEMON-4: Minor alarm set, Fan/Blower Absent Jan 8 18:26:08.422 M120-re0 chassisd[4979]: %DAEMON-5-CHASSISD_SNMP_TRAP7: SNMP trap generated: FRU removal (jnxFruContentsIndex 4, jnxFruL1Index 3, jnxFruL2Index 0, jnxFruL3Index 0, jnxFruName Rear Top Fan Tray, jnxFruType 13, jnxFruSlot 3) lab@M120-re0> show chassis fabric feb Fabric management FEB state FEB 0 Plane 0: Plane enabled Plane 1: Plane enabled Plane 2: Plane enabled Plane 3: Plane enabled FEB 1 Plane 0: Plane enabled Plane 1: Plane enabled Plane 2: Plane enabled Plane 3: Plane enabled FEB 2 Plane 0: Plane enabled Plane 1: Plane enabled Plane 2: Plane enabled Plane 3: Plane enabled FEB 3 Plane 0: Plane enabled Plane 1: Plane enabled Plane 2: Plane enabled Plane 3: Plane enabled FEB 4 Plane 0: Link error --> link error on FEB4 Plane 1: Plane enabled Plane 2: Plane enabled Plane 3: Plane enabled lab@M120-re0> show chassis fabric plane Fabric management PLANE state Plane 0 Plane state: ACTIVE FEB 0: Links ok FEB 1: Links ok FEB 2: Links ok FEB 3: Links ok FEB 4: Links ok Plane 1 Plane state: ACTIVE FEB 0: Links ok FEB 1: Links ok FEB 2: Links ok FEB 3: Links ok FEB 4: Links ok Plane 2 Plane state: ACTIVE FEB 0: Links ok FEB 1: Links ok FEB 2: Links ok FEB 3: Links ok FEB 4: Links ok Plane 3 Plane state: ACTIVE FEB 0: Links ok FEB 1: Links ok FEB 2: Links ok FEB 3: Links ok FEB 4: Links ok
lab@M120-re0> Jan 8 18:26:07 fpm_reno_rcb_intr rcb_ints_pending 0x00050004 Jan 8 18:26:07 re_kontron_gpio_intr: Button events pending 0x00000000 Jan 8 18:26:07 exit fpm_reno_rcb_intr ... Jan 8 18:26:07 fpm_reno_rcb_intr rcb_ints_pending 0x00050000 Jan 8 18:26:07 re_kontron_gpio_intr: Button events pending 0x00000000 Jan 8 18:26:07 exit fpm_reno_rcb_intr ... Jan 8 18:26:07 FM: Received plane state from PFE board 4 Jan 8 18:26:07 FM: plane state ... Jan 8 18:26:07 FM: msg->pb=4 Jan 8 18:26:07 FM: link status ... Jan 8 18:26:07 FM: 4 0 0 0 Jan 8 18:26:07 FM: 2 0 0 0 Jan 8 18:26:07 FM: 2 0 0 0 Jan 8 18:26:07 FM: 2 0 0 0 Jan 8 18:26:07 FM: dest status ... Jan 8 18:26:07 FR: Not all fabric planes in error for FEB 4 Jan 8 18:26:07 CHASSISD_SNMP_TRAP7: SNMP trap generated: Fru Check (jnxFruContentsIndex 12, jnxFruL1Index 1, jnxFruL2Index 0, jnxFruL3Index 0, jnxFruName CB 0, jnxFruType 5, jnxFruSlot 1) Jan 8 18:26:07 send: yellow alarm set, device CB 0, reason Check CB 0 FCHIP0 Jan 8 18:26:08 send: yellow alarm set, device Rear Top Fan Tray 2, reason Fan/Blower Absent Jan 8 18:26:08 CHASSISD_BLOWERS_SPEED_FULL: Fans and impellers being set to full speed [fan/blower missing/failed] Jan 8 18:26:08 Rear Top Fan Tray 2 removed Jan 8 18:26:08 CHASSISD_SNMP_TRAP7: SNMP trap generated: FRU removal (jnxFruContentsIndex 4, jnxFruL1Index 3, jnxFruL2Index 0, jnxFruL3Index 0, jnxFruName Rear Top Fan Tray, jnxFruType 13, jnxFruSlot 3) Jan 8 18:26:08 hwdb: entry for fan at slot 2 deleted lab@M120-re0> show chassis alarms 2 alarms currently active Alarm time Class Description 2008-01-08 18:26:08 JST Minor Fan/Blower Absent 2008-01-08 18:26:07 JST Minor Check CB 0 Fabric Chip 0 --> check CB alarms. syslog messages Jan 8 18:26:06.254 M120-re0 xntpd[4988]: %NTP-7: bind() fd 8, family 2, port 123, addr 10.0.0.1, in_classd=0 flags=0 fails: Can't assign requested address Jan 8 18:26:07.675 M120-re0 chassisd[4979]: %DAEMON-5-CHASSISD_SNMP_TRAP7: SNMP trap generated: Fru Check (jnxFruContentsIndex 12, jnxFruL1Index 1, jnxFruL2Index 0, jnxFruL3Index 0, jnxFruName CB 0, jnxFruType 5, jnxFruSlot 1) Jan 8 18:26:07.676 M120-re0 alarmd[4443]: %DAEMON-4: Alarm set: CB color=YELLOW, class=CHASSIS, reason=Check CB 0 Fabric Chip 0 Jan 8 18:26:07.676 M120-re0 craftd[4444]: %DAEMON-4: Minor alarm set, Check CB 0 Fabric Chip 0 Jan 8 18:26:08.355 M120-re0 chassisd[4979]: %DAEMON-5-CHASSISD_BLOWERS_SPEED_FULL: Fans and impellers being set to full speed [fan/blower missing/failed] Jan 8 18:26:08.354 M120-re0 alarmd[4443]: %DAEMON-4: Alarm set: Fan color=YELLOW, class=CHASSIS, reason=Fan/Blower Absent Jan 8 18:26:08.355 M120-re0 craftd[4444]: %DAEMON-4: Minor alarm set, Fan/Blower Absent Jan 8 18:26:08.422 M120-re0 chassisd[4979]: %DAEMON-5-CHASSISD_SNMP_TRAP7: SNMP trap generated: FRU removal (jnxFruContentsIndex 4, jnxFruL1Index 3, jnxFruL2Index 0, jnxFruL3Index 0, jnxFruName Rear Top Fan Tray, jnxFruType 13, jnxFruSlot 3) lab@M120-re0> show chassis fabric feb Fabric management FEB state FEB 0 Plane 0: Plane enabled Plane 1: Plane enabled Plane 2: Plane enabled Plane 3: Plane enabled FEB 1 Plane 0: Plane enabled Plane 1: Plane enabled Plane 2: Plane enabled Plane 3: Plane enabled FEB 2 Plane 0: Plane enabled Plane 1: Plane enabled Plane 2: Plane enabled Plane 3: Plane enabled FEB 3 Plane 0: Plane enabled Plane 1: Plane enabled Plane 2: Plane enabled Plane 3: Plane enabled FEB 4 Plane 0: Link error --> link error on FEB4 Plane 1: Plane enabled Plane 2: Plane enabled Plane 3: Plane enabled lab@M120-re0> show chassis fabric plane Fabric management PLANE state Plane 0 Plane state: ACTIVE FEB 0: Links ok FEB 1: Links ok FEB 2: Links ok FEB 3: Links ok FEB 4: Links ok Plane 1 Plane state: ACTIVE FEB 0: Links ok FEB 1: Links ok FEB 2: Links ok FEB 3: Links ok FEB 4: Links ok Plane 2 Plane state: ACTIVE FEB 0: Links ok FEB 1: Links ok FEB 2: Links ok FEB 3: Links ok FEB 4: Links ok Plane 3 Plane state: ACTIVE FEB 0: Links ok FEB 1: Links ok FEB 2: Links ok FEB 3: Links ok FEB 4: Links ok
<-- check whether cb is still there lab@M120-re0> show chassis alarms 1 alarms currently active Alarm time Class Description 2008-01-08 18:26:07 JST Minor Check CB 0 Fabric Chip 0
lab@M120-re0> show chassis alarms 1 alarms currently active Alarm time Class Description 2008-01-08 18:26:07 JST Minor Check CB 0 Fabric Chip 0
lab@M120-re0> show chassis fabric feb Fabric management FEB state FEB 0 Plane 0: Plane enabled Plane 1: Plane enabled Plane 2: Plane enabled Plane 3: Plane enabled FEB 1 Plane 0: Plane enabled Plane 1: Plane enabled Plane 2: Plane enabled Plane 3: Plane enabled FEB 2 Plane 0: Plane enabled Plane 1: Plane enabled Plane 2: Plane enabled Plane 3: Plane enabled FEB 3 Plane 0: Plane enabled Plane 1: Plane enabled Plane 2: Plane enabled Plane 3: Plane enabled FEB 4 Plane 0: Link error --> link error on FEB4 Plane 1: Plane enabled Plane 2: Plane enabled Plane 3: Plane enabled
lab@M120-re0> request chassis feb slot 4 restart Restart initiated, use "show chassis feb" to verify lab@M120-re0> show chassis alarms No alarms currently active Note : Restarting the feb will lead to the corresponding fpc going offline/online. If a feb redundancy-group is configured for the fpc (issued feb as primary with another backup feb), please switch over redundancy febbrole from primary to backup, Then you can restart the issued feb.
lab@M120-re0> request chassis feb slot 4 restart Restart initiated, use "show chassis feb" to verify lab@M120-re0> show chassis alarms No alarms currently active
Note : Restarting the feb will lead to the corresponding fpc going offline/online. If a feb redundancy-group is configured for the fpc (issued feb as primary with another backup feb), please switch over redundancy febbrole from primary to backup, Then you can restart the issued feb.
request chassis fabric plane 0 offline request chassis fabric plane 0 online