Description

This article describes HSL2 links and focuses​ on the following syslog message and how to address it:​

fpc Cmerror Op Set: HSL2: HSL2 Fault MQCHIP(0)-chan-rx-62 <= IXCHIP(0)-chan-tx-3 - set   ​

Symptoms

HSL2: The PFE’s ASICs communicate with each other and with the fabric chips at the physical layer via HSL2 (High Speed Link version 2) links. 

  1. When issues are detected in the HSL2 links between the PFE's ASICs, an alarm similar to the following will be active:

    NEO> show chassis alarms
    1 alarms currently active
    Alarm time                          Class  Description
    2020-05-13 15:42:50 PST  Major  FPC 0 Major Errors - HSL2 Error code: 0x200001
  2. The HSL2 also allow communication between the PFE's ASICs and the fabric chips. An alarm similar to the following could also become active:

    NEO> show chassis alarms
    2 alarms currently active
    Alarm time                         Class  Description
    2020-05-13 11:06:30 PST  Minor  Check CB 1 Fabric Chip 1 
    2020-05-13 11:06:30 PST  Major  Fault CB 2 Fabric Chip 0

Solution

For both scenarios, the most common reason of failure is CRC errors. Cyclic redundancy check is one of the most popular methods of error detection for digital signals.

However, the errors could be increasing. In that case, if no FPC action has been configured for the major alarm, manual intervention is needed.

To verify if the errors are increasing, login to the PFE shell and obtain the HSL2 statistics of the FPC (the output below was shortened):

NEO> start shell pfe network fpc0
NPC0(NEO)# show hsl2 statistics

                                                           Cell Received (last)           CRC Errors (last)
------------------------------------------------------
MQCHIP(0) channel statistics :
            MQCHIP(0)-chan-rx-0 <= QXCHIP(0)-chan-tx-1            99520353703489  (13939454)         0  (0)         
            MQCHIP(0)-chan-rx-2 <= LUCHIP(0)-chan-tx-2           758842696983198  (106278017)        0  (0)         
            MQCHIP(0)-chan-rx-3 <= LUCHIP(0)-chan-tx-3           758842696983198  (106278017)        0  (0)         
            MQCHIP(0)-chan-rx-7 <= remote/unknown                     6776073636  (1876871644)       0  (0)         
            MQCHIP(0)-chan-rx-8 <= remote/unknown                     6776073636  (1876871644)       0  (0)         
            MQCHIP(0)-chan-rx-14 <= remote/unknown                    6938103514  (1918669588)       0  (0)         
            MQCHIP(0)-chan-rx-15 <= remote/unknown                    6938103514  (1918669588)       0  (0)         
            MQCHIP(0)-chan-rx-21 <= remote/unknown                    6938118207  (1918669836)       0  (0)         
            MQCHIP(0)-chan-rx-22 <= remote/unknown                    6938118207  (1918669836)       0  (0)         
            MQCHIP(0)-chan-rx-28 <= remote/unknown                    6776078710  (1876871932)       0  (0)         
            MQCHIP(0)-chan-rx-29 <= remote/unknown                    6776078710  (1876871932)       0  (0)         
            MQCHIP(0)-chan-rx-62 <= IXCHIP(0)-chan-tx-3          283631445184166  (39743389)         2  (0)

  1. To clear the errors, restart the affected PFE#0, which is in control of PIC#0 and PIC #1.

    > show chassis hardware 
    Hardware inventory:
    Item                  Description
    Chassis               MX480
    CB 1                  Enhanced MX SCB
    FPC 1                 MPCE Type 2 3D EQ
      CPU                 MPCE PMB 2G 
      MIC 0               3D 20x 1GE(LAN) SFP
        PIC 0             10x 1GE(LAN) SFP
          Xcvr 0          SFP-T
          Xcvr 1          SFP-T
        PIC 1             10x 1GE(LAN) SFP
          Xcvr 8          SFP-SX
          Xcvr 9          SFP-SX​
       MIC 1              3D 4x 10GE  XFP
         PIC 2            2x 10GE  XFP
         PIC 3            2x 10GE  XFP
    
    > request chassis mic mic-slot 0 fpc-slot 1 offline 
    fpc 1 mic 0 offline initiated, use "show chassis fpc pic-status 1" to verify

    Refer to the technical documentation for details on r equest chassis mic

    The errors should stop. If they continue, contact your JTAC Representative to troubleshoot further.

  2. If the alarm is related to a Fabric chip, refer to the following KB article to troubleshoot further:

    KB23173 - Resolution Guide - MX - Troubleshoot Fabric Planes in Fault or Check State [juniper.net]

Modification History

2022-10-22 - Adding code 0x200002 and mark this KB public
2023-12-20 - Adding code 0x200003 and mark this KB public

Related Information