Description

This article details the steps to be performed on the MX SCB when troubleshooting Fabric related issues.

This series of steps is a subroutine that is called from KB23173 - Resolution Guide - MX - Troubleshoot Fabric Planes in Fault or Check State [juniper.net] .

Solution

Perform the following steps for MX SCB Troubleshooting:

Note: For the flowchart version of these steps, click the flowchart icon.

alt

 

[Collect logs.]

  1. As a proactive step, collect logs on the MX router in case they are needed later for diagnosing a problem. The logs to collect are detailed in KB22637 - [M/MX/PTX/T] Data Collection Checklist - Logs/data to collect for troubleshooting [juniper.net] under the "MX Fabric Plane" section.

[Identify the planes with errors.]

  1. Identify the planes with errors by running the following commands:

show chassis fabric planes   
show chassis fabric fpcs

Note:  If you have already identified the faulty planes in KB23173 - Resolution Guide - MX - Troubleshoot Fabric Planes in Fault or Check State [juniper.net] , then you can continue to Step 3 .

Example output 

user@router> show chassis fabric fpcs
Jan 17 17:42:24
Fabric management FPC state
FPC 0
  PFE #0
      Plane 0: Link error
      Plane 1: Plane enabled
      Plane 2: Plane enabled
      Plane 3: Plane enabled
      Plane 4: Plane enabled
      Plane 5: Plane enabled

user@router> show chassis fabric plane 
Fabric management PLANE state
Plane 0
  Plane state: ACTIVE
      FPC 0
          PFE 0 :Links ok
          PFE 1 :Links ok
      FPC 1
          PFE 0 :Links ok
          PFE 1 :Link error
          PFE 2 :Links ok
          PFE 3 :Links ok

In the above example, Plane 0 is reporting an error on FPC1 (PFE1) (which is a type2 MPC card). Example outputs for a single Plane reporting errors and multiple Planes reporting errors are provided in KB23382 - Example outputs for DPC/MPC reporting errors [juniper.net] .

Note: The number of PFEs under each FPC will vary according to the card type present in that slot. For more information about MPCs, refer to MX Series MPC Overview .

[Offline / Online the plane with errors.]

  1. Offline and then Online the Planes with errors (identified in step 2).  

CAUTION: If you are using SCB and at least one MPC or if you are using " chassis fabric redundancy-mode increased-bandwidth " and if any plane is taken offline, it will result in degradation in available Fabric bandwidth. Therefore, it is strongly recommended that you perform this step during a Maintenance Window. If you have an SCBE with plane redundancy available, you should be able to offline a single plane without any service impact. If a plane is already in Check state, restarting the plane which is already in Check state will not cause any impact on the traffic. 

TIP: Refer to KB23067 - [MX] How to Offline/Online an MX Fabric Plane/MX SCB (called from Resolution Guide - Troubleshoot Fabric Plane) [juniper.net] for guidelines on how to offline/online planes.

Then observe the outputs of show chassis alarms and show chassis fabric fpcs .

Has the link error reported in step 2 been corrected and have the alarms cleared (after performing Offline/Online of the plane)?

  • No - Continue to Step 4.

  • Yes - If this is the first occurrence of any Fabric or SCB related link errors on this Fabric Plane, then monitor the SCB for a period of time. If the errors re-appear or if this has happened before on this Fabric Plane, then open a case with your technical support representative .

[Offline / Online the SCB with errors.]

  1. Now Offline and then Online the SCB that is associated with the plane with errors.

CAUTION: In a mixed MPC and DPC card environment, since all the planes are in active state, if any plane is taken offline, it will result in degradation in available Fabric bandwidth. Therefore, it is strongly recommended that you perform this step in a Maintenance Window.

TIP: Refer to KB23067 - [MX] How to Offline/Online an MX Fabric Plane/MX SCB (called from Resolution Guide - Troubleshoot Fabric Plane) [juniper.net] for guidelines on how to offline/online SCBs.

Then observe the outputs of show chassis alarms and show chassis fabric fpcs .

Has the link error reported in step 2 been corrected and have the alarms cleared (after performing the Offline/Online of the SCB)?

[Re-seat the SCB.]

  1. Now physically re-seat the SCB.

For guidelines, refer to the following:

Then observe the outputs of show chassis alarms and show chassis fabric fpcs .

Has the link error reported in step 2 been corrected and have the alarms cleared (after re-seating the SCB)?

Modification History

2022-03-11: Included considerations for redundancy on offlining a single plane on MX