Description

On MX240/480/960 platforms, Fabric drops are observed when changing the fabric redundancy mode with SPC3 card


Symptoms

Fabric drops are observed when changing the fabric redundancy mode dynamically. Multiple planes are going into check state and fabric timeouts/dest-err log messages are seen.

<timestamp> <hostname> fpc2 MQSS(1): FO: Request timeout error - Number of timeouts 4, RC select 0, Stream 12 
<timestamp> <hostname> fpc2 CMTFPC: Fabric request time out pfe 1 plane 3 fab_stream 12, attempting recovery 
<timestamp> <hostname> fpc2 CMTFPC: fabric fab_dest 48 for FPC 2 PFE: 1 fabric plane 0 saw dest-err in stream 12 
<timestamp> <hostname> fpc2 CMTFPC: fabric fab_dest 48 for FPC 2 PFE: 1 fabric plane 1 saw dest-err in stream 12 
<timestamp> <hostname> fpc2 CMTFPC: fabric fab_dest 48 for FPC 2 PFE: 1 fabric plane 2 saw dest-err in stream 12 
<timestamp> <hostname> fpc2 CMTFPC: fabric fab_dest 48 for FPC 2 PFE: 1 fabric plane 3 saw dest-err in stream 12


Solution

  • With SPC3 card installed dynamic change of fabric bandwidth is not supported
  • After changing the fabric bandwidth restart of the system is recommended - "request system reboot both-routing-engines"
  • With the help of PR1670507 fix in chassisd logs we can see this log  "please reboot after fabric mode change for SPC3" when the configuration of fabric bandwidth is changed.  

Modification History

2026-10-20 : Article Created