SFB went offline & when trying to bring it up, FPCs went into fault state
Mar 24 15:29:09.010 2025 JTAC-re0 mgd[77384]: UI_CMDLINE_READ_LINE: User ‘root’, command 'request chassis sfb slot 5 offline '
Mar 24 15:29:12.511 2025 JTAC-re0 mgd[77384]: UI_CMDLINE_READ_LINE: User ‘root’, command 'request chassis sfb slot 5 online '
Mar 24 15:32:46.807 2025 JTAC-re0 craftd[32843]: Major alarm set, FPC 9 Major Errors
Mar 24 15:33:27.475 2025 JTAC-re0 craftd[32843]: Major alarm set, FPC 6 Major Errors
Mar 24 15:33:34.949 2025 JTAC-re0 craftd[32843]: Major alarm set, FPC 8 Major Errors
Mar 24 15:33:45.806 2025 JTAC-re0 craftd[32843]: Major alarm set, FPC 7 Major Errors
Mar 27 03:09:11.504 2025 JTAC-re0 fpc6 Cmerror Op Set: CM[0]: MPC fabric selfping blackhole (1) exceed raising threshold (1) occurrance (0) for module/pfe (10:0) (URI: /fpc/6/pfe/0/cm/0/CM[0]/0/CM_CMERROR_FABRIC_SELFPING)
Mar 27 03:09:51.816 2025 JTAC-re0 fpc8 Cmerror Op Set: CM[0]: MPC fabric selfping blackhole (1) exceed raising threshold (1) occurrance (0) for module/pfe (10:0) (URI: /fpc/8/pfe/0/cm/0/CM[0]/0/CM_CMERROR_FABRIC_SELFPING)
Mar 27 03:09:56.527 2025 JTAC-re0 fpc7 Cmerror Op Set: CM[0]: MPC fabric selfping blackhole (1) exceed raising threshold (1) occurrance (0) for module/pfe (10:0) (URI: /fpc/7/pfe/0/cm/0/CM[0]/0/CM_CMERROR_FABRIC_SELFPING)
Mar 27 03:10:47.340 2025 JTAC-re0 fpc9 Cmerror Op Set: CM[0]: MPC fabric selfping blackhole (1) exceed raising threshold (1) occurrance (0) for module/pfe (10:0) (URI: /fpc/9/pfe/0/cm/0/CM[0]/0/CM_CMERROR_FABRIC_SELFPING)
On MX platforms, in case of a fatal error encountered during SFB reboot ( due to hardware issue or ungrateful power restart ), SPMB will try to offline this SFB during bootup while the system is busy training the fabric links to begin it online.
This may cause a system-wide traffic impact due to the fabric not being consistent.
During such event, below errors can be seen as a trigger for this issue:
fpc CMError: /fpc/0/pfe/0/cm/0/CM[1]/1/CM_CMERROR_FABRIC_SELFPING (0x2e0006), scope: pfe, category: functional, severity: major, module: CM[1], type: MPC fabric selfping blackhole, oc_category: default
fpc CMTFPC: Fabric request time out pfe 0 plane 0 fab_stream 0, attempting recovery
fpc resiliencyd[]: CMError: /fpc/0/fabspoked-pfe/0/cm/0/pfe/0/fpc_link_to_sib_fault (0x410001), scope: pfe, category: internal, severity: minor, module: fab-pfe@0, type: fpc_link_to_sib_fault, oc_category: default
At the same time, the system is busy training the fabric links to begin it online. This may cause a system-wide traffic impact due to the fabric not being consistent.
This is being fixed via below PR.
https://prsearch.juniper.net/problemreport/PR1798780