Description

FPC crash and traffic drop occurred due to "Err Detect Register" and "Data Cache Parity Error"

Symptoms

With some HW memory failure in QFX10008/16 Linecards, the PFE process gets stuck in bad state which takes 2-3min for detection causing traffic drops for that time.

External-Trigger

This issue might be seen if the following conditions are met:
* On QFX10008 and QFX10016 Junos platforms 
* Memory corruption or hardware errors on FPC 

 

Solution

On QFX10008 and QFX10016 Junos platforms due to data parity or related hardware failure in linecard memory which caused FPC to go in bad state. 
It took over 2-3 minutes for system to identify this state and switchover traffic to other working interfaces. This caused traffic black holing for 2-3min. Traffic restored once system identifies the state.
This issue can be identified with "Err Detect Register" and "Data Cache Parity Error" error logs.

Workaround

Inline BFD can be configured which can timeout much quickly in such failures resulting in traffic switchovers

This issue has been fixed through PR1795339 in 23.4R2 and future code versions. 

Modification History

2024-05-02 : Article Created