Description

The "FI Packet CRC error" message indicates that the PFE received packets with the invalid checksum from another PFE over the fabric. The FPC reporting this error is most likely not the one with any issue. It merely reports errors caused by other components.


This is a Troubleshooting Article for a PFE ASIC Syslog Event.
To view other documented syslog events related to XMCHIP, XLCHIP, MQCHIP, LUCHIP, EACHIP, and PECHIP, see KB31893 - Master Index of Articles for Troubleshooting PFE ASIC Syslog Events [juniper.net] .

.

Symptoms

When a "FI Packet CRC error" event occurs, a message similar to the following is reported:
Jan 22 180110 router0 %PFE-3 fpc0 XMCHIP(2) XMCHIP(2) FI Packet CRC error - Stream 28 Count 1

Indications:

  1. Service Impact: The counter in the error message reports the number of packets dropped due to failing packet CRC check
  2. The error is most likely outside of this FPC
  3. No permanent impact of packet forwarding
     

 

Solution

The cause is due to a hardware issue which could be from a remote PFE, the fabric itself or the reporting PFE.  Further analysis will be required to determine the exact source.

This transient or permanent issue is caused by hardware, which could be the remote PFE, fabric itself, or reporting PFE. This error indicates that the PFE received a bad packet from another PFE over the fabric. Usually, multiple occurrences of these log messages are reported. By comparing the stream numbers in each log message, one can determine which is the faulty FPC. If the stream numbers are different, then the reporting PFE is the most likely culprit. If the event is reported from several PFEs with the same stream number, the sending PFE is the culprit. Use the FPC to Stream Mapping Table to map stream numbers to FPC slots.

FPC to Stream Mapping Table
alt

Recommended Actions:

Review the /var/log/messages to determine the most likely cause of packets' CRC errors. Contact JTAC if you need assistance. Please include:​​​​​​​
1. Output of the "request support information | no-more"
2. Content of /var/log directory

{DIDYOUKNOWSERVICENOWTOKEN.EN_US}

Perform these steps to determine the cause and resolve the problem (if any).  Continue through each step until the problem is resolved.

  1. Collect the show command output.

    {SYSLOGSERVICENOWTOKEN.EN_US}

    Capture the output to a file (in case you have to open a technical support case). To do this, configure each SSH client/terminal emulator to log your session.

    show log messages
    show log chassisd
    start shell network pfe <fpc#>
    show nvram
    show syslog messages
    exit

  2. Analyze the show command output.

    1. In the 'show log messages', review the events that occurred at or just before the appearance of the "FI Packet CRC error" message. Frequently these events help identify the cause.

    2. Perform a fabric plane offline/online, one by one, to check if the errors stop. If not, proceed to the FPC reseat in step 3.

      Fabric plane can be off-lined using command “request chassis fabric plane offline 0”:

      user@host> request chassis fabric plane offline 0    
      Offline initiated, use "show chassis fabric plane" to verify
      user@host>

      Fabric plane can be on-lined using command “request chassis fabric plane offline 0”:

      user@host> request chassis fabric plane online 0     
      Online initiated, use "show chassis fabric plane" to verify
      user@host>  
    3. Determine the FPC originating the bad packets:

      1. If there are multiple syslogs from the same FPC listing different stream numbers, then the FPC originating the logs is the potentially faulty FPC.

      2. If there are multiple syslogs from many FPCs or PFEs listing the same stream number, then map the stream number to the originating FPC using the FPC to Stream Mapping Table in the CAUSE section above.

    4. Reseat the FPC determined in the previous step.

    5. If the logs recur after reseat, contact your technical support representative immediately.

 

This article is indexed in KB31893 - Primary Index of Articles for Troubleshooting PFE ASIC Syslog Events [juniper.net] ; tag XMCHIPTSG.


Tip: When looking at an event in the logs, it is important to focus on the first error message in a collection of syslog messages. The first error message is usually the cause of all the follow-on error messages. The follow-on collateral damage error messages can be ignored.

 

Modification History

2022-11-15 Update content of the KB - clearly state that the FI PACKET CRC Error causes by errors outside of the FPC reporting the error