Description

The "Cell packing interface error" messages are caused by corrupted cell received from the HMC (Hybrid Memory Cube) interfaces.

This is a Troubleshooting Article for a PFE ASIC Syslog Event.
To view other documented syslog events related to XMCHIP, XLCHIP, MQCHIP, LUCHIP, EACHIP, and PECHIP, see KB31893 - Master Index of Articles for Troubleshooting PFE ASIC Syslog Events [juniper.net] .

.

Symptoms

When a Cell packing interface error event occurs, messages similar to the following is reported:

2017-07-19 16:39:05.0000 router fpc19 eachip_hmcif_rx_intr_handler(7324): EA[2:0]: HMCIF Rx: Checksum error detected on FO response - Chunk Address 0x13f467
2017-07-19 16:39:05.0000 router fpc19 MQSS(3): FO: Packet Error - Error Packets 15, Stream 43
2017-07-19 16:39:05.0000 router fpc19 eachip_hmcif_rx_intr_handler(7324): EA[3:0]: HMCIF Rx: Checksum error detected on FO response - Chunk Address 0xd7c599
2017-07-19 16:39:06.0000 router fpc10 MQSS(2): FI: Error cell sent to reorder engine - Stream 78, Count 21
2017-07-19 16:39:06.0000 router fpc10 MQSS(3): FI: Error cell sent to reorder engine - Stream 79, Count 18
2017-07-19 16:39:06.0000 router fpc19 MQSS(2): FO: Packet Error - Error Packets 21, Stream 42
2017-07-19 16:39:06.0000 router fpc19 eachip_hmcif_rx_intr_handler(7324): EA[2:0]: HMCIF Rx: Checksum error detected on FO response - Chunk Address 0xbad57b
2017-07-19 16:39:06.0000 router fpc19 MQSS(3): FO: Packet Error - Error Packets 17, Stream 43
2017-07-19 16:39:06.0000 router fpc19 eachip_hmcif_rx_intr_handler(7324): EA[3:0]: HMCIF Rx: Checksum error detected on FO response - Chunk Address 0x53d21e
2017-07-19 16:39:06.0000 router fpc19 Cmerror Op Set: MQSS(2): MQSS(2): FO: Cell packing interface error - data64_err_log 0x5cc028, Stream 40


Indications:

  • Permanent impact on packet forwarding.
  • Major Alarm will be raised.
  • In Junos 17.3R1 and higher, it will perform disable-pfe action automatically.

Solution

Corrupted cells are received from the packet memory ( KB32335 - Syslog message: MQSS.*Checksum error detected on [juniper.net] ) which can cause errors when packing the cells together before sending into the fabric or WAN side. Most of the cells are already dropped due to checksum integrity checks received from the HMCIF as reported in the syslog. Packet cells which are not passing the integrity checks, get marked at Fabric output with ' FO: Packet Error' and sent to the remote PFE, where those cells are getting dropped with the error message ' FI: Error cell sent to reorder engine'

{DIDYOUKNOWSERVICENOWTOKEN.EN_US}

Perform these steps to determine the cause and resolve the problem (if any).  Continue through each step until the problem is resolved.

  1. Collect the show command output.

    {SYSLOGSERVICENOWTOKEN.EN_US}

    Capture the output to a file (in case you have to open a technical support case). To do this, configure each SSH client/terminal emulator to log your session.

    show log messages
    show log chassisd
    start shell network pfe <fpc#>
    show nvram
    show syslog messages
    exit

  2. Analyze the show command output.

    In the 'show log messages', review the events that occurred at or just before the appearance of the "Cell packing interface error" message. Frequently these events help identify the cause.

  • If the FPC is not passing the internal Power on Self Test (POST) and reports "Error initializing PFE asics" Hardware Replacements needs to be initiated.

  • In Junos OS 17.3R1 or higher the default action for major alarm will be disable-pfe   PR1186421.

  • Contact your technical support representative immediately.

This article is indexed in  KB31893 - Primary Index of Articles for Troubleshooting PFE ASIC Syslog Events [juniper.net] ; tag EACHIPTSG


Tip: When looking at an event in the logs, it is important to focus on the first error message in a collection of syslog messages. The first error message is usually the cause of all the follow-on error messages. The follow-on collateral damage error messages can be ignored.

 

Modification History


Note: KB Team - All changes to this article must be approved by the AI-Scripts Review team ( [email protected] ) before re-publishing.


AI-Scripts history:        (Updated by AI-Scripts team only)
 
Date KB Article Version AI-Scripts PR (optional) Notes
1/12/2018 1.0 1332299 New Syslog KB