Description

This document explains the meaning of alarm " FPC x Routing Chip Errors Detected  - Error code: 1040 " and steps to overcome it. The alarm can be seen on M320, T640, T1600, TX Matrix, TX Matrix Plus devices.

Symptoms

Active alarm: 

user@host> show chassis alarms
1 alarms currently active
Alarm time               Class  Description
2019-07-30 06:01:32 UTC  Minor  FPC 3 Routing Chip Errors Detected  - Error code: 1040

Syslog: 

alarmd[1457]: Alarm set: FPC color=YELLOW, class=CHASSIS, reason=FPC 3 Routing Chip Errors Detected  - Error code: 1040
craftd[1458]:  Minor alarm set, FPC 3 Routing Chip Errors Detected  - Error code: 1040

Solution

  1. SRAM Parity errors
  2. Hardware FPC fault
  3. A transient issue with FPC 

Detailed Logs: 

fpc3 RCHIP(1): 1 Discards - data error
fpc3 RCHIP(1): SRAM parity error at loc 0x91fe2c value 0x18689c00 from key engine
fpc 3 RCHIP(1): SRAM parity error at loc 0xadfe2c value 0x3fac2800 from key engine
fpc 3 RCHIP(1): SRAM parity error at loc 0xaebe2c value 0xaa4b8100 from key engine  <-- Make a note or RCHIP number (here it is RCHIP 1)
fpc3 RCHIP(1): 1 Discards - data error
fpc3 RCHIP(1): SRAM parity error at loc 0x81f27c value 0x7d627b00 from key engin

The issue is due to the SRAM parity error on the RCHIP of FPC. The SRAM memory is parity-protected and the parity check is executed when reading the memory. Parity errors can be due to hardware failure or due to transient memory chip errors.

Check the FPC state:

user@host> show chassis fpc detail
<snip> 

Slot 3 information:
  State                               Online
  Temperature                      43 degrees C / 109 degrees F
<snip>

If FPC is online, you need not perform the next step, you can directly perform restart of FPC. If FPC is online, please continue with the below.

Check for errors count on FPC:

user@host> request pfe execute command "show rchip 1 statistics errors" target fpc 3   

Hardware Errors: 

     5 SRAM parity errors  <-- Make a note of it
     0 RRCP errors
     0 RKME errors

Execute the above multiple times to know the rate of increase. If the error count is stable, you may choose to ignore the errors considering it as a one-time occurrence. If error increments, then you might want to proceed with the next steps.

Restart FPC:

user@host> request chassis fpc (offline | online | restart) slot slot-number

OR

Manually reseat/reboot the FPC so that FPC boots back up online.

The above step should help us clear the alarms. If not, please collect the below outputs and open a JTAC case for detailed investigation:

request pfe execute command "show syslog messages" target fpc <FPC#>
request pfe execute command "show nvram" target fpc <FPC#>