Description

This article explains the meaning of the "​XM Chip Error code: 0x70034" alarm that is observed in the Flexible PIC Concentrators (FPCs) in MX Series devices, and the steps to resolve the errors.

 

Symptoms

Active Alarm

user@host> show chassis alarms no-forwarding
 
1 alarms currently active
Alarm time                Class  Description
2019-11-10 07:18:43 UTC   Major  FPC 17 Major Errors - XM Chip Error code: 0x70034

Error logs

Nov 10 07:18:43  alarmd[6931]: Alarm set: FPC color=RED, class=CHASSIS, reason=FPC 17 Major Errors - XM Chip Error code: 0x70034
Nov 10 07:18:43  craftd[12552]:  Major alarm set, FPC 17 Major Errors - XM Chip Error code: 0x70034

 

Solution

To troubleshoot and resolve the error, perform the following steps:

  1. Check the log messages.

user@host> show log messages | match "07:18"
Nov 10 07:18:39  fpc17 XMCHIP(3): XMCHIP(3): CPQ0: DRAM ECC error summary indication for CPQ interface 0 (1X32 DMC 1 DDR 1) - Logical DRAM Bank 0, Logical Address 0xd67f6 
Nov 10 07:18:40  fpc17 Cmerror Op Set: XMCHIP(3): XMCHIP(3): CPQ0: DRAM double bit ECC error for CPQ interface 0 (1X32 DMC 1 DDR 1) - Logical DRAM Bank 0, Logical Address 0xd67f6  
Nov 10 07:18:43  alarmd[6931]: Alarm set: FPC color=RED, class=CHASSIS, reason=FPC 17 Major Errors - XM Chip Error code: 0x70034
Nov 10 07:18:43  craftd[12552]:  Major alarm set, FPC 17 Major Errors - XM Chip Error code: 0x70034
  1. Check the alarm code.

2019-11-10 07:18:43 UTC   Major  FPC 17 Major Errors - XM Chip Error code: 0x70034      
  1. Verify FPC status.

user@host> show chassis fpc detail

Slot 17 information:
  State                               Offline
  <snip>

It is possible that the FPC is offline because of the errors being triggered. In case the FPC status is online, you need not perform any additional steps. The FPC being online indicates that it has recovered on its own. On checking the alarms again, you will see that the alarm has cleared from the device.

If the FPC is offline, perform the subsequent steps.

  1. Check for the alarm module on the FPC.

user@host> request pfe execute command “show cmerror module” target fpc17

This output will list a number of modules with a module number for each cmerror. Make a note of the module number.

Example Output

<snip>
   20   MPC
                  0x1a0001  0  Major  1   0   0   0   0ms  Device Voltage Level High
                  0x1a0002  0  Major  1   0   0   0   0ms  Device Current Level High
                  0x1a0003  0  Major  1   0   0   0   0ms  Bootcpld version does not match
<snip>

For the FPC in this example, the error module is 20.

  1. Check the error information and identify the reason.
user@host> request pfe execute command "show cmerror module 20 error 0x70034" target fpc17  >> Input error code as seen in alarm 
Error-id              : 0x70034
Identifier            : /pfe/0/cm/0/XM[1:0]/1/0x70034
Description           : DRAM ECC error summary indication for CPQ interface 0
State                 : enabled
PFE                   : 1
Configured Level      : Major
Default Level         : Major
Count                 : 0
Threshold             : 1
Error Limit           : 0
Occur Count           : 0
Clear Count           : 0
Last-occurred(ms ago) : 0

The cause is a hardware memory error. "ECC" stands for Error Checking and Correcting, which permits error detection as well as correction of certain errors.

Typically, ECC can detect single and dual bit errors and can correct single-bit errors on its own. The corrected errors are transparent to the system. If the message occurs just once or very sporadically, it can be ignored.

In case the FPC is offline due to the error, there is a possibility that this error is fatal in nature. In order to recover from the error, ideally, the FPC resets on its own. In case the FPC is offline with the active alarm, users can manually restart the FPC.

  1. Restart the FPC.

user@host> request chassis fpc (offline | online | restart) slot slot-number

In this example: user@host> request chassis fpc restart slot 17

In case the restart does not solve the issue or clear the alarm, open a Support case for detailed analysis. It could be a possible hardware issue. Collect the following logs for troubleshooting in addition to the ones collected above:

show log messages
show log chassisd
request execute pfe command "show nvram" target fpc<FPC#>
request execute pfe command "show syslog messages" target fpc<FPC#>