The article explains the meaning of " PE Error code: 0x210532 " reason for FPC on QFX and PTX series devices. The article also explains steps to overcome the alarm.
PE Error code: 0x210532
Active alarm:
user@host> show chassis alarms 1 alarms currently active Alarm time Class Description 2020-06-20 10:41:20 PDT Major FPC 1 Major Errors - PE Error code: 0x210532
Error Log:
Jun 20 10:41:20.546 fpc0 Cmerror Op Set: PE Chip: PE1[1]: FABRORD[0-Data]:ecc: 0x0007f800: Timeout timestamp database Jun 20 10:41:20.585 alarmd[12152]: Alarm set: FPC color=RED, class=CHASSIS, reason=FPC 0 Major Errors - PE Error code: 0x210532 Jun 20 10:41:20.585 craftd[11846]: Receive FX craftd set alarm message: color: 1 class: 100 object: 104 slot: 0 silent: 0 short_reason=FPC 0 Major Errors long_reason=FPC 0 Major Errors - PE Error code: 0x210532 id=150995048 reason=150994944 Jun 20 10:41:20.585 craftd[11846]: Major alarm set, FPC 0 Major Errors - PE Error code: 0x210532
These alarms are due to the Timeout in the Timestamp database which might occur during Fabric Re-ordering. If the PFE is active, the corrupted memory location will be overwritten with the correct timestamp on the new incoming fabric cell.
Ideally, these errors are non-impacting in nature.
The log below indicates the PE chip 1 error.
Jun 20 10:41:20.546 fpc0 Cmerror Op Set: PE Chip: PE1[1]: FABRORD[0-Data]:ecc: 0x0007f800: Timeout timestamp database
Check PFE Error Statistics:
user@host> show pfe statistics error Slot 0 <snip> Slot 1 PEChip Errors: 6 -------------- PEChip 0 IGP errs received from IQM/DBM : 1
IGP errs received from IQM/DBM : 28 PEChip 2 IGP errs received from IQM/DBM : 1 PEChip 3 IGP errs received from IQM/DBM : 0 PEChip 4 IGP errs received from IQM/DBM : 9 PEChip 5 IGP errs received from IQM/DBM : 28
IGP errs received from IQM/DBM : 28 PEChip 2 IGP errs received from IQM/DBM : 1 PEChip 3 IGP errs received from IQM/DBM : 0 PEChip 4 IGP errs received from IQM/DBM : 9
PEChip 5 IGP errs received from IQM/DBM : 28
user@host> request pfe execute command “show cmerror module brief” target fpc1
-------------------------------------- Module Name Active Errors --------------------------------------- 1 PCIe Error 0 2 CPU Error 0 3 Eth Port Error 0 4 Host Loopback 0 5 Generic HMC 0 6 TOE-PE-0:0:0 0 7 TOE-PE-0:0:1 0 8 TOE-PE-0:0:2 0 9 PE Chip 5 10 TOE-PE-1:0:0 0 11 TOE-PE-1:0:1 0 12 TOE-PE-1:0:2 0 13 TOE-PE-2:0:0 0 14 TOE-PE-2:0:1 0 15 TOE-PE-2:0:2 0 16 TOE-PE-3:0:0 0 17 TOE-PE-3:0:1 0 18 TOE-PE-3:0:2 0 19 TOE-PE-4:0:0 0 20 TOE-PE-4:0:1 0 21 TOE-PE-4:0:2 0 22 TOE-PE-5:0:0 0 23 TOE-PE-5:0:1 0 24 TOE-PE-5:0:2 0 25 BCM Switch 0 26 FPC 0
Check the command above multiple times and monitor the error count increase. If the error is not incrementing and the alarm has cleared on its own, you may choose to ignore this. If the alarm persists and the error rate is high, please proceed further.
Verify detailed FPC errors:
user@host> show chassis fpc errors FPC Level Occurred Cleared Threshold Action-Taken Action 0 Minor 0 0 10 0 LOG| Major 0 0 1 10 GET STATE|ALARM| Fatal 0 0 1 0 DISABLE PFE Pfe-State: pfe-0 -ENABLED | pfe-1 -ENABLED | pfe-2 -ENABLED | pfe-3 -ENABLED | pfe-4 -ENABLED | pfe-5 -ENABLED | 1 Minor 0 0 10 0 LOG| Major 5 0 1 0 GET STATE|ALARM| Fatal 0 0 1 0 DISABLE PFE Pfe-State: pfe-0 -ENABLED | pfe-1 -ENABLED | pfe-2 -ENABLED | pfe-3 -ENABLED | pfe-4 -ENABLED | pfe-5 -ENABLED | 2 Minor 0 0 10 0 LOG| Major 0 0 1 0 GET STATE|ALARM| Fatal 0 0 1 0 DISABLE PFE Pfe-State: pfe-0 -ENABLED | pfe-1 -ENABLED | pfe-2 -ENABLED | pfe-3 -ENABLED | pfe-4 -ENABLED | pfe-5 -ENABLED | 3 Minor 0 0 10 0 LOG| Major 0 0 1 0 GET STATE|ALARM| Fatal 0 0 1 0 DISABLE PFE Pfe-State: pfe-0 -ENABLED | pfe-1 -ENABLED | pfe-2 -ENABLED | pfe-3 -ENABLED | pfe-4 -ENABLED | pfe-5 -ENABLED |
From the command above, you can confirm that all the PFE states are enabled for FPC 1 and the Major error indicates that the error occurred 5 times. This matches the error count seen on CMERROR MODULE for PE CHIP. Since this error in non-impacting in nature, this is an expected behavior. There would not be any harm caused to the PFE state.
Clear the alarm:
user@host >request chassis fpc restart slot <slot-number>
Here,
user@host> request chassis fpc restart slot 1
Verify the device alarm:
Once the restart is done, verify the outputs again to see the issue is cleared.
user@host> show chassis alarms user@host> show chassis fpc errors user@host> request pfe execute command “show cmerror module brief” target fpc1
c errors user@host> request pfe execute command “show cmerror module brief” target fpc1
If the alarm and the active errors still exist post restart, contact JTAC for further investigation.