This article helps to resolve major alarm "Major RE 0 PCI Corrected error on dev 0000:00:1c.2" raised on an EVO based router

Major errors present on FPC0:

2 alarms currently active

Alarm time        Class Description

2024-08-14 22:32:05 UTC Major RE 0 PCI Corrected error on dev 0000:00:1c.2

2024-08-14 22:28:01 UTC Major FPC 0 Major Errors

FPC is up and PICs are online.

Below error reported for multiple show commands, - show chassis hardware, show platform dependency-state

error: communication failure with /re0/picd/

  1. The producer app is either down or unresponsive

    Run "show system processes node <node> | grep <app>" to check if app is running

    Run "show system application app <app>" to check the state of the app

  2. If there is a change in mastership recently, please wait for the switchover to complete for the app to be online

    Run "request chassis routing-engine master switch check" to check the status of switchover

Log messages:

Aug 14 22:27:58 2024 ROUTER1 resiliencyd[17288]: CMError: /fpc/0:0/PIC-0/0/cm/0/pic/0/PIC_User_FPGA_Fail (0x310004), scope: board, category: functional, severity: major, module: PIC-0, type: PIC User FPGA Fail

Aug 14 22:27:58 2024 ROUTER1 resiliencyd[17288]: Performing action get-state for error /fpc/0:0/PIC-0/0/cm/0/pic/0/PIC_User_FPGA_Fail (0x310004) in module: PIC-0 with scope: board category: functional level: major

Aug 14 22:28:01 2024 ROUTER1 resiliencyd[17288]: Performing action cmalarm for error /fpc/0:0/PIC-0/0/cm/0/pic/0/PIC_User_FPGA_Fail (0x310004) in module: PIC-0 with scope: board category: functional level: major

Aug 14 22:28:01 2024 ROUTER1 resiliencyd[17288]: EMF_EVO_ALARM_SET: Alarm set: FPC color=red, class=CHASSIS, reason=FPC 0 Major Errors



Aug 14 22:28:01 2024 ROUTER1 resiliencyd[17288]: CMError: /fpc/0:1/PIC-1/0/cm/0/pic/1/PIC_User_FPGA_Fail (0x310004), scope: board, category: functional, severity: major, module: PIC-1, type: PIC User FPGA Fail

Aug 14 22:28:01 2024 ROUTER1 resiliencyd[17288]: Performing action get-state for error /fpc/0:1/PIC-1/0/cm/0/pic/1/PIC_User_FPGA_Fail (0x310004) in module: PIC-1 with scope: board category: functional level: major

Aug 14 22:28:01 2024 ROUTER1 resiliencyd[17288]: Performing action cmalarm for error /fpc/0:1/PIC-1/0/cm/0/pic/1/PIC_User_FPGA_Fail (0x310004) in module: PIC-1 with scope: board category: functional level: major

Aug 14 22:28:01 2024 ROUTER1 resiliencyd[17288]: Cmerror Op Set: PIC-1: Supercon FPGA Access Failed! (URI: /fpc/0:1/PIC-1/0/cm/0/pic/1/PIC_User_FPGA_Fail)



Aug 14 22:28:01 2024 ROUTER1 resiliencyd[17288]: CMError: /fpc/0/evo-cda-bt/0/cm/0/btchip/0/btchip_pwrup_pll_lock_fail (0x452102), scope: pfe, category: functional, severity: major, module: btchip, type: btchip_pwrup_pll_lock_fail

Aug 14 22:32:05 2024 ROUTER1 emfd-fpa[14675]: EMF_FPA_FAULT_REP: createFault: Fault(Location: /Chassis[0]/Re[0] Module: pci_resiliency Object: 0000:00:1c.2 Error: 1) reported

Aug 14 22:32:05 2024 ROUTER1 picd[17061]: signal_buserr_handler: Signal action not correct, continue still Signal code: 2

Aug 14 22:32:05 2024 ROUTER1 last message repeated 30 times

Aug 14 22:32:05 2024 ROUTER1 emfd-fpa[14675]: EMF_EVO_ALARM_SET: Alarm set: PCI_AER color=red, class=CHASSIS, reason=RE 0 PCI Corrected error on dev 0000:00:1c.2

To clear the alarms:

1. Restart the router via CLI if possible

2. If not possible cold reboot the router (power cord unplug or breaker restart)

3. If after the restart the router is still presenting the same alarms then proceed to RMA the RE/chassis.