Description

The following alarm can be raised on PTX EVO device

 

user@ROUTER-re0> show chassis alarms

1 alarms currently active

Alarm time Class Description

2024-07-15 05:11:37 +03 Major FPC 5 PCI Uncorrected error on dev 0000:00:01.0

Symptoms

This alarm coincides with appearance of messages in the system log:

Jul 15 05:11:37.473 2024 ROUTER-fpc5 kernel: %KERN-6: pcieport 0000:00:01.0: AER: Uncorrected (Non-Fatal) error received: 0000:00:01.0
Jul 15 05:11:37.474 2024 ROUTER-fpc5 kernel: %KERN-3: pcieport 0000:00:01.0: AER: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID)
Jul 15 05:11:37.474 2024 ROUTER-fpc5 kernel: %KERN-4: i2c i2c-77: Re-setting i2c master: 17
Jul 15 05:11:37.474 2024 ROUTER-fpc5 kernel: %KERN-3: pcieport 0000:00:01.0: AER: device [8086:6f02] error status/mask=00004000/00000000
Jul 15 05:11:37.474 2024 ROUTER-fpc5 kernel: %KERN-3: pcieport 0000:00:01.0: AER: [14] CmpltTO    (First)
Jul 15 05:11:37.474 2024 ROUTER-fpc5 kernel: %KERN-6: pcieport 0000:00:01.0: AER: Device recovery failed
Jul 15 05:11:37.476 2024 ROUTER-fpc5 emfd-fpa[17357]: %PFE-3-EMF_FPA_FAULT_REP: createFault: Fault(Location: /Chassis[0]/Fpc[5] Module: pci_resiliency Object: 0000:00:01.0 Error: 6) reported
Jul 15 05:11:37.476 2024 ROUTER-fpc5 emfd-fpa[17357]: %PFE-5-EMF_EVO_ALARM_SET: Alarm set: PCI_AER color=red, class=CHASSIS, reason=FPC 5 PCI Uncorrected error on dev 0000:00:01.0
Jul 15 05:11:37.476 2024 ROUTER-fpc5 emfd-fpa[17357]: %PFE-3-EMF_FPA_ALARM_REP: RaiseAlarm: Alarm(Location: /Chassis[0]/Fpc[5] Module: pci_resiliency Object: 0000:00:01.0 Error: 6) reported
Jul 15 05:11:37.492 2024 ROUTER-fpc5 kernel: %KERN-3: i2c i2c-77: (17/0:0x50) i2c transaction error (0x000000ff)

 

 

 

Solution

This alarm is raised when the threshold of PCIe errors on the FPC is exceeded.

A PCI agent keeps monitoring for PCI errors.

For correctable errors, the threshold is 10

for uncorrectable errors, the threshold is 1

The CLI command "show chassis aer-statistics" can be used to show these errors, and observe how they evolve over time.

One single occurrence can be due to transient conditions, and will get cleared automatically after some time:

Jul 15 05:11:37.476 2024 ROUTER-fpc5 emfd-fpa[17357]: %PFE-3-EMF_FPA_FAULT_REP: createFault: Fault(Location: /Chassis[0]/Fpc[5] Module: pci_resiliency Object: 0000:00:01.0 Error: 6) reported
Jul 15 05:11:37.476 2024 ROUTER-fpc5 emfd-fpa[17357]: %PFE-5-EMF_EVO_ALARM_SET: Alarm set: PCI_AER color=red, class=CHASSIS, reason=FPC 5 PCI Uncorrected error on dev 0000:00:01.0

 
Jul 15 11:11:58.696 2024 ROUTER-fpc5 emfd-fpa[17357]: %PFE-5-EMF_EVO_ALARM_CLR: Alarm cleared: PCI_AER color=red, class=CHASSIS, reason=FPC 5 PCI Uncorrected error on dev 0000:00:01.0
Jul 15 11:11:58.697 2024 ROUTER-fpc5 emfd-fpa[17357]: %PFE-3-EMF_FPA_ALARM_CLR: ClearAlarm: Alarm(Location: /Chassis[0]/Fpc[5] Module: pci_resiliency Object: 0000:00:01.0 Error: 6) cleared

 

 

 

Modification History

2024-09-14 : Article Created

2025-04-25 : modify the code format