Our qfx5110-48s-4c has failed and spitting out the below errors repeatedly, and dcpfe core was also generated.
ada1:ata0:0:1:0): Retrying command, 3 more tries remain
(ada1:ata0:0:1:0): WRITE_DMA. ACB: ca 00 62 29 1b 40 00 00 00 00 28 00
(ada1:ata0:0:1:0): CAM status: ATA Status Error
(ada1:ata0:0:1:0): ATA status: 41 (DRDY ERR), error: 04 (ABRT )
(ada1:ata0:0:1:0): RES: 41 04 62 29 1b 00 00 00 00 28 00
(ada1:ata0:0:1:0): Retrying command, 2 more tries remain
(ada1:ata0:0:1:0): Retrying command, 1 more tries remain
1. DMA write errors were seen on console.
2. There DCPFE crash happened due to DMA errors and the fact that DCPFED could not access the disk.
3. The DCPFE restarted repeatedly unsuccessfully, citing ASIC initialization error, also generating a core dumps. So, the core-dumps mentioned ASIC initialization errors.
...(dcpfe getting repawned with the pid 14724) 2024-05-01T23:57:58+00:00 dcpfe0 logger: fpc Stopping dcpfe: --------------------------->restarting dcpfe 2024-05-01T23:57:58+00:00 dcpfe0 logger: has_lxc_platform is TRUE 2024-05-01T23:57:58+00:00 dcpfe0 logger: fpc Starting dcpfe with option(s)[ -D ]: 2024-05-01T23:57:58+00:00 dcpfe0 dcpfe[14724]: Successfully Connected to syslog 2024-05-01T23:57:58+00:00 dcpfe0 dcpfe[14724]: tvp_get_cpu_speed_mhz: CPU speed 1800 MHz 2024-05-01T23:58:00+00:00 dcpfe0 dcpfe[14724]: tvp_is_accton_type: Platform Name: QFX5_48X10G_4X40G_2P ... 2024-05-01T23:58:07+00:00 dcpfe0 dcpfe[14724]: bcm_sdk_init: device ID: dev: 0xb862, rev: 0x2 2024-05-01T23:58:07+00:00 dcpfe0 dcpfe[14724]: bcm_sdk_init:2978 unit:0,bus_type:2 2024-05-01T23:58:07+00:00 dcpfe0 dcpfe[14724]: bcm_sdk_init: device unit no: 0 ... 2024-05-01T23:57:18+00:00 dcpfe0 dcpfe[14724]: bcore_init: after soc_reset_init ------------------->when dcpfe restart happened it tried 3 times for asic intialization. As it was unsuccessful resulted in core-dump. 2024-05-01T23:57:19+00:00 dcpfe0 dcpfe[14724]: bcore_init: before soc_reset_init 2024-05-01T23:57:20+00:00 dcpfe0 dcpfe[14724]: bcore_init: after soc_reset_init 2024-05-01T23:57:21+00:00 dcpfe0 dcpfe[14724]: bcore_init: before soc_reset_init 2024-05-01T23:57:22+00:00 dcpfe0 dcpfe[14724]: bcore_init: after soc_reset_init ==> The crash got generated at the same time due to asic not getting detected. 23:57 /var/crash/core-dcpfe-dcpfe0-0-1714607843.core.tgz
4. The issue was resolved when a power cycle restored the DMA error condition.
View 1809372 - QFX5110 | dcpfe core | #4 0x0808d135 in panic (format_string=0xb9d6424 "Assertion failed:../../../../../src/pfe/common/drivers/dc-bcm-pic/dcbcm_sdk_core.c:3115: %s\n") at ../../../../../src/pfe/platform/tvp/tvp_ panic.c:71 - Bug database (juniper.net)