Description

An RPD core dump was seen on a PTX system. Messages log reported the following:

 

systemd[1]: routing.service: Main process exited, code=dumped, status=11/SEGV
sysman[7523]: SYSTEM_APP_OFFLINE_EVENT: App went offline re0-routing
sysman[7523]: SYSTEM_APP_OFFLINE_EVENT: App went offline re0-rpdagent
systemd[1]: routing.service: Failed with result 'core-dump'.
systemd[1]: rpdagent.service: Succeeded.
systemd[1]: routing.service: Service RestartSec=100ms expired, scheduling restart.
systemd[1]: routing.service: Scheduled restart job, restart counter is at 5.
systemd[1]: rpdagent.service: Service RestartSec=100ms expired, scheduling restart.
systemd[1]: rpdagent.service: Scheduled restart job, restart counter is at 5.
systemd[1]: Stopped "Routing Protocol Daemon on RE".
systemd[1]: Started "Routing Protocol Daemon on RE".
systemd[1]: Stopped "Routing Protocol Daemon Agent (RPD agent) on RE".
Apr 4 01:49:49 cr3-was1 systemd[1]: Started "Routing Protocol Daemon Agent (RPD agent) on RE".
audit[5298]: ANOM_ABEND auid=4294967295 uid=0 gid=0 ses=4294967295 pid=5298 comm="rpd" exe="/usr/sbin/rpd" sig=11 res=1
kernel: rpd[5298]: segfault at 8 ip 0000563038648eb0 sp 00007ffec055f9d0 error 4 in rpd[56303729c000+1cca000]
kernel: Code: 8b 15 8c 0a c9 00 48 8b 85 f8 fc ff ff 48 8b 3c 10 e8 04 0a 20 00 41 c6 47 58 01 e9 62 fa ff ff 66 2e 0f 1f 84 00 00 00 00 00 <4d> 8b 65 08 e9 d2 fa ff ff 0f 1f 80 00 00 00 00 48 8d 15 dd bb c2
kernel: audit: type=1701 audit(1712195351.188:3678): auid=4294967295 uid=0 gid=0 ses=4294967295 pid=5298 comm="rpd" exe="/usr/sbin/rpd" sig=11 res=1
root[5062]: calling exit_handler from savecore.sh
root[5261]: savecore.sh: Started saving core for pid 5298

 

Symptoms

RPD restart and core dump is generated. 

Solution

The backtrace and fault link back to PR1785884:

 

https://prsearch.juniper.net/PR1785884

 

The PR is triggered by an issue with RPD not getting an ack from the PFE for processing entries in the KRT queue.

Modification History

2024-04-26 : Article Created