Core Dumps were generated on the customer PTX device on code 20.4R3-S1.3
PTX5000> show log messages | match dump
init: routing (PID 99731) terminated by signal number 6. Core dumped!
init: Dump Command: /bin/sh (PID 78233) started
dumpd: Core and context for rpd saved in /var/tmp/rpd.core-tarball.4.tgz
After running debug on the core dump:
(gdb) bt
#0 0x00000000cc3e1f3c in thr_kill () at thr_kill.S:2
#1 0x00000000c55709ad in _thr_send_sig (thread=<optimized out>, sig=6) at ../../../../../src/bsd/lib/libthr/thread/thr_kern.c:89
#2 0x00000000c556dd3c in _raise (sig=6) at ../../../../../src/bsd/lib/libthr/thread/thr_syscalls.c:357
#3 0x00000000cc469803 in abort () at ../../../../../src/bsd/lib/libc/stdlib/abort.c:69
#4 0x0000000001c88f15 in __task_quit__ (code=<optimized out>) at ../../../../../src/junos/lib/libjtask/base/task_quit.c:127
#5 0x0000000001c64f84 in task_quit (code=22) at ../../../../../src/junos/lib/libjtask-jthr/jtask_jthr_quit.c:44
#6 0x0000000001c9d603 in task_assert (file=0x25c78a0 "../../../../../../../../src/junos/usr.sbin/rpd/bgp/bgp_io.c", line=1601, test=0x25c7e48 "!(* __error()) || ((* __error()) == 35)") at ../../../../../src/junos/lib/libjtask/base/task_assert.c:87
#7 0x0000000000d2c42e in bgp_rt_send_message (bnp=0xbe10000, force_finalize=0 '\000') at ../../../../../../../../src/junos/usr.sbin/rpd/bgp/bgp_io.c:1601
#8 0x0000000000d30091 in bgp_rt_send_v4_flush (ss=0x7fffffff62e0, bnp=0xbe10000) at ../../../../../../../../src/junos/usr.sbin/rpd/bgp/bgp_io.c:1923
#9 0x0000000000b9a9a7 in bgp_send_flush (ss=0x7fffffff62e0, avs=0x7fffffff2e60, status=0, start=0x7fffffff6070) at ../../../../../../../../src/junos/usr.sbin/rpd/bgp/bgp_rtq.c:1166
#10 0x0000000000ba50c0 in bgp_rt_send_common (ss=0x7fffffff62e0, start=0x7fffffff6070) at ../../../../../../../../src/junos/usr.sbin/rpd/bgp/bgp_rtq.c:2967
#11 0x0000000000ba810a in bgp_rt_send_active_subr (p_num_tokens=<optimized out>, priority=<optimized out>, start=..., nexttime=<optimized out>, state_grp=<optimized out>, bgp=<optimized out>) at ../../../../../../../../src/junos/usr.sbin/rpd/bgp/bgp_rtq.c:4014
#12 bgp_rt_send_active (bgp=0x30bfa000, min_idx_by_pri=0x7fffffffb300, start=...) at ../../../../../../../../src/junos/usr.sbin/rpd/bgp/bgp_rtq.c:4326
#13 0x0000000000d25fd7 in bgp_write (job=<optimized out>) at ../../../../../../../../src/junos/usr.sbin/rpd/bgp/bgp_init.c:4419
#14 0x0000000001c81de8 in task_job_run_common (jp=0x84219ea40, jctx=0x62381c80, dispatcher=0x28de16a "background dispatch") at ../../../../../src/junos/lib/libjtask/base/task_jobs.c:808
#15 0x0000000001c83cb0 in task_job_run_job_bg (log_msg=<optimized out>, jp=<optimized out>, jobs_var=<optimized out>, scheduler=<optimized out>) at ../../../../../src/junos/lib/libjtask/base/task_jobs.c:1028
#16 task_job_bg_dispatch () at ../../../../../src/junos/lib/libjtask/base/task_jobs.c:1246
#17 0x0000000001cada5e in task_scheduler_internal (enable_slip_detector=true, no_exit=true) at ../../../../../src/junos/lib/libjtask/base/task_scheduler.c:299
#18 0x0000000001cae6b3 in task_scheduler () at ../../../../../src/junos/lib/libjtask/base/task_scheduler.c:541
#19 0x000000000086155b in main (argc=8629256, argv=0x32dc8fa <sw_version+186>) at ../../../../../../src/junos/usr.sbin/rpd/os/rpd_main.c:522
(gdb) f 7
1601 assert (!errno || (errno == EAGAIN)); /* must not be ENOBUFS because we are setting blockat limit */
This hits PR1628730.
Has been just removed by PR/1628730 fix as bogus:
PR: 1628730
Reviewed by: skolenchery
Description: Problem: RPD crash on assert in bgp_thrio area for certain errno.
Cause:
This is a too strong assert. It is possible that errno has been overwritten by
other calls in between. It is also possible that errno has never been set as
EAGAIN since bgp_rt_send_message() => bgp_sendv() => bgp_thrio_sendv() =>
bgp_thrio_write_batch() does not rely on errno to set EAGAIN return code. In
this case, the actual errno is EINVAL which is either a leftover errno or an
overwritten errno. Nevertheless, the assert does not add any value since action
is purely based on the return value stored in res.
Fix:
Fix provided in targeted release. Please refer to the PR for the fix info.