This article explain about continuously core-dump generated and RPD crashed leading to all protocol flaps , whenever list-nexthop is configured with aggregate route.

++++++++++++++++core-dumps+++++++++++++++

ca_usr@C-BJSBG-01> show system core-dumps

-rw-r--r-- 1 root wheel 140776543 Apr 3 2024 /var/crash/core-C-BJSBG-01-fpc1-smpc.elf.0.tgz

-rw-r--r-- 1 root wheel 134542528 Apr 24 2024 /var/crash/core-C-BJSBG-01-fpc1-smpc.elf.1.tgz

-rw-r--r-- 1 root wheel 140358610 Apr 3 2024 /var/crash/core-C-BJSBG-01-fpc3-smpc.elf.0.tgz

-rw-r--r-- 1 root wheel 131670036 Apr 23 2024 /var/crash/core-C-BJSBG-01-fpc3-smpc.elf.1.tgz

-rw-r--r-- 1 root wheel 141278695 Apr 3 2024 /var/crash/core-C-BJSBG-01-fpc4-smpc.elf.0.tgz

-rw-r--r-- 1 root wheel 133346821 Apr 23 2024 /var/crash/core-C-BJSBG-01-fpc4-smpc.elf.1.tgz

-rw-r--r-- 1 root wheel 141951641 Apr 3 2024 /var/crash/core-C-BJSBG-01-fpc9-smpc.elf.0.tgz

-rw-r--r-- 1 root wheel 135500719 Apr 23 2024 /var/crash/core-C-BJSBG-01-fpc9-smpc.elf.1.tgz

-rw------- 1 root wheel 7338735 Apr 28 09:50 /var/tmp/ifinfo.core.0.gz

-rw------- 1 root wheel 7341008 Jun 8 21:47 /var/tmp/ifinfo.core.1.gz

-rw-rw---- 1 root wheel 283794505 Jul 30 09:47 /var/tmp/rpd.core-tarball.0.tgz

-rw-rw---- 1 nobody wheel 777279 Sep 14 2022 /var/tmp/pics/mspmand.core.ms11.0.gz

-rw-rw---- 1 nobody wheel 777279 Sep 14 2022 /var/tmp/pics/mspmand.core.ms12.0.gz

/var/crash/kernel.*: No such file or directory

/var/jails/rest-api/tmp\/*core*: No such file or directory

/tftpboot/corefiles\/*core*: No such file or directory

(gdb) bt
#0  thr_kill () at thr_kill.S:4
#1  0x00000000081ffbc1 in __raise (s=s@entry=6) at /volume/build/junos/occam/llvm-13.0/sandbox-bsd15-244-20250502-4791b2b/freebsd/bsd15_244/20250502.082339_builder_bsd15_244.4791b2b/src/lib/libc/gen/raise.c:48
#2  0x000000000829d849 in abort () at /volume/build/junos/occam/llvm-13.0/sandbox-bsd15-244-20250502-4791b2b/freebsd/bsd15_244/20250502.082339_builder_bsd15_244.4791b2b/src/lib/libc/stdlib/abort.c:61
#3  0x00000000081e5e51 in __assert (func=<optimized out>, file=<optimized out>, line=<optimized out>, failedexpr=<optimized out>) at /volume/build/junos/occam/llvm-13.0/sandbox-bsd15-244-20250502-4791b2b/freebsd/bsd15_244/20250502.082339_builder_bsd15_244.4791b2b/src/lib/libc/gen/assert.c:47
#4  0x000000000165a199 in bgp_add_transit_routes (af_ctx=af_ctx@entry=0x85782d0, rtih=rtih@entry=0x9ee0000, label_node=<optimized out>, label_node@entry=0x188b9700, rt=rt@entry=0x645a63f0, rt_forw=rt_forw@entry=0x645a63f0, nh=0x18043b5c, nlri_flags=0, count=0, update_operation=0 '\\000') at ../../../../../../../../../src/layer3/usr.sbin/rpd/bgp/bgp_label_af.c:6643
#5  0x000000000165cf67 in bgp_out_label_node_populate (af_ctx=af_ctx@entry=0x85782d0, label_node=0x188b9700, key=key@entry=0x7fffffffbf70, rtih=rtih@entry=0x9ee0000, rt=rt@entry=0x645a63f0, rt_forw=rt_forw@entry=0x645a63f0, ifa=0x0, mhint=0x0, nhop_addr=0x0, nlri_flags=0, lbl_range=1, label_reuse=0 '\\000', nexthops=0x18043b5c, desired_label_applicable=0 '\\000', desired_label=<optimized out>, label_alloc_fallback_reject=<optimized out>, free_on_failure=1 '\\001', bgp_psid_label=4294967295) at ../../../../../../../../../src/layer3/usr.sbin/rpd/bgp/bgp_label_af.c:8301
#6  0x0000000001652cfe in bgp_out_label_node_add (af_ctx=0x85782d0, key=0x7fffffffbf70, rtih=0x9ee0000, rt=0x645a63f0, rt_forw=0x645a63f0, lbl_range=1, nexthops=0x18043b5c, desired_label_applicable=0 '\\000', desired_label=4294967295, label_alloc_fallback_reject=<optimized out>, ifa=<optimized out>, mhint=<optimized out>, nhop_addr=<optimized out>, nlri_flags=<optimized out>, bgp_psid_label=<optimized out>) at ../../../../../../../../../src/layer3/usr.sbin/rpd/bgp/bgp_label_af.c:8615
#7  bgp_out_label_add (af_ctx=0x85782d0, instance=instance@entry=0x18043b5c, rtih=rtih@entry=0x9ee0000, rt=<optimized out>, rt@entry=0x0, rt_forw=0x645a63f0, nexthops=0x18043b5c, ifa=0x0, nhop_addr=0x0, prtparms=0x7fffffffcdf8, mhint=0x0, exist_afmet=0x5a50a1a8, omet=0x533d6930, mode=0, bgp=0x33c4a000, nlri_flags=0, use_gw_sel=1 '\\001', bgp_psid_label=4294967295) at ../../../../../../../../../src/layer3/usr.sbin/rpd/bgp/bgp_label_af.c:9415
#8  0x00000000016486d4 in bgp_find_out_label (af_ctx=0x85782d0, rtih=0x9ee0000, inst=0x9ee0000, rt=<optimized out>, prtparms=<optimized out>, bs=<optimized out>, mhint=0x0, nlri_flags=0, imet=0x5a50a1a8, omet=0x533d6930) at ../../../../../../../../../src/layer3/usr.sbin/rpd/bgp/bgp_label_af.c:10795
#9  0x00000000015aaf37 in bgp_vpncommon_af_outmetrics (af_ctx=<optimized out>, rt=rt@entry=0x645a63f0, prtparms=<optimized out>, prtparms@entry=0x7fffffffcdf8, outmets=<optimized out>, outmets@entry=0x7fffffffd738, bs=bs@entry=0x33c4ad18, afm=afm@entry=0x0, afmets=0x7fffffffcd70, afi=BGP_AFI_INETVPN_UNI, metrics_block=0x9a28700, local_srv6_sid=0 '\\000', local_srv6_label=3) at ../../../../../../../../../src/layer3/usr.sbin/rpd/bgp/bgp_inetvpn_af.c:2073
#10 0x00000000015acb13 in bgp_inetvpn_uni_setup_af_outmetrics (bs=0x0, rt=0x6, prtparms=0x0, outmets=0x827702a <thr_self+10>, mhint=0x0, afmets=0x7fffffffcd70, local_srv6_sid=0 '\\000', local_srv6_label=3) at ../../../../../../../../../src/layer3/usr.sbin/rpd/bgp/bgp_inetvpn_af.c:2256
#11 0x00000000016b786d in bgp_rt_policy_afmetrics (bgpaf=0x85782d0, bs=bs@entry=0x33c4ad18, new_rt=0x645a63f0, prtparms=prtparms@entry=0x7fffffffcdf8, entp=0x596f3138, rtotsi=<optimized out>, nmetrics=0x7fffffffd738, af_metrics_changed=0x7fffffffd753 "", local_srv6_sid=0 '\\000', local_srv6_label=3) at ../../../../../../../../../src/layer3/usr.sbin/rpd/bgp/bgp_rt.c:6835
#12 0x00000000016b6983 in bgp_rt_policy_rt (rth=<optimized out>, args=<optimized out>) at ../../../../../../../../../src/layer3/usr.sbin/rpd/bgp/bgp_rt.c:7821
#13 0x00000000016ba035 in bgp_rt_policy (rtl=<optimized out>, args=0x7fffffffda00) at ../../../../../../../../../src/layer3/usr.sbin/rpd/bgp/bgp_rt.c:8647
#14 bgp_rt_policy_group (bgp=<optimized out>, rtl=0xd28a000, what=BRTUPD_FLASH, ribix=<optimized out>) at ../../../../../../../../../src/layer3/usr.sbin/rpd/bgp/bgp_rt.c:8703
#15 bgp_rt_group_update (bs=0x33c4ad18, rtl=0xd28a000, what=BRTUPD_FLASH, ribix=<optimized out>) at ../../../../../../../../../src/layer3/usr.sbin/rpd/bgp/bgp_rt.c:9096
#16 bgp_flash_cb (tp=<optimized out>, rtt=<optimized out>, list=<optimized out>, data=<optimized out>, opcode=<optimized out>) at ../../../../../../../../../src/layer3/usr.sbin/rpd/bgp/bgp_rt.c:9135
#17 0x00000000029146b5 in rt_flash_update_callback (rtt=<optimized out>, list=<optimized out>, data=<optimized out>) at ../../../../../../../../src/layer3/usr.sbin/rpd/lib/rt/rt_table.c:9097
#18 0x00000000028bccd9 in rtlist_walker_internals (context=0x5066c308) at ../../../../../../../../src/layer3/usr.sbin/rpd/lib/rt/rt_list.c:659
#19 0x00000000028bbc3a in rtlist_walker_job (jp=0x11883ee0) at ../../../../../../../../src/layer3/usr.sbin/rpd/lib/rt/rt_list.c:703
#20 0x0000000002ba251c in task_job_run_common (jp=jp@entry=0x11883ee0, jctx=jctx@entry=0x665afac0, dispatcher=0x8799ab "background dispatch") at ../../../../../../src/daemon-infra/lib/libjtask/base/task_jobs.c:1000
#21 0x0000000002ba33f6 in task_job_run_job_bg (scheduler=0x306eb10 <task_var_main_context+3008>, jobs_var=0x306e8a0 <task_var_main_context+2384>, jp=0x11883ee0, log_msg=<optimized out>) at ../../../../../../src/daemon-infra/lib/libjtask/base/task_jobs.c:1220
#22 task_job_bg_dispatch () at ../../../../../../src/daemon-infra/lib/libjtask/base/task_jobs.c:1436
#23 0x0000000002bdd4ba in task_scheduler_internal (enable_slip_detector=true, no_exit=false) at ../../../../../../src/daemon-infra/lib/libjtask/base/task_scheduler.c:314
#24 0x0000000002bdcfee in task_scheduler () at ../../../../../../src/daemon-infra/lib/libjtask/base/task_scheduler.c:584
#25 0x000000000117eb46 in rpd_main (argc=<optimized out>, argv=<optimized out>) at ../../../../../../../src/layer3/usr.sbin/rpd/os/rpd_main.c:675
#26 main (argc=<optimized out>, argv=<optimized out>) at ../../../../../../../src/layer3/usr.sbin/rpd/os/rpd_main.c:444
(gdb)

+++++++++configurations++++++++++
set policy-options policy-statement export-RR-IPv6 term reject then reject
set protocols bgp multipath list-nexthop
set protocols bgp source-packet-routing srv6 locator loc-core end-dt4-sid fc00:a000:4008:0002::
set protocols bgp group member-client-RR-IPv6 type internal
set protocols bgp group member-client-RR-IPv6 metric-out igp
set protocols bgp group member-client-RR-IPv6 local-address 2400:9800:7000:4008::
set protocols bgp group member-client-RR-IPv6 family inet unicast local-ipv4-address 202.152.246.188
set protocols bgp group member-client-RR-IPv6 family inet unicast extended-nexthop
set protocols bgp group member-client-RR-IPv6 family inet unicast extended-nexthop-color
set protocols bgp group member-client-RR-IPv6 family inet unicast advertise-srv6-service
set protocols bgp group member-client-RR-IPv6 family inet unicast accept-srv6-service
set protocols bgp group member-client-RR-IPv6 family inet-vpn unicast extended-nexthop
set protocols bgp group member-client-RR-IPv6 family inet-vpn unicast extended-nexthop-tunnel
set protocols bgp group member-client-RR-IPv6 family inet-vpn unicast advertise-srv6-service
set protocols bgp group member-client-RR-IPv6 family inet-vpn unicast accept-srv6-service
set protocols bgp group member-client-RR-IPv6 family inet6 unicast extended-nexthop-color
set protocols bgp group member-client-RR-IPv6 family inet6 unicast advertise-srv6-service
set protocols bgp group member-client-RR-IPv6 family inet6 unicast accept-srv6-service
set protocols bgp group member-client-RR-IPv6 family inet6-vpn unicast advertise-srv6-service
set protocols bgp group member-client-RR-IPv6 family inet6-vpn unicast accept-srv6-service
set protocols bgp group member-client-RR-IPv6 family evpn signaling advertise-srv6-service
set protocols bgp group member-client-RR-IPv6 family evpn signaling accept-srv6-service
set protocols bgp group member-client-RR-IPv6 authentication-key "$9$nnXW6u1MWxs4JWLJDjqQzcylM-bY2aHkPgo"
set protocols bgp group member-client-RR-IPv6 neighbor 2400:9800:7000:4044:: description RR-BJBNH-01
set protocols bgp group member-client-RR-IPv6 neighbor 2400:9800:7000:30b0:: description RR-SBRKO-01
set protocols bgp group member-client-RR-IPv6 vpn-apply-export
set protocols bgp group member-client-RR-IPv6 export export-RR-IPv6

Engineering team just updated that the core happens with the Aggregate route towards regular L3VPN peers. The list-nexthop knob causes the Aggregate-route to have a List-NH, eventually causing the crash. Because Aggregate-route with List-NH is not supported in the code.

We have tested in our VMM lab with discard instead of policy and this resolved the issue post which no more core-dumps seen.

As Engineering already mentioned that list-nexthop and generate route together is not supported in the junos code customer is running. And this configuration together is the trigger and actual root cause of the core-dumps generated.

The list-nexthop knob causes the Aggregate-route to have a List-NH, eventually causing the crash. Because Aggregate-route with List-NH is not supported in the code.

Solution:
[email protected]# show | compare rollback 1
[edit routing-instances red routing-options generate route 0.0.0.0/0]
+      discard;

If you configure this, then the RPD core is avoided, and the VRF-table-label is advertised for the 0/0 route. 

Resolved-In

evo:25.2R2-EVO evo:25.4R1-EVO junos:24.4R2-S1 junos:25.2R2