The bfd sessions can start flapping under the following scenario:

• Restart of the FPC or the FPC change on the running router.
• Many BFD sessions (scaled bfd sessions on that device) on the FPC which is being restarted/changed.

Post restart of FPC, we may see continuous flaps of BFD.

BGP and other protocols may also flap as an after-effect of BFD flapping.

In case, if we take Wireshark traces, we may be found for the same BFD session 2 hellos with different discriminator values.

Nov  5 20:35:44  TEST_MX-RE0 bfdd[9081]: %DAEMON-4-BFDD_TRAP_SHOP_STATE_DOWN: local discriminator: 130, new state: down, interface: ae30.11, peer addr: 172.28.182.xx
Nov  5 20:39:59  TEST_MX-RE0 bfdd[9081]: %DAEMON-5-BFDD_TRAP_SHOP_STATE_UP: local discriminator: 139, new state: up, interface: ae30.11, peer addr: 172.28.182.xx
Nov  5 20:40:00  TEST_MX-RE0 bfdd[9081]: %DAEMON-4-BFDD_TRAP_SHOP_STATE_DOWN: local discriminator: 139, new state: down, interface: ae30.11, peer addr: 172.28.182.xx
Nov  5 20:41:14  TEST_MX-RE0 bfdd[9081]: %DAEMON-5-BFDD_TRAP_SHOP_STATE_UP: local discriminator: 139, new state: up, interface: ae30.11, peer addr: 172.28.182.xx
Nov  5 20:41:17  TEST_MX-RE0 bfdd[9081]: %DAEMON-4-BFDD_TRAP_SHOP_STATE_DOWN: local discriminator: 139, new state: down, interface: ae30.11, peer addr: 172.28.182.xx
Nov  5 20:43:07  TEST_MX-RE0 bfdd[9081]: %DAEMON-5-BFDD_TRAP_SHOP_STATE_UP: local discriminator: 139, new state: up, interface: ae30.11, peer addr: 172.28.182.xx
Nov  5 20:43:09  TEST_MX-RE0 bfdd[9081]: %DAEMON-4-BFDD_TRAP_SHOP_STATE_DOWN: local discriminator: 139, new state: down, interface: ae30.11, peer addr: 172.28.182.xx

We can check the output of “show ppm transmissions detail | no-more” 

From below we can see multiple entries for the same IP, which is incorrect. There should be only one entry per IP.

Destination: 12.83.158.xx, Protocol: BFD, Transmission interval: 300

Distributed: TRUE, Distribution handle: 25, Distribution address: fpc8

IFL-index: 427

Replicated

Destination: 12.83.158.xx, Protocol: BFD, Transmission interval: 1000

Distributed: TRUE, Distribution handle: 1406, Distribution address: fpc9

IFL-index: 451

Replicated

Destination: 12.83.158.xx, Protocol: BFD, Transmission interval: 300

Distributed: TRUE, Distribution handle: 1446, Distribution address: fpc9

IFL-index: 451

Replicated

Destination: 2600:300:3011:200::xyz, Protocol: BFD, Transmission interval: 1000

Distributed: TRUE, Distribution handle: 1442, Distribution address: fpc9

IFL-index: 384

Replicated

Destination: 2600:300:3011:200::xyz, Protocol: BFD, Transmission interval: 150

Distributed: TRUE, Distribution handle: 1478, Distribution address: fpc9

IFL-index: 384

Replicated

Destination: 172.29.175.xy, Protocol: BFD, Transmission interval: 1000

Distributed: TRUE, Distribution handle: 1440, Distribution address: fpc9

IFL-index: 385

Replicated

Destination: 172.29.175.xy, Protocol: BFD, Transmission interval: 150

Distributed: TRUE, Distribution handle: 1486, Distribution address: fpc9

IFL-index: 385

Replicated

The issue is due to a bug in Junos due which in the scaled BFD scenario not all sessions may get cleared post FPC restart.

While distributing XMIT entry if PFE returns error and meanwhile if the XMIT entry is deleted from RE, undo code will kick in and creates stale XMIT entry. This XMIT entry will remain forever.

Below Workaround can be done to clear the issue state.

Restart the PPMD process in MW

e.g.:

> show system processes extensive | match ppm

 9069 root      20  0  725M 14540K select 3 30.1H  0.49% ppmd

> request system process terminate 9069

NOTE: Please be aware, ppmd restart will cause flap for the following processes LACP, BFD, LFM, CFM and VRRP.

IF you do not find any stale entries and still BFD is down or flapping use KB KB26746 [juniper.net] to troubleshoot further.

2024.03.19: the KB has been created
Modified it on how to find the issue and added the workaround, also added PR details