Description

A router running Junos Evolved is seeing an issue with BGP neighbor stability over IPv6 when there is an influx off TTL=1 traffic to the router.

 

For example, DDoS kicks in for TTL=1 indicating an increase in traffic with that time to live:

 

user@router> show ddos-protection protocols statistics detail
Packet types: 86, Received traffic: 26, Currently violated: 0
...

Protocol Group: TTL

 Packet type: aggregate (Aggregate for all TTL exception traffic)
  System-wide information:
   Aggregate bandwidth is no longer being violated
    Last violation started at: 2024-01-06 14:16:25 UTC
    Last violation ended at:  2024-01-23 16:00:24 UTC
    Duration of last violation: 2w3d 01:43 Number of violations: 1048
   Received: 416047021      Arrival rate:   320 pps
   Dropped:  65604914      Max arrival rate: 175273 pps
  Routing Engine information:
   Aggregate policer is no longer being violated
    Last violation started at: 2024-01-06 14:16:25 UTC
    Last violation ended at:  2024-01-06 14:16:25 UTC
    Duration of last violation: 00:00:00 Number of violations: 2
   Received: 137064858      Arrival rate:   62 pps
   Dropped:  2          Max arrival rate: 391 pps
    Dropped by individual policers: 0
    Dropped by aggregate policer:  2
  FPC slot 0 information:
   Aggregate policer is no longer being violated
    Last violation started at: 2024-01-23 05:34:20 UTC
    Last violation ended at:  2024-01-23 16:00:24 UTC
    Duration of last violation: 10:26:04 Number of violations: 1046
   Received: 416047021      Arrival rate:   320 pps
   Dropped:  65604912      Max arrival rate: 175273 pps
    Dropped by individual policers: 0
    Dropped by aggregate policer:  65604912
    Dropped by flow suppression:  0

When that trafifc is present, IPv6 BGP sessions flap...note the sessions in OpenSent or OpenConfirm:

user@router> show bgp summary | match fc00:668:0:3
fc00:668:0:3::1030:52 42459 0 0 0 0 2w6d 12:10:06 Active
fc00:668:0:3::1030:62 44814 8 492 177342 419 1:28 Establ
fc00:668:0:3::1030:92 31042 0 0 0 391 37 OpenSent
fc00:668:0:3:ffff:0:8d88:65aa 13335 16 332 176489 437 8 Establ
fc00:668:0:3:ffff:0:d5fe:d946 60447 0 0 0 0 2w6d 12:10:06 Connect
fc00:668:0:3:ffff:1:0:616 32934 6 134 0 422 17 Establ
fc00:668:0:3:ffff:1:0:61a 32934 7 147 0 433 44 Establ
fc00:668:0:3:ffff:1:0:1852 60068 1 1 0 421 2:44 OpenConfirm
fc00:668:0:3:ffff:1:0:21fe 29049 2 0 0 416 51 Active
fc00:668:0:3:ffff:1:0:233a 396998 0 0 0 0 2w6d 12:10:06 Idle

Or, the messages log is full of neighbor state change logs:

user@router> show log messages| match holdtime | last 10

Jan 23 14:56:37 router rpd[11094]: RPD_BGP_NEIGHBOR_STATE_CHANGED: BGP peer fc00:501b:100:1a::1 (External AS 65000) changed state from Established to Idle (event HoldTime) (instance master)
Jan 23 14:56:37 router rpd[11094]: bgp_io_mgmt_cb:3032: NOTIFICATION sent to fc00:501b:100:1a::1 (External AS 65000): code 4 (Hold Timer Expired Error), Reason: holdtime expired for fc00:501b:100:1a::1 (External AS 65000), socket buffer sndcc: 1659 rcvcc: 0, hold timer 90s, hold timer remain 0s, last sent 1s, TCP port (local 58488, remote 179)
Jan 23 14:57:35 router rpd[11094]: RPD_BGP_NEIGHBOR_STATE_CHANGED: BGP peer fc00:41a8:f00:2::69 (External AS 65001) changed state from Established to Idle (event HoldTime) (instance master)
Jan 23 14:57:35 router rpd[11094]: bgp_io_mgmt_cb:3032: NOTIFICATION sent to fc00:41a8:f00:2::69 (External AS 65001): code 4 (Hold Timer Expired Error), Reason: holdtime expired for fc00:41a8:f00:2::69 (External AS 65001), socket buffer sndcc: 32240 rcvcc: 0, hold timer 90s, hold timer remain 0s, last sent 4s, TCP port (local 49632, remote 179)
Jan 23 14:58:08 router rpd[11094]: RPD_BGP_NEIGHBOR_STATE_CHANGED: BGP peer fc00:668:0:3::1030:62 (External AS 65002) changed state from Established to Idle (event HoldTime) (instance master)
Jan 23 14:58:08 router rpd[11094]: bgp_io_mgmt_cb:3032: NOTIFICATION sent to fc00:668:0:3::1030:62 (External AS 65002): code 4 (Hold Timer Expired Error), Reason: holdtime expired for fc00:668:0:3::1030:62 (External AS 65002), socket buffer sndcc: 26752 rcvcc: 0, hold timer 90s, hold timer remain 0s, last sent 8s, TCP port (local 38340, remote 179)

Symptoms

IPv6 BGP Neighbor instability when there is an influx of TTL=1 traffic to the router

Solution

This problem is tied to PR1732443:
 

https://prsearch.juniper.net/PR1732443
 

BGP IPv6 packets are incorrectly being placed within the same queue as TTL=1 packets instead of the L3 High Priority Queue where they should be, leading to the neighbor instability.

With the fix within the PR, the correct queue is used.

Modification History

2024-01-30 : Article Created