Description

The article describe a problem that might lead to traffic drop while removing BGP flowspec v4 and v6 in one commit

Symptoms

Unexpected packet drop issue during mixed ipv4 and ipv6 FlowSpec routes withdrawn
 

Example Topo:
(Traffic) >>> (et-0/0/0)[router]--- (et-0/0/1) <<< Flowspec redirect next hop 
                                |
                                +-- (et-0/0/2) <<< IGP/BGP route next hop


<< DUT received 11 inetflow routes and 2001 inet6flow routes

user@router# run show bgp summary 
Threading mode: BGP I/O
Default eBGP mode: advertise - accept, receive - accept
Groups: 2 Peers: 4 Down peers: 0

Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
172.16.12.2               100          9          7       0       0        2:30 Establ
  inetflow.0: 11/11/11/0
2001:db8:172:12::2        100         15          7       0       0        2:26 Establ
  inet6flow.0: 2001/2001/2001/0

<< Both inetflow and inet6flow routes has redirect action

route v4flow {
    match destination 172.16.100.0/24;
    then redirect 172.16.12.2;
}

route v6flow {
    match {
        destination 2001:db8:172:100::/64;
    }
    then redirect 2001:db8:172:12::2;
}

<< Shutdown both peers in one commit

user@router# show | compare rollback 0 
[edit protocols bgp group ibgp neighbor 172.16.12.2]
+      shutdown;
[edit protocols bgp group ibgp neighbor 2001:db8:172:12::2]
+      shutdown;
<< During the convergence , related traffic might be dropped as "firewall discard"

user@router-fpc2:pfe> show jnh exceptions level terse inst 0 

     Firewall
     ----------------------
     firewall discard                   DISC( 67)         4881    3816942

<< Based on the rtsockmon output, there is about 10 seconds delay between the flowspecv4 redirect nh delete and flowspecv4 filter delete
<< The delay would vary according to the scaling of flowspec routes.

[15:44:47:564.335] rpd        unknown    PI   nexthop    delete  inet nh=indr flags=0x0 uflags=0x0 idx=1048574 ifidx=0 filteridx=0 tid=0 lr_id=0   infotype = 0 fwdnhidx = 0 fwdnhtype = 0   <<< v4 INDIRECT DELETE

[15:44:57:029.859] rpd        unknown    PI   nexthop    delete  inet6 nh=indr flags=0x0 uflags=0x0 idx=1048575 ifidx=0 filteridx=0 tid=0 lr_id=0   infotype = 0 fwdnhidx = 0 fwdnhtype = 0   <<< v6 INDIRECT DELETE

[15:44:57:047.628] dfwd       self       P    firewall   delete  name=__flowspec_default_inet__ af=2 idx=65025 fulllen=2845   <<< v4 FILTER DELETE after 10sec
[15:44:57:453.828] dfwd       self       P    firewall   delete  name=__flowspec_default_inet6__ af=28 idx=65027 fulllen=437357  <<< v6 FILTER DELETE almost immediately.

Solution

The problem would happen since all flowspec routes has redirect action configured , and there is a gap while handling the delete of redirect action (indirect nexthop) and firewall filter delete.

The problem has been fixed in PR1807693 , and it can be avoid if those flowspecv4 routes and flowspecv6 routes were withdrawn in separate commits or different timestamps.. 

Modification History

2024-06-03 : Article Created