Description

Customer upgraded the software on an SD_WAN related device connected to ACX7100 setup and he started facing the BGP flapping.

Symptoms

The interfaces experienced more ARP broadcast packets getting inside are found using the below parameter under "show interface extensive" option with multiple iterations, you need to check for broadcast packet rate to understand the delta.

 

Physical interface: et-0/0/29, Enabled, Physical link is Up

   Broadcast packets                86266331        61382380

 

Physical interface: et-0/0/29, Enabled, Physical link is Up

   Broadcast packets                86315716        61382425

 

 

Solution

  • Since this EVO software, we cannot interpret using "show policer | match default" for any ARP storms, instead you need to login into FPC AFT shell using below command

 

root@acx7100> start shell pfe network fpc0

 

  • we could see ARP packet rate was quite high from EVOPFEMAND shell at a rate of 2400 and above pps

 

root@acx7100:pfe> show evo-pfemand host pkt-stats 

 

------ ------ ------------- --------- --------------- --------------- ----------- --------------- --------------------

-------

UNIT  CMICQ   Pkt    Pkt_Rate    Bytes    Drp Pkts    DrpPktsRate  DrpBytes        Proto(s)

------ ------ ------------- --------- --------------- --------------- ----------- --------------- --------------------

-------

    

0    201  292711293   2400    18753783947   1536692     0      98348444    arp  

UNIT  CMICQ   Pkt    Pkt_Rate    Bytes    Drp Pkts    DrpPktsRate  DrpBytes        Proto(s)

------ ------ ------------- --------- --------------- --------------- ----------- --------------- --------------------

-------

    

0   201  292736266   2877    18755382227   1536692     0      98348444    arp   

 

  • under the DDOS options, we do observe ARP violations were high

Group      Policer     Id   BWidth Burst P Q Ena Rcvd Packets Drops     Rate(pps) States Scfd Flow

arp       aggregate    0x3400 500   512  2 - on 293001643   15497955   2520   viol  off  0  

 

 

Fixing this ARP packet storm by disabling the customer's Layer2 extension on the downstream has resolved the BGP flaps.

Modification History

2024-05-22 : Article Created