The following article describes a specific issue that involves a monitoring system sending ping probes to verify static route viability.

Customer scenario:

• A firewall is connected to two spines QFX5120-32C via an ESI-LAG with several static routes configured to use ping-based monitors to check the viability of the routes.
• The static routes have specific next hops with pings to these IPs used for monitoring.
• The issue occurs when both path monitors fail simultaneously. During this time, ping probes from the monitoring system to the spine main IRB interface fail, while the ping probes to the firewall remain successful.

[monitoring system]—--------[SPINE1 / SPINE2]------ESI-LAG---------[FIREWALL]

The monitoring system sends ping probes every 10 seconds, the spines stopped responding to these pings when the issue occurs, while the ping towards the firewall remains unaffected.

Investigation and findings that led to the resolution of the issue

• Between the days and hours mentioned by the customer, there is a log messge showing up "EVPN_MAC_IP_THRESHOLD_REACHED: EVPN MAC IP threshold reached". This log is addressed in KB Behavior changes in 22.2R3-S3 due to the introduction of evpn mac-ip-limit | Salesforce; there is a workaround indicating the customer can verify the number of IPs learned per MAC using the commands:

root@switch> show ethernet-switching mac-ip-table | match xx:xx:xx:xx:xx:xx | count. 

If they are more than 200, the customer can use this command to increase the limit:

root@switch# set protocols evpn mac-ip-limit <number>

• We also encountered a lot of ddos violations triggered by L3NHOP

 jddosd[10781]: DDOS_PROTOCOL_VIOLATION_SET: Warning: Host-bound traffic for protocol/exception L3NHOP:aggregate exceeded its allowed bandwidth at fpc 0 for 240 times, started at 2024-10-16 22:30:59 CDT

DDOS messages are not an indication of a problem with the switch itself, they are just informational and it’s an assurance that the control plae is being protected. The intention is that the excess of control traffic packets is dropped to preserve as early as possible system resources so that the Routing Engine receives only the amount of traffic that it can process. DDOS does not represent a problem for the packet forwarding-engine either.

• I also spotted some bgp logs about 2 unconfigured neighbors:

rpd[10696]: bgp_listen_accept: Connection attempt from unconfigured neighbor:

The bgp failing status could make the switch performing exceeding route lookups as per the next hop is in a hold or reject state. It was suggested to ignore packets from these IPs by implementing the following filter:

set firewall family inet filter BGP term 1 from source-address <ip-address>

set firewall family inet filter BGP term 1 from destination-port bgp

set firewall family inet filter BGP term 1 then reject ------------------------>>> Block bgp connections from specific source ip address

set firewall family inet filter BGP term 2 then accept  ------------------------>>> Allows all other traffic not matching terms 1 conditions such as ospf , dhcp , bgp etc.

Important: We highly suggest performing the changes during a maintenance window for testing proposes and apply commit confirmed 5 minutes.