Description

Customers may notice that QFX5120 switches are randomly dropping transit DNS traffic in an EVPN/VxLAN environment.

This article gives the reason for such behavior and how to fix the issue.

 

Symptoms

QFX5120 switches may be seen to randomly drop DNS traffic, especially DNS responses from the DNS server even though the DNS client and server are directly connecting to the switch.

Example Topology 

DNS client --- QFX5120 --- DNS server

Customers may find that DNS responses are being blackholed in the switch as shown in the following output of firewall counters applied to the port that is connecting to the DNS server: 

Filter: DNS_OUT                                                
Counters:
Name                                                Bytes              Packets
DNS_OUT                                         110036876               273267
 
Filter: DNS_IN                                                
Counters:
Name                                                Bytes              Packets
DNS_IN                                           46961378               105066          <<<<<<<<<<<<<<<<<<<<

This issue might be seen if the following conditions are met:

  • When QFX5120 is configured only with EVPN/VXLAN

  • When UDP packets have a source port of 41070 or 52870

 

Solution

This problem is encountered because the DNS packets are wrongly hitting the VxLAN RA filters, which send the packets to the CPU and do not forward them out of the device. In this case, the filter matches on VxLAN Flags and sends the packets to the CPU. Typically, traffic coming on VxLAN access ports should not hit this filter.

 

The problem is tracked by PR1441047 .

Meanwhile, the following shell commands can be used as a workaround for the issue: 

>> cprod -A fpc0 -c "show filter hw all stats 0" | egrep vxlan_ra_option
>> cprod -A fpc0 -c 'set dcb bc "fp entry remove xxx"'