Customer experience a consistent problem where ICMP to devices with even TTL number are dropped , suggesting a VXLAN loop.
Ping is working fine with uneven TTL number, but all other communicaiton protocol to switches switch 1 and switch 2 is impossible (SSH, netconf etc)
Full path looks like
QFX-1→ QFX2 (-l3-vrf) → VXLAN tunnel via QFX2→ QFX-1
root@switch# run ping 10.1.1.1 routing-instance test count 10
PING 10.1.1.1 (10.1.1.1 ): 56 data bytes
36 bytes from 10.1.1.2 : Time to live exceeded
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
--- 10.1.1.1 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss
Pings are working with uneven TTL without setting the interface
root@switch# run ping 10.1.1.1 routing-instance test count 10 ttl 3
PING 10.1.1.1 (10.1.1.1): 56 data bytes
64 bytes from 10.1.1.1 : icmp_seq=0 ttl=64 time=13.156 ms
64 bytes from 10.1.1.1 : icmp_seq=1 ttl=64 time=25.765 ms
^C
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 13.156/19.460/25.765/6.305 ms
Traffic towards 10.1.1.1 is intermittently failing (e.g., ICMP depends on TTL, while TCP-based protocols consistently fail). The behavior initially suggested a VXLAN loop, which has now been confirmed.
Root Cause
The issue is caused by recursive routing between EVPN (overlay) and OSPF (underlay) due to overlapping use of the same IP address.
EVPN (Overlay):
The prefix is advertised as a Type-5 route and resolved via VXLAN:
Protocol next hop: 10.1.1.1
VXLAN tunnel rewrite:
Source VTEP: 10.1.1.1
Destination VTEP: 10.1.1.1
Underlay (OSPF / inet.0):
The same prefix is also installed in inet.0 and resolved via ECMP paths:
10.1.1.1/32 Originating RIB: inet.0
Next hop: 10.0.3.0 via xe-0/1/0.0
Next hop: 10.0.3.9 via xe-0/1/1.0
Resulting behavior
This creates a recursive loop:
This leads to continuous re-encapsulation and a forwarding loop across the fabric.
Based on the outputs:
1. EVPN route using same IP as VTEP
2. Route resolution shows recursion via inet.0
3. Forwarding table shows ECMP paths in underlay
Next-hop: 10.0.3.0 via xe-0/1/0.0
Next-hop: 10.0.3.9 via xe-0/1/1.0
The TTL-dependent behavior occurs because packets destined for 10.1.1.1 enter a two-hop forwarding loop between the EVPN overlay and the underlay OSPF paths: the VXLAN-encapsulated packet points to a VTEP IP that is also advertised in the underlay, creating recursive routing. Each hop decrements the TTL by 1, so packets with an even TTL expire on the originating node before reaching the actual VTEP, resulting in “TTL exceeded” messages, while packets with an odd TTL happen to reach the correct VTEP before the TTL expires and are successfully decapsulated. This explains why ICMP may succeed only with certain TTL values, while TCP sessions fail completely due to looping, reordering, and packet drops.
Conclusion
The issue was caused by using the same IP (10.1.1.1 ) simultaneously as:
An EVPN Type-5 routed prefix (overlay)
An OSPF-advertised loopback (underlay)
This led to recursive VXLAN encapsulation and a forwarding loop. Removing the prefix from OSPF should resolves the recursion.