ISSUE: Ping reachability not working outside of the new data center when sourced from irb IP
ISSUE TRIGGER: Bringing up new datacenter
ISSUE IMPACT: No application impact. Failure of this test stalls the new DC migration activity.
APSTRA/MIST managed?: No
DEVICE:
Old DC Spines
Model: QFX10K2-36q
Code: 20.2R3-S3.6 (EOL & EOS)
New DC Spines
Model: QFX5120-32c
Code: 22.2R3-S5.4
TOPOLOGY:
Nutanix HCI Servers < --- > | NEW DC| Collapsed Spines QFX5120-32c 1/2 |NEW DC| < --- DCI --- > |OLD DC1| Collapsed Spines QFX5120-32c 1/2 |OLD DC1| (Nutanix HCI servers connected to OLD DC1) < --- > |OLD DC2| Collapsed Spines QFX5700 1/2 |OLD DC2| < --- > Nutanix HCI Servers
TROUBLESHOOTING SUMMARY:
1. Customer is running EVPN/VXLAN with a collapsed core design in their new datacenter (pair of QFX5120's) along with old DC1 and DC2 (pair of QFX10K2's in each DC). New DC is connected to old DC1 and DC2 with DCI connection and in turn OLD DC1 is connected to OLD DC2 via DCI connection using both type 2 and type 5. L3 gateway irb's are stretched among these DC's.
2. Customer is testing ping reachability from irb of new DC to servers connected and irb's of OLD DC1
3. ARP and mac learning looked good. We couldn't correlated PFE packet drops due to large amount of traffic traversing these interfaces.
5. We monitored the irb interfaces and see that ICMP request is being received by OLD DC1 QFX1 but no response being sent back to NEW DC QFX1.
6. We reviewed the routing configuration with import and export policies and checked the BGP route advertisement between OLD DC1 and NEW DC switches where we found that type2 and type 5 EVPN route inconsistency between these DC's.
7. We tweaked the EVPN_Export policy in NEW QFX switches from default reject to accept to accept the IRB routes of OLD DC1 QFX switches.
Once this was fixed, ping reachability between the IRB's were successful.
BEFORE CHANGE:
admin@newDC-jnpr-qfx1> show configuration policy-options policy-statement EVPN-Export
term Export-Direct {
from protocol direct;
then accept;
}
term Export-BGP {
from protocol bgp;
term Export-EVPN {
from {
protocol evpn;
route-filter-list EVPN-Networks;
term default {
then reject;
AFTER CHANGE:
then accept; ----> changed to accept
Ping reachability not working outside of the new data center when sourced from irb IP
We tweaked the EVPN_Export policy in NEW QFX switches from default reject to accept to accept the IRB routes of OLD DC1 QFX switches.