Our customer has an 8 x QFX5120’s running a collapsed spine EVPN network.
+---------------------------------------+ | 192.168.1.1 | | +--------+ +--------+ | | | CORE-1 | | CORE-2 | | | +--------+ +--------+ | | |xe-0/0/40 |xe-0/0/40 | | | | | | | ESI-LAG ae0 | | | | | | | | +--------+ | | | +-----| EX3400 |-----+ | | +--------+ | | |ge-0/0/1 | | | | | | | | | | | +-------+ | | | MITEL | | | +-------+ VLAN 1200 | | 192.168.1.5 | +---------------------------------------+
They are seeing an issue with vlan1200 which is any casted across 6 of the QFX’s. There is a MITEL phone system plugged into a EX3400 on vlan1200 on ge-0/0/1. This EX3400 is patched into CORE1 xe-0/0/40 and CORE2 xe-0/0/40 on an ESI-LAG ae0.
When both legs of the ae0 are up there is partial reachability to the MITEL phone system. Traffic that traverses via CORE1 seems to work but CORE2 does not work. When we shutdown xe-0/0/40 on CORE1 and force all traffic onto CORE2 we loose complete access to the MITEL Phone system. We also cannot see the MAC address on the ESI-LAG on CORE2 from the ACCESS switch.
From CORE2 debugging, when xe-0/0/40 is shutdown on CORE1 it shows that the former CORE2 is trying to send out ARP who-has from irb.1200 but this never gets sent out on the ESI-LAG ae0:
root@CORE2> monitor traffic interface irb.1200 no-resolve
verbose output suppressed, use <detail> or <extensive> for full protocol decode
Address resolution is OFF.
Listening on irb.1200, capture size 96 bytes
13:01:23.780819 Out arp who-has 192.168.1.5 tell 192.168.1.1
13:01:24.403036 Out arp who-has 192.168.1.5 tell 192.168.1.1
13:01:25.102388 Out arp who-has 192.168.1.5 tell 192.168.1.1
13:01:25.702984 Out arp who-has 192.168.1.5 tell 192.168.1.1
13:01:26.602980 Out arp who-has 192.168.1.5 tell 192.168.1.1
root@CORE2> monitor traffic interface ae0 size 1500 no-resolve
Listening on ae3.0, capture size 96 bytes
We also never see the ARP Request arriving on the EX3400 (ACCESS switch).
Workaround:
Issue fixed by deleting and then re-applying the configuration related to VLAN 1200 on CORE2 and it worked as expected:
> delete interfaces irb unit 1200 family inet address 192.168.1.1/24
> delete interfaces irb unit 1200 mac 00:00:5e:xx:xx:xx
> delete routing-instances INSTANCE-NAME interface irb.1200
> delete protocols evpn vni-options vni 1200 vrf-target target:xxxxx:1200
> delete vlans v1200 vlan-id 1200
> delete vlans v1200 l3-interface irb.1200
> delete vlans v1200 vxlan vni 1200
> commit
> rollback 1
> commit full
This issue is tracked via the following PR:
https://prsearch.juniper.net/problemreport/PR1651827