Description

After a virtual machine (VM) was moved from DC1 to DC2 in the same broadcast domain, there was no IP reachability of this VM for several minutes.

Symptoms

The topology can be seen on the picture:

Screenshot 2024-03-15 at 13.20.36.png

After the VM movement the two spine QFX switches in DC1 still had the information that the MAC address was reachable via AE24 interface.

user@spine1_1> show evpn database extensive

VN Identifier: 102000, MAC address: aa:bb:cc:00:00:01
 State: 0x0
 Mobility history
   Mobility event time    Type   Source                                  Seq num
   Feb 14 21:00:41.604439 Remote 00:01:00:01:00:02:00:00:00:24           1
 Source: 00:01:00:01:00:02:00:00:00:24, Rank: 1, Status: Active.                           <<<<< via local ae interface
   Local origin: ae24.0
   Mobility sequence number: 1 (minimum origin address 10.0.0.1)
   Timestamp: Feb 14 21:10:06.890271 (0x65ccded6)
   State: <Local-MAC-Only Local-To-Remote-Adv-Allowed>
   MAC advertisement route status: Created
   IP address: 192.168.1.10
   Flags: <Local-Adv Local-Mh-Adv-Proxy-Bit Interconnect-DC>
     Local origin: ae24.0
 Source: 00:02:00:01:00:02:00:00:00:21, Rank: 2, Status: Inactive.                    <<<<< via vxlan tunnel and it is Inactive
   Remote origin: 10.0.0.3
   Remote state: <Mac-Only-Adv>
   Mobility sequence number: 0 (minimum origin address 10.0.0.3)
   Timestamp: Feb 14 21:03:56.260729 (0x65ccdd64)
   State: <>
   MAC advertisement route status: Not created (inactive source)
   IP address: 192.168.1.10
     Remote origin: 10.0.0.3
   History db:
     Time                      Event
     Feb 14 21:00:41.868 2024  00:02:00:01:00:02:00:00:00:21 : Remote peer 10.0.0.3 deleted
     Feb 14 21:00:41.868 2024  00:02:00:01:00:02:00:00:00:21 : Deleting
     Feb 14 21:00:41.868 2024  Updating output state (change flags 0x2 <ESI-Deleted>)
     Feb 14 21:00:42.025 2024  00:01:00:01:00:02:00:00:00:24 : Updating output state (change flags 0x41 <Intf Local-Seq-Num>)
     Feb 14 21:00:42.026 2024  00:01:00:01:00:02:00:00:00:24 : Updating output state (change flags 0x20 <ESI-Local-State>)
     Feb 14 21:03:56.260 2024  00:02:00:01:00:02:00:00:00:21 : Remote peer 10.0.0.3 created
     Feb 14 21:03:56.260 2024  00:02:00:01:00:02:00:00:00:21 : Created
     Feb 14 21:03:56.261 2024  Updating output state (change flags 0x1 <ESI-Added>)
     Feb 14 21:10:06.474 2024  00:01:00:01:00:02:00:00:00:24 : Remote peer 10.0.0.2 deleted
     Feb 14 21:10:06.474 2024  00:01:00:01:00:02:00:00:00:24 : Updating output state (change flags 0x110 <Remote-Seq-Num ESI-Peer-Deleted>)


{master:0}
user@spine1_1> show ethernet-switching table extensive

MAC address: aa:bb:cc:00:00:01
 Routing instance: default-switch
 VLAN name: VL-2000, VLAN ID: 2000
  Learning interface: ae24.0
  ELP-NH: 0
  Layer 2 flags: in_hash,in_ifd,in_ifl,in_vlan,in_rtt,kernel,in_ifbd,advt_to_remote,rcvd_from_remote,local_gen_proxy_mac
  Epoch: 0                           Sequence number: 3
  Learning mask: 0x4000000000000000010000000000000000000001 BGP sequence number: 0

 

 

And the information is not updated until the information about the MAC address over local AE interface is not expired. 

 

 

 



 

 

Solution

The issue was due to the presence of the following configuration on Spine2_1 and Spine2_2 devices:

protocols {
    evpn {
       mac-mobility {
           no-sequence-numbers;
       }
    }
}

 

This command disables mac-mobility feature in Junos. To implement mac-mobility Junos uses mac-mobility extended community in BGP EVPN route to track the version of the update (using the secuence number). The bigger the sequence number – the newer the information about the MAC address whereabouts. This is the behavior by default. You can check this extended community using the following command (for example):

 show route protocol bgp 10.0.0.3 table bgp.evpn.0 extensive


In the default configuration:

* 2:10.0.0.3:1::102000::aa:bb:cc:00:00:01/304 MAC/IP (1 entry, 0 announced)
    Import Accepted
    Route Distinguisher: 10.0.0.3:1
    Route Label: 102000
    ESI: 00:02:00:01:00:02:00:00:00:21
    Nexthop: 10.0.0.3
    Localpref: 100
    AS path: I
    Communities: target:65000:268537456 encapsulation:vxlan(0x8) mac-mobility:0x0 (sequence 2)


It is possible to disable the support of this extended community (set protocols evpn mac-mobility no-sequence-numbers). In this case the update regarding the MAC address will look like:

* 2:10.0.0.3:1::102000::aa:bb:cc:00:00:01/304 MAC/IP (1 entry, 0 announced)
    Import Accepted
    Route Distinguisher: 10.0.0.3:1
    Route Label: 102000
    ESI: 00:02:00:01:00:02:00:00:00:21
    Nexthop: 10.0.0.3
    Localpref: 100
    AS path: I
    Communities: target:65000:268537456 encapsulation:vxlan(0x8)


And in this case the local knowledge is more preferable (via ae interface) - as shown above.

 

Modification History

2024-02-26 : Article Created