Description

This article explains about how to test anycast irb without VGA in an IP Fabric environment. Sometimes we see customers test active-active solution the same way VGA is tested.

image.png

In case of anycast irb without VGA, below is how the configuration looks like in the leaf A

root@Leaf-A>
show configuration vlans vn1000 
vlan-id 1000;
l3-interface irb.1000;
vxlan
{
  vni 1000;
}
 {master:0}
root@LEAF-A> show configuration interfaces irb.1000 
family inet {
  mtu 9000;
  address 10.18.64.1/24;
}
mac 00:00:00:00:01:01;

Below is the configuration from LEAF B

root@Leaf-B> show configuration vlans vn1000 
vlan-id 1000;
l3-interface irb.1000;
vxlan {
  vni 2000;
}
 {master:0}
root@LEAF-B> show configuration interfaces irb.1000

family inet {
  mtu 9000;
  address 10.18.64.1/24;
}
mac 00:00:00:00:01:01;

The servers are multi-homed to the Leaf A and Leaf B. The ip configured in the irb.1000 would act as a default gateway for the server. The server can hash the traffic across leaf A and Leaf B.

 

Symptoms

Verifying the connectivity by pinging from the leaf to server can cause the icmp replies to go to the other leaf in the pair making it look like the connectivity is not there. 

For example:
  
{master:0}
root@LEAF-A> ping 10.18.64.200 routing-instance TEST

PING 10.18.64.200 (10.18.64.200): 56 data bytes
64 bytes from 10.18.64.200: icmp_seq=0 ttl=64 time=3.962 ms
64 bytes from 10.18.64.200: icmp_seq=1 ttl=64 time=4.578 ms
^C
--- 10.18.64.200 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 3.962/4.270/4.578/0.308 ms

root@LEAF-A> ping 10.18.64.201 routing-instance TEST   
PING 10.18.64.201 (10.18.64.201): 56 data bytes
64 bytes from 10.18.64.201: icmp_seq=0 ttl=64 time=11.951 ms
^C
--- 10.18.64.201 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 11.951/11.951/11.951/0.000 ms

Perform Ping from Leaf B

root@LEAF-B> ping 10.18.64.200 routing-instance TEST 
PING 10.18.64.200 (10.18.64.200): 56 data bytes
^C
--- 10.18.64.200 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss

{master:0}
root@LEAF-B> ping 10.18.64.201 routing-instance TEST   
PING 10.18.64.201 (10.18.64.201): 56 data bytes
^C
--- 10.18.64.201 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss
 
On checking monitor traffic interface on Leaf A, icmp replies are seen. This shows packets are replied to but are sent to Leaf A. This can happen and is expected behavior with anycast irb. 
root@LEAF-A> monitor traffic interface irb.1000 

verbose output suppressed, use <detail> or <extensive> for full protocol decode
Address resolution is ON. Use <no-resolve> to avoid any reverse lookup delay.
Address resolution timeout is 4s.
Listening on irb.1000, capture size 96 bytes
Reverse lookup for 10.18.64.200 failed (check DNS reachability).
Other reverse lookup failures will not be reported.
Use <no-resolve> to avoid reverse lookups on IP addresses.

15:58:47.398631 In IP 10.18.64.200 > 10.18.64.1: ICMP echo reply, id 18771, seq 0, length 64
15:58:48.404898 In IP 10.18.64.200 > 10.18.64.1: ICMP echo reply, id 18771, seq 1, length 64
16:01:13.452167  In IP 10.18.64.201 > 10.18.64.1: ICMP echo reply, id 20307, seq 0, length 64
16:01:14.450543  In IP 10.18.64.201 > 10.18.64.1: ICMP echo reply, id 20307, seq 1, length 64

Solution

The correct way to verify connectivity is to perform

• Ping between the servers

{master:0}
root@Server2> ping 10.18.64.200 
PING 10.18.64.200 (10.18.64.200): 56 data bytes
64 bytes from 10.18.64.200: icmp_seq=0 ttl=64 time=1.219 ms
^C
--- 10.18.64.200 ping statistics ---

1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 1.219/1.219/1.219/0.000 ms

{master:0}
root@Server1> ping 10.18.64.201   
PING 10.18.64.201 (10.18.64.201): 56 data bytes
64 bytes from 10.18.64.201: icmp_seq=0 ttl=64 time=1.760 ms
64 bytes from 10.18.64.201: icmp_seq=1 ttl=64 time=4.968 ms
^C
--- 10.18.64.201 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 1.760/3.364/4.968/1.604 ms

 

 

• Ping from the server to the anycast gateway

{master:0}
root@Server2> ping 10.18.64.1
PING 10.18.64.1 (10.18.64.1): 56 data bytes
64 bytes from 10.18.64.1: icmp_seq=0 ttl=64 time=16.712 ms
^C
--- 10.18.64.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 16.712/16.712/16.712/0.000 ms

{master:0}
root@Server1> ping 10.18.64.1 
PING 10.18.64.1 (10.18.64.1): 56 data bytes
64 bytes from 10.18.64.1: icmp_seq=0 ttl=64 time=7.475 ms
^C
--- 10.18.64.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 7.475/7.475/7.475/0.000 ms

• Ping from outside Fabric to the servers

root@Router> ping 10.18.64.201 routing-instance TEST-1   
PING 10.18.64.201 (10.18.64.201): 56 data bytes
64 bytes from 10.18.64.201: icmp_seq=0 ttl=64 time=13.811 ms
64 bytes from 10.18.64.201: icmp_seq=1 ttl=64 time=14.408 ms
64 bytes from 10.18.64.201: icmp_seq=2 ttl=64 time=9.209 ms

root@Router> ping 10.18.64.200 routing-instance TEST-1  
PING 10.18.64.200 (10.18.64.200): 56 data bytes
64 bytes from 10.18.64.200: icmp_seq=0 ttl=64 time=10.321 ms
64 bytes from 10.18.64.200: icmp_seq=1 ttl=64 time=11.724 ms
64 bytes from 10.18.64.200: icmp_seq=2 ttl=64 time=8.749 ms

 

Modification History

06-02-2023 Initial Draft