Description

This KB article explains ECMP support for bridged flows on vRouter, which was added in the following Contrail releases:

  • r3.0.3.2
  • r3.1.2.0
  • r3.2.2.0
  • r4.0.0.0

Symptoms

In Contrail releases - r3.2, r3.0, r3.1, r2.22.x, r3.0.2.x, r3.0.3.x, r3.1.1.x, if a packet is bridged, and even if there is ECMP route back to the source, the ECMP index of the source is not stored as part of the flow. Therefore, return traffic (if routed) would be sent to only one of the SI/sources, the one that has ECMP index 0.

Solution

In contrail releases - r3.0.3.2, r3.1.2.0, r3.2.2.0, r4.0.0.0, the flow entry is continuously updated with the source of the packet. In case of tunnel, the tunnel source is recorded. And in case of VMI, the VIF index is recorded. Agent ensures that all L2 flows are added with ECMP index as -1. When such flow is being processed, if the nexthop is an ECMP Nexthop, the reverse flow source information is used to choose the component NH.

Example:

Flow from compute node 10.173.150.156

    Index                Source:Port/Destination:Port                      Proto(V)
-----------------------------------------------------------------------------------
   120032<=>441780       1.2.3.4:49408                                       1 (1)
                         1.2.3.3:0    
(Gen: 1, K(nh):20, Action:F, Flags:, QOS:-1, S(nh):24,  Stats:6339/621222,
SPort 55648, TTL 0, Sinfo 10.173.150.156)

root@comp155:~# nh --get 24
Id:24         Type:Tunnel         Fmly: AF_INET  Rid:0  Ref_cnt:4          Vrf:0
              Flags:Valid, MPLSoUDP,
              Oif:0 Len:14 Flags Valid, MPLSoUDP,  Data:52 54 00 ce 12 c2 52 54 00 13 0e f0 08 00
              Vrf:0  Sip:10.173.150.155  Dip:10.173.150.156

root@comp155:~# nh --get 20
Id:20         Type:Encap          Fmly: AF_INET  Rid:0  Ref_cnt:4          Vrf:1
              Flags:Valid, Policy,
              EncapFmly:0806 Oif:3 Len:14
              Encap Data: 02 84 4f c3 40 2b 00 00 5e 00 01 00 08 00

 Flow from VMI tap844fc340-2b

    Index                Source:Port/Destination:Port                      Proto(V)
-----------------------------------------------------------------------------------
   441780<=>120032       1.2.3.3:49408                                       1 (1)
                         1.2.3.4:0    
(Gen: 1, K(nh):20, Action:F, Flags:, QOS:-1, S(nh):20,  Stats:6339/621222,
SPort 53218, TTL 0, Sinfo 3.0.0.0)


root@comp155:~# nh --get 20
Id:20         Type:Encap          Fmly: AF_INET  Rid:0  Ref_cnt:4          Vrf:1
              Flags:Valid, Policy,
              EncapFmly:0806 Oif:3 Len:14
              Encap Data: 02 84 4f c3 40 2b 00 00 5e 00 01 00 08 00

root@comp155:~# vif --get 3
Vrouter Interface Table

Flags: P=Policy, X=Cross Connect, S=Service Chain, Mr=Receive Mirror
       Mt=Transmit Mirror, Tc=Transmit Checksum Offload, L3=Layer 3, L2=Layer 2
       D=DHCP, Vp=Vhost Physical, Pr=Promiscuous, Vnt=Native Vlan Tagged
       Mnp=No MAC Proxy, Dpdk=DPDK PMD Interface, Rfl=Receive Filtering Offload, Mon=Interface is Monitored
       Uuf=Unknown Unicast Flood, Vof=VLAN insert/strip offload, Df=Drop New Flows, Proxy=MAC Requests Proxied Always
       Mn=Mirror without Vlan Tag

vif0/3      OS: tap844fc340-2b
            Type:Virtual HWaddr:00:00:5e:00:01:00 IPaddr:1.2.3.3
            Vrf:1 Flags:PL3L2D QOS:-1 Ref:5
            RX packets:14082  bytes:1741762 errors:0
            TX packets:14113  bytes:1121547 errors:0
            Drops:6