Description

Traffic passing a Contrail compute node will trigger flows established in Contrail vRouter. 

This article explains the flow behavior in a Contrail vRouter when it receives ICMP messages triggered by ping command.

Solution

In this test, there are two cirros VMs: cirros-1 and cirros-2, each with a different VM attached to it.  The IP addresses of the two VMs are 1.1.1.3 and 2.2.2.5 respectively. The test is done by first logging into VM1 and ping VM2. Then collect the following data to investigate the flow behavior in the system:

  1. Capture the ICMP packets via "tcpdump" command on tap interface of any of the VMs
  2. Monitor the generated flows in vRouter via vRouter "flow" command

First ping test

From one terminal, start the ping from cirros-1 VM and keep it running:

    cirros-1 $ ping 2.2.2.5
    PING 2.2.2.5 (2.2.2.5): 56 data bytes
    64 bytes from 2.2.2.5: seq=0 ttl=63 time=2.057 ms
    64 bytes from 2.2.2.5: seq=1 ttl=63 time=0.762 ms
    64 bytes from 2.2.2.5: seq=2 ttl=63 time=1.063 ms
    64 bytes from 2.2.2.5: seq=3 ttl=63 time=0.772 ms
    64 bytes from 2.2.2.5: seq=4 ttl=63 time=0.717 ms

On another terminal, monitor the flow:

    root@comp48:~# flow --match "1.1.1.3 & 2.2.2.5"
    ......
    Listing flows matching ([1.1.1.3]:*, [2.2.2.5]:*)

        Index                Source:Port/Destination:Port                      Proto(V)
    -----------------------------------------------------------------------------------
       258388<=>433616       1.1.1.3:9218                                        1 (1)
                             2.2.2.5:0
    (Gen: 1, K(nh):20, Action:F, Flags:, QOS:-1, S(nh):20,  Stats:122/11956,
     SPort 64928, TTL 0, Sinfo 3.0.0.0)

       433616<=>258388       2.2.2.5:9218                                        1 (1)
                             1.1.1.3:0
    (Gen: 1, K(nh):20, Action:F, Flags:, QOS:-1, S(nh):22,  Stats:122/10248,
     SPort 52109, TTL 0, Sinfo 10.169.4.8)

On the third terminal, start packet capture:

    root@comp47:~# tcpdump -ni tap7b048c9e-82 icmp
    tcpdump: WARNING: tap7b048c9e-82: no IPv4 address assigned
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on tap7b048c9e-82, link-type EN10MB (Ethernet), capture size 65535 bytes
    15:56:30.015114 IP 1.1.1.3 > 2.2.2.5: ICMP echo request, id 9218, seq 0, length 64
    15:56:30.016707 IP 2.2.2.5 > 1.1.1.3: ICMP echo reply, id 9218, seq 0, length 64
    15:56:31.015574 IP 1.1.1.3 > 2.2.2.5: ICMP echo request, id 9218, seq 1, length 64
    15:56:31.016033 IP 2.2.2.5 > 1.1.1.3: ICMP echo reply, id 9218, seq 1, length 64
    15:56:32.016082 IP 1.1.1.3 > 2.2.2.5: ICMP echo request, id 9218, seq 2, length 64
    15:56:32.016693 IP 2.2.2.5 > 1.1.1.3: ICMP echo reply, id 9218, seq 2, length 64
    15:56:33.016221 IP 1.1.1.3 > 2.2.2.5: ICMP echo request, id 9218, seq 3, length 64
    15:56:33.016812 IP 2.2.2.5 > 1.1.1.3: ICMP echo reply, id 9218, seq 3, length 64

This first ping command triggered one pair of flows. Packet capture shows all ICMP packets generated by this ping command is with the same 'id 9218', while the 'seq' number keeps increasing in each pair of ICMP request/reply messages. Comparing with flow command output, the "port" number of the flow entry in this test is actually the value of the ICMP 'id'. Obviously there is no "port" number in an ICMP packet, so vRouter treats ICMP 'id' as the "source port" in each entry of the forwarding/reverse flow, making the ICMP flow look consistent with the TCP/UDP flow. This also complies to the usage example briefed in RFC792 .

The identifier and sequence number may be used by the echo sender to aid in matching the replies with the echo requests.  For example, the identifier might be used like a port in TCP or UDP to identify a session, and the sequence number might be incremented on each echo request sent.  The echoer returns these same values in the echo reply.

Second ping test

Stop the first ping command and restart it again. Monitor the flows and packets.

    64 bytes from 2.2.2.5: seq=269 ttl=63 time=1.090 ms
    64 bytes from 2.2.2.5: seq=270 ttl=63 time=0.764 ms
    64 bytes from 2.2.2.5: seq=271 ttl=63 time=0.556 ms
    64 bytes from 2.2.2.5: seq=272 ttl=63 time=0.716 ms
    ^C  #<---press ctrl-c to stop the ping command
    --- 2.2.2.5 ping statistics ---
    273 packets transmitted, 273 packets received, 0% packet loss
    round-trip min/avg/max = 0.384/0.805/2.057 ms

    cirros-1 $ ping 2.2.2.5
    PING 2.2.2.5 (2.2.2.5): 56 data bytes
    64 bytes from 2.2.2.5: seq=0 ttl=63 time=1.198 ms
    64 bytes from 2.2.2.5: seq=1 ttl=63 time=0.729 ms
    64 bytes from 2.2.2.5: seq=2 ttl=63 time=0.503 ms

flow table:

        Index                Source:Port/Destination:Port                      Proto(V)
    -----------------------------------------------------------------------------------
        87600<=>185536       1.1.1.3:9474                                        1 (1)
                             2.2.2.5:0
    (Gen: 1, K(nh):20, Action:F, Flags:, QOS:-1, S(nh):20,  Stats:5/490,  SPort 57496,
     TTL 0, Sinfo 3.0.0.0)

       185536<=>87600        2.2.2.5:9474                                        1 (1)
                             1.1.1.3:0
    (Gen: 1, K(nh):20, Action:F, Flags:, QOS:-1, S(nh):22,  Stats:5/420,  SPort 56194,
     TTL 0, Sinfo 10.169.4.8)

       258388<=>433616       1.1.1.3:9218                                        1 (1)
                             2.2.2.5:0
    (Gen: 1, K(nh):20, Action:F, Flags:, QOS:-1, S(nh):20,  Stats:273/26754,
     SPort 64928, TTL 0, Sinfo 3.0.0.0)

       433616<=>258388       2.2.2.5:9218                                        1 (1)
                             1.1.1.3:0
    (Gen: 1, K(nh):20, Action:F, Flags:, QOS:-1, S(nh):22,  Stats:273/22932,
     SPort 52109, TTL 0, Sinfo 10.169.4.8)


Packet capture:

    16:01:02.074109 IP 1.1.1.3 > 2.2.2.5: ICMP echo request, id 9218, seq 272, length 64
    16:01:02.074623 IP 2.2.2.5 > 1.1.1.3: ICMP echo reply, id 9218, seq 272, length 64
    16:01:05.219698 IP 1.1.1.3 > 2.2.2.5: ICMP echo request, id 9474, seq 0, length 64
    16:01:05.220803 IP 2.2.2.5 > 1.1.1.3: ICMP echo reply, id 9474, seq 0, length 64
    16:01:06.219907 IP 1.1.1.3 > 2.2.2.5: ICMP echo request, id 9474, seq 1, length 64
    16:01:06.220404 IP 2.2.2.5 > 1.1.1.3: ICMP echo reply, id 9474, seq 1, length 64
    16:01:07.220048 IP 1.1.1.3 > 2.2.2.5: ICMP echo request, id 9474, seq 2, length 64
    16:01:07.220408 IP 2.2.2.5 > 1.1.1.3: ICMP echo reply, id 9474, seq 2, length 64
    16:01:08.220217 IP 1.1.1.3 > 2.2.2.5: ICMP echo request, id 9474, seq 3, length 64
    16:01:08.220604 IP 2.2.2.5 > 1.1.1.3: ICMP echo reply, id 9474, seq 3, length 64

One additional pair of flows is being established in vRouter flow table, with a different source "Port" 9474 this time. This is essentially an ICMP "id" number, as seen from the packet capture again. This concludes our test:
  • Each time a ping command starts, a new ICMP 'id' value will be used by the system, which will trigger a new pair of flows in vRouter.
  • During the running of the same ping command, only one pair of flows will be generated, because all packets share a same ICMP "id". Only the first few ping packets will invoke vRouter flow establishment process.
  • vRouter displays ICMP 'id' as "source port" of each flow entry. To trigger more flows by ping command, start ping with packet count 1 and repeat the command in a script.