Article Detail

Description

When dpdkvifstats.py script is used to measure throughput of a particular vif (virtual interface) or a contrail-vrouter, there can be noticeable differences of outputs depending on the pps of vif. This may cause incorrect throughput calculations.

Symptoms

Example 1:

1000 PPS packets were sent from a VM1 towards a VM2 on remote compute and measured the throughput of VIF of VM1. On the local compute, the following results are seen where PPS count of packets seems to be higher than what was sent:

root@xenial-1:~# ping 172.76.77.36 -i 0.001 -c 10000000000
PING 172.76.77.36 (172.76.77.36) 56(84) bytes of data.
64 bytes from 172.76.77.36: icmp_seq=1 ttl=64 time=0.807 ms
64 bytes from 172.76.77.36: icmp_seq=2 ttl=64 time=0.136 ms
64 bytes from 172.76.77.36: icmp_seq=3 ttl=64 time=0.116 ms
64 bytes from 172.76.77.36: icmp_seq=4 ttl=64 time=0.131 ms
[root@overcloud-contraildpdk-0 heat-admin]# python dpdkstats.py -v 4 -t 1
-------------------------------------------------------------------------------------------------------------------------------------
|Core 1  | TX pps: 1018      | RX pps: 1018      | TX bps: 99764     | RX bps: 99764     | TX error: 0         | RX error 0         |
-------------------------------------------------------------------------------------------------------------------------------------
|Core 2  | TX pps: 0         | RX pps: 0         | TX bps: 0         | RX bps: 0         | TX error: 0         | RX error 0         |
-------------------------------------------------------------------------------------------------------------------------------------
|Total   | TX pps: 1018      | RX pps: 1018      | TX bps: 798112    | RX bps: 798112    | TX error: 0         | RX error 0         |
-------------------------------------------------------------------------------------------------------------------------------------
[root@overcloud-contraildpdk-0 heat-admin]# python dpdkstats.py -v 4 -t 1
-------------------------------------------------------------------------------------------------------------------------------------
|Core 1  | TX pps: 1016      | RX pps: 1016      | TX bps: 99568     | RX bps: 99568     | TX error: 0         | RX error 0         |
-------------------------------------------------------------------------------------------------------------------------------------
|Core 2  | TX pps: 0         | RX pps: 0         | TX bps: 0         | RX bps: 0         | TX error: 0         | RX error 0         |
-------------------------------------------------------------------------------------------------------------------------------------
|Total   | TX pps: 1016      | RX pps: 1016      | TX bps: 796544    | RX bps: 796544    | TX error: 0         | RX error 0         |
-------------------------------------------------------------------------------------------------------------------------------------
[root@overcloud-contraildpdk-0 heat-admin]# python dpdkstats.py -v 4 -t 1
-------------------------------------------------------------------------------------------------------------------------------------
|Core 1  | TX pps: 1020      | RX pps: 1020      | TX bps: 99960     | RX bps: 99960     | TX error: 0         | RX error 0         |
-------------------------------------------------------------------------------------------------------------------------------------
|Core 2  | TX pps: 0         | RX pps: 0         | TX bps: 0         | RX bps: 0         | TX error: 0         | RX error 0         |
-------------------------------------------------------------------------------------------------------------------------------------
|Total   | TX pps: 1020      | RX pps: 1020      | TX bps: 799680    | RX bps: 799680    | TX error: 0         | RX error 0         |
-------------------------------------------------------------------------------------------------------------------------------------
[root@overcloud-contraildpdk-0 heat-admin]#

Example 2:

100 PPS packets were sent from a VM1 towards a VM2 on remote compute and measuring throughput of VIF of VM1. On the local compute, the following results were seen where the PPS count of packets seems to be lower than what was sent:

root@xenial-1:~# ping 172.76.77.36 -i 0.01 -c 10000000000
PING 172.76.77.36 (172.76.77.36) 56(84) bytes of data.
64 bytes from 172.76.77.36: icmp_seq=1 ttl=64 time=0.861 ms
[root@overcloud-contraildpdk-0 heat-admin]# python dpdkstats.py -v 4 -t 1
-------------------------------------------------------------------------------------------------------------------------------------
|Core 1  | TX pps: 85        | RX pps: 85        | TX bps: 8330      | RX bps: 8330      | TX error: 0         | RX error 0         |
-------------------------------------------------------------------------------------------------------------------------------------
|Core 2  | TX pps: 0         | RX pps: 0         | TX bps: 0         | RX bps: 0         | TX error: 0         | RX error 0         |
-------------------------------------------------------------------------------------------------------------------------------------
|Total   | TX pps: 85        | RX pps: 85        | TX bps: 66640     | RX bps: 66640     | TX error: 0         | RX error 0         |
-------------------------------------------------------------------------------------------------------------------------------------
[root@overcloud-contraildpdk-0 heat-admin]# python dpdkstats.py -v 4 -t 1
-------------------------------------------------------------------------------------------------------------------------------------
|Core 1  | TX pps: 85        | RX pps: 85        | TX bps: 8330      | RX bps: 8330      | TX error: 0         | RX error 0         |
-------------------------------------------------------------------------------------------------------------------------------------
|Core 2  | TX pps: 0         | RX pps: 0         | TX bps: 0         | RX bps: 0         | TX error: 0         | RX error 0         |
-------------------------------------------------------------------------------------------------------------------------------------
|Total   | TX pps: 85        | RX pps: 85        | TX bps: 66640     | RX bps: 66640     | TX error: 0         | RX error 0         |
-------------------------------------------------------------------------------------------------------------------------------------
[root@overcloud-contraildpdk-0 heat-admin]# python dpdkstats.py -v 4 -t 1
-------------------------------------------------------------------------------------------------------------------------------------
|Core 1  | TX pps: 84        | RX pps: 84        | TX bps: 8232      | RX bps: 8232      | TX error: 0         | RX error 0         |
-------------------------------------------------------------------------------------------------------------------------------------
|Core 2  | TX pps: 0         | RX pps: 0         | TX bps: 0         | RX bps: 0         | TX error: 0         | RX error 0         |
-------------------------------------------------------------------------------------------------------------------------------------
|Total   | TX pps: 84        | RX pps: 84        | TX bps: 65856     | RX bps: 65856     | TX error: 0         | RX error 0         |
-------------------------------------------------------------------------------------------------------------------------------------
[root@overcloud-contraildpdk-0 heat-admin]#

Solution

The dpdkvifstats.py script can be used for getting the rate of a particular vif or all vif's on dpdk based contrail-vrouter. Multiple switches like time (-t) core (-c) exists which can be used to get time sliced data and amount of traffic on particular core.

When the script is executed as "dpdkvifstats.py -v <interface num > -t <time in seconds>" it will internally call "vif --get X --core Y | grep -i "TX\|RX\" command and sleep for the stipulated time given in the command with -t switch and run a second iteration of the command and then subtracts TX & RX values from two iterations respectively. Then it formats and displays the result on the console.

Time slicing is provided by the dpdkvifstats script, where we can provide a time slice to measure traffic in bps and pps. There is no time slicing option available internally with the vif --get command. Though the script calls the "vif --get " CLI, the CLI takes some time to fetch the available packets stats from the core, henceforth displaying packet counts on the core at the time of fetch, rather at the time interval exactly when the call to "vif --get" is made. Due to the processing overhead at the vrouter, we can't exactly map with the quantified values that are received by vrouter and processed by vrouter at high rates. For the default cases where ping interval with "1" second it works fine. There can be delta in count expected when we are sending the packets at higher rates i.e. very low intervals between the echo requests.

The dpdkvifstats.py script can be used for troubleshooting purposes, but it may not be accurate to measure data rate for vrouter or a particular vif. Alternatively, the "vif --get X --rate" command can be used to get the rate of a particular vif during traffic flow.

Modification History

2023-04-07: Initial publication.