Description

Contrail DPDK vrouter runs in compute node in the form of a single process. This single process comprises a few threads to fulfill its features and functions. This article summarizes the role of each thread and how to locate each of them using the Linux command line.

Solution

In DPDK vrouter, check the processes by running 'contrail-status' command: 

$ contrail-status -d
== Contrail vRouter ==
supervisor-vrouter:           active
contrail-vrouter-agent        active       pid 3722, uptime 7:59:42
contrail-vrouter-dpdk         active       pid 3638, uptime 7:59:45
contrail-vrouter-nodemgr      active       pid 3637, uptime 7:59:45

The process 'contrail-vrouter-dpdk' with pid 3638 , represent the forwarding plane of dpdk vrouter. 

DPDK vrouter process is a multi-threads application. There are 3 sets of threads into a DPDK vrouter. Each set of threads is made up of one of several threads:

  • Control threads : Used for DPDK internal processing. They are generated by the DPDK library itself.

  • Service threads : Used for connectivity between vrouter agent and vrouter forwarding plane (DPDK vrouter). For example, pkt0 and netlink each are served by a dedicated thread in dpdk vrouter process.  These provide internal connections between DPDK vrouter and vrouter agent. Service thread names are are 'lcore-slave-x' , where x is a number ranging 0-9 .

  • Processing threads : Used for packet polling and processing (forwarding plane). Thread names are `lcore-slave-x` where x is a number starting from 10 .  Processing threads are running with 'polling' mode in a loop, so new packets can be immediately processed without interrupt involved. This is why CPUs bound to these threads always show as busy. In DPDK compute node, you can use different Linux commands to locate each threads, check it's running status, etc. In this article, 'pstree' , 'ps' , and 'pidstat' will be used. You may also use other tools/commands to achieve the same goal.


First, the 'pstree' command will print a good hierarchical structure of all threads belonging to the same vrouter process: 
$ pstree -p $(ps -ef | awk '$8=="/usr/bin/contrail-vrouter-dpdk" {print $2}')
contrail-vroute(3638)─┬─{contrail-vroute}(4026)
                      ├─{contrail-vroute}(4027)
                      ├─{contrail-vroute}(4028)
                      ├─{contrail-vroute}(4029)
                      ├─{contrail-vroute}(4030)
                      ├─{contrail-vroute}(4031)
                      ├─{contrail-vroute}(4032)
                      ├─{contrail-vroute}(4033)
                      ├─{contrail-vroute}(4034)
                      ├─{contrail-vroute}(4035)
                      ├─{contrail-vroute}(4036)
                      ├─{contrail-vroute}(4037)
                      ├─{contrail-vroute}(4038)
                      └─{contrail-vroute}(5941)

It shows the main DPDK process 3638 is composed of threads with ID ranged from 4026 through 4038 plus 5941. Next, we'll look at the name of each threads: 
$ ps -Tp `pidof contrail-vrouter-dpdk`
      PID  SPID TTY          TIME CMD
     3638  3638 ?        00:10:07 contrail-vroute       #<---main process
     3638  4026 ?        00:00:06 eal-intr-thread       #<---control thread
     3638  4027 ?        00:13:09 lcore-slave-1         #<---service thread
     3638  4028 ?        00:00:00 lcore-slave-2         #<---service thread
     3638  4029 ?        00:00:11 lcore-slave-8         #<---service thread
     3638  4030 ?        00:01:01 lcore-slave-9         #<---service thread
     3638  5941 ?        00:00:01 lcore-slave-9         #<---service thread
     3638  4031 ?        07:37:41 lcore-slave-10        #<---forwarding thread
     3638  4032 ?        07:37:41 lcore-slave-11        #<---forwarding thread
     3638  4033 ?        07:37:41 lcore-slave-12        #<---forwarding thread
     3638  4034 ?        07:37:38 lcore-slave-13        #<---forwarding thread
     3638  4035 ?        07:37:41 lcore-slave-14        #<---forwarding thread
     3638  4036 ?        07:37:41 lcore-slave-15        #<---forwarding thread
     3638  4037 ?        07:37:41 lcore-slave-16        #<---forwarding thread
     3638  4038 ?        07:37:41 lcore-slave-17        #<---forwarding thread

Here, according to the name, we can identify there in DPDK process, we are running one control thread, 5 service threads and 8 forwarding threads.