Description

This KB describes the various messages that are generated while the DPDK application is negotiating interface using vhost user protocol.

Symptoms

During the investigation of an issue where the interfaces on the VMs have flapped, it is required to determine the cause of the flap. The contrail-dpdk-vrouter.log is used to analyze if the interface flap was initiated from the VM side or the vRouter.

Example logs:

2022-11-28 21:00:13,619 UVHOST: Client _tapcf7a6154-d2: handling message 11
2022-11-28 21:00:13,619 UVHOST:     GET VRING BASE: vring 0
2022-11-28 21:00:13,619 UVHOST:     GET VRING BASE: returns 40757
2022-11-28 21:00:13,619 UVHOST: Client _tapcf7a6154-d2: handling message 11
2022-11-28 21:00:13,619 UVHOST:     GET VRING BASE: vring 1
2022-11-28 21:00:13,619 UVHOST:     GET VRING BASE: returns 22855
2022-11-28 21:00:13,696 UVHOST: Client _tapcf7a6154-d2: handling message 18
2022-11-28 21:00:13,696 UVHOST: Client _tapcf7a6154-d2: setting vring 0 ready state 1
2022-11-28 21:00:13,696 UVHOST: Client _tapcf7a6154-d2: handling message 18
2022-11-28 21:00:13,696 UVHOST: Client _tapcf7a6154-d2: setting vring 1 ready state 1
2022-11-28 21:00:13,697 UVHOST: Client _tapcf7a6154-d2: handling message 2
2022-11-28 21:00:13,697 UVHOST:     SET FEATURES(original): 0x40009983
2022-11-28 21:00:13,697 UVHOST:     SET FEATURES( updated): 0x40009983
2022-11-28 21:00:13,698 UVHOST: Client _tapcf7a6154-d2: handling message 5
2022-11-28 21:00:13,699 UVHOST:     SET MEM TABLE:
2022-11-28 21:00:13,699 UVHOST: Client _tapcf7a6154-d2: unmapping 2 memory regions:
2022-11-28 21:00:13,699 UVHOST:     0: unmapping addr 0x0 size 0xc0000000
2022-11-28 21:00:13,699 UVHOST:     1: unmapping addr 0x100000000 size 0x340000000
2022-11-28 21:00:13,699 UVHOST: Client _tapcf7a6154-d2: mapping 2 memory regions:
2022-11-28 21:00:13,699 UVHOST:     0: FD 84 addr 0x0 size 0xc0000000 off 0x0
2022-11-28 21:00:13,699 UVHOST:     1: FD 94 addr 0x100000000 size 0x340000000 off 0xc0000000
2022-11-28 21:00:13,699 UVHOST: Client _tapcf7a6154-d2: handling message 8
2022-11-28 21:00:13,699 UVHOST:     SET VRING NUM: vring 0 num 512
2022-11-28 21:00:13,699 UVHOST: Client _tapcf7a6154-d2: handling message 10
2022-11-28 21:00:13,699 UVHOST:     SET VRING BASE: vring 0 base 0
2022-11-28 21:00:13,699 UVHOST: Client _tapcf7a6154-d2: handling message 9
2022-11-28 21:00:13,699 UVHOST:     SET VRING ADDR: vring 0 flags 0x0 desc 0x7eff15350000 used 0x7eff15353000 avail 0x7eff15352000
2022-11-28 21:00:13,699 UVHOST: Client _tapcf7a6154-d2: vring 0 is ready
2022-11-28 21:00:13,699 UVHOST: Client _tapcf7a6154-d2: no handler defined for message 12
2022-11-28 21:00:13,699 UVHOST: Client _tapcf7a6154-d2: handling message 13
2022-11-28 21:00:13,699 UVHOST:     SET VRING CALL: vring 0 FD 84
2022-11-28 21:00:13,699 UVHOST: Client _tapcf7a6154-d2: handling message 8
2022-11-28 21:00:13,699 UVHOST:     SET VRING NUM: vring 1 num 512
2022-11-28 21:00:13,700 UVHOST: Client _tapcf7a6154-d2: handling message 10
2022-11-28 21:00:13,700 UVHOST:     SET VRING BASE: vring 1 base 0
2022-11-28 21:00:13,700 UVHOST: Client _tapcf7a6154-d2: handling message 9
2022-11-28 21:00:13,700 UVHOST:     SET VRING ADDR: vring 1 flags 0x0 desc 0x7eff153f8000 used 0x7eff153fb000 avail 0x7eff153fa000
2022-11-28 21:00:13,700 UVHOST: Client _tapcf7a6154-d2: vring 1 is ready
2022-11-28 21:00:13,700 UVHOST: Client _tapcf7a6154-d2: no handler defined for message 12
2022-11-28 21:00:13,700 UVHOST: Client _tapcf7a6154-d2: handling message 13
2022-11-28 21:00:13,700 UVHOST:     SET VRING CALL: vring 1 FD 94
2022-11-28 21:00:13,700 UVHOST: Client _tapcf7a6154-d2: handling message 18
2022-11-28 21:00:13,700 UVHOST: Client _tapcf7a6154-d2: setting vring 0 ready state 1
2022-11-28 21:00:13,700 UVHOST: Client _tapcf7a6154-d2: handling message 18
2022-11-28 21:00:13,700 UVHOST: Client _tapcf7a6154-d2: setting vring 1 ready state 1
2022-11-28 21:00:13,754 UVHOST: Client _tapa5845703-9f: handling message 11
2022-11-28 21:00:13,754 UVHOST:     GET VRING BASE: vring 0
2022-11-28 21:00:13,754 UVHOST:     GET VRING BASE: returns 37971
2022-11-28 21:00:13,755 UVHOST: Client _tapa5845703-9f: handling message 11
2022-11-28 21:00:13,755 UVHOST:     GET VRING BASE: vring 1
2022-11-28 21:00:13,755 UVHOST:     GET VRING BASE: returns 35157

The logs indicate there are several messages exchanged between VM and DPDK vrouter. However, the message number do not help in understanding what these messages are for.
 

Solution

The Vhost User protocol is a communication protocol between a virtualization layer (i.e QEMU) and the underlying DPDK-based vRouter. The protocol is based on exchanging messages between these two components, with each message being defined by a message type.

The following is the list of dpdk message numbers and their function. This information has been compiled using the https://github.com/tungstenfabric/tf-vrouter/blob/99cad229e434539f1b66726f4b54fd998a846aa1/dpdk/qemu_uvhost.h 
Message type 1: VHOST_USER_GET_FEATURES: This message is used to retrieve the features supported by the vRouter.
Message type 2: VHOST_USER_SET_FEATURES: This message is used to configure the features supported by the vRouter.
Message type 3: VHOST_USER_SET_OWNER: This message is used to specify the owner of the vRouter.
Message type 4: VHOST_USER_RESET_OWNER: This message is used to reset the owner of the vRouter.
Message type 5: VHOST_USER_SET_MEM_TABLE: This message is used to configure the memory mapping for the vRouter.
Message type 6: VHOST_USER_SET_LOG_BASE: This message is used to set the base address for the vRouter's log buffer.
Message type 7: VHOST_USER_SET_LOG_FD: This message is used to set the file descriptor for the vRouter's log buffer.
Message type 8: VHOST_USER_SET_VRING_NUM: This message is used to set the number of descriptor rings used by the vRouter.
Message type 9: VHOST_USER_SET_VRING_ADDR: This message is used to set the base address for the vRouter's descriptor rings.
Message type 10: VHOST_USER_SET_VRING_BASE: This message is used to set the base index for the vRouter's descriptor rings.

These are some of the most commonly used message types in the Vhost User protocol. Use these message types as a reference to determine which direction you should be troubleshooting in.
  • Message type 11: VHOST_USER_GET_VRING_BASE is a message type in the Vhost User protocol. It is used to retrieve the current base index for a specific descriptor ring in the vRouter. The message specifies the index of the ring and the virtualization layer initiates this message to retrieve the current state of the ring.
The virtualization layer typically initiates the VHOST_USER_GET_VRING_BASE message when it needs to retrieve the current status of a descriptor ring, for example, to determine the next available descriptor for processing. This message is typically used in conjunction with other messages in the protocol, such as VHOST_USER_SET_VRING_ENABLE, to fully control and monitor the state of the vRouter's descriptor rings.
  • Message Type 12: VHOST_USER_SET_VRING_KICK
This message type is used in the DPDK (Data Plane Development Kit) framework for virtualization. It is used to configure the kick feature of a specific vring in the vhost-user backend. The kick feature allows the virtual machine to notify the backend when it has new data available to process.
  • Message Type 13: VHOST_USER_SET_VRING_CALL
This message type is also part of the DPDK vhost-user framework. It is used to configure the call feature of a specific vring in the vhost-user backend. The call feature enables the backend to notify the virtual machine when it has completed processing data.
  • Message Type 14: VHOST_USER_SET_VRING_ERR
This message type is used to configure the error feature of a specific vring in the vhost-user backend. The error feature allows the backend to report any errors encountered during data processing to the virtual machine.
  • Message Type 15: VHOST_USER_GET_PROTOCOL_FEATURES
This message type is used to retrieve the protocol features supported by the vhost-user backend. The virtual machine can send this message to query the backend for its supported features and capabilities.
  • Message Type 16: VHOST_USER_SET_PROTOCOL_FEATURES
This message type is used to set the protocol features for the vhost-user backend. The virtual machine can use this message to negotiate and enable specific features supported by the backend.
  • Message Type 17: VHOST_USER_GET_QUEUE_NUM
This message type is used to retrieve the number of available queues supported by the vhost-user backend. The virtual machine can send this message to query the backend for the maximum number of queues it can handle.
  • Message Type 18: VHOST_USER_SET_VRING_ENABLE
This message type is used to enable or disable a specific vring in the vhost-user backend. The virtual machine can use this message to control the activation or deactivation of a vring for data processing.
  • Message Type 19: VHOST_USER_SEND_RARP
This message type is used to send a Reverse Address Resolution Protocol (RARP) request from the virtual machine to the vhost-user backend. RARP is a protocol used to obtain an IP address from a MAC address in a local network. This message allows the virtual machine to request network configuration information from the backend.

 

Modification History

2023-07-31: Initial publication