Description

Port mirroring is an important monitoring/debugging tool for tenant traffic. This article describes two best practices to follow when enabling port mirroring on virtual machine interface to ensure it works correctly.

For a related article to configure port mirroring on TAP interface, refer to KB34705 - Port mirroring may not take effect if analyzer_name is not given [juniper.net]

Symptoms

Scenario 1

Configuring port mirroring on virtual machine interface (VMI) and an analyzer_name is not given or left blank on purpose. Therefore, mirroring instance does not function.

Scenario 2

Configuring port mirroring on two VMIs on the same compute node and the same analyzer_name is provided. Therefore, one mirroring instance overrides the other, causing the wrong mirroring index to be programmed.

Solution

For a detailed description of the failure due to scenario 1, refer to KB34705 - Port mirroring may not take effect if analyzer_name is not given [juniper.net]

For scenario 2, the example below shows how a mirroring instance overrides the other if the same name is mistakenly provided.

Initially, the user configures two unrelated port mirroring instances on VMI 'c6df2ab2-1c3e-4ec1-b47c-653a84627631' ​ and VMI '4d070662-bd76-48aa-9b9d-bc76dc07d24e' .
Two different names are given, which are highlighted in blue below.
 
user@device:~# curl -u admin:Juniper localhost:8095/virtual-machine-interface/c6df2ab2-1c3e-4ec1-b47c-653a84627631|python -mjson.tool|grep -A10 "mirror"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3449  100  3449    0     0   316k      0 --:--:-- --:--:-- --:--:--  336k
            "interface_mirror": {
                "mirror_to": {
                    "analyzer_ip_address": "2.2.2.4",
                    "analyzer_mac_address": "02:7d:fe:fe:59:d3",
                    "analyzer_name": "Analyzer_Nir",
                    "juniper_header": false,
                    "nh_mode": "dynamic",
                    "routing_instance": "default-domain:nzhong:analyzer-vn:analyzer-vn",
                    "static_nh_header": null,
                    "udp_port": 8099
                },
                "traffic_direction": "both"
 
user@device:~# curl -u admin:Juniper localhost:8095/virtual-machine-interface/4d070662-bd76-48aa-9b9d-bc76dc07d24e|python -mjson.tool|grep -A10 "mirror"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3408  100  3408    0     0   433k      0 --:--:-- --:--:-- --:--:--  475k
            "interface_mirror": {
                "mirror_to": {
                    "analyzer_ip_address": "4.4.4.4",
                    "analyzer_mac_address": null,
                    "analyzer_name": "Analyzer_Nir2",
                    "juniper_header": true,
                    "nh_mode": "dynamic",
                    "routing_instance": "default-domain:admin:VM-MGT:VM-MGT",
                    "static_nh_header": null,
                    "udp_port": null
                },
                "traffic_direction": "both"
 

Mirroring indices are created separately as 2 and 4 respectively, and point to two different destinations:

Index    NextHop    Flags    VNI
------------------------------------------------
    0         44       D          0
    1          1                  0
    2        190       D          0
    3        185       D          0
    4        191       D          0​
 
vif0/30     PMD: tapc6df2ab2-1c
            Type:Virtual HWaddr:00:00:5e:00:01:00 IPaddr:10.11.12.3
            Vrf:23 Flags:MrMtL3L2D QOS:-1 Ref:23 Mirror index 2
            RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
            RX packets:0  bytes:0 errors:0
            TX packets:412358  bytes:17319036 errors:0
            Ingress Mirror Metadata: 3 20 64 65 66 61 75 6c 74 2d 64
                                     6f 6d 61 69 6e 3a 61 64 6d 69 6e
                                     3a 56 4e 2d 4e 69 72 2d 54 65 73
                                     74 ff 0
            Egress Mirror Metadata: 4 20 64 65 66 61 75 6c 74 2d 64 6f
                                    6d 61 69 6e 3a 61 64 6d 69 6e 3a
                                    56 4e 2d 4e 69 72 2d 54 65 73 74
                                    ff 0
            TX port   packets:412358 errors:0 syscalls:412357
            Drops:0
vif0/11     PMD: tap4d070662-bd
            Type:Virtual HWaddr:00:00:5e:00:01:00 IPaddr:10.85.190.114
            Vrf:6 Flags:PMrMtL3L2D QOS:-1 Ref:23 Mirror index 4
            RX port   packets:29077 errors:0
            RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
            RX packets:29077  bytes:1279420 errors:0
            TX packets:412664  bytes:17332128 errors:0
            Ingress Mirror Metadata: 3 1b 64 65 66 61 75 6c 74 2d 64
                                     6f 6d 61 69 6e 3a 61 64 6d 69 6e
                                     3a 56 4d 2d 4d 47 54 ff 0
            Egress Mirror Metadata: 4 1b 64 65 66 61 75 6c 74 2d 64 6f
                                    6d 61 69 6e 3a 61 64 6d 69 6e 3a
                                    56 4d 2d 4d 47 54 ff 0
            TX port   packets:412664 errors:0 syscalls:412664
            Drops:29077

Then user changed the second mirroring instance name from Analyzer_Nir2 to Analyzer_Nir, which overwrites the first instance. Now with two mirroring instances having the same mirror index, the first instance sends mirroring traffic from the wrong port.
user@device:~# curl -u admin:Juniper localhost:8095/virtual-machine-interface/4d070662-bd76-48aa-9b9d-bc76dc07d24e|python -mjson.tool|grep -A10 "mirror"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3407  100  3407    0     0   719k      0 --:--:-- --:--:-- --:--:--  831k
            "interface_mirror": {
                "mirror_to": {
                    "analyzer_ip_address": "4.4.4.4",
                    "analyzer_mac_address": null,
                    "analyzer_name": "Analyzer_Nir",
                    "juniper_header": true,
                    "nh_mode": "dynamic",
                    "routing_instance": "default-domain:admin:VM-MGT:VM-MGT",
                    "static_nh_header": null,
                    "udp_port": null
                },
                "traffic_direction": "both"

$ mirror --dump
Mirror Table

Flags:D=Dynamic Mirroring

Index    NextHop    Flags    VNI
------------------------------------------------
    0         44       D          0
    1          1                  0
    2        191       D          0
    3        185       D          0

vif0/11     PMD: tap4d070662-bd
            Type:Virtual HWaddr:00:00:5e:00:01:00 IPaddr:10.85.190.114
            Vrf:6 Flags:PMrMtL3L2D QOS:-1 Ref:23 Mirror index 2
            RX port   packets:29077 errors:0
            RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
            RX packets:29077  bytes:1279420 errors:0
            TX packets:412646  bytes:17331372 errors:0
            Ingress Mirror Metadata: 3 1b 64 65 66 61 75 6c 74 2d 64
                                     6f 6d 61 69 6e 3a 61 64 6d 69 6e
                                     3a 56 4d 2d 4d 47 54 ff 0
            Egress Mirror Metadata: 4 1b 64 65 66 61 75 6c 74 2d 64 6f
                                    6d 61 69 6e 3a 61 64 6d 69 6e 3a
                                    56 4d 2d 4d 47 54 ff 0
            TX port   packets:412646 errors:0 syscalls:412646
            Drops:29077

vif0/30     PMD: tapc6df2ab2-1c
            Type:Virtual HWaddr:00:00:5e:00:01:00 IPaddr:10.11.12.3
            Vrf:23 Flags:MrMtL3L2D QOS:-1 Ref:23 Mirror index 2
            RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
            RX packets:0  bytes:0 errors:0
            TX packets:412358  bytes:17319036 errors:0
            Ingress Mirror Metadata: 3 20 64 65 66 61 75 6c 74 2d 64
                                     6f 6d 61 69 6e 3a 61 64 6d 69 6e
                                     3a 56 4e 2d 4e 69 72 2d 54 65 73
                                     74 ff 0
            Egress Mirror Metadata: 4 20 64 65 66 61 75 6c 74 2d 64 6f
                                    6d 61 69 6e 3a 61 64 6d 69 6e 3a
                                    56 4e 2d 4e 69 72 2d 54 65 73 74
                                    ff 0
            TX port   packets:412358 errors:0 syscalls:412357
            Drops:0

The best practices to follow are:

  • Always provide an analyzer_name when configuring port mirroring
  • Always avoid using the same analyzer_name to more than one port mirroring instance