This article discusses OVSDB issues that occur frequently.
Make sure all basic configurations are valid. Please refer to OVSDB and VXLAN Feature Guide (Chapter 2 Configuring OVSDB and VXLANS). Make sure VMWare NSX-V controllers are reachable. Ping the IP addresses of the controllers in CLI. In the following example, 100.0.0.1 is the IP address of a NSX-V controller. root@QFX5100> ping 100.0.0.1 count 5 PING 100.0.0.1 (100.0.0.1): 56 data bytes 64 bytes from 100.0.0.1: icmp_seq=0 ttl=64 time=10.600 ms 64 bytes from 100.0.0.1: icmp_seq=1 ttl=64 time=11.151 ms 64 bytes from 100.0.0.1: icmp_seq=2 ttl=64 time=11.137 ms 64 bytes from 100.0.0.1: icmp_seq=3 ttl=64 time=49.540 ms 64 bytes from 100.0.0.1: icmp_seq=4 ttl=64 time=11.156 ms Make sure the following certificates are copied under /var/db/certs . Refer to OVSDB and VXLAN Feature Guide (Chapter 2 for creating and installing an SSL Key and Certificate on a Juniper Networks Device for Connection with SDN Controllers). If the issue still exists, please open a case with Support for further investigation.
root@QFX5100> ping 100.0.0.1 count 5 PING 100.0.0.1 (100.0.0.1): 56 data bytes 64 bytes from 100.0.0.1: icmp_seq=0 ttl=64 time=10.600 ms 64 bytes from 100.0.0.1: icmp_seq=1 ttl=64 time=11.151 ms 64 bytes from 100.0.0.1: icmp_seq=2 ttl=64 time=11.137 ms 64 bytes from 100.0.0.1: icmp_seq=3 ttl=64 time=49.540 ms 64 bytes from 100.0.0.1: icmp_seq=4 ttl=64 time=11.156 ms
/var/db/certs
Logical switch (LS) should be created by the controller and it should be operational in Junos. LS Flags can contain any of the following values. Created by Controller – Created by controller, but not operational within Junos. Created by L2ALD – LS is operational within Junos, but not created by controller. L2ALD stands for Layer 2 Address Learning Daemon. Created by both – Created by NSX as well as operational within Junos. LS should be created by both for OVSDB to be functional. If LS is not created by both, please open a case with Support for further analysis. root@QFX5100> show ovsdb logical-switch Logical switch information: Logical Switch Name: cccedb61-0231-4aea-b799-5e9035e67b58 Flags: Created by both VNI: 1 Num of Remote MAC: 2 Num of Local MAC: 3
root@QFX5100> show ovsdb logical-switch Logical switch information: Logical Switch Name: cccedb61-0231-4aea-b799-5e9035e67b58 Flags: Created by both VNI: 1 Num of Remote MAC: 2 Num of Local MAC: 3
Whenever VLANs or port bindings are missing, verify the following: Verify whether logical switch and port bindings are pushed by NSX – Verify this information in the OVSDB server database on Juniper Networks Devices. If not, issue with NSX Controller. Use the OVSDB-client tool to dump the database and check the Logical_Switch table and the Physical_Port table. OVSDB@sw-ubuntu28:/home/OVSDB/openvswitch/openvswitch-2.3.3$ ./OVSDB/OVSDB-client -v dump tcp:10.92.82.94:6645 > ~/OVSDB_dbl OVSDB@sw-ubuntu28:/home/OVSDB/openvswitch/openvswitch-2.3.3$ vi ~/OVSDB_dbl Logical_Switch table _uuid description name tunnel_key ------------------------------------ ----------- -------- ---------- b18c2a4a-46f0-40fe-86f3-f6d7c14b1fdd "" "cccedb61-0231-4aea-b799-5e9035e67b58" 1 Physical_Port table _uuid description name port_fault_status vlan_bindings vlan_stats ------------------------------------ ----------- ---------- ----------------- ---------------------------------------- ---------- b61a7e02-a98b-4722-ba63-d28799dcc21c "" "xe-0/0/0" [] {1=b18c2a4a-46f0-40fe-86f3-f6d7c14b1fdd} {} In the above example, 10.92.82.94 is the management IP of the Juniper device. For installation and usage of the tool, refer to Open Virtual Switch website . For how to interpret OVSDB database, refer to the hardware VTEP database schema . Verify whether the configuration is pushed to Juniper Networks devices. Verify whether config is pushed. root@QFX5100> show configuration | display set | match cccedb61-0231-4aea-b799-5e9035e67b58 set vlans cccedb61-0231-4aea-b799-5e9035e67b58 interface xe-0/0/0.0 set vlans cccedb61-0231-4aea-b799-5e9035e67b58 vxlan vni 10 In the above example, a logical switch cccedb61-0231-4aea-b799-5e9035e67b58 has been pushed to the Juniper device. The logical switch is bound to VLAN 10 on PORT ge-0/0/19. Verify whether there are any OVSDB commit failures. Use CLI command, show ovsdb commit failures . root@QFX5100> show ovsdb commit failures Txn ID Logical-switch Port VLAN ID 1 cccedb61-0231-4aea-b799-5e9035e67b58 1 xe-0/0/0 1 If there are any OVSDB commit failures, see syslogs for any error messages. After correcting the error, cleared OVSDB commit failures will push the config. Currently, two error logs are displayed in syslog. UI_CONFIGURATION_ERROR from MGD (Management Daemon). Sep 22 21:20:53 QFX5100 dcd[42177]: UI_CONFIGURATION_ERROR: Process: dcd, path: [edit interfaces xe-0/0/0], statement: unit 0, VLAN-ID must be specified on tagged ethernet interfaces netconf txn failed error message from VGD (Virtual-Tunnel-End-Point-Management Daemon). Sep 22 21:20:53 QFX5100 vgd[42040]: Netconf txn failed: 1 To recover from the above issue, delete IFL configuration and "clear OVSDB commit failures". {master:0}[edit] root@QFX5100# delete interfaces xe-0/0/0 {master:0}[edit] root@QFX5100# commit {master:0}[edit] root@QFX5100> clear OVSDB commit failures {master:0} If there are no OVSDB commit failures and config is not pushed to MGD, please make sure that the switch is configured with a host name ('set system host-name ...'). The switch may need to be deleted and re-added back into the controller after configuring the host name. If there are still no OVSDB commit failures and config is not pushed to MGD, please open a case with Support for further debugging. If bindings exists in MGD (show configuration vlans) and not in L2ALD (show vlans), open a case with Support .
OVSDB@sw-ubuntu28:/home/OVSDB/openvswitch/openvswitch-2.3.3$ ./OVSDB/OVSDB-client -v dump tcp:10.92.82.94:6645 > ~/OVSDB_dbl OVSDB@sw-ubuntu28:/home/OVSDB/openvswitch/openvswitch-2.3.3$ vi ~/OVSDB_dbl Logical_Switch table _uuid description name tunnel_key ------------------------------------ ----------- -------- ---------- b18c2a4a-46f0-40fe-86f3-f6d7c14b1fdd "" "cccedb61-0231-4aea-b799-5e9035e67b58" 1 Physical_Port table _uuid description name port_fault_status vlan_bindings vlan_stats ------------------------------------ ----------- ---------- ----------------- ---------------------------------------- ---------- b61a7e02-a98b-4722-ba63-d28799dcc21c "" "xe-0/0/0" [] {1=b18c2a4a-46f0-40fe-86f3-f6d7c14b1fdd} {}
root@QFX5100> show configuration | display set | match cccedb61-0231-4aea-b799-5e9035e67b58 set vlans cccedb61-0231-4aea-b799-5e9035e67b58 interface xe-0/0/0.0 set vlans cccedb61-0231-4aea-b799-5e9035e67b58 vxlan vni 10 In the above example, a logical switch cccedb61-0231-4aea-b799-5e9035e67b58 has been pushed to the Juniper device. The logical switch is bound to VLAN 10 on PORT ge-0/0/19.
root@QFX5100> show configuration | display set | match cccedb61-0231-4aea-b799-5e9035e67b58 set vlans cccedb61-0231-4aea-b799-5e9035e67b58 interface xe-0/0/0.0 set vlans cccedb61-0231-4aea-b799-5e9035e67b58 vxlan vni 10
Use CLI command, show ovsdb commit failures . root@QFX5100> show ovsdb commit failures Txn ID Logical-switch Port VLAN ID 1 cccedb61-0231-4aea-b799-5e9035e67b58 1 xe-0/0/0 1
show ovsdb commit failures
root@QFX5100> show ovsdb commit failures Txn ID Logical-switch Port VLAN ID 1 cccedb61-0231-4aea-b799-5e9035e67b58 1 xe-0/0/0 1
Sep 22 21:20:53 QFX5100 dcd[42177]: UI_CONFIGURATION_ERROR: Process: dcd, path: [edit interfaces xe-0/0/0], statement: unit 0, VLAN-ID must be specified on tagged ethernet interfaces
Sep 22 21:20:53 QFX5100 vgd[42040]: Netconf txn failed: 1
{master:0}[edit] root@QFX5100# delete interfaces xe-0/0/0 {master:0}[edit] root@QFX5100# commit {master:0}[edit] root@QFX5100> clear OVSDB commit failures {master:0}
Logical switch can only be deleted by the controller after deleting all corresponding remote macs and port bindings. The following bindings need to be deleted before deleting logical switch. Remote macs Port bindings After port bindings get deleted, VGD deletes local mcast mac entries automatically. When logical switch is not deleted, verify in OVSDB database and VGD whether all corresponding dependencies are deleted for that logical switch. Use the OVSDB-client tool to dump the database and check all corresponding dependencies are deleted for that logical switch. If entries are deleted from OVSDB but not in VGD (show ovsdb interface, show ovsdb mac), see OVSDB commit failures (show ovsdb commit failures). If there are any entries in OVSDB commit failures, see syslogs for error messages and correct them. After correcting the errors, cleared OVSDB commit failures will push the config. If no entries in OVSDB commit failures and you still see the problem, open a case with Support for further debugging.
Verify whether logical switch is created by both controller and l2ald In the following example, a local MAC 00:10:94:00:00:01 is learned at OVSDB-managed interface xe-0/0/0.1 on QFX device QFX5100. The interface is bound to logical-switch cccedb61-0231-4aea-b799-5e9035e67b58. root@QFX5100> show ovsdb logical-switch Logical switch information: Logical Switch Name: cccedb61-0231-4aea-b799-5e9035e67b58 Flags: Created by both VNI: 1 Num of Remote MAC: 1 Num of Local MAC: 2 Verify whether the entry exists in show ethernet switching table root@QFX5100> show ethernet-switching table MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - OVSDB MAC) Ethernet switching table : 2 entries, 2 learned Routing instance : default-switch Vlan MAC MAC Age Logical name address flags interface cccedb61-0231-4aea-b799-5e9035e67b58 00:10:94:00:00:01 D - xe-0/0/0.1 cccedb61-0231-4aea-b799-5e9035e67b58 00:10:94:00:00:10 DO - vtep.32769 Verify whether the entry is added to OVSDB. Only if entry is added to OVSDB server database, it can be pushed as remote mac. First, verify with the ' show ovsdb mac ' CLI command. root@QFX5100> show ovsdb mac Logical Switch Name: cccedb61-0231-4aea-b799-5e9035e67b58 Mac IP Encapsulation Vtep Address Address Address ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 10.0.0.2 00:10:94:00:00:01 0.0.0.0 Vxlan over Ipv4 10.0.0.2 00:10:94:00:00:10 0.0.0.0 Vxlan over Ipv4 10.0.0.1 Second, verify if the MAC is in the Ucast_Macs_Local table. 10.92.82.94 is the IP of the QFX device QFX5100 where the MAC is locally learned. OVSDB@sw-ubuntu28:/home/OVSDB/openvswitch/openvswitch-2.3.3$./OVSDB/OVSDB-client -v dump tcp:10.92.82.94:6645 > ~/OVSDB_dbl Ucast_Macs_Local table MAC _uuid ipaddr locator logical_switch ------------------- ------------------------------------ --------- ------------------------------------ ------------------------------------ "00:10:94:00:00:01" 5e2d6a48-df15-4d26-a7f3-f3765e3c0eb0 "0.0.0.0" bceab452-92af-4b57-a39a-457074fb897d 27ae644b-f944-4fc5-9f57-fbeb92dcc3db If the MAC is not present, please open a case with Support for further investigation.
In the following example, a local MAC 00:10:94:00:00:01 is learned at OVSDB-managed interface xe-0/0/0.1 on QFX device QFX5100. The interface is bound to logical-switch cccedb61-0231-4aea-b799-5e9035e67b58. root@QFX5100> show ovsdb logical-switch Logical switch information: Logical Switch Name: cccedb61-0231-4aea-b799-5e9035e67b58 Flags: Created by both VNI: 1 Num of Remote MAC: 1 Num of Local MAC: 2
root@QFX5100> show ovsdb logical-switch Logical switch information: Logical Switch Name: cccedb61-0231-4aea-b799-5e9035e67b58 Flags: Created by both VNI: 1 Num of Remote MAC: 1 Num of Local MAC: 2
root@QFX5100> show ethernet-switching table MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - OVSDB MAC) Ethernet switching table : 2 entries, 2 learned Routing instance : default-switch Vlan MAC MAC Age Logical name address flags interface cccedb61-0231-4aea-b799-5e9035e67b58 00:10:94:00:00:01 D - xe-0/0/0.1 cccedb61-0231-4aea-b799-5e9035e67b58 00:10:94:00:00:10 DO - vtep.32769
show ovsdb mac
root@QFX5100> show ovsdb mac Logical Switch Name: cccedb61-0231-4aea-b799-5e9035e67b58 Mac IP Encapsulation Vtep Address Address Address ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 10.0.0.2 00:10:94:00:00:01 0.0.0.0 Vxlan over Ipv4 10.0.0.2 00:10:94:00:00:10 0.0.0.0 Vxlan over Ipv4 10.0.0.1
OVSDB@sw-ubuntu28:/home/OVSDB/openvswitch/openvswitch-2.3.3$./OVSDB/OVSDB-client -v dump tcp:10.92.82.94:6645 > ~/OVSDB_dbl Ucast_Macs_Local table MAC _uuid ipaddr locator logical_switch ------------------- ------------------------------------ --------- ------------------------------------ ------------------------------------ "00:10:94:00:00:01" 5e2d6a48-df15-4d26-a7f3-f3765e3c0eb0 "0.0.0.0" bceab452-92af-4b57-a39a-457074fb897d 27ae644b-f944-4fc5-9f57-fbeb92dcc3db
Local mac entries are deleted either through mac aging out or logical switch deletion. When mac entries are aged out, local mac entries are withdrawn from L2ALD and then deleted from VGD. Verify whether the entry exists in show ethernet switching table root@QFX5100> show ethernet-switching table MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - OVSDB MAC) Ethernet switching table : 2 entries, 2 learned Routing instance : default-switch Vlan MAC MAC Age Logical name address flags interface cccedb61-0231-4aea-b799-5e9035e67b58 00:10:94:00:00:01 D - xe-0/0/0.1 cccedb61-0231-4aea-b799-5e9035e67b58 00:10:94:00:00:10 DO - vtep.32769 If the MAC entry exists, it will not be deleted in OVSDB. If the MAC entry should be removed, but is still present, please open a case with Support for further debugging. Verify whether the entry exists in OVSDB. First, verify with the ' show ovsdb mac ' CLI command.root@QFX5100> show ovsdb mac Logical Switch Name: cccedb61-0231-4aea-b799-5e9035e67b58 Mac IP Encapsulation Vtep Address Address Address ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 10.0.0.2 00:10:94:00:00:01 0.0.0.0 Vxlan over Ipv4 10.0.0.2 00:10:94:00:00:10 0.0.0.0 Vxlan over Ipv4 10.0.0.1 If the MAC entry should be removed but is still present, please open a case with Support for further debugging.
Logical Switch Name: cccedb61-0231-4aea-b799-5e9035e67b58 Mac IP Encapsulation Vtep Address Address Address ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 10.0.0.2 00:10:94:00:00:01 0.0.0.0 Vxlan over Ipv4 10.0.0.2 00:10:94:00:00:10 0.0.0.0 Vxlan over Ipv4 10.0.0.1
Local mcast mac is added only if there exists any logical switch to port binding. Below is an example which shows a local mcast mac FF:FF:FF:FF:FF:FF associated to the local VTEP 10.0.0.2. root@QFX5100> show ovsdb mac Logical Switch Name: cccedb61-0231-4aea-b799-5e9035e67b58 Mac IP Encapsulation Vtep Address Address Address ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 10.0.0.2 00:10:94:00:00:01 0.0.0.0 Vxlan over Ipv4 10.0.0.2 When local mcast mac is not added, verify if there are any logical switch to port bindings pushed from the controller. root@QFX5100> show ovsdb logical-switch Logical switch information: Logical Switch Name: cccedb61-0231-4aea-b799-5e9035e67b58 Flags: Created by both VNI: 1 Num of Remote MAC: 1 Num of Local MAC: 1 If not, debug using the steps specified in section 1.3 vlans and port bindings missing. If the issue still exists, please open a case with Support for further investigation.
root@QFX5100> show ovsdb mac Logical Switch Name: cccedb61-0231-4aea-b799-5e9035e67b58 Mac IP Encapsulation Vtep Address Address Address ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 10.0.0.2 00:10:94:00:00:01 0.0.0.0 Vxlan over Ipv4 10.0.0.2
root@QFX5100> show ovsdb logical-switch Logical switch information: Logical Switch Name: cccedb61-0231-4aea-b799-5e9035e67b58 Flags: Created by both VNI: 1 Num of Remote MAC: 1 Num of Local MAC: 1
Local mcast mac is deleted only if all the port bindings for a particular logical switch are deleted. Below is an example which shows a local mcast mac FF:FF:FF:FF:FF:FF associated to the local VTEP 10.0.0.2. root@QFX5100> show ovsdb mac Logical Switch Name: cccedb61-0231-4aea-b799-5e9035e67b58 Mac IP Encapsulation Vtep Address Address Address ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 10.0.0.2 00:10:94:00:00:01 0.0.0.0 Vxlan over Ipv4 10.0.0.2 If local mcast mac is not deleted, verify whether any ls to port bindings exists. If any ls to port bindings exists, debug further using the steps specified in number 3 - Vlans and port bindings are missing.
Verify whether remote mac is pushed from OVSDB server. In the following example, 00:10:94:00:00:10 is a remote MAC added by controller. Verify whether the remote MAC is in the OVSDB Ucast_Macs_Remote table. 10.92.82.94 is the IP of the QFX device. OVSDB@sw-ubuntu28:/home/OVSDB/openvswitch/openvswitch-2.3.3$ ./OVSDB/OVSDB-client -v dump tcp:10.92.82.94:6645 > ~/OVSDB_dbl Ucast_Macs_Remote table MAC _uuid ipaddr locator logical_switch ------------------- ------------------------------------ ------ ------------------------------------ ------------------------------------ "00:10:94:00:00:10" aafa3df9-0bbb-418f-bb79-ebabfee6d77d "" 41290872-9c32-4939-a1c3-3f535d61ce49 27ae644b-f944-4fc5-9f57-fbeb92dcc3db If entry is in OVSDB server database and not listed in the output of CLI command, “show ovsdb mac”, open a case with Support for further debugging. root@QFX5100> show ovsdb mac Logical Switch Name: cccedb61-0231-4aea-b799-5e9035e67b58 Mac IP Encapsulation Vtep Address Address Address ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 10.0.0.2 00:10:94:00:00:01 0.0.0.0 Vxlan over Ipv4 10.0.0.2 00:10:94:00:00:10 0.0.0.0 Vxlan over Ipv4 10.0.0.1 If remote mac entry exists in both OVSDB server database and is in VGD (“show ovsdb mac”), but not in L2ALD (show ethernet switching table), open a case with Support for triaging the issue further. root@QFX5100> show ethernet-switching table MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - OVSDB MAC) Ethernet switching table : 2 entries, 2 learned Routing instance : default-switch Vlan MAC MAC Age Logical name address flags interface cccedb61-0231-4aea-b799-5e9035e67b58 00:10:94:00:00:01 D - xe-0/0/0.1 cccedb61-0231-4aea-b799-5e9035e67b58 00:10:94:00:00:10 DO - vtep.32769 If remote mac entry does not exist in OVSDB server, verify whether local macs are learned under remote vtep and are pushed to OVSDB server. For further debugging, refer to number 5 - Local mac not added in OVSDB.
OVSDB@sw-ubuntu28:/home/OVSDB/openvswitch/openvswitch-2.3.3$ ./OVSDB/OVSDB-client -v dump tcp:10.92.82.94:6645 > ~/OVSDB_dbl Ucast_Macs_Remote table MAC _uuid ipaddr locator logical_switch ------------------- ------------------------------------ ------ ------------------------------------ ------------------------------------ "00:10:94:00:00:10" aafa3df9-0bbb-418f-bb79-ebabfee6d77d "" 41290872-9c32-4939-a1c3-3f535d61ce49 27ae644b-f944-4fc5-9f57-fbeb92dcc3db
Verify whether remote mac entry is deleted from OVSDB server database. Refer to number 9. If remote mac entry is deleted from OVSDB server, but not from VGD (show ovsdb mac), open a case with Support . If remote mac entry is deleted from both OVSDB server and VGD, but not from L2ALD (show ethernet-switching table), open a case with Support for triaging this issue further. If remote mac entry is not deleted from OVSDB server, then verify under corresponding remote vtep, whether these local macs were deleted and updated correctly in OVSDB server database. Refer to number 6.
Remote mcast mac entry will be created only when service node is added in controller. Remote mcast entry will be deleted only when service node is deleted from controller. Whenever remote mcast mac entry is not added to VGD, verify whether remote mcast mac entry exists in OVSDB server. If it exists, then open a case with Support for further debugging. OVSDB@sw-ubuntu28:/home/OVSDB/openvswitch/openvswitch-2.3.3$ ./OVSDB/OVSDB-client -v dump tcp:10.92.82.94:6645 > ~/OVSDB_dbl Mcast_Macs_Remote table MAC _uuid ipaddr locator_set logical_switch ------------------- ------------------------------------ ------ ------------------------------------ ------------------------------------ "FF:FF:FF:FF:FF:FF" d3b154c5-441b-4dbf-ab15-dbd7e67f42b7 "" 6693d343-07d0-48a6-b948-4107b19c15f8 b18c2a4a-46f0-40fe-86f3-f6d7c14b1fdd The following example shows a remote mcast mac entry which is associated with a service node of IP 12.12.12.12. root@QFX5100> show ovsdb mac Logical Switch Name: cccedb61-0231-4aea-b799-5e9035e67b58 Mac IP Encapsulation Vtep Address Address Address ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 10.0.0.2 00:10:94:00:00:10 0.0.0.0 Vxlan over Ipv4 10.0.0.1 ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 12.12.12.12
OVSDB@sw-ubuntu28:/home/OVSDB/openvswitch/openvswitch-2.3.3$ ./OVSDB/OVSDB-client -v dump tcp:10.92.82.94:6645 > ~/OVSDB_dbl Mcast_Macs_Remote table MAC _uuid ipaddr locator_set logical_switch ------------------- ------------------------------------ ------ ------------------------------------ ------------------------------------ "FF:FF:FF:FF:FF:FF" d3b154c5-441b-4dbf-ab15-dbd7e67f42b7 "" 6693d343-07d0-48a6-b948-4107b19c15f8 b18c2a4a-46f0-40fe-86f3-f6d7c14b1fdd
root@QFX5100> show ovsdb mac Logical Switch Name: cccedb61-0231-4aea-b799-5e9035e67b58 Mac IP Encapsulation Vtep Address Address Address ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 10.0.0.2 00:10:94:00:00:10 0.0.0.0 Vxlan over Ipv4 10.0.0.1 ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 12.12.12.12
Verify whether “show ethernet-switching vxlan-tunnel-end-point source” has correct SVTEP-IP information. root@QFX5100> show ethernet-switching vxlan-tunnel-end-point source Logical System Name Id SVTEP-IP IFL L3-Idx <default> 0 10.0.0.2 lo0.0 0 L2-RTT Bridge Domain VNID MC-Group-IP default-switch cccedb61-0231-4aea-b799-5e9035e67b58 1 0.0.0.0 If not, verify whether inet address is configured for lo0 and following switch-options configuration is specified. root@QFX5100> show interfaces terse lo0 Interface Admin Link Proto Local Remote lo0 up up lo0.0 up up inet 10.0.0.2 --> 0/0 iso lo0.16385 up up inet set groups global interfaces lo0 unit 0 family inet address 10.0.0.2/32 primary set switch-options OVSDB-managed set switch-options vtep-source-interface lo0.0 If the issue still exists, please open a case with Support for further investigation.
root@QFX5100> show ethernet-switching vxlan-tunnel-end-point source Logical System Name Id SVTEP-IP IFL L3-Idx <default> 0 10.0.0.2 lo0.0 0 L2-RTT Bridge Domain VNID MC-Group-IP default-switch cccedb61-0231-4aea-b799-5e9035e67b58 1 0.0.0.0
root@QFX5100> show interfaces terse lo0 Interface Admin Link Proto Local Remote lo0 up up lo0.0 up up inet 10.0.0.2 --> 0/0 iso lo0.16385 up up inet set groups global interfaces lo0 unit 0 family inet address 10.0.0.2/32 primary set switch-options OVSDB-managed set switch-options vtep-source-interface lo0.0
Remote vtep will be created only upon receiving remote macs. Verify whether controller is pushing remote macs and entries exist in OVSDB server database. Verify whether the remote MAC is in the OVSDB Ucast_Macs_Remote table. In the following example, 10.92.82.94 is the IP of the QFX device. OVSDB@sw-ubuntu28:/home/OVSDB/openvswitch/openvswitch-2.3.3$ ./OVSDB/OVSDB-client -v dump tcp:10.92.82.94:6645 > ~/OVSDB_dbl Ucast_Macs_Remote table MAC _uuid ipaddr locator logical_switch ------------------- ------------------------------------ ------ ------------------------------------ ------------------------------------ "00:10:94:00:00:10" aafa3df9-0bbb-418f-bb79-ebabfee6d77d "" 41290872-9c32-4939-a1c3-3f535d61ce49 27ae644b-f944-4fc5-9f57-fbeb92dcc3db Verify whether VGD pushed remote macs to L2ALD. root@QFX5100> show ovsdb mac Logical Switch Name: cccedb61-0231-4aea-b799-5e9035e67b58 Mac IP Encapsulation Vtep Address Address Address ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 10.0.0.2 00:10:94:00:00:01 0.0.0.0 Vxlan over Ipv4 10.0.0.2 00:10:94:00:00:10 0.0.0.0 Vxlan over Ipv4 10.0.0.1 root@QFX5100> show ethernet-switching vxlan-tunnel-end-point remote mac-table MAC flags (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC) Logical system : <default> Routing instance : default-switch Bridging domain : cccedb61-0231-4aea-b799-5e9035e67b58, VLAN : NA, VNID : 1 MAC MAC Logical Remote VTEP address flags interface IP address 00:10:94:00:00:10 DO vtep.32769 10.0.0.1 If remote macs don’t exist in VGD (show ovsdb mac) and L2ALD (show ethernet-switching table), debug further by referring to number 9 - Remote mac not added in OVSDB. If remote mac entry exists in VGD and L2ALD, see whether corresponding ifl entry exists in L2ALD for remote vtep. If issues occur with remote vtep ifl entry, open a case with Support for further debugging. In the following example, 10.0.0.1 is the IP of the remote VTEP. root@QFX5100> show ethernet-switching vxlan-tunnel-end-point remote Logical System Name Id SVTEP-IP IFL L3-Idx <default> 0 10.0.0.2 lo0.0 0 RVTEP-IP IFL-Idx NH-Id VNID MC-Group-IP 10.0.0.1 558 1735 1 0.0.0.0 If corresponding remote vtep entry exists in L2ALD, but not in VGD, open a case with JTAC for further debugging.
root@QFX5100> show ovsdb mac Logical Switch Name: cccedb61-0231-4aea-b799-5e9035e67b58 Mac IP Encapsulation Vtep Address Address Address ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 10.0.0.2 00:10:94:00:00:01 0.0.0.0 Vxlan over Ipv4 10.0.0.2 00:10:94:00:00:10 0.0.0.0 Vxlan over Ipv4 10.0.0.1 root@QFX5100> show ethernet-switching vxlan-tunnel-end-point remote mac-table MAC flags (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC) Logical system : <default> Routing instance : default-switch Bridging domain : cccedb61-0231-4aea-b799-5e9035e67b58, VLAN : NA, VNID : 1 MAC MAC Logical Remote VTEP address flags interface IP address 00:10:94:00:00:10 DO vtep.32769 10.0.0.1
root@QFX5100> show ethernet-switching vxlan-tunnel-end-point remote Logical System Name Id SVTEP-IP IFL L3-Idx <default> 0 10.0.0.2 lo0.0 0 RVTEP-IP IFL-Idx NH-Id VNID MC-Group-IP 10.0.0.1 558 1735 1 0.0.0.0
When default factory settings are loaded on QFX5100, by default, unit 0 is created with family ethernet-switching for all interfaces. root@QFX5100# load factory-default warning: activating factory configuration root@QFX5100# show interfaces { xe-0/0/0 { unit 0 { family ethernet-switching { storm-control default; } } } In this case, when interface is added to OVSDB Configuration and port bindings are pushed from NSX, the transaction fails and moves to failed queue (show ovsdb commit failures). root@QFX5100> show ovsdb commit failures Txn ID Logical-switch Port VLAN ID 1 cccedb61-0231-4aea-b799-5e9035e67b58 1 xe-0/0/0 1 Currently, two error logs are displayed in syslog. UI_CONFIGURATION_ERROR from MGD. Sep 22 21:20:53 QFX5100 dcd[42177]: UI_CONFIGURATION_ERROR: Process: dcd, path: [edit interfaces xe-0/0/0], statement: unit 0, VLAN-ID must be specified on tagged ethernet interfaces netconf txn failed error message from VGD. Sep 22 21:20:53 QFX5100 VGD[42040]: Netconf txn failed: 1 Sep 22 21:20:53 QFX5100 VGD[42040]: <load-configuration action="merge"> <configuration> <vlans> <vlan> <vlan-name> cccedb61-0231-4aea-b799-5e9035e67b58 <vxlan> <vni>1 <interfaces> <interface> <name>xe-0/0/0 <flexible-vla Sep 22 21:20:53 QFX5100 VGD[42040]: n-tagging/> <encapsulation>extended-vlan-bridge <unit> < Sep 22 21:20:53 QFX5100 VGD[42040]: name>1 <vlan-id>1 < Sep 22 21:20:53 QFX5100 VGD[42040]: vlans> <vlan> <vlan-name> cccedb61-0231-4aea-b799-5e9035e67b58 <interface> <name>x Sep 22 21:20:53 QFX5100 VGD[42040]: e-0/0/0.1 To recover from the above issue, delete IFL configuration and "clear OVSDB commit failures" {master:0} root@ QFX5100# delete interfaces xe-0/0/0 {master:0}[edit] root@ QFX5100# commit {master:0} root@QFX5100> clear OVSDB commit failures If the issue still exists please open a case with Support for further investigation.
root@QFX5100# load factory-default warning: activating factory configuration root@QFX5100# show interfaces { xe-0/0/0 { unit 0 { family ethernet-switching { storm-control default; } } }
Sep 22 21:20:53 QFX5100 VGD[42040]: Netconf txn failed: 1 Sep 22 21:20:53 QFX5100 VGD[42040]: <load-configuration action="merge"> <configuration> <vlans> <vlan> <vlan-name> cccedb61-0231-4aea-b799-5e9035e67b58 <vxlan> <vni>1 <interfaces> <interface> <name>xe-0/0/0 <flexible-vla Sep 22 21:20:53 QFX5100 VGD[42040]: n-tagging/> <encapsulation>extended-vlan-bridge <unit> < Sep 22 21:20:53 QFX5100 VGD[42040]: name>1 <vlan-id>1 < Sep 22 21:20:53 QFX5100 VGD[42040]: vlans> <vlan> <vlan-name> cccedb61-0231-4aea-b799-5e9035e67b58 <interface> <name>x Sep 22 21:20:53 QFX5100 VGD[42040]: e-0/0/0.1
{master:0} root@ QFX5100# delete interfaces xe-0/0/0 {master:0}[edit] root@ QFX5100# commit {master:0} root@QFX5100> clear OVSDB commit failures
Verify whether vteps are reachable and it both of them are reachable from each other based on the lo0 address as source address. First, identify the IP address of the source VTEP. root@QFX5100> show ethernet-switching vxlan-tunnel-end-point source Logical System Name Id SVTEP-IP IFL L3-Idx <default> 0 10.0.0.2 lo0.0 0 L2-RTT Bridge Domain VNID MC-Group-IP default-switch cccedb61-0231-4aea-b799-5e9035e67b58 1 0.0.0.0 Second, find the IP address of the remote VTEP. root@QFX5100> show ethernet-switching vxlan-tunnel-end-point remote mac-table MAC flags (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC) Logical system : <default> Routing instance : default-switch Bridging domain : cccedb61-0231-4aea-b799-5e9035e67b58, VLAN : NA, VNID : 1 MAC MAC Logical Remote VTEP address flags interface IP address 00:10:94:00:00:10 DO vtep.32769 10.0.0.1 And then, ping the remote vtep ip with the lo0 address as source address root@QFX5100> ping 10.0.0.1 source 10.0.0.2 count 5 PING 10.0.0.1 (10.0.0.1): 56 data bytes 64 bytes from 10.0.0.1: icmp_seq=0 ttl=64 time=10.600 ms 64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=11.151 ms 64 bytes from 10.0.0.1: icmp_seq=2 ttl=64 time=11.137 ms 64 bytes from 10.0.0.1: icmp_seq=3 ttl=64 time=49.540 ms 64 bytes from 10.0.0.1: icmp_seq=4 ttl=64 time=11.156 ms Service node should be reachable for BUM traffic. Ping the service node ip with the lo0 address as source address. In the following example, 11.0.0.1 is the ip address of the service node. root@QFX5100> ping 11.0.0.1 source 10.0.0.2 count 5 PING 11.0.0.1 (11.0.0.1): 56 data bytes 64 bytes from 11.0.0.1: icmp_seq=0 ttl=64 time=10.600 ms 64 bytes from 11.0.0.1: icmp_seq=1 ttl=64 time=11.151 ms 64 bytes from 11.0.0.1: icmp_seq=2 ttl=64 time=11.137 ms 64 bytes from 11.0.0.1: icmp_seq=3 ttl=64 time=49.540 ms 64 bytes from 11.0.0.1: icmp_seq=4 ttl=64 time=11.156 ms If the issue still exists please open a case with Support for further investigation.
root@QFX5100> show ethernet-switching vxlan-tunnel-end-point remote mac-table MAC flags (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC) Logical system : <default> Routing instance : default-switch Bridging domain : cccedb61-0231-4aea-b799-5e9035e67b58, VLAN : NA, VNID : 1 MAC MAC Logical Remote VTEP address flags interface IP address 00:10:94:00:00:10 DO vtep.32769 10.0.0.1
root@QFX5100> ping 10.0.0.1 source 10.0.0.2 count 5 PING 10.0.0.1 (10.0.0.1): 56 data bytes 64 bytes from 10.0.0.1: icmp_seq=0 ttl=64 time=10.600 ms 64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=11.151 ms 64 bytes from 10.0.0.1: icmp_seq=2 ttl=64 time=11.137 ms 64 bytes from 10.0.0.1: icmp_seq=3 ttl=64 time=49.540 ms 64 bytes from 10.0.0.1: icmp_seq=4 ttl=64 time=11.156 ms
root@QFX5100> ping 11.0.0.1 source 10.0.0.2 count 5 PING 11.0.0.1 (11.0.0.1): 56 data bytes 64 bytes from 11.0.0.1: icmp_seq=0 ttl=64 time=10.600 ms 64 bytes from 11.0.0.1: icmp_seq=1 ttl=64 time=11.151 ms 64 bytes from 11.0.0.1: icmp_seq=2 ttl=64 time=11.137 ms 64 bytes from 11.0.0.1: icmp_seq=3 ttl=64 time=49.540 ms 64 bytes from 11.0.0.1: icmp_seq=4 ttl=64 time=11.156 ms