This article describes how to dump various tables on nfp-vrouter SmartNIC-based Contrail compute nodes. These tables can be helpful while troubleshooting issues with a customer's nfp-vrouter environment.
A SmartNIC-based vRouter has an offloaded structure so an offloaded flow, l2, MPLS, mirror, NH table, and route table are maintained by the nfp_vrouter kernel module. The following specifies the list of files that contain a formatted dump of tables that are read directly from the NFP as well as the method to display the corresponding vRouter kernel table.
/sys/kernel/debug/nfp_vrouter/bond
cat /proc/net/bonding/bond0
/sys/kernel/debug/nfp_vrouter/flowtable
flow -l
/sys/kernel/debug/nfp_vrouter/l2
rt --family bridge --dump [vrf]
/sys/kernel/debug/nfp_vrouter/mirror
mirror --dump
/sys/kernel/debug/nfp_vrouter/mpls
mpls --dump
/sys/kernel/debug/nfp_vrouter/nexthop
nh --list
/sys/kernel/debug/nfp_vrouter/vif
vif --list
/sys/kernel/debug/nfp_vrouter/vxlan
vxlan --dump
/sys/kernel/debug/nfp_vrouter/routes/nfplpm_state
rt --dump [vrf]
Some examples of the above table
To access tables from the agilio_vrouter container:
root@compute:~# docker exec -it agilio_vrouter bash
To dump flow -l:
Note the offloaded flow entries as shown here:
(agilio_vrouter):/# cat /host/sys/kernel/debug/nfp_vrouter/flowtable INDEX HASH KEY(SIP|DIP|FAMILY|PROTO|SPORT|DPORT|NH) ACTION 1704648 0x00D9E703 (11.95.56.172|2.95.56.172|2|6|22|32794|23)NAT 1960824 0x00C3AA21 (11.95.56.172|11.37.90.172|2|1|18834|0|23)FWD 2486184 0x001E24A7 (7.101.168.192|4.0.254.169|2|6|32794|22|5)NAT 3514216 0x00F67577 (11.37.90.172|11.95.56.172|2|1|18834|0|23)FWD
To dump bond interface information:
(agilio_vrouter):/# cat /host/sys/kernel/debug/nfp_vrouter/bond Mode=0 Xmit_policy=0 active_slaves=0 0 0 0 0 0 0 0 0
To dump the L2 nfp-vrouter offloaded table:
(agilio_vrouter):/# cat /host/sys/kernel/debug/nfp_vrouter/l2 Index Hash KEY(VRF|MAC) LABEL NH_index 9288 0x0076D313 (2|00:15:4d:12:21:fd) 0 3 16144 0x00306B45 (0|00:00:5e:00:01:00) 0 3 27328 0x00DCDB03 (2|02:34:37:89:a9:c6) 29 34 53928 0x00EC0405 (1|00:00:5e:00:01:00) 0 3 55216 0x0091BB03 (0|9a:6f:50:0c:fa:6f) 16777215 1 57104 0x00736DED (0|00:25:90:c4:98:b2) 16777215 1 57248 0x006A7839 (2|02:00:00:00:00:02) 16777215 12 65344 0x00ADE413 (1|ff:ff:ff:ff:ff:ff) 2 25 65384 0x0068AFB5 (2|02:28:80:65:3e:ca) 34 20 68248 0x000E8AB9 (2|02:14:b9:2b:2f:bf) 29 20 70272 0x001262D1 (1|00:15:4d:12:21:fd) 19 8 93136 0x0072A3BB (1|00:15:4d:12:21:f3) 19 20 95208 0x00853A0B (2|02:00:00:00:00:01) 16777215 12 99944 0x00AECCFB (0|00:15:4d:12:21:f3) 16777215 1 114000 0x00B1A2FF (0|00:25:90:c4:98:aa) 16777215 1 117128 0x00C955D1 (2|ff:ff:ff:ff:ff:ff) 6 36 124000 0x0088B5C7 (2|00:00:5e:00:01:00) 0 3 130872 0x00CE0D91 (0|00:15:4d:12:21:fd) 0 3
To dump the mirror table:
(Note that we do not see any entries in the example because no mirroring is configured.)
(agilio_vrouter):/# cat /host/sys/kernel/debug/nfp_vrouter/mirror Index RID NH_ID Flags VNI
To retrieve the MPLS table that is offloaded on a SmartNIC:
(agilio_vrouter):/# cat /host/sys/kernel/debug/nfp_vrouter/mpls Label NH_index 16 5 17 6 18 7 19 8 20 9 21 21 22 25 23 26 24 36 25 28 26 29 27 30 28 31 29 34 4609 25 4621 36
To dump the next-hop:
Observe the offloaded NH entries in the following example:
(agilio_vrouter):/# cat /host/sys/kernel/debug/nfp_vrouter/nexthop Index TYPE FAMILY FLAGS VRF OIF ECMP(nh|label) 0 DISCARD 2 0x00001 0 0 1 DISCARD 2 0x200001 0 0 3 L2 RCV 2 0x200001 0 0 4 ENCAP 2 0x200001 0 0 5 ENCAP 2 0x200003 0 1 6 ENCAP 2 0x200001 0 1 7 ENCAP 2 0x200021 0 1 8 ENCAP 7 0x200001 0 1 9 ENCAP 7 0x200003 0 1 10 RCV 2 0x200101 1 1 11 RCV 2 0x200003 1 1 12 ENCAP 2 0x200001 -1 2 13 ENCAP 2 0x200003 -1 2 14 RESOLVE 2 0x200001 0 0
To view the VIF entries as observed by nfp-vrouter:
(agilio_vrouter):/# cat /host/sys/kernel/debug/nfp_vrouter/vif VIF VRFID TYPE FLAGS NH MTU VLAN VF PORT/QID BOND 0 0 P 0x2002C0 4 1514 65535 0 0 -1 1 0 H 0x200140 5 0 65535 0 0 0 2 65535 A 0x200040 0 0 65535 0 0 0 3 2 V 0x2001C1 28 9000 65535 58 0 0
To dump the VXLAN table:
(agilio_vrouter):/# cat /host/sys/kernel/debug/nfp_vrouter/vxlan VNID NH_index 2 21 6 26
To dump the forwarding table:
Observe the offloaded entries per vrf for both v4 and v6 families here:
(agilio_vrouter):/# cat /host/sys/kernel/debug/nfp_vrouter/routes/nfplpm_state <clipped> vrfid_len: 8 v4_s0_nbits: 16 V4 FW active buf: 1 V4 last trie generate: Successful V4 last trie commit: Successful <clipped> v4 routes count: 21 v4 routes max pfxlen (ip + vrfid): 40 1,0.0.0.0/0 --> [NextHop 1, label 0] 0,0.0.0.0/0 --> [NextHop 15, label 0] 0,224.0.0.0/8 --> [NextHop 10, label 0] 0,192.168.40.0/24 --> [NextHop 14, label 0] 2,59.0.0.0/24 --> [NextHop 1, label 0] 1,192.168.40.51/32 --> [NextHop 6, label 0] 0,192.168.40.254/32 --> [NextHop 15, label 0] 0,192.168.40.51/32 --> [NextHop 10, label 0] 0,192.168.40.48/32 --> [NextHop 16, label 0] 0,192.168.40.49/32 --> [NextHop 17, label 0] 1,192.168.40.52/32 --> [NextHop 20, label 17] 0,192.168.40.52/32 --> [NextHop 19, label 0] 1,169.254.169.254/32 --> [NextHop 11, label 0] 0,192.168.40.9/32 --> [NextHop 18, label 0] 2,59.0.0.254/32 --> [NextHop 13, label 0] 2,8.8.8.8/32 --> [NextHop 13, label 0] 2,169.254.169.254/32 --> [NextHop 11, label 0] 2,59.0.0.1/32 --> [NextHop 28, label 0] 0,169.254.0.3/32 --> [NextHop 28, label 0] 2,59.0.0.2/32 --> [NextHop 20, label 25] 2,59.0.0.3/32 --> [NextHop 20, label 30] V6 FW active buf: 1 V6 last trie generate: Successful V6 last trie commit: Successful <clipped> v6 active per-vrfid route sets: 3 route set for VRFID 2, routes count 3: 2,ff02:0000:0000:0000:0000:0000:0000:0002/128 --> [NextHop 12, label 0] 2,ff02:0000:0000:0000:0000:0000:0000:0001/128 --> [NextHop 12, label 0] 2,fe80:0000:0000:0000:0000:0000:5e00:0100/128 --> [NextHop 12, label 0] route set for VRFID 1, routes count 3: 1,ff02:0000:0000:0000:0000:0000:0000:0002/128 --> [NextHop 12, label 0] 1,ff02:0000:0000:0000:0000:0000:0000:0001/128 --> [NextHop 12, label 0] 1,fe80:0000:0000:0000:0000:0000:5e00:0100/128 --> [NextHop 12, label 0] route set for VRFID 0, routes count 1: 0,ff00:0000:0000:0000:0000:0000:0000:0000/8 --> [NextHop 10, label 0] (agilio_vrouter):/#