Description

This article provides a scenario where the vrf-table-label is not working, along with a work-around.

Symptoms

The vrf-table-label is not working when configured on multiple logical systems (or a logical and main instance) when the core-facing physical interface (IFD) is the same and hosted on ichip (DPC)/ABC/LMNR.

alt

The common interface for the main router and logical-router is ge-5/1/5 .

Configuration on R1:

root@R1-re0# show routing-instances
VPN_B {
    instance-type vrf;
    interface lo0.3;
    route-distinguisher 2:1;
    vrf-target target:2:1;
    vrf-table-label;
}

root@R1-re0# show logical-systems R1 routing-instances
VPN_A {
    instance-type vrf;
    interface ge-5/1/9.0;
    interface lo0.2;
    route-distinguisher 1:1;
    vrf-target target:1:1;
    vrf-table-label;


root@R1-re0# run show route table mpls logical-system R1 | match VPN
16        *[VPN/0] 01:37:53
          to table VPN_A.inet.0, Pop

[edit]
root@R1-re0# run show route table mpls | match VPN
16       *[VPN/0] 01:13:55
         to table VPN_B.inet.0, Pop


root@R1-re0# run show route advertising-protocol bgp 172.16.1.3

VPN_B.inet.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
* 200.1.1.2/32 Self 100 I


root@R1-re0# run show route advertising-protocol bgp 172.16.1.4 logical-system R1

VPN_A.inet.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
* 100.1.1.2/32 Self 100 I
* 100.1.2.0/24 Self 100 I


The PFE (Packet Forwarding Engine) has the label 16 programmed on the same core facing interface IFD, ge-5/1/5, for all logical-systems/main router.

Now, the problem happens depending upon which logical systems/main router VRF (Virtual Routing and Forwarding) label gets programmed last in the iif (incoming interface) channel table. IFL (logical interface) which gets programmed last will work fine and it will override what was earlier programmed on the PFE.


R1 output when ping from R3:

lab@R3-#run ping 200.1.1.2 routing-instance VPN_B rapid count 10000000 source 200.2.2.1
PING 200.1.1.2 (200.1.1.2): 56 data bytes
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!


root@R1-re0> show chassis hardware | match "FPC 5" 
FPC 5            REV 25   750-022765   CAAA5577          DPCE 20x 1GE + 2x 10GE R


root@R1-re0> request pfe execute command "show ifd brief" target fpc5 | match ge-5/1/5     
GOT:   159  ge-5/1/5             Ethernet     0x0000000000008000     5  Up


root@R1-re0> request pfe execute command "show iif ifd 159" target fpc5 | no-more | except "    0    "
SENT: Ukern command: show iif ifd 159
GOT:
GOT: ge-5/1/5, stream 0, chip 1, lookup idx 2
GOT: Stream default 0xe0022710, srw 0x00000000, iif 141072
GOT: Lookup tree:
GOT:     idx  chan_type  key_len  level  tree_type  stream  refcnt
GOT:     -----  -----    -----    -----  -----     -----    -----
GOT:     Chan1      Chan2      iif    Pkt_cnt       pkt_rate    byte_cnt    continue
GOT:     ---------- --------- ----- -------------  --------- ------------- --------
GOT:      0x50000        0x88   364        247129        650      20758836
GOT:
GOT:   total channel entries entries 6
LOCAL: End of file
For IFD 159 (ge-5/1/5) IFL 364 gets programmed in the PFE.
root@R1-re0> request pfe execute command "show ifl 364" target fpc5| match "Logical|IPv4"
GOT: Logical interface lsi.1 (Index 364, Alias-Index 0 Peer-Index 0)
GOT:   Protocol: IPv4, MTU: 1496 bytes, Flags: 0x0000000000000080, Route table: 10


Routing-table 10 is associated with IFL 364 , which belongs to VPN_B.

root@R1-re0> request pfe execute command "show route ip table" target fpc5 | match "Name|10"
GOT:     Table Name                       Table Index (lrid ) # of Routes  Bytes        FRRP TID
GOT:     VPN_B.10                         10          (0    ) 7            864          low  65535  (seg-0) 65535  (seg-0) 65535  (seg-0) 65535  (seg-0)


R1 output when ping from R4 :

root@R4# run ping 100.1.1.2 routing-instance VPN_A rapid count 100000 source 100.2.2.1
PING 100.1.1.2 (100.1.1.2): 56 data bytes
..................................................................................


root@R1-re0> request pfe execute command "show ifd brief" target fpc5 | match ge-5/1/5
GOT:   159  ge-5/1/5             Ethernet     0x0000000000008000     5  Up

root@R1-re0> request pfe execute command "show iif ifd 159" target fpc5 | no-more | except "    0    "
SENT: Ukern command: show iif ifd 159
GOT:
GOT: ge-5/1/5, stream 0, chip 1, lookup idx 2
GOT: Stream default 0xe0022710, srw 0x00000000, iif 141072
GOT: Lookup tree:
GOT:     idx  chan_type  key_len  level  tree_type  stream  refcnt
GOT:     -----  -----    -----    -----  -----     -----    -----
GOT:     Chan1      Chan2      iif    Pkt_cnt       pkt_rate    byte_cnt    continue
GOT:     ---------- --------- ----- -------------  --------- ------------- --------
GOT:      0x50002         n/a   321          5753          1        431830
GOT:      0x50003         n/a 65541          4710          1        323765
GOT:
GOT:   total channel entries entries 6
LOCAL: End of file

Two IFLs are programmed in the PFE for IFD 159.

root@R1-re0> request pfe execute command "show ifl 321" target fpc5| match "Logical|IPv4"

GOT: Logical interface ge-5/1/5.10 (Index 321, Alias-Index 0 Peer-Index 0)
GOT:   Protocol: IPv4, MTU: 1500 bytes, Flags: 0x8000000200000000, Route table: 0


root@R1-re0> request pfe execute command "show ifl 65541" target fpc5| match "Logical|IPv4"
GOT: Logical interface ge-5/1/5.100 (Index 65541, Alias-Index 0 Peer-Index 0)
GOT:   Protocol: IPv4, MTU: 1500 bytes, Flags: 0x8000000200000080, Route table: 6

root@R1-re0> request pfe execute command "show route ip table" target fpc5 | match "Name|R1/"
GOT:     Table Name                       Table Index (lrid ) # of Routes  Bytes        FRRP TID
GOT:     R1/VPN_A.9                       9           (1    ) 12           1484         low  65535  (seg-0) 65535  (seg-0) 65535  (seg-0) 65535  (seg-0)
GOT:     R1/__master.anon__.7             7           (1    ) 5            616          low  65535  (seg-0) 65535  (seg-0) 65535  (seg-0) 65535  (seg-0)
GOT:     R1/default.6                     6           (1    ) 21           2600         low  65535  (seg-0) 65535  (seg-0) 65535  (seg-0) 65535  (seg-0)

For IFL 321 & 65541, the interface is showing ge-5/1/5.x instead of the lsi interface, and the route table is pointing towards inet.0 & R1/inet.0 respectively. This is indicating that due to wrong programming, traffic is getting leaked into the inet.0 or R1/inet.0 table instead of the VPN_A table and hence getting dropped. If you keep flapping the VRF (or vrf-table-label) it would fix the problem until the correct destination logical systems/vrf gets programmed last.  However, this also means that it would break the traffic for other VRFs which use the same label.
root@R1-re0# show | compare
[edit logical-systems R1 routing-instances VPN_A]
-     vrf-table-label;


root@R1-re0# commit
commit complete

lab@R1-re0# run show route table mpls logical-system R1 | match VPN
299840             *[VPN/170] 00:00:23
                    receive table VPN_A.inet.0, Pop


root@R1-re0# show | compare
[edit logical-systems R1 routing-instances VPN_A]
+     vrf-table-label;


root@R1-re0# commit
commit complete


lab@R1-re0# run show route table mpls logical-system R1 | match VPN
17                 *[VPN/0] 00:00:05
                    to table VPN_A.inet.0, Pop


root@R4# run ping 100.1.1.2 routing-instance VPN_A rapid count 100000 source 100.2.2.1
PING 100.1.1.2 (100.1.1.2): 56 data bytes
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

At R3;

lab@R3#run ping 200.1.1.2 routing-instance VPN_B rapid count 10000000 source 200.2.2.1
PING 200.1.1.2 (200.1.1.2): 56 data bytes
......................................................................................

Now VPN_A traffic is getting leaked into VPN_B or in inet.0.

Solution

MPCs (Modular Port Concentrator) do not have this problem as they do not use same IIF channel table infrastructure like DPC's (Dense Port Concentrator), hence this issue would not happen there.

Workaround :
  • Disable vrf-table-label in all VRF's, using the vt interface instead if necessary
  • Use separate physical core-facing interfaces for each logical system