CE1--et-0/0/0-----et-0/0/1—PE1--et-0/1/0-----et-0/1/0—PE2--et-0/0/1-----et-0/0/0--CE2
labroot@PE1-re0> show vpls connections
Instance: 00/VLXM/000008/LUMN
Mesh-group connections: Pseudowire_Static
Neighbor Type St Time last up # Up trans
100.124.108.10(vpls-id 1000005136) rmt Up Nov 15 07:47:38 2024 1
Remote PE: 100.124.108.10, Negotiated control-word: No
Incoming label: 39942, Outgoing label: 39941
Negotiated PW status TLV: No
Local interface: lsi.1048598, Status: Up, Encapsulation: ETHERNET
Description: Intf - vpls 00/VLXM/000008/LUMN neighbor 100.124.108.10 vpls-id 1000005136
Flow Label Transmit: No, Flow Label Receive: No
100.124.108.6(vpls-id 1000005136)( rmt Up Nov 15 07:47:38 2024 1
Remote PE: 100.124.108.6, Negotiated control-word: No
Incoming label: 39940, Outgoing label: 39939
Local interface: lsi.1048599, Status: Up, Encapsulation: ETHERNET
Description: Intf - vpls 00/VLXM/000008/LUMN neighbor 100.124.108.6 vpls-id 1000005136
100.124.108.2(vpls-id 1000005136)( rmt Up Nov 15 07:53:40 2024 2
Remote PE: 100.124.108.2, Negotiated control-word: No
Incoming label: 39937, Outgoing label: 39938
Local interface: lsi.1048600, Status: Up, Encapsulation: ETHERNET
Description: Intf - vpls 00/VLXM/000008/LUMN neighbor 100.124.108.2 vpls-id 1000005136
labroot@PE2-re0> show vpls connections
Instance: green
VPLS-id: 1000005136
100.124.108.9(vpls-id 1000005136)( rmt Up Nov 15 07:44:20 2024 1
Remote PE: 100.124.108.9, Negotiated control-word: No
Incoming label: 39941, Outgoing label: 39942
Local interface: lsi.1048594, Status: Up, Encapsulation: ETHERNET
Description: Intf - vpls green neighbor 100.124.108.9 vpls-id 1000005136
100.124.108.5(vpls-id 1000005136)( rmt Up Nov 15 07:44:20 2024 1
Remote PE: 100.124.108.5, Negotiated control-word: No
Incoming label: 39939, Outgoing label: 39940
Local interface: lsi.1048595, Status: Up, Encapsulation: ETHERNET
Description: Intf - vpls green neighbor 100.124.108.5 vpls-id 1000005136
100.124.108.1(vpls-id 1000005136)( rmt Up Nov 15 07:44:20 2024 1
Remote PE: 100.124.108.1, Negotiated control-word: No
Incoming label: 39938, Outgoing label: 39937
Local interface: lsi.1048596, Status: Up, Encapsulation: ETHERNET
Description: Intf - vpls green neighbor 100.124.108.1 vpls-id 1000005136
labroot@CE2 > show interfaces terse
Interface Admin Link Proto Local Remote
et-0/0/0 up up
et-0/0/0.101 up up inet 192.168.1.2/24
multiservice
et-0/0/0.102 up up inet 192.168.2.2/24
et-0/0/0.103 up up inet 192.168.3.2/24
labroot@CE1> show interfaces terse
et-0/0/0.101 up up inet 192.168.1.1/24
et-0/0/0.102 up up inet 192.168.2.1/24
et-0/0/0.103 up up inet 192.168.3.1/24
<<< vpls flood to mesh-group
labroot@PE1-re0> show vpls flood extensive
Warning: License key missing; requires 'vpls' license
Name: 00/VLXM/000008/LUMN
CEs: 3
VEs: 3
Flood route prefix: 0x30004/51
Flood route type: FLOOD_GRP_COMP_NH
Flood route owner: __all_ces__
Flood group name: __all_ces__
Flood group index: 1
Nexthop type: comp
Nexthop index: 581
Flooding to:
Name Type NhType Index
__all_ces__ Group comp 580
Composition: split-horizon
et-0/0/1.3001 CE ucst 621
et-0/0/1.3002 CE ucst 584
et-0/0/1.3003 CE ucst 585
Pseudowire_Static Group comp 623
Composition: flood-to-all
lsi.1048598 VE indr 1048577
lsi.1048599 VE indr 1048578
lsi.1048600 VE indr 1048574
Flood route prefix: 0x30005/51
Flood route owner: Pseudowire_Static
Flood group name: Pseudowire_Static
Flood group index: 2
Nexthop index: 624
Flood route prefix: 0x30008/51
Flood route owner: __re_flood__
Flood group name: __re_flood__
Flood group index: 65534
Nexthop index: 589
<<< disable 1 interface on PE2 to PE1
[edit]
labroot@PE2-re0# set interfaces et-0/1/0.5001 disable
labroot@PE2-re0# commit
[edit routing-instances green protocols]
'vpls'
warning: requires 'vpls' license
[edit routing-options static]
'route 0.0.0.0/0'
warning: requires 'l3static' license
[edit protocols mpls]
'static-label-switched-path PET01/MESA042N4P001/PHNDAZ91RT009'
warning: requires 'static-mpls' license
commit complete
labroot@PE2-re0# exit
Exiting configuration mode
labroot@PE2re0> show interfaces terse
et-0/1/0.5001 down up inet 100.124.108.2/30
mpls
et-0/1/0.5002 up up inet 100.124.108.6/30
et-0/1/0.5003 up up inet 100.124.108.10/30
labroot@PE1-re0> show arp no-resolve expiration-time
MAC Address Address Interface TTE Flags
20:93:39:d5:4b:79 100.124.108.2 et-0/1/0.5001 33 none
20:93:39:d5:4b:79 100.124.108.6 et-0/1/0.5002 61 none
20:93:39:d5:4b:79 100.124.108.10 et-0/1/0.5003 24 none
labroot@PE1-re0> show pfe route ip prefix 100.124.108.2
Slot 0
Index Destination NH Id NH Type NH Token GUID
----- -------------------------------- --------- --------- --------- --------
0 100.124.108.2 586 Unicast 7283 0
<<< no longer arp reply to vpls neighbor 100.124.108.2
20:93:39:d5:4b:79 100.124.108.6 et-0/1/0.5002 12 none
20:93:39:d5:4b:79 100.124.108.10 et-0/1/0.5003 15 none
<<< NH type transitions from Unicast to Hold as expected
0 100.124.108.2 586 Hold 7283 0
labroot@PE1-re0> start shell
% nhinfo -Ri 623
623(comp, vpls, ref:3, fn:Flood, sub:vpls)
1048577(indr, inet, et-0/1/0.5003, ref:4)
606(ucst, inet, et-0/1/0.5003, ref:2)
1048578(indr, inet, et-0/1/0.5002, ref:4)
618(ucst, inet, et-0/1/0.5002, ref:2)
1048574(indr, inet, et-0/1/0.5001, ref:4)
592(hold, inet, et-0/1/0.5001, ref:2)
<<< no longer able to flood to flood group once 1 interface in the group is on hold status.
labroot@CE1> ping 192.168.2.2 count 1
PING 192.168.2.2 (192.168.2.2): 56 data bytes
--- 192.168.2.2 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
labroot@CE1> show arp no-resolve
MAC Address Address Interface Flags
00:00:5e:00:01:0b 10.85.165.1 fxp0.0 none
d0:07:ca:58:9a:a0 10.85.165.3 fxp0.0 none
fe:54:00:04:11:47 128.0.0.16 em3.0 none
fe:54:00:04:11:47 128.0.0.101 em3.0 none
b4:f9:5d:2b:4a:5f 192.168.1.2 et-0/0/0.101 none
fe:54:00:af:4d:ca 192.168.1.2 em2.32768 none
Total entries: 6
On PE2:
When one of the interfaces is disabled, the lsi is removed from the flood composite nh.
As static lsp is configured the lsi interface will not go down.
set interfaces et-0/1/0 flexible-vlan-tagging
set interfaces et-0/1/0 mtu 9192
set interfaces et-0/1/0 encapsulation flexible-ethernet-services
set interfaces et-0/1/0 unit 5001 disable
set interfaces et-0/1/0 unit 5001 vlan-id 131
set interfaces et-0/1/0 unit 5001 family inet address 100.124.108.2/30
set interfaces et-0/1/0 unit 5001 family mpls
set interfaces et-0/1/0 unit 5002 vlan-id 132
set interfaces et-0/1/0 unit 5002 family inet address 100.124.108.6/30
set interfaces et-0/1/0 unit 5002 family mpls
set interfaces et-0/1/0 unit 5003 vlan-id 133
set interfaces et-0/1/0 unit 5003 family inet address 100.124.108.10/30
set interfaces et-0/1/0 unit 5003 family mpls
labroot@PE2-re0# run show vpls flood
Name: green
VEs: 2
Flood Routes:
Prefix Type Owner NhType NhIndex
0x30005/51 FLOOD_GRP_COMP_NH __all_ces__ comp 624
0x30006/51 FLOOD_GRP_COMP_NH Pseudowire_Static comp 574
0x30003/51 FLOOD_GRP_COMP_NH __re_flood__ comp 625
root@PE2-re0-fpc0:pfe> show nh recursive index 624
nexthop-id: 624, type: Composite, Func-Type: Flood, proto: vpls, flags: 0x0, token:3166
nexthop-id: 623, type: Composite, Func-Type: SH, proto: vpls, flags: 0x0, token:3036
nexthop-id: 575, type: Unicast, Func-Type: -, proto: vpls, flags: 0x0, token:3789
nexthop-id: 633, type: Unicast, Func-Type: -, proto: vpls, flags: 0x0, token:4054
nexthop-id: 634, type: Unicast, Func-Type: -, proto: vpls, flags: 0x0, token:4093
nexthop-id: 573, type: Composite, Func-Type: Flood, proto: vpls, flags: 0x0, token:5828. >>>>>>>>>>>>>>>>>> Has only two entries.
nexthop-id: 1048575, type: Indirect, Func-Type: -, proto: vpls, flags: 0x0, token:5818
nexthop-id: 571, type: Unicast, Func-Type: -, proto: vpls_tag, flags: 0x5, token:5806
nexthop-id: 1048574, type: Indirect, Func-Type: -, proto: vpls, flags: 0x0, token:5811
nexthop-id: 569, type: Unicast, Func-Type: -, proto: vpls_tag, flags: 0x5, token:5802
On PE1:
labroot@PE1-re0# run show vpls flood
0x30005/51 FLOOD_GRP_COMP_NH __all_ces__ comp 691
0x30006/51 FLOOD_GRP_COMP_NH Pseudowire_Static comp 692
0x30003/51 FLOOD_GRP_COMP_NH __re_flood__ comp 689
root@PE1-re0-fpc0:pfe> show nh recursive index 691
nexthop-id: 642, type: Composite, Func-Type: Flood, proto: vpls, flags: 0x0, token:4589
nexthop-id: 641, type: Composite, Func-Type: SH, proto: vpls, flags: 0x0, token:4582
nexthop-id: 733, type: Unicast, Func-Type: -, proto: vpls, flags: 0x0, token:3022
nexthop-id: 734, type: Unicast, Func-Type: -, proto: vpls, flags: 0x0, token:3061
nexthop-id: 735, type: Unicast, Func-Type: -, proto: vpls, flags: 0x0, token:3100
nexthop-id: 639, type: Composite, Func-Type: Flood, proto: vpls, flags: 0x0, token:4566
nexthop-id: 1048574, type: Indirect, Func-Type: -, proto: vpls, flags: 0x0, token:4520
nexthop-id: 636, type: Hold, Func-Type: -, proto: vpls, flags: 0x0, token:4491 >>>>>>>>>> Corresponding NH is set to HOLD from UNICAST.
nexthop-id: 1048575, type: Indirect, Func-Type: -, proto: vpls, flags: 0x0, token:4536
nexthop-id: 637, type: Unicast, Func-Type: -, proto: vpls_tag, flags: 0x5, token:4492
nexthop-id: 1048576, type: Indirect, Func-Type: -, proto: vpls, flags: 0x0, token:4543
nexthop-id: 638, type: Unicast, Func-Type: -, proto: vpls_tag, flags: 0x5, token:4495
root@PE1-re0-fpc0:pfe> show sandbox token detail 4491 >>>>>>>>>>>>>>>>>>> The NH token is programmed to discard.
AftNode : AftDiscard token:4491 group:0 tag:nh(nexthop) NhIndex:685 Type:6 proto:27 ifl-index:359 nodeMask:Default
JnhHandle : JnhHandleDiscard Jnh:0x1080332e90000000 PfeInst:ff Paddr:0x0 Vaddr:0x19af9 AppType:NH
JNH decode:
ModifyNH: Subcode=SetErr
JNH_BUF_MOD_DATA_ERR:
reason = 0x000000c2
Reason name: discard route
Previous nodes count: 1
Next nodes count : 0
Entries count : 0
Now when a packet from to PE1 from the CE1 facing IFL, the flood NH will be executed.
root@PE1-re0-fpc0:pfe> show nh detail index 642
Nexthop Info:
NH Index : 642
NH Type : Composite
NH Proto : vpls
NH Flags : 0x0
IF Name : -
NH Token Id : 4589
NH Route Table Id : 0
Sgid : 0
NH Composite Func type : Flood
Flood nexthop information :
num nexthops : 2
nexthop 1 : 641
nexthop 2 : 639
Platform Info
-------------
FabricToken: 4592
IngressToken: 4589
EgressToken: 4591
root@PE1-re0-fpc0:pfe> show sandbox token detail 4589
AftNode : AftIndirect token:4589 group:0 tag:nh(layer2) NhIndex:642 Type:24 proto:27 comp-func:1 nodeMask:Default indirect:4588 hwInstall:1 proto: index:0
JnhHandle : JnhHandleIndirect Jnh:0x20066f240000000c PfeInst:ff Paddr:0x1019be1 Vaddr:0x19be1 AppType:NH
UcodeNH: Indirect Decode: Indirect, Next = 0x19bc9, ,
Indirect target nexthop details:Addr:Nexthop 0x19bc9 Paddr:0x1019bc9, Data = 0x082c04e000030000
Previous nodes count: 3
Next nodes count : 1
NextNode: 4588
root@PE1-re0-fpc0:pfe> show sandbox token detail 4588
AftNode : AftReplicateIngress token:4588 group:0 tag:nh(layer2) NhIndex:642 Type:24 proto:27 comp-func:1 nodeMask:Default DebugStr: AftReplicateIngress-Flood
JnhHandle : 25JnhHandleReplicateIngress Jnh:0x82c04e000030000 PfeInst:ff Paddr:0x0 Vaddr:0x0 AppType:NH Mcast List VA24
CallNH:desc_ptr:0xb0138, mode=0, count=0x4
0x0b0134 0 : 0x1100337200c00282
0x0b0135 1 : 0x20066f800000000c
0x0b0136 2 : 0x20066f3c0000000c
0x0b0137 3 : 0x1100337200c00282
BmMcast JNH : 0x1100337200c00282
Final entries description:
--------------------------
Number of Final Entries : 0
Non Final entries description:
------------------------------
NH : 639 CallTblIdx : 2
NH : 641 CallTblIdx : 1
Number of Non Final Entries : 2
Entries count : 2
AftEntry: parentToken:4588 entryMask:ffffffffffffffff index:639 nextToken:4566
AftEntry: parentToken:4588 entryMask:ffffffffffffffff index:641 nextToken:4582
root@PE1-re0-fpc0:pfe> show jnh vread inst 0 vaddr 0x19bcf nwords 1 NH
Addr:Nexthop 0x19bcf Paddr:0x1019bcf, Data = 0x0806469800040000
root@piastri-re0-fpc0:pfe> show jnh decode inst 0 word 0x0806469800040000 >>>>>>> Entries 2, 3, and 4 correspond to the three indirect NH in the NH topo.
CallNH:desc_ptr:0x191a6, mode=0, count=0x5
0x0191a1 0 : 0x1100336c60c0027f
0x0191a2 1 : 0x20066b480000000c
0x0191a3 2 : 0x20066b700000000c
0x0191a4 3 : 0x20066b780000000c
0x0191a5 4 : 0x1100336c60c0027f
Once the NH is set to discard, during this call table execution, the packets will be dropped without further processing, with the hold route as an exception.
PR1849854 - When static lsp is configured, lsi is not deleted when the ifl goes down and are not removed from the flood group. The underlying unicast NH changes to HOLD type and the nh token is updated as discard. Issue Is applicable on all MX devices with AFT line cards (MPC10, MPC11, LC9600, Mx304) Fix is available from 22.4R3-S6