Issue:
With ~8M or more route scale, VRFs are not cleaned up after its deactivation / deletion via commit.
Model: mx2020
Junos: 21.2R3-S3.5
labroot@router> show route summary | no-more
Autonomous system number: 714
Router ID: 17.0.1.58
Maximum ECMP: 64
Highwater Mark (All time / Time averaged watermark)
RIB unique destination routes: 3001146 at 2024-01-26 23:11:35 / 300335
RIB routes : 21000588 at 2024-01-26 23:11:36 / 2100286
FIB routes : 2000786 at 2024-01-26 23:11:35 / 200249
VRF type routing instances : 2 at 2024-01-26 21:46:55
inet.0: 1000353 destinations, 10000072 routes (1000350 active, 0 holddown, 3 hidden)
Direct: 9 routes, 7 active
Local: 41 routes, 41 active
BGP: 10000014 routes, 1000297 active
Static: 3 routes, 3 active
Aggregate: 4 routes, 1 active
LDP: 1 routes, 1 active
CDNVRF.inet.0: 1000344 destinations, 10000060 routes (1000344 active, 0 holddown, 0 hidden)
Direct: 6 routes, 6 active
Local: 40 routes, 40 active
BGP: 10000014 routes, 1000298 active
apple.inet.0: 10 destinations, 10 routes (9 active, 0 holddown, 1 hidden)
Direct: 1 routes, 1 active
Local: 7 routes, 7 active
Aggregate: 2 routes, 1 active
mpls.0: 12 destinations, 12 routes (12 active, 0 holddown, 0 hidden)
MPLS: 10 routes, 10 active
VPN: 2 routes, 2 active
bgp.l3vpn.0: 1000306 destinations, 1000306 routes (1000306 active, 0 holddown, 0 hidden)
Direct: 7 routes, 7 active
BGP: 1000298 routes, 1000298 active
Aggregate: 1 routes, 1 active
inet6.0: 86 destinations, 88 routes (81 active, 0 holddown, 5 hidden)
Direct: 5 routes, 4 active
Local: 71 routes, 71 active
BGP: 2 routes, 1 active
Static: 2 routes, 2 active
Aggregate: 7 routes, 2 active
INET6: 1 routes, 1 active
CDNVRF.inet6.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
Local: 2 routes, 2 active
apple.inet6.0: 17 destinations, 17 routes (16 active, 0 holddown, 1 hidden)
Local: 12 routes, 12 active
Aggregate: 3 routes, 2 active
bgp.l3vpn-inet6.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
Direct: 2 routes, 2 active
Aggregate: 2 routes, 2 active
---------------------------------------------------------------------------------------------
We removed the routing instance CDNVRF and all the groups associated with this instance. The config was committed around the time mentioned below.
labroot@router# show | compare
[edit routing-instances]
! inactive: CDNVRF { ... }
[edit protocols bgp family inet unicast]
! inactive: rib-group { ... }
[edit protocols bgp family inet6 unicast]
[edit protocols bgp group IBGP-NG-V4 family inet6 labeled-unicast]
[edit protocols bgp group GLOBAL-VRR-JUNIPER-V4-PEER family inet6 labeled-unicast]
[edit protocols bgp group GLOBAL-VRR-NOKIA-V4-PEER family inet6 labeled-unicast]
{master}[edit]
labroot@router# commit check
re0:
configuration check succeeds
re1:
labroot@router# commit
commit complete
**** AFTER DEACTIVATING CDNVRF *******
{master}
root@<...>> start shell
root@<...>:~ # vty fpc18
SMPC platform (1750Mhz Intel(R) Atom(TM) CPU processor, 3168MB memory, 8192KB flash)
SMPC18(<...> vty)# show route table
Protocol: IPv4
Table Name Table Index (lrid ) # of Routes Bytes LOCAL FRRP TID
-------------------------------------------------------------------------------------------------------
CDNVRF.7 7 (0 ) 5 656 LOCAL low ---- ===============================> still listed
__juniper_private1__.1 1 (0 ) 42 5540 LOCAL low ----
__juniper_private2__.2 2 (0 ) 14 1844 LOCAL low ----
__juniper_private4__.36736 36736 (0 ) 5 656 LOCAL low ----
__juniper_services__.3 3 (0 ) 9 1184 LOCAL low ----
__master.anon__.6 6 (0 ) 5 656 LOCAL low ----
__pfe_private__.4 4 (0 ) 5 656 LOCAL low ----
apple.8 8 (0 ) 15 1976 LOCAL low ----
default.0 0 (0 ) 1000360 132047516 LOCAL low ----
Protocol: MPLS
__mpls-oam__.9 9 (0 ) 1 128 LOCAL low ----
apple.8 8 (0 ) 1 128 LOCAL low ----
default.0 0 (0 ) 11 1448 LOCAL low ----
Protocol: IPv6
CDNVRF.7 7 (0 ) 6 824 LOCAL low ----
__juniper_private1__.1 1 (0 ) 6 824 LOCAL low ----
__master.anon__.6 6 (0 ) 6 824 LOCAL low ----
apple.8 8 (0 ) 28 3968 LOCAL low ----
default.0 0 (0 ) 106 15184 LOCAL low ----
Protocol: ARP
ARP.0 0 (0 ) 0 0 LOCAL low ----
Protocol: CLNP
__master.anon__.6 6 (0 ) 1 128 LOCAL low ----
default.0 0 (0 ) 1 128 LOCAL low ----
Protocol: CCC
CCC.0 0 (0 ) 0 0 LOCAL low ----
Protocol: MLPPP
MLPPP.0 0 (0 ) 0 0 LOCAL low ----
Protocol: TCC
TCC.0 0 (0 ) 0 0 LOCAL low ----
Protocol: BRIDGE
__juniper_private1__.1 1 (0 ) 29 3944 LOCAL low ----
default-switch.5 5 (0 ) 0 0 LOCAL low ----
Protocol: Multiservice
Multiservice.0 0 (0 ) 0 0 LOCAL low ----
Protocol: DHCP-Snooping
SMPC18(<...> vty)# show route ip table index 7
IPv4 Route Table 7, CDNVRF.7, 0x41000:
Destination NH IP Addr Type NH ID Interface
--------------------------------- --------------- -------- ----- ---------
default Reject 550 RT-ifl 0 .local..7 ifl 327
0.0.0.0 Discard 548 RT-ifl 0 .local..7 ifl 327
224/4 mdiscard 549 RT-ifl 0 .local..7 ifl 327
224.0.0.1 Mcast 552 RT-ifl 0 .local..7 ifl 327
255.255.255.255 Bcast 553 RT-ifl 0 .local..7 ifl 327
Logical interface .local..7 (Index 327, Alias-Index 0 Peer-Index 0 ifl address 0x3bef7630)
Channel Mode DISABLED (channel1 0 channel2 0)
Flags: (0x0000000000000052) Shutdown Point-To-Point No-Multicast
GEN Flags: (0x0000)
Addresses:
Media address: Family: Unspecified (0), Chan: 0, Length: 0
IRB ifl BD index 65535
Reroute Ref: 0, Restore Ref: 0, LRID: 0
Residue Stats in: 0 out: 0
Protocols:
Protocol: IPv4, MTU: 65535 bytes, TCP MSS 0 bytes, Flags: 0x0000000000000000, Route table: 7
Maximum labels: 0
Input filter: 0, Output filter: 0, Interface class: 0, Dialer Filter: 0
Input Simple Filter: 0, Output Simple Filter: 0
Input implicit filters: None
Output implicit filters: None
L2 Input policer: 0, L2 Output policer: 0
Input policer: 0, Output policer: 0
RPF fail-filter: 0, Reroute Ref: 0, Restore Ref: 0
Service filters in: 0, 0, 0, 0, 0, 0, out: 0, 0, 0, 0, 0, 0, psf: 0
Protocol: IPv6, MTU: 65535 bytes, TCP MSS 0 bytes, Flags: 0x0000000000000000, Route table: 7
Media:
Type: Unspecified, Encapsulation: Unspecified (0x00000000)
MTU: 0 bytes, Flags: 0x0000
Dependencies:
Parent ifl index: 327
Storm control:
BC: 0, UC: 0, Flags: 0x0
Creation time: Jan 26 21:55:27 2024
root@<...>> show route instance
Instance Type
Primary RIB Active/holddown/hidden
master forwarding
inet.0 1000351/0/3
mpls.0 11/0/0
bgp.l3vpn.0 2/0/0
inet6.0 83/0/5
bgp.l3vpn-inet6.0 3/0/0
__juniper_private1__ forwarding
__juniper_private1__.inet.0 5/0/0
__juniper_private1__.inet6.0 4/0/0
__juniper_private2__ forwarding
__juniper_private2__.inet.0 0/0/1
__juniper_private4__ forwarding
__master.anon__ forwarding
apple vrf
apple.inet.0 9/0/1
apple.inet6.0 16/0/1
mgmt_junos forwarding
root@<...>> show krt error-statistics
Error code Error Type Count Timestamp
2 ENOENT 1 Fri Jan <...>
More information can be found in PR1788272 (internal while compiling this KB).