Description

This article explains the following:

  • Default Hash computation algorithm on PTX1K for MPLS lookup and IP lookup.
  • Why traffic gets evenly distributed on PTX1K when it does an MPLS lookup (with default forwarding-options config)
  • Why the same even distribution does not work in case of an IP lookup (in case of MPLSoUDP).
  • Mitigation to achieve even load distribution on PTX1K in case of IP lookup (MPLSoUDP as transport)

Symptoms

When no explicit hash-key is configured under forwarding-options, on PTX1K egress LAG member links, the traffic will see even distribution when the PTX1K has to do a MPLS swap operation on the traffic. However, with the same default configuration, the traffic on MPLSoUDP will not be evenly distributed on the Egress LAG member links of PTX1K.

Solution

  • By Default, PTX1K includes all MPLS labels in the stack for hash computation for MPLS traffic. So, in case there are multiple VRFs between a pair of PE routers, the different VPN label for each VRF will ensure entropy for the flows

  • By Default, PTX1K includes only Source IP and Destination IP in the hash computation for any IP traffic (even if there is one or more MPLS headers are present underneath IP header i.e. MPLSoUDP case). So, even if the VPN label is different for multiple VRFs between a pair of PE routers, the Top IP header will be same (for the same pair of PE routers) and will not help in achieving entropy. Thus for such IP flow, the traffic on egress of PTX1K will not be evenly distributed 

The following default hash-key is supported for forwarding options, in case of PTX1K:

labroot@P1> show configuration groups junos-defaults forwarding-options hash-key |display set 
set groups junos-defaults forwarding-options hash-key family mpls all-labels
set groups junos-defaults forwarding-options hash-key family mpls payload ip enable


'junos-defaults' is the default group which involves all the configuration supported by default on a particular platform.

Topology in case of MPLS as the underlying transport:

alt

 

traffic streams:

Flow-1 : SIP = 100.100.100.0/30, DIP = 200.200.200.0/30, Rate = 10,000 PPS
Flow-2 : SIP = 100.100.101.0/30, DIP = 200.200.201.0/30, Rate = 10,000 PPS
Flow-3 : SIP = 100.100.102.0/30, DIP = 200.200.202.0/30, Rate = 10,000 PPS

As seen below, the PE2 loopback IP - 4.4.4.4/32 is reachable on PE1 via BGP-LU router:

labroot@PE1> show route 4.4.4.4 
 
inet.0: 18 destinations, 22 routes (18 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
 
4.4.4.4/32         *[BGP/170] 00:06:55, localpref 100
                      AS path: 65002 65010 I, validation-state: unverified
                    >  to 10.10.10.2 via ae1.10, Push 27
 
inet.3: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
 
4.4.4.4/32         *[BGP/170] 00:06:55, localpref 100
                      AS path: 65002 65010 I, validation-state: unverified
                    >  to 10.10.10.2 via ae1.10, Push 27
 
The PE2 loopback address on P1 is also reachable via BGP-LU
 
labroot@P1> show route 4.4.4.4 
 
inet.3: 17 destinations, 17 routes (17 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
 
4.4.4.4/32         *[BGP/170] 00:07:28, localpref 100
                      AS path: 65010 I, validation-state: unverified
                    > to 10.10.10.5 via ae27.0, Push 28
 

Reachability to the Remote VPN routes:

labroot@PE1> show route 200.200.200.0 table CUST1    
 
CUST1.inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
 
200.200.200.0/30   *[BGP/170] 00:56:58, localpref 100, from 4.4.4.4
                      AS path: I, validation-state: unverified
                    >  to 10.10.10.2 via ae1.10, Push 16, Push 27(top)
 
labroot@PE1> show route 200.200.201.0 table CUST2    
 
CUST2.inet.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
 
200.200.201.0/30   *[BGP/170] 00:57:21, localpref 100, from 4.4.4.4
                      AS path: I, validation-state: unverified
                    >  to 10.10.10.2 via ae1.10, Push 20, Push 27(top)
 
labroot@PE1> show route 200.200.202.0 table CUST3   
 
CUST3.inet.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
 
200.200.202.0/30   *[BGP/170] 00:57:28, localpref 100, from 4.4.4.4
                      AS path: I, validation-state: unverified
                    >  to 10.10.10.2 via ae1.10, Push 21, Push 27(top)
 
With no explicit hash-key configuration under forwarding-options for MPLS family, the load would still be close to evenly distributed on the egress LAG member links of P1. Since by default, in case of MPLS lookup, all labels in the label stack are used as input to hash computation logic. Thus, different VPN labels for multiple VRFs in the label stack, will give entropy to achieve load distribution on all egress links
 
labroot@P1> monitor interface traffic
Interface    Link  Input packets        (pps)     Output packets        (pps)
 et-0/0/1:0    Up         953653          (1)        153856322        (10471)
 et-0/0/1:1    Up            123          (0)        162848065        (8147)
 et-0/0/1:2    Up         117132          (0)        170852426        (7809)
 et-0/0/1:3    Up           7914          (0)        153242840        (3333)


Topology in case of MPLSoUDP as the underlying transport:

Traffic streams:

Flow-1 : SIP = 100.100.100.0/30, DIP = 200.200.200.0/30, Rate = 10,000 PPS
Flow-2 : SIP = 100.100.101.0/30, DIP = 200.200.201.0/30, Rate = 10,000 PPS
Flow-3 : SIP = 100.100.102.0/30, DIP = 200.200.202.0/30, Rate = 10,000 PPS

As seen below, the PE2 loopback IP - 4.4.4.4/32 is reachable on PE1 via a Tunnel Composite NH, MPLSoUDP here:

labroot@PE1> show route 4.4.4.0/24  
 
inet.0: 17 destinations, 20 routes (17 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
 
4.4.4.0/24         *[BGP/170] 1w2d 04:05:00, localpref 100
                      AS path: 65002 65010 I, validation-state: unverified
                    >  to 10.10.10.18 via ae1.11
 
inet.3: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
 
4.4.4.0/24         *[Tunnel/305] 1w2d 04:05:00
                       Tunnel
4.4.4.4/32         *[Tunnel/300] 1w2d 04:04:57
                       Tunnel Composite
 
Similarly, remote PE address 4.4.4.4/32 is reachable on PE via IPv4 route.
 
labroot@P1> show route 4.4.4.4 
 
inet.0: 15 destinations, 15 routes (15 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
 
4.4.4.0/24         *[BGP/170] 1w2d 04:19:26, localpref 100
                      AS path: 65010 I, validation-state: unverified
                    > to 10.10.10.5 via ae27.0
 
  
Reachability to the Remote VPN routes:
 
labroot@PE1> show route 200.200.200.0 table CUST1 
 
CUST1.inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
 
200.200.200.0/30   *[BGP/170] 05:43:59, localpref 100, from 4.4.4.4
                      AS path: I, validation-state: unverified
                    >  via Tunnel Composite, Push 16
 
 
labroot@PE1> show route 200.200.201.0 table CUST2    
 
CUST2.inet.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
 
200.200.201.0/30   *[BGP/170] 05:29:22, localpref 100, from 4.4.4.4
                      AS path: I, validation-state: unverified
                    >  via Tunnel Composite, Push 20
 
 
labroot@PE1> show route 200.200.202.0 table CUST3    
 
CUST3.inet.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
 
200.200.202.0/30   *[BGP/170] 05:26:56, localpref 100, from 4.4.4.4
                      AS path: I, validation-state: unverified
                    >  via Tunnel Composite, Push 21
 
As seen in below output, the traffic on PTX1K on the egress side (AE27) will have uneven distribution among the LAG member links. Only 1 link is carrying all the traffic 
 
labroot@P1> monitor interface traffic
Interface    Link  Input packets        (pps)     Output packets        (pps)
 et-0/0/1:0    Up         950159          (0)        140514649          (0)
 et-0/0/1:1    Up             33          (0)        149515968          (0)
  et-0/0/1:2    Up         116739          (0)        147153676      (30033)
 et-0/0/1:3    Up           7904          (0)        138602639          (0)
 
The solution to this will be to configure 'hash-key family inet layer-4' and 'hash-key family inet layer-3' under forwarding-options on the P1 router. This will include Layer-4 Source Port and Destination Port as input to hash computation.
 
labroot@P1# show |compare 
[edit]
+  forwarding-options {
+      hash-key {
+          family inet {
+              layer-3;
+              layer-4;
+          }
+      }
+  }
 
After adding the hash-key configuration shown above, you will see more even distribution on AE27 LAG member links on the egress side of P1 router.
 
labroot@P1> monitor interface traffic
Interface    Link  Input packets        (pps)     Output packets        (pps)
 et-0/0/1:0    Up         950403          (0)        140562163       (7497)
 et-0/0/1:1    Up             33          (0)        149563577        (7558)
 et-0/0/1:2    Up         116768          (0)        152854607        (7496)
 et-0/0/1:3    Up           7904          (0)        138649728        (7439)