Description

We see an issue with ACX7100 that its not load balancing traffic over AE (LAG) member interfaces. This issue is causing one of the member interfaces of AE to be overutilized while leaving other interfaces of same bundle underutilized.

 

Example: AE98 has 3xMember interfaces but only of the link egressing traffic

 

     Interface   Link Input packets       (pps)    Output packets       (pps)

 

      ae98         Up   11026676739     (82558)     11015676744     (82560)

      et-0/0/12    Up    2261556661     (16512)      3786390177         (0)

      et-0/0/13    Up     111524961         (0)      3900062964     (82560)

      et-0/0/14    Up    8653594767     (66046)      3329213610         (0)

Symptoms

In my test cases, I used ACX7100 as transit device to bridge traffic between upstream and downstream nodes.

 

Test Case 1: Load Balancing - Single S.MAC and D.MAC

IXIA_1  --- MX_DUT1 ----3x---- [AE203] ACX7100 [AE98] ---3x----MX_DUT2 --- IXIA_2  

 

  • Multiple bidirectional IP flows between IXIA_1 and IXIA_2.
  • MX_DUT1 and MX_DUT2 is connected to ACX7100 via AE bundles and the bundles has multiple sub-interfaces to route the traffic flows.
  • ACX7100 is used to bridge traffic between MX_DUT1 and MX_DUT2
  • Below Output displays that ACX7100 learns same S.MAC/D.MAC for each VLAN

 

Ethernet switching table : 72 entries, 72 learned

Routing instance : default-switch

   Vlan               MAC                MAC        Age  GBP    Logical               NH       RTR

   name               address            flags            Tag    interface             Index    ID

   Juniper_Lab_VLAN_1936 58:00:bb:0b:d7:cc D            -           ae203.500             0        0      

   Juniper_Lab_VLAN_1936 58:00:bb:0b:d7:cd D            -           ae98.1936             0        0      

   Juniper_Lab_VLAN_1902 58:00:bb:0b:d7:cc D            -           ae203.500             0        0      

   Juniper_Lab_VLAN_1902 58:00:bb:0b:d7:cd D            -           ae98.1902             0        0      

   Juniper_Lab_VLAN_1903 58:00:bb:0b:d7:cc D           -           ae203.500             0        0      

   Juniper_Lab_VLAN_1903 58:00:bb:0b:d7:cd D           -           ae98.1903             0        0      

   Juniper_Lab_VLAN_1924 58:00:bb:0b:d7:cc D            -           ae203.500             0        0      

   Juniper_Lab_VLAN_1924 58:00:bb:0b:d7:cd D            -           ae98.1924             0        0      

   Juniper_Lab_VLAN_1904 58:00:bb:0b:d7:cc D            -           ae203.500             0        0      

   Juniper_Lab_VLAN_1904 58:00:bb:0b:d7:cd D            -           ae98.1904             0        0      

     ...  

 

  • Below output displays that the egress traffic is not efficiently load balanced on ACX7100 AE member interfaces

 

     Interface   Link Input packets       (pps)    Output packets       (pps)

 

      ae98         Up   11026676739     (82558)     11015676744     (82560)

      et-0/0/12    Up    2261556661     (16512)      3786390177         (0)

      et-0/0/13    Up     111524961         (0)      3900062964     (82560)

      et-0/0/14    Up    8653594767     (66046)      3329213610         (0)

 

      ae203        Up   11015634975     (82559)     11026704832     (82560)

      et-0/0/15    Up    2257140516     (16512)      1991698540         (0)

      et-0/0/16    Up    4406022188     (33024)      3236813467         (0)

      et-0/0/17    Up    4352472271     (33023)      5798192825     (82560)

 

 

 

Test Case 2: Load Balancing - Multiple S.MACs and D.MACs

IXIA_1  ---3x--- [AE203] ACX7100 [AE98] ---3x--- IXIA_2  

 

  • Multiple bidirectional IP flows between IXIA_1 and IXIA_2. Each flow uses unique S.MAC
  • ACX7100 is used to bridge traffic between IXIA_1 and IXIA_2
  • Below Output displays that ACX7100 learns unique S.MACs for each flow

 

  Routing instance : default-switch

        Vlan               MAC                MAC        Age  GBP    Logical               NH       RTR

        name               address            flags            Tag    interface             Index    ID

        V100               00:00:44:16:de:3f  D            -           ae203.500             0        0      

        V100               00:00:44:16:de:41  D            -           ae203.500             0        0      

        V100               00:00:44:16:de:43  D            -           ae203.500             0        0      

        V100               00:00:44:16:de:45  D            -           ae203.500             0        0      

        V100               00:00:44:16:de:47  D            -           ae203.500             0        0      

        V100               00:00:44:17:48:a1  D            -           ae98.100              0        0      

        V100               00:00:44:17:48:a3  D            -           ae98.100              0        0      

        V100               00:00:44:17:48:a5  D            -           ae98.100              0        0      

        V100               00:00:44:17:48:a7  D            -           ae98.100              0        0      

        V100               00:00:44:17:48:a9  D            -           ae98.100              0        0     

 

  • Below output displays that the egress traffic is not efficiently load balanced on ACX7100 AE member interfaces

 

     Interface   Link Input packets       (pps)    Output packets       (pps)

 

      ae98         Up   11053952175    (165016)     11042952227    (165017)

      et-0/0/12    Up    2268357607     (66006)      3788516160         (0)

      et-0/0/13    Up     114215456     (66007)      3923086509    (165017)

      et-0/0/14    Up    8671378762     (33003)      3331339565         (0)

 

      ae203        Up   11042910445    (165019)     11053980343    (165022)

      et-0/0/15    Up    2263941480     (66008)      1992761591         (0)

      et-0/0/16    Up    4416932066     (66007)      3240002365         (0)

      et-0/0/17    Up    4362036899     (33004)      5821216387    (165022)

Solution

There is no default load balancing hashing enabled on ACX7100 platforms, hence it’s expected that the egress traffic will take one of the member links of AE.

 

labroot@jtac-acx7100-48l-r2017> show configuration forwarding-options         

 

labroot@jtac-acx7100-48l-r2017>

 

labroot@jtac-acx7100-48l-r2017:pfe> show evo-pfemand hashkey

Multiservice:

-------------

_hashL2Fields:0x0

   srcMac:disabled

   dstMac:disabled

 

Inet:

------

_hashInetFields:0x0

   isLayer3:disabled

   isLayer4:disabled

 

Inet6:

------

_hashInet6Fields:0x0

   isLayer3:disabled

   isLayer4:disabled

 

Mpls:

-----

_hashMplsFields:0x0

   isLabel:disabled

   isIpPayload:disabled

 

 

 

Test Case 1: Load Balancing - Single S.MAC and D.MAC

 

labroot@jtac-acx7100-48l-r2017> show configuration forwarding-options   

hash-key {

   family inet {

       layer-3;

   }

}

 

labroot@jtac-acx7100-48l-r2017:pfe> show evo-pfemand hashkey

…

Inet:

------

_hashInetFields:0x10000f10

   isLayer3:enabled

   isLayer4:disabled

…

 

     Interface   Link Input packets       (pps)    Output packets       (pps)

 

      ae98         Up   11041319896     (82571)     11030319920     (82569)

      et-0/0/12    Up    2264485662     (16516)      3786488425     (33028)

      et-0/0/13    Up     111524967         (0)      3914509641     (16515)

      et-0/0/14    Up    8665308917     (66055)      3329311861     (33026)

 

      ae203        Up   11030278133     (82569)     11041347999     (82570)

      et-0/0/15    Up    2260069520     (16517)      1991747676     (16514)

      et-0/0/16    Up    4411879264     (33026)      3236960848     (49541)

      et-0/0/17    Up    4358329349     (33026)      5812639475     (16515)

 

 

Test Case 2:  Load Balancing - Multiple S.MACs and D.MACs

 

With multiple unique S.MACs, Load balance for L2 traffic will not happen if we are not adding any hash keys under forwarding-options as there is no default hashing on ACX7100 platforms

 

With multiple unique S.MACs, Load balance for L2 traffic will not happen if we are enabling only one of the following “set forwarding-options hash-key family multi-services S.MAC” or “set forwarding-options hash-key family multi-services D.MAC”

 

labroot@jtac-acx7100-48l-r2017> show configuration forwarding-options   

hash-key {

   family multiservice {

       source-mac;

       destination-mac;

   }

}

 

labroot@jtac-acx7100-48l-r2017:pfe> show evo-pfemand hashkey

Multiservice:

-------------

_hashL2Fields:0x10fc0000

   srcMac:enabled

   dstMac:enabled

…

 

 

     Interface   Link Input packets       (pps)    Output packets       (pps)

 

      ae98         Up   11074260574    (165143)     11063260647    (165141)

      et-0/0/12    Up    2276481151     (66057)      3789123409     (33029)

      et-0/0/13    Up     122338691     (66058)      3940966016     (33028)

      et-0/0/14    Up    8675440382     (33028)      3333161229     (99084)

 

      ae203        Up   11063218844    (165144)     11074288744    (165145)

      et-0/0/15    Up    2272065026     (66059)      1993368851     (33030)

      et-0/0/16    Up    4425055301     (66057)      3240609598     (33028)

      et-0/0/17    Up    4366098517     (33028)      5840310295     (99087)

 

Modification History

2024-01-25 : Article Created