Customers may want to add a new member link to an existing Aggregated Ethernet (AE) interface, and then want to rebalance subscribers across the new member link. However, this operation may not work sometimes.
This article gives the reason, while also detailing the configuration that ensures that subscribers are rebalanced across a new member link automatically.
When you configure targeted-distribution in a dynamic-profile and apply the configuration on a subscriber interface, subscribers will be distributed to each link automatically.
dynamic-profiles { pppoe { interfaces { pp0 { unit "$junos-interface-unit" { no-traps; ppp-options { chap; pap; } pppoe-options { underlying-interface "$junos-underlying-interface"; server; } targeted-distribution; <<<<<< targeted-distribution configuration in dynamic-profile family inet { unnumbered-address lo0.0; } } } } } }
Now when you bring up a single link AE interface and connect 100 subscribers, you will find that all the subscribers are distributed on that active link.
user@mx> show lacp interfaces ae0 Aggregated interface: ae0 LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity xe-0/2/0 Actor No No Yes Yes Yes Yes Fast Active xe-0/2/0 Partner No No Yes Yes Yes Yes Fast Active xe-0/2/1 Actor No Yes No No No Yes Fast Active xe-0/2/1 Partner No Yes No No No Yes Fast Passive LACP protocol: Receive State Transmit State Mux State xe-0/2/0 Current Fast periodic Collecting distributing xe-0/2/1 Port disabled No periodic Detached user@mx> show interfaces targeting ae0 Aggregated interface: ae0 Targeting Type: Auto Redundancy mode: Link Level Redundancy Total number of distribution groups: 1 Total number of distributed interfaces: 100 Distribution Group name: default Number of distributed interfaces: 100 Physical interface: xe-0/2/0, Link status: Up Number of primary distributions: 100 <<<<<< All subscribers distributed on this link Number of backup distributions: 0 Physical interface: xe-0/2/1, Link status: Down Number of primary distributions: 0 Number of backup distributions: 100
When you bring up a new member link on the AE interface now, and try to rebalance subscriber distribution, the operation may not work as is shown below:
user@mx> show lacp interfaces ae0 Aggregated interface: ae0 LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity xe-0/2/0 Actor No No Yes Yes Yes Yes Fast Active xe-0/2/0 Partner No No Yes Yes Yes Yes Fast Active xe-0/2/1 Actor No No Yes Yes Yes Yes Fast Active xe-0/2/1 Partner No No Yes Yes Yes Yes Fast Active LACP protocol: Receive State Transmit State Mux State xe-0/2/0 Current Fast periodic Collecting distributing xe-0/2/1 Current Fast periodic Collecting distributing user@mx> request interface rebalance ae0 <<<<<< Rebalance operation Rebalance operation on interface ae0 started user@mx> show interfaces targeting ae0 Aggregated interface: ae0 Targeting Type: Auto Redundancy mode: Link Level Redundancy Total number of distribution groups: 1 Total number of distributed interfaces: 100 Distribution Group name: default Number of distributed interfaces: 100 Physical interface: xe-0/2/0, Link status: Up Number of primary distributions: 100 <<<<<< All subscribers are still distributed on the original link. Number of backup distributions: 0 Physical interface: xe-0/2/1, Link status: Up Number of primary distributions: 0 Number of backup distributions: 100
As per Junos OS design, if the subscriber count difference between the members of the AE interface is below 500, rebalance is not done.
A configurable knob, rebalance-subscriber-granularity , controls the aggregate client weight for each targeted pseudo IFL. This knob's default value is 500, which is the reason that rebalance works at a higher number (> 500) of subscribers.
rebalance-subscriber-granularity
user@mx> show interfaces targeting ae0 Aggregated interface: ae0 Targeting Type: Auto Redundancy mode: Link Level Redundancy Total number of distribution groups: 1 Total number of distributed interfaces: 1000 Distribution Group name: default Number of distributed interfaces: 1000 Physical interface: xe-0/2/0, Link status: Up Number of primary distributions: 1000 Number of backup distributions: 0 Physical interface: xe-0/2/1, Link status: Down Number of primary distributions: 0 Number of backup distributions: 1000 user@mx> request interface rebalance ae0 Rebalance operation on interface ae0 started {MASTER} user@mx> show interfaces targeting ae0 Aggregated interface: ae0 Targeting Type: Auto Redundancy mode: Link Level Redundancy Total number of distribution groups: 1 Total number of distributed interfaces: 1000 Distribution Group name: default Number of distributed interfaces: 1000 Physical interface: xe-0/2/0, Link status: Up Number of primary distributions: 500 Number of backup distributions: 500 Physical interface: xe-0/2/1, Link status: Up Number of primary distributions: 500 <<<<<<500 subscribers rebalanced to new member link Number of backup distributions: 500
When you set rebalance-subscriber-granularity to 1, the rebalance operation always take place.
user@mx# set interfaces ae0 aggregated-ether-options targeted-options rebalance-subscriber-granularity 1 {MASTER}[edit] user@mx# show | compare [edit interfaces ae0 aggregated-ether-options] + targeted-options { + rebalance-subscriber-granularity 1; + } user@mx> show interfaces targeting ae0 Aggregated interface: ae0 Targeting Type: Auto Redundancy mode: Link Level Redundancy Total number of distribution groups: 1 Total number of distributed interfaces: 100 Distribution Group name: default Number of distributed interfaces: 100 Physical interface: xe-0/2/0, Link status: Up Number of primary distributions: 100 Number of backup distributions: 0 Physical interface: xe-0/2/1, Link status: Down Number of primary distributions: 0 Number of backup distributions: 100 user@mx> request interface rebalance ae0 Rebalance operation on interface ae0 started user@mx> show interfaces targeting ae0 Aggregated interface: ae0 Targeting Type: Auto Redundancy mode: Link Level Redundancy Total number of distribution groups: 1 Total number of distributed interfaces: 100 Distribution Group name: default Number of distributed interfaces: 100 Physical interface: xe-0/2/0, Link status: Up Number of primary distributions: 50 Number of backup distributions: 50 Physical interface: xe-0/2/1, Link status: Up Number of primary distributions: 50 Number of backup distributions: 50
Note: Setting this knob to a low value, as shown above, can result in undesirable effects such as the system running out of pseudo IFLs, the convergence time increasing for rebalance, and so on. It is therefore NOT recommended that the value should be set to less than a few hundreds.
However, customers need not worry about subscriber distribution difference being below 500 because new subscribers will always be distributed to a new member link that has low weight.
user@mx> show interfaces targeting ae0 Aggregated interface: ae0 Targeting Type: Auto Redundancy mode: Link Level Redundancy Total number of distribution groups: 1 Total number of distributed interfaces: 100 Distribution Group name: default Number of distributed interfaces: 100 Physical interface: xe-0/2/0, Link status: Up Number of primary distributions: 100 Number of backup distributions: 0 Physical interface: xe-0/2/1, Link status: Down Number of primary distributions: 0 Number of backup distributions: 100
For example, when you bring up a new member link and connect 100 new subscribers, all new subscribers will distribute to the new member link.
user@mx> show interfaces targeting ae0 Aggregated interface: ae0 Targeting Type: Auto Redundancy mode: Link Level Redundancy Total number of distribution groups: 1 Total number of distributed interfaces: 200 Distribution Group name: default Number of distributed interfaces: 200 Physical interface: xe-0/2/0, Link status: Up Number of primary distributions: 100 Number of backup distributions: 100 Physical interface: xe-0/2/1, Link status: Up Number of primary distributions: 100 <<<<<< 100 new subscribers distributed to new member link Number of backup distributions: 100