This article provides information to avoid double bandwidth issues for subscribers terminating on MX over an AE interface. Strategies are provide for using the Shared-Bandwidth-Policer knob and Targeted-Distribution.
A standard policer restricts traffic at the configured rate per PFE. In the case of AE bundles spanning multiple FPCs, the overall rate for the subscriber could exceed the configured rate (depending on the number of PFEs involved). An example of this would be a policer with bandwidth-limit 40mbps and burst-size 40Kbytes configured on an AE interface that has member links ge-0/0/0 and ge-1/0/0. When the policer is applied to the AE interface, this will result in a total bandwidth of 80Mbps as policer is configured for two PFE's. In the example below, AE has 4 member links configured over FPC11. FPC11 is an MPC5 which has 1 PFE. The subscriber is configured with 100M policer.
FPC 11 REV 66 750-045715 CAFE6382 MPC5EQ-40G10G jtac@ERX-MX960-II-RE0> show lacp interfaces Aggregated interface: ae0 LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity xe-11/0/0 Actor No No Yes Yes Yes Yes Fast Active xe-11/0/0 Partner No No Yes Yes Yes Yes Fast Active xe-11/0/1 Actor No No Yes Yes Yes Yes Fast Active xe-11/0/1 Partner No No Yes Yes Yes Yes Fast Active xe-11/1/0 Actor No No Yes Yes Yes Yes Fast Active xe-11/1/0 Partner No No Yes Yes Yes Yes Fast Active xe-11/1/1 Actor No No Yes Yes Yes Yes Fast Active xe-11/1/1 Partner No No Yes Yes Yes Yes Fast Active LACP protocol: Receive State Transmit State Mux State xe-11/0/0 Current Fast periodic Collecting distributing xe-11/0/1 Current Fast periodic Collecting distributing xe-11/1/0 Current Fast periodic Collecting distributing xe-11/1/1 Current Fast periodic Collecting distributing jtac@ERX-MX960-II-RE0# show firewall family inet filter 100M interface-specific; term 1 { then { policer 100M; accept; } } jtac@ERX-MX960-II-RE0# show firewall policer 100M filter-specific; if-exceeding { bandwidth-limit 100m; burst-size-limit 8m; } then discard; jtac@ERX-MX960-II-RE0# run show subscribers extensive Type: VLAN Logical System: default Routing Instance: default Interface: demux0.3221225476 Interface type: Dynamic Underlying Interface: ae0 Dynamic Profile Name: DYN-VLAN-Q-IN-Q-AE-PROFILE Dynamic Profile Version: 1 State: Active Session ID: 5 PFE Flow ID: 21 Stacked VLAN Id: 0x8100.2 VLAN Id: 0x8100.200 Login Time: 2017-04-03 21:23:57 IST Type: PPPoE User Name: [email protected] IP Address: 10.200.0.4 IP Netmask: 255.255.255.255 Domain name server inet: 8.8.4.4 Domain name server inet6: 2001:4860:4860::8844 Logical System: default Routing Instance: default Interface: pp0.3221225477 Interface type: Dynamic Underlying Interface: demux0.3221225476 Dynamic Profile Name: PPPOE-PROFILE Dynamic Profile Version: 1 MAC Address: 00:16:01:00:00:01 State: Active Radius Accounting ID: 6 Session ID: 6 PFE Flow ID: 22 Stacked VLAN Id: 2 VLAN Id: 200 Login Time: 2017-04-03 21:23:57 IST IP Address Pool: ipv4-pool IPv4 Input Filter Name: 100M-pp0.3221225477-in IPv4 Output Filter Name: 100M-pp0.3221225477-out Accounting interval: 1800 Dynamic configuration: junos-input-filter: 100M junos-output-filter: 100M jtac@ERX-MX960-II-RE0# run show interfaces pp0.3221225477 extensive | match bps Input bytes : 32693175289 0 bps Output bytes : 6120028012 99865464 bps <-- 100Mbps
Aggregated interface: ae0 LACP state:
As seen above, the subscriber is getting the expected 100M bandwdith as configured. The egress policer is configured on a single PFE (MPC5).
FPC 8 REV 21 750-054563 CADH0955 MPC5E-40G10G FPC 11 REV 66 750-045715 CAFE6382 MPC5EQ-40G10G jtac@ERX-MX960-II-RE0> show lacp interfaces Aggregated interface: ae0 LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity xe-11/0/0 Actor No No Yes Yes Yes Yes Fast Active xe-11/0/0 Partner No No Yes Yes Yes Yes Fast Active xe-11/0/1 Actor No No Yes Yes Yes Yes Fast Active xe-11/0/1 Partner No No Yes Yes Yes Yes Fast Active xe-11/1/0 Actor No No Yes Yes Yes Yes Fast Active xe-11/1/0 Partner No No Yes Yes Yes Yes Fast Active xe-11/1/1 Actor No No Yes Yes Yes Yes Fast Active xe-11/1/1 Partner No No Yes Yes Yes Yes Fast Active xe-8/0/6 Actor No No Yes Yes Yes Yes Fast Active xe-8/0/6 Partner No No Yes Yes Yes Yes Fast Active LACP protocol: Receive State Transmit State Mux State xe-11/0/0 Current Fast periodic Collecting distributing xe-11/0/1 Current Fast periodic Collecting distributing xe-11/1/0 Current Fast periodic Collecting distributing xe-11/1/1 Current Fast periodic Collecting distributing xe-8/0/6 Current Fast periodic Collecting distributing jtac@ERX-MX960-II-RE0# run show subscribers extensive Type: VLAN Logical System: default Routing Instance: default Interface: demux0.3221225478 Interface type: Dynamic Underlying Interface: ae0 Dynamic Profile Name: DYN-VLAN-Q-IN-Q-AE-PROFILE Dynamic Profile Version: 1 State: Active Session ID: 7 PFE Flow ID: 24 Stacked VLAN Id: 0x8100.2 VLAN Id: 0x8100.200 Login Time: 2017-04-04 08:10:46 IST Type: PPPoE User Name: [email protected] IP Address: 10.200.0.5 IP Netmask: 255.255.255.255 Domain name server inet: 8.8.4.4 Domain name server inet6: 2001:4860:4860::8844 Logical System: default Routing Instance: default Interface: pp0.3221225479 Interface type: Dynamic Underlying Interface: demux0.3221225478 Dynamic Profile Name: PPPOE-PROFILE Dynamic Profile Version: 1 MAC Address: 00:16:01:00:00:01 State: Active Radius Accounting ID: 8 Session ID: 8 PFE Flow ID: 26 <-- VBF FLOW ID Stacked VLAN Id: 2 VLAN Id: 200 Login Time: 2017-04-04 08:10:46 IST IP Address Pool: ipv4-pool IPv4 Input Filter Name: 100M-pp0.3221225479-in IPv4 Output Filter Name: 100M-pp0.3221225479-out Accounting interval: 1800 Dynamic configuration: junos-input-filter: 100M junos-output-filter: 100M jtac@ERX-MX960-II-RE0# run show interfaces pp0.3221225479 extensive | match bps Input bytes : 684 0 bps Output bytes : 11820983986 200031952 bps <-- 200Mbps
The purpose of the “Shared-bandwidth-policer” feature is to match the effective bandwidth/burst-size made available to an aggregate bundle by enhancing the implementation of the policer on the PFE. The implementation will statically carve up the bandwidth and burst size among the PFE-complex by a factor derived from number of member-links that belong to a given PFE-complex. The carve-up factor shown below. Carve-up-factor= (Number of links on that PFE-complex)/(Total Number of links in the bundle)
Bandwidth-limit=(Total Bandwidth-limit)*(Carve-up-factor) Burst-Size-limit=(Total burst-size-limit)* (Carve-up-factor)
policer 100M { filter-specific; shared-bandwidth-policer; <-- Configured shared-bandwidth-policer under policer if-exceeding { bandwidth-limit 100m; burst-size-limit 8m; } then discard; jtac@ERX-MX960-II-RE0# run show subscribers extensive Type: VLAN Logical System: default Routing Instance: default Interface: demux0.3221225480 Interface type: Dynamic Underlying Interface: ae0 Dynamic Profile Name: DYN-VLAN-Q-IN-Q-AE-PROFILE Dynamic Profile Version: 1 State: Active Session ID: 9 PFE Flow ID: 27 Stacked VLAN Id: 0x8100.2 VLAN Id: 0x8100.200 Login Time: 2017-04-04 12:24:43 IST Type: PPPoE User Name: [email protected] IP Address: 10.200.0.6 IP Netmask: 255.255.255.255 Domain name server inet: 8.8.4.4 Domain name server inet6: 2001:4860:4860::8844 Logical System: default Routing Instance: default Interface: pp0.3221225481 Interface type: Dynamic Underlying Interface: demux0.3221225480 Dynamic Profile Name: PPPOE-PROFILE Dynamic Profile Version: 1 MAC Address: 00:16:01:00:00:01 State: Active Radius Accounting ID: 10 Session ID: 10 PFE Flow ID: 29 Stacked VLAN Id: 2 VLAN Id: 200 Login Time: 2017-04-04 12:24:43 IST IP Address Pool: ipv4-pool IPv4 Input Filter Name: 100M-pp0.3221225481-in IPv4 Output Filter Name: 100M-pp0.3221225481-out Accounting interval: 1800 Dynamic configuration: junos-input-filter: 100M junos-output-filter: 100M
Regular policer '100M' dfw:167772160 pfe_id:0 carve-up factor:0.200 vcuf:0.200 <-- AE0 has only one member link from FPC8, so 1/5=0.2 Carve-up factor bandwidth:2500000 bytes/sec, burst size:1600000 bytes
Regular policer '100M' dfw:167772160 pfe_id:0 carve-up factor:0.800 vcuf:0.800 <-- AE0 has 4 links from FPC11, so 4/5=0.8 Carve-up factor bandwidth:10000000 bytes/sec, burst size:6400000 bytes jtac@ERX-MX960-II-RE0# run show interfaces pp0.3221225481 extensive | match bps Input bytes : 684 0 bps Output bytes : 2256293826 80189776 bps <-- ~100Mbps
Targeted distribution enables you to target the egress traffic for an IP or VLAN demux subscriber on a single member link using a single scheduler resource. To achieve load balancing over the member links, the system distributes the subscriber interfaces equally among the links.
interfaces { ae0 { hierarchical-scheduler maximum-hierarchy-levels 2; interface-transmit-statistics; flexible-vlan-tagging; auto-configure { stacked-vlan-ranges { dynamic-profile DYN-VLAN-Q-IN-Q-AE-PROFILE { accept pppoe; ranges { any,any; } } } remove-when-no-subscribers; } mtu 9192; encapsulation flexible-ethernet-services; aggregated-ether-options { no-flow-control; minimum-links 1; link-speed 10g; lacp { active; periodic fast; } } } } jtac@ERX-MX960-II-RE0# show dynamic-profiles DYN-VLAN-Q-IN-Q-AE-PROFILE routing-instances { "$junos-routing-instance" { interface "$junos-interface-name"; } } interfaces { demux0 { unit "$junos-interface-unit" { actual-transit-statistics; demux-source [ inet inet6 ]; no-traps; proxy-arp unrestricted; vlan-tags outer "$junos-stacked-vlan-id" inner "$junos-vlan-id"; demux-options { underlying-interface "$junos-underlying-interface"; } family inet { unnumbered-address "$junos-loopback-interface"; } family inet6 { unnumbered-address "$junos-loopback-interface"; } family pppoe { access-concentrator MX960; dynamic-profile PPPOE-PROFILE; } } } } jtac@ERX-MX960-II-RE0# show dynamic-profiles PPPOE-PROFILE routing-instances { "$junos-routing-instance" { interface "$junos-interface-name"; } } interfaces { pp0 { unit "$junos-interface-unit" { no-traps; ppp-options { pap; } pppoe-options { underlying-interface "$junos-underlying-interface"; server; } targeted-distribution; <-- Enable targeted distribution
keepalives interval 20; family inet { filter { input "$junos-input-filter"; output "$junos-output-filter"; } unnumbered-address "$junos-loopback-interface"; } family inet6 { unnumbered-address "$junos-loopback-interface"; } } } }
Verify
jtac@ERX-MX960-II-RE0# run show subscribers extensive Type: VLAN Logical System: default Routing Instance: default Interface: demux0.3221225482 Interface type: Dynamic Underlying Interface: ae0 Dynamic Profile Name: DYN-VLAN-Q-IN-Q-AE-PROFILE Dynamic Profile Version: 1 State: Active Session ID: 11 PFE Flow ID: 30 Stacked VLAN Id: 0x8100.2 VLAN Id: 0x8100.200 Login Time: 2017-04-04 12:44:09 IST Type: PPPoE User Name: [email protected] IP Address: 10.200.0.7 IP Netmask: 255.255.255.255 Domain name server inet: 8.8.4.4 Domain name server inet6: 2001:4860:4860::8844 Logical System: default Routing Instance: default Interface: pp0.3221225483 Interface type: Dynamic Underlying Interface: demux0.3221225482 Dynamic Profile Name: PPPOE-PROFILE Dynamic Profile Version: 1 MAC Address: 00:16:01:00:00:01 State: Active Radius Accounting ID: 12 Session ID: 12 PFE Flow ID: 33 Stacked VLAN Id: 2 VLAN Id: 200 Login Time: 2017-04-04 12:44:09 IST IP Address Pool: ipv4-pool IPv4 Input Filter Name: 100M-pp0.3221225483-in IPv4 Output Filter Name: 100M-pp0.3221225483-out Accounting interval: 1800 Dynamic configuration: junos-input-filter: 100M junos-output-filter: 100M jtac@ERX-MX960-II-RE0# run show interfaces targeting Aggregated interface: ae0 Targeting Type: Auto Redundancy mode: Link Level Redundancy Total number of distribution groups: 1 Total number of distributed interfaces: 1 Distribution Group name: default Number of distributed interfaces: 1 Physical interface: xe-11/0/0, Link status: Up Number of primary distributions: 1 Number of backup distributions: 0 Physical interface: xe-11/0/1, Link status: Up Number of primary distributions: 0 Number of backup distributions: 1 Physical interface: xe-11/1/0, Link status: Up Number of primary distributions: 0 Number of backup distributions: 0 Physical interface: xe-11/1/1, Link status: Up Number of primary distributions: 0 Number of backup distributions: 0 Physical interface: xe-8/0/6, Link status: Up Number of primary distributions: 0 Number of backup distributions: 0 jtac@ERX-MX960-II-RE0# run show interfaces pp0.3221225483 extensive | match bps Input bytes : 190 0 bps Output bytes : 66540630 100378728 bps <-- 100Mbps jtac@ERX-MX960-II-RE0# run show interfaces xe-11/0/0 extensive | match bps Link-level type: Flexible-Ethernet, MTU: 9192, MRU: 9200, LAN-PHY mode, Speed: 10Gbps, BPDU Error: None, Loop Detect PDU Error: None, MAC-REWRITE Error: None, Loopback: None, Input bytes : 7879396 992 bps Output bytes : 66352704835 99880936 bps <-- Traffic going out from one member link