This article details about a critical issue regarding the IPv4 packet drops that can experiencing on the Aggregate Ethernet (AE) member interface. These packet drops seem to create an illusion of unequal load balancing across the AE bundle, which could significantly impact network performance and efficiency.
Model: PTX5000
Junos: 20.4R3-S1.3
Working State
labroot@jtac-ptx5000dc-r2004-re0> show interfaces et-4/0/8 | match "Output rate"
Output rate : 18418288 bps (2321 pps)
labroot@jtac-ptx5000dc-r2004-re0> show interfaces et-4/0/10 | match "Output rate"
Output rate : 12755656 bps (1632 pps)
labroot@jtac-ptx5000dc-r2004-re0> show interfaces et-4/0/11 | match "Output rate"
Output rate : 18783184 bps (2353 pps)
Non Working State
Output rate : 18576952 bps (2345 pps)
{master}
Output rate : 336 bps (0 pps)
Output rate : 0 bps (0 pps)
too long# show pechip trapstats
DEV TRAPCODE count rate
=========================================================================================
0 ( 1) dlu.ucode.inv_start_pc 1 0
0 ( 12) dlu.ucode.pkt_to_cpu 696 1
0 ( 165) irp.pdct.pol_drop_trapcode 34257 0
0 ( 252) sw.nh.discard_sampling_trap 6 0
1 ( 12) dlu.ucode.pkt_to_cpu 693 0
1 ( 162) sw.mflt.dmac_dflt_drop 6 0
4 ( 12) dlu.ucode.pkt_to_cpu 916 1
4 ( 250) sw.egnh.cfg_discard 266459 4021 << PFE 4 hosting et-4/0/10 and et-4/0/11
labroot@jtac-ptx5000dc-r2004-re0> show route forwarding-table destination 99.82.177.118 << Unicast NH
Routing table: default.inet
Internet:
Destination Type RtRef Next hop Type Index NhRef Netif
99.82.177.118/32 dest 1 98:49:25:1c:67:51 ucst 581 4 ae1002.0
too long# show nhdb id 581 extensive
ID Type Interface Next Hop Addr Protocol Encap MTU Flags PFE internal Flags
----- -------- ------------- --------------- ---------- ------------ ---- ------------------ ------------------
581 Unicast ae1002.0 99.82.177.118 IPv4 Ethernet 0 0x0000000000000000 0x0000002000002000
BFD Session Id: 320
...
for fe_id = 5, Ingress NH handle:
handle 0x486d8b60, flags 0x0, refcount 1
NH installed at addr NH 8, INT_SEQ 0x10000042
Raw dump of the nh words of size 2 words
0x1000023b 0x04030ffa
SEQ [1000023b] Interm, SIZE 2, NO_ACT 0, USE_REMAP 0, NEXT_ADDR: 00047, SZ: 3
Nexthop points to:
SEQ [480005e9] Eq_List, SIZE 3, FIN 1, BASE ADDR/SZ 0xbd/1, HASH 3 MASK: SZ 1, SEL 1, PTR/MASK 0x60 OFST 2
EQ-List Mask Words via mask pointer [mask/size]: 0xe0000000/4
Final members:0x20299800(3264) 0x20282400(288) 0x20299d00(3304)
ACT: [04030ffa] EgNHId: EG_NHID: 030ffa
too long# show jencap show_flabel 4 0x030ffa regular << PFE4
Flabel : 200698 Segment table index: : 48[1] Page Table index : 143[3] desc start addr:: 5146[1]
T: DMAC dmac: 00009849251c6751
F: P_NEXT p_next: 201792
F: COUNTER counter: 130774 cix_tc_en: 0
F: P_NEXT p_next: 201784
F: IDX_P_NEXT base_addr: 201846 entry_sz: 1 field_sel: 3 msb: 0 lsb: 0 << IPv6 Unicast NH feature and not IPv4 unicast NH feature
F: TIX tix: 5
F: VLAN_TBL address: 15814 use_l2dom: 0 use_vlan: 1 use_l3_vpn: 1 use_def_vfid: 1 use_stp_set: 1 outer_etype: 0 inner_etype: 0 format_selet: 31
VLAN TBL INFO L2D entry type 0 location 0/7907/0
L2 Domain id :2 Lport id : 2 outer_vlan : 4095 inner_vlan : 4095 vlan ctxt : 1 Baidx: 67
F: P_CONST bv_flags: 2 address: 14
[ 14] Refcount 5 SMAC 30:b6:4f:7c:ba:ee
T: PORT_L2DOMAIN eg_l2_dom: 2 stream: 0
T: L2_DOMAIN eg_l2_dom: 3 eg_gl2dom: 70
T: L3_MTU l3_mtu: 1500 bv_l3type: 3 inc_mpls: 0 en_fragmentation: 1 clear_df: 0
T: IP_REDIRECT use_prefix_len: 0 cmp_hash_val: 0
Flabel show Success
too long# show jencap show_descr 4 201846
Flabel : 16777217 Segment table index: : 0[2] Page Table index : 128[308] desc start addr:: 33[1]
F: P_NEXT p_next: 201854
F: TIX tix: 1
T: QOS_CTRL st_code: 2 st_val: 250 s_tc: 0 tc: 0 s_dp: 0 dp: 0 allow_plct_update: 0
Descr show Success
In PTX platforms, with the AE IFL in both IPv4 and IPv6 egress traffic scenarios, they share the same next-hop descriptor address. When the configuration “forwarding-options family inet6 route-accounting” is present, and if we flap the inet6 family from the AEx bundle, it results in the IPv6 next-hop being created as an IPv4 next-hop. This can lead to improper handling of the egress next-hop by the PFE, potentially causing egress traffic to be forwarded through the newly created IPv4 next-hop of that AE IFL, which may ultimately lead to egress packet drops.
Issue recovered by bouncing (deactivate/activate) affected interfaces
labroot@jtac-ptx5000dc-r2004-re0# deactivate interfaces et-4/0/10
{master}[edit]
labroot@jtac-ptx5000dc-r2004-re0# deactivate interfaces et-4/0/11
labroot@jtac-ptx5000dc-r2004-re0# commit
labroot@jtac-ptx5000dc-r2004-re0# show | compare
[edit interfaces]
! active: et-4/0/10 { ... }
! active: et-4/0/11 { ... }
labroot@jtac-ptx5000dc-r2004-re0# commit and-quit
0 ( 1) dlu.ucode.inv_start_pc 3 0
0 ( 12) dlu.ucode.pkt_to_cpu 920 0
1 ( 12) dlu.ucode.pkt_to_cpu 918 0
1 ( 162) sw.mflt.dmac_dflt_drop 12 0
4 ( 12) dlu.ucode.pkt_to_cpu 1270 1
4 ( 250) sw.egnh.cfg_discard 861196 0