Description

Output errors (tail drops) may increment when an interface is experiencing microburst traffic loads contributed by end hosts that are connected to the downstream interfaces, or due to a scale issue.

This article suggests a few workarounds to address the problem.

Symptoms

In the example, the MX Series device has cascade ports connecting to the satellite devices QFX 201 and 202 as extended fabric line-cards by using Junos Fusion. And the output discards are seen to happen only from the cascade ports that are connected to QFX202.
 

image.png

user@MX960> show interfaces xe-3/3/3 extensive | match drop 
  Dropped traffic statistics due to STP State:
    Errors: 0, Drops: 0, Framing errors: 0, Runts: 0, Policed discards: 0, L3 incompletes: 0,
    Carrier transitions: 0, Errors: 0, Drops: 770, Collisions: 0, Aged packets: 0, FIFO errors: 0,
  Queue counters:       Queued packets  Transmitted packets      Dropped packets
    Traffic Class        Received Packets   Transmitted  Packets      Dropped Packets
    Max nh cache: 75000, New hold nh limit: 75000, Curr nh cnt: 1, Curr new hold cnt: 0, NH drop cnt: 0

user@MX960> show interfaces xe-4/3/3 extensive | match drop
  Dropped traffic statistics due to STP State:
    Errors: 0, Drops: 0, Framing errors: 0, Runts: 0, Policed discards: 0, L3 incompletes: 0,
    Carrier transitions: 0, Errors: 0, Drops: 954, Collisions: 0, Aged packets: 0, FIFO errors: 0,
  Queue counters:       Queued packets  Transmitted packets      Dropped packets
    Traffic Class        Received Packets   Transmitted  Packets      Dropped Packets
    Max nh cache: 75000, New hold nh limit: 75000, Curr nh cnt: 1, Curr new hold cnt: 0, NH drop cnt: 0

user@MX960> show interfaces queue xe-4/3/3
...
Tail-dropped packets :                    58

MX960> show interfaces queue xe-3/3/3
...
Tail-dropped packets :                    34

To see which queue dropped a packet, click KB69632 - [MX] How can users determine the queue in which drops happen? [juniper.net]

Cause

The output discards and tail drops can be due to microburst traffic contributed by end hosts that are connected to QFX 202, or a scale issuefor example, more end hosts connect to QFX202 since the utilization of cascade ports on QFX201 is much less than connections to QFX 202.

QFX 201

user@MX960> show interfaces xe-3/3/2 |match pps            
  Input rate     : 1168 bps (0 pps)
  Output rate    : 8184 bps (6 pps)
user@MX960> show interfaces xe-4/3/2 |match pps
  Input rate     : 2904 bps (3 pps)
  Output rate    : 6840 bps (9 pps)

QFX 202

user@MX960> show interfaces xe-3/3/3 |match pps
  Input rate     : 4478808 bps (6103 pps)
  Output rate    : 247992984 bps (21287 pps)

user@MX960> show interfaces xe-4/3/3 |match pps
  Input rate     : 3932008 bps (5205 pps)
  Output rate    : 193360536 bps (16763 pps)

Solution

Use any one of the following to work around the problem:

  • If the tail drops happen from the low priority queues, fine-tune the Class of Service (CoS) configuration to assign resources from high priority queues.

  • Add more cascade ports from the Aggregation Device MX960.

  • Re-balance traffic load between the satellite devices QFX201 and QFX202 by moving the connected end hosts.

Modification History

First draft 2022-05-20

Related Information

[EX/QFX] Output tail drops increment on interface although traffic rate is within interface capacity