This article discusses a scenario where BFD can flap due to the absence of COS configuration on certain IFLs.

Topology Diagram -

• BFD is configured between MX10K4 (DUT) and IXIA2.
• IXIA1 was used to generate traffic towards DUT.
• Total IFLs configured on the device numbered 900, ranging from 1 to 900. Initially, 400 IFLs were tested, each having one IPv4 and IPv6 BFD and BGP session between DUT and IXIA.
• A scheduler-map and different shaping rate of 650m,500m,520m  were applied and tested to 400 IFLs (from 1 to 400).
• On the remaining interfaces (401 to 900), although BFD, BGP, and COS configurations were present on both DUT and IXIA, these IFLs lacked a scheduler-map or shaper.
• With no scheduler-map or scheduler, BFDs were flapping randomly on different IFLs (Flapping was observed for both IFLs with or without scheduler-map or shaper configuration).
• We confirmed that BFD packets were correctly directed to the network-control queue as per the configuration. No drops were observed on that NC queue.
• Other queues on the egress interface were congested.
• The transmit-rate and drops on all congested queues were consistent with the configured schedulers for each queue, indicating that COS shaping and scheduling behaviour met expectations.
• COS queues evaluate traffic-rate to make drop decisions and do not have the capability to differentiate between packet types within the ASIC queuing block. Therefore, COS queues cannot selectively drop specific protocol packets (such as BFD packets in this case).
• We subsequently applied the scheduler-map and shaper to the remaining 500 IFLs, ensuring all 900 IFLs now had scheduler-maps and shapers configured.
• Each IFL maintained one IPv4 and IPv6 BFD and BGP session, with a shaper of 600m and 500m applied cyclically for 15 minutes. No BFD flapping occurred during these tests.

Observations:

For IFLs lacking COS configuration, even though there were no drops on the network-control queue-7, packet delays may occur due to GL priority and bandwidth congestion at this priority level, potentially causing BFD flaps. The weights of these unconfigured network-control queues were notably lower compared to other data-traffic queues.

Recommendations for such scenarios:

1. Always set network-control queues (handling control packets) to High priority.
2. Allocate guaranteed bandwidth to the network-control queue based on expected control traffic to prevent bandwidth starvation.
3. In scenarios where some IFLs have COS scheduler-maps and some do not, consider working in HCOS (hierarchical-scheduler) mode instead of per-unit-scheduler mode for improved COS resource and queue utilization. Use a 2-level HCOS if IFL sets are unnecessary and IFL scale is high.

• When using HCOS mode, configure COS output-traffic-control-profile and output-traffic-control-profile-remaining at the IFD level. Ensure to allocate guaranteed bandwidth to the network-control queue with High priority. This setup applies to IFLs lacking explicit COS configuration, utilizing these IFD-level configurations.