Description

When making the configuration change for the minimum number of links in an aggregate bundle, with optimize-aggregate-frr configured on the bundle, a link flap can occur.

Symptoms

Whenever a change to the configuration below is committed, an unexpected flap will occur with the aggregated interface.  This only occurs when optimize-aggregate-frr is configured and the specific conditions are met.

set interfaces ae50 aggregated-ether-options minimum-links 3 
[MASTER][edit]
labroot@hypnos-re1# show interfaces ae50 Jun 10 23:39:20 aggregated-ether-options {
    minimum-links 3;  ==> Configured minimum-links = 3 changed from 1
    optimize-aggregate-frr;
    link-speed 10g;
    lacp {
        active;
        periodic fast;
    }
}
unit 0 {
    family inet {
        address 10.80.4.1/30;
    }
}


Log messages seen: 

Jun 10 11:10:42.461  hypnos-re1 mgd[69382]: %INTERACT-6-UI_CMDLINE_READ_LINE: User 'labroot', command 'show lacp interfaces ae50 '
Jun 10 11:10:54.795  hypnos-re1 mgd[69382]: %CHANGE-6-UI_CFG_AUDIT_SET: User 'labroot' set: [interfaces ae50 aggregated-ether-options minimum-links] "1 -> "3"
Jun 10 11:10:54.795  hypnos-re1 mgd[69382]: %INTERACT-6-UI_CMDLINE_READ_LINE: User 'labroot', command 'set interfaces ae50 aggregated-ether-options minimum-links 3 '
Jun 10 11:11:02.002  hypnos-re1 cosd[15244]: %DAEMON-5: cosd_add_ae_parent_to_snmp_db:2670: Reset AE cache flag for AE ae50
Jun 10 11:11:02.809  hypnos-re1 rpd[14562]: %DAEMON-6: EVENT <Delete UpDown> ae50.0 index 736 <Broadcast Multicast> address #0 c.86.10.eb.f.c3
Jun 10 11:11:02.810  hypnos-re1 kernel: %KERN-6-LAG: Deleting cifl xe-5/1/1.0 from bundle ae50
Jun 10 11:11:02.810  hypnos-re1 kernel: %KERN-6-LAG: Deleting cifl xe-5/1/2.0 from bundle ae50
Jun 10 11:11:02.810  hypnos-re1 kernel: %KERN-6-LAG: Deleting cifl xe-5/1/3.0 from bundle ae50
Jun 10 11:11:02.810  hypnos-re1 kernel: %KERN-4: lag_bundlestate_ifd_change: bundle ae50: bundle IFD minimum bandwidth or minimum links not met, Bandwidth (Current : Required) 0 : 1 Number of links (Current : Required) 0 : 1
Jun 10 11:11:02.810  hypnos-re1 kernel: %KERN-6: lag_add_cifl_to_pifl: added a new ifl xe-5/1/1.0 to bundle ae50
Jun 10 11:11:02.810  hypnos-re1 kernel: %KERN-6: lag_add_cifl_to_pifl: added a new ifl xe-5/1/2.0 to bundle ae50
Jun 10 11:11:02.810  hypnos-re1 kernel: %KERN-6: lag_add_cifl_to_pifl: added a new ifl xe-5/1/3.0 to bundle ae50
Jun 10 11:11:02.810  hypnos-re1 kernel: %KERN-4: lag_bundlestate_ifd_change: bundle ae50 is now Up. uplinks 3 >= min_links 3
Jun 10 11:11:02.811  hypnos-re1 rpd[14562]: %DAEMON-6: EVENT <Add> ae50.0 index 735 <Up Broadcast Multicast> address #0 c.86.10.eb.f.c3
Jun 10 11:11:02.816  hypnos-re1 rpd[14562]: %DAEMON-6: EVENT <MTU> ae50.0 index 735 <Up Broadcast Multicast> address #0 c.86.10.eb.f.c3
Jun 10 11:11:02.816  hypnos-re1 rpd[14562]: %DAEMON-6: EVENT <Flags> ae50.0 index 735 <Up Broadcast Multicast> address #0 c.86.10.eb.f.c3
Jun 10 11:11:02.827  hypnos-re1 mib2d[15294]: %DAEMON-4-SNMP_TRAP_LINK_DOWN: ifIndex 1105, ifAdminStatus down(2), ifOperStatus down(2), ifName ae50.0
Jun 10 11:11:02.828  hypnos-re1 mib2d[15294]: %DAEMON-5-MIB2D_IF_FLAPPING_MISSING: if_track_flap_update: interface flapping missing: ifname(ae50) old if_anydown_flaps(3) if_anydown_flaps(5)
Jun 10 11:11:02.829  hypnos-re1 mib2d[15294]: %DAEMON-4-SNMP_TRAP_LINK_DOWN: ifIndex 1101, ifAdminStatus down(2), ifOperStatus down(2), ifName ae50
Jun 10 11:11:02.829  hypnos-re1 mib2d[15294]: %DAEMON-5-SNMP_TRAP_LINK_UP: ifIndex 1101, ifAdminStatus up(1), ifOperStatus up(1), ifName ae50
Jun 10 11:11:02.837  hypnos-re1 rpd[14562]: %DAEMON-6: EVENT <SNMP Index> ae50.0 index 735 <Up Broadcast Multicast> address #0 c.86.10.eb.f.c3
Jun 10 11:11:02.928  hypnos-re1 mib2d[15294]: %DAEMON-6-SNMP_TRAP_LINK_UP: ifIndex 1105, ifAdminStatus up(1), ifOperStatus up(1), ifName ae50.0
Jun 10 11:11:03.059  hypnos-re1 cosd[15244]: %DAEMON-7: link protection 0 for intf ae50
Jun 10 11:11:04.918  hypnos-re1 ifinfo: %USER-6-IFINFO_IFD_NAME: IFD: ae50
Jun 10 11:11:04.922  hypnos-re1 mgd[69382]: %INTERACT-6-UI_CMDLINE_READ_LINE: User 'labroot', command 'show interfaces ae50 '

Solution

Due to the way DCD handles the variable for optimize-aggregate-frr, DCD will trigger events due to the number of minimum links available to the link.

The values for the OPTIMIZE_AGGREGATE_FRR knob are defined as:

* 0 - Unconfigured.
* 1 - min links == num of links case.
* 2 - min links < num of links case.

Whenever the value of OPTIMIZE_AGGREGATE_FRR changes, DCD will trigger a catastrophic below event on AE IFD which causes in deletion and addition of IFLs, IFFs and IFAs on AE.

The issue will be seen in the following cases:

  • ‘minimum-links’ configured is same as number of links available and changes to a value lesser than no. of links available
  • ‘minimum-links’ configured is less than number of links available and changes to same value as no. of links available

This behavior is working as designed with optimize-aggregate-frr, so there is no workaround.  The flap is expected for the configuration.