When the VPLS instance is on ACX, with one out of 2 child interface in detached state & a single packet is sent, the MAC is relearned from Egress interface in VPLS instance instead of ingress.
We are not seeing this issue when core is on MX.
Customer has a setup wherein they have below design.
Switch1 <-> Core (with VPLS) <-> Switch2
The switch ae bundle will have one out of 2 links as backup & hence the same is in detached state at core.
When switch sent a BPDU packet, it get forwarded to peer switch & then ACX put the MAC as learned from Egress interface, which cause traffic blackholing.
If we sent single ICMP packet, we can see same behaviour & with a 2nd icmp packet, its re-learned correctly from ingress.
It also get cleared after default mac flush timer.
If we remove the detached link from ACX bundle, the issue is not seen.
Lab: MX80 <-> ACX7348 <-> MX80 (2nd link) *Lab outputs* ACX: labroot@ACX7348-C-EG-re0> show lacp interfaces Aggregated interface: ae1 LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity et-0/0/0 Actor No No Yes Yes Yes Yes Fast Active et-0/0/0 Partner No No Yes Yes Yes Yes Fast Active et-0/0/1 Actor No Yes No No No Yes Fast Active et-0/0/1 Partner No Yes No No No Yes Fast Passive LACP protocol: Receive State Transmit State Mux State et-0/0/0 Current Fast periodic Collecting distributing et-0/0/1 Defaulted Fast periodic Detached labroot@ACX7348-C-EG-re0> clear ethernet-switching table {master} labroot@ACX7348-C-EG-re0> clear ethernet-switching mac-learning-log {master} labroot@ACX7348-C-EG-re0> show ethernet-switching table {master} labroot@ACX7348-C-EG-re0> show ethernet-switching mac-learning-log {master} MX: labroot@MX80> show configuration interfaces irb | display set set interfaces irb unit 101 family inet address 10.10.101.1/24 labroot@MX80> show configuration routing-instances test | display set set routing-instances test instance-type virtual-router set routing-instances test interface ge-1/0/2.101 labroot@MX80> show configuration interfaces ge-1/0/2 | display set set interfaces ge-1/0/2 vlan-tagging set interfaces ge-1/0/2 mac ca:fe:00:11:22:33 set interfaces ge-1/0/2 unit 101 vlan-id 101 set interfaces ge-1/0/2 unit 101 family inet address 10.10.101.20/24 labroot@MX80> clear arp 10.85.161.3 deleted 10.85.161.2 deleted 10.85.161.5 deleted 10.85.161.37 deleted 10.10.101.1 deleted 10.10.101.20 deleted labroot@MX80> clear arp vpn test labroot@MX80> ACX: labroot@ACX7348-C-EG-re0> show ethernet-switching table MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static, C - Control MAC SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - ovsdb MAC GBP - group based policy, B - Blocked MAC) Ethernet switching table : 2 entries, 2 learned Routing instance : VPLS Vlan MAC MAC Age GBP Logical NH MAC RTR name address flags Tag interface Index property ID v101 08:81:f4:82:47:e0 D - ae1.101 0 0 v101 ca:fe:00:11:22:33 D - ae1.101 0 0 {master} labroot@ACX7348-C-EG-re0> show ethernet-switching mac-learning-log Fri Jan 30 14:09:29 vlan_name v101+101 mac ca:fe:00:11:22:33 was learned on et-0/0/2.101 with flags: 0x2001f Fri Jan 30 14:09:29 vlan_name v101+101 mac ca:fe:00:11:22:33 was moved from et-0/0/2.101 to ae1.101 with flags: 0x2001f Fri Jan 30 14:09:29 vlan_name v101+101 mac 08:81:f4:82:47:e0 was learned on ae1.101 with flags: 0x2001f Cleared after mac flush: Fri Jan 30 14:14:06 vlan_name v101+101 mac ca:fe:00:11:22:33 was deleted from ae1.101 with flags: 0x1000 Fri Jan 30 14:14:29 vlan_name v101+101 mac ca:fe:00:11:22:33 was learned on et-0/0/2.101 with flags: 0x2001f labroot@ACX7348-C-EG-re0> show ethernet-switching table MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static, C - Control MAC SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - ovsdb MAC GBP - group based policy, B - Blocked MAC) Ethernet switching table : 2 entries, 2 learned Routing instance : VPLS Vlan MAC MAC Age GBP Logical NH MAC RTR name address flags Tag interface Index property ID v101 08:81:f4:82:47:e0 D - ae1.101 0 0 v101 ca:fe:00:11:22:33 D - et-0/0/2.101 0 0
This was replicated in lab & was analysed by Engineering via internal PR 1934724.
The behaviour flagged in this PR is expected by current design from both Broadcom and PFE perspective and this is day-1 behaviour in ACX7k.
We are planning to change this behaviour to follow mx but this will be a behavioural change and code churn will be high.
According to ACX7K design, a LAG/Lag members with asymmetric LACP configuration (one side LACP, one side non-LACP) will not block the ingress/incoming traffic.
Broadcom has a limitation in which it can block only egress traffic for a lag member. So, only egress traffic is blocked. This will be applicable for all of the ACX varieties.
So for the customer suggestion in this case is: to have symmetric LACP configuration on both of the peers.
We can resolve this issue by deleting the members from lag bundle(only in relevant cases), but this will take good amount of design and test cycles, we are planning commit this only in master branch.