Article Detail

Description

The intention of the article is to provide steps for log collection when MAC- PINNING is observed on the log when there was no mac-pinning configured provides a workaround to fix.

Symptoms

In a similar setup in customer environment it was observed that MAC’s were learnt from both directly connected interface and from the bundle interface . In customer environment they had load balancers connected to the MX’s . Load balancers have Physical IP as well as VIP between them . There was no mac-pinning configuration on customer configuration . To simulate the issue in local lab we had below setup .

1.Upon checking Bridge table it was observed that MAC was moving between directly connected interfaces and the bundle interface .

labroot@jtac-mx480-r2064-re0> show bridge mac-table vlan-id 652 | refresh 2
Apr 05 22:32:32
Routing instance : default-switch
Bridging domain : 652, VLAN : 652
MAC   MAC   Logical NH MAC active
address flags interface Index property   source
  00:00:00:00:00:05   D ae4.652
  00:00:00:00:00:06   D xe-2/0/16.0
  00:00:00:00:00:07   D xe-2/0/16.0
  00:00:00:00:00:08   D xe-2/0/16.0
  00:00:00:00:00:09   D xe-2/0/16.0
  00:00:00:00:00:0a D xe-2/0/16.0

MAC flags     (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC
O -OVSDB MAC, SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC, P -Pinned MAC)

Routing instance : default-switch
Bridging domain : 652, VLAN : 652
MAC   MAC   Logical NH MAC active
address flags interface Index property   source
  00:00:00:00:00:05   D xe-2/0/16.0
  00:00:00:00:00:06   D ae4.652
  00:00:00:00:00:07   D ae4.652
  00:00:00:00:00:08   D ae4.652
  00:00:00:00:00:09   D xe-2/0/16.0
  00:00:00:00:00:0a D xe-2/0/16.0

l2ald[26018]: L2ALD_MAC_PINNING_DISCARD_NOTIFICATION: MAC: 00:00:00:00:00:06, learned for BD 652+652 at MAC Pinning interface ae4.652 is discarded on interface xe-2/0/16.0
Apr 5 21:47:53.670 jtac-mx480-r2064-re0 l2ald[26018]: L2ALD_MAC_PINNING_DISCARD_NOTIFICATION: MAC: 00:00:00:00:00:09, learned for BD 652+652 at MAC Pinning interface xe-2/0/16.0 is discarded on interface ae4.652
Apr 5 21:47:55.026 jtac-mx480-r2064-re0 l2ald[26018]: L2ALD_MAC_MOVE_NOTIFICATION: MAC Moves detected in the system
Apr 5 21:47:57.013 jtac-mx480-r2064-re0 l2ald[26018]: L2ALD_MAC_MOVE_NOTIFICATION: MAC Moves detected in the system
Apr 5 21:47:59.002 jtac-mx480-r2064-re0 l2ald[26018]:
L2ALD_MAC_PINNING_DISCARD_NOTIFICATION: MAC: 00:00:00:00:00:05, learned for BD 652+652 at MAC Pinning interface ae4.652 is discarded on interface xe-2/0/16.0

2. Check the control plane related commands for Bridge Domains and check to see if there are any MAC-PINNING enabled on the control plane.

show bridge mac-table vlan-id 652 extensive
show l2-learning interface
show l2-learning global-information

3. If no MAC-PIN flag was enabled on the control plane then next check the PFE related commands if we see MAC_PIN is set or not .

Identify the FPC type card .
Identify the PFE ID of the interface on which the interface is hosted .

4. Upon checking exceptions observed MAC-PINNIG discards are incrementing or not.

Bridging
----------------------
bridge ucast split horizon         DISC( 26)            1        102
iif STP blocked                    DISC( 3)        35348    2121332
oif STP blocked                    DISC( 31)       741378   44553484
mlp pkt                            PUNT( 11)   11267190731 326748531199
MAC pinning packet discards        PUNT( 79)   31970631323 2028452638886

Bridging
----------------------
bridge ucast split horizon         DISC( 26)            1        102
iif STP blocked                    DISC( 3)        35348    2121332
oif STP blocked                    DISC( 31)       741378   44553484
mlp pkt                            PUNT( 11)   11267190731 326748531199
MAC pinning packet discards        PUNT( 79)   31975407031 2028739

Solution

First we need to check the BD ID on PFE level .

SMPC2(jtac-mx480-r2064-re0 vty)# show bridge-domain
651+651                                        33        3   Regular            0
652+652                                        34        3

Next, check for the suspected MAC on the PFE on the SW . Check the flags and see if there is any mac-pin set or not.

cprod -A fpc2 -c 'show l2metro 0 mac sw bridge-domain 0x34' | grep 00:00:00:00:00:07 -A 20

52 0 00:00:00:00:00:08 566 1 6 67 0000b6 1 7 8/ 7 Drop 0015 62095 0 4

unrslvd ignr_drp lrn_rtry static bm_dmac_tk ignore_cvlan_4lsbits
---------------------------------------------------------------
0 1 1 0 1 1

Current Age: 0
MAC aging time : 300
cvlan_list   : 0xff00
mac_pin   : 0x1

Next, check if on the HW level what is the action specified .

52 0 00:00:00:00:00:0d    669 75   6    67   012084 1 0 254/254 Drop 0015 98808 0 2

During continuous mac move between two IFLs in same BD, there can be a scenario where by the time MLP ack sent by l2pd updates the first learnt Mac entry, the mac has moved and learnt on a different IFL.Since the hash table lookup key only has MAC, Learn vlan and BD ID, even for the newly Learnt Mac on a different IFL,hash table lookup succeeds, while L2pd ack tries to update the Mac in HW. So, Mac update for the older L2pd ack is corrupting the newly learnt Mac's sideband data.

On customer environment as a temporary workaround bridge table was cleared from the CLI to clear the issue.

clear bridge mac-table bridge-domain 652

Modification History

2024-05-08 : Article Created