Link Aggreggate Group formation fails and traffic is sent on only one internal port from EX4550 to the EX4200. The correction is to display the speed as unidirectional (16Gb) rather than bidirectional (32Gb). This article explains the symptoms and cause, and recommends that users configure additional VC-ports if the traffic between EX4200 <-> EX4550 is greater than 16Gb.
Traffic is sent on only one internal port from EX4550 to the EX4200.
labroot@MixedVC> show virtual-chassis Virtual Chassis ID: e450.08dd.3f6d Virtual Chassis Mode: Mixed Mstr Mixed Route Neighbor List Member ID Status Serial No Model prio Role Mode Mode ID Interface 0 (FPC 0) Prsnt BN0208190433 ex4200-24p 128 Master* Y VC 1 vcp-0 2 vcp-1 1 (FPC 1) Prsnt LY0212500079 ex4550-32t 128 Linecard Y VC 0 vcp-255/2/0 2 vcp-255/2/2 2 vcp-255/2/3 2 (FPC 2) Prsnt LY0212500081 ex4550-32t 128 Backup Y VC 0 vcp-255/2/0 1 vcp-255/2/2 1 vcp-255/2/3 labroot@MixedVC> show virtual-chassis vc-port fpc0: -------------------------------------------------------------------------- Interface Type Trunk Status Speed Neighbor or ID (mbps) ID Interface PIC / Port vcp-0 Dedicated 2 Up 32000 1 vcp-255/2/0 vcp-1 Dedicated 1 Up 32000 2 vcp-255/2/0 fpc1: -------------------------------------------------------------------------- Interface Type Trunk Status Speed Neighbor or ID (mbps) ID Interface PIC / Port 2/0 Configured -1 Up 16000 0 vcp-0 2/1 Configured -1 Up 16000 2/2 Configured 1 Up 16000 2 vcp-255/2/2 2/3 Configured 1 Up 16000 2 vcp-255/2/3 fpc2: -------------------------------------------------------------------------- Interface Type Trunk Status Speed Neighbor or ID (mbps) ID Interface PIC / Port 2/0 Configured -1 Up 16000 0 vcp-1 2/1 Configured -1 Up 16000 2/2 Configured 1 Up 16000 1 vcp-255/2/2 2/3 Configured 1 Up 16000 1 vcp-255/2/3 labroot@MixedVC> show virtual-chassis vc-port statistics member 0 fpc0: -------------------------------------------------------------------------- Interface Input Octets/Packets Output Octets/Packets Input Output Util Util internal-0/24 200589161251 / 2785831743 6555844 / 19362 0 0 internal-0/25 33068940 / 23190 6099215 / 63329 0 0 internal-1/26 200572464931 / 2785659506 13082026 / 52793 0 0 internal-1/27 31888212 / 22362 29629123 / 97163 0 0 vcp-0 200622230191 / 2785854933 12655059 / 82691 2 0 vcp-1 200604353143 / 2785681868 42711149 / 149956 2 0 <----port towards EX4550 internal-0/26 5266220 / 60992 29639056 / 97181 0 0 internal-0/27 1115274 / 12351 7787988 / 45964 0 0 internal-1/24 7787988 / 45964 1115274 / 12351 0 0 internal-1/25 29639056 / 97181 5266220 / 60992 0 0 labroot@MixedVC> show virtual-chassis vc-port statistics member 2 fpc2: -------------------------------------------------------------------------- Interface Input Octets/Packets Output Octets/Packets Input Output Util Util vcp-255/2/0 13020040 / 52533 197789944917 / 2747013725 0 5 --->port towards EX4200 vcp-255/2/1 29601275 / 96823 31749890 / 22265 0 0 --->port towards EX4200 vcp-255/2/2 4315913 / 4713 197790222951 / 2747029387 0 5 vcp-255/2/3 5260310 / 6754 13209411 / 88437 0 0
The EX4200 is sending 149956 packets towards the EX4550 on vcp-1 that is connected to EX4550 vcp-255/2/0 and vcp-255/2/1. On these last two ports, there are 52533 + 96823 = 149356 packets received. Thus the direction EX4200 ------> EX4550 is load balancing between the two ports in VCP. In the other direction, EX4550------>EX4200, the EX4550 is sending 2747013725 towards the EX4200 (on 2/0) and only 22265 (2/1) and the EX4200 is receiving 2785681868 on vcp1. In conclusion, traffic is sent only on 2/0 from EX4550 towards EX4200.
The virtual chassis (VC) module on the EX4550 has two virtual chassis ports (VCP). Each VCP has two internal ports, for a total of 4 internal ports. When a VC module is inserted into the rear module slot (PIC-2), PFE internal ports 40 and 42 are configured as DXAUI 16G MAC and a Link Aggregation Group (LAG) is created to form one VC port. Ports 56 and 58 are configured as DXAUI 16G MAC, both of which are link aggregated to form another VC port, both running at 32Gb. The rear slot VC ports are vcp-2/0 and vcp-2/1. Each internal port speed is 16Gbps unidirectional, for a total of 64Gbps unidirectional, 128Gbps bidirectional in total. The speed is not 32Gbps as one might see on some outputs from customer VCs. This is misleading as the speed is counted bidirectional. There is a software fix that has changed the above output to reflect the correct speed. This fix has not been committed to 12.3 throttle. In 15.1 throttle, the display has been changed to reflect the actual speed of the ports. The correction is to display the speed as unidirectional (16Gb) rather than bidirectional (32Gb).
fpc6: -------------------------------------------------------------------------- Interface Type Trunk Status Speed Neighbor or ID (mbps) ID Interface PIC / Port 0/26 Configured -1 Up 10000 4 vcp-255/1/0 2/0 Configured -1 Up 16000 1 vcp-1 2/1 Configured -1 Up 16000 2/2 Configured -1 Up 16000 2/3 Configured -1 Up 16000 5 vcp-255/2/0
In the EX4550, a VCCPD interface is created for both the internal ports (255/2/0 and 255/2/1) connecting to the EX4200. In the case of the EX4200, only one virtual chassis control protocol (VCCPD) interface is created per VCP (vcp-0/vcp-1) though internally the vcp-0/vcp-1 maps to two Marvell ports. Due to this, one of the adjacencies in EX4550 is not formed and hence the LAG( Link Aggregation Group) is not created.
Attempts to change this will be a change at design level and impacts all platforms, as the code is common. Also, after this change, NSSU will be affected if the upgrade is from the older release, which does not have this change to new release.
Considering the high risk involved, we recommend customers configure additional VC-ports if the traffic between 4200 <-> 4550 is greater than 16Gb.
This behavior is only seen when using EX4550 in a mixed VC setup. It is not applicable for EX4550-VC , EX4500-VC or Mixed EX4500-EX4200-VC.