This article describes the scenario where Juniper SFP-T interfaces stops to be operational after the upgrade to Junos 21.4R3 and how to resolve this issue.
1. With 19.4R3-S11.2 version, the SFP-T interfaces from PIC 1 [ge-0/1/0, ge-0/1/2] were UP.
{master:0} user@switch> show version fpc0: -------------------------------------------------------------------------- Hostname: switch Model: qfx5100-24q-2p Junos: 19.4R3-S11.2
Output of "show chassis hardware detail"
FPC 0 REV 50 650-049942 ABCD1111 QFX5100-24Q-2P CPU BUILTIN BUILTIN FPC CPU PIC 0 BUILTIN BUILTIN 24x 40G-QSFP Xcvr 0 REV 01 740-056705 ABCD1112 QSFP+40GE-LX4 Xcvr 1 REV 01 740-056705 ABDC1113 QSFP+40GE-LX4 PIC 1 REV 02 611-064275 ABCD1114 EX4600-EM-8F Xcvr 0 REV 02 740-013111 ABCD1115 SFP-T Xcvr 2 REV 02 740-013111 ABCD1116 SFP-T
Output of "show interfaces terse" [where interfaces with SFP-T module are in "UP" state]
ge-0/1/0 up up ge-0/1/0.858 up up inet 192.168.0.1/24 ge-0/1/0.859 up up inet 192.168.0.1/24 ge-0/1/0.32767 up up ge-0/1/2 up up ge-0/1/2.0 up up inet 192.168.1.1/24
2. After upgrading it to 21.4R3-S3.4, the SFP-T interfaces moved to down state.
user@switch> show version fpc0: -------------------------------------------------------------------------- Hostname: switch Model: qfx5100-24q-2p Junos: 21.4R3-S3.4
Output of "show interfaces terse" [ where interfaces are in "Down" state]
ge-0/1/0 up down ge-0/1/0.858 up down inet 192.168.0.1/24 ge-0/1/0.859 up down inet 192.168.0.1/24 ge-0/1/0.32767 up down ge-0/1/2 up down ge-0/1/2.0 up down inet 192.168.1.1/24
When the remote end server/system reboots, QFX5100 platform ports with SFP-T 1G inserted may go into a hung state and remain in that state even after the reboot is complete. This may affect traffic after the remote end system comes online and resumes traffic transmission. When this issue occurs, the affected physical link status (which can be checked by 'show interface' commands) will show up as 'down'. Status LED (Light Emitting Diode) status will also show up as OFF when 'show chassis led' command is issued for the interface. Even when the reboot is complete, the connecting link may not come up so it may impact traffic flow through the said ports.
Root-Cause:
SFP-T 1G Ports was going down on qfx5100 due to auto negotiation being enabled by default.
Work-Around:
Disable auto-negotiation from the Broadcom shell after remote device reboots.
From vty shell,
show dcbcm ifd all --> this command will give the bcm port numbers for all the interface. Example: user@switch> start shell % vty fpc0 TOR platform (1500 Mhz Pentium processor, 511MB memory, 0KB flash) too long# show dcbcm ifd all ---------------------------------------------------------- ifd name global-dev local-dev port-num port-name ---------------------------------------------------------- ge-0/1/0 0 0 86 xe84 ge-0/1/1 0 0 87 xe85 ge-0/1/2 0 0 88 xe86 <<Interface which is in Down state and port_name is xe86. ge-0/1/3 0 0 89 xe87
Now to enter into the bcm shell:From vty shell,
set dcbcm diag --> this command to enter into the bcm shell.
After this :
port <port_name> AN=0 --> to disable auto-negotiation for the specific port. Example:
# set dcbcm diag Boot flags: Probe NOT performed Boot flags: Cold boot Boot flags: initialization scripts NOT loaded BCM.0> port xe86 AN=0 BCM.0>
Note: This workaround needs to be applied every time the server or QFX reboots.** The cli knob to disable auto-negotiation may result in unpredictable behavior. Hence, VTY workaround is suggested.
This is fixed through PR1742565 .