Some 100G interfaces between QFX5210-64C and QFX5110 using QSFP-100G-SR4-T2 optic does not come up on certain ports.
Example:
Port 49 on QFX5210 and port 40 on QFX5100 does not come up, but if we move the port location on the 5210 to another port (for example port 50, the link comes up).
QFX5210-64C ----et-0/0/49-----------------et-0/0/40----QFX5110 <-- Link is down QFX5210-64C ----et-0/0/50-----------------et-0/0/40----QFX5110 <-- Link is up
This may be related to FEC errors on the QFX5210-64C side. Check if the link has any FEC errors.
root@QFX5210> show interfaces et-0/0/40 Physical interface: et-0/0/40, Enabled, Physical link is Down Interface index: 744, SNMP ifIndex: 515 Link-level type: Ethernet, MTU: 1514, LAN-PHY mode, Speed: 100Gbps, BPDU Error: None, Loop Detect PDU Error: None, Ethernet-Switching Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Disabled, Media type: Fiber Device flags : Present Running Down Interface flags: Hardware-Down SNMP-Traps Internal: 0x4000 Link flags : None CoS queues : 10 supported, 10 maximum usable queues Current address: f4:bf:a8:06:0b:47, Hardware address: f4:bf:a8:06:0b:47 Last flapped : 2021-06-07 08:23:52 BOT (07:18:58 ago) Input rate : 0 bps (0 pps) Output rate : 0 bps (0 pps) Active alarms : LINK Active defects : LINK, LOCAL-FAULT Pre-emphasis(reg 236/237) Register-value 0 Ctle(reg 234/235) Register-value 0 Differential-amplitude(reg 238/239) Register-value 0 PCS statistics Seconds Bit errors 0 Errored blocks 0 Ethernet FEC Mode : FEC91 Ethernet FEC statistics Errors FEC Corrected Errors 258874756 FEC Uncorrected Errors 27714745 FEC Corrected Errors Rate 0 FEC Uncorrected Errors Rate 0 PRBS Statistics : Disabled Interface transmit statistics: Disabled
From the "show interfaces extensive" output, both ports have Ethernet FEC Mode set to FEC91
"show interfaces extensive"
{master:0} root@QFX5210-64c> show interfaces et-0/0/40 extensive | match "FEC" Active defects : None Ethernet FEC Mode : FEC91 Ethernet FEC statistics Errors FEC Corrected Errors 258874756 FEC Uncorrected Errors 27714745 FEC Corrected Errors Rate 0 FEC Uncorrected Errors Rate 0 {master:0} root@QFX5110-48S> show interfaces et-0/0/49 | match FEC Active defects : None Ethernet FEC Mode : FEC91 Ethernet FEC statistics Errors FEC Corrected Errors 0 FEC Uncorrected Errors 0 FEC Corrected Errors Rate 0 FEC Uncorrected Errors Rate 0
Change the port configuration on both sides and set the Ethernet FEC Mode to none.
Refer to the technical documentation on FEC Edit Interfaces .
{master:0}[edit] root@QFX5110-48S# set interfaces et-0/0/49 gigether-options fec none {master:0}[edit] root@QFX5110-48S# commit and-quit configuration check succeeds commit complete Exiting configuration mode {master:0}[edit] root@QFX5210-64C# set interfaces et-0/0/40 gigether-options fec none {master:0}[edit] root@QFX5210-64C# commit and-quit configuration check succeeds commit complete Exiting configuration mode {master:0} root@QFX5210-64C> show interfaces terse et-0/0/40 Interface Admin Link Proto Local Remote et-0/0/40 up up {master:0} root@QFX5110-48S> show interfaces terse et-0/0/49 Interface Admin Link Proto Local Remote et-0/0/49 up up