CTLE is a parameter for the 100G optic (not applicable for 100G DAC cable). The CTLE value of the 100G optic is based on the lengths of the trace for each front port and the loss. When the 100G optic is inserted into the Junos device, it will be programmed with a value by Junos software based on the port number.
The CTLE value is measured with a DCA-X, OIF compliance software with normative CTLE using an HCB plugged into the system.
In the output of 'show interface et-x/x/x extensive' from QFX5200, some of the ports show FEC Corrected Errors and FEC Uncorrected Errors.
'show interface et-x/x/x extensive'
There is no fiber issue, optic issue, and the light level is normal for each lane.
root@test> show interfaces et-0/0/11 extensive Physical interface: et-0/0/11, Enabled, Physical link is Up Interface index: 658, SNMP ifIndex: 519, Generation: 150 Link-level type: Ethernet, MTU: 9216, LAN-PHY mode, Speed: 100Gbps, BPDU Error: None, Loop Detect PDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, ........................... FEC statistics Errors FEC MODE FEC91 FEC Corrected Errors 233002817 FEC Uncorrected Errors 3
Incorrect CTLE setting for the optic is one of the reasons that cause this issue. QFX5200 may not have been able to program the 100G optics with the correct CTLE value and caused the errors.
This issue has been fixed with PR1389478 - Link problems might occur with 100G-AOC on QFX series . Junos 15.1X53-D236 and higher has the fix.
With QFX5200, each front port has different lengths of trace and loss, so the system has defined different CTLE values for each front port's CTLE.
Here is the list of CTLE values for each port in QFX5200:
0 7 1 6 2 6 3 6 4 6 5 5 6 6 7 5 8 5 9 5 10 7 11 7 12 6 13 5 14 4 15 3 16 4 17 5 18 6 19 6 20 7 21 6 22 5 23 5 24 5 25 5 26 6 27 5 28 6 29 6 30 7 31 7
Procedure to check if the CTLE value is correct:
start shell vty fpc0 show qsfp list test qsfp <index> write 127 3 test qsfp <index> read 234 test qsfp <index> read 235
root@test> start shell root@test:~ # vty fpc0 TOR platform (1799 Mhz Pentium processor, 383MB memory, 0KB flash) TFXPC0(vty)# show qsfp list QSFP Toolkit summary: wakeup count: 3148779, debug: 0 thread: 0xbf689200, itable: 0xbf676da8 Index Name State ----- ---- ----- 1 qsfp-0/0/0 Present 3 qsfp-0/0/2 Present 5 qsfp-0/0/1 Present 6 qsfp-0/0/3 Present 7 qsfp-0/0/10 Present 8 qsfp-0/0/11 Present TFXPC0(vty)# test qsfp 7 write 127 3 TFXPC0(vty)# test qsfp 7 read 234 234: 77 <-- 77 both means the lane0 and lane 1 with CTLE value 7 TFXPC0(vty)# test qsfp 7 read 235 235: 77 <-- 77 both means the lane2 and lane 3 with CTLE value 7
Procedure to change the CTLE value of the 100G optics: Note: Changing the CTLE value will not impact service and it will take affect immediately.
start shell vty fpc0 show qsfp list test qsfp <index> write 127 3 test qsfp <index> write 234 77 <-- This command to set the CTLE of the 100G optics as 7 for both lane0 and lane1 test qsfp <index> write 235 77 <-- This command to set the CTLE of the 100G optics as 7 for both lane2 and lane2
root@test> start shell root@test:~ # vty fpc0 TOR platform (1799 Mhz Pentium processor, 383MB memory, 0KB flash) TFXPC0(vty)# show qsfp list QSFP Toolkit summary: wakeup count: 3148779, debug: 0 thread: 0xbf689200, itable: 0xbf676da8 Index Name State ----- ---- ----- 1 qsfp-0/0/0 Present 3 qsfp-0/0/2 Present 5 qsfp-0/0/1 Present 6 qsfp-0/0/3 Present 7 qsfp-0/0/10 Present 8 qsfp-0/0/11 Present TFXPC0(vty)# test qsfp 7 write 127 3 TFXPC0(vty)# test qsfp 7 write 234 77 TFXPC0(vty)# test qsfp 7 write 235 77