QSFP56-DD-400G-ZR-M-HP channelized to 100G, link not up when added to AE bundle
ACX7322 configured for channelized 400g -> 100G used in an AE interface. It is seen that if the interfaces are defined first then committed, then add the chassis stanza for aggregation-device ethernet device count, the physical links may never come up. No TX is seen at the optic level. At this point, the ACX needs to be rebooted.
The issue is seen on PTX10k1-36mr as well with the same optics. Seems to be an EVO generic issue.
Topology:
PTX10001-36mr-et-0/0/0:x------ae0----------et-1/0/0---acx7332
Example:
Configure interfaces:
set interfaces et-1/0/0 description "400G test port"
set interfaces et-1/0/0 number-of-sub-ports 3
set interfaces et-1/0/0 speed 100g
set interfaces et-1/0/0:0 description test
set interfaces et-1/0/0:0 ether-options 802.3ad ae0
set interfaces et-1/0/0:0 optics-options wavelength 1561.42
set interfaces et-1/0/0:0 optics-options tx-power -10
set interfaces et-1/0/0:1 description test
set interfaces et-1/0/0:1 ether-options 802.3ad ae0
set interfaces et-1/0/0:2 ether-options 802.3ad ae0
set interfaces et-1/0/1 unused
set interfaces et-1/0/3 unused
set interfaces et-1/0/4 unused
set interfaces et-1/0/5 unused
set interfaces ae0 aggregated-ether-options minimum-links 1
set interfaces ae0 aggregated-ether-options link-speed 100g
set interfaces ae0 aggregated-ether-options lacp active
Commit:
Step 2:
Add chassis stanza
set chassis aggregated-devices ethernet device-count 5
commit
Physical links will never come up.
# run show interfaces ae0
Physical interface: ae0, Enabled, Physical link is Down
Interface index: 1271, SNMP ifIndex: 784
Link-level type: Ethernet, MTU: 1514, Speed: Unspecified, BPDU Error: None,
Ethernet-Switching Error: None, MAC-REWRITE Error: None, Loopback: Disabled,
Source filtering: Disabled, Flow control: Enabled, Minimum links needed: 1,
Minimum bandwidth needed: 1bps
Device flags : Present Running
Interface flags: Hardware-Down SNMP-Traps
Current address: 20:ed:47:ef:23:67, Hardware address: 20:ed:47:ef:23:67
Last flapped : 2024-09-05 12:26:25 PDT (00:59:27 ago)
Input rate : 0 bps (0 pps)
Output rate : 0 bps (0 pps)
Members:
Statistics Packets pps Bytes bps
et-1/0/0:0 <-- down
Input : 0 0 0 0
Output: 0 0 0 0
et-1/0/0:1 <-- down
et-1/0/0:2 <-- down
# run show interfaces diagnostics optics et-1/0/0:0
Physical interface: et-1/0/0:0
Module temperature : 45 degrees C / 113 degrees F
Module voltage : 3.316 V
Module max power : 23 W
Wavelength channel number : 42
Wavelength setpoint : 1561.42 nm
Tx dither : Disabled
Frequency error : 0.00 GHz
Wavelength error : 0 nm
TEC fault alarm : False
Wavelength unlocked alarm : True
Tx tune alarm : False
Module temperature high alarm : Off
Module temperature low alarm : Off
Module temperature high warning : Off
Module temperature low warning : Off
Module voltage high alarm : Off
Module voltage low alarm : Off
Module voltage high warning : Off
Module voltage low warning : Off
Module temperature high alarm threshold : 76 degrees C / 168 degrees F
Module temperature low alarm threshold : -11 degrees C / 12 degrees F
Module temperature high warning threshold : 73 degrees C / 163 degrees F
Module temperature low warning threshold : -3 degrees C / 26 degrees F
Module voltage high alarm threshold : 3.7 V
Module voltage low alarm threshold : 3 V
Module voltage high warning threshold : 3.599 V
Module voltage low warning threshold : 3.049 V
Laser bias current high alarm threshold : Not supported
Laser bias current low alarm threshold : Not supported
Laser bias current high warning threshold : Not supported
Laser bias current low warning threshold : Not supported
Laser output power high alarm threshold : 3.98 mW / 5.99 dBm
Laser output power low alarm threshold : 0.015 mW / -18.01 dBm
Laser output power high warning threshold : 2.511 mW / 3.99 dBm
Laser output power low warning threshold : 0.025 mW / -16.00 dBm
Laser rx power high alarm threshold : 6.309 mW / 7.99 dBm
Laser rx power low alarm threshold : 0 mW / -32.21 dBm
Laser rx power high warning threshold : 3.98 mW / 5.99 dBm
Laser rx power low warning threshold : 0.001 mW / -28.53 dBm
Laser temperature high alarm threshold : 80 degrees C / 176 degrees F
Laser temperature low alarm threshold : -80 degrees C / -112 degrees F
Laser temperature high warning threshold : 75 degrees C / 167 degrees F
Laser temperature low warning threshold : -40 degrees C / -40 degrees F
Lane 0
Laser bias current : 0 mA
Laser output power : 0 mW / -40.00 dBm
Laser temperature : 0 degrees C / 32 degrees F
Laser receiver power : 0 mW / -40.00 dBm
Rx power (signal) : 0.00 mW / -40.00 dBm
Lane chromatic dispersion : 0.0 ps/nm
Lane differential group delay : 0.0 ps
Lane carrier frequency offset : 0.0 MHz
Lane polarization dependent loss : 0.0 dB
Lane snr : 0.0 dB
Lane Optical signal-to-noise ratio : 0.0 dB
Lane sopmd : 0.0
Laser bias current high alarm : Off
Laser bias current low alarm : Off
Laser bias current high warning : Off
Laser bias current low warning : Off
Laser temperature high alarm : Off
Laser temperature low alarm : Off
Laser temperature high warning : Off
Laser temperature low warning : Off
Laser receiver power high alarm : Off
Laser receiver power low alarm : Off
Laser receiver power high warning : Off
Laser receiver power low warning : Off
Laser output power high alarm : Off
Laser output power low alarm : On
Laser output power high warning : Off
Laser output power low warning : On
Tx loss of signal functionality alarm : Off
Rx loss of signal alarm : Off
Tx laser disabled alarm : On
channel-1/0/0:0 100G Chan_Online Tune_Ok up down down down 0 81
channel-1/0/0:1 100G Chan_Online Tune_Ok up down down down 0 83
channel-1/0/0:2 100G Chan_Online Tune_Ok up down down down 0 85
# run request system reboot
The entire system (all nodes) will reboot causing traffic loss, do you wish to continue? [yes,no] (no) yes
*** System shutdown message from root@posh-re0 ***
reboot the system at Thu Sep 5 13:27:58 2024
.
Last login: Thu Sep 5 09:21:06 2024 from 172.29.135.101
--- JUNOS 24.2R1.18-EVO Linux (none) 5.15.76-yocto-standard-juniper-02085-gce2d521ddcf4 #1 SMP PREEMPT Mon May 27 05:52:24 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
> show interfaces ae0
Physical interface: ae0, Enabled, Physical link is Up
Interface index: 1215, SNMP ifIndex: 784
Link-level type: Ethernet, MTU: 1514, Speed: 300Gbps, BPDU Error: None, Ethernet-Switching Error: None, MAC-REWRITE Error: None, Loopback: Disabled,
Source filtering: Disabled, Flow control: Enabled, Minimum links needed: 1, Minimum bandwidth needed: 1bps
Interface flags: SNMP-Traps
Last flapped : 2024-09-05 13:34:53 PDT (00:00:59 ago)
et-1/0/0:0
et-1/0/0:1
et-1/0/0:2
{master}
> show interfaces diagnostics optics et-1/0/0:0
Module temperature : 54 degrees C / 129 degrees F
Module voltage : 3.255 V
Wavelength unlocked alarm : False
Laser bias current : 290 mA
Laser output power : 0.1 mW / -9.99 dBm
Laser temperature : 53 degrees C / 127 degrees F
Laser receiver power : 0.064 mW / -11.87 dBm
Rx power (signal) : 0.07 mW / -11.67 dBm
Lane chromatic dispersion : 1.0 ps/nm
Lane differential group delay : 1.0 ps
Lane carrier frequency offset : 200.0 MHz
Lane polarization dependent loss : 0.4 dB
Lane snr : 17.1 dB
Lane Optical signal-to-noise ratio : 35.7 dB
Lane sopmd : 42.0
Laser output power low alarm : Off
Laser output power low warning : Off
Tx laser disabled alarm : Off
Resolved in PR1688222 (https://prsearch.juniper.net/problemreport/PR1688222)
evo:24.4R1-EVO
evo:25.1R1-EVO
junos:25.1R1