Note: A product listed in this article has either reached hardware End of Life (EOL) OR software End of Engineering (EOE). Refer to End of Life Products & Milestones for the EOL, EOE, and End of Support (EOS) dates.
When two SRX nodes have a fabric link between them, which is connected through a Cisco 4500/6500 switch, the secondary node is always going to disabled state. This problem may occur in Junos OS version 10.0 or earlier.
In the following topology:
SRX ---fab0----------------------Cisco6500----------------------fab1---SRX
Secondary SRX is always going to disabled state. When you check " show chassis cluster statistic " you see that no probes have been received on both nodes:
Fabric link statistics: Probes sent: 944 Probes received: 0
Both fabric links are physically UP and they are connected in the same VLAN on the switch as only two ports members of that VLAN. If you configure fabric link interfaces as normal L3 interfaces, traffic is flowing between two nodes without any problems
By default, Cisco 4500/6500 switches are performing IP header checksum in multilayer switching.
Fabric link probes have following format (as displayed in Wireshark):
In Junos OS version 10.0 and earlier, the fabric link probes are using Juniper proprietary IP datagrams, where IP Total Length field is set to 0 (zero). Wireshark will always assume that this is a faulty IP datagram and will not read other fields. Other network equipment might do the same as IP header can be reported as bogus.
This is the error that you might see on the Cisco 4500/6500 switch:
%MLS_STAT-SP-4-IP_CSUM_ERR: IP checksum errors
Also, when you check switch ports that are connected to SRX fabric ports, you will see only incoming packets (packets received from SRX), but no outgoing packets (fabric link probes that should be switched from one SRX to another)
Switch#show interfaces ge0/22 GigabitEthernet0/22 is up, line protocol is up Hardware is GigabitEthernet, address is 0007.abcd.abcd (bia 0007.abcd.abcd) MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA, loopback not set Keepalive not set Auto-duplex (Full), Auto Speed (1000), 1000BaseTX/FX ARP type: ARPA, ARP Timeout 04:00:00 Last input never, output 00:00:00, output hang never Last clearing of "show interface" counters 2d18h Queueing strategy: fifo Output queue 0/40, 0 drops; input queue 0/75, 0 drops 5 minute input rate 9000 bits/sec, 1 packets/sec 5 minute output rate 0 bits/sec, 0 packets/sec 5021 packets input, 6493552 bytes Received 0 broadcasts, 0 runts, 0 giants, 0 throttles 1 input errors, 0 CRC, 1 frame, 0 overrun, 0 ignored 0 watchdog, 0 multicast 0 input packets with dribble condition detected 0 packets output, 0 bytes, 0 underruns 0 output errors, 0 collisions, 0 interface resets 0 babbles, 0 late collision, 0 deferred 0 lost carrier, 0 no carrier
Important: If you see the output packet counter increasing, and the problem is still present, check if your port is running STP BPDU hello messages and/or DTP probes (if port is dynamically access and not static access port).
To resolve this issue, disable multilayer switching IP header checksum:
Switch(config)#no mls verify ip checksum
config)#no mls verify ip checksum
After this command is enabled, output counter for both ports on switch will start incrementing, and SRX devices will start receiving fabric link probes.
2021-07-16: Tagged for EOL/EOE; issue not seen in latest code. Content is still relevant.