Model: EX4300-48mp Junos: 20.4R3.8
This KB article provides a solution for the failure and unexpected crash of the chassis daemon on the EX4300-48MP platform that occurs after performing an interface Time Domain Reflectometer [TDR] test.
The CLI crash issue is observed with a very specific configuration, namely the interface speed. This issue is unrelated to faulty cables or frame errors, as these are not incrementing. This problem occurs in JUNOS version 20.4R3.8 when the interface speed is configured to 100 Mbps. In this scenario, a camera connected at the other end supports a maximum speed of 100 Mbps. The link operates at this speed. Regarding the TDR CLI commands: when "request diagnostics tdr start interface <if-name>" is executed, it collects the information as expected. However, when "show diagnostics tdr interface <if-name>" is executed, the Chassisd crashes. This is because, at 1000 Mbps (1 Gbps), all 4 MDI pairs are in use, but at 100 Mbps, only 2 MDI pairs are active. As a result, we observe: MDI Pair 1-2: Cable Status is Unknown MDI Pair 3-6: Cable Status is Normal MDI Pair 4-5: Cable Status is Normal For MDI Pair 7-8, the Cable Status is set to NULL, but the system attempts to print data from this invalid address instead of handling it as NULL. This causes the Chassisd to crash due to an unexpected segmentation fault. > show diagnostics tdr interface ge-1/0/3
Interface TDR detail:
Interface name : ge-1/0/3
Test status : Passed
Link status : Down
MDI pair : 1-2
Cable status : Unknown
Cable length/Distance To Fault: 0 Meters
MDI pair : 3-6
Cable status : Normal
MDI pair : 4-5
MDI pair : 7-8
error: abnormal communication termination with chassis-control daemon
++ We have observed crash message logs.
Message from switch1xyz at May 29 12:30:52 ...
switch1xyz fpc0 CMLC: Going disconnected; Routing engine chassis socket closed abruptly
Message from syslogd@Switch at May 29 12:30:52 ...
Switch fpc1 CMLC: Going disconnected; Routing engine chassis socket closed abruptly
Message from syslogd@Switch at May 29 12:30:58 ...
Switch fpc1 SCHED: Thread 37 (PFE Manager) ran for 1088 ms without yielding
Switch fpc1 Scheduler Oinker
Switch fpc1 Frame 0: sp = 0xd93c7248, pc = 0x8068353
Switch fpc1 Frame 1: sp = 0xd93c7268, pc = 0x8db5706
Switch fpc1 Frame 2: sp = 0xd93c7288, pc = 0x828eee0
Switch fpc1 Frame 3: sp = 0xd93c72e8, pc = 0x80686f0
Switch fpc1 SCHED: Thread 37 (PFE Manager) ran for 1195 ms without yielding
Switch fpc0 SCHED: Thread 37 (PFE Manager) ran for 1158 ms without yielding
Switch fpc0 Scheduler Oinker
Switch fpc0 Frame 0: sp = 0xd93f3e18, pc = 0x8068353
Switch fpc0 Frame 1: sp = 0xd93f3e38, pc = 0x8db5706
Switch fpc0 Frame 2: sp = 0xd93f3e58, pc = 0x828eee0
Switch fpc0 Frame 3: sp = 0xd93f3eb8, pc = 0x80686f0
Switch fpc0 SCHED: Thread 37 (PFE Manager) ran for 1207 ms without yielding
++ Core files are generated.
user@device> show system core-dumps no-forwarding
-rw-rw---- 1 root wheel 2898628 May 29 11:11 /var/tmp/chassisd.core-tarball.0.tgz
-rw-rw---- 1 root wheel 2881612 May 29 11:12 /var/tmp/chassisd.core-tarball.1.tgz
-rw-rw---- 1 root wheel 2883730 May 29 11:13 /var/tmp/chassisd.core-tarball.2.tgz
-rw-rw---- 1 root wheel 2892651 May 29 11:18 /var/tmp/chassisd.core-tarball.3.tgz
-rw-rw---- 1 root wheel 2894349 May 29 12:31 /var/tmp/chassisd.core-tarball.4.tgz
total files: 5
Host Crash Info:
------------------------------------------------------
No core files present on the host.
++ No Alarms are active.
user@device> show chassis alarms no-forwarding
No alarms currently active
We found that the "show diagnostics tdr interface <if-name>", which will be executed as a follow up CLI after executing "request diagnostics tdr start interface <if-name>", was crashing in Junos: 21.1R1.11, and same issue disappeared in Junos: 21.2R1.10.
user@device> show diagnostics tdr interface ge-0/0/4
Interface name : ge-0/0/4
Test status : Started
{master:0}
Link status : UP
Cable status : ?D$\x0w\x01
Cable length/Distance To Fault: 128 Meters
Polartiy swap : N/A
Pair swap : N/A
Downshift : N/A
2024-09-16 : Article Created