Sometimes a QFX device crashes with no logs available during the time of the issue. This is applicable to standalone as well a VC setup.

Switch going down without any log messages reported in varlogs or hostshell logs and starts logging only after the switch is UP by a power cycle.

For eg:
Customer reported the switch went down around 11:03 on Feb 15. Same can be confirmed from the other VC member where it lost connectivity with its member.

• If it is a VC setup, logs on it's VC member confirms that it lost VC membership at 11:03

Feb 15 11:03:11 2024  /kernel: tcp_timer_keep: Dropping socket connection due to keepalive timer expiration, idle/intvl/cnt: 2000/2000/20

Feb 15 11:03:11 2024  /kernel: tcp_timer_keep:Local(0x80000001:6011) Foreign(0x80000011:65429)

Feb 15 11:03:27 2024 vccpd[2023]: VCCPD_PROTOCOL_ADJDOWN: Lost adjacency to c8e7.f0ab.d6c0 on vcp-255/0/99.32768, reason: Aged out

• Varlogs on the problematic device does not show any activities during that time

Feb 15 11:01:42 2024  ksyncd[2264]: mchassis_re_command_execute_status_by_iri:{errno=2,pid=60300,status=256} Child did not exit cleanly. Remote command failed

Feb 15 14:14:10 2024 eventd: sendto: No route to host

Feb 15 14:14:10 2024  eventd[1510]: SYSTEM_ABNORMAL_SHUTDOWN: System abnormally shut down

• Even hostshell logs does not report anything when the device went down.

Feb 15 11:02:21 localhost vehostd[7086]: vjunos_vm_state_transition Calling get_next_state for vjuons vjunos0 with current_state=3 current_ret=0

Feb 15 11:02:21 localhost vehostd[7086]: vjunos_get_next_state vjunos transition to next state 3

Feb 15 14:11:53 localhost kernel: imklog 5.8.10, log source = /proc/kmsg started.

Feb 15 14:11:53 localhost rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="3674" x-info="http://www.rsyslog.com"] start

Feb 15 14:11:53 localhost kernel: Initializing cgroup subsys cpuset

Feb 15 14:11:53 localhost kernel: Initializing cgroup subsys cpu

1. Device went down unexpectedly and remains down
   • To recover, power cycle the device. If power cycle does not bring up the device back, then it is a permanent hardware failure and needs to be replaced.
   • If the power cycle restores the device, it could be considered as a transient hardware issue.If the issue recurs, then it is recommended to replace the chassis.
   • In this situation, since the device stopped logging, it is suspected that the device might have run into some transient hardware (could be SSD related) issue and hence went unresponsive.
2. If the device reboots on its own, then it could be due to Watchdog timeout. Check for the last reboot reason in "show chassis routing-engine" output.

2024-02-19 : Article Created
2024-07-26 : Added a scenario of chassis rebooting with no logging.

2025-04-28 : added ex4400 to categories as same symptoms were seen.

2026-06-04: Added Internal comment with dev feedback and PR#