Description

This article explains the issue with EX4300 standalone or Virtual Chassis where the device becomes in accessible and in Not Prsnt State without any logs.

Symptoms

  1. The device goes down without any logging or device stops logging on the local member.
  2. The device is recovered only with a power cycle.

Following logs can be observed while the device is booting up or in the file ">show /var/run/dmesg.boot.detail"

 

Automatic reboot in progress...

** Last Mounted on /instrootmnt

** Root file system

** Phase 1 - Check Blocks and Sizes

** Phase 2 - Check Pathnames

** Phase 3 - Check Connectivity

** Phase 4 - Check Reference Counts

UNREF FILE I=31720 OWNER=0 MODE=120755

SIZE=42 MTIME=Jul 17 04:49 2023

CLEAR? yes

 

UNREF FILE I=32151 OWNER=0 MODE=120755

SIZE=43 MTIME=Jul 17 04:49 2023

CLEAR? yes

 

** Phase 5 - Check Cyl groups

FREE BLK COUNT(S) WRONG IN SUPERBLK

SALVAGE? yes

 

SUMMARY INFORMATION BAD

SALVAGE? yes

 

BLK(S) MISSING IN BIT MAPS

SALVAGE? yes

 

4624 files, 149694 used, 12243 free (11 frags, 1529 blocks, 0.0% fragmentation)

 

***** FILE SYSTEM WAS MODIFIED *****

RE-CHECKING FILE SYSTEM

NO WRITE ACCESS

UNEXPECTED SOFT UPDATE INCONSISTENCY. >>>>>>>>>File System showing errors

(NO WRITE)** Last Mounted on /instrootmnt

** Root file system

** Phase 1 - Check Blocks and Sizes

** Phase 2 - Check Pathnames

** Phase 3 - Check Connectivity

** Phase 4 - Check Reference Counts

** Phase 5 - Check Cyl groups

4624 files, 149694 used, 12243 free (11 frags, 1529 blocks, 0.0% fragmentation)

** Last Mounted on /var

** Phase 1 - Check Blocks and Sizes

INCORRECT BLOCK COUNT I=5802 (8 should be 0)

CORRECT? yes

 

INCORRECT BLOCK COUNT I=11417 (148 should be 144). >>>>>>>>Device correcting the file system errors on power cycle

CORRECT? yes

 

** Phase 2 - Check Pathnames

** Phase 3 - Check Connectivity

UNREF DIR I=5870 OWNER=0 MODE=40750

SIZE=512 MTIME=Aug 9 20:40 2023

RECONNECT? yes

 

DIR I=5870 CONNECTED. PARENT WAS I=5641

 

UNREF DIR I=5841 OWNER=0 MODE=40750

SIZE=512 MTIME=Aug 9 20:40 2023

RECONNECT? yes

 

DIR I=5841 CONNECTED. PARENT WAS I=5641

 

UNREF DIR I=5753 OWNER=0 MODE=40750

SIZE=512 MTIME=Aug 9 20:40 2023

RECONNECT? yes

 

DIR I=5753 CONNECTED. PARENT WAS I=5641

 

** Phase 4 - Check Reference Counts

UNREF FILE I=84 OWNER=0 MODE=140700

SIZE=0 MTIME=Aug 9 20:42 2023

RECONNECT? yes

 

UNREF FILE I=91 OWNER=0 MODE=140700

SIZE=0 MTIME=Aug 9 20:42 2023

RECONNECT? yes

 

UNREF FILE I=94 OWNER=0 MODE=140700

SIZE=0 MTIME=Aug 9 20:42 2023

RECONNECT? yes

 

UNREF FILE I=99 OWNER=0 MODE=140700

SIZE=0 MTIME=Aug 9 20:42 2023

RECONNECT? yes

 

UNREF FILE I=420 OWNER=0 MODE=140755

SIZE=0 MTIME=Jul 26 09:12 2023

CLEAR? yes

 

UNREF FILE I=428 OWNER=0 MODE=140755

SIZE=0 MTIME=Jul 26 09:12 2023

CLEAR? yes

 

UNREF FILE I=437 OWNER=0 MODE=140755

SIZE=0 MTIME=Jul 26 09:12 2023

CLEAR? yes

 

Computing slice and partition sizes for /dev/da0 ...

savecore: Reboot reason(s): 0x1: power cycle/failure

UNREF FILE I=470 OWNER=0 MODE=140755

SIZE=0 MTIME=Jul 26 09:12 2023

CLEAR? yes

 

UNREF FILE I=473 OWNER=0 MODE=140755

SIZE=0 MTIME=Jul 26 09:12 2023

CLEAR? yes

 

LINK COUNT DIR I=5641 OWNER=0 MODE=40750

SIZE=512 MTIME=Aug 9 20:40 2023 COUNT 5 SHOULD BE 2

ADJUST? yes

 

UNREF FILE I=11284 OWNER=0 MODE=100640

SIZE=3000090 MTIME=Jul 27 18:09 2023

CLEAR? yes

 

UNREF FILE I=11296 OWNER=0 MODE=100644

SIZE=2999950 MTIME=Jul 27 03:28 2023

CLEAR? yes

 

UNREF FILE I=11332 OWNER=0 MODE=100640

SIZE=10761901 MTIME=Aug 6 01:10 2023

CLEAR? yes

 

** Phase 5 - Check Cyl groups

FREE BLK COUNT(S) WRONG IN SUPERBLK

SALVAGE? yes

 

SUMMARY INFORMATION BAD

SALVAGE? yes

 

BLK(S) MISSING IN BIT MAPS

SALVAGE? yes

 

981 files, 15714 used, 71162 free (450 frags, 8839 blocks, 0.5% fragmentation)

 

***** FILE SYSTEM MARKED CLEAN *****

 

***** FILE SYSTEM WAS MODIFIED *****

RE-CHECKING FILE SYSTEM

** Last Mounted on /var

** Phase 1 - Check Blocks and Sizes

** Phase 2 - Check Pathnames

** Phase 3 - Check Connectivity

** Phase 4 - Check Reference Counts

** Phase 5 - Check Cyl groups

981 files, 15714 used, 71162 free (450 frags, 8839 blocks, 0.5% fragmentation)

checking for core dump...

No dump exists

** Last Mounted on /var/tmp

** Phase 1 - Check Blocks and Sizes

** Phase 2 - Check Pathnames

** Phase 3 - Check Connectivity

** Phase 4 - Check Reference Counts

** Phase 5 - Check Cyl groups

20 files, 51001 used, 403354 free (34 frags, 50415 blocks, 0.0% fragmentation)

 

***** FILE SYSTEM MARKED CLEAN *****

** Last Mounted on /config

** Phase 1 - Check Blocks and Sizes

INCORRECT BLOCK COUNT I=7691 (4 should be 0)

CORRECT? yes

 

** Phase 2 - Check Pathnames

** Phase 3 - Check Connectivity

** Phase 4 - Check Reference Counts

** Phase 5 - Check Cyl groups

FREE BLK COUNT(S) WRONG IN SUPERBLK

SALVAGE? yes

 

SUMMARY INFORMATION BAD

Solution

The device can be recovered by doing a Power cycle/reboot. Once the device is rebooted the bad blocks are corrected and device comes back up.

Modification History

10/01/2024 - Added QFX5100 into categories