Description

In this article we will discuss an issue occurred during the boot-up of RE-S-2X00x6 routing-engine.

Symptoms

  • During the RE boot-up process your will notice a message with keyword "VMHOST crash"
  • VMHOST is crashed on this routing-engine and kernel loaded at reserved memory hence it wont start vehostd service.
  • If VMHOST crashes continuously it would put the routing-engine in a boot-loop situation.

 

RE boot log:

VMHost crash kernel loaded at reserved memory 

Current Boot: Boot0005* EFI Hard Drive (SFSA050GV3AA2TO-I-LB-24A-JUN)

detected current boot medium.. ssd

Starting file system check on rootfs

fsck from util-linux 2.28.1

e2fsck 1.43 (17-May-2016)

jrootp_P: recovering journal

Setting free inodes count to 103240 (was 103284)

Setting free blocks count to 156016 (was 159189)

jrootp_P: clean, 27832/131072 files, 368272/524288 blocks

fsck from util-linux 2.28.1

e2fsck 1.43 (17-May-2016)

jrootp_P: clean, 27832/131072 files, 3682systemd[1]: [/etc/systemd/system/reboot.target.d/override.conf:2] Failed to parse JobTimeoutSec= parameter, ignoring:

insmod: ERROR: could not load module /lib/modules/4.8.28-rt10-WR9.0.0.24_ovp/extra/uio_fpga.ko: No such file or directory

sh[2858]: 4 logical volume(s) in volume group "jvg_S" now active

sh[2858]: 4 logical volume(s) in volume group "jvg_P" now active

sh[2858]: Reading volume groups from cache.

sh[2858]: Found volume group "jvg_S" using metadata type lvm2

sh[2858]: Found volume group "jvg_P" using metadata type lvm2

sh[2858]: Cleaned RW Root, proceeding...

[FAILED] Failed to start vehostd service.

See 'systemctl status vehostd.service' for details.

[ OK ] Created slice system-failure_handler.slice.

[ OK ] Started Failure Handler Service.

Additional logs that can be seen along with this:

Cannot get driver information: No such device
BUG: unable to handle kernel NULL pointer dereference at (null)
IP: [<0000000066554540>] i40e_remove+0xb1/0x380 [i40e]
PGD 8000001034157067 PUD 1032d02067 PMD 0
Oops: 0000 [#1] SMP
Modules linked in: iTCO_wdt iTCO_vendor_support intel_rapl intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crct10dif_common aesni_intel aes_x86_64 glue_helper lrw gf128mul ablk_helper cryptd i2c_i801 igb lpc_ich e1000e i40e(O-) configfs acpi_power_meter nfsd fuse
CPU: 1 PID: 2383 Comm: rmmod Tainted: G O 4.8.28-rt10-WR9.0.0.24_ovp #1
...
sh[2233]: PF driver reset failed, trying to reinitialize

 

 

Solution

 

Step 1 : Recover RE by booting via SSD2

Recover the RE by booting up again (Use CB online/offline) and use Boot-Option and boot the RE using SSD2.

Once RE is booted via SSD2, recover the SSD1 using "request vmhost snapshot recovery"

Reboot the RE to boot via SSD1. 

 

Step 2: Use USB method to recover the RE.

 

If above steps does not help replace the RE in a MW.

 

Modification History

2024-12-25 : Article Created