Description

This document explains the reason for the appearance of minor alarm, 'RE x Mismatch in total memory detected' on devices that have Next Generation REs installed on them. By the end of this document, the reader will be aware of the steps to follow in case one encounters such an error.

Symptoms

Error Logs:

Feb 22 11:08:28   chassisd[5134]: re_rainier_perform_misc_checks:RE memory size mismatch expected 65536 M , actual 0 M
Feb 22 11:09:03   alarmd[5451]: Alarm set: RE color=YELLOW, class=CHASSIS, reason=RE 1 Mismatch in total memory detected
Feb 22 11:09:03   alarmd[5451]: Alarm set: RE color=YELLOW, class=CHASSIS, reason=RE 0 Mismatch in total memory detected

On the device, active alarms are displayed when the system alarms command is executed:

>show system alarms 
2 alarms currently active
Alarm time               Class  Description
2020-02-22 11:09:03 UTC  Minor  RE 1 Mismatch in total memory detected
2020-02-22 11:09:03 UTC  Minor  RE 0 Mismatch in total memory detected

Solution

This behavior is seen when one or more of the following conditions are met:

  1. VM Host requires an upgrade
  2. Transient hardware error
  3. False alarm
  4. RE faulty

Verify if the device is single RE or dual RE:

user@host> show chassis routing-engine

If the above command displays Primary RE and Backup RE, then verify which RE is throwing the error. In case the error is seen on the Primary RE, then perform a primary role switch for further troubleshooting.

Perform Primary Role Switch:

Note:  Please perform this during a maintenance window.

  1. Connect to primary RE
  2. Perform a primary switch and check if the alarm is cleared

user@host> request chassis routing-engine master switch

 

Verify the RE Type: 

user@host> show chassis hardware detail
<snip>
Routing Engine 1 REV 16   7xxxxxxxxx   xxxxxxxx    RE-S-2X00x6     <-- Next Generation RE


On routers with NG-RE installed, during router bootup time - chassisd init time, chassisd may fail to read DIMM size from host.  As a result, the memory mismatch error may be triggered. This can be a transient error seen once in a while during bootup. Due to broken host communication, Junos may fail to get host memory size and generate a false alarm.

Verify if there are any RE errors:

user@host> show chassis routing-engine errors

The 'show chassis routing-engine errors ' is part of a new error Reporting and Resiliency framework that enhances the debugging capability of hardware failures. It provides error reporting on I2C/SMBUS DEVICES, Power failure/errors, Voltage monitoring, Temperature Sensors and PCI-E among others.

 

Restart/Reboot the RE:

user@host> request system reboot

Note: Be sure to reboot the RE that is currently the backup.

Restart/Reboot vmhost:

user@host> request vmhost reboot  <re0 | re1> <routing engine>

In case the above does not clear the alarm, check if the Junos OS and vmhost versions are on the latest recommended releases. 

Routing Engine Upgrade:

For detailed upgrade procedure, refer to KB8911 - Procedure to upgrade JUNOS on a Dual Routing Engine System [juniper.net]

For installing the host software as well as Junos OS, specify the package name  junos-vmhost-install-x.tgz  in the  request vmhost software add  command. Junos OS installation alone can be achieved by specifying the regular package name  junos-install-x.tgz   in the  request system software add  command. However, installation using the vmhost package is recommended as it upgrades both the host software and Junos OS

VM Host Upgrade:

Copy image   junos-vmhost-install-x.tgz to /var/tmp directory. You can navigate to downloads site to find the image: select OS—Junos, Version -18.3----Install Package (Vmhost 64 bit)

user@host> request vmhost software add <package name>

Reboot:

user@host> request system reboot

In case the above does not work, perform USB upgrade as below:

Vmhost upgrade can be done using image “ junos-vmhost-install-usb ” format and it should solve the issue. In the case of USB upgrade please be mindful of the fact that all config and log files would be lost. 

USB installation will install both Linux and Junos OS. This installation will erase all files in Linux and Junos. Create back up files if needed before proceeding with the installation. For USB installation, you will need a USB with at least 8GB storage.

  1. Copy junos-vmhost-install-usb-x.img.gz  to RE on /var/tmp   (On downloads site: select OS—Junos, Version -18.3, Install Media--- VMHost USB Installer 64-Bit)

  2. Unzip the copied file in RE in shell mode.   % gunzip junos-vmhost-install-usb-x.img.gz  

  3. Write the above image to USB via the dd command ( dd if=/path/to/downloaded.img of=/dev/devicenode bs=4M ) from BSD shell prompt as the root user

  4. Insert USB on RE and reboot via CLI " request vmhost reboot " or power cycle RE.

  5. Press Y to install image from USB to SSDs. After installation is completed, Press Y again to reboot the router.

  6. For any reason, if the user key is in a different character, then it will fall to the shell.
    The user is requested to do an exit from the shell in order for the installation to proceed. When prompted to install, press Y to continue. Once completed, press Y again to reboot the router.

  7. After successful Junos OS boot, remove the USB from RE. Don’t remove USB in between, as it might corrupt file systems of USB. 

Note: Ensure console connection to the device before performing upgrade as backup access. 

If the steps above do not resolve the issue, contact your Support Representative for further investigation.

***WARNING: Please take adequate measures while restarting or upgrading RE as this could be traffic impacting if mishandled. In case you need assistance during the activity, please contact Support .

 

Modification History

2021-03-25: Updated the article terminology to align with Juniper's Inclusion & Diversity initiatives
2022-09-01: Article reviewed for accuracy; no changes required; article valid and relevant
 

Related Information