Description

This article details the unexpected master switchover that resulted from taking a snapshot.

Symptoms

user@re0> request system snapshot recovery routing-engine 
re0:
--------------------------------------------------------------------------
Creating image ...
Compressing image ...
Image size is 2966MB
mount: /dev/gpt/oam: R/W mount of /.mount/oam denied. Filesystem is not clean - run fsck.: Operation not permitted
umount: /mnt: not a file system root directory
ERROR: The OAM volume is too small to store a snapshot.     <<<<<<<<<<<<<<< Insufficient memory


++ The OAM volume has insufficient space, so we perform a recovery on OAM-volume, but it failed with the error oam-volume not present.

user@re0> request system recover oam-volume    
NOTICE: Recovering the OAM volume ...
warning: unable to create volume: oam
warning: the storage device that holds it is not present
ERROR: oam recovery error: failed to create oam-volume

Solution

Please run "show chassis hardware detail | grep Routing" and verify the status of ada1. 

Routing Engine 0 REV 07   xxxxxxxx   xxxxxxxx        RE-S-1800x4
  ada0  30533 MB  ATP 1.8" SSD xxxxxxxx xxxxx            Disk 1 
  usb0 (addr 0.1) EHCI root HUB 0      Intel             uhub0

As shown from the above output, ada1 (OAM volume) is not detected in the system. Please perform a reload/reseat on the affected RE.

The unexpected switchover happened due to "failover on-disk-failure" command under configuration. This instructs the backup router to take mastership if it detects hard disk errors on the master routing engine (RE).

user.re0> show configuration 

chassis {
    redundancy {
        failover {
            on-disk-failure;   <<<<<<<<<<<< 


NOTE:

 
ada1: This designation is commonly used for the secondary storage device, which houses the OAM volume. The OAM volume is used for various operational and administrative tasks, including:

  • Logs: Storing system logs and trace files.
  • Snapshots: Saving configuration snapshots.
  • Rescue Configuration: Hold the rescue configuration.  
  • Other operational files: Supporting various management functions.

If the issue persists, please open a case with JTAC Support.

 

Modification History

2025-04-07 : Article Created