Description

Note: this article describes how to extend disk space in Apstra versions *prior* to 4.1.2. From 4.1.2 onward the aos_extend_disk utility can be used. Refer to the Installation and Upgrade guide for details.

When using any supported version of Juniper Apstra, you may be filling up the available space in the default Juniper Apstra server "root" disk partition by uploading several OS images to it. The uploaded OS images will then share the Juniper Apstra server "root" disk partition along with other functions. However, filling up the disk partition may have an impact on the operation of the server.

This article details how to check for issues with the Juniper Apstra server "root" disk partition utilization, and provides methods to resolve any problems thus identified, including adding space to the Juniper Apstra server "root" disk partition.

Symptoms

In general, the Juniper Apstra server manages the utilization of disk partitions on the Juniper Apstra server virtual machine (VM). The VM's disk partitions are sized for normal operations but not sized to hold too many uploaded OS images. Depending on the size of the image, as few as four uploaded images may cause disk utilization issues.

Therefore, if you are unable to upload an OS image, you can check the status of the disk partitions.

alt

From the Apstra UI, go to Platform > AOS/Apstra Cluster and click the controller node. Any errors will be shown. Then check the disk usage for the aos--server--vg-root partition.

From the Juniper Apstra server Linux CLI, use the df -m command and check the Use  percentage for the /dev/mapper/aos--server--vg-root partition.

admin@aos-server:~$ df -m
Filesystem                                 1M-blocks  Used Available Use% Mounted on
udev                                            7997     0      7997   0% /dev
tmpfs                                           1604    10      1595   1% /run
/dev/mapper/aos--server--vg-root               11224 10319       316  98% /
tmpfs                                           8020     0      8020   0% /dev/shm
tmpfs                                              5     0         5   0% /run/lock
tmpfs                                           8020     0      8020   0% /sys/fs/cgroup
/dev/mapper/aos--server--vg-var                27153  2527     23225  10% /var
/dev/sda1                                       1890   118      1659   7% /boot
/dev/mapper/aos--server--vg-var+log            13150   204     12258   2% /var/log
/dev/mapper/aos--server--vg-var+lib+aos+db     22940    52     21701   1% /var/lib/aos/db
/dev/sr0                                           1     1         0 100% /media/cdrom
tmpfs                                           1604     0      1604   0% /run/user/1001
admin@aos-server:~$

We see many cases where the VM has multiple backup files. Those should be copied to external server then deleted to reduce the disk utilization. 

admin@aos-server:~$ sudo du /var/lib/aos/snapshot/
1387520  /var/lib/aos/snapshot/2024-10-21_00_01_23
1387526  /var/lib/aos/snapshot/2024-10-22_00_01_18
1387531  /var/lib/aos/snapshot/2024-10-23_00_01_22
1387538  /var/lib/aos/snapshot/2024-10-24_00_01_19
1387546  /var/lib/aos/snapshot/2024-10-25_00_01_23
6937670  /var/lib/aos/snapshot/
admin@aos-server:~$

To verify that OS images are the reason for disk utilization, use the du command and check that the disk space used by the /opt/aos/frontend/www/dos_images/ directory corresponds to the increased partition utilization.

admin@aos-server:~$ sudo du /opt/aos/frontend/www/dos_images/
5496996    /opt/aos/frontend/www/dos_images/
admin@aos-server:~$

You can also use the ncdu command to explore filesystem utilization. e.g. You may have some files saved under /home/admin directory. 

 

If you are experiencing high disk utilization but have not uploaded any OS images to your Juniper Apstra server, contact Juniper Support for assistance.

alt

Solution

If you are experiencing high disk utilization on your Juniper Apstra server due to multiple uploaded OS images, it is recommended that you immediately correct this by deleting one or more OS images and freeing up disk space. Alternatively, you can register an OS image URL instead of uploading the OS image to the Juniper Apstra server.

If you wish to add available disk space to the Juniper Apstra server "root" disk partition, you can use the following procedure. Note that it is assumed here that your Juniper Apstra server VM is running on a VMware ESXi server managed by vSphere. The procedure for a Linux KVM will be similar. Contact  Juniper Support for more information.

This procedure can be carried out while the Juniper Apstra server VM is running, without any impact to the Juniper Apstra server or the managed network.

  1. In VMware vSphere, edit the setting for the Juniper Apstra server VM. Under Virtual Hardware, add a new Hard Disk device.

  1. For KVM follow thebelow steps to add disk space to the Apstra VM 

               a. ⁠virsh list ( to check the Apstra server name )
               b. virsh dumpxml <server-name> | more ( to check the source file installed path )
               c. ⁠qemu-img create -f qcow2 /var/lib/libvert/images/aos-server-v412-disk-extend 30G  
               d. Go to 4.1.2 & 4.2.2 server and check "lsblk" output 
               e. virsh attach-disk aos-server-v412 /var/lib/libvirt/images/aos-server-v412-disk-extend vdb --driver=qemu --subdriver=qcow2 --persistent ( execute this for both 4.1.2 and 4.2.2 )
               f. Go to 4.1.2 & 4.2.2 server and check "lsblk" output

               g. Now follow the below steps as given below to mount/add the space to the partition

 

  1. After vCenter has completed the VM change, use the Juniper Apstra server Linux CLI dmesg command to verify that the new disk is available. Note the disk device (for example sdb).

admin@aos-server:~$ sudo dmesg | tail
[673451.645443] scsi target2:0:0: FAST-40 WIDE SCSI 80.0 MB/s ST (25 ns, offset 127)
[673451.646392] sd 2:0:0:0: [sdb] 83886080 512-byte logical blocks: (42.9 GB/40.0 GiB)
[673451.646412] sd 2:0:0:0: [sdb] Write Protect is off
[673451.646413] sd 2:0:0:0: [sdb] Mode Sense: 61 00 00 00
[673451.646425] sd 2:0:0:0: [sdb] Cache data unavailable
[673451.646426] sd 2:0:0:0: [sdb] Assuming drive cache: write through
[673451.646758] sd 2:0:0:0: Attached scsi generic sg2 type 0
[673451.648034] sd 2:0:0:0: [sdb] Attached SCSI disk
admin@aos-server:~$
  1. Verify that the new disk is empty by using the fdisk command and ensure that no disk device partitions (for example /dev/sdb1) are listed.

admin@aos-server:~$ sudo fdisk -l /dev/sdb
Disk /dev/sdb: 40 GiB, 42949672960 bytes, 83886080 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
admin@aos-server:~$
  1. Partition the new disk (for example sdb) with a single Linux LVM partition by using the fdisk command.

admin@aos-server:~$ sudo fdisk /dev/sdb

Welcome to fdisk (util-linux 2.31.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0xfb2af457.

Command (m for help): n
Partition type
p   primary (0 primary, 0 extended, 4 free)
e   extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1): [ENTER]
First sector (2048-83886079, default 2048): [ENTER]
Last sector, +sectors or +size{K,M,G,T,P} (2048-83886079, default 83886079): [ENTER]

Created a new partition 1 of type 'Linux' and of size 40 GiB.

Command (m for help): t
Selected partition 1
Hex code (type L to list all codes): 8e
Changed type of partition 'Linux' to 'Linux LVM'.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks
admin@aos-server:~$
  1. Use the vgextend command to add the new partition to the aos-server-vg LVM Volume Group.

admin@aos-server:~$ sudo vgextend aos-server-vg /dev/sdb1
Physical volume "/dev/sdb1" successfully created.
Volume group "aos-server-vg" successfully extended
admin@aos-server:~$
  1. Use the vgdisplay command to verify that the new space is made available in the Volume Group. Note the available "Free  PE" (for example 10239).

admin@aos-server:~$ sudo vgdisplay
--- Volume group ---
VG Name               aos-server-vg
System ID
Format                lvm2
Metadata Areas        2
Metadata Sequence No  7
VG Access             read/write
VG Status             resizable
MAX LV                0
Cur LV                5
Open LV               5
Max PV                0
Cur PV                2
Act PV                2
VG Size               118.08 GiB
PE Size               4.00 MiB
Total PE              30229
Alloc PE / Size       19990 / <78.09 GiB
Free  PE / Size       10239 / <40.00 GiB
VG UUID               Hu46hG-tjKZ-9T4i-pL2w-UI20-SuRb-XUVXFr
admin@aos-server:~$
  1. Use the lvextend command with the "Free PE" value to add the new space to the "root" LVM Logical Volume.

admin@aos-server:~$ sudo lvextend -l +10239 /dev/aos-server-vg/root
Size of logical volume aos-server-vg/root changed from <11.20 GiB (2867 extents) to <51.20 GiB (13106 extents).
Logical volume aos-server-vg/root successfully resized.
admin@aos-server:~$
  1. Use the resize2fs command to resize the filesystem.

admin@aos-server:~$ sudo resize2fs /dev/aos-server-vg/root
resize2fs 1.44.1 (24-Mar-2018)
Filesystem at /dev/aos-server-vg/root is mounted on /; on-line resizing required
old_desc_blocks = 2, new_desc_blocks = 7
The filesystem on /dev/aos-server-vg/root is now 13420544 (4k) blocks long.
admin@aos-server:~$
  1. Finally, use the df -m command to verify that the new space is available on the Juniper Apstra server "root" disk partition.

admin@aos-server:~$ df -m
Filesystem                                 1M-blocks  Used Available Use% Mounted on
udev                                            7997     0      7997   0% /dev
tmpfs                                           1604    10      1595   1% /run
/dev/mapper/aos--server--vg-root               51540  8989     40326  19% /
tmpfs                                           8020     0      8020   0% /dev/shm
tmpfs                                              5     0         5   0% /run/lock
tmpfs                                           8020     0      8020   0% /sys/fs/cgroup
/dev/mapper/aos--server--vg-var                27153  2527     23225  10% /var
/dev/sda1                                       1890   118      1659   7% /boot
/dev/mapper/aos--server--vg-var+log            13150   208     12254   2% /var/log
/dev/mapper/aos--server--vg-var+lib+aos+db     22940    52     21701   1% /var/lib/aos/db
/dev/sr0                                           1     1         0 100% /media/cdrom
tmpfs                                           1604     0      1604   0% /run/user/1001
admin@aos-server:~$

 

Note: This same procedure can be used to increase the disk space on the other LVM partitions on the Juniper Apstra server (for example var).

Note: Other methods such as https://kb.vmware.com/s/article/1006371 [(Extending a logical volume in a virtual machine running Red Hat or Cent OS (1006371)] might be possible. Please contact VMWARE for detail and support.

Modification History

2023-02-03: Added a note about another method to extend a logical volume
2024-02-26: adding a note about the 'aos_extend_disk' utility and Apstra version

2025-10-17 : Adding KVM section to add space to Apstra-VM