Description

This article explains why users may encounter the "Minimum 60 GB of disk space is required in /var partition" error while performing the Contrail Healthbot setup and what they should do to ensure that the setup is successful.

 

Symptoms

CentOS VM is installed by using a 150GB HD but upon installing on Kubernetes, the user gets the following error during the Healthbot setup:

[root@localhost ~]# sudo healthbot setup
Setting new password for postgres db
Do you want to install on kubernetes? [y/N] y
Checking disk space
Minimum 60 GB of disk space is required in /var partition. Available: 30 GB

While checking the VM disk space, we see that /var , which is part of /dev/mapper/centos-root , has only 50G in total, which is insufficient for the Healthbot installation. The minimum disk space required for the Contrail Healthbot installation is 60G.

[root@localhost ~]# df -h
Filesystem               Size  Used Avail Use% Mounted on
devtmpfs                 9.8G     0  9.8G   0% /dev
tmpfs                    9.9G     0  9.9G   0% /dev/shm
tmpfs                    9.9G  8.8M  9.8G   1% /run
tmpfs                    9.9G     0  9.9G   0% /sys/fs/cgroup
/dev/mapper/centos-root   50G  6.6G   44G  14% /
/dev/sda1               1014M  210M  805M  21% /boot
/dev/mapper/centos-home   90G   33M   90G   1% /home
tmpfs                    2.0G     0  2.0G   0% /run/user/0

 

Solution

 

Option 1: (Recommended) Re-installing the OS

Re-install the host Operating System. During the OS installation, manually adjust the disk partitions before starting the installation. See the OS Installation documentation for guidance.

Option 2:  Re-allocating the home partition

If the OS installation has placed a large amount of data in the /home volume, this disk space can be reallocated. Note that this should be done ONLY on a system that was just installed.

Caution: The process to reallocate disk space will DESTROY all data in the /home/* partition so it is a good practice to back up data to /tmp and then restore it.

Note: The first two steps in the following process are to back up data to /tmp . (This can be done only if the used space in /home is less than the available disk space in / .)

  1. Confirm that the /home partition was created and has low usage.

[root@localhost ~]# df -h
Filesystem               Size  Used Avail Use% Mounted on
devtmpfs                 9.8G     0  9.8G   0% /dev
tmpfs                    9.9G     0  9.9G   0% /dev/shm
tmpfs                    9.9G  8.8M  9.8G   1% /run
tmpfs                    9.9G     0  9.9G   0% /sys/fs/cgroup
/dev/mapper/centos-root   50G  6.6G   44G  14% /
/dev/sda1               1014M  210M  805M  21% /boot
/dev/mapper/centos-home   90G   33M   90G   1% /home

[root@localhost ~]# lvs
  LV   VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  home centos -wi-ao---- <89.18g                                                    
  root centos -wi-ao----  50.00g                                                    
  swap centos -wi-ao----   9.81g        
  1. Back up the contents of /home .

To follow this process, ensure that you have logged in to the system as the "root" user directly (not sudo).

If the data used in /home is small, back up the data ( /tmp needs to have more disk space available than the used space in /home ).

tar czf /tmp/home_backup.tgz /home
  1. To expand /root , unmount the home volume, DESTROY the home volume, and then apply the home volume disk space to the root volume.

[root@localhost ~]# umount -fl /dev/mapper/centos-home
  1. Remove the unmounted volume.

[root@localhost ~]# lvremove /dev/mapper/centos-home
Do you really want to remove active logical volume centos/home? [y/n]: y
  Logical volume "home" successfully removed
  1. Re-check the logical volumes.

[root@localhost ~]# lvs
  LV   VG     Attr       LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root centos -wi-ao---- 50.00g                                                    
  swap centos -wi-ao----  9.81g                                                    
  1. Extend the logical volume of root with the space created by unmounting/removing the centos-home volume.

[root@localhost ~]# lvextend -L+50G /dev/mapper/centos-root
  Size of logical volume centos/root changed from 50.00 GiB (12800 extents) to 100.00 GiB (25600 extents).
  Logical volume centos/root successfully resized.
  1. Re-grow the centos-root volume.

[root@localhost ~]# xfs_growfs /dev/mapper/centos-root
meta-data=/dev/mapper/centos-root isize=512    agcount=4, agsize=3276800 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0 spinodes=0
data     =                       bsize=4096   blocks=13107200, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal               bsize=4096   blocks=6400, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
data blocks changed from 13107200 to 26214400
  1. Remove the /dev/mapper/centos-home line from file fstab so that the Linux OS will not look for that volume during the next boot.

[root@localhost ~]# vi /etc/fstab
#
# /etc/fstab
# Created by anaconda on Tue Jul 21 09:13:31 2020
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/centos-root /                       xfs     defaults        0 0
/dev/mapper/centos-home /                       xfs     defaults        0 0   <<<<< Delete this.
/dev/mapper/centos-swap swap                    swap    defaults        0 0
~                                                                                                                                                      
~                                                                                                                                                      
~                                                                                                                                                      
~                                                                                                                                                      
~                                                                                                                                                      
~                                                                                                                                                      
~                                                                                                                                                      
~                                                                                                                                                      
~                                                                                                                                                      
~                                                                                                                                                      
~                                                                                                                                                      
~                                                                                                                                                      
~                                                                                                                                                      
~                                                                                                                                                      
~                                                                                                                                                      
~                                                                                                                                                      
~                                                                                                                                                      
~                                                                                                                                                      
"/etc/fstab" 11L, 465C written
  1. Re-check the space for the root volume.

[root@localhost ~]# lvs
  LV   VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root centos -wi-ao---- 100.00g                                                    
  swap centos -wi-ao----   9.81g    
  1. The disk space is now good enough for the Healthbot installation to proceed.                                              

[root@localhost ~]# df -h             
Filesystem               Size  Used Avail Use% Mounted on
devtmpfs                 9.8G     0  9.8G   0% /dev
tmpfs                    9.9G     0  9.9G   0% /dev/shm
tmpfs                    9.9G  8.7M  9.8G   1% /run
tmpfs                    9.9G     0  9.9G   0% /sys/fs/cgroup
/dev/mapper/centos-root  100G  6.6G   94G   7% /    <<<<<<<<<<
/dev/sda1               1014M  210M  805M  21% /boot
  1. Re-create/restore /home data.

cd /
tar zxf /tmp/home_backup.tgz