On VM Host platforms, insufficient available memory (less than 7.5G/8G) during software installation leads to failures and subsequent issues, including RE (Routing Engine) failure. The error arises during VM Host software installation, displaying messages like "tar: Cannot write: No space left on device" and "tar: Unexpected EOF in archive." If a reboot follows a failed installation, the RE goes dead or boots from an alternative disk. This issue affects various platforms using VMHost releases, such as ACX5448, EX9200, MX series, PTX series, QFX10002-60C, and others.
VM Host software installation failure with error messages related to insufficient space.
RE goes dead or boots from an alternative disk after a reboot following a failed installation.
Misleading CLI messages, such as "upgrade complete," even when space-related errors occur.
Log entries indicating lack of space during the upgrade process.
Below is an excerpt of the CLI messages displayed during the issue:-
tar: junos/junos-install-mx-x86-xx.xxxx.xxxx.img.gz: Cannot write: No space left on device tar: ./vmhost-core-x86-xxx.xxx-xxx.tgz: Wrote only 512 of 10240 bytes tar: Exiting with failure status due to previous errors gzip: stdin: unexpected end of file tar: Unexpected EOF in archive tar: Unexpected EOF in archive tar: Error is not recoverable: exiting now ls: cannot access '/tmp/partdisk-RYsWnN/unpack/jhost*core*.rpm': No such file or directory ... upgrade complete. Initiating vmhost reboot ... Initiating vmhost reboot... warning: Initiating Junos shutdown... shutdown: [pid xxxx]
Ensure the VM Host has sufficient available memory (at least 7.5G/8G) before initiating upgrades.
If encountering memory issues during an upgrade, restart VMHOST to clear up the memory. The issue is tracked in PR1755585
Example:
root@starfire-re1-node:~# cat /proc/meminfo MemTotal: 64757372 kB MemFree: 10282600 kB MemAvailable: 9904172 kB <-- should be more than 7.5/8G to make vmhost install upgrade success Buffers: 2276 kB Cached: 2473976 kB
This issue is addressed via PR 1759077. With the fix, you'll receive an error message when there is low memory available during the upgrade process.Example:
root@doon-a> ...6-64-24.1I20240105_1300_nlagisetty.tgz Junos Validation begin. Procedure will take few minutes. Running image compatibility pre-checks for /var/tmp/junos-vmhost-install-mx-x86-64-24.1I20240105_1300_nlagisetty.tgz. Current platform is [mx] Image compatibility pre-checks pass successfully. Required: 7.6 GB Available: 31.6 GB in Junos guest /var/tmp Not enough free memory in host Required: 7.6 GB Available: 0.0 GB in host warning: Host software installation has failed.