Description

This article describes one of the possible issues we may face during an upgrade attempt (here, upgrading the QFX device to 22.2R3-S4.10 version)

Symptoms

You may see the following or similar output:


root@host> request system software add /var/tmp/junos-arm-32-22.4R3.25.tgz no-validate reboot
Removing /var/log/wtmp.1.gz
Removing /var/log/wtmp.0.gz
Removing /var/log/messages.0.gz
Removing /var/log/interactive-commands.0.gz
setting unlink by default.
setting unlink by default.
veriexec: cannot validate /packages/db/pkginst.13734/manifest.ecerts: certificate is not yet valid: /C=US/ST=CA/L=Sunnyvale/O=Juniper Networks/OU=Juniper CA/CN=PackageProductionECP256_2024/[email protected]
veriexec: cannot validate junos-arm-32-22.4R3.25.ecerts: certificate is not yet valid: /C=US/ST=CA/L=Sunnyvale/O=Juniper Networks/OU=Juniper CA/CN=PackageProductionECP256_2024/[email protected]
veriexec: /packages/db/pkginst.13734/package.xml: Authentication error
usage: /usr/sbin/pkg add <pkg> ...
where <pkg> is a compressed tar file
ERROR: Failed to add junos-arm-32-22.4R3.25.tgz

OR

2025-02-20 03:30:10,858 INFO:Command (timeout-1200): request system software add /var/tmp/jinstall-host-qfx-5e-x86-64-22.2R3-S4.10-secure-signed.tgz force no-validate  force-host | display xml
2025-02-20 03:32:06,018 INFO:Add software image: <rpc-reply xmlns:junos="http://xml.juniper.net/junos/21.2R0/junos">
    <cli>
        <ignore-signals>
            hup
        </ignore-signals>
    </cli>
    <output>
        veriexec: cannot validate /packages/db/pkginst.85409/manifest.ecerts: certificate is not yet valid: /C=US/ST=CA/L=Sunnyvale/O=Juniper Networks/OU=Juniper CA/CN=PackageProductionECP256_2024/[email protected]
        FIPS veriexec ECDSA Verify Known Answer Test: Passed
        veriexec: cannot validate jinstall-host-qfx-5e-x86-64-22.2R3-S4.10-secure-signed.ecerts: certificate is not yet valid: /C=US/ST=CA/L=Sunnyvale/O=Juniper Networks/OU=Juniper CA/CN=PackageProductionECP256_2024/[email protected]
        FIPS veriexec ECDSA Verify Known Answer Test: Passed
        veriexec: /packages/db/pkginst.85409/package.xml: Authentication error
        usage: /usr/sbin/pkg add &lt;pkg&gt; ...
        where &lt;pkg&gt; is a compressed tar file
        ERROR: Failed to add jinstall-host-qfx-5e-x86-64-22.2R3-S4.10-secure-signed.tgz
    </output>
    <package-result>1</package-result>
    <cli>
        <banner>{master:0}</banner>
    </cli>
</rpc-reply> back_re_present: False
2025-02-20 03:32:06,019 ERROR:Image install failed with error.
2025-02-20 03:32:06,019 ERROR:FAILED Image install failed with error.
2025-02-20 03:32:06,019 INFO:Deleting Existing NOS image file /var/tmp/jinstall-host-qfx-5e-x86-64-22.2R3-S4.10-secure-signed.tgz from device
2025-02-20 03:32:06,020 INFO:Command (timeout-120): file delete /var/tmp/jinstall-host-qfx-5e-x86-64-22.2R3-S4.10-secure-signed.tgz
2025-02-20 03:32:06,481 INFO:
2025-02-20 03:32:06,481 ERROR:Image install failed with error.

You can see from the above error messages that the upgrade has been failed due to failure to add the image.

  • The QFX upgrade failed with an error related to certificate validation issues.
  • The error message: "veriexec: cannot validate /packages/db/pkginst.85409/manifest.ecerts: certificate is not yet valid" indicates that the certificate timestamp is incorrect or the system clock is out of sync.
  • "veriexec: /packages/db/pkginst.85409/package.xml: Authentication error" means that the package verification failed, likely due to the invalid certificate.
  • "ERROR: Failed to add jinstall-host-qfx-5e-x86-64-22.2R3-S4.10-secure-signed.tgz" indicates that the upgrade process aborted.

Solution

Step-by-Step Troubleshooting:

The error "certificate is not yet valid" suggests that the system date/time is incorrect.

1.Check the current date/time on the device:

show system uptime
show system date

If incorrect, set the correct time:

set date YYYYMMDDHHMM
commit

OR

use NTP sync:

set system ntp server <ntp-server-ip>
commit

2. Verify image file Integrity: Check if the file is corrupted. Also, Manually verify the image signature using: 

file list /var/tmp/jinstall-host-qfx-5e-x86-64-22.2R3-S4.10-secure-signed.tgz
request system software validate /var/tmp/jinstall-host-qfx-5e-x86-64-22.2R3-S4.10-secure-signed.tgz

3. Re-download the Image: 

file delete /var/tmp/jinstall-host-qfx-5e-x86-64-22.2R3-S4.10-secure-signed.tgz
scp user@juniper-server:/path/to/image /var/tmp/

4. Retry Installation with Proper Flags: 

  • Try installing without validation: request system software add /var/tmp/jinstall-host-qfx-5e-x86-64-22.2R3-S4.10-secure-signed.tgz no-validate force
  • If that fails, try: request system software add /var/tmp/jinstall-host-qfx-5e-x86-64-22.2R3-S4.10-secure-signed.tgz no-validate no-verify force

5. Check Logs for more details and analyse:

show log messages | match "error|fail"
show system software detail

6. If the issue persists: Please check Juniper's release notes for known issues with 22.2R3-S4.10. Try a different software version or rollback to a working version: 

request system software rollback
request system reboot

7. If it's an RTC (Real-Time Clock) issue, a reboot might fix it.

If the issue remains the same after changing the time, please perform format installation instead of upgrade

Modification History

KB Creation

Related Information

Configure Date and Time Locally