Description

This document provides steps for installing & rollback of hotpatch on Paragon automation.

Solution

Installation steps:

 Step 1: Copy the patch on to the Master Node.
# ls -lrt | grep EVAL
-rw------- 1 root root 1173698560 Jul 8 08:54 patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9.tar

 

Step 2: Un-tar the file and navigate to the patch folder.

root@davinci-master:/var/tmp# tar -xvf patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9.tar

patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9/

patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9/ns_celery_scheduler.tar.gz

patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9/ns_pcviewer.tar.gz

patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9/rollback-images.sh

patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9/ns_pcserver.tar.gz

patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9/dcscheduler.tar.gz

patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9/push-external-registry.sh

patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9/REPO_TAG

patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9/deploy-images.sh

patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9/check-images.sh

patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9/patch.cfg

root@davinci-master:/var/tmp# cd patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9/

root@davinci-master:/var/tmp/patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9# ls -lrt

total 1146240

-rw------- 1 32972 707 293204869 Jun 12 20:28 ns_celery_scheduler.tar.gz

-rw------- 1 32972 707 438745382 Jun 12 20:30 dcscheduler.tar.gz

-rw------- 1 32972 707 173757439 Jun 12 20:30 ns_pcserver.tar.gz

-rwx------ 1 32972 707     2720 Jun 12 20:31 rollback-images.sh

-rw------- 1 32972 707      512 Jun 12 20:31 REPO_TAG

-rwx------ 1 32972 707     1468 Jun 12 20:31 push-external-registry.sh

-rw------- 1 32972 707     1406 Jun 12 20:31 patch.cfg

-rw------- 1 32972 707 267967019 Jun 12 20:31 ns_pcviewer.tar.gz

-rwx------ 1 32972 707     8975 Jun 12 20:31 deploy-images.sh

-rwx------ 1 32972 707      469 Jun 12 20:31 check-images.sh

 

Step 3: Verify current installed image details.

# ./check-images.sh

Current images in running pods:

ns_celery_scheduler

localhost:5000/eng-registry.juniper.net/northstar-scm/northstar-containers/ns_celery_scheduler:release-23-1-ge572e4b914

dcscheduler

localhost:5000/eng-registry.juniper.net/northstar-scm/northstar-containers/dcscheduler:release-23-1-ge572e4b914

ns_pcserver

localhost:5000/eng-registry.juniper.net/northstar-scm/northstar-containers/ns_pcserver:release-23-1-ge572e4b914

ns_pcviewer

localhost:5000/eng-registry.juniper.net/northstar-scm/northstar-containers/ns_pcviewer:release-23-1-ge572e4b914

 

Step 4: Proceed with installation by executing “./deploy”, at the end of the installation we will notice “Patch deployment is complete.”

 

root@davinci-master:/var/tmp/patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9# ./deploy-images.sh

 

NAME                             READY  UP-TO-DATE  AVAILABLE  AGE

docker-registry-docker-registry  2/2    2           2          67m

 

NAME                                   DESIRED  CURRENT  READY  UP-TO-DATE  AVAILABLE  NODE SELECTOR  AGE

docker-registry-docker-registry-proxy  4        4        4      4           4          <none>         67m

 

Patch deployment is complete.

Patch deployment backup file is saved to /var/tmp/patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9/patch_deploy.backup_20240708_0858

 

Step 5: Once installation is done, we can cross the images installed.

root@davinci-master:/var/tmp/patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9# ./check-images.sh

Current images in running pods:

ns_celery_scheduler

localhost:5000/eng-registry.juniper.net/northstar-scm/northstar-containers/ns_celery_scheduler:patch_23.1-PF_EVAL_11330-d8fdcd9

dcscheduler

localhost:5000/eng-registry.juniper.net/northstar-scm/northstar-containers/dcscheduler:patch_23.1-PF_EVAL_11330-d8fdcd9

ns_pcserver

localhost:5000/eng-registry.juniper.net/northstar-scm/northstar-containers/ns_pcserver:patch_23.1-PF_EVAL_11330-d8fdcd9

ns_pcviewer

localhost:5000/eng-registry.juniper.net/northstar-scm/northstar-containers/ns_pcviewer:patch_23.1-PF_EVAL_11330-d8fdcd9

 

Step 6: Make sure all the pods are running.

# kubectl get pods -A

 

Rollback steps:

Step 1: To rollback the installed patch, navigate to the patch folder and check for backup file created at the time of installation. [Refer to step 4 of installation]

root@davinci-master:/var/tmp/patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9# ls -lrt | grep backup

-rw-r--r-- 1 root root      262 Jul 8 08:59 patch_deploy.backup_20240708_0858

 

Step 2: Execute the "./rollback-images.sh" with the backup file as input.

root@davinci-master:/var/tmp/patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9# ./rollback-images.sh patch_deploy.backup_20240708_0858

 

Restoring /etc/kubernetes/po/pcviewer/kube-cfg.yml.bak_20240708_0858

[Run] mv /etc/kubernetes/po/pcviewer/kube-cfg.yml.bak_20240708_0858 /etc/kubernetes/po/pcviewer/kube-cfg.yml

[Run] kubectl delete -f /etc/kubernetes/po/pcviewer/kube-cfg.yml

deployment.apps/dcscheduler created

Updated /etc/kubernetes/po/dcscheduler/kube-cfg.yml with original image has been applied.

 Patch rollback is complete.

 

Step 2: Verify the images post the rollback.images should have rolled back to older versions.

root@davinci-master:/var/tmp/patch_23.1-PF_EVAL_11330-20240612_1328-d8fdcd9# ./check-images.sh

Current images in running pods:

ns_celery_scheduler

localhost:5000/eng-registry.juniper.net/northstar-scm/northstar-containers/ns_celery_scheduler:release-23-1-ge572e4b914

dcscheduler

localhost:5000/eng-registry.juniper.net/northstar-scm/northstar-containers/dcscheduler:release-23-1-ge572e4b914

ns_pcserver

localhost:5000/eng-registry.juniper.net/northstar-scm/northstar-containers/ns_pcserver:release-23-1-ge572e4b914

ns_pcviewer

localhost:5000/eng-registry.juniper.net/northstar-scm/northstar-containers/ns_pcviewer:release-23-1-ge572e4b914

 

Note : If you face any issues, please contact JUNIPER JTAC Support for assistance.

Modification History

2024-08-15 : Article Created