Description

This article describes how to recover from the situation where Junos image upgrade failed for SRX1500 Dual CPE site through CSO.

Symptoms

  • Customer managing SDWAN SRX1500 Dual CPE site through CSO SaaS portal running with 6.3 version. 
  • SRX1500 Dual CPE site running with Junos version 20.4X15.1.
  • Customer staged Junos 22.2R3-S4.11 on to the Site through CSO. 
  • Customer tried to upgrade Junos image on SRX1500 Dual CPE site through the CSO.
  • Junos image upgraded successfully only one node (i.e node1) and other node (node 0) failed to upgrade. 
  • Junos NSD process core file observed on node (node 0) at the time of upgrade. 

 

Solution

Due to know Junos bug PR 1724777 ( https://prsearch.juniper.net/problemreport/PR1724777 ) , when ISSU upgrade is performed from a Junos version lower than 21.4R3 to 21.4R3 or higher, the nsd process might crash. This bug is fixed in 21.4R3-S5 and higher Junos versions.

Hence when ISSU upgrade performed from 21.4R3-S5 or greater to higher Junos version then this problem will not be observed. 

 

 

Workaround : Follow below steps as part of workaround. 

  • At times due to Junos version mistmatch there could be multiple corefiles ( eg: crashinfo, ksyncd) generated on the node that failed to upgrade. It is advised to delete all corefiles to avoid storage issue before proceeding with next step i,e software upgrade. 
Following command can be used to sort the files in shell prompt based on the size.

sh -c 'find / -size +10485760c 2> /dev/null' | xargs du -h | sort -nr
Note:  When there is less sway memory available due to continuous core file generation, device may go for reset. Once device comes up proceed to next step by upgrading the software . 

 

Modification History

2024-08-30 : Article Created