Description

This is article how to deal with the Major Alarm Application routing fail on node Re1 & Application rpdagent fail on node Re1

Symptoms

You will see the below alarm with the Crash File

 

show chassis alarms

 

2 alarms currently active

Alarm time Class Description

2024-05-18 01:24:33 PDT Major Application routing fail on node Re1

2024-05-18 01:24:34 PDT Major Application rpdagent fail on node Re1

*******************************

re0:
--------------------------------------------------------------------------
-rw-r--r-- 1 root root  1883323232 Nov 15 2023 /var/core/fpc1/msvcsd.fpc_x86_64.fpc1.15683.2023_11_15.04_07_03.tar.gz

total files: 1

re1:
--------------------------------------------------------------------------
-rw-r--r-- 1 root root  1937718513 May 18 01:26 /var/core/re1/rpdagent.re.re1.17982.2024_05_18.01_23_02.tar.gz
-rw-r--r-- 1 root root  1801746159 May 18 01:31 /var/core/re1/rpdagent.re.re1.26331.2024_05_18.01_24_00.tar.gz
-rw-r--r-- 1 root root  1802440946 May 18 01:33 /var/core/re1/rpdagent.re.re1.29197.2024_05_18.01_24_27.tar.gz

total files: 3

 

If the child objects are published by the different applications, the applications of the publishing parent might crash randomly. Then Junos OS Evolved system might be stuck in malfunction state. 

 

The following log messages might be seen when running into this issue.

 

user@device> show system core-dumps 

re0:

--------------------------------------------------------------------------

xxx /var/core/re0/ifmand.xxx.tar.gz

xxx /var/core/re0/vmcore.xxx

 

re1:

--------------------------------------------------------------------------

xxx /var/core/re1/ifmand.xxx.tar.gz

xxx /var/core/re1/vmcore.xxx

 

This issue might be seen if the following conditions are met:

* Junos OS Evolved platform only

* Different number of self-owned parent objects are publishing (e.g. The ifl/iff/ifa object published by different applications (e.g. ifmand/cosd/dfwd) link to the parent interface object, where it might crash)

* Any scenarios which trigger the publishing of self-owned parent objects (e.g. scaled bgp sessions flap/NSR switchover)

 

Solution

In Junos OS Evolved platform scenario, the committed hierarchy CLI configuration will be published by applications as the object in hierarchy order and form the parent-child link between them (e.g. the ifl (logical device) will be published by ifmand as ifl object (iflo), and ifd (physical device) object (ifdo) is the parent of iflo). But in some corner cases, if the child objects are published by the different applications, the application publishing the parent object might crash for some reason. Then some issues might happen on this kind of parent-child relationship, the related applications might crash randomly, even after posting any restart.

 

PR : https://prsearch.juniper.net/problemreport/PR1595846

 

Please follow the below,

To clear the alarm, can you please restart the apps namely routing and rpdagent, let us know the status post restart.

 

> request system application app rpdagent restart node re1

> request system application app routing restart node re1 

 

After restarting please verify the alarms alarms and please provide the command output.

 

> show system applications app routing detail 

> show system applications app rpdagent detail 

 

Modification History

2024-07-25 : Article Created