Description

RE0 was stuck in the following state and we were not able to configure or run show commands on RE0 (Master):

 

Retry connection attempts ? [yes,no] (yes) could not open user interface connection: management daemon not responding

Retry connection attempts ? [yes,no] (yes) could not open user interface connection: management daemon not responding

Symptoms

Could see that several users were stuck in lockf state, as shown in the example below:

 

10740 root 21 0 480M 41724K lockf 1 0:00 0.00% mgd

11133 root 20 0 480M 41724K lockf 1 0:00 0.00% mgd

11213 root 20 0 480M 41724K lockf 1 0:00 0.00% mgd

11246 root 20 0 480M 41724K lockf 1 0:00 0.00% mgd

Solution

Log in to the other RE- RE1, and then enter RE1 shell, and from there, enter RE0 shell using the below commands:

 

RE1 > start shell user root

login to member0 RE0:

# rsh -T re0

 

  • On re0, check the mgd processes and kill the individual processes, in other words, kill the users stuck in lockf state one by one:

-----------

 #ps -Au |grep mgd

 

You will get an output like the following

root  33835  0.0 0.1 756848 39908 - Is  10:31    0:00.07 mgd: (mgd) (labroot)/dev/pts/0 (mgd) <<<<user is labroot

 

terminate the process

# kill 33835  

 

Do this for all the users (all the mgd processes), and you should be able to resolve the issue, and enter the CLI.

 

  • If the above workaround does not work, restart the mgd daemon using the command "restart management" from operational mode. Please do this in a MW as restarting the mgd process may close existing sessions on the device and cancel pending commits.
  • Reboot the device as last resort.

This errors seems to be because of some commit which did not go through on RE causing a lock on the RE database.

We can verify this by comparing the show system commit. We can also see mgd process being high during issue time causing a database lock. 

 

On Junos and Junos OS Evolved platforms, a situation arises where the mgd (Management Daemon) process fails to release the commit lock, preventing CLI users from modifying the configuration. 

Modification History

2024-09-03 : Article Created