Description

In some rare conditions, a Junos-based device may become unresponsive for configuration or other operations after executing the command "commit confirmed x" twice.

Symptoms

After executing the CLI command commit confirmed 5 , if the command is still executing as per the set timer value (5 minutes, in this example), and the user sends another “commit confirm” command (during the time slot between the timeout of 5 minutes and the auto-rollback operation), it causes the commit to lock and impacts the operation.

Example:

The user enters the commit confirmed 5 command. The MGD process is undergoing a commit. Then at the same time, the user enters another MGD session ( commit confirmed x ).

The following message is logged as part of the second commit but the first commit has not completed:

router-re0 mgd[71402]: UI_CFG_AUDIT_OTHER: User 'root‘ override:/var/run//changes.xml

The following message is logged to show that the rollback action of the first “commit confirmed” started, which eventually is blocked:

router-re0 mgd[74126]: UI_CFG_AUDIT_OTHER: User 'root' override: /config/juniper.conf.unconfirmed.gz

The following message will continue to show that the first "commit confirmed" is still in progress:.

router-re0 mgd[74126]: UI_CFG_AUDIT_OTHER: User 'root' override: /var/run//changes.xml+

Solution

Due to a race condition, it enters into a cyclic dependency between two management daemon (MGD) sessions acquiring the db_lock for editing the configuration. Due to this condition, any subsequent user is not be able to enter into the edit mode. The output of the process extensive also shows the MGD process stuck in the lockf state.

Work-around:

The work-around is to kill the MGD process in the lockf state in the UNIX shell. (You must be logged in as root to perform these steps.)

user@router-re1> show system process extensive | grep mgd
 2616  root  2  0  13012K 7000K select 0:30 0.00% 0.00% mgd
45570  root  2  0  13068K 9020K lockf 0:02 0.00% 0.00% mgd
45525  root  2  0  14128K 8440K lockf 0:00 0.00% 0.00% mgd
45601  root  2  0  13068K 7712K lockf 0:00 0.00% 0.00% mgd

45624  root  2  0  13052K 7468K select 0:00 0.00% 0.00% mgd


user@router-re1> start shell user root
Password:
root@router-re1%
root@router-re1% kill -9 45570
root@router-re1% kill -9 45525
root@router-re1% kill -9 45601

root@router-re1% exit
exit

user@router-re1> show system process extensive | grep mgd
45629  root  2  0  13052K 7468K select 0:00 1.00% 0.05% mgd
2616   root  2  0  13012K 7000K select 0:30 0.00% 0.00% mgd

Configuration mode will then be available again.
 

Fix:

See PR1415042 for fixed releases where this race condition is addressed.