This KB Describes RE network-service mode mismatch between configuration and kernel setting alarm due to missing script files and how to resolve the same.

• RE can report network service mismatch alarm if event script is not present in folder specified which is configured in event options.
• If an RE goes through reboot in such condition, it boots up with partial config.
• In boot messages we can see mgd deamon reports "config checkout failed due to missing script files" and hence booting up with partial config which can result in network-service mismatch alarm on RE with missing script file.

In this scenario RE0 is master and RE1 is backup 

• "RE1 network-service mode mismatch between configuration and kernel setting" alarm seen on Master RE0:

root@LAB_RE0> show chassis alarms no-forwarding

 
1 alarms currently active

Alarm time              Class Description

2023-07-10 05:51:10 UTC Major RE1 network-service mode mismatch between configuration and kernel setting

• Network services configured is Enhanced IP:

On RE0:

chassis {

.

.

   network-services enhanced-ip;

}

On RE1:

chassis {

.

.

   }

   network-services enhanced-ip;

}

• But in kernel RE1 shows mode as IP mode instead of enhanced IP.
• Refer below example for troubleshooting this issue.

root@LAB_RE0> show chassis network-services 

Network Services Mode: Enhanced-IP
 

Network service mode in Kernel:

root@LAB_RE0> request routing-engine execute command "sysctl net.netsvc" 
net.netsvc: 2

2 in above output means Enhanced IP Mode. On RE0 kernel and config is matching the Mode, Enhanced IP.

On RE1 (Backup RE):

Network service mode in CLI:

root@LAB_RE1> show chassis network-services  >>> RE1

Network Services Mode: Enhanced-IP

Network service mode in Kernel:

root@LAB_RE1> request routing-engine execute command "sysctl net.netsvc" 
net.netsvc: 0  >>>>> IP mode

0 in above output means IP Mode in Kernel. This means on RE1 network services mode on kernel and config is mismatched.

Refer to KB: KB70169 [juniper.net] for additional details:

https://supportportal.juniper.net/s/article/What-does-network-service-mode-mismatch-between-configuration-and-kernel-mean

Console  logs from RE1:

• We can see scripts file TEST_1.slax and TEST_2.slax is showing invalid file name and RE 1 is booting up with partial config:

invalid filename: /var/db/scripts/event//TEST_1.slax

</message>

</xnm:error>

<xnm:error xmlns="http://xml.juniper.net/xn

FreeBSD/amd64 (Amnesiac) (ttyu0)

<xnm:error xmlns="http://xml.juniper.net/xnm/1.1/xnm" xmlns:xnm="http://xml.juniper.net/xnm/1.1/xnm">

<source-daemon>

commit-script

</source-daemon>

<message>

invalid filename: /var/db/scripts/event//TEST_2.slax

</message>i

</xnm:error>

• RE Boots up with partial config and mgd reports Config checkout failed:

mgd: error: configuration check-out failed

Warning: Commit failed, activating partial configuration.

Warning: Edit the router configuration to fix these errors.

@ 1688968232 [2023-07-10 05:50:32 UTC] mgd done

• Upon further checking, We could find script files were missing from the location specified but the event options were pointing to these script files:

event-options {

   generate-event {

       0600-CST time-of-day "12:00:00 +0000";

       1800-CST time-of-day "12:00:00 +0000";

       EVERY-2H time-interval 7200;

   }

   policy TEST_1{

       events [ 0600-CST 1800-CST ];

       then {

           event-script TEST_1.slax;

       }

   }

   policy TEST_2 {

       events EVERY-2H;

       then {

           event-script TEST_2.slax;

       }

   }

   event-script {

       file TEST_1.slax;

       file TEST_2.slax;

   }

}

Solution 1:

• Add the missing script file to RE or if files are present in other RE, then synchronize the script files between the REs with using below command:

request system scripts synchronize all 

• Reboot affected RE to resolve issue after synchronized.

Solution 2:

Or we can enable the below knob in event-option config:

 set event-options event-script optional 

 optional knob will allow commit to succeed if the script is missing

By default, optional is not set. Event scripts are required to be accessible during commit operations.

Refer Document: https://www.juniper.net/documentation/us/en/software/junos/automation-scripting/topics/ref/statement/optional-edit-event-options.html

2023-07-12: initial version