Description
On SRX Series devices starting from "junos:21.4R3-S5", In Chassis Cluster, when the Tenant system configuration enabled, then NSD will remain high on primary device, if one of the below mentioned trigger is hit:
Trigger:
* When one of the devices are rebooted
* When one of the devices are powered on
* When RG0 is failed over
Symptoms
>> When the below mentioned Tenant configuration is present and any one of the description mentioned triggers are hit, then the code gets stuck and run into the loop which causes nsd to keep on running in the background and stays nearly 100% all the time.
>> Tenant system configuration:
set tenants Juniper routing-instances CUST_Juniper instance-type virtual-router
set tenants Juniper routing-instances CUST_Juniper routing-options static route 0.0.0.0/0 next-hop 10.10.1.1
set tenants Juniper routing-instances CUST_Juniper interface lt-0/0/0.1
set tenants Juniper routing-instances CUST_Juniper interface reth1.123
set tenants Juniper security policies from-zone Juniper_A to-zone Juniper_B policy ANY_IN match source-address any
set tenants Juniper security policies from-zone Juniper_A to-zone Juniper_B policy ANY_IN match destination-address any
set tenants Juniper security policies from-zone Juniper_A to-zone Juniper_B policy ANY_IN match application any
set tenants Juniper security policies from-zone Juniper_A to-zone Juniper_B policy ANY_IN then permit
set tenants Juniper security policies from-zone Juniper_A to-zone Juniper_B policy ANY_IN then log session-init
set tenants Juniper security zones security-zone Juniper_A host-inbound-traffic system-services all
set tenants Juniper security zones security-zone Juniper_A host-inbound-traffic protocols all
set tenants Juniper security zones security-zone Juniper_A interfaces reth1.123
set tenants Juniper security zones security-zone Juniper_B host-inbound-traffic system-services all
set tenants Juniper security zones security-zone Juniper_B host-inbound-traffic protocols all
set tenants Juniper security zones security-zone Juniper_B interfaces lt-0/0/0.1
>>RE CPU Idle will go to 0 on the primary device as shown below:
{primary:node0}
root@SRX> show chassis routing-engine
Nov 30 19:36:33
node0:
--------------------------------------------------------------------------
Routing Engine status:
Temperature 38 degrees C / 100 degrees F
CPU temperature 38 degrees C / 100 degrees F
Total memory 1900 MB Max 475 MB used ( 25 percent)
Memory utilization 24 percent
5 sec CPU utilization:
User 31 percent
Background 0 percent
Kernel 68 percent
Interrupt 1 percent
Idle 0 percent
>> NSD process will be showing 100% utilized all the time on primary device as shown below:
{primary:node0}
root@SRX> show system processes extensive
Nov 30 19:36:56
node0:
--------------------------------------------------------------------------
last pid: 65345; load averages: 1.50, 1.82, 1.78 up 8+08:24:48 19:36:56
373 threads: 3 running, 310 sleeping, 1 zombie, 59 waiting
CPU: 32.2% user, 0.0% nice, 66.7% system, 0.5% interrupt, 0.6% idle
Mem: 40M Active, 1307M Inact, 188K Laundry, 397M Wired, 190M Buf, 123M Free
Swap: 1639M Total, 1639M Free
PID USERNAME PRI NICE SIZE RES STATE TIME WCPU COMMAND
23829 root 102 0 758M 73M RUN 187.0H 100.00% nsd
Solution
Solution:
It has been addressed with PR1776480, and Engineering has committed fix in versions: junos:21.4R3-S6 junos:23.2R2 junos:23.4R2
Workaround:
Please remove the tenant system from the configuration or follow the below steps, if tenant system is required for the device.
If the tenant system is required then, failover of RG0 or reboot or power outage of the device can cause this issue to reoccur.
Please follow the below steps as a work around to overcome the issue, if it reoccurs:
Step:1, Remove Tenant system configuration and commit
Step:2, Restart network-security using command "restart network-security immediately"
Step:3, Add the Tenant system configuration back and commit
Note
:
The above WA is only applicable till the next failover of RG0 or reboot or power outage.
Modification History
2023-12-22 : Article Created
2024-01-02 : Article Updated and Published
On SRX Series devices, In Chassis Cluster, when the Tenant system configuration enabled, then NSD will remain high on primary device.