Customer has a issue with CGNAT box using MX-SPC3 after upgrading the Junos version to 23.4R2-S7
There are some abnormal on this box.
- When the customer shows some commands relevant to service NAT, there is no any output
- The nsd process is not appeared with "show system processes extensive"
- The box can not user NAT service
When the Routing Engine restarts, the nsd may remain down or fails to program internal subsystems like service sets, causing the device to lose control plane functionality and forwarding stops completely.
Issue can be identified by checking if nsd is missing from the process list when issuing the command:
> show system processes extensive | match nsd
Additionally, the system logs will show repeated connection failure messages indicating that the daemon is unreachable.
An example of the log message indicating nsd is not responding is:
re0 alarmd 37538 - - ALARMD_CONNECTION_FAILURE: after 60 attempts nsd connect returned error: Connection refused
re0 alarmd 37538 - - ALARMD_CONNECTION_FAILURE: after 120 attempts nsd connect returned error: Connection refused
re0 alarmd 37538 - - ALARMD_CONNECTION_FAILURE: after 180 attempts nsd connect returned error: Connection refused
re0 alarmd 37538 - - ALARMD_CONNECTION_FAILURE: after 240 attempts nsd connect returned error: Connection refused
re0 alarmd 37538 - - ALARMD_CONNECTION_FAILURE: after 300 attempts nsd connect returned error: Connection refused
We are hitting a known issue addressed in the PR 1913560
NSD will not start, and this issue triggers when the Routing Engine is restarted and should not be seen on every reboot.
It is a timing issue during the bootup sequence. Not happen to see in every reboot.
For workaround, we have to start the NSD process manually:
>restart network-security
And confirm nsd is up with command
>show system processes extensive | match nsd
For more information, please find the below:
https://prsearch.juniper.net/problemreport/PR1913560