Description

Satellite devices might stuck at "SyncWait" status.

Customers may report traffic impact on clients connecting to satellite devices connecting to the AD node which shows every satellite device "SyncWait".

Symptoms

The status of satellite devices on the affected AD node is "SyncWait" when the issue occurs.

Triggers:
This issue might be seen if the following conditions are met:
* On Junos MX, EX, and QFX platforms * Junos Fusion Satellite device * Rebooting the device
* Upon Upgrading the Aggregation Device to the 21.4 release

Example output:

{master} show system processes extensive | match spmd 
17559 root           52    0   742M 13156K wait    0   0:00   0.00% spmd

RE0> show chassis hardware satellite 
error: the satellite-platform-management-process subsystem is not running


root@Fusion-AD-2> show chassis satellite

                    Device          Cascade      Port       Extended Ports
Alias            Slot   State           Ports        State      Total/Up
SD-L7_FPC:71     71     SyncWait        et-72/1/1    present    51/0   <<<<<<<<<<
                                        et-73/1/0    present
                                        ae0*         backup
SD-L7_FPC:72     72     SyncWait        et-71/1/0    present    50/0   <<<<<<<<<<
                                        et-73/1/1    present
                                        ae0*         backup
SD-L7_FPC:73     73     SyncWait        et-72/1/0    present    51/0   <<<<<<<<<<
                                        et-71/1/1    present
                                        xe-0/0/0:3   present
                                        ae0*         backup
SD-L8_FPC:81     81     SyncWait        et-82/1/1    present    51/0   <<<<<<<<<<
                                        et-83/1/0    present
                                        ae0*         backup
<Output Truncated>

Solution

We are aware that some customers have encountered an issue related to public PRs PR1733558 and PR1682680. This issue occurs when satellite devices attempt to generate new SSH keys without first clearing the old ones, which can result in unexpected behavior.

The temporary workaround for this issue is to restart or kill the spmd process on the affected Aggregation Device (AD). This can be done using the following command:

restart satellite-platform-management-process

 

OR

 

show system processes extensive | match spmd
pid root           52    0   742M 13156K wait    0   0:00   0.00% spmd


start shell user root
kill -9 pid

 

This should help mitigate the issue by ensuring the old SSH keys are cleared before generating new ones.

 

External PR Links:

PR1733558
PR1682680

 

Further Assistance:

If the issue remains unresolved, please contact Juniper Technical Support for further investigation.

Modification History

2024-02-01 : Article Created