This is caused by a Power outage + multiple reboots that left the Unit in an unclean state

The console screen has this constant message - Process secure communication manager has exited unexpectedly within the threshold time of 250(ms) after being restarted 2 times.

The status of 128T (output below) is: Caught exception in main: Public key for node is not in /etc/128technology/ssh/pdc_ssh_key.pub

$ systemctl status 128T -l

● 128T.service - 128T service

  Loaded: loaded (/usr/lib/systemd/system/128T.service; enabled; vendor preset: disabled)

  Active: active (running) since Thu 2026-01-01 20:55:01 UTC; 23s ago

 Process: 34615 ExecStartPre=/sbin/sysctl -p /etc/sysctl.d/95-sysctl.conf (code=exited, status=0/SUCCESS)

 Process: 34531 ExecStartPre=/usr/bin/minionStarter (code=exited, status=0/SUCCESS)

Main PID: 34618 (processManager)

   Tasks: 54

  Memory: 104.6M

  CGroup: /system.slice/128T.service

          ├─34618 /usr/bin/processManager

          ├─34743 /usr/bin/java -Dlog4j.configuration=file:///etc/128technology/log4j.properties -Dzookeeper.root.logger=ERROR -Dzookeeper.log.dir=/var/log/128technology -Dznode.container.checkIntervalMs=10000 -cp /usr/lib64/*:/usr/lib/* -Dzookeeper.nio.numSelectorThreads=1 -Dzookeeper.nio.numWorkerThreads=2 -Dzookeeper.commitProcessor.numWorkerThreads=1 -Djute.maxbuffer=83886064 -Dzookeeper.cnxTimeout=5000 -Dzookeeper.tcpKeepAlive=false -Xmx200m org.apache.zookeeper.server.quorum.QuorumPeerMain /var/lib/128technology/pds.cfg

          └─34779 /usr/bin/configDirector --startup-log-level Info

Jan 01 20:55:02 ar36240002 secureCommunicationManager[34734]: Failed to get pid for: SSHD-127.0.0.1:931 Process SSHD-127.0.0.1:931 is not being managed

Jan 01 20:55:02 ar36240002 processManager[34618]: Secure communication Manager: Caught exception in main: Public key for node is not in /etc/128technology/ssh/pdc_ssh_key.pub

Jan 01 20:55:02 ar36240002 secureCommunicationManager[34847]: Failed to get pid for: SSHD-0.0.0.0:930 Process SSHD-0.0.0.0:930 is not being managed

Jan 01 20:55:02 ar36240002 secureCommunicationManager[34847]: Failed to get pid for: SSHD-127.0.0.1:931 Process SSHD-127.0.0.1:931 is not being managed

Jan 01 20:55:02 ar36240002 processManager[34618]: Secure communication Manager: Caught exception in main: Public key for node is not in /etc/128technology/ssh/pdc_ssh_key.pub

Jan 01 20:55:03 ar36240002 secureCommunicationManager[34880]: Failed to get pid for: SSHD-0.0.0.0:930 Process SSHD-0.0.0.0:930 is not being managed

Jan 01 20:55:03 ar36240002 secureCommunicationManager[34880]: Failed to get pid for: SSHD-127.0.0.1:931 Process SSHD-127.0.0.1:931 is not being managed

Jan 01 20:55:03 ar36240002 processManager[34618]: Secure communication Manager: Caught exception in main: Public key for node is not in /etc/128technology/ssh/pdc_ssh_key.pub

Jan 01 20:55:03 ar36240002 processManager[34618]: Process Secure communication Manager (secureCommunicationManager:34880) has exited unexpectedly within the threshold time of 250(ms) after being restarted 2 times.

Both SSR 1200 are not passing traffic after power outage, Both SSR's are managed via Mist

Console screen has this constant message - Process secure communication manager has exited unexpectedly within the threshold time of 250(ms) after being restarted 2 times.

Key error:

log processManager[97955]: Secure communication Manager: Caught exception in main: Public key for node is not in /etc/128technology/ssh/pdc_ssh_key.pub

.

Resolution:

1. Verify various pdc_* files were all zero byte files
   1. cd /etc/128technology/ssh followed by ls -l 
2. Remove files 
   1. rm pdc_*
3. Regenerate keys
   1. ssh-keygen -t rsa -N "" -f pdc_ssh_key
4. Copy contents of pdc_ssh_key.pub from each node and put that in the authorized_keys of both (both public keys on both systems)