The following logs is printed continuously in the system log:
Sep 29 11:15:28.703 2024 ROUTER scpd[58582]: %DAEMON-3-JTASK_OS_MEMHIGH: Using 2584195 KB of memory, 80 percent of available
Sep 29 11:16:28.703 2024 ROUTER scpd[58582]: %DAEMON-3-JTASK_OS_MEMHIGH: Using 2584211 KB of memory, 80 percent of available
Sep 29 11:17:28.704 2024 ROUTER scpd[58582]: %DAEMON-3-JTASK_OS_MEMHIGH: Using 2584220 KB of memory, 80 percent of available
Sep 29 11:18:28.703 2024 ROUTER scpd[58582]: %DAEMON-3-JTASK_OS_MEMHIGH: Using 2584244 KB of memory, 80 percent of available
This occurs for the SCPD process (Satellite Control Plane Daemon) on the master routing-engine of Fusion AD (aggregator device)
Monitoring of the memory consumption by SCPD shows that it is high and possibly continuously increasing:
ROUTER> show system processes extensive | match scpd
58582 root 20 0 3570M 2820M kqread 2 32.8H 0.00% scpd
There is no functionality impact due to these logs. This is a sign of memory leak for the SCPD process.
One possible cause of this leak is the continuous flapping of EP ports (Extended Ports)
Sep 29 11:16:02.677 2024 ROUTER mib2d[58597]: %DAEMON-4-SNMP_TRAP_LINK_DOWN: ifIndex 66588792, ifAdminStatus up(1), ifOperStatus down(2), ifName ge-102/0/30
Sep 29 11:16:05.878 2024 ROUTER mib2d[58597]: %DAEMON-5-SNMP_TRAP_LINK_UP: ifIndex 66588792, ifAdminStatus up(1), ifOperStatus up(1), ifName ge-102/0/30
Sep 29 11:16:08.877 2024 ROUTER mib2d[58597]: %DAEMON-4-SNMP_TRAP_LINK_DOWN: ifIndex 66588792, ifAdminStatus up(1), ifOperStatus down(2), ifName ge-102/0/30
Sep 29 11:16:10.877 2024 ROUTER mib2d[58597]: %DAEMON-5-SNMP_TRAP_LINK_UP: ifIndex 66588792, ifAdminStatus up(1), ifOperStatus up(1), ifName ge-102/0/30
Sep 29 11:16:13.077 2024 ROUTER mib2d[58597]: %DAEMON-4-SNMP_TRAP_LINK_DOWN: ifIndex 66588792, ifAdminStatus up(1), ifOperStatus down(2), ifName ge-102/0/30
Sep 29 11:16:15.078 2024 ROUTER mib2d[58597]: %DAEMON-5-SNMP_TRAP_LINK_UP: ifIndex 66588792, ifAdminStatus up(1), ifOperStatus up(1), ifName ge-102/0/30
Sep 29 11:16:17.077 2024 ROUTER mib2d[58597]: %DAEMON-4-SNMP_TRAP_LINK_DOWN: ifIndex 66588792, ifAdminStatus up(1), ifOperStatus down(2), ifName ge-102/0/30
To stop the leak, the cause needs to be stopped (flapping of EP ports). Stopping the leak would not recover the extra memory allocated by SCPD.
To do that, either of the following can be done:
1- Restart the SCPD process from the CLI, in a maintenance window
USER@ROUTER> restart satellite-control-plane-process
Satellite Control Plane Process signalled but still running, waiting 28 seconds more
Satellite Control Plane Process started, pid 20351
USER@ROUTER>
2- GRES switchover, then restart the new backup routing-engine