Description

EX4300 platform hangs due to low memory and displays continuously KERNEL_MEMORY_CRITICAL: System low on free memory|low_mem_signal_processes and Processing low memory signal

Symptoms

We see logs like below in console/messages :  
May 1 18:28:25 2024 lab mgd[67050]: UI_CMDLINE_READ_LINE: User 'p2186821', command 'show log messages | match "KERNEL_MEMORY_CRITICAL: System low on free memory|low_mem_signal_processes:" '
May 1 18:28:41 2024 lab mgd[67050]: UI_CMDLINE_READ_LINE: User 'p2186821', command 'show log messages | match "KERNEL_MEMORY_CRITICAL: System low on free memory|low_mem_signal_processes:" '
May 1 19:35:42 2024 lab mgd[67205]: UI_CMDLINE_READ_LINE: User 'P2971117', command 'show log messages | match "KERNEL_MEMORY_CRITICAL: System low on free memory|low_mem_signal_processes:" '
May 1 19:58:16 2024 lab mgd[67276]: UI_CMDLINE_READ_LINE: User 'P2971117', command 'show log messages | match "KERNEL_MEMORY_CRITICAL: System low on free memory|low_mem_signal_processes:" '
May 1 20:33:18 2024 lab /kernel: KERNEL_MEMORY_CRITICAL: System low on free memory, notifying init (#117)
May 1 20:33:18 2024 lab rpd[1757]: Processing low memory signal
May 1 20:33:18 2024 lab l2cpd[1774]: Processing low memory signal
 
These error happens due to the increase in number of ssh sessions which results in spike in memory utilization
 

Solution

Power cycle the device would restore the device and issue would be resolved. But the issue can happen again when the number of SSH sessions are increased which results in spike in memory utilization

>>> It is recommended to limit the number of ssh session inorder to fix the issue 

# set system services ssh connection-limit 7
  
   services {

       ssh {

           root-login deny;

           protocol-version v2;

           connection-limit 7; <<<<<<<<<<<<<<<<<

       }

       netconf {

           ssh;

       }

   }

Modification History

2024-05-16 : Article Created