This KB explains the meaning of log message "kernel: rt_pfe_veto: Severe memory pressure, low-free-pages is below threshold. Op 4 err 55, rtsm_id 0:-1, msg type 5" seen on EX platforms and also lists down steps which can be used to find the cause and mitigate the same.
-- PFE veto messages
-- Protocol Flaps
-- JTASK_SCHED_SLIP_KEVENT logs
The logs are related to severe memory usage on the device and these logs inform us that excessive memory is in use on the device. Swap memory might be also used when the amount of physical memory (RAM) is full. If the system needs more memory resources and the RAM is full, inactive pages in memory are moved to the swap space. While swap space can help machines with a small amount of RAM, it should not be considered a replacement for more RAM. Swap space is located on hard drives, which have a slower access time than physical memory. Hence the below logs might be expected.
kernel: rt_pfe_veto: Possible slowest client is lacpd. States processed - 0. States to be processed - 11293645
kernel: rt_pfe_veto: Possible second slowest client is dcd. States processed - 11293568. States to be processed - 77
Follow the below steps to troubleshoot:-
Step-1:- Check if there were any changes performed on the device or in the network around the time when these logs started generating. Analyze the event as it will help to understand the trigger for severe memory pressure and usage.
Step 2:- Collect Multiple Iterations of the below Outputs as this will help to understand which process is utilizing the maximum memory
show system virtual memory (Execute in a gap of 5-10secs and at least 4-5times)
show system processes extensive | except 0.00 ((Execute in a gap of 5-10secs and at least 4-5times)
Step 3:- As a workaround, reboot the device to release swap memory/inactive memory and the default memory allocation will be returned.
Step 4:- Consider upgrading the device to the latest version which includes bug fixes/improvements to handle memory crunch situations better
If you have any further questions, contact JTAC
2024-09-08 - Initial Publication