Description

 

This article describes a scenario in which the CPU utilization of the backup RE can be observed to be higher than that of the master RE due to a memory leak.

Symptoms

Customers can report a spike in memory utilization of the backup RE compared to the master RE.

 

user@RE0> show chassis routing-engine

Routing Engine status:

Slot 0:

Current state Master

Election priority Master (default)

Temperature 36 degrees C / 96 degrees F

CPU temperature 37 degrees C / 98 degrees F

DRAM 16324 MB (16384 MB installed)

Memory utilization 23 percent

5 sec CPU utilization:

User 1 percent

Background 0 percent

Kernel 3 percent

Interrupt 1 percent

Idle 95 percent

1 min CPU utilization:

User 2 percent

Background 0 percent

Kernel 3 percent

Interrupt 1 percent

Idle 94 percent

5 min CPU utilization:

User 2 percent

Background 0 percent

Kernel 3 percent

Interrupt 1 percent

Idle 94 percent

15 min CPU utilization:

User 2 percent

Background 0 percent

Kernel 3 percent

Interrupt 1 percent

Idle 94 percent

Model RE-S-1800x4

Serial ID 9009118072

Start time 2021-09-03 06:09:58 UTC

Uptime 942 days, 3 hours, 35 minutes, 40 seconds

Last reboot reason 0x100:reset from debugger

Load averages: 1 minute 5 minute 15 minute

0.33 0.28 0.25

Routing Engine status:

Slot 1:

Current state Backup

Election priority Backup (default)

Temperature 37 degrees C / 98 degrees F

CPU temperature 36 degrees C / 96 degrees F

DRAM 16324 MB (16384 MB installed)

Memory utilization 93 percent >>>>>>>>>>>>>>>>>>>>>>>>

5 sec CPU utilization:

User 1 percent

Background 0 percent

Kernel 12 percent

Interrupt 0 percent

Idle 87 percent

Model RE-S-1800x4

Serial ID 9009119518

Start time 2021-09-03 06:33:41 UTC

Uptime 942 days, 3 hours, 11 minutes, 49 seconds

Last reboot reason Router rebooted after a normal shutdown.

Load averages: 1 minute 5 minute 15 minute

0.89 0.76 0.65

Solution

We can extract a live RPD core from the backup RE to align with internal PR. If issues arise during the core extraction process, the following action plan can be implemented:

  1. Restart the RPD on the backup RE or reboot the backup RE in maintenance mode. This action will reduce memory utilization on the backup RE.
  2. Execute the "show task memory detail" command and verify the "RT" section. For example:

RT 459705281 14710570576 459705283 14710571264 4544301473

 

If you notice high numbers or a continuous increase in these values, indicative of the current device state, proceed to extract the core file from the device.

 

Extract the core file from the backup RE by running the command:

request system core-dump routing running

Modification History

2024-04-12 : Article Created