Description

High CPU was observed on FPC and RE when the traffic was shifted away

Symptoms

>Customer observed High CPU based on  jnxOperatingCPU metrics monitored .

 

The device was showing FPC CPU as 100% as seen from below command,

 

root@QFX5100-48s-6q> show chassis fpc no-forwarding

           Temp CPU Utilization (%)  CPU Utilization (%) Memory  Utilization (%)

Slot State (C)  Total Interrupt   1min  5min  15min DRAM(MB) Heap   Buffer

 0 Online    31   100    0 0     0     0   1920     0     70

 1 Empty

 

>Below is the "show chassis routing engine" command during issue time which shows high utilisation under user,

 

root@QFX5100-48s-6q> show chassis routing-engine no-forwarding

Routing Engine status:

 Slot 0:

  Current state         Master

  Temperature         31 degrees C / 87 degrees F

  CPU temperature       31 degrees C / 87 degrees F

  DRAM           1920 MB

  Memory utilization     70 percent

  5 sec CPU utilization:

   User           89 percent

   Background         0 percent

   Kernel          11 percent

   Interrupt         0 percent

   Idle            0 percent

  1 min CPU utilization:

   User           68 percent <<<<<<<<<<<<<<<<

   Background         0 percent

   Kernel          14 percent

   Interrupt         0 percent

   Idle           18 percent

  5 min CPU utilization:

   User           55 percent <<<<<<<<<<<<<<<<

   Background         0 percent

   Kernel          13 percent

   Interrupt         0 percent

   Idle           32 percent

  15 min CPU utilization:

   User           57 percent <<<<<<<<<<<<<<<<

   Background      0 percent

   Kernel         13 percent

   Interrupt       0 percent

   Idle            29 percent

  Model            QFX Routing Engine

  Serial ID         BUILTIN

  Uptime             140 days, 20 hours, 24 minutes, 9 seconds

  Last reboot reason    0x2000:hypervisor reboot

  Load averages:         1 minute  5 minute 15 minute

                     2.69    1.89    1.99

 

>>Upon checking "show system process extensive | no-more" output we see "mgd" process is consuming more cpu as seen below

 

 

root@QFX5100-48s-6q> show system processes extensive no-forwarding

 

last pid: 1843; load averages: 2.71, 1.90, 2.00 up 140+20:26:57  03:10:41

172 processes: 3 running, 147 sleeping, 1 zombie, 21 waiting

Mem: 794M Active, 113M Inact, 487M Wired, 220M Cache, 69M Buf, 234M Free

Swap: 1106M Total, 1106M Free

 

 PID USERNAME PRI NICE  SIZE  RES STATE  TIME  WCPU COMMAND

 

 1753 root   70  0 72196K 34816K RUN   0:02 30.68% mgd <<<<<<<<<<<<<

11840 root   43  -1 1255M  562M select 803.3H 17.77% fxpc

  10 root   155  52   0K  12K RUN  2322.2 13.96% idle

 1697 root   46  0 72224K 35068K select  0:02 12.39% mgd

 1759 root   53  0 65972K 52424K select  0:00 2.45% cli

 313 root   -8  0   0K  12K mdwait  0:01 1.37% md6

 1730 remote  47  0 66000K 52516K select  0:00 1.35% cli

 1692 remote  41  0 66008K 52520K select  0:00 0.74% cli

11840 root   -52 -52 1255M  562M select 803.3H 0.59% fxpc

 1760 root   71  0 72060K 34908K select  0:00 0.56% mgd

  6 root   -8  0   0K  12K -    12:19 0.34% g_down

 332 root   -8  0   0K  12K mdwait  0:01 0.34% md7

12013 root   40  0 53428K 39224K select 36.7H 0.20% mib2d

 1318 root   41  0 15680K 8016K select 473:27 0.15% eventd

  24 root   -80 -183   0K  12K WAIT   2:46 0.10% irq14: ata0

 273 root   -8  0   0K  12K mdwait  0:00 0.05% md4

12040 root   40  0 28968K 22380K select 23.7H 0.00% snmpd

 

 

Some of the reasons on why "mgd" process might be consuming more can be due to below reason,

 

  1. CLI sessions are not closed gracefully on the router. In this case, one would see mgd running high on CPU
  2. Multiple users or sessions being run in parallel executing commands (different PID or user # running similar or different commands which tend to spike the CPU as there will be multiple MGD daemons spawned)

 

 

Solution

  1. For high CPU , Please collect below outputs from the device in issue state multiple times,

 

 

show chassis fpc no-forwarding

show chassis routing-engine

show system processes extensive | no-more

show interfaces extensive | save /var/tmp/interface-stats

request pfe execute target fpc0 command "show threads”    <<<< check which thread occupy high cpu

request pfe execute target fpc0 command "show sched"

request pfe execute target fpc0 command "show pfe statistics traffic"

request pfe execute target fpc0 command "show pfe statistics notification"

 

Turn on task accounting and collect the task accounting detail output

(three times with a gap of 30 seconds).

Don't forget to turn it off once done.

 

set task accounting on

show task accounting detail

set task accounting off

 

show task memory detail

show task memeory summary

show task io

show task history

show task statistics

show task job

show task jobs

show krt queue

show krt state

 

 

2.Generate live core during issue state using root access

 

Login to the shell

 

ps -aux | grep mgd

gcore -c mgd.live-core.0 <pid>

 

 

If We are unable to collect the core and need to recover the device, kill the process consuming more cpu or reboot the box to resolve the issue

 

 

 

Modification History

2024-02-29 : Article Created