Description

Customer is reporting HIGH CPU on RE on SRX1500

Symptoms

We can check the RE utilization by issue the following command in CLI:

 

root@test> show chassis routing-engine no-forwarding

 

Routing Engine status:

  Temperature         39 degrees C / 102 degrees F

  CPU temperature       39 degrees C / 102 degrees F

  Total memory       1954 MB Max   0 MB used ( 0 percent)

  Memory utilization     37 percent

  5 sec CPU utilization:

   User           76 percent

   Background         0 percent

   Kernel          15 percent

   Interrupt         7 percent

   Idle            2 percent. <<<<<<<<<<

  1 min CPU utilization:

   User           70 percent

   Background         0 percent

   Kernel           8 percent

   Interrupt         10 percent

   Idle           12 percent. <<<<<<<<<<<<

  5 min CPU utilization:

   User           71 percent

   Background         0 percent

   Kernel           6 percent

   Interrupt         5 percent

   Idle           19 percent. <<<<<<<<<<

  15 min CPU utilization:

   User           71 percent

   Background         0 percent

   Kernel           5 percent

   Interrupt         4 percent

   Idle           21 percent. <<<<<<<<<

  Model             SRX Routing Engine

  Serial ID           BUILTIN

  Start time           2018-08-16 03:49:50 UTC

  Uptime             1980 days, 10 hours, 5 minutes, 9 seconds

  Last reboot reason       0x4000:VJUNOS reboot

  Load averages:         1 minute  5 minute 15 minute

                    1.71    1.12    0.99

 

It can be seen that the RE utilization is more that 80%. Based on the upper output, we can check the system processes by using the command :show system processes extensive no-forwarding

 

root@test> show system processes extensive no-forwarding

 

last pid: 81992; load averages: 1.81, 1.15, 1.00 up 1980+10:05:41  13:55:01

182 processes: 5 running, 156 sleeping, 2 zombie, 19 waiting

 

Mem: 571M Active, 103M Inact, 105M Wired, 937M Cache, 69M Buf, 185M Free

Swap: 615M Total, 615M Free

 

 

 PID USERNAME   THR PRI NICE  SIZE  RES STATE  TIME  WCPU COMMAND

 1328 root      1 138  0 6008K 3808K RUN    ??? 66.46% eventd. <<<<<<<<

  11 root      1 -40 -159   0K  12K WAIT  154.0H 6.98% swi2: netisr 0

  24 root      1 -68 -187   0K  12K WAIT  716.5H 3.96% irq11: uhci0 em3++*

  10 root      1 171  52   0K  12K RUN    ??? 2.49% idle: cpu0

81236 root      1 76  0 51792K 23276K select  0:01 0.98% mgd

81885 root      1 77  0 31936K 25816K select  0:00 0.49% cli

81886 root      1 137  0 51768K 23432K select  0:00 0.49% mgd

81992 root      1 137  0 22740K 1968K RUN   0:00 0.49% top

 

We can see that eventd is the processes that is using about 67% of the RE. Checking the configuration, found that there are security flow traceoptions configured:

 

 

security {

flow {

    traceoptions {

      file flowTrace size 1000000 files 3;

      flag basic-datapath;

      packet-filter t1 {

        source-prefix 10.10.10.10/32;

        destination-prefix 192.168.181.1/32;

      }

    }

  }

}

Solution

Delete security flow traceoptions from the configuration

Modification History

2024-01-30 : Article Created