Description

When using J-Web for devices management, high CPU usage on Routing Engine (RE) may occur.

Symptoms

When using J-Web for devices management, high CPU usage on the RE is observed.  High CPU usage on the httpd process, stale mgd and php processes may also ocurr.  J-Web may be unable to respond. The command, 'show system processes extensive' was used to check it: 


root@SRX> show system processes extensive no-forwarding

last pid: 6519; load averages: 1.59, 1.20, 1.09 up 3+22:26:04 02:06:19
147 processes: 19 running, 116 sleeping, 1 zombie, 11 waiting

Mem: 199M Active, 171M Inact, 1087M Wired, 164M Cache, 112M Buf, 352M Free
Swap:

PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
1596 root 7 76 0 1025M 79608K select 0 293.8H 285.55% flowd_octeon_hm
1654 nobody 2 -8 0 11060K 4360K piperd 0 20.7H 49.61% httpd
.....

root@SRX> show system processes extensive | match mgd
14997 root 1 76 0 39428K 1280K select 0 0:00 0.00% mgd
38401 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
15007 root 1 76 0 39428K 1280K select 0 0:00 0.00% mgd
48580 root 1 139 0 39428K 1360K select 0 0:00 0.00% mgd
19125 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
61200 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
30709 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
71845 root 1 76 0 39428K 1356K select 0 0:00 0.00% mgd
52989 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
65078 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
32996 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
53019 root 1 139 0 39428K 1356K select 0 0:00 0.00% mgd
14999 root 1 76 0 39428K 1280K select 0 0:00 0.00% mgd
35784 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
34562 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
47296 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
23957 root 1 76 0 39428K 1288K select 0 0:00 0.00% mgd
34607 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
25713 root 1 76 0 39428K 1288K select 0 0:00 0.00% mgd
29488 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
28221 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
65969 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
30778 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
28227 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
52995 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
51760 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
66456 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
45873 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
52991 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
52123 root 1 139 0 39428K 1360K select 0 0:00 0.00% mgd
52993 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
45396 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
5897 root 1 76 0 39428K 1276K select 0 0:00 0.00% mgd
52999 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
63706 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
68273 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
39855 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
714 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
52997 root 1 76 0 39428K 1360K select 0 0:00 0.00% mgd
65046 root 1 76 0 39428K 1356K select 0 0:00 0.00% mgd
53035 root 1 139 0 39428K 1360K select 0 0:00 0.00% mgd

root@SRX> show system processes extensive | match php
15000 nobody 1 4 0 16488K 7936K sbwait 0 0:00 0.00% php
52124 nobody 1 4 0 16488K 8580K sbwait 0 0:00 0.00% php
34608 nobody 1 4 0 16276K 8104K sbwait 0 0:00 0.00% php
30779 nobody 1 4 0 16244K 8124K sbwait 0 0:00 0.00% php
47297 nobody 1 4 0 16276K 8128K sbwait 0 0:00 0.00% php
65970 nobody 1 4 0 16220K 8380K sbwait 0 0:00 0.00% php
52992 nobody 1 4 0 16220K 8300K sbwait 0 0:00 0.00% php
30712 nobody 1 4 0 16244K 8108K sbwait 0 0:00 0.00% php
52990 nobody 1 4 0 16220K 8316K sbwait 0 0:00 0.00% php
53020 nobody 1 4 0 16220K 8300K sbwait 0 0:00 0.00% php
53000 nobody 1 4 0 16220K 8300K sbwait 0 0:00 0.00% php
53040 nobody 1 4 0 16276K 8100K sbwait 0 0:00 0.00% php
71846 nobody 1 4 0 16276K 8184K sbwait 0 0:00 0.00% php
45874 nobody 1 4 0 16244K 8120K sbwait 0 0:00 0.00% php
35785 nobody 1 4 0 16276K 8112K sbwait 0 0:00 0.00% php
14998 nobody 1 4 0 16392K 7764K sbwait 0 0:00 0.00% php
52996 nobody 1 4 0 16392K 8412K sbwait 0 0:00 0.00% php
52994 nobody 1 4 0 16392K 8420K sbwait 0 0:00 0.00% php
63707 nobody 1 4 0 16276K 8188K sbwait 0 0:00 0.00% php
65047 nobody 1 4 0 16212K 8168K sbwait 0 0:00 0.00% php
32997 nobody 1 4 0 15956K 8064K sbwait 0 0:00 0.00% php
25714 nobody 1 4 0 16276K 7464K sbwait 0 0:00 0.00% php
52998 nobody 1 4 0 16392K 8412K sbwait 0 0:00 0.00% php
34563 nobody 1 4 0 16276K 8104K sbwait 0 0:00 0.00% php
715 nobody 1 4 0 16276K 8180K sbwait 0 0:00 0.00% php
38403 nobody 1 4 0 16276K 8108K sbwait 0 0:00 0.00% php
29489 nobody 1 4 0 16244K 7964K sbwait 0 0:00 0.00% php
28228 nobody 1 4 0 16276K 7888K sbwait 0 0:00 0.00% php
51761 nobody 1 4 0 15956K 8068K sbwait 0 0:00 0.00% php
28222 nobody 1 4 0 16276K 7888K sbwait 0 0:00 0.00% php
66457 nobody 1 4 0 15956K 8152K sbwait 0 0:00 0.00% php
45397 nobody 1 4 0 16212K 8084K sbwait 0 0:00 0.00% php
65079 nobody 1 4 0 16212K 8168K sbwait 0 0:00 0.00% php
23958 nobody 1 4 0 16276K 7452K sbwait 0 0:00 0.00% php
19126 nobody 1 4 0 16276K 8192K sbwait 0 0:00 0.00% php
39856 nobody 1 4 0 15956K 8156K sbwait 0 0:00 0.00% php
68274 nobody 1 4 0 16276K 8188K sbwait 0 0:00 0.00% php
15009 nobody 1 4 0 16276K 7456K sbwait 0 0:00 0.00% php
48581 nobody 1 4 0 16212K 8088K sbwait 0 0:00 0.00% php
61201 nobody 1 4 0 15956K 8072K sbwait 0 0:00 0.00% php
5896 nobody 1 4 0 16212K 7396K sbwait 0 0:00 0.00% php

Solution

On all SRX Series devices, when using J-Web for devices management, the mgd process might get stuck, and high CPU usage on Routing Engine (RE) may occur.  When one or multiple J-Web sessions are established in a browser after navigating through different tabs and multiple PHP processes remain, this causes high CPU usage on the RE.

This issue is resolved in the Junos versions below. Please upgrade to a fixed version:

  • 12.1X46-D56
  • 12.1X46-D60
  • 12.3X48-D36
  • 12.3X48-D40
  • 15.1X49-D62
  • 15.1X49-D70

Workaround:

Take the following steps to restore service:
  1. Restart web-management

    user@srx> restart web-management
  2. Use ‘ps -aux | grep php’ to find the long run php process. Even though J-Web is idle, end the php process by issuing ‘kill -9’

    root@SRX> start shell user root
    root@SRX% ps -aux | grep php
    root@SRX% kill -9 <<<--- kill -9 +PID of the process

Related Information