This article walks you through troubleshooting high CPU utilization on Juniper Networks EX Series Switches.
The customer reports high CPU.
When you observe high CPU utilization, perform the following steps to understand the cause and isolate the source.
Depending on the load on the device, CPU utilization may vary. For example, if the device has several routes / MACs and routing protocols, high CPU can be expected.
Check the outputs of show chassis routing-engine from the Command Line Interface and top from shell.
show chassis routing-engine
top
root> show chassis routing-engine Routing Engine status: Slot 0: Current state Master Temperature 54 degrees C / 129 degrees F CPU temperature 54 degrees C / 129 degrees F DRAM 2048 MB Memory utilization 49 percent CPU utilization: User 1 percent Background 0 percent Kernel 3 percent Interrupt 0 percent Idle 96 percent Model EX4300-24T Serial ID PG3716110055 Start time 2018-06-28 03:06:24 UTC Uptime 9 days, 2 hours, 4 minutes, 35 seconds Last reboot reason Router rebooted after a normal shutdown. Load averages: 1 minute 5 minute 15 minute 2.34 2.89 2.94
The idle percentage depends on the device configuration and usage. High CPU might be expected and not cause an impact depending on the configured protocols. Always verify if the customer is reporting any impact. From the above output, you can determine which process is causing high CPU utilization. If PFEX, FXPC or L2CPD values are high it may mean that several MAC move / flood / STP related events are happening on the device. Chassisd spiking may mean that the issue is related to interface delete / reconfigure / temperature of device or some chassis operations. MGD means that some Junos Space / configuration / user login / netconf polling is hogging the CPU. MCSNOOPD means some multicast traffic is hogging the switch. MIB2D will indicate any issues with SNMP such as intensive polling. RPD will indicate any issues with routing updates, for example massive protocol updates. Rtinfo being high will mean there is high polling for route table information. Authd or dot1xd will indicate if there is a problem with dot1x. Jdhcpd being high indicate a problem with DHCP, it can be with the switch working as DHCP client, DHCP server or DHCP relay.cscript means event scripts being ran in the device are causing high CPU.
The idle percentage depends on the device configuration and usage. High CPU might be expected and not cause an impact depending on the configured protocols. Always verify if the customer is reporting any impact.
From the above output, you can determine which process is causing high CPU utilization.
If PFEX, FXPC or L2CPD values are high it may mean that several MAC move / flood / STP related events are happening on the device.
Chassisd spiking may mean that the issue is related to interface delete / reconfigure / temperature of device or some chassis operations.
MGD means that some Junos Space / configuration / user login / netconf polling is hogging the CPU.
MCSNOOPD means some multicast traffic is hogging the switch.
MIB2D will indicate any issues with SNMP such as intensive polling.
RPD will indicate any issues with routing updates, for example massive protocol updates. Rtinfo being high will mean there is high polling for route table information.
Authd or dot1xd will indicate if there is a problem with dot1x.
last pid: 12631; load averages: 2.34, 2.89, 2.94 up 9+02:06:20 05:12:14 70 processes: 2 running, 67 sleeping, 1 zombie CPU states: 0.6% user, 0.0% nice, 1.0% system, 0.0% interrupt, 98.4% idle Mem: 703M Active, 66M Inact, 118M Wired, 312M Cache, 112M Buf, 664M Free Swap: PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 1137 root 2 44 -52 560M 200M select 31.2H 16.75% pfex_junos 1235 root 1 107 0 18968K 11656K select 764:21 9.91% ppmd 1246 root 1 4 0 25124K 15616K kqread 472:03 4.98% l2cpd 1262 root 1 4 0 26636K 12876K kqread 149:09 3.03% mcsnoopd 1131 root 1 101 0 46708K 19952K select 111:26 0.10% chassisd 1250 root 1 96 0 86252K 42408K select 49:15 0.00% authd 1224 root 1 96 0 42264K 20400K select 21:58 0.00% mib2d 1138 root 1 4 0 10628K 4088K kqread 17:12 0.00% chassism 1227 root 1 96 0 41976K 26240K select 16:59 0.00% l2ald 1263 root 1 96 0 15464K 8912K select 11:20 0.00% license-check 1249 root 1 96 0 68636K 21988K select 8:57 0.00% jdhcpd 1231 root 1 96 0 29544K 13000K select 8:27 0.00% pfed 1223 root 1 96 0 31524K 19164K select 8:24 0.00% snmpd 1259 root 1 96 0 15192K 3912K select 7:05 0.00% shm-rtsdbd 1234 root 1 96 0 23732K 12656K select 3:37 0.00% cosd 1219 root 1 96 0 2728K 1164K select 3:02 0.00% bslockd
last pid: 12631; load averages: 2.34, 2.89, 2.94 up 9+02:06:20 05:12:14 70 processes: 2 running, 67 sleeping, 1 zombie CPU states: 0.6% user, 0.0% nice, 1.0% system, 0.0% interrupt, 98.4% idle Mem: 703M Active, 66M Inact, 118M Wired, 312M Cache, 112M Buf, 664M Free Swap:
PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 1137 root 2 44 -52 560M 200M select 31.2H 16.75% pfex_junos 1235 root 1 107 0 18968K 11656K select 764:21 9.91% ppmd 1246 root 1 4 0 25124K 15616K kqread 472:03 4.98% l2cpd 1262 root 1 4 0 26636K 12876K kqread 149:09 3.03% mcsnoopd 1131 root 1 101 0 46708K 19952K select 111:26 0.10% chassisd 1250 root 1 96 0 86252K 42408K select 49:15 0.00% authd 1224 root 1 96 0 42264K 20400K select 21:58 0.00% mib2d 1138 root 1 4 0 10628K 4088K kqread 17:12 0.00% chassism 1227 root 1 96 0 41976K 26240K select 16:59 0.00% l2ald 1263 root 1 96 0 15464K 8912K select 11:20 0.00% license-check 1249 root 1 96 0 68636K 21988K select 8:57 0.00% jdhcpd 1231 root 1 96 0 29544K 13000K select 8:27 0.00% pfed 1223 root 1 96 0 31524K 19164K select 8:24 0.00% snmpd 1259 root 1 96 0 15192K 3912K select 7:05 0.00% shm-rtsdbd 1234 root 1 96 0 23732K 12656K select 3:37 0.00% cosd 1219 root 1 96 0 2728K 1164K select 3:02 0.00% bslockd
show ethernet-switching mac-learning-log >>> Verify if MAC addresses are getting deleted and relearned very quickly. show system statistics bridge | match mac >>> Check the number of MAC address moves and learning. show system core-dumps >>> See if there are any cores for any daemons. show system alarms >>> Check for alarms. show chassis alarms >>> Check for alarms.
show ethernet-switching mac-learning-log >>> Verify if MAC addresses are getting deleted and relearned very quickly.
show system statistics bridge | match mac >>> Check the number of MAC address moves and learning.
show system core-dumps >>> See if there are any cores for any daemons.
show system alarms >>> Check for alarms.
show chassis alarms >>> Check for alarms.
Check for current logs and see if there are any abnormal logs or alarm messages. You can get the logs from Helloqa or from the customer. During high CPU events QBOT will not be able to poll data from the switch. Ask the customer to upload RSI and logs as needed. By abnormal messages we mean something like a process core dumping, memory being exhausted or a process not yielding.
See if there are any interfaces that are sending / receiving a high amount of traffic. If you notice any interface spiking, consider disabling the interface and verify CPU utilization again. You can verify this in the UI with the right permissions or with the customer during a call.
root> monitor interface traffic Interface Link Input packets (pps) Output packets (pps) ge-0/0/0 Down 226528864551 (0) 16685266 (0) gr-0/0/0 Up 0 (0) 0 (0) pfh-0/0/0 Up 0 0 ge-0/0/1 Down 0 (0) 0 (0) ge-0/0/2 Down 0 (0) 0 (0) ge-0/0/3 Down 0 (0) 0 (0) ge-0/0/4 Down 0 (0) 0 (0) ge-0/0/5 Down 0 (0) 0 (0) ge-0/0/6 Down 0 (0) 0 (0) ge-0/0/7 Down 0 (0) 0 (0) ge-0/0/8 Down 0 (0) 0 (0) ge-0/0/9 Down 0 (0) 0 (0) ge-0/0/10 Down 0 (0) 0 (0) ge-0/0/11 Down 0 (0) 0 (0) ge-0/0/12 Down 0 (0) 0 (0) ge-0/0/13 Down 0 (0) 0 (0) ge-0/0/14 Down 0 (0) 0 (0) ge-0/0/15 Down 0 (0) 0 (0) ge-0/0/16 Down 0 (0) 0 (0)
Note: If you notice some interfaces spiking and sending/receiving a high amount of traffic, do a port mirroring of the interface and see what kind of traffic is coming in. If you notice packets such as SSDP / MLD / IPV6 that are not expected, check why the interface is sending / receiving such traffic. You can create a filter to block it temporarily and verify if the CPU issue is resolved.
monitor traffic interface ge-0/0/0 no-resolve
tcpdump -i ge-0/0/0 -s 9000 -w /var/tmp/file.pcap
Now let’s say you identify some traffic / packets flooding in the above two methods. You could apply a filter on the interface to block this traffic. You could also apply this filter to the loopback interface.
Example with SNMP
[edit firewall family ethernet-switching] + filter BLOCK-SNMP { + term 1 { + from { + destination-port snmp; >>>We will be blocking only traffic with destination port SNMP. + ip-source-address { + 10.210.96.13/32; >>>Only traffic coming from this IP will be blocked (any other SNMP traffic coming from different IP will not be blocked.) + } + } + then { + discard; >>>>Whenever the filter is encountered, the packet will be discarded. + count SNMP-BLOCK; >>>> We include a counter to make sure that the SNMP packets are hitting the filter. + } + } + term 2 { + then accept; >>>> We include a second term with "accept" so that any other traffic is accepted and doesn’t get affected by the filter. + } + }
The outputs of the following commands would also be very helpful in understanding the cause of high CPU utilization. Contact Support if further assistance is required. Remember to always capture timestamps of the incident. You can use from the operational mode 'set cli timestamp'.
show system uptime show chassis routing-engine show system process extensive show task memory detail show task io show system buffers set task accounting on show task accounting detail show task jobs show task io show krt queue show krt state show system process extensive
show system uptime
show system process extensive
show task memory detail
show task io
show system buffers
set task accounting on
show task accounting detail
show task jobs
show krt queue
show krt state
Wait for 30 seconds and execute the above commands a few times, and then run the following command:
set task accounting off
When you have intermittent high CPU in usage you can also configure a small script to capture some data as needed when a user-defined CPU threshold is reached. For example:
set snmp rmon alarm 1 interval 10
set snmp rmon alarm 1 variable jnxOperatingCPU.9.1.0.0
set snmp rmon alarm 1 sample-type absolute-value
set snmp rmon alarm 1 rising-threshold 75 >>>>>>>>>>>>>>>>>>>>> Threshold
set snmp rmon alarm 1 rising-event-index 1
set snmp rmon event 1 type log
set event-options policy test events snmpd_rmon_eventlog
set event-options policy test then execute-commands commands "show system processes extensive"
set event-options policy test then execute-commands commands "show system virtual-memory"
set event-options policy test then execute-commands commands "set task accounting on"
set event-options policy test then execute-commands commands "show task accounting detail"
set event-options policy test then execute-commands commands "set task accounting off"
set event-options policy test then execute-commands commands "show task memory detail"
set event-options policy test then execute-commands commands "show task memory summary"
set event-options policy test then execute-commands commands "show task io"
set event-options policy test then execute-commands commands "show task history"
set event-options policy test then execute-commands commands "show task statistics"
set event-options policy test then execute-commands commands "show task job"
set event-options policy test then execute-commands commands "show task jobs"
set event-options policy test then execute-commands commands "show krt queue"
set event-options policy test then execute-commands output-filename CPU-high
set event-options policy test then execute-commands destination var-log
set event-options destinations var-log archive-sites /var/log
Something important to note is that the script above will increment the CPU usage by some amount so it is expected to see this. The process that will be executing the script is cscript. Running 'show system processes extensive is more CPU intensive than running 'top' from shell.
2020-08-06: Article reviewed for accuracy; no changes required.
2025-07-09: Article updated.
2025-12-01: Typo in Solution corrected