Description

The system is generating repeated kernel log messages indicating an interrupt storm on IRQ10. This results in high CPU usage by the intr (interrupt) process. The issue stems from the control plane (RE/CB) hardware generating excessive interrupts, overwhelming the CPU.

Symptoms

1) Repeated syslog messages:

Mar 20 14:30:06 re0 kernel: interrupt storm detected on "irq10:"; throttling interrupt source

Mar 20 14:30:07 re0 kernel: interrupt storm detected on "irq10:"; throttling interrupt source

Mar 20 14:30:09 re0 kernel: interrupt storm detected on "irq10:"; throttling interrupt source

Mar 20 14:30:10 re0 kernel: interrupt storm detected on "irq10:"; throttling interrupt source

2) High CPU utilization observed in intr process:

> show system processes extensive no-forwarding

last pid: 92679; load averages: 0.15, 0.28, 0.36 up 540+06:55:36 01:28:16

362 processes: 5 running, 316 sleeping, 41 waiting

 

Mem: 1647M Active, 11G Inact, 1754M Wired, 419M Buf, 32G Free

Swap: 3072M Total, 3072M Free

 

 PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND

 18 root  -80 -  0K 656K WAIT 0 22.1H 91.00% intr{irq11: ichsmb0 acb0}

3) System performance degradation or delay in control plane operations.

Solution

The log message "interrupt storm detected on 'irq10:'; throttling interrupt source" indicates that system has detected an unusually high rate of hardware interrupts from the RE/CB on interrupt request line 10 (irq10). This condition is known as an interrupt storm.

 

An interrupt storm happens when an interrupt source (such as RE/CB hardware) generates an excessive number of interrupts.

 

As a workaround:

  • Perform a RE switchover to move mastership to the standby Routing Engine.
  • Reboot the affected RE.

After these actions, the interrupt storm messages should stop, and CPU utilization may returns to normal.

Modification History

2025-04-16 : Article Created