Description

SRX1500 chassis cluster node 1 routing engine CPU high:

 

root@HXJY_B02_IDC_SRX1500_01> show chassis cluster status

 

Monitor Failure codes:

  CS Cold Sync monitoring    FL Fabric Connection monitoring

  GR GRES monitoring       HW Hardware monitoring

  IF Interface monitoring    IP IP monitoring

  LB Loopback monitoring     MB Mbuf monitoring

  NH Nexthop monitoring     NP NPC monitoring

  SP SPU monitoring       SM Schedule monitoring

  CF Config Sync monitoring   RE Relinquish monitoring

  IS IRQ storm

 

Cluster ID: 12

Node  Priority Status        Preempt Manual  Monitor-failures

 

Redundancy group: 0 , Failover count: 1

node0 200   primary       no   no    None

node1 100   secondary      no   no    None

 

Redundancy group: 1 , Failover count: 19

node0 200   primary       no   no    None

node1 100   secondary      no   no    None

 

root@HXJY_B02_IDC_SRX1500_02> show chassis routing-engine no-forwarding

 

Routing Engine status:

  Temperature         38 degrees C / 100 degrees F

  CPU temperature       38 degrees C / 100 degrees F

  Total memory       1900 MB Max  342 MB used ( 18 percent)

  Memory utilization     17 percent

  5 sec CPU utilization:

   User           11 percent

   Background         0 percent

   Kernel          44 percent

   Interrupt         45 percent

   Idle            0 percent

  1 min CPU utilization:

   User           11 percent

   Background         0 percent

   Kernel          42 percent

   Interrupt         46 percent

   Idle            0 percent

  5 min CPU utilization:

   User           11 percent

   Background         0 percent

   Kernel          37 percent

   Interrupt         49 percent

   Idle            3 percent

  15 min CPU utilization:

   User           10 percent

   Background         0 percent

   Kernel          33 percent

   Interrupt         49 percent

   Idle            9 percent

  Model             SRX Routing Engine

  Serial ID           BUILTIN

  Start time           2023-12-03 20:30:01 CST

  Uptime             234 days, 14 hours, 10 minutes, 23 seconds

  Last reboot reason       0x4000:VJUNOS reboot

  Load averages:         1 minute  5 minute 15 minute

                    2.84    2.79    2.61

 

root@HXJY_B02_IDC_SRX1500_02> show system processes extensive no-forwarding

 

last pid: 37224; load averages: 3.10, 2.84, 2.63 up 234+14:10:26  10:40:27

325 threads:  3 running, 262 sleeping, 1 zombie, 59 waiting

CPU: 0.7% user, 0.0% nice, 2.7% system, 1.6% interrupt, 95.0% idle

Mem: 87M Active, 1495M Inact, 251M Wired, 32M Buf, 66M Free

Swap: 1639M Total, 1639M Free

 

 PID USERNAME  PRI NICE  SIZE  RES STATE  TIME  WCPU COMMAND

  12 root    -92  -   0B  944K WAIT  302:49 38.67% intr{irq11: em0:irq0+}

18934 root     80  0  726M  15M RUN  164.1H 28.37% eventd

11917 root     -8  -   0B  16K mdwait  0:01  3.56% md37

 7535 root     -8  -   0B  16K mdwait  0:02  2.10% md19

  12 root    -92  -   0B  944K WAIT  49:14  1.27% intr{irq259: virtio_pci1}

36997 root     21  0  86M  69M select  0:01  1.27% cli

36998 root     52  0  738M  41M select  0:00  1.07% mgd

10417 root     -8  -   0B  16K mdwait  0:01  0.98% md26

11918 root     -8  -   0B  16K wrkwai  0:00  0.88% md37.uzip

  12 root    -88  -   0B  944K WAIT   0:01  0.78% intr{irq268: virtio_pci4}

 6189 root     -8  -   0B  16K mdwait  0:05  0.68% md13

 7536 root     -8  -   0B  16K wrkwai  0:02  0.39% md19.uzip

 

Solution

Upon checking the node 1 log files, there are lots of below messages in the log file:

 

Jul 25 10:15:00 HXJY_B02_IDC_SRX1500_02 newsyslog[36433]: logfile turned over due to size>5120K

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: could not bind to address 10.9.254.3: Can't assign requested address

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: Trying bind to default address: Inappropriate ioctl for device

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: bind: Invalid argument

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: could not bind to address 10.9.254.3: Can't assign requested address

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: Trying bind to default address: Can't assign requested address

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: bind: Invalid argument

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: could not bind to address 10.9.254.3: Can't assign requested address

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: Trying bind to default address: Can't assign requested address

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: bind: Invalid argument

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: could not bind to address 10.9.254.3: Can't assign requested address

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: Trying bind to default address: Can't assign requested address

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: bind: Invalid argument

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: could not bind to address 10.9.254.3: Can't assign requested address

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: Trying bind to default address: Can't assign requested address

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: bind: Invalid argument

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: could not bind to address 10.9.254.3: Can't assign requested address

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: Trying bind to default address: Can't assign requested address

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: bind: Invalid argument

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: could not bind to address 10.9.254.3: Can't assign requested address

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: Trying bind to default address: Can't assign requested address

Jul 25 10:15:00 2024 HXJY_B02_IDC_SRX1500_02 eventd: %SYSLOG-3: bind: Invalid argument

 

  syslog {

    archive size 5m files 10;

    user * {

      any emergency;

    }

    host 10.6.60.139 {

      any info;

    }

    file interactive-commands {

      interactive-commands any;

    }

    file messages {

      any notice;

      authorization info;

      explicit-priority;

    }

    time-format year;

     source-address 10.9.254.3; ----------------------- Remove this part of configuration then the backup node RE doesn't report the above logs any more and also routing engine CPU utilization back to normal

Modification History

2024-07-31 : Article Created