Description

A PTX10001-36 with 1,000,000 routes:

 

labroot@e05-40> show route summary   

Autonomous system number: 65001

Router ID: 1.1.1.1

 

Highwater Mark (All time / Time averaged watermark)

  RIB unique destination routes: 1010024 at 2024-04-17 13:12:14 / 0

  RIB routes          : 1010024 at 2024-04-17 13:12:14 / 0

  FIB routes          : 1010006 at 2024-04-17 13:12:14 / 0

  VRF type routing instances  : 0 at 2024-04-17 10:54:51

 

inet.0: 1010009 destinations, 1010009 routes (1010009 active, 0 holddown, 0 hidden)

       Direct:   3 routes,   3 active

        Local:   2 routes,   2 active

        OSPF: 10003 routes, 10003 active

         BGP: 1000000 routes, 1000000 active

       Static:   1 routes,   1 active

  

Sees a CPU spike for EvoAftManBt-main to 99.9% if the 'show pfe route ip' command is run then interrupted via Ctrl_C:

 

labroot@e05-40> show pfe route ip   

Apr 17 13:33:13

^C[abort]

 

labroot@e05-40> 

 

CPU goes to 99.9%:

 

---(refreshed at 2024-04-17 13:33:11 PDT)---

-------------------------------

node: re0

-------------------------------

top - 13:33:12 up 2:39, 4 users, load average: 1.46, 1.49, 1.68

Threads: 2018 total,  3 running, 2014 sleeping,  0 stopped,  1 zombie

%Cpu(s): 5.2 us, 6.3 sy, 0.0 ni, 88.2 id, 0.0 wa, 0.2 hi, 0.2 si, 0.0 st

MiB Mem : 63148.3 total, 41817.9 free, 13820.6 used,  7509.7 buff/cache

MiB Swap:  4096.0 total,  4096.0 free,   0.0 used. 47523.0 avail Mem

 

 PID USER    TID PRI NI  SIZE   RSS S   TIME %MEM %CPU COMMAND

21801 root   21801 19  0 3028092 3105356 R 00:06:25 4.8 95.0 EvoAftManBt-mai{EvoAftManBt-mai}

 7961 root   20585 19  0  51420  36092 S 00:02:13 0.1 0.0 marvd{marvd}

 7987 root   20807 19  0  58324  80156 S 00:01:21 0.1 0.0 marvell-cpss-ap{luaCLI_0}

10122 root   10122 19  0 1069868 2319956 S 00:04:59 3.6 0.0 fibd{fibd}

10480 root   19733 19  0 353828 253584 S 00:02:46 0.4 0.0 evo-cda-bt{AftTimer-0}

10480 root   19734 19  0 353828 253584 S 00:02:46 0.4 0.0 evo-cda-bt{AftTimer-1}

10480 root   19735 41  - 353828 253584 S 00:02:44 0.4 0.0 evo-cda-bt{QPollMainThd}

10480 root   19752 19  0 353828 253584 S 00:04:31 0.4 0.0 evo-cda-bt{WorkChannel-0}

10694 root   10694 19  0 110288 193028 S 00:01:31 0.3 0.0 fabspoked-fchip{fabspoked-fchip}

11135 root   11135 19  0 119308 171788 S 00:01:02 0.3 0.0 hwdre{hwdre}

12523 root   12523 19  0 184924  50376 S 00:01:03 0.1 0.0 trace-writer{trace-writer}

13875 root   13875 19  0  75784 874648 S 00:01:10 1.4 0.0 objmon{objmon}

14372 root   14372 19  0 157464 239804 S 00:01:14 0.4 0.0 picd{picd}

14756 root   14756 19  0 1536572 922928 S 00:01:04 1.4 0.0 rpd{rpd}

21801 root   22334 19  0 3028092 3105356 S 00:01:13 4.8 0.0 EvoAftManBt-mai{AftTimer-0}

21801 root   22335 19  0 3028092 3105356 S 00:01:17 4.8 0.0 EvoAftManBt-mai{AftTimer-1}

21801 root   22336 19  0 3028092 3105356 S 00:01:14 4.8 0.0 EvoAftManBt-mai{AftTimer-2}

 

And it stays that way for over 4 minutes:

 

---(refreshed at 2024-04-17 13:37:26 PDT)---

-------------------------------

node: re0

-------------------------------

top - 13:37:26 up 2:43, 4 users, load average: 2.46, 2.03, 1.86

Threads: 2015 total,  2 running, 2013 sleeping,  0 stopped,  0 zombie

%Cpu(s): 4.1 us, 1.8 sy, 0.0 ni, 94.1 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st

MiB Mem : 63148.3 total, 41811.1 free, 13821.5 used,  7515.7 buff/cache

MiB Swap:  4096.0 total,  4096.0 free,   0.0 used. 47521.8 avail Mem

 

 PID USER    TID PRI NI  SIZE   RSS S   TIME %MEM %CPU COMMAND

21801 root   21801 19  0 3028092 3105356 R 00:10:38 4.8 99.9 EvoAftManBt-mai{EvoAftManBt-mai}

 7961 root   20585 19  0  51420  36092 S 00:02:15 0.1 0.0 marvd{marvd}

 7987 root   20807 19  0  58324  80156 S 00:01:23 0.1 0.0 marvell-cpss-ap{luaCLI_0}

10122 root   10122 19  0 1069868 2319956 S 00:04:59 3.6 0.0 fibd{fibd}

10480 root   19733 19  0 353828 253584 S 00:02:50 0.4 0.0 evo-cda-bt{AftTimer-0}

10480 root   19734 19  0 353828 253584 S 00:02:50 0.4 0.0 evo-cda-bt{AftTimer-1}

10480 root   19735 41  - 353828 253584 S 00:02:48 0.4 0.0 evo-cda-bt{QPollMainThd}

10480 root   19752 19  0 353828 253584 S 00:04:38 0.4 0.0 evo-cda-bt{WorkChannel-0}

10694 root   10694 19  0 110288 193028 S 00:01:33 0.3 0.0 fabspoked-fchip{fabspoked-fchip}

11135 root   11135 19  0 119308 172188 S 00:01:03 0.3 0.0 hwdre{hwdre}

12523 root   12523 19  0 184924  50376 S 00:01:03 0.1 0.0 trace-writer{trace-writer}

13875 root   13875 19  0  75784 874644 S 00:01:11 1.4 0.0 objmon{objmon}

14372 root   14372 19  0 157464 239804 S 00:01:16 0.4 0.0 picd{picd}

14756 root   14756 19  0 1536572 922928 S 00:01:04 1.4 0.0 rpd{rpd}

21801 root   22334 19  0 3028092 3105356 S 00:01:15 4.8 0.0 EvoAftManBt-mai{AftTimer-0}

21801 root   22335 19  0 3028092 3105356 S 00:01:19 4.8 0.0 EvoAftManBt-mai{AftTimer-1}

21801 root   22336 19  0 3028092 3105356 S 00:01:16 4.8 0.0 EvoAftManBt-mai{AftTimer-2}

 

Before falling off:

 

---(refreshed at 2024-04-17 13:37:41 PDT)---

-------------------------------

node: re0

-------------------------------

top - 13:37:42 up 2:44, 4 users, load average: 2.34, 2.03, 1.86

Threads: 2015 total,  1 running, 2014 sleeping,  0 stopped,  0 zombie

%Cpu(s): 1.1 us, 1.1 sy, 0.0 ni, 97.7 id, 0.0 wa, 0.2 hi, 0.0 si, 0.0 st

MiB Mem : 63148.3 total, 41807.9 free, 13824.5 used,  7515.9 buff/cache

MiB Swap:  4096.0 total,  4096.0 free,   0.0 used. 47518.8 avail Mem

 

 PID USER    TID PRI NI  SIZE   RSS S   TIME %MEM %CPU COMMAND

10480 root   19735 41  - 353828 253584 S 00:02:48 0.4 10.5 evo-cda-bt{QPollMainThd}

 

Symptoms

High CPU for EvoAftManBt-main

Solution

The behavior is FAD and intentional. The EvoAftManBt-main daemon is not coded to respond to the abort tied to the Ctrl-C, in contrast to MGD and the CLI which stop. Thus, it continues to process the command in the background until all of the routes are "displayed". This is the reason for the high CPU. There is no other adverse impact. 

Modification History

2024-04-30 : Article Created