Switch is consuming 99% of swap memory due to auditd process.
The following outputs and logs are described:
user@device> show system processes extensive no-forwarding last pid: 3441; load averages: 0.53, 0.48, 0.51 up 158+11:44:54 08:34:30 413 threads: 5 running, 364 sleeping, 44 waiting CPU: 5.1% user, 0.0% nice, 4.1% system, 0.1% interrupt, 90.8% idle Mem: 288M Active, 145M Inact, 2433M Laundry, 584M Wired, 132M Buf, 89M Free Swap: 1024M Total, 1023M Used, 816K Free, 99% Inuse PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND AUDITD release 23.4R2-S4.11 built by builder on 2025-03-14 21:17:59 UTC 25678 root 20 0 2816M 2100M nanslp 0 206:07 0.00% auditd >>>>>>>>>>>>>>>>> 25678 25671 root 20 0 459M 5588K select 1 0:00 0.00% auditd 10 root -16 - 0B 16K audit_ 2 0:00 0.00% audit audit_evclass 230 8K - 287 32 audit_record: 1312, 0, 0, 0, 0 file user-audit { 10 0(00.00) 0(00.00) [audit] 25671 13992(85.40) 6860(00.18) /usr/sbin/auditd -N 25678 13988(85.38) 6956(00.19) /usr/sbin/auditd -N Log messages: Apr 13 08:34:00 device phone-home[23966]: PHCD_CULR_EASY_PERFORM_ERR: curl_easy_perform() failed: Couldn't connect to server Apr 13 08:34:00 device mgd[3150]: UI_CHILD_EXITED: Child exited: PID 3151, status 1, command '/sbin/ifinfo' Apr 13 08:34:10 device phone-home[23966]: PHCD_CULR_EASY_PERFORM_ERR: curl_easy_perform() failed: Couldn't connect to server Apr 13 08:34:10 device mgd[3195]: UI_CHILD_EXITED: Child exited: PID 3196, status 1, command '/sbin/ifinfo' Apr 13 08:34:21 device phone-home[23966]: PHCD_CULR_EASY_PERFORM_ERR: curl_easy_perform() failed: Couldn't connect to server Apr 13 08:34:21 device mgd[3225]: UI_CHILD_EXITED: Child exited: PID 3226, status 1, command '/sbin/ifinfo' Apr 13 08:34:25 device kernel: swap_pager_getswapspace(32): failed Apr 13 08:34:27 device last message repeated 3 times Apr 13 08:34:27 device kernel: swap_pager_getswapspace(20): failed Apr 13 08:34:27 device kernel: swap_pager_getswapspace(18): failed Apr 13 08:34:27 device kernel: swap_pager_getswapspace(9): failed Apr 13 08:34:28 device kernel: swap_pager_getswapspace(32): failed Apr 13 08:34:28 device kernel: swap_pager_getswapspace(32): failed Apr 13 08:34:28 device kernel: swap_pager_getswapspace(20): failed Apr 13 08:34:28 device kernel: swap_pager_getswapspace(18): failed Apr 13 08:34:28 device kernel: swap_pager_getswapspace(32): failed Apr 13 08:34:28 device kernel: swap_pager_getswapspace(20): failed Apr 13 08:34:29 device kernel: swap_pager_getswapspace(32): failed Apr 13 08:34:30 device last message repeated 2 times Apr 13 08:34:30 device kernel: swap_pager_getswapspace(20): failed Apr 13 08:34:31 device phone-home[23966]: PHCD_CULR_EASY_PERFORM_ERR: curl_easy_perform() failed: Couldn't connect to server Apr 13 08:34:31 device mgd[3455]: UI_CHILD_EXITED: Child exited: PID 3456, status 1, command '/sbin/ifinfo' User-audit logs are also flooding with PHC logs: Apr 13 03:45:00 device newsyslog[31536]: logfile turned over due to size>10240K Apr 13 03:45:03 device auditd[25678]: %DAEMON-6-AUDITD_TACPLUS_MSG_SENT: suceessfully sent tacacs+ Accounting-Request message >>>>>>>>>>>>>>>> PID 25768 Apr 13 03:46:18 device last message repeated 43 times Apr 13 03:46:21 device mgd[31828]: %DAEMON-5-UI_CHILD_EXITED: Child exited: PID 31829, status 1, command '/sbin/ifinfo' Apr 13 03:46:23 device auditd[25678]: %DAEMON-6-AUDITD_TACPLUS_MSG_SENT: suceessfully sent tacacs+ Accounting-Request message Apr 13 03:46:28 device last message repeated 43 times Apr 13 06:00:08 device mgd[64796]: UI_LOGIN_EVENT: User 'root' login, class 'super-user' [64796], ssh-connection '', client-mode 'cli' Apr 13 06:00:08 device mgd[64796]: UI_CMDLINE_READ_LINE: User 'root', command 'show dhcp client binding interface irb.0 detail | display xml | save /etc/phone-home/phc_dns_dhcp.xml ' Apr 13 06:00:08 device mgd[64796]: UI_LOGOUT_EVENT: User 'root' logout Apr 13 06:00:08 device mgd[64798]: UI_AUTH_EVENT: Authenticated user 'root' assigned to class 'super-user' From interactive-commands I can see the following generating over and over: Apr 13 08:15:00 device mgd[98272]: UI_LOGOUT_EVENT: User 'apstra' logout Apr 13 08:15:01 device mgd[98288]: UI_AUTH_EVENT: Authenticated user 'root' assigned to class 'super-user' Apr 13 08:15:01 device mgd[98288]: UI_LOGIN_EVENT: User 'root' login, class 'super-user' [98288], ssh-connection '', client-mode 'cli' Apr 13 08:15:01 device mgd[98288]: UI_CMDLINE_READ_LINE: User 'root', command 'show route | display xml | save /etc/phone-home/phc_route_info.xml ' Apr 13 08:15:01 device mgd[98288]: UI_LOGOUT_EVENT: User 'root' logout Apr 13 08:15:01 device mgd[98290]: UI_AUTH_EVENT: Authenticated user 'root' assigned to class 'super-user' Apr 13 08:15:01 device mgd[98290]: UI_LOGIN_EVENT: User 'root' login, class 'super-user' [98290], ssh-connection '', client-mode 'cli' Apr 13 08:15:01 device mgd[98290]: UI_CMDLINE_READ_LINE: User 'root', command 'show interfaces irb.0 brief | display xml | save /etc/phone-home/phc_intf_info.xml '
Switch configuration under system stanza:
authentication-order [ tacplus password ]; accounting { events [ login change-log interactive-commands ]; >>>>> destination { tacplus { server { <IP address> { routing-instance mgmt_junos; port 49; single-connection; } } } } } phone-home { server https://redirect.juniper.net; rfc-compliant; }
The issue is due to phone-home trying to perform ZTP operation, this will generate UI_CMDLINE_READ_LINE on interactive commands, which will trigger accounting to send authentication request to the tacacs+ server.
The PID is related to authentication requests, however the requests are excessive which is causing high memory swap condition.
Removing phone-home configuration will help to prevent this issue.
#delete system phone-home