This Article explains the meaning of the log message and details of the mgd crash when <request-shell-execute> is run via an event script. The issue is reported on qfx5100-24q-2p in 14.1X53-D28

The below log messages are seen continuously on the device in addition to the core-dumps.

lab@qfx> show system core-dumps no-forwarding

-rw-rw---- 1 root wheel 4272988 Jan 3 2023 /var/tmp/mgd.core.0.gz

-rw-rw---- 1 root wheel 4272992 Jan 3 2023 /var/tmp/mgd.core.1.gz

-rw-rw---- 1 root wheel 4273002 Jan 3 2023 /var/tmp/mgd.core.2.gz

-rw-rw---- 1 root wheel 4273045 Jan 3 2023 /var/tmp/mgd.core.3.gz

-rw-rw---- 1 root wheel 4272938 Oct 6 17:53 /var/tmp/mgd.core.4.gz

total files: 5

Oct 6 15:05:11 qfx /kernel: %KERN-3-BAD_PAGE_FAULT: pid 41188 (mgd), uid 0: pc 0x82276fb got a read fault at 0x0, x86 fault flags = 0x4

Oct 6 15:05:11 qfx /kernel: %KERN-3: Trapframe Register Dump:

Oct 6 15:05:11 qfx /kernel: %KERN-3: eax: 085fdc00 ecx: ffffffff edx: afb751c0 ebx: afb748dc

Oct 6 15:05:11 qfx /kernel: %KERN-3: esp: afb748c0 ebp: afb74ce8 esi: 00000000 edi: 00000000

Oct 6 15:05:11 qfx /kernel: %KERN-3: eip: 082276fb eflags: 00210286

Oct 6 15:05:11 qfx /kernel: %KERN-3: cs: 0033 ss: 003b ds: afb7003b es: 003b

Oct 6 15:05:11 qfx /kernel: %KERN-3: fs: afb7003b trapno: 0000000c err: 00000004

Oct 6 15:05:11 qfx /kernel: %KERN-3: Page table info for PC address 0x82276fb: PDE = 0x2c2e0067, PTE = 1a43c425Oct 6 15:05:11 qfx /kernel: %KERN-3: Dumping 16 bytes starting at PC address 0x82276fb:

Oct 6 15:05:11 qfx /kernel: %KERN-3:  f2 ae 89 ce f7 d6 b9 ff ff ff ff 89 df f2 ae f7 

If a page fault occurs for a reference to an address that's not part of the virtual address space, so that there cannot be a page in memory corresponding to it, then it is called an invalid page fault. This is what constitutes the "BAD" in the BAD_PAGE_FAULT message.  The page fault handler in Junos then needs to terminate the code that made the reference, and/or deliver an indication to that code that the reference was invalid.

Bad page faults can result in process crashes, segmentation errors, and often core-dumps. The cause of the invalid reference can be software errors, or hardware faults - bad memory that can corrupt pointers causing even correct software to fail.
 

The core-dump generated can be decoded and following is the back-trace.

#0 0x08276e4b in mgd_execute_shell_cmd (daap=0xffef51f0) at ../../../../src/ui/lib/actions/ui_afuncs.c:1807

#1 0xc863275e in ddl_invoke_action (amp=0xfaab7620, action=1, daap=0xffef51f0) at ../../../../src/ui/lib/access/action.c:91

#2 0xc8632a2f in ddl_invoke_all_actions (act=1, args=0xffef51f0) at ../../../../src/ui/lib/access/action.c:113

#3 0x08159242 in gram_execute (gdp=0xffef5844, gsp=0xffef5a14) at /volume/build/junos/14.1/release/14.1R3-S1/src/ui/lib/util/pipe.c:654

#4 0x081a6721 in gram_yyparse (yyparse_void_param=0xffef5844) at ../../../../src/ui/lib/engine/gram.y:3101

#5 0x0818ec83 in ddl_start_command_mode (peer=0x85d3d00, username=0x85da180 "root", req=18446743936270598143, xml_mode=1, netconf_mode=0) at ../../../../src/ui/lib/engine/loadogram.c:3659

#6 0x080ea3c2 in main (argc=4, argv=0xffefdeb8) at ../../../../src/ui/usr.sbin/mgd/master.c:2643


1807 in #0 indicates size = strlen("cd ") + strlen(js_client_cwd()) + strlen(buf) + 2;


 

Root-Cause:

• This crash happens when <request-shell-execute> is run via an event script. If the same script is run with an OP script, the crash does not happen.
• The root cause for the core-dump is that the current directory is not set in js_client_data. when rpc from event script is invoked by the eventd daemon.
• At function mgd_execute_shell_cmd (ui/lib/actions/ui_afuncs.c:2138), js_client_cwd() tries to fetch the current directory which is "NULL" and hence  strlen(js_client_cwd()) causes segmentation fault.
• OP script is invoked from CLI and current directory is set in js_client_data and hence the issue is not seen with OP script.
• This is a known software issue which is fixed in 14.1R3-S3 and later releases.

Solution:

• Remove the event script under [edit event-options event-script] hierarchy and reboot the device to stop seeing the log messages/cores as reported.
• Upgrade the device to a fix version of PR1051943  (14.1R3-S3 or later).
• In some cases, these may also indicate either transient or persistent HW failure event. If the issue persists, post any of the workarounds shared above, then you might consider to replace the device.