Description

Routing-engine might panic with vm_fault unhandled status error 9

Symptoms

Below logs can be seen during the time of panic

 

Aug 17 01:04:33  kernel: panic: vm_fault: Unhandled status 9

Aug 17 01:04:33  kernel: cpuid = 6

Aug 17 01:04:33  kernel: time = 1692234109

Aug 17 01:04:33  kernel: Uptime: 85d5h22m3s

Aug 17 01:04:33  kernel: Automatic reboot in 15 seconds - press a key on the console to abort

Aug 17 01:04:33  kernel: Rebooting...

Aug 17 01:04:33  kernel: cpu_reset: Restarting BSP

Aug 17 01:04:33  kernel: cpu_reset_proxy: Stopped CPU 6

Aug 17 01:04:33  kernel: ---<<BOOT>>---

 

Host:

2023-08-17T01:01:49.579028+00:00 a00 kernel: High host-mode time vcpu 6 delay 53819 us exit reason EPT_MISCONFIG rip 0xffffffff8389b5e6 info 0 0

2023-08-17T01:02:07.985030+00:00 a00 kernel: High host-mode time vcpu 0 delay 1402017 us exit reason IO_INSTRUCTION rip 0xfff0 info 640040 0

2023-08-17T01:02:08.141658+00:00 a00 kernel: High host-mode time vcpu 0 delay 155297 us exit reason IO_INSTRUCTION rip 0xec1f0 info cfc0003 0

2023-08-17T01:02:08.302030+00:00 a00 kernel: High host-mode time vcpu 0 delay 161406 us exit reason IO_INSTRUCTION rip 0xec1f0 info cfc0003 0

2023-08-17T01:02:09.555029+00:00 a00 kernel: High host-mode time vcpu 0 delay 1252377 us exit reason TRIPLE_FAULT rip 0xfff0 info 0 0

2023-08-17T01:02:10.834029+00:00 a00 kernel: High host-mode time vcpu 0 delay 1278891 us exit reason IO_INSTRUCTION rip 0xfff0 info 710048 0

2023-08-17T01:02:10.867034+00:00 a00 kernel: High host-mode time vcpu 1 delay 4284268 us exit reason EXTERNAL_INTERRUPT rip 0x0 info 0 800000fd

2023-08-17T01:02:10.867045+00:00 a00 kernel: High host-mode time vcpu 2 delay 4284269 us exit reason EXTERNAL_INTERRUPT rip 0x0 info 0 800000fd

2023-08-17T01:02:10.867046+00:00 a00 kernel: High host-mode time vcpu 3 delay 4284268 us exit reason EXTERNAL_INTERRUPT rip 0x0 info 0 800000fd

2023-08-17T01:02:10.867049+00:00 a00 kernel: High host-mode time vcpu 4 delay 4284267 us exit reason EXTERNAL_INTERRUPT rip 0x0 info 0 800000fd

2023-08-17T01:02:10.867051+00:00 a00 kernel: High host-mode time vcpu 5 delay 4284265 us exit reason EXTERNAL_INTERRUPT rip 0x0 info 0 800000fd

2023-08-17T01:02:10.867052+00:00 a00 kernel: High host-mode time vcpu 6 delay 4284265 us exit reason EXTERNAL_INTERRUPT rip 0x0 info 0 800000fd

2023-08-17T01:02:10.867052+00:00 a00 kernel: High host-mode time vcpu 7 delay 4284263 us exit reason EXTERNAL_INTERRUPT rip 0x0 info 0 800000fd

2023-08-17T01:02:11.165031+00:00 a00 kernel: High host-mode time vcpu 0 delay 155150 us exit reason IO_INSTRUCTION rip 0xec1f0 info cfc0003 0

2023-08-17T01:03:52.975034+00:00 a00 kernel: i40e 0000:04:00.0: VF 0 failed opcode 11, retval: -10

 

 

 

Solution

Panic is caused because of the consolidate page allocation did not account for vm_fault_allocate(), returning KERN_OUT_OF_BOUNDS, vm_fault() operation is supposed to handle all possible allocation options ,but an unhandled value triggered the panic, routing-engine will be recover by itself.

This has been addressed by below PR
https://prsearch.juniper.net/PR1709013

 

 

Modification History

9/15/2023 -KB created
9/20/2023 - added PR information