Description

Some PTX10008 routers may experience an unexpected RE reboot, with the reason indicated as a "Panic", which generates a "VMCORE". This behaviour is typically associated with a deadlock in FreeBSD processes caused by BlockIO issues on vmhost platforms.

Symptoms

re0> show system core-dumps no-forwarding

-rw------- 1 root wheel 1514668032 Nov 7 03:41 /var/crash/vmcore.0

/var/tmp/pics/*core*: No such file or directory

/var/crash/kernel.*: No such file or directory

/var/jails/rest-api/tmp/*core*: No such file or directory

/tftpboot/corefiles/*core*: No such file or directory

total files: 1

 

re0> show chassis routing-engine no-forwarding

Routing Engine status:

 Slot 0:

  Current state         Backup

  Election priority       Master

  Temperature         31 degrees C / 87 degrees F

  CPU temperature       32 degrees C / 89 degrees F

  DRAM           20424 MB (20480 MB installed)

  Model             RE-PTX-2X00x4

  Start time           2024-11-7 03:39:24 PST

  Uptime             2 days, 19 hours, 40 minutes, 7 seconds

  Last reboot reason       0x1000:reboot due to panic

  Load averages:         1 minute  5 minute 15 minute

                    0.21    0.15    0.15

re1> show chassis alarms no-forwarding

1 alarms currently active

Alarm time        Class Description

2024-11-7 03:37:22 PST Minor Backup RE Active

Solution

Root Cause:

The observed issue on vmhost platforms is related to "BlockIO", which results in a "deadlock" among FreeBSD processes. This deadlock leads to "vmcore" files being generated as part of the system's resolution process during the panic. 


Solution:

The recommended resolution for this issue is to "upgrade the router to a fixed Junos version". The upgrade will address the underlying BlockIO deadlock issue, preventing further panics and vmcore generation.


Steps to Resolve:

Upgrade Junos Version:

  - Identify the current Junos version running on the PTX10008.

  - Upgrade to the appropriate fixed Junos version as recommended by the engineering team. The fixed version will resolve the BlockIO deadlock issue causing the panic and vmcore generation.

  

Note: Ensure you follow the official upgrade procedures to avoid configuration issues.


Logs Collection for Debug:

In order to investigate the issue further and gather the necessary data for resolution, the following logs should be collected:

- RSI from both the RE (request support information | no-more)

- /var/logs from both Routing Engines (REs) to provide a complete view of system behaviour before and after the panic.

- Collect the VMCORE file generated under "show system core-dump" during the panic for further analysis to check if there is any known PR after decoding it.


Next Steps:

- If you have already collected the logs and upgraded to the fixed Junos version, monitor the router for further issues.

- If the issue persists after the upgrade, escalate the case to engineering with the collected logs for further investigation.


Additional Notes:

- It is important to ensure that the router is running the latest recommended software version to avoid encountering similar issues in the future.

- For any questions or additional assistance, please contact [Support Team Contact Information].

Modification History

2024-12-09 : Article Created

2025-7-31 : Replaced hostname 

2025-8-12 : Filled in Modification History