Description

The NFX device was stuck in a boot loop that appeared to be hardware failure related. Upon further investigation the issue was related to environmental conditions that fell below temperature specifications.

Symptoms

The device would run in a stable state, then intermittently go into a boot loop. Initially, this was believed to be hardware issue.

The following error was seen in the boot logs:

  • RE-FPGA-DRV: reboot notifier called with 0x0003
    reboot: Power down

Solution

The device is pushed into a boot loop when the external temperature, as detected by tmpsensor on the device, is below the minimal specified level; which is 32°F . The boot loop on the device will continue until the temperature detected by tmpsensor is within the specified temperature range of 32° to 122° F (0° to 50°).

To recover a device stuck in the boot loop due to temperature below the minimal specified level, the external temperature has to be maintained at a level where the temperature detected by tmpsensor on the device is within the specified range that is 32° to 122° F (0° to 50° C).

The following temperature specs are documented in NFX Series Network Services Platform .

  • Environmental Ranges NFX250 
  • Operating temperature: 32° to 122° F (0° to 50° C) 
  • Storage temperature: -40° to 158° F (-40° to 70° 

The following log was from hypervisor found in  /var/log/lcmd.log | grep nfx_shutdown .  This command was run during stable conditions.

ABC01-host lcmd[8248]: nfx_shutdown:184: Shutting down due to thermal event 1 Temp = -1 notification from FRU TmpSensor

The following command can be run from vJUNOS0 to see the current temperature on the device:

root@vJUNOS0> show chassis environment no-forwarding​
 
Class Item                      Status   Measurement
Power FPC 0 Power Supply   0    OK
Temp  FPC 0 Sensor 1            OK       5 degrees C / 41 degrees F
FPC 0 Sensor 2                  OK       2 degrees C / 35 degrees F
​FPC 0 Coretemp                  OK       20 degrees C / 68 degrees F
Fans  FPC 0 Fan Tray 0 Fan 0    OK       Spinning at normal speed

FPC 0 Fan Tray 1 Fan 0          OK       Spinning at normal speed

 

Modification History

2019-05-25: Solution description updated.