Description

This KB provides steps to troubleshoot an alarm for all fan failed on the device:-
  
user@device> show chassis alarms
5 alarms currently active
Alarm time Class Description
2023-08-14 02:10:54 UTC Major FPC 0 Fan 4 not spinning
2023-08-14 02:10:49 UTC Major FPC 0 Fan 3 not spinning
2023-08-14 02:10:29 UTC Major FPC 0 Fan 2 not spinning
2023-08-14 02:10:29 UTC Major FPC 0 Fan 1 not spinning
2023-08-14 02:10:29 UTC Major FPC 0 Fan 0 not spinning

Symptoms

Below Major alarms that will be seen on the device:- 
 

user@device> show chassis alarms
5 alarms currently active
Alarm time Class Description
2023-08-14 02:10:54 UTC Major FPC 0 Fan 4 not spinning
2023-08-14 02:10:49 UTC Major FPC 0 Fan 3 not spinning
2023-08-14 02:10:29 UTC Major FPC 0 Fan 2 not spinning
2023-08-14 02:10:29 UTC Major FPC 0 Fan 1 not spinning
2023-08-14 02:10:29 UTC Major FPC 0 Fan 0 not spinning


Device may also go in an unstable state and may reboot multiple times.
This can lead to multiple convergence of protocols like STP causing instability in the network.

Solution

When the alarm for "FAN not spinning" is seen for all fans on the device and generated around the same time then many times this can be a transient hardware issue on the chassis or can be permanent hardware failure for the chassis or the Fan module.

Below steps can be followed to narrow down the issue further, if it is related to the chassis or fan.

STEP 1:-  Verify the outputs of the command  "show chassis hardware" to see if all fans are of the same type.

On QFX devices there are two types of FAN module 
  1. Air Flow in  [AFI]
  2. Air Flow Out [AFO]
Power supplies are also available as AFO and AFI.

Airflow In and Airflow Out fans and power supplies cannot be mixed in the same chassis.
Please refer to the below documentation for more details on the same:-

https://www.juniper.net/documentation/us/en/hardware/qfx5100/topics/topic-map/qfx5100-cooling-system.html

The device will generate an alarm when there is a MIX of AFI and AFO type fan modules on the device.

Below is an example of when AFi and AFO-type fans are present on the device 
user@device> show chassis hardware detail no-forwarding
<Truncated output>
Power Supply 0   REV 04   740-041741                     JPSU-650W-AC-AFO
Power Supply 1   REV 04   740-041741                     JPSU-650W-AC-AFO
Fan Tray 0                                               QFX5100 Fan Tray 0, Front to Back Airflow - AFO
Fan Tray 1                                               QFX5100 Fan Tray 1, Front to Back Airflow - AFO
Fan Tray 2                                               QFX5100 Fan Tray 2, Front to Back Airflow - AFO
Fan Tray 3                                               QFX5100 Fan Tray 3, Back to Front Airflow - AFI
Fan Tray 4                                               QFX5100 Fan Tray 4, Front to Back Airflow - AFO

STEP 2:- If the alarm does not clear after Step-1 then check if the alarm is getting set and then cleared continuously. This can be identified by checking the time when the alarm was generated. If this is happening then power cycle the device and check if the alarm clears.

STEP 3:- If the alarm is still not clear then the issue might not be transient and needs to be investigated further.

STEP 4:- Reseat all Fan modules one by one and check the status of the alarm. If it clears then this might be a transient issue on the Fan module

STEP 5:-  If the alarm is still not clear then swap the FAN module with a spare unit if available onsite and check the status. If the alarm clears for the slot where the spare Fan module is inserted then the issue is with the Fan module. Repeat this for all Fan modules to confirm. In this case, the affected fan modules need to be replaced on the device.

STEP 6:-  If the Alarm stays even after using a spare Fan module then the issue is with the chassis and it needs to be replaced.

All the above steps should be performed within a maintenance window and if there are unexpected results seen at any step then please contact JTAC for further assistance.
 

Modification History

22-08-2023 :- Version 1
21-05-2023:- Updated symptoms