Description

This article explains the cause and possible solution for one of the major alarm that is seen on QFX10002 devices.

The alarm observed is as follows:

root@lab> show chassis alarms no-forwarding
2 alarms currently active
Alarm time        Class Description
2024-04-14 03:00:31 UTC Major Vsensor 12:PE1 AVDD 1.0V Access Failed

Symptoms

The major alarm "Vsensor 12:PE1 AVDD 1.0V Access Failed" indicates that there is a failure in accessing the AVDD (Analog Voltage Digital-to-Digital) power supply for PE1 (Processing Element 1) with a voltage of 1.0V.

Possible causes:
 

  • Voltage exceeded beyond margin value

  • Overheating of FPC

  • Inadequate airflow system

Error logs:
 

  • Missing details of interfaces in "show chassis hardware"
root@spine1-a05-dub> show chassis hardware detail no-forwarding
Hardware inventory:
Item             Version  Part number  Serial number     Description
Chassis                                             XXXXXXX
Pseudo CB 0
  ada0   9316 MB  QEMU                 QM00001           Virtio Block Disk
  ada1   4096 MB  QEMU                 QM00002           Virtio Block Disk
  ada2    512 MB  QEMU                 QM00003           Virtio Block Disk
  ada3   1024 MB  QEMU                 QM00004           Virtio Block Disk
  usb0 (addr 0.1) UHCI root HUB 0      Intel             uhub0
  usb1 (addr 1.1) EHCI root HUB 0      Intel             uhub1
  usb2 (addr 1.2) product 0x0020 32    vendor 0x8087     uhub2
Power Supply 0   REV 05   XXX-XXXXXX   XXXXXXXXXXX       AC AFO 1600W PSU
Power Supply 1   REV 05   XXX-XXXXXX   XXXXXXXXXXX       AC AFO 1600W PSU
Fan Tray 0                                               QFX10002 Fan Tray 0, Front to Back Airflow - AFO
Fan Tray 1                                               QFX10002 Fan Tray 1, Front to Back Airflow - AFO
Fan Tray 2                                               QFX10002 Fan Tray 2, Front to Back Airflow - AFO
  • Interfaces shut down, and that can be verified in "show interface terse"

  • FPC slot show "empty" in "show chassis fpc"

root@lab> show chassis fpc
Temp CPU Utilization (%) CPU Utilization (%) Memory Utilization (%)
Slot State (C) Total Interrupt 1min 5min 15min DRAM (MB) Heap Buffer
 0     Empty
  • Temperature values is shown as "0" for all PFE chipset in "show chassis environment"
root@lab> show chassis environment no-forwarding
Class Item                                            Status     Measurement
Temp  
      FPC 0 OCXO Temp Sensor         OK         34 degrees C / 93 degrees F
      FPC 0 PFE0-HMC0-DIE            OK         0 degrees C / 32 degrees F
      FPC 0 PFE0-HMC0-TOP            OK         0 degrees C / 32 degrees F
      FPC 0 PFE0-HMC0-BOT            OK         0 degrees C / 32 degrees F
      FPC 0 PFE0-HMC1-DIE            OK         0 degrees C / 32 degrees F
      FPC 0 PFE0-HMC1-TOP            OK         0 degrees C / 32 degrees F
      FPC 0 PFE0-HMC1-BOT            OK         0 degrees C / 32 degrees F
      FPC 0 PFE0-HMC2-DIE            OK         0 degrees C / 32 degrees F
      FPC 0 PFE0-HMC2-TOP            OK         0 degrees C / 32 degrees F
      FPC 0 PFE0-HMC2-BOT            OK         0 degrees C / 32 degrees F
      FPC 0 PFE1-HMC0-DIE            OK         0 degrees C / 32 degrees F
      FPC 0 PFE1-HMC0-TOP            OK         0 degrees C / 32 degrees F
      FPC 0 PFE1-HMC0-BOT            OK         0 degrees C / 32 degrees F
      FPC 0 PFE1-HMC1-DIE            OK         0 degrees C / 32 degrees F
      FPC 0 PFE1-HMC1-TOP            OK         0 degrees C / 32 degrees F
      FPC 0 PFE1-HMC1-BOT            OK         0 degrees C / 32 degrees F
      FPC 0 PFE1-HMC2-DIE            OK         0 degrees C / 32 degrees F
      FPC 0 PFE1-HMC2-TOP            OK         0 degrees C / 32 degrees F
      FPC 0 PFE1-HMC2-BOT            OK         0 degrees C / 32 degrees F
      FPC 0 PFE2-HMC0-DIE            OK         0 degrees C / 32 degrees F
      FPC 0 PFE2-HMC0-TOP            OK         0 degrees C / 32 degrees F
      FPC 0 PFE2-HMC0-BOT            OK         0 degrees C / 32 degrees F
      FPC 0 PFE2-HMC1-DIE            OK         0 degrees C / 32 degrees F
      FPC 0 PFE2-HMC1-TOP            OK         0 degrees C / 32 degrees F
      FPC 0 PFE2-HMC1-BOT            OK         0 degrees C / 32 degrees F
      FPC 0 PFE2-HMC2-DIE            OK         0 degrees C / 32 degrees F
      FPC 0 PFE2-HMC2-TOP            OK         0 degrees C / 32 degrees F
      FPC 0 PFE2-HMC2-BOT            OK         0 degrees C / 32 degrees F
  • Var log messages shows the below error:
April 10 07:04:01 [Alarm SET] Object 150 reason 1 Device FRU VOLT SENSOR 12, reason VSensor 12 Read Fai(Vsensor 12:PE1 AVDD 1.0V Access Failed), color 1, slot 0 Alarm id 2516582401
April 10 07:04:01 send: red alarm set, device FRU VOLT SENSOR 12, reason Vsensor 12:PE1 AVDD 1.0V Access Failed

Solution

  • Since the device will be unreachable in the network, collecting RSI and var logs will be difficult. Instead, collect the below commands to verify if issue is matching to the symptoms mentioned in this KB
root@lab> show chassis hardware
root@lab> show chassis alarms
root@lab> show log messages | match VSensor
root@lab> show chassis fpc
root@lab> show chassis fpc detail
root@lab> show chassis environment 
root@lab> show chassis environment fpc 0
root@lab> show chassis temperature-thresholds
Possible workaround:
 
  •  Perform a restart of chassis control​. 
user@host> restart chassis-control
  • Since FAN is and FRU in the QFX10002 device, try removing the FAN units, clean and reseat the same.
  • If issue still persists, then perform a hard reboot / power cycle the switch and then check the voltage supplies
Note: If issue persists, even after performing the above mentioned workaround, then open JTAC case for further investigation.

Modification History

15-04-2024: Article created.
18-04-2024: Validated the article.

Related Information

https://www.juniper.net/documentation/us/en/hardware/qfx10002/topics/topic-map/qfx10002-system-overview.html#:~:text=QFX10002%20Field%2DReplaceable%20Units,-Field%2Dreplaceable%20units&text=The%20QFX10002%20FRUs%20are%20hot,removal%20to%20prevent%20chassis%20overheating.