Description

On PTX10004/8/16 platforms, the Routing Engine suffered a switchover with multiple alarms of CB <x> Power Failure and CB <x> Temp/Volt/Power Sensor Fail.

Symptoms

The RE0 suffered a reboot with the reason "MSMI error/hw misc" and caused a switchover. There are multiple alarms on CB0 indicating power and sensors failures. The RE0 is still operational and not showing any other issues.

 

{master}
user@Optx10008-re1> show chassis alarms 
6 alarms currently active
Alarm time               Class  Description
2025-06-24 02:52:49 JST  Major  CB 0 Power Failure
2025-06-24 02:57:52 JST  Minor  CB 0 Temp Sensor Fail
2025-06-24 02:57:52 JST  Minor  CB 0 Volt Sensor Fail
2025-06-24 02:57:52 JST  Minor  CB 0 Power Sensor Fail
2025-06-24 02:57:50 JST  Minor  2 VRF(MPLS) Scales(320) require a license
2025-06-24 02:52:30 JST  Minor  Backup RE Active

{master}
user@Optx10008-re1> show chassis routing-engine 
Routing Engine status:
  Slot 0:
    Current state                  Backup
    Election priority              Master
    Temperature                 35 degrees C / 95 degrees F
    CPU temperature             35 degrees C / 95 degrees F
    DRAM                      59466 MB (65536 MB installed)
    Memory utilization           9 percent
    5 sec CPU utilization:
      User                       0 percent
      Background                 0 percent
      Kernel                     0 percent
      Interrupt                  0 percent
      Idle                      99 percent
    1 min CPU utilization:
      User                       1 percent
      Background                 0 percent
      Kernel                     1 percent
      Interrupt                  0 percent
      Idle                      99 percent
    5 min CPU utilization:
      User                       1 percent
      Background                 0 percent
      Kernel                     0 percent
      Interrupt                  0 percent
      Idle                      77 percent
    15 min CPU utilization:
      User                       0 percent
      Background                 0 percent
      Kernel                     0 percent
      Interrupt                  0 percent
      Idle                      39 percent
    Model                          JNP10K-RE1-E
    Serial ID                      BUILTIN
    Start time                     2025-06-24 02:54:00 JST
    Uptime                         7 minutes, 40 seconds
    Load averages:                 1 minute   5 minute  15 minute
                                       0.28       0.51       0.41
    Last reboot reason             MSMI error/hw misc <<-----------
Routing Engine status:
  Slot 1:
    Current state                  Master
    Election priority              Backup
    Temperature                 37 degrees C / 98 degrees F
    CPU temperature             37 degrees C / 98 degrees F
    DRAM                      56889 MB (65536 MB installed)
    Memory utilization          13 percent
    5 sec CPU utilization:
      User                       2 percent
      Background                 0 percent
      Kernel                     1 percent
      Interrupt                  0 percent
      Idle                      97 percent
    1 min CPU utilization:
      User                       2 percent
      Background                 0 percent
      Kernel                     1 percent
      Interrupt                  0 percent
      Idle                      97 percent
    5 min CPU utilization:
      User                       2 percent
      Background                 0 percent
      Kernel                     1 percent
      Interrupt                  0 percent
      Idle                      86 percent
    15 min CPU utilization:
      User                       1 percent
      Background                 0 percent
      Kernel                     1 percent
      Interrupt                  0 percent
      Idle                      50 percent
    Model                          JNP10K-RE1-E
    Serial ID                      BUILTIN
    Start time                     2025-06-24 02:47:00 JST
    Uptime                         14 minutes, 42 seconds
    Load averages:                 1 minute   5 minute  15 minute
                                       0.41       0.74       0.68
    Last reboot reason             software reboot

Solution

The RE0 rebooted due to Machine Check SMI - System Management Interrupt handler receiving uncorrectable interrupts. This is a transient hardware issue. Currently the RE0 is stable in backup state. Since the RE is operational and is built-in on the CB, rebooting the CB first is recommended before going for a replacement. After the reboot, the alarms are cleared and the system remains stable.

Modification History

2024-09-29 : Article Created

2025-07-16 : Major edit with relevant logs by habe

Related Information

PR1501848 [Confidential]-[fabric] [fabric_resiliency] Evo: Scapa: CB1 power failure alarm incorrectly raised

PR1509556 [Confidential]-[Chassis] : lazurite : CB1 - Power Failure & Temp Sensor Fail

PR1518673 [Confidential]-Scapa: CMIS: CB 0 Power Failure/Temp Sensor Fail alarms seen on Scapa
PR1557418 [Confidential]-Scapa: "CB 0 Power Failure" is seen, while performing back to back mastership switch-over with link flap event

PR1627032 [Confidential]-CB0 power failure on ptx10008

PR1649781 [Confidential]-[scapa16] CB 1 Power Failure and Temp Sensor Fail alarm

PR1712000 [Confidential]-[CCL]RE1 not booting due to power failure post power on/off using cli

PR1760618 [Confidential]-22.3X50_Scapa_Google:: Observing "CB 1 Power Failure" when CB offline and online was performed via push button

PR1836005 [Confidential]-Aegon:FT: [ALC-Resiliency]:Observed "CB 0 Power Failure" and "MSMI error/hw misc" without any event

PR1843196 [Confidential]-[Aegon] After doing GRES observing CB1 power failure alarm

PR1846079 [Confidential, Duplicate]-Aegon - EDVT: CB0 [BCEG2935]: power failure at -5C

PR1849482 [Confidential]-CB 1 Power Failure alarm seen on PTX10016