During the replacement of a faulty PEM#2 module in a maintenance window (MW), the customer observed failures in both the FPC and CB. This document outlines a scenario that may lead to CB alarms and FPC failures during PEM replacement procedures.
Although the MW was intended solely for replacing PEM2, the changes also affected PEM0. In typical configurations, PEM0 and PEM2 provide redundant power to the system—each acting as a backup for the other. Therefore, it is critical to avoid simultaneously cutting power to both PEMs, as this can disrupt power delivery to Field Replaceable Units (FRUs) that rely on these modules.
For more information about PEM redundancy and Power Zones in the MX chassis, please refer to Knowledge Base article KB94329 [juniper.net].
Log Indicating that the changes were affecting PEM0
May 30 08:17:23 2025 re0.labrouter alarmd[52257]: Alarm set: Pwr supply id=402653797, color=RED, class=CHASSIS, reason=PEM 2 Dipswitch 1 Feed Connection 1
May 30 08:20:01 2025 re0.labrouter craftd[28820]: Major alarm set, PEM 2 Dipswitch 1 Feed Connection 1
May 30 08:20:07 2025 re0.labrouter alarmd[52257]: Alarm set: Pwr supply id=150995557, color=RED, class=CHASSIS, reason=PEM 2 Not OK
May 30 08:20:07 2025 re0.labrouter craftd[28820]: Major alarm set, PEM 2 Not OK
May 30 08:20:07 2025 re0.labrouter chassisd[28804]: CHASSISD_PEM_INPUT_BAD: status failure for power supply 2 (status bits: 0x4e); check circuit breaker
May 30 08:20:07 2025 re0.labrouter alarmd[52257]: Alarm set: Pwr supply id=134218341, color=RED, class=CHASSIS, reason=PEM 2 Input Failure
May 30 08:20:07 2025 re0.labrouter craftd[28820]: Major alarm set, PEM 2 Input Failure
May 30 08:20:07 2025 re0.labrouter chassisd[28804]: CHASSISD_PEM_INPUT_BAD: Input failure for power supply 2 (status bits: 0x4e); check circuit breaker
May 30 08:20:07 2025 re0.labrouter alarmd[52257]: Alarm cleared: Pwr supply id=33555045, color=YELLOW, class=CHASSIS, reason=PEM 2 Fan Failed
May 30 08:20:07 2025 re0.labrouter craftd[28820]: Minor alarm cleared, PEM 2 Fan Failed
May 30 08:20:11 2025 re0.labrouter alarmd[52257]: Alarm set: Pwr supply id=150995045, color=RED, class=CHASSIS, reason=PEM 0 Not OK
May 30 08:20:11 2025 re0.labrouter craftd[28820]: Major alarm set, PEM 0 Not OK
May 30 08:20:11 2025 re0.labrouter chassisd[28804]: CHASSISD_PEM_INPUT_BAD: status failure for power supply 0 (status bits: 0x5e); check circuit breaker
May 30 08:20:11 2025 re0.labrouter alarmd[52257]: Alarm set: Pwr supply id=134217829, color=RED, class=CHASSIS, reason=PEM 0 Input Failure
May 30 08:20:11 2025 re0.labrouter craftd[28820]: Major alarm set, PEM 0 Input Failure
May 30 08:20:11 2025 re0.labrouter chassisd[28804]: CHASSISD_PEM_INPUT_BAD: Input failure for power supply 0 (status bits: 0x5e); check circuit breaker
May 30 08:20:11 2025 re0.labrouter alarmd[52257]: Alarm set: Pwr supply id=33554533, color=YELLOW, class=CHASSIS, reason=PEM 0 Fan Failed
May 30 08:20:11 2025 re0.labrouter craftd[28820]: Minor alarm set, PEM 0 Fan Failed
May 30 08:20:11 2025 re0.labrouter chassisd[28804]: CHASSISD_PEM_INPUT_BAD: status failure for power supply 2 (status bits: 0x4e); check circuit breaker
May 30 08:20:11 2025 re0.labrouter chassisd[28804]: CHASSISD_PEM_INPUT_BAD: Input failure for power supply 2 (status bits: 0x4e); check circuit breaker