Description

We have received a report regarding the RE1 failure from customer. Upon investigation, we observed a "CB Power Disabled" error in the output of the "show system errors active" command.


Other relevant logs in the symptoms section for your review.

Symptoms

Hostname: labrouter.RD.NO-RE0-re0

Model: ptx10008

Junos: 23.4R2-S3.10-EVO


0 alarms currently active


[email protected]> show system errors active detail

System Active Errors Detail Information

CB 0

----------------------------------------------------------------

Error Name      : CB Power Disabled

Identifier      : /cb/0/plat-mra/0/Cb[1]/0/scapa_refpga_i2cs/0/CB Power Disabled

Description      : CB Power Disabled

State         : enabled

Scope         : board

Category       : functional

Level         : major

Threshold       : 1

Error limit      : 1

Occur count      : 1

Clear count      : 0

Last occurred(ms ago) : 2025-04-28 21:14:55 UTC (18973150 ms ago)


 

[email protected]> show chassis routing-engine no-forwarding

Routing Engine status:

 Slot 0:

  Current state         Master

  Election priority       Master

...

  Model             JNP10K-RE1-E128

  Serial ID           BUILTIN

  Start time           2025-03-12 06:53:00 UTC

  Uptime             47 days, 19 hours, 36 minutes, 59 seconds


Routing Engine status:

 Slot 1:

  Current state         Backup

  Election priority       Backup

...

  Model             JNP10K-RE1-E128

  Serial ID           BUILTIN

  Start time           2025-04-28 21:08:00 UTC

  Uptime             5 hours, 13 minutes, 11 seconds

  Last reboot reason       MSMI error/hw misc


 

[email protected]> show system firmware


Part       Type       Tag Current        Available       Status

                    version        version

...


RE 0       CBC FPGA     0  0x10B         0x10B         OK

RE 0       RE FPGA      1  0x2B         0x2B         OK

RE 0       SSD Primary    2  SBR12050       SBR12050       OK

RE 0       SSD Secondary   3  SBR12050       SBR12050       OK

RE 0       i40e       4  6.01         7.00         OK

RE 0       BIOS ROM     5  19.01         19.01         OK

RE 1       CBC FPGA     0  0x10B         0x10B         OK

RE 1       RE FPGA      1  0x2B         0x2B         OK

RE 1       SSD Primary    2  SBR12050       SBR12050       OK

RE 1       SSD Secondary   3  SBR12050       SBR12050       OK

RE 1       i40e       4  6.01         7.00         OK

RE 1       BIOS ROM     5  19.01         19.01         OK



Routing Engine 0     BUILTIN   BUILTIN      JNP10K-RE1-E128

Routing Engine 1     BUILTIN   BUILTIN      JNP10K-RE1-E128

CB 0       REV 12  750-101823  XXXXX163     Control Board

CB 1       REV 12  750-101823  XXXXX247     Control Board



Apr 28 21:14:38.227 2025 labrouter.RD.NO-RE0-re1 kernel: {1}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 0

Apr 28 21:14:38.275 2025 labrouter.RD.NO-RE0-re1 kernel: {1}[Hardware Error]: It has been corrected by h/w and requires no further action

Apr 28 21:14:38.275 2025 labrouter.RD.NO-RE0-re1 kernel: {1}[Hardware Error]: event severity: corrected

Apr 28 21:14:38.275 2025 labrouter.RD.NO-RE0-re1 kernel: {1}[Hardware Error]: Error 0, type: corrected

Apr 28 21:14:38.275 2025 labrouter.RD.NO-RE0-re1 kernel: {1}[Hardware Error]:  section_type: memory error

Apr 28 21:14:38.275 2025 labrouter.RD.NO-RE0-re1 kernel: {1}[Hardware Error]:  error_status: 0x0000000000000400

Apr 28 21:14:38.275 2025 labrouter.RD.NO-RE0-re1 kernel: {1}[Hardware Error]:  node: 0


Apr 28 21:14:39.710 2025 labrouter.RD.NO-RE0-re1 kernel: spi-cbc spi-cbc.10: cbc_spi_mstr_wait_rdy: TIMEOUT: 8407263098.100.1: 0x00000006!

Apr 28 21:14:39.710 2025 labrouter.RD.NO-RE0-re1 kernel: spi-cbc spi-cbc.10: cbc_spi_xfer_run: wait_rdy(100) WR failed: 23418581.0, 70256255.1: -16.

Apr 28 21:14:39.710 2025 labrouter.RD.NO-RE0-re1 kernel: spi_master spi0: cbc_spi_xfer_one: failed: 2.93674836.2 187349672.1: -16.

Apr 28 21:14:39.796 2025 labrouter.RD.NO-RE0-re1 kernel: spi_master spi0: failed to transfer one message from queue


Apr 28 21:14:48.504 2025 labrouter.RD.NO-RE0-re0 hwdre: HWD_FRU_OFFLINE_NOTICE: FRU offline /Chassis[0]/Cb[1]

Apr 28 21:14:48.504 2025 labrouter.RD.NO-RE0-re0 hwdre: HWD_FRU_OFFLINE_NOTICE: FRU offline cb1

Apr 28 21:14:48.504 2025 labrouter.RD.NO-RE0-re0 hwdre: HWD_FRU_SNMP_TRAP_NOTICE: SNMP trap generated: jnxFruOffline for /Chassis[0]/Cb[1]

Apr 28 21:14:48.502 2025 labrouter.RD.NO-RE0-re0 hwdre[18612]: process_change_event Fru name /Chassis[0]/Cb[1] get_attr enable state: 1 fruState: 8

Apr 28 21:14:48.504 2025 labrouter.RD.NO-RE0-re0 hwdre[18612]: fru_state_transition /Chassis[0]/Cb[1] ONLINE -> OFFLINE

Apr 28 21:14:48.508 2025 labrouter.RD.NO-RE0-re0 hwdre[18612]: fru_state_transition /Chassis[0]/Cb[1] OFFLINE -> POWER_OFF_START

Apr 28 21:14:48.510 2025 labrouter.RD.NO-RE0-re0 hwdre[18612]: process_change_event Fru name /Chassis[0]/Cb[1] get_attr enable state: 0 fruState: 17

Apr 28 21:14:48.670 2025 labrouter.RD.NO-RE0-re0 emfd-fpa[18356]: EMF_FPA_FAULT_REP: createFault: Fault(Location: /Chassis[0]/Re[0] Module: platform Object: eth0 Error: 6) reported

Apr 28 21:14:48.670 2025 labrouter.RD.NO-RE0-re0 emfd-fpa[18356]: EMF_EVO_ALARM_SET: Alarm set: RE color=red, class=CHASSIS, reason=Host 0 Ethernet Interface Link Down

Apr 28 21:14:48.671 2025 labrouter.RD.NO-RE0-re0 emfd-fpa[18356]: EMF_FPA_ALARM_REP: RaiseAlarm: Alarm(Location: /Chassis[0]/Re[0] Module: platform Object: eth0 Error: 6) reported


Apr 28 21:14:49.349 2025 labrouter.RD.NO-RE0-re0 hwdre: HWD_FRU_SNMP_TRAP_NOTICE: SNMP trap generated: jnxFruPowerOff for /Chassis[0]/Cb[1]

Apr 28 21:14:49.349 2025 labrouter.RD.NO-RE0-re0 hwdre[18612]: fru_state_transition /Chassis[0]/Cb[1] POWER_OFF_START -> POWER_OFF

Apr 28 21:14:50.700 2025 labrouter.RD.NO-RE0-re0 emfd-fpa[18356]: EMF_EVO_ALARM_CLR: Alarm cleared: RE color=red, class=CHASSIS, reason=Host 0 Ethernet Interface Link Down

Apr 28 21:14:50.700 2025 labrouter.RD.NO-RE0-re0 emfd-fpa[18356]: EMF_FPA_ALARM_CLR: ClearAlarm: Alarm(Location: /Chassis[0]/Re[0] Module: platform Object: eth0 Error: 6) cleared

Apr 28 21:14:50.700 2025 labrouter.RD.NO-RE0-re0 emfd-fpa[18356]: EMF_FPA_FAULT_CLR: destroyFault: Fault(Location: /Chassis[0]/Re[0] Module: platform Object: eth0 Error: 6) cleared

Apr 28 21:14:51.422 2025 labrouter.RD.NO-RE0-re0 hwdre[18612]: process_change_event Fru name /Chassis[0]/Cb[1] get_attr enable state: 0 fruState: 18

Apr 28 21:14:51.960 2025 labrouter.RD.NO-RE0-re0 hwdre: HWD_FRU_ONLINE_NOTICE: FRU online /Chassis[0]/Cb[1]

Apr 28 21:14:51.960 2025 labrouter.RD.NO-RE0-re0 hwdre: HWD_FRU_SNMP_TRAP_NOTICE: SNMP trap generated: jnxFruOnline for /Chassis[0]/Cb[1]

Apr 28 21:14:51.960 2025 labrouter.RD.NO-RE0-re0 hwdre: HWD_FRU_ONLINE_NOTICE: FRU online cb1

Apr 28 21:14:51.953 2025 labrouter.RD.NO-RE0-re0 kernel: max1363 104-0034: 104-0034 supply vcc not found, using dummy regulator


Apr 28 21:14:51.960 2025 labrouter.RD.NO-RE0-re0 hwdre[18612]: process_change_event Fru name /Chassis[0]/Cb[1] get_attr enable state: 0 fruState: 18

Apr 28 21:14:51.960 2025 labrouter.RD.NO-RE0-re0 hwdre[18612]: fru_state_transition /Chassis[0]/Cb[1] POWER_OFF -> ONLINE


Apr 28 21:14:52.311 2025 labrouter.RD.NO-RE0-re0 sysman[16382]: SYSTEM_NODE_UNREACHABLE_EVENT: Node 2201170739205 nonce 1780269778 is unreachable


Apr 28 21:14:54.726 2025 labrouter.RD.NO-RE0-re0 emfd-fpa[18356]: SYSTEM_PSET_STATE_CHANGE: Production set [fpa, re0, 50341892] has changed state to: active at 21:14:54.726

...


Solution

From the below logs, we see CB1 is reporting an error condition even after the soft reboot (offline/online of CB). CB1 replaced to resolve the issue 

 

[vrf:none] root@labrouter:~# i2csc cb0

 

Main I2C Bus -- i2c-sam.10:0, channel number 0

I2C bus of fru (cb0) -- i2c-39-mux (chan_id 0), bus # 40

...

          Power Control       0x20     0x01

...

 

[vrf:none] root@labrouter:~# i2csc cb1

 

Main I2C Bus -- i2c-sam.10:0, channel number 1

I2C bus of fru (cb1) -- i2c-39-mux (chan_id 1), bus # 41

...

          Power Control       0x20     0x80      <<< Indicating Power disabled

 

 

Modification History

2025-05-06 : Article Created