Description

This article addresses the issue of "Major SPMB 0 Major errors" on the PTX5000, which is often accompanied by voltage fluctuations on the Control Boards (CBs), as seen in the chassisd logs.

Symptoms

root@XYZ> show version detail no-forwarding
Hostname: XYZ
Model: ptx5000

root@XYZ> show chassis alarms no-forwarding

1 alarms currently active
Alarm time               Class  Description
2024-05-11 12:31:40 PDT  Major  SPMB 0 Major errors

root@XYZ> show chassis hardware detail no-forwarding

Hardware inventory:
Item             Version  Part number  Serial number     Description
Chassis                                123456ABCDEF     PTX5000
CB 0             REV 21   750-055537   ABAB1123         Control Board 2
SPMB 0           REV 12   711-056400   ABNCE173          PTX5K CB PMB

May 11 12:31:40.919  XYZ alarmd[12753]: Alarm set: SPMB id=67108979, color=RED, class=CHASSIS, reason=SPMB 0 Major errors
May 11 12:31:40.919  XYZ craftd[11888]:  Major alarm set, SPMB 0 Major errors

May 11 12:31:40  spmb_error_action: SPMB 0 sent action 0x4  configured 0x4 

May 11 12:31:40  send: red alarm set, device SPMB 0, reason SPMB 0 Major errors
May 11 12:31:41  CB 0:    raw 205000 voltage 12.40
May 11 12:31:41  CB 0:    raw 205000 voltage 5.11
May 11 12:31:41  CB 0:    raw 205000 voltage 8.86
May 11 12:31:41  CB 1:    raw 205000 voltage 12.40
May 11 12:31:41  CB 1:    raw 205000 voltage 4.73
May 11 12:31:41  CB 1:    raw 206000 voltage 7.85
May 11 12:31:41  CB 0:    raw 205000 voltage 12.40
May 11 12:31:41  CB 0:    raw 205000 voltage 5.11
May 11 12:31:41  CB 0:    raw 205000 voltage 8.86
May 11 12:31:41  CB 1:    raw 205000 voltage 12.40

 

Solution

The Switch Processor Mezzanine Board (SPMB) is a component on PTX5000 series devices that programs, monitors, and controls the Switch Interface Boards (SIBs). The SPMB programs the ASICs on the Switching Fabric (SIBs

The "Major SPMB 0 Major errors" alarm typically indicates an issue with the System Power Management Board (SPMB), which could be triggered by voltage fluctuations on the Control Boards.

Workaround:
1. Restart SPMB 0: Prior to this step, we can do a switch over before restarting the SPMB_0. As restarting the SPMB_0 will cause restart of the sibs and can cause traffic issue. Hence, we perform this under MW. 
   - Restarting the SPMB can help resolve the issue by refreshing the board's state.
  > request chassis spmb slot 0 restart
  > show chassis alarms

2. Take CB0 Offline and Online:
   - If RE0 is the master Routing Engine (RE), switch mastership to RE1 before performing this action. If only a single RE is available, connect directly to the console to complete the steps.
>   request chassis cb slot 0 offline
>  request chassis cb slot 0 online

3. Reseat CB0:
   - If the alarms persist after performing the above steps, try physically reseating CB0 to ensure it is securely connected.

 Next Steps:
- After performing each action, check the status using the `show chassis alarms` command.
- If the issue persists after reseating the Control Board, it may require hardware replacement or further investigation.

Modification History

2024-09-17 : Article Created