This article explains the meaning of the alarm, 'Host x SMPC CPLD Golden Image Active' on MX series devices and how to resolve it.
'Host x SMPC CPLD Golden Image Active'
This alarm may be seen after upgrading the Routing Engine or during the upgrade. One of the routing engines may trigger this error.
Active alarm:
user@host> show system alarms 1 alarms currently active Alarm time Class Description 2019-04-25 16:50:58 UTC Minor Host 0 SPMC CPLD Golden Image active Syslog Messages:
Feb 6 00:33:06.956 2020 alarmd[23623]: %DAEMON-4: Alarm set: RE color=YELLOW, class=CHASSIS, reason=Host 1 SPMC CPLD Golden Image active Feb 6 00:33:06.981 2020 craftd[18436]: %DAEMON-4: Minor alarm set, Host 1 SPMC CPLD Golden Image active
Verify Hardware details:
user@host> show chassis hardware Jun 21 03:38:07 Hardware inventory: Item Version Part number Serial number Description Chassis BLANK JNP10003 [MX10003] Midplane REV 01 750-066883 BLANK MX10003 Midplane Routing Engine 0 BUILTIN BLANK JNP10003-RE1 CB 0 REV 07 750-067071 BLANK JNP10003-SPM
Verify the Firmware version on affected RE x:
user@host> show system firmware
Part Type Version RE 0 PRI BIOS CBEP_P_VAL1_00.14.01 FPGA 265.0 RE-FPGA 42.0 RE-SSD1 SF-SBR12050 RE-SSD2 SF-SBR12050 i40e-NVM 6.01 RE 1 PRI BIOS CBEP_P_VAL1_00.14.01 FPGA 265.0 RE-FPGA 42.0 RE-SSD1 SF-SBR12050 RE-SSD2 SF-SBR12050
Login to Linux Client ( VM Host ):
user@host>start shell user root % vhclient -s
Verify FPGA version on RE that is problematic:
root@RE:~# re_fpga -V FPGA version : 0x2a <-- Verify this to be same on both REs JSPEC version : 0x12 PCB version : 0x02 Board Name : MX 10K
On problematic RE (Here, it is RE 0): root@RE0:/var/log# re_fpga -r 0x1104 RE FPGA: Read reg 0x1104 -> Data 0x4f On RE with no alarm: root@RE1:/var/log# re_fpga -r 0x1104 RE FPGA: Read reg 0x1104 -> Data 0x17
root@RE0:/var/log# re_fpga -r 0x1104 RE FPGA: Read reg 0x1104 -> Data 0x4f
root@RE1:/var/log# re_fpga -r 0x1104 RE FPGA: Read reg 0x1104 -> Data 0x17
To clear the alarm, perform FPGA Firmware Upgrade.
The firmware should be updated to the latest available version. Refer to the technical documentation on Installing and Upgrading Firmware
Preferably, both REs should be on the same firmware version.
When the SPMC FPGA user image is corrupted and the good golden image is available, the following is expected:
In certain cases, the alarm may be triggered due to a mismatch in RE versions as well. Ensure both the REs are on the same version, then check if the alarm clears. In the middle of an upgrade on dual RE devices, when both REs are on different versions, this alarm could be seen. In that case, upgrade the device with the best practice procedure, then check once both the REs are on the same page.
You may also see a BIOS mismatch/secure boot disabled alarm in addition to the existing alarm. Refer to KB35975 - Minor 'VMHost RE x Secure Boot Disabled' alarm [juniper.net]
From the affected RE x, collect the following logs (applicable for NGRE ):
user@host> start shell user root
Enter Linux Host:
:~#vhclient -s
Collect VM Host Logs:
root@host:~# tar -cf ~/host_varlog_REx.tar -C /var/log/* tar: Removing leading `/' from member names root@host:~# ls -l | grep varlog -rw-r--r--. 1 root root 75509760 Jun 18 02:55 host_varlog_REx.tar
Copy the file to VM Routing Engine /var/tmp .
/var/tmp
-- MX systems -- root@host:~# scp host_varlog_REx.tar [email protected]:/var/tmp/ Password: host_varlog_REx.tar 100% 73MB 36.5MB/s 00:02
root@host:~# scp host_varlog_REx.tar [email protected]:/var/tmp/ Password: host_varlog_REx.tar 100% 73MB 36.5MB/s 00:02
The above-collected VM Host logs will help to determine the root cause for the issue and for JTAC analysis. In addition to the above-collected outputs, you might need to have the following (preferable) for JTAC analysis:
Refer to Net Generation Routing Engine- NGRE for more details.
Please contact Juniper Support if the alarm does not clear after performing the above-mentioned steps for detailed analysis.