Description

This article describes the issue of Present being displayed as the status of the RE on the T1600 device.

Symptoms

Symptoms :

When the issue occurs, the immediate impact is the loss of communication with the Backup RE being displayed in chassis alarms. Then the Present status of both RE and CB is displayed in the output of the show chassis environment CLI command:

user@T1600-re0> show chassis alarms

2 alarms currently active
Alarm time Class Description
2012-11-17 13:32:34 MYT Minor Loss of communication with Backup RE
2012-11-17 13:32:24 MYT Minor Backup RE Active

 

user@T1600-re0 > show chassis environment

Class Item Status Measurement

Temp PEM 0 OK 27 degrees C / 80 degrees F
PEM 1 OK 27 degrees C / 80 degrees F
SCG 0 OK 32 degrees C / 89 degrees F
SCG 1 OK 30 degrees C / 86 degrees F
Routing Engine 0 Present
Routing Engine 1 OK 39 degrees C / 102 degrees F
Routing Engine 1 CPU OK 44 degrees C / 111 degrees F
CB 0 Present

After rebooting specific CB for the RE, it still cannot be booted up; also the SPMB is stuck in the image loading status:

user@T1600-re0> start shell pfe direct spmb0
Automatic boot request timed out, no boot command string from host
Automatic boot request timed out, no boot command string from host
Automatic boot request timed out, no boot command string from host
Automatic boot request timed out, no boot command string from host

user@T1600-re0> show chassis environment | match spmb
SPMB 1 OK

Solution

Possible causes :

If the issue occurs on RE slot 0 , it could be due to the RE slot 0 PCI bus issue, RE slot 1's em0 private interface issue, or the CB ethernet switch issue. Also, the mid-plane was suspected in such cases. As per the T1600 RE architecture, RE slot 0 is connected to the local components via the bcm0 interface and the local ethernet switch, whose components includes the local CB, local SPMB, and all of the FPCs.

The RE slot 0 is connected to backup components, such as the backup RE and CB, via the em0 interface and the ethernet switch on the backup CB. Refer to the following TNP address information:

On the primary RE : 
user@T1600-re0> show tnp addresses
Name TNPaddr MAC address IF MTU E H R
master 0x1 02:00:01:00:00:04 em0 1500 0 1 3
master 0x1 02:00:00:00:00:04 bcm0 1500 0 0 3
re0 0x4 02:00:01:00:00:04 em0 1500 0 1 3
re0 0x4 02:00:00:00:00:04 bcm0 1500 0 0 3
re1 0x5 02:01:00:00:00:05 em0 1500 3 0 3
backup 0x6 02:01:00:00:00:05 em0 1500 3 0 3
fpc0 0x10 02:00:00:00:00:10 bcm0 1500 5 0 3
fpc2 0x12 02:00:00:00:00:12 bcm0 1500 5 0 3
fpc3 0x13 02:00:00:00:00:13 bcm0 1500 4 0 3
fpc4 0x14 02:00:00:00:00:14 bcm0 1500 5 0 3
fpc5 0x15 02:00:00:00:00:15 bcm0 1500 5 0 3
fpc6 0x16 02:00:00:00:00:16 bcm0 1500 5 0 3
fpc7 0x17 02:00:00:00:00:17 bcm0 1500 5 0 3
spmb0 0x30 02:00:00:00:00:30 bcm0 1500 5 0 3
spmb1 0x31 02:00:00:00:00:31 em0 1500 5 0 3
bcast 0xffffffff ff:ff:ff:ff:ff:ff em0 1500 0 1 3
bcast 0xffffffff ff:ff:ff:ff:ff:ff bcm0 1500 0 0 3

On the backup RE: 

user@T1600-re1> show tnp addresses
Name TNPaddr MAC address IF MTU E H R
master 0x1 02:00:00:00:00:04 em0 1500 2 0 3
re0 0x4 02:00:00:00:00:04 em0 1500 2 0 3
re1 0x5 02:01:01:00:00:05 em0 1500 0 1 3
re1 0x5 02:01:00:00:00:05 bcm0 1500 0 0 3
backup 0x6 02:01:01:00:00:05 em0 1500 0 1 3
backup 0x6 02:01:00:00:00:05 bcm0 1500 0 0 3
fpc0 0x10 02:00:00:00:00:10 em0 1500 4 0 3
fpc2 0x12 02:00:00:00:00:12 em0 1500 4 0 3
fpc3 0x13 02:00:00:00:00:13 em0 1500 4 0 3
fpc4 0x14 02:00:00:00:00:14 em0 1500 4 0 3
fpc5 0x15 02:00:00:00:00:15 em0 1500 4 0 3
fpc6 0x16 02:00:00:00:00:16 em0 1500 4 0 3
fpc7 0x17 02:00:00:00:00:17 em0 1500 4 0 3
spmb0 0x30 02:00:00:00:00:30 em0 1500 4 0 3
spmb1 0x31 02:00:00:00:00:31 bcm0 1500 4 0 3
bcast 0xffffffff ff:ff:ff:ff:ff:ff em0 1500 0 1 3
bcast 0xffffffff ff:ff:ff:ff:ff:ff bcm0 1500 0 0 3
Confirm causes :

The following outputs can be examined to determine the cause of this message:
- request support information | no-more
- show log messages | no-more
- show log chassisd | no-more
- show chassis environment | no-more
- show chassis spmb
- show chassis enviroment cb
- show tnp address

- start shell pfe direct spmb0
- show nvram
- show syslog messages
- ping 0x1
ctrl^C

To resolve this issue, perform the following procedure:
 

  1. When the symptoms first occur, do not restart the CB, specifically for the problematic RE; instead, logon to the CB.
  2. Test the reachability via the ping command on SPMB firmware.
  3. If all the SPMBs and FPCs can be reached, except for the primary, the RE PCI bus issue can be confirmed and the RE hardware issue was suspected by the ping test.
  4. If you are unable to logon to the SPMB, the SPMB on the CB hardware issue can be suspected.
  5. If the SPMB cannot reach the remote RE via em0 , the remote RE interface and mid-plane hardware issue can be suspected.

Note : If the CB, which displayed the 'Present' status, is restarted, but the RE still cannot be booted up, SPMB will not bootup, as the image cannot be loaded; which might cause more difficulties for troubleshooting.

Modification History

2021-03-24: Updated the article terminology to align with Juniper's Inclusion & Diversity initiatives.