Description

A Junos Fusion Aggregate Device MX router may suddenly lose connection to a Satellite Device EX switch and cause Simple Network Management Protocol (SNMP) traps FRu to go offline and online due to restarting the Extended Mode (em) service.

This article indicates that there is no service impact to the SNMP traps FRu going offline and coming online and so no action would be required.

 

Symptoms

In a Junos Fusion setup, an Aggregate Device (MX_AD) can lose connection to a Satellite Device and cause SNMP traps FRu to go offline and online due to restarting the Extended Mode (em) service.

*** messages *** 

Aug 16 19:10:42.334 2019 MX_AD spmd[1857]: %DAEMON-5-SPMD_SNMP_TRAP10: SNMP trap generated: Fru Offline (jnxFruContentsIndex 108, jnxFruL1Index 106, jnxFruL2Index 1, jnxFruL3Index 0, jnxFruName SD106 PIC:48x 10/100/1000 Base-T @ 106/0/*, jnxFruType 11, jnxFruSlot 0, jnxFruOfflineReason 1, jnxFruLastPowerOff 0, jnxFruLastPowerOn 0)

Aug 16 19:10:42.335 2019 MX_AD spmd[1857]: %DAEMON-5-SPMD_SNMP_TRAP10: SNMP trap generated: Fru Offline (jnxFruContentsIndex 108, jnxFruL1Index 106, jnxFruL2Index 2, jnxFruL3Index 0, jnxFruName SD106 PIC:4x 40GE @ 106/1/*, jnxFruType 11, jnxFruSlot 1, jnxFruOfflineReason 1,jnxFruLastPowerOff 0, jnxFruLastPowerOn 0)

Aug 16 19:10:42.335 2019 MX_AD spmd[1857]: %DAEMON-5-SPMD_SNMP_TRAP10: SNMP trap generated: Fru Offline (jnxFruContentsIndex 108, jnxFruL1Index 106, jnxFruL2Index 3, jnxFruL3Index 0, jnxFruName SD106 PIC:4x 1G/10G SFP/SFP+ @ 106/2/*, jnxFruType 11, jnxFruSlot 2, jnxFruOfflineReason 1, jnxFruLastPowerOff 0, jnxFruLastPowerOn 0)

Aug 16 19:10:42.335 2019 MX_AD spmd[1857]: %DAEMON-5-SPMD_SNMP_TRAP7: SNMP trap generated: Fru Online (jnxFruContentsIndex 108, jnxFruL1Index 106, jnxFruL2Index 1, jnxFruL3Index 0, jnxFruName SD106 PIC: 48x 10/100/1000 Base-T @ 106/0/*, jnxFruType 11, jnxFruSlot 0)

Aug 16 19:10:42.335 2019 MX_AD spmd[1857]: %DAEMON-5-SPMD_SNMP_TRAP7: SNMP trap generated: Fru Online (jnxFruContentsIndex 108, jnxFruL1Index 106, jnxFruL2Index 2, jnxFruL3Index 0, jnxFruName SD106 PIC: 4x 40GE @ 106/1/*, jnxFruType 11, jnxFruSlot 1)

Aug 16 19:10:42.335 2019 MX_AD spmd[1857]: %DAEMON-5-SPMD_SNMP_TRAP7: SNMP trap generated: Fru Online (jnxFruContentsIndex 108, jnxFruL1Index 106, jnxFruL2Index 3, jnxFruL3Index 0, jnxFruName SD106 PIC: 4x 1G/10G SFP/SFP+ @ 106/2/*, jnxFruType 11, jnxFruSlot 2)

When AD loses connection to SD106, the em service is restarted. 

User@MX_AD> show chassis services satellite

Slot: 106
Service Name: em
Service ID: 1661
Service Status: UP <--------------------------------- em service is Up after restart.
Service Last Restart: 2019-08-16 19:10:41 JST <------ em service is restarted
API Statistics:

User@MX_AD> show chassis satellite 

             Device          Cascade      Port       Extended Ports
Alias         Slot   State           Ports        State      Total/Up
MX_AD-101     101    Online          xe-3/2/2     online     50/48
                                     xe-3/3/2     online     
MX_AD-102     102    Online          xe-3/2/3     online     50/37
                                     xe-3/3/3     online     
MX_AD-103     103    Online          xe-3/0/2     online     50/49
                                     xe-3/1/2     online     
MX_AD-104     104    Online          xe-3/2/5     online     50/3
                                     xe-3/3/5     online     
MX_AD-105     105    Online          xe-3/0/3     online     50/9
                                     xe-3/1/3     online     
MX_AD-106     106    Online          xe-3/0/6     online     50/50 <------ online and stable after em restart
                                     xe-3/2/6     online <---------------- online and stable after em restart

The following messages are logged on lcmd.log .  

User@MX_AD> file show /var/tmp/sd106_201908170930/lcmd.log

Aug 16 19:10:42 localhost lcmd[1661]: Request to change management IP parameters
Aug 16 19:10:42 localhost lcmd[1661]: env_module_mgmt_addr_set:Resettingeth0 IP parameters
Aug 16 19:10:42 localhost lcmd[1661]: lcmd_get_username_crc:547: Sendingusername CRC 0x0
Aug 16 19:10:43 localhost lcmd[1661]: lcmd_get_username_crc:547: Sendingusername CRC 0x0
Aug 16 19:10:43 localhost lcmd[1661]: lcmd_plugin_set_tz:352: Timezone path:
Aug 16 19:10:43 localhost lcmd[1661]: env_module_tz_set:249: Setting timezone --
Aug 16 19:10:43 localhost lcmd[1661]: lcmd_get_username_crc:870: Sendingusername CRC 0x0
Aug 16 19:10:43 localhost lcmd[1661]: process_tz_setting:172: ftp completed ..

Note: lcmd is chassis-management-daemon. 

User@MX_AD> show chassis satellite extensive fpc-slot 106
                        Device          Cascade      Port       Extended
Alias            Slot   State           Ports        State      Ports

MX_AD-106        106    Online          xe-3/0/6     online     50/50
                                        xe-3/2/6     online

When                 Event                         Action

Aug 17 13:43:21.865  Local Provisioned
Aug 17 10:21:43.218  Local Provisioned
Aug 16 18:29:03.256  LocalProvisioned <----- No event found around Aug 16 19:10:42

 

Solution

It is common to see FRu offline/online traps if devices lose connection. The SNMP traps FRu offline/online does not mean any actual FRu restart. From the above log, you can see that the em service is restarted when traps are raised. Restarting the em service causes SNMP traps FRu to go offline/online.

The EM service is "Extended Mode" service, which supports multiple uplinks, and the ability to manage SD by using 802.1BR.

Restarting the em service causes SNMP traps FRu to go offline/online, but in this scenario, the em restart reason is not identified. The connection comes online quickly, but the SNMP traps FRu going offline/online does not indicate an actual FRu restart.

 

No action is required because this symptom does not have any service impact. There is no actual restart of the FRu.

 

Related Information