Hardware failure of Master RCB caused mastership switchover. After this event, router became inaccessible via inband access. Removal of faulty RCB helped in regaining access to the router but caused impact to transit traffic. FPC Major, FPC SIB Link errors Alarms on new Master appeared after the removal of faulty RCB
Active Alarms
Minor LCM Peer Connection un-stable
Major FPC * Major Errors
Minor FPC * SIB Link Error
Core file on Host System
root@Router> show vmhost crash
Compute cluster: rainier-re-cc
Compute node: rainier-re-cn
Crash Info
==========
total 13932
-rw-r--r--. 1 root root 3137219 Oct 15 03:24 core-Router-node-lcmd.0.tgz
-rw-r--r--. 1 root root 3142967 Oct 15 03:24 core-Router-node-lcmd.1.tgz
...
Host Kern Logs reports LCMD process segfault
2023-10-15T03:24:38.933302+00:00 Router-node kernel: lcmd[21165]: segfault at 68 ip 00000000f7373cdd sp 00000000f6aaf1e0 error 4 in libplugin_pviultimat.so.1.0[f722d000+1b7000]
2023-10-15T03:24:50.379279+00:00 Router-node kernel: lcmd[2726]: segfault at 82 ip 00000000f733bcdd sp 00000000f6a771e0 error 4 in libplugin_pviultimat.so.1.0[f71f5000+1b7000]
Host LCMD Logs reports crash and LCMD process number change in every few seconds
Remove Faulty RCB from the chassis and perform a reboot of newly elected Master RE