Description

This article outlines the steps & commands needed to generate a live core dump from SPC3 PICx

Solution

Live core dump to be taken during service impacting MW as this would offline the PIC.

Here is a snippet of the commands used for live core-dump collection taken from a lab node for reference [replace FPC & PIC id as needed (example refers to FPC 0 PIC 1); instructions are underlined and key values in bold for ease of understanding]:

1. root@LAB-re0> start shell user root ---->  Login to RE shell as root user
Password:
root@LAB-re0:/var/home/root #


2. root@LAB-re0:/var/home/root # cty -f fpc0 pic1 ---->  Command to console access fpc0 pic1, hit enter after cmd to login to to fpc0 pic1 shell


3. root@LAB-re0-fpc0:~# ps aux | grep flowd ---->  Command to grep flowd process, in bold is the correct process & pid
root       4300 0.0 0.0  9716 2608 ?       Ss  11:27  0:00 /bin/bash /usr/bin/pfe-app-wrapper /etc/init.d/flowd_spc3 0 start
root       4309 0.0 0.0  9452 2416 ?       S   11:27  0:00 /bin/bash /etc/init.d/flowd_spc3 start
root       4315 2206 1.3 128242376 1782460 ?  SLl 11:27 308:28 /usr/sbin/flowd_spc3.elf -d -r 1
root      11041 0.0 0.0  4392  748 ttyS0   S+  11:41  0:00 grep flowd

 

4. root@LAB-re0-fpc0:~# kill -6 <PID> ----> Use PID in bold from the previous output for process /usr/sbin/flowd_spc3.elf 
Message from syslogd@LAB-re0 at Sep 28 11:41:13.000 ...
LAB-re0 (FPC Slot 0, PIC Slot 1) daemon.emerg srxpfe[4315]: --------------------------------------
root@LAB-re0-fpc
Message from syslogd@LAB-re0 at Sep 28 11:41:13.000 ...
LAB-re0 (FPC Slot 0, PIC Slot 1) daemon.emerg srxpfe[4315]: ABORT!
0:~#
Message from syslogd@LAB-re0-fpc0 at Sep 28 11:41:13 ...
 srxpfe[4315]: --------------------------------------

Message from syslogd@LAB-re0-fpc0 at Sep 28 11:41:13 ...
 srxpfe[4315]: ABORT!
INIT: Sending processes the TERM signal

root@LAB-re0-fpc0:~# cd /var/tmp
root@LAB-re0-fpc0:/var/tmp# ls -ltr
total 28
-rw-r--r-- 1 root root  31 Sep 28 11:29 flowd_spc3.elf.log.LAB-re0-fpc0.pic1.0 ---> Live core dump generated & stored in below location


5. root@LAB-re0-fpc0:/var/tmp# scp flowd_spc3.elf.log.LAB-re0-fpc0.pic1.0 re0:/var/tmp ----> to copy core dump from FPC/PIC /var/tmp to RE0 /var/tmp location [assuming RE0 as mater RE]
The authenticity of host 're0 (128.0.0.4)' can't be established.
ECDSA key fingerprint is SHA256:KL1mB/2KFSberJ3/8AkMNYYD+53KpdqOyBJQxPb4nsg.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 're0,128.0.0.4' (ECDSA) to the list of known hosts.
Password:
flowd_spc3.elf.log.LAB-re0-f 100%  31   83.0KB/s  00:00
root@LAB-re0:/var/home/labroot # ----> Ctrl+C to exit
root@LAB-re0:/var/home/labroot #

 

labroot@LAB-re0> file list detail /var/tmp/ | match spc3   
Sep 28 11:42:44
-rw-r--r-- 1 root wheel        31 Sep 28 11:42 flowd_spc3.elf.log.LAB-re0-fpc0.pic1.0 ---->  Upload the file to case

 

labroot@LAB-re0> show chassis fpc pic-status 0   ---->  Generating a live core dump will bring the PIC offline
Slot 0  Online    SPC3                      
 PIC 0 Online    SPC3-PIC
 PIC 1 Offline   SPC3-PIC

 

labroot@LAB-re0> request chassis pic pic-slot 1 fpc-slot 0 online  ---->  To bring PIC online, if it gets stuck/fails to transition online, proceed with FPC restart
FPC 0 PIC 1 already transitioning to online

labroot@LAB-re0> request chassis fpc slot 0 restart   ---->  restart FPC 0 to bring PIC online
Restart initiated, use "show chassis fpc" to verify

Modification History

2024-09-29 : Article Created