1. When an fpc is removed ungracefully from the chassis , hwdre may crash.
2. This may also lead to chassis alarm with "FPC x Power Failure" & "Fpcx Node unreachable" and FPC SN can still reflect against same slot even after removal. This issue can lead to FPC never coming online in this slot.
3. System reboot is required to recover from this fault state.

{master}

re1> show chassis alarms 

7 alarms currently active

Alarm time              Class Description

2024-03-19 11:13:34 PDT Major FPC 0 Power Failure

2024-03-19 11:13:41 PDT Major Fpc0 Node unreachable

{master}

re1> show chassis hardware | match fpc    

Mar 19 11:18:38

FPC 0           REV 50  750-093524   BCDR0293         JNP10K-LC1201 >>>> FPC removed from this slot

FPC 3           REV 50  750-093524   BCDR0293         JNP10K-LC1201 >>>> FPC inserted into this slot

re0> show system core-dumps | match hwd 

Mar 19 12:17:02

-rw-r--r-- 1 root root   12513666 Mar 19 11:39 /var/core/re0/hwdre.re.re0.12970.2024_03_19.11_37_23.tar.gz

-rw-r--r-- 1 root root   21494722 Mar 19 11:15 /var/core/re1/hwdre.re.re1.23319.2024_03_19.11_13_39.tar.gz

Mar 05 02:31:25.033359 re1 kernel: jnx-connector fpc1: empty:insert_ev -> insert_wait:insert_done_ev

Mar 05 02:31:25.033511 re1 kernel: jnx-connector fpc1: insert_wait:insert_done_ev

Mar 05 02:31:25.033751 re1 kernel: jnx-connector fpc1: insert_wait:insert_done_ev -> loading0:timeout_ev

<..>

Mar 05 02:31:25.033359 re1 kernel: jnx-connector fpc1: empty:insert_ev -> insert_wait:insert_done_ev

Mar 05 02:31:25.033511 re1 kernel: jnx-connector fpc1: insert_wait:insert_done_ev

Mar 05 02:31:25.033751 re1 kernel: jnx-connector fpc1: insert_wait:insert_done_ev -> loading0:timeout_ev

<..>

Mar 05 02:18:18 re1 hwdre[16629]: fru_state_transition /Chassis[0]/Fpc[1] UNKNOWN -> PRESENT

Mar 05 02:26:12 re1 hwdre[16629]: fru_state_transition /Chassis[0]/Fpc[1] PRESENT -> POWER_ON_WAIT

<..>

/Chassis[0]/Fpc[1] ONLINE -> FAULT

Mar 05 02:31:20 re1 hwdre[16629]: fru_state_transition /Chassis[0]/Fpc[1] FAULT -> EMPTY

<..>

Mar 05 02:31:15 re1 kernel: jnx-connector fpc1: active:powerbad_ev -> failed:powerbad_ev

Mar 05 02:31:18 re1 kernel: jnx-connector fpc1: failed:remove_ev

Mar 05 02:31:19 re1 kernel: jnx-connector fpc1: Failed to read i2c client @54[20], error (-5)

Mar 05 02:31:19 re1 kernel: jnx-connector fpc1: failed:remove_ev -> 

 

As a precautionary method, always OFFLINE the FPC before removing from the chassis.

For permanent solution, device needs to be upgraded to fixed release of PR: 1781493

This issue is fixed in PR: 1781493.