Description

The SRX went unresponsive when the customer were trying to perform manual failover. 

 

1.Cluster status before user executed "request chassis cluster failover node 0 redundancy-group 1"

Cluster ID: 1

Node   Priority Status               Preempt Manual   Monitor-failures

Redundancy group: 0 , Failover count: 1

node0  100      secondary            no      no       None          

node1  1        primary              no      no       None          


Redundancy group: 1 , Failover count: 1

node0  100      secondary            no      no       None          

node1  1        primary              no      no       None          



2.Cluster status after user executed "request chassis cluster failover node 0 redundancy-group 1"

Redundancy group: 0 , Failover count: 1

node0  100      secondary            no      no       None          

node1  1        primary              no      no       None          

Redundancy group: 1 , Failover count: 2

node0  255      primary              no      yes      None          

node1  1        secondary            no      yes      None       
   

3.User then executed request chassis cluster failover node 0 redundancy-group 0

4.The device lost connection after this.

Symptoms

  • System went down and rebooted on it's own. 
  • Phase-2 did not come up automatically on reboot, needed clearing the sessions.

Solution

  1. Be careful with redundancy group 0 manual failovers. This type of failover means switching to a new primary Routing Engine (RE), which kills all processes on the old RE and restarts them on the new one. This can lead to loss of routing state and performance issues due to system churn.
  2. Make sure the node you're doing failover is a healthy state. (Logs had indicated that node-0 was lost after power-off).
  3. After doing a manual failover, you must issue the reset failover command before requesting another failover.
  4. You need to wait for at-least 5 minutes before executing next failover. Check for cluster-status before executing next failover.

For more information on manual failovers procedure, please refer to this documentation Understanding Chassis Cluster Redundancy Group Manual Failover.

Modification History

2024-10-20 : Article Created