The article explains the meaning of the PFEMAN logs that are being heavily printed in the syslog file and provides guidance on how to address and resolve the issue

The syslog files are getting flooded with a large number of the following logs:

Nov 13 14:45:01 SRX node0.fpc1 Routing engine PFEMAN reconnection succeeded after 1 tries
Nov 13 14:45:01 SRX node0.fpc1 PFEMAN master RE reconnection made with 0x1008081
Nov 13 14:45:01 SRX node0.fpc1 Probe exists for str bulkget_dfw
Nov 13 14:45:01 SRX node0.fpc1 PFEMAN: Master socket closed
Nov 13 14:45:01 SRX node0.fpc1 PFEMAN disconnected; PFEMAN socket closed abruptly
Nov 13 14:45:01 SRX node0.fpc0 Routing engine PFEMAN reconnection succeeded after 1 tries
Nov 13 14:45:01 SRX node0.fpc0 PFEMAN master RE reconnection made with 0x1008081
Nov 13 14:45:01 SRX node0.fpc0 Probe exists for str bulkget_dfw
Nov 13 14:45:01 SRX node0.fpc0 PFEMAN: Master socket closed
Nov 13 14:45:01 SRX node0.fpc0 PFEMAN disconnected; PFEMAN socket closed abruptly
Nov 13 14:45:01 SRX node0.fpc1 Routing engine PFEMAN reconnection succeeded after 1 tries
Nov 13 14:45:01 SRX node0.fpc1 PFEMAN master RE reconnection made with 0x1008081

For example, the log file named messages is rotating extremely quickly:

messages Size: 5180753, Last changed: Nov 13 14:51:55
messages.0.gz Size: 172433, Last changed: Nov 13 14:45:00
messages.1.gz Size: 169829, Last changed: Nov 13 14:30:00
messages.2.gz Size: 171178, Last changed: Nov 13 14:15:00
messages.3.gz Size: 171336, Last changed: Nov 13 14:00:00
messages.4.gz Size: 169851, Last changed: Nov 13 13:45:00
messages.5.gz Size: 169998, Last changed: Nov 13 13:30:00
messages.6.gz Size: 169974, Last changed: Nov 13 13:15:00
messages.7.gz Size: 170925, Last changed: Nov 13 13:00:00
messages.8.gz Size: 170218, Last changed: Nov 13 12:45:00
messages.9.gz Size: 170003, Last changed: Nov 13 12:30:00

This issue was observed after the customer performed the following steps:

1. Node 1: ISSU was initiated and completed successfully.
2. Node 0: After the ISSU completion on Node 1, Node 0 remained the primary for all RGs.
3. Node 0: Manual failovers for RG0 and RG1 were triggered to Node 1.
4. Node 0: After the failover in Step 3, the messages log file started showing the logs mentioned in the symptom section.

After the failover of RG0 from Node 0 to Node 1, all the cards on Node 0 (now secondary) are expected to connect to the master RE IP on Node 1. However, the pfeman connection from Node 0 incorrectly attempts to use the local RE address instead of the master RE address on Node 1. Initially, the connection succeeds, but the kernel subsequently rejects it, causing the process to enter a continuous loop of connection attempts and rejections.

Nov 13 14:45:01 SRX node0.fpc1 Routing engine PFEMAN reconnection succeeded after 1 tries
Nov 13 14:45:01 SRX node0.fpc1 PFEMAN master RE reconnection made with 0x1008081
Nov 13 14:45:01 SRX node0.fpc1 Probe exists for str bulkget_dfw
Nov 13 14:45:01 SRX node0.fpc1 PFEMAN: Master socket closed

• Expected Master RE Address: 130.160.0.1 (0x1008082)
• Actual Connection Attempt: 129.160.0.1 (0x1008081)

This connection issue can be resolved by rebooting the impacted node.