To provide resolution we need to first start collecting the Debug collector from both the RE’s . From the journalctl logs it was noticed that on the RE1 logs JSR(Junos Socket Replication) was removed. May 30 10:31:33.958886 JTASK_SYSTEM: JSR socket peek failed for task BGP_15169. 192.168.1.3 error:Unknown error Unknown error 1070731176As suspicion was with JSR , below logs and traceoptions needs to be enabled in a service impacting MW.
set protocols bgp traceoptions file bgp.log size 1g files 100set protocols bgp traceoptions flag nsr-synchronization detailset protocols bgp traceoptions flag commit-synchronizeset protocols bgp traceoptions flag thread-ioset protocols bgp traceoptions flag stateset protocols bgp traceoptions flag taskset protocols bgp traceoptions flag timerRPD live core(Both RE's) and below outputs to be collected as well from both RE's. request system core-dump routing-running show bgp neighborshow bgp replication detailFrom the JSR (Kernel side)
jtac-bgp.log:Jun 7 07:26:49.580545 rsync session terminated because New connection from backup jtac-bgp.log:Jun 7 07:26:59.995919 rsync session terminated because New connection from backup jtac-bgp.log:Jun 7 07:28:41.299967 rsync session terminated because Socket replication deactivated jtac-bgp.log.0:Jun 7 07:24:11.154031 rsync session terminated because New connection from backup jtac-bgp.log.0:Jun 7 07:24:21.578347 rsync session terminated because New connection from backup. From the debugs and logs it was later identified that there could possibly be a prefix that is causing the rsynch connection to fail. In order to catch the faulty prefix . TCPDUMP is required to be collected for the internal interface which is being used by RE’s to identify which prefix is causing the rsynch session to flap . JTDV0 is internal interface. Along with the above logs of BGP & NSR traces below tcpdump can be collected from shell of both RE's . tcpdump -w /var/tmp/master_re_dut_jtdv0.pcap -S -n -i jtdv0 tcp port 179 From the TCPDUMP below prefix was identified which did not have a local address when checked on the customer RSI & the same IP was configured on a local interface. 2024/174 06:07:01.086790 192.168.10..1 192.168.10.1 TCP 134 0 53317 ? 179 [SYN] Seq=7276056 Win=16384 Len=0 MD5 MSS=8192 SACK_PERM WS=1 Internet Protocol Version 4, Src: 128.0.0.4, Dst: 128.0.0.1 Internet Protocol Version 4, Src: 192.168.10.1, Dst: 192.168.10.1 Transmission Control Protocol, Src Port: 53317, Dst Port: 179, Seq: 7276056, Len: 0 Peer: 192.168.10.1 AS 115 Local: unspecified AS 101 Description: RR Group: AS115_MARKET: master Forwarding routing-instance: master Type: External State: Idle Flags: <> Last State: NoState Last Event: NoEvent Last Error: None Options: <AuthKey LogUpDown AddressFamily PeerAS PrefixLimit Refresh> Options: <MtuDiscovery> Options: <GracefulShutdownRcv> Authentication key is configured Address families configured: inet-unicast Holdtime: 90 Preference: 170 Graceful Shutdown Receiver local-preference: 0 Prefixlimit configured for NLRI: inet-unicast Limit: 100 Action: Log Only Number of flaps: 0 Malformed attributes log interval: 300 route limit: 1000 Trace options: all Trace file: /var/log//bgp_log size 1073741824 files 25 ae8 { mtu 9192; unit 0 { family inet { address 192.168.10.1/31; } } } This is causing rscynch in BGP code as the Master RE is unable to identify whether this is a genuine rsynch connection or not. Removing the BGP configuration in idle state fixed the issue and NSR synch was successful .