This article explains the behavior change of the notification queue (PFE to RE queue) used to carry keepalive for IQ2 PIC.
IQ2 PIC reboots when a large amount of host bound traffic reaches the M320 switch. The following logs were observed: 3 22:06:12.546 M320 /kernel: %KERN-5: tcp_timer_rexmt: Dropping socket connection due to error: 60 3 22:06:12.546 M320 /kernel: %KERN-4: pfe_listener_disconnect: conn dropped: listener idx=3, tnpaddr=0x11030080, reason: generic peer error
3 22:06:12.546 M320 /kernel: %KERN-5: tcp_timer_rexmt: Dropping socket connection due to error: 60 3 22:06:12.546 M320 /kernel: %KERN-4: pfe_listener_disconnect: conn dropped: listener idx=3, tnpaddr=0x11030080, reason: generic peer error
In releases prior to 13.3R8, 14.1R3, 14.2R1, and 15.1R1, IQ2 PIC keepalives used notification queue Q1, which is a low priority queue. ICMP and other types of traffic may cause congestion in Q1, which can result in a PIC reboot.
SFPC1(MX320 vty)# show ichip 1 ptop counters Traffic stats: Counter Name Total Rate Peak Rate ---------------------- ---------------- -------------- -------------- Read Req RX 1217744 5385 5604 Read Resp TX 1217744 5385 5604 Read Resp Segment TX 2999647 18204 18204 Write Req RX 434755 122 2442 Write Req Segment RX 1457843 264 17409 Write Resp TX 434755 122 2442 Write Resp Segment TX 0 0 0 MTU Redirect TX 0 0 0 Age Check TX 0 0 0 Flow Control RX 496405 12325 12325 Flow Control TX 2 0 0 PIO Read RX 30430143 18204 18357 PIO Read TX 30430143 18204 18357 PIO Write RX 14612696 16412 18777 PIO Write TX 14612696 16412 18777 Notification Q[0] TX 730012 385 1934 Notification Q[1] TX 476751 5000 5043 Notification Q[2] TX 0 0 0 Notification Q[3] TX 0 0 0 Notification Q[4] TX 0 0 0 Notification Q[5] TX 0 0 0 Notification Q[6] TX 11029 0 334 Notification Q[7] TX 0 0 0
SFPC1(MX320 vty)# show ichip 1 r counters Traffic stats: Counter Name Total Rate Peak Rate ---------------------- ---------------- -------------- -------------- rcp_input_ucast 113089099 99880 113353 (BYTE) 6079865052 7744727 11869915 rcp_output_ucast 110955919 91166 94945 (BYTE) 5566313793 4109262 5533400 rcp_input_mcast 70395639 0 65007 (BYTE) 23962977142 0 22040548 rcp_output_mcast 70395630 0 65007 (BYTE) 23962976548 0 22040412 Discarded Packets: Counter Name Total Rate Peak Rate ---------------------- ---------------- -------------- -------------- JTREE_TRUNC_KEY 0 0 0 NEXHOP_TRUNC_KEY 0 0 0 ITABLE_TRUNC_KEY 0 0 0 TIMEOUT 0 0 0 IIF_CHECK 0 0 0 SRAM_PAR_ERR 0 0 0 EXEC_STACK_OFLOW 0 0 0 SID_ZERO 515585 1 1616 FW_DROP 1511 0 3 POLICER_DROP 63185 159 2898 INDEX_NBITS 0 0 0 FW_ILLEGAL_INSTR 0 0 0 RCP_INPUT_MCAST_DROP 0 0 0 RCP_UBR_CNTR 0 0 0 HNP discards: Counter Name Total Rate Peak Rate ---------------------- ---------------- -------------- -------------- Q0_DCS_CNTR 0 0 0 Q1_DCS_CNTR 315547 3161 11496 Q2_DCS_CNTR 0 0 0 Q3_DCS_CNTR 0 0 0 Q4_DCS_CNTR 0 0 0 Q5_DCS_CNTR 0 0 0 Q6_DCS_CNTR 0 0 0 Q7_DCS_CNTR 0 0 0
In Junos versions 13.3R8, 14.1R3, 14.2R1, 15.1R1 and later, this behavior is changed. The IQ2 PIC keepalives use Q0, which is a high priority queue. The chances of an IQ2 PIC reboot due to high ICMP traffic or other similar types of traffic is reduced.