Article Detail

Description

This article describes the issue of SRX responding with the incorrect ACK packet for the RSH session, which causes the client to send the RST packet (SYN and SEQ checks are disabled).

Symptoms

Specific to RSH ALG sessions; after the session has been closed with full 4-way FIN close, it goes into the Time Wait state.

Sample output of session in Time Wait state:

root@SRX210> show security flow session session-identifier 10293 Session ID: 10293, Status: Normal Flag: 0x80000000 Policy name: test1/4 Source NAT pool: interface, Application: junos-rsh/2 Maximum timeout: 2, Current timeout: 2 Session State: Valid Start time: 611066, Duration: 19 In: 10.7.105.59/1023 --> 10.8.48.1/514;tcp, Interface: fe-0/0/4.0, Session token: 0x7, Flag: 0x0x621 Route: 0x190010, Gateway: 192.168.33.10, Tunnel: 0 Port sequence: 0, FIN sequence: 1497941576, FIN state: 2, Pkts: 10, Bytes: 438 Out: 10.8.48.1/514 --> 10.7.105.59/1023;tcp, Interface: ge-0/0/0.0, Session token: 0x8, Flag: 0x0x620 Route: 0x180010, Gateway: 172.27.117.46, Tunnel: 0 Port sequence: 0, FIN sequence: 3700428809, FIN state: 2, Pkts: 7, Bytes: 421 Total sessions: 1

Note : For more information about session FIN state, refer to KB22738 - [SRX] SYN packet gets dropped in the TCP session. [juniper.net]

At this time, if the client starts another RSH session (take note that RSH tends to reuse source ports), the SYN packet will match the old session in Time Wait state. In such instances, the SRX may respond with an incorrect ACK packet to the client, as the TCP Proxy module on SRX is in an incorrect state.

The following diagram illustrates the traffic flow:

alt

The flow trace will also show this:

The session is created with the following session ID.

[0897] T31 <192.168.1.100/1023->1.1.1.100/514;6> : [0898] T31 packet [48] ipid = 5082, @79bdd918 .... [0906] T31 reth1.0:192.168.1.100/1023->1.1.1.100/514, tcp, flag 2 syn [0907] T31 find flow: table 0x586ec398, hash 164727(0x7ffff), sa 192.168.1.100, da 1.1.1.100, sp 1023, dp 514, proto 6, tok 6 [0908] T31 no session found, start first path. in_tunnel - 0, from_cp_flag - 2048 [0909] T31 flow_first_create_session [0910] T31 Installing pending sess (1705) in ager [0911] T31 First path alloc and instl pending session, natp=0x378aad98, id=1705 [0912] T31 flow_first_in_dst_nat: in <reth1.0> , out <N/A> dst_adr 1.1.1.100, sp 1023, dp 514 [0913] T31 chose interface reth1.0 as incoming nat if. [0914] T31 flow_first_rule_dst_xlate: DST no-xlate: 0.0.0.0(0) to 1.1.1.100(514) [0915] T31 flow_first_routing: call flow_route_lookup(): src_ip 192.168.1.100, x_dst_ip 1.1.1.100, in ifp reth1.0, out ifp N/A sp 1023, dp 514, ip_proto 6, tos 0 [0916] T31 Doing DESTINATION addr route-lookup [0917] T31 flow_rt_lkup: nh word 0x84663c2 [0918] T31 flow_ipv4_rt_lkup success 1.1.1.100, iifl 0x4e, oifl 0x4f [0919] T31 routed (x_dst_ip 1.1.1.100) from Trust (reth1.0 in 1) to reth2.0, Next-hop: 1.1.1.100 [0920] T31 policy search from zone Trust-> zone Untrust (0x0,0x3ff0202,0x202) [0921] T31 app 2, timeout 1800s, curr ageout 20s [0922] T31 Permitted by policy 4 [0923] T31 flow_first_src_xlate: nat_src_xlated: False, nat_src_xlate_failed: False [0924] T31 flow_first_src_xlate: src nat returns status: 0, rule/pool id: 0/0, pst_nat: False. [0925] T31 dip id = 0/0, 192.168.1.100/1023->192.168.1.100/1023 [0926] T31 choose interface reth2.0 as outgoing phy if [0927] T31 is_loop_pak: No loop: on ifp: reth2.0, addr: 1.1.1.100, rtt_idx:0 [0928] T31 check nsrp pak fwd: in_tun=0x0, VSD 1 for out ifp reth2.0 [0929] T31 vsd 1 is active .... [0952] T31 Session (id:1705) created for first pak 80a2 <<<< [0953] T31 first pak processing successful

After the 4-way FIN has completed, you can see that the client starts the new session, almost immediately. This new SYN packet matches the old session ID, as seen below:

[3035] T30 <192.168.1.100/1023->1.1.1.100/514;6> : [3036] T30 packet [48] ipid = 5094, @7a2c78e4 [3037] T30 ---- flow_process_pkt: (thd 30): flow_ctxt type 13, common flag 0x3, mbuf 0xfce6600, rtbl_idx = 7 .... [3046] T30 in_ifp <Trust:reth1.0> .... [3053] T30 flow session id 1705 <<<<

SRX then responds with the incorrect ACK packet and the session is subsequently aged out. This is also seen in the above diagram (new2).

However, the client responds to the incorrect ACK packet with a RST packet, which then creates a new session with the timeout of 1800 seconds (or the configured default timeout).

[3156] T12 <192.168.1.100/1023->1.1.1.100/514;6> : [3157] T12 packet [40] ipid = 5095, @7a330918 .... [3165] T12 reth1.0:192.168.1.100/1023->1.1.1.100/514, tcp, flag 4 rst <<<<< [3167] T12 no session found, start first path. in_tunnel - 0, from_cp_flag - 2048 .... [3172] T12 flow_first_create_session

This could result in subsequent packets being dropped, due to the invalid session being created by the RST packet.

Solution

There are two issues here:

Incorrect ACK response packet from SRX, due to the TCP Proxy module on SRX being in an incorrect state.
RST packet creates a new session with the timeout of 1800 seconds, which results in the new traffic that matches this session being dropped for 1800 seconds.

The fix for this issue is in the following releases 10.4R10, 11.4R3, 11.2R7, and 12.1R2.

These issues are addressed by the following two ways:

SRX will no longer respond with the incorrect ACK packet, if the SYN packet matches the session in Time Wait state.
IF the RST packet is sent from the client (for RSH traffic) and SYN check is disabled, a session will be created only for 2 seconds; instead of 1800 seconds.

This issue is specific to the behavior of RSH in combination with SYN and SEQ checking being disabled. To further explain point 2, as the SYN check is disabled, RST packet will be able to create a new session.

In terms of normal traffic (non-RSH), it has been noticed that with SYN check being disabled, if the RST packet is received by SRX, a session will be created. The session is usually created with the timeout of 2 seconds.

This is considered a kind of invalid session; so, during the 2 seconds, any traffic matching this session will be dropped. But generally, this behavior is not often noticed; except for RSH traffic, which reuses its source port.