This resolution KB article provides detailed information about the behaviour of TCP flows when the No-SYN-Check option is enabled in the TCP flow settings and TCP-RST is configured within the security zone.

When you configure "no-syn-check" under security flow and TCP-RST under the source security zone, the "no-syn-check" option takes precedence. This means that, even if TCP-RST is configured in the source security zone; if the first packet is not a SYN, the SRX will allow the packet to pass without checking the SYN flag, effectively bypassing the TCP-RST configuration.

• Scenario 1: No-SYN-Check Enabled + TCP-RST Configured
• If the first packet is not a SYN, the SRX allows it without triggering a TCP-RST.

set security flow tcp-session no-syn-check
set security zones security-zone untrust tcp-rst

• Scenario 2: SYN Check Enabled(By Default enabled) + TCP-RST Configured
• If the SYN check is enabled and the first packet is not a SYN, the SRX will send a TCP-RST to the source machine.

set security zones security-zone untrust tcp-rst

• Scenario 3: SYN Check Enabled + TCP-RST Not Configured
• If the SYN check is enabled but TCP-RST is not configured in the source security zone, the SRX will silently drop the packet without sending a TCP-RST to the source.