Description

 This article describes the issue of high end SRX dropping fragmented packets.

Symptoms

The following excerpt is the Security Flow Trace Options output for the fragmented packets. We notice that SRX has received 23 packets, each of them being 32k in size. The attempt to re-assemble or merge them fails, with the Cannot allocate the net-pak error message being generated.


Sep 6 15:32:42 15:32:41.798759:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: a231e43->aa0017a,0570, cookie:00000046, nsp:3dbd8c54

Sep 6 15:32:42 15:32:41.798790:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: state:1, listlevel:0, hole_cnt:0, queued_frag:23

Sep 6 15:32:42 15:32:41.798805:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT:Level 0 packet:

Sep 6 15:32:42 15:32:41.798812:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836eed8: frag_offset 0, frag_end 1480

Sep 6 15:32:42 15:32:41.798831:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836f258: frag_offset 1480, frag_end 2960

Sep 6 15:32:42 15:32:41.798852:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836f6b8: frag_offset 2960, frag_end 4440

Sep 6 15:32:42 15:32:41.798874:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836ff08: frag_offset 4440, frag_end 5920

Sep 6 15:32:42 15:32:41.798927:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836fa38: frag_offset 5920, frag_end 7400

Sep 6 15:32:42 15:32:41.798948:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836f178: frag_offset 7400, frag_end 8880

Sep 6 15:32:42 15:32:41.798970:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836f028: frag_offset 8880, frag_end 10360

Sep 6 15:32:42 15:32:41.798991:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836f568: frag_offset 10360, frag_end 11840

Sep 6 15:32:42 15:32:41.799014:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836f2c8: frag_offset 11840, frag_end 13320

Sep 6 15:32:42 15:32:41.799036:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836fdb8: frag_offset 13320, frag_end 14800

Sep 6 15:32:42 15:32:41.799059:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836f808: frag_offset 14800, frag_end 16280

Sep 6 15:32:42 15:32:41.799081:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836f8e8: frag_offset 16280, frag_end 17760

Sep 6 15:32:42 15:32:41.799104:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836f958: frag_offset 17760, frag_end 19240

Sep 6 15:32:42 15:32:41.799127:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836f418: frag_offset 19240, frag_end 20720

Sep 6 15:32:42 15:32:41.799150:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836ff78: frag_offset 20720, frag_end 22200

Sep 6 15:32:42 15:32:41.799172:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836fc68: frag_offset 22200, frag_end 23680

Sep 6 15:32:42 15:32:41.799195:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836f488: frag_offset 23680, frag_end 25160

Sep 6 15:32:42 15:32:41.799217:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836eca8: frag_offset 25160, frag_end 26640

Sep 6 15:32:42 15:32:41.799240:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836fd48: frag_offset 26640, frag_end 28120

Sep 6 15:32:42 15:32:41.799262:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836ed18: frag_offset 28120, frag_end 29600

Sep 6 15:32:42 15:32:41.799284:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836f728: frag_offset 29600, frag_end 31080

Sep 6 15:32:42 15:32:41.799306:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836fe28: frag_offset 31080, frag_end 32560

Sep 6 15:32:42 15:32:41.799327:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: net-pak at 2836f3a8: frag_offset 32560, frag_end 32904

Sep 6 15:32:42 15:32:41.799349:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: Fragment reassemble complete

Sep 6 15:32:42 15:32:41.799363:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: FCB 0x509d6ef8, being removed from hash entry 0x51051608

Sep 6 15:32:42 15:32:41.799384:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: start to merge all fragments

Sep 6 15:32:42 15:32:41.799394:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: Cannot allocate the net-pak

Sep 6 15:32:42 15:32:41.799458:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT:defrag_begin returns fail: -1

Sep 6 15:32:42 15:32:41.799468:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT:defrag returned -1 for non-first frag

Sep 6 15:32:42 15:32:41.799478:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: L2 flow processing success. ret:-1

Sep 6 15:32:42 15:32:41.799488:CID-00:FPC-01:PIC-00:THREAD_ID-20:RT: ----- flow_process_pkt rc 0x7 (fp rc -1)

Solution

This issue occurs on all SRX platforms running 11.2 and earlier. This issue occurs as a result of SRX platforms having limited interface buffers to store fragmented packets, that is size<10k.

If the fragmented packets come in with a combined size of >10k, at the time of re-assembly, SRX will drop the packets due to the lack of interface buffer space to re-assemble the fragmented chuncks in one big packet.

With 11.4 and later, the Interface Buffer limit to re-assemble the fragmented chunks is increased to 64k on high end SRX; on the branch, the limit is still 10k.