This article explains jflow/sampling behaviour with Power mode IPsec.
The customer notices a huge difference in the number of input/output packets from the interface statistics of SRX compared to the jfow collecter. However, the SNMP server reports the correct statistics.
The device mainly handels IPSec traffic, according to counters ~99% of the traffic was IPSec and Power-mode Ipsec is in action for better IPSEC throughput. The Customer has the below sampling configured on the SRX and applied it on the interface.
root@srx# show forwarding-options sampling { instance { qos_instance { input { rate 1000; run-length 0; } root@srx# show interfaces ae0 unit 0 { family inet { sampling { input; output; } } }
When the packet input/output rate was checked from the SNMP and IPFIX server, a huge difference was noticed. From the SNMP server we could see the packet in/out on the interfaces around ~10G. But when we took a capture from the IPFIX server and checked the number of packets we could see ~58K packets.
IPFIX_Server~$ tshark -r netflow_30min_1.pcap -V | grep Packets | awk '{print $3}' | awk '{sum+=$1;}END{print sum;}' Running as user "root" and group "root". This could be dangerous. 57967
Considering the sampling rate of 1000, we were expecting to see 100M packets not ~58K packets which seems to be a huge difference.
The device has been set up to handle only IPSEC traffic and the below sampling configuration is added to the device. Also, it's applied to the ae0 interface.
root@srx# show forwarding-options sampling { instance { qos_instance { input { rate 100; run-length 0; } root@srx# show interfaces ae0 ae0 { aggregated-ether-options { lacp { active; periodic slow; } } unit 0 { family inet { sampling { input; output; }
>> PMI is by default disabled on versions before 21.2R1, so we need to enable it using the command "set security flow power-mode-ipsec". From 21.2R1 PMI is by default enabled and we need to disable it using the below command.
root@srx# show security flow | display set set security flow power-mode-disable
root@srx:~ # srx-cprod.sh -s spu -c "show usp flow config" | grep -i power Flow power mode express path: disabled Flow power mode IPsec: disabled Flow power mode IPsec QAT: disabled Flow power mode express path: disabled Flow power mode IPsec: disabled Flow power mode IPsec QAT: disabled
>> Pushed the traffic to the device and monitored the interface utilisation. Sampling is enabled only on the et- interface.
srx Seconds: 1328 Time: 18:16:40 Interface Link Input packets (pps) Output packets (pps) gr-0/0/0 Up 0 (0) 0 (0) ip-0/0/0 Up 0 (0) 0 (0) lt-0/0/0 Up 0 (0) 0 (0) xe-1/0/0 Up 0 (0) 0 (0) xe-1/0/1 Up 0 (0) 0 (0) et-1/1/0 Up 4827477 (0) 9616481 (0) xe-1/2/0 Up 9616681 (0) 4827278 (0) xe-1/2/1 Down 0 (0) 0 (0) et-1/3/0 Up 238 (0) 65 (0) ae0 Up 4827715 (0) 9616546 (0) avs0 Up 0 (0) 0 (0)
>> The number of input/output packets is 4827715 + 9616546 = 14444261
>> Now if we check the packets sampled we can see almost equal to the packets seen on the interface as well.
root@srx:~ # srx-cprod.sh -z0 -s spu -c "show fwdd statistics sample" | grep Sampled Packets: 0, Sampled: 0 Packets: 0, Sampled: 0 Packets: 0, Sampled: 0 Packets: 0, Sampled: 0 Packets: 14443637, Sampled: 144436 Packets: 0, Sampled: 0 Packets: 0, Sampled: 0
root@srx> show configuration security flow | display set set security flow load-distribution session-affinity ipsec
root@srx:~ # srx-cprod.sh -s spu -c "show usp flow config" | grep power Flow power mode express path: enabled Flow power mode IPsec: enabled Flow power mode IPsec QAT: disabled Flow power mode express path: enabled Flow power mode IPsec: enabled Flow power mode IPsec QAT: disabled
>> Interface traffic statistics during the tests.
srx Seconds: 144 Time: 18:34:41 Interface Link Input packets (pps) Output packets (pps) gr-0/0/0 Up 0 (0) 0 (0) ip-0/0/0 Up 0 (0) 0 (0) lt-0/0/0 Up 0 (0) 0 (0) xe-1/0/0 Up 0 (0) 0 (0) xe-1/0/1 Up 0 (0) 0 (0) et-1/1/0 Up 191 (0) 10506230 (0) xe-1/2/0 Up 10506438 (0) 5274741 (0) xe-1/2/1 Down 0 (0) 0 (0) et-1/3/0 Up 5274875 (0) 18 (0) ae0 Up 5275066 (0) 10506248 (0) avs0 Up 0 (0) 0 (0)
The number of input/output packets is 5275066 + 10506248 = 15781314, the expectation is to see the same number of packets in the fwdd statistics however we see less number since PMI handles most of the traffic.
root@srx:~ # srx-cprod.sh -z0 -s spu -c "show fwdd statistics sample" | grep Sampled Packets: 0, Sampled: 0 <trimmed> Packets: 0, Sampled: 0 Packets: 0, Sampled: 0 Packets: 0, Sampled: 0 Packets: 0, Sampled: 0 Packets: 7519494, Sampled: 75194 Packets: 0, Sampled: 0
root@srx:~ # cli show security flow pmi statistics Flow power-mode-IPSec statistics: PMI statistics of FPC3 PIC0: PMI received packets: 2329275 <trimmed> PMI statistics summary: PMI received packets: 27182548 PMI transmitted packets: 7494697 <<<<<<<<<< Half of the packets are transmitted by PMI. PMI regular flow path: 19687851 PMI dropped packets: 0 PMI encapsulation bytes: 0 PMI decapsulation bytes: 0 PMI encapsulation packets: 0 PMI decapsulation packets: 0
Since all the packets are not reaching the flowd SRX will not be able to sample the packets. Since the IPFIX reported numbers will also not match the interface statistics. According to the current design packets that are handled only by PMI or SOF cannot be sampled.