When enabling IPsec fat-core feature, it may not function correctly - the FPC CPU load is still topping only one CPU thread instead of distributing the load among fat-core threads group.
For example with one IPsec VPN tunnel carrying a fat-flow session, the FPC CPU load will top at 100% on only one FPC CPU thread even after enabling IPsec PMI and IPsec fat-core features:
user@srx# run show security monitoring performance spu extensive CPU Util Wutil Status SchedCounter FPC0 PIC0 0 99 95 alive 412677 1 0 0 alive 412677 2 0 0 alive 412677 3 0 0 alive 412677 4 0 0 alive 412677 5 0 0 alive 412677 6 0 0 alive 412677 7 0 0 alive 412677 8 0 0 alive 412677 9 0 0 alive 412677 10 0 0 alive 412677 11 0 0 alive 412677 12 0 0 alive 412677 13 0 0 alive 412677 14 0 0 alive 412677 15 0 0 alive 412677 16 0 0 alive 412677 17 0 0 alive 412677 18 0 0 alive 412677 19 0 0 alive 412677 20 0 0 alive 412677 21 0 0 alive 412677 22 0 0 alive 412677 23 0 0 alive 412677
This happens because IPsec fat-core features do not support flexible VLAN tagging to be configured on the interface handling the traffic sessions going to/from IPsec tunnel.
The solution is to disable flexible VLAN tagging on the interfaces handling the traffic sessions going to/from IPsec tunnel.
For example, below is an unidirectional fat-flow session, the traffic enters SRX on If: xe-1/1/4.3024 and is encrypted to the IPsec tunnel exiting the SRX on If: st0.0:
Session ID: 4356036590, Policy name: JTAC-test/15, HA State: Active, Timeout: 60, Session State: Valid
In: 10.1.1.1/63 --> 10.2.2.2/5000;udp, Conn Tag: 0x0, If: xe-1/1/4.3024, Pkts: 411768, Bytes: 526239504, HA Wing State: Active,
Out: 10.2.2.2/5000 --> 10.1.1.1/63;udp, Conn Tag: 0x0, If: st0.0, Pkts: 0, Bytes: 0, HA Wing State: Active,
The interface xe-1/1/4.3024 is configured with flexible-vlan-tagging and this is preventing the IPsec fat-core from distributing the processing load for this session among the IPsec fat-core group of CPU threads.
After deleting the interface xe-1/1/4.3024 flexible-vlan-tagging and replacing it with regular vlan-tag the IPsec fat-core starts distributing the processing load for this session among the IPsec fat-core group of CPU threads resolving the one CPU thread bottleneck problem:
user@srx> show security monitoring performance spu extensive CPU Util Wutil Status SchedCounter FPC0 PIC0 0 39 39 alive 493537 1 87 83 alive 493537 2 84 83 alive 493537 3 88 84 alive 493537 4 88 85 alive 493537 5 88 85 alive 493537 6 82 82 alive 493537 7 88 84 alive 493537 8 89 86 alive 493537 9 89 85 alive 493537 10 77 71 alive 493537 11 89 86 alive 493537 12 0 0 alive 493537 13 0 0 alive 493537 14 0 0 alive 493537 15 0 0 alive 493537 16 0 0 alive 493537 17 0 0 alive 493537 18 0 0 alive 493537 19 0 0 alive 493537 20 0 0 alive 493537 21 0 0 alive 493537 22 0 0 alive 493537 23 0 0 alive 493537
The IPsec fat-core features do not support flexible VLAN tagging to be configured on the interface handling the traffic going to/from IPsec tunnel.