Description

On QFX10k, cannot install ARP entries up to the maximum limit. This KB describes this behavior and how to solve this issue.

Symptoms

According to the datasheet of QFX10k, ARP entries can be installed up to 192,000, or 64,000 per PFE. However, QFX10k cannot install all of them with default config.
 

Solution

On QFX10k, the default value of system ARP limit is 75,000 and Public ARP limit is 59,840. Public ARP is used for users, so you can refer to this value in the test. And you can check this value by command below.

root@QFX10k# run show system statistics | grep limit
   0 option packets dropped due to rate limit
   0 drops due to rate limit
   75000 Max System ARP nh cache limit <<<=== System ARP
   59840 Max Public ARP nh cache limit <<<=== Public ARP

This value can be configurable by arp-system-cache-limit option described below.
Transport and Internet Protocols User Guide

This is the command to change the System ARP, and Public ARP will be changed along with the value of System ARP.

Here is the calculation for Public ARP:
System ARP x 0.8 - 160 = Public ARP


And currently we are supporting up to 96k entry per PFE and 288k in system. So config below will help increase the Public ARP to maximum scaled value.

set system arp-system-cache-limit 360000

root@QFX10k# run show system statistics | grep limit
     0 option packets dropped due to rate limit
     0 drops due to rate limit
     360000 Max System ARP nh cache limit  <<<=== System ARP
     287840 Max Public ARP nh cache limit  <<<=== Public ARP

And ARP learnt on IRB will be programmed on PFE0 as per design. If exceeded the limit, you will see the error message like below.
Mar 31 12:34:37.855 2023 QFX10k kernel: %KERN-3: prds_token_alloc: Token 98304 in use for global allocation. Allocation failed for type L3 on PE (id=0).

Modification History

2023-06-27 : Created a KB