Active Directory authentication failures may occur on Windows client terminals when MSRPC ALG is enabled. Despite having policies configured to permit traffic from the trust zone to the untrust zone, users are unable to authenticate, and the following log messages appear in system logs:

• RT_ALG_ERR_RES_ALLOC: [name="MSRPC" message="create ASL group failed"]
• RT_ALG_ERR_MEM_ALLOC: [name="MSRPC" message="reach maximum msrpc group num"]
• RT_ALG_ERR_MEM_ALLOC: [name="MSRPC" message="reach maximum msrpc group num"]

The maximum number of MSRPC groups allowed is 40,960.

node0.fpc0> show usp algs ms-rpc map-table  
MSRPC active map entry:          0  
MSRPC active group number:       42462  >> Higher then the maximum group number 
MSRPC maximum group number:      40960  
MSRPC map lifetime (min):        480  
MSRPC map reserved timeout factor: 0.500

To verify the current active group usage for MSRPC, use the following command:

root@device> show security resource-manager group active
Total groups 76800, active groups 42462

Note: The total 42462 includes all ALG active groups, including MSRPC ones.

To resolve the issue of resource exhaustion caused by MSRPC traffic, follow these steps.
Note that these actions may impact active user sessions, requiring users to re-establish their connections.

1. Disable MSRPC ALG (if not required)
 
If MSRPC traffic is not required, disable the MSRPC ALG to free up resources:
 
> set security alg msrpc disable
> commit

Alternatively, if MSRPC is required, ensure proper configuration and resource optimization.

2. Clear MSRPC Resources: Free up MSRPC resources using the following command, which will terminate existing sessions. This command must be executed at the PFE level:

FLOWD_SRX4100# clear usp algs ms-rpc ?
    group-usage          Clear MS-RPC ALG group-usage value
    map-table            Clear MS-RPC ALG mapping table
    stats                Clear MS-RPC ALG statistics

3. Adjust Mapping Entry Timeout:
 
Reduce the MSRPC ALG mapping entry timeout value to conserve resources. By default, the timeout is 8 hours. To further reduce resource usage, configure a lower value (e.g., 1 hour):
 
set security alg msrpc map-entry-timeout <timeout-value>
commit
 
Example:
 
set security alg msrpc map-entry-timeout 1  // Set timeout to 1 hour

4. Enable Traceoptions for Debugging:
 
Enable trace options to collect detailed logs for further troubleshooting:
 
set security alg msrpc traceoptions flag all extensive
commit
 
This will log comprehensive details about MSRPC traffic and resource allocation.