Customer is observing below log message:

fpc0 dnx_nh_unilist_install_multipath: Failed to create shadow obj 0x20001eb7 for NH 3185(FEC 0x20001b7b) unilist nh 2100031. Error -14(No resources for operation)

fpc0 dnx_nh_unilist_install_multipath: Failed to create shadow obj 0x20001eb7 for NH 1773(FEC 0x20001580) unilist nh 2100984. Error -14(No resources for operation)

Customer is observing below log message:

fpc0 dnx_nh_unilist_install_multipath: Failed to create shadow obj 0x20001eb7 for NH 3185(FEC 0x20001b7b) unilist nh 2100031. Error -14(No resources for operation)

fpc0 dnx_nh_unilist_install_multipath: Failed to create shadow obj 0x20001eb7 for NH 1773(FEC 0x20001580) unilist nh 2100984. Error -14(No resources for operation)

fpc0 PFE_ERROR_NOT_FOUND: NH: Unilist nh (2097185) with unknown nh in list (1030)

 fpc0 PFE_ERROR_FAIL_OPERATION: Type specific Add failed invalid argument nh-id:2097185

 fpc0 nh_indirect_initialize:949Failed to find Target 2097185 for indirect 2097168 creating a discard

 fpc0 nh_indirect_initialize:957Failed to create discard 2097185 

 fpc0 PFE_ERROR_FAIL_OPERATION: Params Initialization Failed generic failure nh-id:2097168

 fpc0 PFE_ERROR_FAIL_OPERATION: Failed to instantiate new copy for NH(2097168)

 fpc0 PFE_ERROR_FAIL_OPERATION: Failed to change state for NH (2097168)

 fpc0 PFE_ERROR_NOT_FOUND: NH: Unilist nh (2097186) with unknown nh in list (1034)

 fpc0 PFE_ERROR_FAIL_OPERATION: Type specific Add failed invalid argument nh-id:2097186

fpc0 dnx_nh_unilist_install_multipath: Failed to create shadow obj 0x20001eb7 for NH 3808(FEC 0x20001d9a) unilist nh 2101277. Error -14(No resources for operation)

fpc0 dnx_nh_unilist_install_multipath: Failed to create shadow obj 0x20001eb7 for NH 3478(FEC 0x20001c92) unilist nh 2099695. Error -14(No resources for operation)

fpc0 dnx_nh_unilist_install_multipath: Failed to create shadow obj 0x20001eb7 for NH 2401(FEC 0x200011a7) unilist nh 2103922. Error -14(No resources for operation)

From the above error message, error happened when next-hop is allocated with particular hw egress fec id - 0x20001eb7.

• So, it’s not the case of fec resource exhaustion, but that particular fec id is not available for usage in hw.
• It looks device has landed in a state that sw/hw are not in sync w.r.t. fec id 0x20001eb7 since this fec id is allocated for next-hop but when trying to configure hw using that fec id, hw returns error that fec resource is not available.
• This memory error also not seem to have impacted the required sw memory allocation. If its an issue with device memory exhaution, we would have seen in sw structure allocation also.
• It seems memory allocation failure in bcm hw programming as hw is in an inconsistent state.

Try Restart chassis-control / restart router to recover from issue state if the issue observe again.