Description

This article describes the Inventory Synchronization job in addition to a failure scenario and the cause behind it.

Note: Inventory Synchronization is an add-on job which is patched to every JOB for DVPN creation or deletion to get the device stats/configuration to make sure that configuration duplication does not happen (for example the GRE tunnel interface configured or deleted).

Symptoms

The job fails with the following error:

Task: resync-topo-device

Sep 15 2020 11:37:07 AM discover resources task in-progress
Sep 15 2020 11:37:07 AM Task started
Sep 15 2020 11:37:07 AM discover resources task in-progress
Sep 15 2020 11:38:36 AM discover resources task failed
Sep 15 2020 11:38:36 AM Task complete
Sep 15 2020 11:38:36 AM discover resources task failed

Solution

From the TSSM logs, it was discovered that a command execution on the CPE failed.

The error that was discovered:

Upon checking the command output (XML), the error came from the device side:

<output-pps>
96072


<logical-interface>
nc:rpc-error
nc:error-typeprotocol
nc:error-tagoperation-failed
nc:error-severityerror
<source-daemon>
ifinfo

nc:error-message
remote statistics get for interface reth4.0 failed: Resource temporarily unavailable (35)






]]>]]>
"},

Since this RPC is relative to retrieve interface utilization, checking the same output from the platform side, the same error is seen for reth4:

junoscript interactive
rpc <get-interface-information>

When checking the said interface statistics, an error was seen:

root@srx> show interfaces reth4
Physical interface: reth4, Enabled, Physical link is Up
Interface index: 132, SNMP ifIndex: 565
Description: AUX_0 Connected to JCP
Link-level type: Ethernet, MTU: 1518, Speed: 10Gbps, BPDU Error: None, Ethernet-Switching Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled,
Flow control: Disabled, Minimum links needed: 1, Minimum bandwidth needed: 1bps
Device flags   : Present Running
Interface flags: SNMP-Traps Internal: 0x4000
Current address: 00:10:db:ff:10:04, Hardware address: 00:10:db:ff:10:04
Last flapped   : 2020-09-24 15:40:04 UTC (4d 00:53 ago)
Input rate     : 944 bps (0 pps)
Output rate    : 0 bps (0 pps)
error: remote statistics get for interface reth4.0 failed: Resource temporarily unavailable (35)  

Hence, the error of the Job failure was due to a command execution failure on the CPE and not in the CSO.
The issue needs to be debugged from a platform prospective.