Description

Burst of "RPD_KRT_Q_RETRIES: list nexthop ADD: Device busy" logs are appearing after protocol flaps on multiple uplink/core interfaces 


No service impact was observed. 

Symptoms

May 1 08:56:28 2025 labrouter rpd[11048]: RPD_ISIS_ADJDOWN: IS-IS lost L1 adjacency to labneighbor on ae1.20, reason: Aged out


May 1 08:56:28 2025 labrouter rpd[11048]: RPD_RSVP_NBRDOWN: RSVP neighbor 96.34.XX.XXX down on interface ae1.20 nbr-type Direct, triggered by IGP neighbor down event

May 1 08:56:30 2025 labrouter rpd[11048]: RPD_RSVP_NBRUP: RSVP neighbor 96.34.XX.XXX up on interface ae1.20 nbr-type Direct


May 1 08:57:03 2025 labrouter rpd[11048]: RPD_ISIS_ADJDOWN: IS-IS lost L1 adjacency to labneighbor on ae3.20, reason: Aged out


May 1 08:57:03 2025 labrouter rpd[11048]: RPD_RSVP_NBRDOWN: RSVP neighbor 96.34.XX.XXX down on interface ae3.20 nbr-type Direct, triggered by IGP neighbor down event


May 1 08:57:04 2025 labrouter rpd[11048]: RPD_ISIS_ADJDOWN: IS-IS lost L1 adjacency to labneighbor on ae1.20, reason: Aged out


May 1 08:57:04 2025 labrouter rpd[11048]: RPD_RSVP_NBRDOWN: RSVP neighbor 96.34.XX.XXX down on interface ae1.20 nbr-type Direct, triggered by IGP neighbor down event


May 1 08:57:04 2025 labrouter rpd[11048]: RPD_ISIS_ADJDOWN: IS-IS lost L1 adjacency to labneighbor on ae2.20, reason: Aged out


May 1 08:57:04 2025 labrouter rpd[11048]: RPD_RSVP_NBRDOWN: RSVP neighbor 96.34.XX.XXX down on interface ae2.20 nbr-type Direct, triggered by IGP neighbor down event


May 1 08:57:07 2025 labrouter rpd[11048]: RPD_RSVP_NBRUP: RSVP neighbor 96.34.XX.XXX up on interface ae1.20 nbr-type Direct

May 1 08:57:07 2025 labrouter rpd[11048]: RPD_RSVP_NBRUP: RSVP neighbor 96.34.XX.XXX up on interface ae3.20 nbr-type Direct

May 1 08:57:07 2025 labrouter rpd[11048]: RPD_RSVP_NBRUP: RSVP neighbor 96.34.XX.XXX up on interface ae2.20 nbr-type Direct

May 1 08:57:24 2025 labrouter rpd[11048]: RPD_KRT_Qlabrouter_RETRIES: list nexthop ADD: Device busy

May 1 08:57:25 2025 labrouter last message repeated 72 times


....

Solution

The logs indicate that the kernel is throttling KRT, typically caused by a slow state consumer such as an FPC or PFE. These log entries are usually transient and should resolve without manual intervention. As observed in the output below, the KRT is currently clear and does not show any stuck entries.


If you notice any stuck entries in the KRT, kindly open a ticket with JTAC for further assistance.


labuser@labrouter> show krt state   

May 01 10:24:40

General state:

    Options: IndirectPFE IndirectChangeACK

    Install job is not running

    Number of operations queued: 0

        Routing table adds: 0

        Interface routes: 0

        High pri multicast  Adds/Changes: 0

        Indirect Next Hop  Adds/Changes: 0    Deletes: 0

        MPLS    Adds: 0    Changes: 0

        High pri  Adds: 0    Changes: 0    Deletes: 0

        Normal pri Indirects: 0

        Normal pri Adds: 0    Changes: 0    Deletes: 0

        GMP GENCFG Objects: 0

        Routing Table deletes: 0

    Number of operations deferred: 0

    Number of operations canceled: 0

    Number of async queue entries: 0

    Number of async non queue entries: 0

    Time until next queue run: 0

    Routes learned from kernel: 1310451


Routing socket lossage:

    Time until next scan: 54

Modification History

2025-05-06 : Article Created