In a rare scenario, it can happen that the Dynamic Host Configuration Protocol (DHCP) may assign an IP address to the client interface. After the Zero Touch Provisioning (ZTP) process is complete and the configuration file obtained by ZTP is committed, a change may occur in the client's DHCP-assigned IP address based on the configuration file obtained through ZTP.

This can result in the addition of an extra access-internal route, which does not get cleared after the jdhcpd is terminated, in turn resulting in stale access-internal routes. This article suggests to restart the RPD to clear any additional access-internal routes that may have been added.

​The issue can be detected by using the show route 0.0.0.0 command.

root@hostname> show route    

inet.0: 9 destinations, 10 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0.0.0.0/0          *[Access-internal/12] 00:03:03, metric 0
                    > to 4.4.4.4 via em0.0

The sequence of events during the DHCP phase of the ZTP process on affected routers is given below. You can verify the following logs from image_load_log and dhcp_logfile .

Addition of a default route is required for config-file transfer (4.4.4.1 is the IP address assigned by DHCP server in this case):

Jul 17 01:10:13.432210 [MSTR][INFO] [default:default][CLN][INET][em0.0] change_sus_if_addr: Changing if_addr to 4.4.4.1: intf em0.0 (stack 0x9ce2900); INET safd(0x9c17e80); dhcp_type 3; LSRI default:default      
Jul 17 01:10:13.432264 [MSTR][INFO] [default:default][CLN][INET][em0.0] change_sus_if_addr: Changed intf em0.0 INET safd if_addr from 0.0.0.0 to 4.4.4.1
Jul 17 01:10:13.432289 [MSTR][INFO] [default:default][CLN][INET][em0.0] sus_by_ip_addr_add: [INET] - Added 'subunit by ip-addr' entry 0x9f3e600, safd 0x9c17e80 for ip ADDR 4.4.4.1 type 3 in routing context LR default/RI default (ref_count 1)
Jul 17 01:10:13.432331 [MSTR][DEBUG][default:default][CLN][INET][em0.0] jdhcpd_ifam_handler: Added new IFA: ifa name em0, family INET, sub 0, addr 4.4.4.1/26
Jul 17 01:10:13.434290 [MSTR][INFO] [default:default][CLN][INET][em0.0][SID=0] JDHCPD_CLIENT_EVENT: Client(0x9c05400) got event CLIENT_EVENT_IFA_CHANGE in state LOCAL_CLIENT_STATE_BOUNDJul 17 01:10:13.434327 [MSTR][DEBUG][default:default][CLN][INET][em0.0][SID=0] jdhcpd_client_state_bound: ifa change ifl = em0.0, install default static route
Jul 17 01:10:13.434345 [MSTR][INFO] [default:default][CLN][INET][em0.0][SID=0] jdhcpd_rpd_add_default_route: set rt_params with rta_flags 4 cookie 7
Jul 17 01:10:13.435115 [MSTR][DEBUG][default:default][CLN][INET][em0.0][SID=0] jdhcpd_rpd_add_default_route: Add default route successfully. local 4.4.4.1 nh 4.4.4.4 ifl em0.0/7

The bootstrap/initial configuration is then committed:

[Wed Jul 17 01:10:24 UTC 2019] Terminating child process 13809
[Wed Jul 17 01:10:24 UTC 2019] Termination result 0
[Wed Jul 17 01:10:24 UTC 2019] PID val 13186
[Wed Jul 17 01:10:24 UTC 2019] Terminating image_load [13186] process
[Wed Jul 17 01:10:24 UTC 2019] Auto Image Upgrade: Committed Configuration DR993?circuit_id=%2C%2C7%2C%2C1001&relay_agent_ip=4.4.4.4 received from 10.10.10.10 through em0
[Wed Jul 17 01:10:24 UTC 2019] Clear dhcp client bindings

A duplicate access-internal default route (2.2.2.1 is the IP address of the em0 interface configured in the bootstrap/initial configuration) is added:

Jul 17 01:10:24.597731 [MSTR][INFO] [default:default][CLN][INET][em0.0] change_sus_if_addr: Changed intf em0.0 INET safd if_addr from 4.4.4.1 to 2.2.2.1
Jul 17 01:10:24.597766 [MSTR][DEBUG][default:default][CLN][INET][em0.0] jdhcpd_ifam_handler: Added new IFA: ifa name em0, family INET, sub 0, addr 2.2.2.1/26
Jul 17 01:10:24.597821 [MSTR][INFO] [default:default][CLN][INET][em0.0][SID=0] JDHCPD_CLIENT_EVENT: Client(0x9c05400) got event CLIENT_EVENT_IFA_CHANGE in state LOCAL_CLIENT_STATE_BOUND
Jul 17 01:10:24.597839 [MSTR][DEBUG][default:default][CLN][INET][em0.0][SID=0] jdhcpd_client_state_bound: ifa change ifl = em0.0, install default static route
Jul 17 01:10:24.597855 [MSTR][INFO] [default:default][CLN][INET][em0.0][SID=0] jdhcpd_rpd_add_default_route: set rt_params with rta_flags 4 cookie 7
Jul 17 01:10:24.597890 [MSTR][DEBUG][default:default][CLN][INET][em0.0][SID=0] jdhcpd_rpd_add_default_route: Add default route successfully. local 4.4.4.1 nh 4.4.4.4 ifl em0.0/7

Note: Normally, the jdhcpd process is terminated as soon as the bootstrap/initial configuration is committed, so a second CLIENT_EVENT_IFA_CHANGE event does not happen and a duplicate access-internal default route is not added.

The CLIENT_EVENT_IFA_CHANGE event to a BOUND client triggers the addition of an access-internal default route by the client.

Multiple such events to the client in BOUND state will result in duplicate access-internal default route additions and will result in stale access-internal routes.

To avoid this problem, users need to restart the RPD.

1. Verify default route:

   root@hostname> show route 0.0.0.0/0 exact

   inet.0: 9 destinations, 10 routes (9 active, 0 holddown, 0 hidden)
   + = Active Route, - = Last Active, * = Both
   
   0.0.0.0/0          *[Access-internal/12] 00:03:03, metric 0
                       > to 4.4.4.4 via em0.0

2. Restart RPD and verify the stale default route is cleared:
   
   root@hostname> restart routing

   root@hostname> show route 0.0.0.0/0 exact

Note: There is a software fix for this issue in the followingJunos versions:

• 15.4X49-D140
• 17.4R2, 17.3R3, 17.2R1-S9
• 18.1R2, 18.2
• 20.1R1, 20.2

2021-06-08: Added example in the solution and fixed Junos releases.