In this article, we discuss a situation where a device is unable to synchronize with an NTP server, resulting in a failure to sync with the Reference ID ".LOCL." We also explain how to resolve this issue.
Symptoms
user@> show ntp associations
remote refid auth st t when poll reach delay offset jitter
====================================================================================
10.10.10.10 .LOCL. - 1 - 823 1024 377 5.373 -994801 38.107
10.10.10.11 .LOCL. - 1 - 631 1024 377 3.775 -994807 21.887
user@> show system uptime
fpc0:
--------------------------------------------------------------------------
Current time: 2025-05-22 12:26:41 EDT
Time Source: LOCAL CLOCK
System booted: 2023-11-16 14:48:49 EST (78w6d 20:37 ago)
Last configured: 2025-05-06 11:53:40 EDT (2w2d 00:33 ago) by aa2260
12:26PM up 552 days, 20:38, 1 users, load averages: 0.46, 0.35, 0.33
fpc1:
Current time: 2025-05-22 12:26:42 EDT
System booted: 2023-11-16 14:48:51 EST (78w6d 20:37 ago)
Protocols started: 2023-11-16 14:53:54 EST (78w6d 20:32 ago)
Last configured: 2025-05-06 11:53:48 EDT (2w2d 00:32 ago) by aa2260
12:26PM up 552 days, 20:38, 2 users, load averages: 0.86, 0.75, 0.65
In this case, you need to ensure that there is bidirectional connectivity and that UDP port "123" is open across the path. If you are stuck in this state without synchronization between the NTP client and NTP server, you can take a PCAP on the egress interface connected to the server to validate how they are sending the "Reference ID," which might show as "uncalibrated local clock" under the NTP header. Additionally, you can monitor the traffic on the interface where you are sourcing this connectivity to the server to validate this.
NOTE: An "uncalibrated local clock" in NTP refers to a situation where a server is using its own internal clock as the time source instead of synchronizing with an external time server. The local clock is considered "uncalibrated" because it hasn't been adjusted or corrected based on a more accurate external time source. This can lead to inaccuracies in timekeeping, as internal clocks are generally less precise than dedicated time servers.
We will monitor the traffic on irb.10, which is the source of the NTP communication on this device:
user@> monitor traffic interface irb.10 size 1500 detail no-resolve matching "host 10.10.10.10"
Address resolution is OFF.
Listening on irb.10, capture size 1500 bytes
12:28:02.152797 Out IP (tos 0x0, ttl 64, id 61725, offset 0, flags [none], proto: UDP (17), length: 76) 10.10.10.1.59902 > 10.10.10.10.123: NTPv4, length 48
Client, Leap indicator: clock unsynchronized (192), Stratum 0, poll 4s, precision -6
Root Delay: 1.000000, Root dispersion: 1.000000, Reference-ID: (unspec)
Reference Timestamp: 0.000000000
Originator Timestamp: 0.000000000
Receive Timestamp: 0.000000000
Transmit Timestamp: 3956920082.152488423
Originator - Receive Timestamp: 0.000000000
Originator - Transmit Timestamp: 3956920082.152488423
12:28:02.157900 In IP (tos 0x0, ttl 127, id 33681, offset 0, flags [none], proto: UDP (17), length: 76) 10.10.10.10.123 > 10.10.10.1.59902: NTPv3, length 48
Server, Leap indicator: (0), Stratum 1, poll 4s, precision -23
Root Delay: 0.000000, Root dispersion: 10.205154, Reference-ID: LOCL
Reference Timestamp: 3956901361.256555299
Originator Timestamp: 3956920082.152488423
Receive Timestamp: 3956919087.334549399
Transmit Timestamp: 3956919087.334556699
Originator - Receive Timestamp: -994.817939024
Originator - Transmit Timestamp: -994.817931724
You can force the NTP to sync using the "set date ntp" command. However, it would synchronize for a short period and then revert to the same state. At this point, we can conclude that the switch is not synchronizing with the servers because they are using their own internal clocks as the time source instead of synchronizing with an external time server. To confirm this, you can change the servers to a trusted external time server, which will synchronize correctly with the device. We will change it to 129.6.15.28, which is one of the NIST Internet Time Servers. We observed the difference in the Reference-ID when we pointed the device to the NIST servers:
user@> monitor traffic interface irb.10 size 1500 detail no-resolve matching "host 129.6.15.28"
12:14:22.191947 Out IP (tos 0x10, ttl 64, id 21853, offset 0, flags [none], proto: UDP (17), length: 76) 10.10.10.1.123 > 129.6.15.28.123: NTPv4, length 48
Client, Leap indicator: (0), Stratum 2, poll 6s, precision -19
Root Delay: 0.044540, Root dispersion: 0.193557, Reference-ID: 129.6.15.28
Reference Timestamp: 3956919260.235012421
Originator Timestamp: 3956919260.207867605
Receive Timestamp: 3956919260.235012421
Transmit Timestamp: 3956919262.191806008
Originator - Receive Timestamp: +0.027144816
Originator - Transmit Timestamp: +1.983938402
12:14:22.237675 In IP (tos 0x0, ttl 49, id 27997, offset 0, flags [none], proto: UDP (17), length: 76) 129.6.15.28.123 > 10.10.10.1.123: NTPv3, length 48
Server, Leap indicator: (0), Stratum 1, poll 13s, precision -29
Root Delay: 0.000244, Root dispersion: 0.000488, Reference-ID: NIST
Reference Timestamp: 3956919168.000000000
Originator Timestamp: 3956919262.191806008
Receive Timestamp: 3956919262.210701421
Transmit Timestamp: 3956919262.210702734
Originator - Receive Timestamp: +0.018895413
Originator - Transmit Timestamp: +0.018896726
The customer needs to ensure that the NTP configuration on the servers points to reliable external NTP servers so the devices can synchronize properly. As we can see from the Reference-ID, we are now receiving it as "NIST" instead of "LOCL" as before.
2025-05-22 : Article Created
2025-05-18 : Changed visibility to Internal, Partner and Customer and categories to EX-Series.