Time synchronization

A Synced Clock Is a Dependency, Not a Cosmetic Setting

TLS validation, log correlation, and scheduled jobs quietly assume correct time. Verify synchronization state itself, not just a running NTP daemon.

What skew actually breaks

A wrong clock fails quietly. Certificate validation compares notBefore and notAfter against local time, so a host that is hours off can reject every valid certificate or accept an expired one. Log lines from different hosts stop correlating, which turns a two-minute incident walk-through into guesswork. Scheduled jobs fire late or twice, tokens and one-time windows expire immediately, and database timestamps disagree about event order.

None of these present as a clock problem. They present as TLS errors, missing logs, or "random" application failures on exactly one host.

Check sync state, not service state

An NTP daemon can be active and completely unsynchronized — after losing its upstream, it keeps running on its own drift. The service state answers "is the daemon alive"; only the sync state answers "is the clock correct".

timedatectl show -p NTPSynchronized -p NTP
chronyc tracking
chronyc sources -v

NTPSynchronized=yes is the claim that matters. In chronyc tracking, check Leap status (normal, not not synchronised), the current and RMS offsets, and the stratum. In sources, a source marked ^? is unreachable and does not contribute; a server list full of question marks with an active daemon is exactly the silent failure described above.

Virtual machines jump; make the correction explicit

chrony prefers slewing — small continuous adjustments — over stepping the clock, which is right for drift and wrong after a snapshot restore or host migration throws the clock minutes or hours off. Make the step policy explicit instead of waiting out a long slew.

# /etc/chrony.conf
makestep 1.0 3
logchange 1.0

This allows a step only in the first few updates after startup, keeping normal operation slew-only, and logs any change larger than a second. After restoring a machine from a snapshot, re-check chronyc tracking before trusting certificates or timestamps on that host.

Compare against an independent source

For a host that must be right, verify against a source that does not share the NTP daemon's opinion. An HTTPS response's Date header is a coarse independent reference, accurate to seconds — enough to catch minute-scale errors that break certificates.

date -u
curl -sI https://www.example.com | grep -i '^date:'

Alert on the offset itself, with a threshold of a few seconds, rather than on the daemon's unit state. An offset alert catches a clock that is drifting toward breakage while the service still looks healthy.

Clock checklist

  • Treat certificate and log-correlation failures as possible clock faults.
  • Check NTPSynchronized, not just the service state.
  • Review chronyc tracking leap status, offsets, and stratum.
  • Confirm reachable sources; question marks mean no correction.
  • Allow a bounded startup step for restored virtual machines.
  • Cross-check local time against an independent source.
  • Alert on offset, not on daemon liveness.