usage anomaly detector drifted after the daylight saving change: daily scoring ran an hour late and missed drops

TL;DR

An hour sounds trivial until it moves the window you score. When clocks change, a daily job pinned to a fixed UTC time or a naive local time shifts against the data it examines, comparing partial days, scoring before the warehouse has finished loading, or missing the exact hours where a drop begins. Pin the schedule to an explicit local timezone with a stated daylight saving behaviour, align it with data readiness rather than the clock alone, and backfill the scoring for the drift window so the missed drops are finally seen.

usage anomaly detector drifted after the daylight saving change: daily scoring ran an hour late and missed drops

Use this when

• Daily scoring or anomaly results degraded right after a daylight saving transition and stayed off until noticed
• Run timestamps show the job executing an hour earlier or later relative to local data windows than before the change
• The missed anomalies are concentrated in the hours the shifted window no longer covers

Not for this skill when

• The detector model itself degraded over weeks with no timing change at the transition
• The warehouse load, not the scoring job, shifted at the change and scoring inherited stale inputs

Steps

1. Reconstruct the timing before and after the change. List the jobs scheduled and actual run times in both UTC and the business local time for the weeks around the transition, and the data windows each run scored, because the drift is only provable as a timeline, and several schedules, the job, the warehouse load, and the alert digest, may each have shifted differently. Verify: the before and after timelines are documented for every schedule in the chain
2. Find which clock each component obeys. Check the scheduler configuration for the scoring job, cron expressions are usually UTC or host local depending on the platform, the warehouse load schedule, and any code that computes today or yesterday from a system clock, and identify the component whose assumption about the hour broke at the transition. Verify: the component with the broken time assumption is identified
3. Restate the schedule in explicit terms. Configure the job with a named timezone and a written rule for transition days, run at the same local time, accept a twenty three or twenty five hour day, or hold to data readiness, and trigger scoring from the warehouse loads completion where the tooling allows, because data ready is the event that actually matters and clocks are only its usual proxy. Verify: the schedule is reconfigured with its timezone and transition rule written in the job documentation
4. Handle the partial day problem in the detector. Make the anomaly comparison use complete comparable windows, same hours of day across the days compared, and skip or flag scoring when the input window is incomplete rather than calling a short window a usage drop, since the transition weeks real damage is false and missed anomalies mixed together. Verify: a test scoring run over a simulated transition day produces flagged incomplete windows instead of false anomalies
5. Backfill the drift window. Rescore every day from the transition to the fix with correct windows, deliver any genuine drops the detector missed to account owners as a catch-up list with the affected dates, and mark the original runs in the scoring history as timing compromised so nobody cites them as a quiet period. Verify: the window is rescored and missed drops are delivered with dates
6. Add clock change rehearsals to the calendar. Before each spring and autumn transition, run a checklist over time sensitive jobs, schedules, window computations, and alert digests, and add a monitoring comparison of scheduled versus expected local run times that alerts when any job drifts by more than a few minutes for any reason, clocks being only the most predictable cause. Verify: the rehearsal checklist exists and the drift monitor is active

Why it happens

Time is a configuration surface that changes twice a year. Schedulers, host clocks, and business calendars each hold a different assumption about what nine in the morning means across a transition, and a pipeline assembled from components that each look correct can drift by an hour as a system. Anomaly detection is exquisitely sensitive to that hour because it compares windows, so a small universal shift becomes missed drops at the edges and false alarms in the middle, all wearing the costume of customer behaviour.

Compatibility: Usage anomaly detection, job scheduling with timezones and daylight saving transitions, and warehouse load orchestration, in current scheduling platform versions.

Variants and edge cases

daylight saving scoring ran hour late anomalies missed

['The direct phrasing. Timelines in both UTC and local time are the diagnostic, single timezone views hide the drift they are meant to reveal.']

usage detector drifted DST daily window

['The window variant. Comparable complete windows matter as much as run timing, a correctly timed job scoring a ragged window still manufactures anomalies.']

cron timezone daylight saving churn job

['The scheduling variant. Prefer data readiness triggers over clock triggers wherever the platform supports them, the load finishing is the real start signal.']

Edge cases to work through before you close the incident:

• Not every region observes daylight saving, and customers span regions. Scoring windows should follow the data and the accounts region deliberately, not the server teams location by accident.
• A twenty five hour day in autumn duplicates an hour of data in naive windows. Deduplicate or explicitly include it by rule, do not discover it in an anomaly argument.
• Log both UTC and local timestamps in run records permanently. Future timing incidents are unprovable from single zone logs.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_oxs9DHy8FTv0hkLy-sYUAg