agent's scan scheduler broke: overlapping runs corrupted the baseline
Fixes an agent's scan scheduler corrupting the baseline with overlapping runs: serialize with locking. Use when concurrent scans corrupt shared state. Not for single-run failures.
TL;DR
Overlapping scans write the baseline concurrently and corrupt it. Serialize runs with a lock, skip a run if the previous is still going, and write state atomically.
Error
agent's scan scheduler broke: overlapping runs corrupted the baselineSteps
- Confirm overlap: check scheduler logs for runs starting before the previous finished. Expected: the overlap proven.
- Add a lock around the scan so only one runs at a time. Expected: serialization.
- Configure the scheduler to skip rather than queue when a run is in progress. Expected: no pileup.
- Write the baseline atomically (temp file plus rename). Expected: readers never see a half-written file.
- Validate the baseline after each run. Expected: corruption detected immediately if it recurs.
When to use
- Corrupted baselines with overlapping schedules.
- Designing scan schedulers.
When not to use
- Single runs corrupting the baseline (different bug).
- The scheduler never overlaps (look elsewhere).
Tool compatibility
- Cron and job schedulers; file locks; atomic writes.
Variant phrasings
baseline corrupted by concurrent scans
Lock them.
overlapping scan runs
Skip-or-lock.
Why it happens
Schedulers fire on time, not on completion; slow scans overlap the next trigger and both write the same file.
Edge cases
- Locks must have timeouts; a dead holder must not block forever.
- Distributed schedulers need distributed locks, not file locks.
- Skipped runs should alert if skips become chronic.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_JnBQlvX7oc-N5o1n1XqPcQ