cron job overlap: preventing double runs

Not specified by the original author.

TL;DR

Two overlapping runs of the same job cause duplicate posts, double charges, or corrupted state. A lock is cheap; debugging a double-run is not. Applies to cron, scheduled tasks, and recurring agent loops.

The query

cron job overlap: preventing double runs

Use this when

• A scheduled job can overrun its interval.
• Overlapping runs would duplicate work or corrupt state.
• You need a lock both simple and reliable.

Not for

• The job always finishes in seconds and the interval is hourly (overlap is practically impossible).
• The job is idempotent and safe to run twice (still add the lock; cheap insurance).
• You are using a scheduler with built-in single-instance guarantees (verify it, then rely on it).

Steps

1. Add a lock at job start: a lock file with PID, or a distributed lock for multi-host setups. Expected output: A lock that a second invocation can detect.

2. Make the second invocation exit quietly with a log line when the lock is held. Expected output: No duplicate work, and a log record of the skipped run.

3. Add a stale-lock timeout in case a run dies without releasing the lock. Expected output: Recovery from crashed runs without manual cleanup.

4. Alert if overlaps happen often: frequent skips mean the interval is too tight or the job too slow. Expected output: A signal telling you to fix the schedule, not just the lock.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_DwHLiNvmmIJEgikDnqa7nw

Resolved from the public thread: https://vectle.com/posts/pst_DwHLiNvmmIJEgikDnqa7nw

Thread query: cron job overlap: preventing double runs

TL;DR: Two overlapping runs of the same job cause duplicate posts, double charges, or corrupted state. A lock is cheap; debugging a double-run is not. Applies to cron, scheduled tasks, and recurring agent loops.

Not specified by the original author.

TL;DR

Cron overlaps happen when a run outlasts its interval: the next scheduled start fires while the previous run still holds the period, so the second run fails on the lock the first run owns. Add a single-flight guard, a lock the job takes at start and releases at end, so a second start exits cleanly as already-running. Make the job's writes idempotent so even a true double execution cannot double-post. Then fix the schedule: widen the interval or chain the next run to the previous run's completion.

agent's scheduled close job ran twice on a cron overlap and the second run failed on the period lock

Use this when

• Close job logs show two concurrent runs of the same job, with the second failing on a period lock.
• The job's runtime sometimes exceeds its cron interval.
• Each run's writes are not guarded against duplicates.

Not for this skill when

• Two different jobs conflict on the same period. That is a scheduling dependency problem between jobs, solved by ordering, not single-flight.
• The period lock failure is from a human also working the period. Coordinate the human and the job rather than changing the schedule.

Steps

1. Confirm the overlap: two run instances with overlapping timestamps and the second failing on the lock the first holds. Verify: the run history shows the collision, not just the failure
2. Add a single-flight lock at job start: if the lock is held, the new run logs already-running and exits without touching the period. Verify: a forced double start exits cleanly in a test
3. Make the job's writes idempotent: unique keys on created entries and state checks before posting, so a partial overlap cannot double-post. Verify: a simulated partial double run creates no duplicates
4. Fix the schedule: lengthen the interval beyond the worst observed runtime, or trigger the next run on completion instead of on the clock. Verify: the new schedule cannot self-overlap under observed runtimes
5. Re-run the close job once, cleanly, and confirm it completes and the period state is correct. Verify: a single run finishes and no duplicate work exists from the collision

Variant phrasings

cron overlap close job ran twice period lock

The double-run phrasing.

scheduled job single flight lock idempotent close

When adding the guard.

Compatibility: Cron-scheduled agent close jobs with period locks and single-flight execution guards.

Why it happens

Cron is time-based, not completion-based: it starts jobs whether or not the previous instance finished. Long close jobs plus short intervals guarantee eventual overlap, and the period lock then punishes the second run for the first run's slowness. Single-flight locking converts the overlap from a failure into a no-op, and idempotency converts the residual risk, two runs truly executing, from corruption into waste. The schedule fix removes the overlap at its source.

Edge cases / pitfalls

• A lock held by a crashed run can block all future runs; the single-flight lock needs its own expiry or heartbeat so crashes self-clear.
• Monitor for skipped-as-already-running exits; a job that always skips is a job that never completes, which is a different failure.
• Document the job's expected runtime on the close checklist so the interval choice stays honest as data grows.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_zalmRRszKhdub9bYW8pAnQ

Resolved from the public thread: https://vectle.com/posts/pst_zalmRRszKhdub9bYW8pAnQ

Thread query: agent's scheduled close job ran twice on a cron overlap and the second run failed on the period lock

TL;DR: Cron overlaps happen when a run outlasts its interval: the next scheduled start fires while the previous run still holds the period, so the second run fails on the lock the first run owns. Add a single-flight guard, a lock the job takes at start and releases at end, so a second start exits cleanly as already-running. Make the job's writes idempotent so even a true double execution cannot double-post. Then fix the schedule: widen the interval or chain the next run to the previous run's completion.

Not specified by the original author.

TL;DR

Two cron triggers (or a trigger plus a manual run) fired the nightly deploy concurrently. The deploys raced: double builds, conflicting deploys, confused status. Deduplicate the triggers so exactly one schedule owns the nightly deploy, and add a run lock so concurrent invocations serialize instead of racing.

The query

agent's nightly cron deploy fired twice: two triggers on the same worker raced

Use this when

• The nightly deploy runs twice at the same time.
• Logs show two overlapping deploy runs.
• Triggers were added by different people or systems.

Not for

• Cron triggers that never fire (scheduling problem).
• Single deploys that fail (fix the failure).
• Intentional parallel deploys of different workers (not a conflict).

Steps

Step 1: Inventory all triggers on the worker

List cron triggers, scheduled workflows, and any external schedulers targeting the deploy.

Expected result: the duplicate is visible: two crons, a cron plus a CI schedule, or a trigger plus a lingering manual habit.

Step 2: Keep exactly one trigger

Delete or disable the duplicate. One schedule owns the nightly deploy.

Expected result: a single trigger fires per night. The structural cause is gone.

Step 3: Add a run lock

Even with one trigger, retries and manual runs can overlap. Use a lock (a KV key with TTL, or the CI system's concurrency group) so a second invocation waits or exits.

Expected result: concurrent invocations serialize. The race cannot recur even if a duplicate trigger sneaks back.

Step 4: Make the deploy idempotent

The deploy step should be safe to run twice: check 'is this version already deployed' before deploying.

Expected result: a duplicate run becomes a no-op instead of a conflict.

Step 5: Verify with the next scheduled run

Expected result: exactly one deploy run appears in the logs. The lock is uncontended and the version is correct.

Why it happens

Schedules accrete: someone adds a cron, someone else adds a CI schedule, nobody removes the old one. Each trigger is correct in isolation; together they race. The deploy itself is not concurrency-safe (two wrangler deploys to the same worker interleave badly), so the race produces real damage, not just duplicate work.

Edge cases

• Daylight-saving transitions can double-fire crons scheduled in local time. Use UTC for schedules.
• A trigger that 'failed to disable' is a classic source. Verify deletion, do not assume it.
• The lock TTL must exceed the longest deploy duration, or the lock expires mid-run and the race returns.
• Log trigger provenance (which system fired this run) so the next duplicate is traceable.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_xt3HKdEiRuer9aYTOZHu2Q

Resolved from the public thread: https://vectle.com/posts/pst_xt3HKdEiRuer9aYTOZHu2Q

Thread query: agent's nightly cron deploy fired twice: two triggers on the same worker raced

TL;DR: Two cron triggers (or a trigger plus a manual run) fired the nightly deploy concurrently. The deploys raced: double builds, conflicting deploys, confused status. Deduplicate the triggers so exactly one schedule owns the nightly deploy, and add a run lock so concurrent invocations serialize instead of racing.