## TL;DR

Two cron triggers (or a trigger plus a manual run) fired the nightly deploy concurrently. The deploys raced: double builds, conflicting deploys, confused status. Deduplicate the triggers so exactly one schedule owns the nightly deploy, and add a run lock so concurrent invocations serialize instead of racing.

## The query

```text
agent's nightly cron deploy fired twice: two triggers on the same worker raced
```

## Use this when

- The nightly deploy runs twice at the same time.
- Logs show two overlapping deploy runs.
- Triggers were added by different people or systems.

## Not for

- Cron triggers that never fire (scheduling problem).
- Single deploys that fail (fix the failure).
- Intentional parallel deploys of different workers (not a conflict).

## Steps

### Step 1: Inventory all triggers on the worker

List cron triggers, scheduled workflows, and any external schedulers targeting the deploy.

Expected result: the duplicate is visible: two crons, a cron plus a CI schedule, or a trigger plus a lingering manual habit.

### Step 2: Keep exactly one trigger

Delete or disable the duplicate. One schedule owns the nightly deploy.

Expected result: a single trigger fires per night. The structural cause is gone.

### Step 3: Add a run lock

Even with one trigger, retries and manual runs can overlap. Use a lock (a KV key with TTL, or the CI system's concurrency group) so a second invocation waits or exits.

Expected result: concurrent invocations serialize. The race cannot recur even if a duplicate trigger sneaks back.

### Step 4: Make the deploy idempotent

The deploy step should be safe to run twice: check 'is this version already deployed' before deploying.

Expected result: a duplicate run becomes a no-op instead of a conflict.

### Step 5: Verify with the next scheduled run

Expected result: exactly one deploy run appears in the logs. The lock is uncontended and the version is correct.

## Why it happens

Schedules accrete: someone adds a cron, someone else adds a CI schedule, nobody removes the old one. Each trigger is correct in isolation; together they race. The deploy itself is not concurrency-safe (two wrangler deploys to the same worker interleave badly), so the race produces real damage, not just duplicate work.

## Edge cases

- Daylight-saving transitions can double-fire crons scheduled in local time. Use UTC for schedules.
- A trigger that 'failed to disable' is a classic source. Verify deletion, do not assume it.
- The lock TTL must exceed the longest deploy duration, or the lock expires mid-run and the race returns.
- Log trigger provenance (which system fired this run) so the next duplicate is traceable.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_xt3HKdEiRuer9aYTOZHu2Q