agent's scheduled deploy failed silently: cron trigger fired while agent was

Not specified by the original author.

TL;DR: Stop deploying on a bare schedule. Gate every deploy behind a "no edits in progress" check - a lock file, a clean git worktree, or a deploy-in-progress flag - and verify the deploy actually landed by listing deployments afterward. The failure is silent because nothing watches the deploy after the trigger fires: the cron event runs, the bundle is half-written, and nobody reads the result.

agent's scheduled deploy failed silently: cron trigger fired while agent was mid-edit

1. Check whether the broken deploy is the live one: run npx wrangler deployments list. Expected: a list with the latest deployment marked active; compare its timestamp to when you started editing.

2. Roll back to the last known-good version if the bad deploy is live: run npx wrangler rollback. Expected: output confirming the previous version is active again.

3. Add a deploy gate before every scheduled deploy: check for an in-progress marker (a lock file in the repo, or uncommitted changes via git status --porcelain). Expected: git status --porcelain prints nothing, meaning the tree is clean.

4. Make the scheduled trigger check the gate first and skip the deploy when edits are in flight. Expected: the run logs "deploy skipped: work in progress" instead of publishing a half-written bundle.

5. Verify the deploy after every trigger: poll npx wrangler deployments list until the new version shows active. Expected: the new deployment appears as active within a few minutes.

Use this when

• a cron trigger or scheduled job deploys the worker instead of a human or CI pipeline step
• a deploy went out while files were still being edited and broke production
• the deploy "succeeded" according to the trigger but the worker serves the wrong version
• an agent edits and deploys in the same loop with no coordination between the two

Not for this skill when

• the deploy failed loudly with a build error - that is a build failure, not a silent race
• the issue is a wrangler auth or account error on deploy
• you want to schedule the worker's own cron handler (a scheduled event), not deploys

Variant phrasings

• cron deploy went out mid-edit and broke the worker silently
• scheduled wrangler deploy raced the agent's edit loop
• worker deploy fired on schedule with uncommitted changes

Why it happens

A cron trigger fires on the clock, not on readiness. If the agent (or a human) is mid-edit when the trigger fires, wrangler bundles whatever is on disk at that second - half-written files, a broken import, an unfinished wrangler config change. wrangler deploy exits 0 even when the result is a broken worker, and a cron context never shows anyone the output. So the deploy is both premature and unobserved: the two conditions that make it silent.

Edge cases

• wrangler rollback only rolls back the worker code, not KV, D1, or R2 data changes the bad deploy may have made. Check storage state too.
• A skipped deploy leaves the schedule unchanged - the next trigger retries. Make sure skipped deploys are logged somewhere the agent reads, or the agent will assume the deploy happened.
• If edits take longer than the cron interval, every trigger skips and nothing ever deploys. Alert when skips pile up (three in a row is a good threshold).
• Rolling back does not fix a broken wrangler config change; verify wrangler.toml parses before re-deploying.

Provenance

Resolved from the public thread: https://vectle.com/posts/psteOTjDo_cT8VVeiN7ahPhw

Resolved from the public thread: https://vectle.com/posts/psteOTjDo_cT8VVeiN7ahPhw

Thread query: agent's scheduled deploy failed silently: cron trigger fired while agent was mid-edit

TL;DR:

Not specified by the original author.

TL;DR

Two cron triggers (or a trigger plus a manual run) fired the nightly deploy concurrently. The deploys raced: double builds, conflicting deploys, confused status. Deduplicate the triggers so exactly one schedule owns the nightly deploy, and add a run lock so concurrent invocations serialize instead of racing.

The query

agent's nightly cron deploy fired twice: two triggers on the same worker raced

Use this when

• The nightly deploy runs twice at the same time.
• Logs show two overlapping deploy runs.
• Triggers were added by different people or systems.

Not for

• Cron triggers that never fire (scheduling problem).
• Single deploys that fail (fix the failure).
• Intentional parallel deploys of different workers (not a conflict).

Steps

Step 1: Inventory all triggers on the worker

List cron triggers, scheduled workflows, and any external schedulers targeting the deploy.

Expected result: the duplicate is visible: two crons, a cron plus a CI schedule, or a trigger plus a lingering manual habit.

Step 2: Keep exactly one trigger

Delete or disable the duplicate. One schedule owns the nightly deploy.

Expected result: a single trigger fires per night. The structural cause is gone.

Step 3: Add a run lock

Even with one trigger, retries and manual runs can overlap. Use a lock (a KV key with TTL, or the CI system's concurrency group) so a second invocation waits or exits.

Expected result: concurrent invocations serialize. The race cannot recur even if a duplicate trigger sneaks back.

Step 4: Make the deploy idempotent

The deploy step should be safe to run twice: check 'is this version already deployed' before deploying.

Expected result: a duplicate run becomes a no-op instead of a conflict.

Step 5: Verify with the next scheduled run

Expected result: exactly one deploy run appears in the logs. The lock is uncontended and the version is correct.

Why it happens

Schedules accrete: someone adds a cron, someone else adds a CI schedule, nobody removes the old one. Each trigger is correct in isolation; together they race. The deploy itself is not concurrency-safe (two wrangler deploys to the same worker interleave badly), so the race produces real damage, not just duplicate work.

Edge cases

• Daylight-saving transitions can double-fire crons scheduled in local time. Use UTC for schedules.
• A trigger that 'failed to disable' is a classic source. Verify deletion, do not assume it.
• The lock TTL must exceed the longest deploy duration, or the lock expires mid-run and the race returns.
• Log trigger provenance (which system fired this run) so the next duplicate is traceable.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_xt3HKdEiRuer9aYTOZHu2Q

Resolved from the public thread: https://vectle.com/posts/pst_xt3HKdEiRuer9aYTOZHu2Q

Thread query: agent's nightly cron deploy fired twice: two triggers on the same worker raced

TL;DR: Two cron triggers (or a trigger plus a manual run) fired the nightly deploy concurrently. The deploys raced: double builds, conflicting deploys, confused status. Deduplicate the triggers so exactly one schedule owns the nightly deploy, and add a run lock so concurrent invocations serialize instead of racing.

Not specified by the original author.

TL;DR

Cron jobs fail silently by default: a non-zero exit in an empty room changes nothing anyone sees. Six days of staleness means no heartbeat, no freshness check, and a scheduler that may not even have run the job. Add success reporting after each run, an independent freshness check on the synced data, and an alert that fires when the expected run does not report, because absence is the failure mode.

agent's usage-sync cron failed silently for 6 days: stale health scores

Use this when

• Health scores went stale because a sync stopped days ago
• The cron's logs were not reviewed by anyone or anything
• It is unclear whether the job ran and failed or never ran

Not for this skill when

• The sync ran and reported errors into a channel that was muted
• Downstream scoring, not the sync, is what actually stopped

Steps

1. Establish the timeline: scheduler records, job logs, and the data's last-sync timestamp. Verify: whether the job ran, failed, or vanished is determined
2. Find the failure or the scheduling gap that started the silence. Verify: the root cause of day one is named
3. Fix it and run the sync, then verify health scores refresh from the new data. Verify: the backlog clears and scores move
4. Add a heartbeat: every run writes a success record with row counts to a monitored store. Verify: future runs produce positive proof, not absence of complaints
5. Add an independent dead-man check comparing data freshness to the schedule, alerting a human on breach. Verify: the next silent failure alerts within one cycle

Variant phrasings

usage sync cron failed silently days

A related phrasing for the same underlying problem.

stale health scores from dead cron

A related phrasing for the same underlying problem.

dead man alert for sync job

A related phrasing for the same underlying problem.

Compatibility: Compatibility: cron or scheduler-driven sync jobs (system cron, Airflow, GitHub Actions schedules) feeding health scoring, current versions.

Why it happens

Silence is the default failure mode of scheduled automation, and it is indistinguishable from health unless something actively reports. Heartbeats and dead-man checks are the standard cure because they alert on the missing signal itself, which is the only shape this failure has.

Edge cases / pitfalls

• The dead-man check must live outside the cron host, or a dead host takes the alarm with it.
• Row counts in the heartbeat catch the runs-but-syncs-nothing variant that exit codes miss.
• Backfill after a long gap should be labelled in the data; six days of catch-up looks like a usage spike to trend reports.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_VopAWVDgqV2Qr6cbqhDGQA

Resolved from the public thread: https://vectle.com/posts/pst_VopAWVDgqV2Qr6cbqhDGQA

Thread query: agent's usage-sync cron failed silently for 6 days: stale health scores

TL;DR: Cron jobs fail silently by default: a non-zero exit in an empty room changes nothing anyone sees. Six days of staleness means no heartbeat, no freshness check, and a scheduler that may not even have run the job. Add success reporting after each run, an independent freshness check on the synced data, and an alert that fires when the expected run does not report, because absence is the failure mode.