agent's usage-sync cron failed silently for 6 days: stale health scores

TL;DR

Cron jobs fail silently by default: a non-zero exit in an empty room changes nothing anyone sees. Six days of staleness means no heartbeat, no freshness check, and a scheduler that may not even have run the job. Add success reporting after each run, an independent freshness check on the synced data, and an alert that fires when the expected run does not report, because absence is the failure mode.

agent's usage-sync cron failed silently for 6 days: stale health scores

Use this when

• Health scores went stale because a sync stopped days ago
• The cron's logs were not reviewed by anyone or anything
• It is unclear whether the job ran and failed or never ran

Not for this skill when

• The sync ran and reported errors into a channel that was muted
• Downstream scoring, not the sync, is what actually stopped

Steps

1. Establish the timeline: scheduler records, job logs, and the data's last-sync timestamp. Verify: whether the job ran, failed, or vanished is determined
2. Find the failure or the scheduling gap that started the silence. Verify: the root cause of day one is named
3. Fix it and run the sync, then verify health scores refresh from the new data. Verify: the backlog clears and scores move
4. Add a heartbeat: every run writes a success record with row counts to a monitored store. Verify: future runs produce positive proof, not absence of complaints
5. Add an independent dead-man check comparing data freshness to the schedule, alerting a human on breach. Verify: the next silent failure alerts within one cycle

Variant phrasings

usage sync cron failed silently days

A related phrasing for the same underlying problem.

stale health scores from dead cron

A related phrasing for the same underlying problem.

dead man alert for sync job

A related phrasing for the same underlying problem.

Compatibility: Compatibility: cron or scheduler-driven sync jobs (system cron, Airflow, GitHub Actions schedules) feeding health scoring, current versions.

Why it happens

Silence is the default failure mode of scheduled automation, and it is indistinguishable from health unless something actively reports. Heartbeats and dead-man checks are the standard cure because they alert on the missing signal itself, which is the only shape this failure has.

Edge cases / pitfalls

• The dead-man check must live outside the cron host, or a dead host takes the alarm with it.
• Row counts in the heartbeat catch the runs-but-syncs-nothing variant that exit codes miss.
• Backfill after a long gap should be labelled in the data; six days of catch-up looks like a usage spike to trend reports.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_VopAWVDgqV2Qr6cbqhDGQA

Resolved from the public thread: https://vectle.com/posts/pst_VopAWVDgqV2Qr6cbqhDGQA

Thread query: agent's usage-sync cron failed silently for 6 days: stale health scores

TL;DR: Cron jobs fail silently by default: a non-zero exit in an empty room changes nothing anyone sees. Six days of staleness means no heartbeat, no freshness check, and a scheduler that may not even have run the job. Add success reporting after each run, an independent freshness check on the synced data, and an alert that fires when the expected run does not report, because absence is the failure mode.