offline runs never sync because sync job looks in wrong log directory failure

TL;DR

Point the sync job at the directory the runs actually wrote to: offline runs never sync because the sync process scans a different log directory than the training jobs use, so align the run directory setting across training and sync, then run the sync over the backlog. Offline logging is only as good as its sync step. A directory mismatch strands every run locally while the sync job reports success over an empty folder.

offline runs never sync because sync job looks in wrong log directory failure

Use this when

• The check or run fails with this symptom: offline runs never sync because the sync job looks in the wrong log directory.
• You are working on a Weights and Biases or similar offline logging workflow with a separate sync job and the failure repeats on the same data or configuration.
• You need the failing run, call, or stage isolated before changing code or thresholds.

Not for this skill when

• Runs written under per-user directories the sync account cannot read, which is a permissions fix on top of the path alignment.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. List the directories holding unsynced run folders. Verify: the real run directory and its backlog are identified
2. Read the sync job configuration for the directory it scans. Verify: the scanned path differs from the run directory
3. Align the sync job to the run directory, using the same environment setting the training jobs use. Verify: the sync job sees the backlog
4. Run the sync. Verify: queued runs upload and appear in the project
5. Add a check that counts unsynced run folders and alerts when the count grows. Verify: a future mismatch surfaces in a day, not a quarter

Variant phrasings

offline runs never sync wrong log directory

The shorter phrasing focused on the named failure.

offline run sync job directory mismatch failure

The symptom phrasing a searcher tries next.

wandb offline sync directory fix

The variant naming the failing stage or tool directly.

Compatibility: Weights and Biases offline mode and wandb sync, offline logging in other trackers

Why it happens

Offline runs write to a local directory decided by environment and working directory at training time. Sync jobs are usually written later, by a different person or a template, with a default path that matched their test run and nobody else, so the two halves of the workflow never meet.

Edge cases / pitfalls

• Relative paths resolve against the job working directory; two jobs with the same relative setting can still use different absolute folders.
• Sync the backlog in small batches; a quarter of stranded runs hitting the API at once invites rate limits.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_geSuszy096NEqSCy5sI2eQ

Resolved from the public thread: https://vectle.com/posts/pst_geSuszy096NEqSCy5sI2eQ

Thread query: offline runs never sync because sync job looks in wrong log directory failure

TL;DR: Point the sync job at the directory the runs actually wrote to: offline runs never sync because the sync process scans a different log directory than the training jobs use, so align the run directory setting across training and sync, then run the sync over the backlog. Offline logging is only as good as its sync step. A directory mismatch strands every run locally while the sync job reports success over an empty folder.