whisper timestamp drift on 90-minute earnings call audio

TL;DR

Timestamp drift on long earnings calls happens because Whisper processes audio in 30-second windows and small alignment errors accumulate over 90 minutes. Split the call into prepared-remarks and Q&A chunks with a few seconds of overlap, transcribe each separately, and rebase the timestamps per chunk.

The error

segment 842: start=5023.4 end=5028.1 (expected ~4980)
Cumulative drift: +43s at 90min mark.
Q&A question boundaries misaligned with audio.

Fix it

1. Confirm the drift is cumulative by comparing a known landmark (the Q&A start announcement) in the audio versus the transcript timestamp. Expected: Expected: the offset grows with audio position, proving accumulation rather than a constant shift.

2. Split the audio at natural boundaries (end of prepared remarks, Q&A start) into chunks under 30 minutes with 10 seconds of overlap. Expected: Expected: each chunk transcribes with its own local timeline and near-zero internal drift.

3. Transcribe each chunk separately and offset each chunk's timestamps by its start time in the original audio, trimming the overlap. Expected: Expected: the stitched transcript aligns with the audio across the full 90 minutes.

4. In the pipeline, always chunk calls over 30 minutes and store per-chunk offsets so downstream QA alignment stays correct. Expected: Expected: long calls transcribe with the same timestamp accuracy as short ones.

When to use

• Your earnings call transcript timestamps drift progressively on long audio.
• You need Q&A boundaries aligned to the audio for clipping or review.
• You run Whisper over 60-plus-minute earnings calls.

When not to use

• The drift is constant across the whole file - then it is a sample-rate mismatch, not accumulation.
• The transcript text itself is wrong - that is a transcription quality issue.
• You use word-level timestamps from a model that supports long-form natively - different architecture.

Why it happens

Whisper's architecture processes 30-second windows and stitches them. Each stitch can introduce a small alignment error, and over 180 windows those errors compound. The model was also trained mostly on short clips, so its timing degrades on very long inputs.

Variant phrasings

whisper timestamps drift long audio

Same failure and fix; some searchers phrase the query this way.

whisper cumulative timestamp error fix

Same failure and fix; some searchers phrase the query this way.

Edge cases

• Variable-bitrate MP3s can report wrong durations, which looks like drift but is a container issue.
• Music or long silences (hold music between segments) confuse the aligner more than speech does.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_McsBCOLYHy1AbY81gf6bbQ

Resolved from the public thread: https://vectle.com/posts/pst_McsBCOLYHy1AbY81gf6bbQ

Thread query: whisper timestamp drift on 90-minute earnings call audio

TL;DR: Timestamp drift on long earnings calls happens because Whisper processes audio in 30-second windows and small alignment errors accumulate over 90 minutes. Split the call into prepared-remarks and Q&A chunks with a few seconds of overlap, transcribe each separately, and rebase the timestamps per chunk.