Auto language detection returns English only for mixed English/Spanish audio

Auto language detection returns English only for mixed English/Spanish audio

Batch transcription of a 12-minute English/Spanish video labeled every word as English, even though large stretches were Spanish. A maintainer clarified this is expected behavior: Speechmatics supports a single language per file, and the auto option just samples the audio to pick the predominant language for the whole file. The expected_languages option only helps determine which language model to use, not per-word language labels. A bilingual Spanish/English pack was planned for mixed audio.

How to handle it

The auto language option picks one predominant language for the whole file, so mixed English/Spanish audio will come back labeled as whichever dominated the samples. This is by design since Speechmatics supports a single language per file. If you need better Spanish handling, look at the bilingual Spanish/English pack rather than relying on auto detection. For per-segment language labels, you will need a different approach, since even the bilingual pack does not label results by language. Source: https://github.com/speechmatics/speechmatics-python/issues/80

Question asked in the thread: Batch transcription of a 12-minute English/Spanish video labeled every word as English, even though large stretches were Spanish. A maintainer clarified this is expected behavior: Speechmatics supports a single language per file, and the auto option just samples the audio to pick the predominant language for the whole file. The expected_languages option only helps determine which language model to use, not per-word language labels. A bilingual Spanish/English pack was planned for mixed audio.

Verified answer: The auto language option picks one predominant language for the whole file, so mixed English/Spanish audio will come back labeled as whichever dominated the samples. This is by design since Speechmatics supports a single language per file. If you need better Spanish handling, look at the bilingual Spanish/English pack rather than relying on auto detection. For per-segment language labels, you will need a different approach, since even the bilingual pack does not label results by language. Source: https://github.com/speechmatics/speechmatics-python/issues/80

Thread: https://vectle.com/threads/thr_CsHMpOKKKdZ7s021MtQohQ