Airbyte MongoDB incremental sync fails repeatedly with "Saved offset is not valid"

TL;DR

The fix shipped in source-mongodb-v2 v2.0.4, announced by an Airbyte contributor on the thread. This was fixed in source-mongodb-v2 v2.0.4.

Steps

1. Incremental syncs from MongoDB to Postgres kept failing with java.lang.IllegalArgumentException: Saved offset is not valid. The sync would then retry from a full refresh, which was expensive on a large collection. A community member traced the bug in the connector code: the saved offset wasn't being updated correctly, so the next sync started from a stale offset. The fix shipped in source-mongodb-v2 v2.0.4, announced by an Airbyte contributor on the thread. A user ran the dev build for several days before the release and confirmed no further failures, then confirmed again on v2.0.4.
2. The connector had a bug where it didn't update the saved offset properly, so every incremental sync after the first choked on a stale offset and fell back to a full refresh. This was fixed in source-mongodb-v2 v2.0.4. Fix: upgrade the MongoDB source connector to v2.0.4 (or later) in your connection settings. Your next incremental syncs should pick up from the saved offset instead of erroring out. One user tested the dev build for days with zero failures before the release, and the Airbyte contributor confirmed the fix is live in v2.0.4.

When to use

You are seeing this: The sync would then retry from a full refresh, which was expensive on a large collection. Use this skill when you run into "Airbyte MongoDB incremental sync fails repeatedly with "Saved offset is not valid"".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

Versions mentioned in the source: source-mongodb-v2 v2.0.4. If you are on something much newer or older, the details may have shifted.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

The thread asked: Incremental syncs from MongoDB to Postgres kept failing with java.lang.IllegalArgumentException: Saved offset is not valid. The sync would then retry from a full refresh, which was expensive on a large collection. A community member traced the bug in the connector code: the saved offset wasn't being updated correctly, so the next sync started from a stale offset. The fix shipped in source-mongodb-v2 v2.0.4, announced by an Airbyte contributor on the thread. A user ran the dev build for several days before the release and confirmed no further failures, then confirmed again on v2.0.4.

The verified answer: The connector had a bug where it didn't update the saved offset properly, so every incremental sync after the first choked on a stale offset and fell back to a full refresh. This was fixed in source-mongodb-v2 v2.0.4. Fix: upgrade the MongoDB source connector to v2.0.4 (or later) in your connection settings. Your next incremental syncs should pick up from the saved offset instead of erroring out. One user tested the dev build for days with zero failures before the release, and the Airbyte contributor confirmed the fix is live in v2.0.4.

Thread: https://vectle.com/threads/thr_3hS2bArMaRF7SiFaW9tT6w