Estuary capture from Google Cloud Postgres fails because watermarksTable is not fully qualified

Estuary capture from Google Cloud Postgres fails because watermarksTable is not fully qualified

Whats going on

A Google Cloud Postgres capture in Estuary Flow kept failing even though the replication slot and publications were set up. The task logs said the materialization couldn't find flow_watermarks, and the user suspected they had configured the wrong watermarks table. An Estuary maintainer spotted that the task was connecting to a different database than the capture, so the unqualified flow_watermarks reference didn't resolve. The fix was to fully qualify the table name as public.flow_watermarks in the config. The reporter edited their config, the capture connected, and the user confirmed it worked.

What actually fixes it

The capture was fine. The problem was that watermarksTable was set to the bare flow_watermarks, and the task was connecting to a different database where that unqualified name didn't resolve. Fully qualifying it as public.flow_watermarks fixed it, which matches what the docs say: the watermarks table must be fully qualified. Fix: in your capture config set watermarksTable to public.flow_watermarks (or your own fully qualified name) and re-run the capture. The reporter did exactly this and confirmed the task connected and ran. Source: https://github.com/estuary/flow/issues/398

Original thread

https://vectle.com/threads/thrPCdJV2Px1a7zvMMEz1DxA

In the thread https://vectle.com/threads/thrPCdJV2Px1a7zvMMEz1DxA, the question was: A Google Cloud Postgres capture in Estuary Flow kept failing even though the replication slot and publications were set up. The task logs said the materialization couldn't find flow_watermarks, and the user suspected they had configured the wrong watermarks table. An Estuary maintainer spotted that the task was connecting to a different database than the capture, so the unqualified flow_watermarks reference didn't resolve. The fix was to fully qualify the table name as public.flow_watermarks in the config. The reporter edited their config, the capture connected, and the user confirmed it worked. The verified resolution was: The capture was fine. The problem was that watermarksTable was set to the bare flow_watermarks, and the task was connecting to a different database where that unqualified name didn't resolve. Fully qualifying it as public.flow_watermarks fixed it, which matches what the docs say: the watermarks table must be fully qualified. Fix: in your capture config set watermarksTable to public.flow_watermarks (or your own fully qualified name) and re-run the capture. The reporter did exactly this and confirmed the task connected and ran. Source: https://github.com/estuary/flow/issues/398