postgres listen notify vs polling pipelines
Compares Postgres LISTEN/NOTIFY with polling for pipeline triggers. Use it when deciding how downstream jobs learn about new data, when poll intervals cause lag, or when notifications go missing. Not for cross-database messaging.
TL;DR
LISTEN/NOTIFY pushes events to connected clients with almost no latency, which is great for cache invalidation and waking up jobs. But it is not durable: notifications sent while nobody is listening are lost, and payloads are capped around 8KB. Use NOTIFY as a signal and keep the actual data in a table or queue; poll or read the table when you need reliability. Signals via NOTIFY, data via polling, is the standard split.
The query
postgres listen notify vs polling pipelinesUse this when
- poll intervals are adding latency you cannot afford
- you are waking up workers or invalidating caches on database changes
- notifications seem to go missing under load
Not for
- guaranteed message delivery, which needs a real queue
- sending large payloads through the database
Steps
- Keep NOTIFY payloads tiny: an id or a table name, never the row data. Payloads over the size cap get dropped.
Expected output: Notifications carry only references, and nothing is silently dropped.
- On receiving a notification, read the actual state from the table. Never trust the signal alone, since signals can be lost.
Expected output: The worker always re-reads from the source of truth after a wakeup.
- Handle reconnects: on reconnect, poll for anything missed while disconnected before resuming the listen loop.
Expected output: A reconnect path exists and catches up missed events.
- Use one channel per event type so listeners only wake for what they care about.
Expected output: Channels are named by event, not one giant catch-all.
- If you need durability or multiple consumers, switch to a queue table with SKIP LOCKED instead of NOTIFY.
Expected output: You have a documented reason for choosing signals over a queue.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_s8oTi4ayD-rmfXN6ORkPaw
Maintainer review
No maintainer verification is recorded for this version.
This records the version a maintainer checked. It does not assert that the version is the latest upstream release.