airflow sensor timeout: poke interval exceeded

TL;DR

The poke interval controls how often the sensor checks, and the timeout controls how long it waits total. An interval too short hammers the external system; too long and the sensor misses the arrival window before the timeout. Set the interval from the expected arrival granularity and the timeout from the worst realistic delay. For waits over an hour, use deferrable mode.

airflow sensor timeout: poke interval exceeded

Use this when

• A sensor times out or polls aggressively.
• The sensed system complains about poll load.
• Sensor tasks occupy worker slots for a long time.

Not for this skill when

• The sensor errors on the first poke. That is a poke-function bug.
• A non-sensor task times out. Check its execution_timeout.

Steps

1. Note the current poke_interval, timeout, and mode from the sensor definition. Note the current values. Verify: the config is on the table
2. Determine the realistic arrival window for the sensed thing: when it usually lands and the worst delay seen. Note the window. Verify: reality is measured
3. Set poke_interval to a fraction of the arrival granularity (minutes for hourly arrivals) and timeout beyond the worst delay. Note the new values. Verify: the config fits the arrival pattern
4. For long waits, switch to a deferrable sensor or reschedule mode to free the worker slot. Note slot usage drops. Verify: waiting stops costing slots
5. Watch one cycle: the sensor detects the arrival without timeout and without spamming the external system. Note the clean cycle. Verify: the tuning is verified

Variant phrasings

airflow sensor poke_interval best practice

The tuning-shaped search.

airflow sensor timeout configuration

The config phrasing.

Compatibility: Airflow 2.x sensors. Deferrable sensors need the triggerer; reschedule mode is available on most sensors.

Why it happens

A sensor is a polling loop, and polling has two knobs that people set by gut feel: how often to check and how long to wait. Gut feel produces 60-second pokes against systems that update hourly, which is 59 wasted checks per hour, or 24-hour timeouts on data that lands in ten minutes, which burns a worker slot all day. The right values come from the arrival pattern of the thing being sensed, which is measurable from history. Sensors tuned from data are quiet and reliable; sensors tuned from defaults are noisy or fragile.

Edge cases / pitfalls

• mode='poke' holds the worker slot the whole time. For waits longer than the slot is worth, reschedule or defer.
• Some sensors ignore poke_interval in reschedule mode nuances; verify the effective interval from the logs, not the code.
• External systems with aggressive rate limits need intervals in minutes or the sensor gets blocked and never sees the arrival.
• Timeout=None waits forever and blocks downstream silently. Always set an explicit timeout.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_dBWOWHpiWZ983FfLkNyU5A