airflow sla miss alerts setup

TL;DR

Set sla=timedelta(...) on tasks (or default_args), configure an sla_miss_callback to route the alert, and make sure email/SMTP or your alerting integration works. SLA misses are detected by the scheduler, not by task failure.

airflow sla miss alerts setup

Use this when

• DAGs finish late with no notification
• You are setting SLA expectations on tasks
• SLA miss alerts never arrive

Not for this skill when

• You are configuring deferrable operators
• You need dynamic task mapping
• Sensors time out

Steps

1. Set the SLA on the tasks that matter:

from datetime import timedelta
default_args = {"sla": timedelta(hours=2)}
@dag(default_args=default_args, schedule="@daily", ...)
def my_dag(): ...

Expected output: any task instance running longer than 2 hours past its scheduled time registers an SLA miss. Set SLA relative to the schedule period: a daily DAG with a 2-hour SLA means "done within 2 hours of the period start."

1. Route the alert with a callback instead of relying on email alone:

def sla_alert(dag, task_list, blocking_task_list, slas, blocking_tis):
    send_to_pagerduty(f"SLA missed: {[t.task_id for t in task_list]}")
default_args = {"sla": timedelta(hours=2), "sla_miss_callback": sla_alert}

Expected output: misses go to your real alerting system. The default email path needs working SMTP config, which is the usual reason "SLA never alerted."

1. Verify email works if you use the default path:

[smtp]
smtp_host = YOUR_HOST
smtp_mail_from = [alert sender address]

Expected output: test with airflow dags test on a task with a tiny SLA. If no email arrives, the SMTP config is wrong long before the SLA logic is.

1. Understand what SLA miss actually means:

SLA miss fires when a task instance has not SUCCEEDED within the
sla period after its logical date. It fires once per miss. It does
NOT fail the task; the task keeps running. It is a timeliness
alert, not a failure alert.

Expected output: correct expectations. Teams that expect the task to fail on SLA miss misconfigure their paging.

1. Keep SLAs honest or they become noise:

Set SLA from measured p95 task durations plus headroom, per task.
A DAG-level SLA on a DAG with one slow task pages for the wrong
reason. Review SLA misses monthly; a miss that nobody acts on
is a threshold that needs raising or deleting.

Expected output: alerts people act on. SLA fatigue looks exactly like having no SLAs.

Variant phrasings

airflow sla not working

Check the callback/SMTP path (steps 2-3) and that the scheduler is running; SLA detection lives in the scheduler loop.

airflow slamisscallback example

Step 2 is the pattern. The callback receives the dag, the missed tasks, and the blocking task instances.

airflow sla vs timeout

SLA is "should have finished by now" (alerts, task continues). Timeouts (execution_timeout) kill the task. Use SLA for timeliness, timeouts for runaway protection.

Why it happens

Airflow tracks each task instance's logical date and compares success time against the SLA. The check runs in the scheduler, independent of task state, which is why a hung-but-running task still generates the miss while never failing.

Edge cases

• SLA only applies to scheduled runs; manually triggered runs skip SLA checks.
• Backfills generate SLA misses in bulk; exclude or pause SLAs during backfill windows.
• The scheduler must be healthy for SLA detection; a down scheduler means no SLA alerts either.
• sla_miss_callback on default_args applies to all tasks; override per task where the SLA differs.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstwAfZNkum1NY1ipWuuKKuQ