Airflow dag processor exits with too many open files after sometime

TL;DR

Upgrade to an Airflow release that includes PR #50558. Issue apache/airflow#49887 (closed, 55 comments): ### Apache Airflow version 3.0.0 ### If "Other Airflow 2 version" selected, which one? No response ### What happened?

Steps

1. Issue apache/airflow#49887 (closed, 55 comments): ### Apache Airflow version 3.0.0 ### If "Other Airflow 2 version" selected, which one? No response ### What happened? Upgraded Airflow from 2.10 to 3.0.0 and DagProcessorJob getting failed after few hours. with below exception OSError: [Errno 24] Too many open files ``` [2025-04-25T21:30:41.765+0100] {dagprocessorjobrunner.py:63} ERROR - Exception when executing DagProcessorJob Traceback (most recent call last): File "/var/opt/rdos-airflow/venv/lib/python3.10/site-packages/airflow/jobs/dagprocessorjobrunner.py", line 61, in _execute self.processor.run() File...
2. Upgrade to an Airflow release that includes PR #50558. That PR fixed the file-descriptor leak behind airflow dags test (and other CLI paths) leaking descriptors, and the reporter confirmed the FD count stopped growing after the fix. If you are pinned to an older Airflow, there is no config workaround in the thread; upgrading to the fixed release is the resolution.

When to use

Use this skill when you run into "Airflow dag processor exits with too many open files after sometime".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

Versions mentioned in the source: 3.0.0. If you are on something much newer or older, the details may have shifted.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

In gh:apache/airflow#49887 (Apache Airflow), the issue reported was: Issue apache/airflow#49887 (closed, 55 comments): ### Apache Airflow version 3.0.0 ### If "Other Airflow 2 version" selected, which one? No response ### What happened? Upgraded Airflow from 2.10 to 3.0.0 and DagProcessorJob getting failed after few hours. with below exception OSError: [Errno 24] Too many open files `` [2025-04-25T21:30:41.765+0100] {dag_processor_job_runner.py:63} ERROR - Exception when executing DagProcessorJob Traceback (most recent call last): File "/var/opt/rdos-airflow/venv/lib/python3.10/site-packages/airflow/jobs/dag_processor_job_runner.py", line 61, in _execute self.processor.run() File... The verified resolution from the thread: Upgrade to an Airflow release that includes PR #50558. That PR fixed the file-descriptor leak behind airflow dags test` (and other CLI paths) leaking descriptors, and the reporter confirmed the FD count stopped growing after the fix. If you are pinned to an older Airflow, there is no config workaround in the thread; upgrading to the fixed release is the resolution. Source: mount the host path https at container path //github.com/apache/airflow/issues/49887