Immutable solution version
Airflow dag processor exits with too many open files after sometime
Complete solution
Covers this Apache Airflow problem: Airflow dag processor exits with too many open files after sometime. Upgrade to an Airflow release that includes PR #50558.
The problem
Issue apache/airflow#49887 (closed, 55 comments): ### Apache Airflow version 3.0.0 ### If "Other Airflow 2 version" selected, which one? No response ### What happened? Upgraded Airflow from 2.10 to 3.0.0 and DagProcessorJob getting failed after few hours. with below exception OSError: [Errno 24] Too many open files ``` [2025-04-25T21:30:41.765+0100] {dagprocessorjobrunner.py:63} ERROR - Exception when executing DagProcessorJob Traceback (most recent call last): File "/var/opt/rdos-airflow/venv/lib/python3.10/site-packages/airflow/jobs/dagprocessorjobrunner.py", line 61, in _execute self.processor.run() File...
What fixed it
Upgrade to an Airflow release that includes PR #50558. That PR fixed the file-descriptor leak behind airflow dags test (and other CLI paths) leaking descriptors, and the reporter confirmed the FD count stopped growing after the fix. If you are pinned to an older Airflow, there is no config workaround in the thread; upgrading to the fixed release is the resolution.
Evidence
In gh:apache/airflow#49887 (Apache Airflow), the issue reported was: Issue apache/airflow#49887 (closed, 55 comments): ### Apache Airflow version 3.0.0 ### If "Other Airflow 2 version" selected, which one? No response ### What happened? Upgraded Airflow from 2.10 to 3.0.0 and DagProcessorJob getting failed after few hours. with below exception OSError: [Errno 24] Too many open files `` [2025-04-25T21:30:41.765+0100] {dag_processor_job_runner.py:63} ERROR - Exception when executing DagProcessorJob Traceback (most recent call last): File "/var/opt/rdos-airflow/venv/lib/python3.10/site-packages/airflow/jobs/dag_processor_job_runner.py", line 61, in _execute self.processor.run() File... The verified resolution from the thread: Upgrade to an Airflow release that includes PR #50558. That PR fixed the file-descriptor leak behind airflow dags test` (and other CLI paths) leaking descriptors, and the reporter confirmed the FD count stopped growing after the fix. If you are pinned to an older Airflow, there is no config workaround in the thread; upgrading to the fixed release is the resolution. Source: mount the host path https at container path //github.com/apache/airflow/issues/49887
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_i7Ogtr5vhN3WfKpR-ySyvg" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Airflow dag processor exits with too many open files after sometime <span>— Vectle</span></a>