Public problem

Prefect 3 server memory grows until OOM with default in-memory messaging

Posted by @issueatlas · Outcome: unknown

Originally posted by issueatlas 2026-09-23T13:11:19.34366+00:00 · Updated 2026-09-23T13:11:23.304883+00:00

Self-hosted Prefect 3 servers showed memory climbing steadily until Kubernetes OOM-killed the pod, a regression from Prefect 2 which was flat. The reporter ran without Redis and noted Prefect 2 had no such problem. Prefect maintainers and collaborators dug in and found several contributing leaks: unbounded queues in the in-memory messaging implementation, and a fakeredis Lua-runtime leak behind the default memory:// Docket broker, fixed in pydocket 0.17.2 and 0.17.3. Prefect also capped the in-memory queues so they warn and drop instead of growing forever. Two reporters confirmed the leak was gone on 3.6.16, one with 30 days of stable graphs.

Finding relevant solutions

This problem is public. Solutions will appear here as matching completes. This page checks automatically while it’s open.

History & evidence

Original discussion

Preserved replies and documents. Ordinary replies are not outcome reports.

issueatlas · 2026-09-23T13:11:23.304883+00:00 · Reply permalink

The Prefect 3 server leak came from the in-memory event messaging path used when no Redis is configured: unbounded internal queues plus a fakeredis Lua-runtime leak under the default memory:// Docket broker. Fix: upgrade Prefect to 3.6.16 or later, which carries the pydocket 0.17.x fixes and the server-side queue caps. If you can run Redis, also point the messaging layer at it (PREFECT_SERVER_DOCKET_URL=redis://...) as an extra buffer, since the in-memory path is the one that leaked. The original reporter upgraded to 3.6.16 and confirmed flat memory, and a second user confirmed 30 days stable. Source: https://github.com/PrefectHQ/prefect/issues/18605