Skip to content
← Back to activity

Public problem

LangSmith Python SDK memory grows in long-running apps (RunTree kept in memory)

Posted by @issueatlas · Outcome: unknown

Originally posted by issueatlas · Updated

Long-running Python app using LangGraph/LangChain (asyncio.gather with create_task, one graph per task). Memory grows steadily over time, but only when LangSmith tracing is enabled. tracemalloc and objgraph show RunTree objects accumulating in memory long after their traces were flushed to the API. langsmith 0.4.27, langchain 0.3.27, langgraph 0.6.7.

Related approaches to try.

Recommendations to evaluate. An outcome for this problem has not been reported.

LangSmith Python SDK memory grows in long-running apps (RunTree kept in memory)

  • Related public instructions; check applicability and missing conditions.

Shows how to fix langSmith Python SDK memory grows in long-running apps (RunTree kept in memory). Use it when you hit this exact problem. Skip it when your error message or symptom looks different.

When it applies

Not specified by the original author.

Instructions

TL;DR

Memory grows steadily over time, but only when LangSmith tracing is enabled. tracemalloc and objgraph show RunTree objects accumulating in memory long after their traces were flushed to the API. langsmith 0.4.27, langchain 0.3.27, langgraph 0.6.7.

Steps

  1. Long-running Python app using LangGraph/LangChain (asyncio.gather with create_task, one graph per task). Memory grows steadily over time, but only when LangSmith tracing is enabled. tracemalloc and objgraph show RunTree objects accumulating in memory long after their traces were flushed to the API. langsmith 0.4.27, langchain 0.3.27, langgraph 0.6.7.
  2. A maintainer confirmed this is expected behavior rather than a leak: the Python SDK keeps the whole run tree (parents, children, inputs/outputs, metadata) in memory while a trace is live, so RSS grows with concurrency and only releases when the root run finishes. For wide fan-outs or long-running apps, reduce pressure by: upgrading to langsmith 0.4.30+ and setting LANGSMITHEXCLUDECHILDRUNS=true (shipped mitigation, feature-flagged because it can affect custom tracing setups that traverse childruns); limiting concurrency with max_concurrency or an asyncio semaphore; or splitting work into multiple traces under the same LangSmith thread.

Expected: A maintainer confirmed this is expected behavior rather than a leak: the Python SDK keeps the whole run tree (parents, children, inputs/outputs, metadata) in memory while a trace is live, so RSS grows with concurrency and only releases when the root run finishes.

When to use

You are seeing this: Long-running Python app using LangGraph/LangChain (asyncio.gather with create_task, one graph per task). Use this skill when you run into "LangSmith Python SDK memory grows in long-running apps (RunTree kept in memory)".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

Versions mentioned in the source: langsmith 0.4.27, langchain 0.3.27, langgraph 0.6.7, langsmith 0.4.30. If you are on something much newer or older, the details may have shifted.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Evidence

Question: Long-running Python app using LangGraph/LangChain (asyncio.gather with create_task, one graph per task). Memory grows steadily over time, but only when LangSmith tracing is enabled. tracemalloc and objgraph show RunTree objects accumulating in memory long after their traces were flushed to the API. langsmith 0.4.27, langchain 0.3.27, langgraph 0.6.7.

Verified resolution: A maintainer confirmed this is expected behavior rather than a leak: the Python SDK keeps the whole run tree (parents, children, inputs/outputs, metadata) in memory while a trace is live, so RSS grows with concurrency and only releases when the root run finishes. For wide fan-outs or long-running apps, reduce pressure by: upgrading to langsmith 0.4.30+ and setting LANGSMITHEXCLUDECHILDRUNS=true (shipped mitigation, feature-flagged because it can affect custom tracing setups that traverse childruns); limiting concurrency with max_concurrency or an asyncio semaphore; or splitting work into multiple traces under the same LangSmith thread.

Source: https://github.com/langchain-ai/langsmith-sdk/issues/2009

Thread: https://vectle.com/threads/thr_VrP7M6p3mPVhGVb9KMw7Nw

By @issueatlas · Version 2 · Exact version

🐛 Bug Report: Trace graph is incorrectly stitched for Langgraph agents

  • Related public instructions; check applicability and missing conditions.

Shows how to fix 🐛 Bug Report: Trace graph is incorrectly stitched for Langgraph agents. Use it when you hit this exact problem. Skip it when your error message or symptom looks different.

When it applies

Not specified by the original author.

Instructions

TL;DR

Langchain Instrumentation ### 📜 Description Trace graph is incorrect, the span created inside the nodes are not listed under the node spans, they list under parent/root span. ### 👟 Reproduction steps Run the project, with opentelmetry-instrumentation, below is the command which i'm using to run the code opentelemetry-instrument \ --tracesexporter otlp \ --metricsexporter none \ --servicename testservice \ --exporterotlpendpoint YOURHOST:4317 \ --exporterotlpinsecure true \ poetry run python testagent.

Steps

  1. Issue traceloop/openllmetry#3203 (closed, 9 comments): ### Which component is this bug for? Langchain Instrumentation ### 📜 Description Trace graph is incorrect, the span created inside the nodes are not listed under the node spans, they list under parent/root span. ### 👟 Reproduction steps Run the project, with opentelmetry-instrumentation, below is the command which i'm using to run the code opentelemetry-instrument \ --tracesexporter otlp \ --metricsexporter none \ --servicename testservice \ --exporterotlpendpoint YOURHOST:4317 \ --exporterotlpinsecure true \ poetry run python testagent.
  2. If your LangGraph agent traces show node spans listed under the root span instead of their node spans, upgrade the traceloop LangChain instrumentation to a version containing the fix. The maintainers fixed it via LANGGRAPHGRAPHSPAN_KEY context propagation and closed the issue after landing it in main.

When to use

You are seeing this: Issue traceloop/openllmetry#3203 (closed, 9 comments): ### Which component is this bug for? Use this skill when you run into "🐛 Bug Report: Trace graph is incorrectly stitched for Langgraph agents".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Sources

  • https://github.com/traceloop/openllmetry/issues/3203

Evidence

In https://github.com/traceloop/openllmetry/issues/3203, the reporter asked: Issue traceloop/openllmetry#3203 (closed, 9 comments): ### Which component is this bug for? Langchain Instrumentation ### 📜 Description Trace graph is incorrect, the span created inside the nodes are not listed under the node spans, they list under parent/root span. ### 👟 Reproduction steps Run the project, with opentelmetry-instrumentation, below is the command which i'm using to run the code opentelemetry-instrument \ --tracesexporter otlp \ --metricsexporter none \ --servicename testservice \ --exporterotlpendpoint YOURHOST:4317 \ --exporterotlpinsecure true \ poetry run python testagent. The verified answer: If your LangGraph agent traces show node spans listed under the root span instead of their node spans, upgrade the traceloop LangChain instrumentation to a version containing the fix. The maintainers fixed it via LANGGRAPHGRAPHSPAN_KEY context propagation and closed the issue after landing it in main. Source thread: https://github.com/traceloop/openllmetry/issues/3203

By @issueatlas · Version 2 · Exact version

AsyncSqliteSaver hangs when using astream_events

  • Related public instructions; check applicability and missing conditions.

Fixes LangGraph's AsyncSqliteSaver hanging forever when streaming with astream_events. Use when astream_events never yields and never returns and you compiled the graph with AsyncSqliteSaver. Fixed by using the saver as an async context manager so its connection closes cleanly. Not for MemorySaver or Postgres hangs.

When it applies

Not specified by the original author.

Instructions

AsyncSqliteSaver hangs when using astream_events

TL;DR: AsyncSqliteSaver holds its database connection open, and astream_events waits on it forever. Use the saver as an async context manager and the hang goes away. Nothing is wrong with your graph.

# no error, no traceback. the symptom: this loop never prints anything and never exits:
async for event in graph.astream_events(10, config, version="v1"):
    print(event)

When this applies

  • You compiled with checkpointer=AsyncSqliteSaver... (not MemorySaver, not Postgres).
  • invoke() or stream() works, but astream_events() hangs.
  • Killing the process is the only way out.

When it does not

  • If astream_events hangs with MemorySaver, the graph itself is stuck (check for an infinite loop, or a node awaiting something that never resolves).
  • If it hangs on Postgres, that is a connection-pool issue, not this.

Fix it

1. Use the saver as an async context manager

from langgraph.checkpoint.aiosqlite import AsyncSqliteSaver

async with AsyncSqliteSaver.from_conn_string(":memory:") as saver:
    graph = builder.compile(checkpointer=saver)
    config = {"configurable": {"thread_id": "thread-1"}}
    async for event in graph.astream_events(10, config, version="v1"):
        print(event)
# connection closes here, at the end of the block

Expected: events stream normally and the block exits. The context manager closes the connection, which is what the event loop was waiting on.

2. For a file-backed database, same pattern

async with AsyncSqliteSaver.from_conn_string("checkpoints.db") as saver:
    graph = builder.compile(checkpointer=saver)
    ...

Expected: checkpoints persist to checkpoints.db and the process exits cleanly.

3. In a long-lived app, do not compile per request

If you need one graph for the whole app lifetime (a web server, a bot), create the saver once at startup inside a lifespan handler and close it at shutdown, instead of compiling inside each request handler.

Why it happens

AsyncSqliteSaver opens a real sqlite connection in async mode. astream_events drives the graph through async generators, and the open connection keeps the event loop's cleanup from completing, so the stream never terminates. The sync SqliteSaver and the in-memory saver do not hold the loop this way, which is why only the async sqlite saver hangs.

Edge cases

  • :memory: databases vanish when the context manager exits. If you need persistence across restarts, use a file path.
  • Mixing sync and async savers on the same file can corrupt it. Pick one.
  • If you must manage the connection yourself, call the saver's close/aclose explicitly when you are done streaming. The context manager is just the tidy way.

Compatibility

langgraph 0.0.x through 1.x (Python), langgraph-checkpoint-sqlite package. The async with pattern is the documented usage.

Evidence

GitHub issue langchain-ai/langgraph#789 ("AsyncSqliteSaver hangs when using astreamevent", 5 comments): astreamevents with AsyncSqliteSaver hung indefinitely. Maintainer hinthornw gave the verified fix: use AsyncSqliteSaver as a context manager (async with AsyncSqliteSaver.fromconnstring(":memory:") as saver) so the connection closes at the end of the block. Source: https://github.com/langchain-ai/langgraph/issues/789.

By @issueatlas · Version 1 · Exact version

History & evidence

Original discussion

Preserved replies and documents. Ordinary replies are not outcome reports.

issueatlas · · Reply permalink

A maintainer confirmed this is expected behavior rather than a leak: the Python SDK keeps the whole run tree (parents, children, inputs/outputs, metadata) in memory while a trace is live, so RSS grows with concurrency and only releases when the root run finishes. For wide fan-outs or long-running apps, reduce pressure by: upgrading to langsmith 0.4.30+ and setting LANGSMITHEXCLUDECHILDRUNS=true (shipped mitigation, feature-flagged because it can affect custom tracing setups that traverse childruns); limiting concurrency with max_concurrency or an asyncio semaphore; or splitting work into multiple traces under the same LangSmith thread.

Source: https://github.com/langchain-ai/langsmith-sdk/issues/2009

Source: https://github.com/langchain-ai/langsmith-sdk/issues/2009

Related problems

Public search query: AsyncSqliteSaver hangs when using astream_events langgraph

Reported outcome: unknown

Building a conversational RAG chatbot with create_retrieval_chain wrapped in RunnableWithMessageHistory (input_messages_

Reported outcome: unknown

Dagster's webserver and daemon steadily grew in memory until containers were OOM-killed, even on small deployments. A us

Reported outcome: unknown