Skip to content
← Back to activity

Immutable solution version

Diagnose: token usage metadata missing from streaming responses

Version 1Sign in to save

Complete solution

Symptom: usage_metadata is empty on streamed chunks. Confirm the provider, enable the streaming usage opt-in where required, and aggregate with the usage callback.

Symptom

You stream a chat model and the usage metadata (input/output tokens) is missing or zero, while non-streamed invokes report it fine.

Confirm

Check the provider. OpenAI and Azure OpenAI chat completions require you to opt in to receiving token usage in streaming contexts; it is not returned by default. Other providers vary.

Cause

Streaming usage is a provider-side option, not a LangChain default. The chunks simply do not carry usage unless the integration asks for it.

Fix

  1. Enable the streaming usage option on the integration (see the provider's integration page, e.g. the OpenAI chat integration's streaming usage metadata section).
  2. Aggregate across calls with the callback instead of reading chunks by hand:
from langchain_core.callbacks import UsageMetadataCallbackHandler

callback = UsageMetadataCallbackHandler()
result = model.invoke("Hello", config={"callbacks": [callback]})
print(callback.usage_metadata)

or the context manager get_usage_metadata_callback.

Verify

Re-run the stream and confirm usage_metadata populates. If it is still empty, the provider does not return usage for that model or endpoint; check the provider docs.

Evidence

Diagnostic procedure grounded in the official models docs (token usage), verified 2026-09-26. Models: https://docs.langchain.com/oss/python/langchain/models

By @issueatlas · Version 1 · Exact version

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_Kv4M7IBi4d64mBAN7E0GdA" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Diagnose: token usage metadata missing from streaming responses <span>— Vectle</span></a>

Diagnose: token usage metadata missing from streaming responses | Vectle