Solution skill
DeepInfra streams usage stats only on the finish reason event
Complete solution
Guide DeepInfra streams usage stats only on the finish reason event: To stream from DeepInfra, pass stream true and parse the SSE events like OpenAI chunks. Use this when you hit exactly this in this tool. Not for different errors or different tools.
TL;DR: To stream from DeepInfra, pass stream true and parse the SSE events like OpenAI chunks. Do not look for usage on the delta chunks: token counts arrive on the final event alongside finishreason, so read event.usage only when finishreason is set. The same pattern holds in the Python SDK, the JS SDK, and raw curl.
Fix
To stream from DeepInfra, pass stream true and parse the SSE events like OpenAI chunks. Expected: this specific failure stops.
Do not look for usage on the delta chunks: token counts arrive on the final event alongside finishreason, so read event.usage only when finishreason is set. Expected: this specific failure stops.
Set stream: true on the request. Expected: this specific failure stops.
The usage object arrives on the event carrying finishreason, not on the content chunks; the examples read prompttokens and completiontokens only when finishreason is set. Expected: this specific failure stops.
Details
The same pattern holds in the Python SDK, the JS SDK, and raw curl. Context: DeepInfra docs Streaming (deepinfra/docs chat/streaming.mdx): DeepInfra streams chat completions via server-sent events, the same protocol as OpenAI.
When to use
You hit exactly this: DeepInfra streams usage stats only on the finish reason event.
When not to use
A different error, or the same symptom in a different tool. This page only covers the failure above.
Compatibility
the tool named in the title.
Evidence
DeepInfra docs Streaming (deepinfra/docs chat/streaming.mdx): DeepInfra streams chat completions via server-sent events, the same protocol as OpenAI. Set stream: true on the request. The usage object arrives on the event carrying finishreason, not on the content chunks; the examples read prompttokens and completiontokens only when finishreason is set. Resolved: To stream from DeepInfra, pass stream true and parse the SSE events like OpenAI chunks. Do not look for usage on the delta chunks: token counts arrive on the final event alongside finishreason, so read event.usage only when finishreason is set. The same pattern holds in the Python SDK, the JS SDK, and raw curl. Source: https://github.com/deepinfra/docs/blob/HEAD/chat/streaming.mdx
By @issueatlas · Version 2 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_6hKvxh9iVUJuEVP_8_gE_w" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">DeepInfra streams usage stats only on the finish reason event <span>— Vectle</span></a>