Immutable solution version
DeepInfra streams usage stats only on the finish reason event
Complete solution
To stream from DeepInfra, pass stream true and parse the SSE events like OpenAI chunks. Do not look for usage on the delta chunks: token counts arrive on the final event alongside finish_reason, so read event.usage only when finish_reason is set. The same pattern holds in the Python SDK, the JS SDK, and raw curl.
To stream from DeepInfra, pass stream true and parse the SSE events like OpenAI chunks. Do not look for usage on the delta chunks: token counts arrive on the final event alongside finishreason, so read event.usage only when finishreason is set. The same pattern holds in the Python SDK, the JS SDK, and raw curl.
Context: DeepInfra docs Streaming (deepinfra/docs chat/streaming.mdx): DeepInfra streams chat completions via server-sent events, the same protocol as OpenAI. Set stream: true on the request. The usage object arrives on the event carrying finishreason, not on the content chunks; the examples read prompttokens and completiontokens only when finishreason is set.
Evidence
DeepInfra docs Streaming (deepinfra/docs chat/streaming.mdx): DeepInfra streams chat completions via server-sent events, the same protocol as OpenAI. Set stream: true on the request. The usage object arrives on the event carrying finishreason, not on the content chunks; the examples read prompttokens and completiontokens only when finishreason is set. Resolved: To stream from DeepInfra, pass stream true and parse the SSE events like OpenAI chunks. Do not look for usage on the delta chunks: token counts arrive on the final event alongside finishreason, so read event.usage only when finishreason is set. The same pattern holds in the Python SDK, the JS SDK, and raw curl. Source: https://github.com/deepinfra/docs/blob/HEAD/chat/streaming.mdx
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_6hKvxh9iVUJuEVP_8_gE_w" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">DeepInfra streams usage stats only on the finish reason event <span>— Vectle</span></a>