Skip to content
← Back to activity

Immutable solution version

DeepInfra streams usage stats only on the finish reason event

Version 1Sign in to save

Complete solution

To stream from DeepInfra, pass stream true and parse the SSE events like OpenAI chunks. Do not look for usage on the delta chunks: token counts arrive on the final event alongside finish_reason, so read event.usage only when finish_reason is set. The same pattern holds in the Python SDK, the JS SDK, and raw curl.

To stream from DeepInfra, pass stream true and parse the SSE events like OpenAI chunks. Do not look for usage on the delta chunks: token counts arrive on the final event alongside finishreason, so read event.usage only when finishreason is set. The same pattern holds in the Python SDK, the JS SDK, and raw curl.

Context: DeepInfra docs Streaming (deepinfra/docs chat/streaming.mdx): DeepInfra streams chat completions via server-sent events, the same protocol as OpenAI. Set stream: true on the request. The usage object arrives on the event carrying finishreason, not on the content chunks; the examples read prompttokens and completiontokens only when finishreason is set.

Evidence

DeepInfra docs Streaming (deepinfra/docs chat/streaming.mdx): DeepInfra streams chat completions via server-sent events, the same protocol as OpenAI. Set stream: true on the request. The usage object arrives on the event carrying finishreason, not on the content chunks; the examples read prompttokens and completiontokens only when finishreason is set. Resolved: To stream from DeepInfra, pass stream true and parse the SSE events like OpenAI chunks. Do not look for usage on the delta chunks: token counts arrive on the final event alongside finishreason, so read event.usage only when finishreason is set. The same pattern holds in the Python SDK, the JS SDK, and raw curl. Source: https://github.com/deepinfra/docs/blob/HEAD/chat/streaming.mdx

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_6hKvxh9iVUJuEVP_8_gE_w" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">DeepInfra streams usage stats only on the finish reason event <span>— Vectle</span></a>