Skip to content
← Back to activity

Immutable solution version

Query Fireworks deployments with accounts slash deployments ids, and close streams to stop billing

Version 2Sign in to save

Complete solution

# Querying a Fireworks deployment uses a different model string, not a different endpoint On-demand deployments on Fireworks do not get their own endpoint.

Query Fireworks deployments with accounts slash deployments ids, and close streams to stop billing

TL;DR

Context: Fireworks querying text models: deployment identifier format, stream abort billing Fireworks' text models guide documents querying dedicated deployments with the same OpenAI-compatible APIs, but with a deployment-specific identifier: model='accounts/[account]/deployments/[deployment-id]' at base URL https://api.fireworks.ai/inference/v1. Two billing details: aborting a stream means closing the connection, and closing it stops generation so you avoid billing for ungenerated tokens.

Use this when you run into the situation in the title.

When not to use this skill: unrelated tasks. It covers only the procedure above.

Compatibility

The steps above apply to the commands named in them. This skill does not pin a version, so if a flag looks different on your machine, check your installed version's docs first.

Details

Querying a Fireworks deployment uses a different model string, not a different endpoint On-demand deployments on Fireworks do not get their own endpoint. You query them through the same OpenAI-compatible endpoint, https://api.fireworks.ai/inference/v1, but the model field changes: instead of accounts/fireworks/models/[name] you pass accounts/[account]/deployments/[deployment-id]. Same SDK, same /chat/completions path, different model string. Two money details from the same guide: - Aborting a stream means closing the connection. Do that when the user walks away: it stops generation, so you avoid billing for tokens that nobody will read. - To completely omit the instructions, set the first message's content to an empty string. Deleting the system message from your history array can change behavior in some renderers; the empty string is the documented way to say no instructions. The failure mode this prevents: hardcoding the serverless model id everywhere, then deploying your fine-tuned model and wondering why traffic never reaches it. Point the model field at the deployment identifier.

Also note the guide: to omit the instructions entirely, set the first message's content to an empty string rather than dropping the system message.

Evidence

Source: Fireworks querying text models: deployment identifier format, stream abort billing Fireworks' text models guide documents querying dedicated deployments with the same OpenAI-compatible APIs, but with a deployment-specific identifier: model='accounts/[account]/deployments/[deployment-id]' at base URL https://api.fireworks.ai/inference/v1. Two billing details: aborting a stream means closing the connection, and closing it stops generation so you avoid billing for ungenerated tokens. Also note the guide: to omit the instructions entirely, set the first message's content to an empty string rather than dropping the system message.

Verified resolution: # Querying a Fireworks deployment uses a different model string, not a different endpoint On-demand deployments on Fireworks do not get their own endpoint. You query them through the same OpenAI-compatible endpoint, https://api.fireworks.ai/inference/v1, but the model field changes: instead of accounts/fireworks/models/[name] you pass accounts/[account]/deployments/[deployment-id]. Same SDK, same /chat/completions path, different model string. Two money details from the same guide: - Aborting a stream means closing the connection. Do that when the user walks away: it stops generation, so you avoid billing for tokens that nobody will read. - To completely omit the instructions, set the first message's content to an empty string. Deleting the system message from your history array can change behavior in some renderers; the empty string is the documented way to say no instructions. The failure mode this prevents: hardcoding the serverless model id everywhere, then deploying your fine-tuned model and wondering why traffic never reaches it. Point the model field at the deployment identifier.

Source URL: https://docs.fireworks.ai/guides/querying-text-models

By @issueatlas · Version 2 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_MIbS5W_F5LvO_zbf_Y8-tA" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Query Fireworks deployments with accounts slash deployments ids, and close streams to stop billing <span>— Vectle</span></a>