agent misread the dbt cloud run artifacts url, download failed

TL;DR

dbt Cloud artefacts live behind the API at paths built from the account id, the job or run id, and the artefact name, and a URL assembled from memory, from a UI link, or from a different run will download an error, not the file. Resolve it properly: query the run through the dbt Cloud API, read the artefact hrefs or construct the documented artefact path from the verified run id, download with the API credential header, and confirm the response is the artefact JSON, not an HTML error page saved with a JSON name.

agent misread the dbt cloud run artifacts url, download failed

Use this when

• An agent's download of a manifest, run results, or catalogue artefact fails or returns an error.
• The downloaded file will not parse as JSON and starts with markup or an error message.
• The URL in the agent's code was typed from an example or copied from a browser session.

Not for this skill when

• The run itself failed before generating artefacts. Check the run state first; there may be nothing to download.
• You need artefacts from a local dbt Core run. Those are files in the target folder, no URL involved.

Steps

1. Verify the run exists and finished: query the dbt Cloud API for the run by its id and note its state, job id, and account id from the response. Note the identifiers from the API, not from the agent's notes. Verify: every identifier in the URL is now sourced from the API
2. Get the artefact location the documented way for your API version: the run's artefact links if provided, or the artefacts path for that run and artefact name. Note the exact path you will call. Verify: the URL is constructed from the verified run, per the API shape
3. Call it with the API credential supplied the way the dbt Cloud API expects, following redirects, and save the response with its status code recorded. Note the status and content type. Verify: the download is authenticated and its outcome is recorded
4. Validate what arrived before using it: the file parses as JSON and contains the artefact's expected top level keys, such as metadata for a manifest. Note the check result. Verify: a saved error page can no longer masquerade as an artefact
5. Fix the agent procedure so artefact URLs are always derived from a run lookup in the same session, and an unparsable download is reported as a failed download rather than analysed. Note the guard. Verify: future downloads are grounded in a live run lookup

Variant phrasings

dbt cloud api get run artifacts manifest.json download 404

The API artefact phrasing.

dbt cloud run artifacts url account id run id

The identifier construction search.

Compatibility: dbt Cloud administrative and metadata API versions in current use; artefact availability differs per artefact type and run outcome. Local dbt Core artefacts are plain files and out of scope.

Why it happens

Artefact URLs are identifiers all the way down: change one segment, the account, the run, or the artefact name spelling, and the path points at nothing, or at a login or error response that a naive downloader happily saves. Agents are prone to this because a URL looks like knowledge they should have, and examples online mix API versions and UI links freely. The run lookup first pattern removes the guesswork, because every segment of the final URL is copied from a response the API just gave about the run you actually want.

Edge cases / pitfalls

• A 200 response can still be wrong if an expired credential redirects to a login page. The parse check in step four is the real validator.
• Run results and manifest artefacts are generated at different points in a run. A run cancelled midway may have one and not the other.
• Keep artefact downloads per run id in separate folders. Same named files from different runs overwrite each other and create phantom analysis errors.
• API base URLs differ by dbt Cloud region and deployment type. Take the base from your account settings, not from an example.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst66yZ9nJWqEW8_1govbugA