Comet API rate limit hit mid-experiment export error failed to download logs

TL;DR

Export in pages and expect the limit: the Comet API rate limit stopped the export mid download, so request logs in paged chunks, store each chunk before requesting the next, and resume the export from the last stored chunk after a limit pause. Exports are long reads over many objects. A single all-at-once request gives the limit one target and the workflow nothing to resume from.

Comet API rate limit hit mid-experiment export error failed to download logs

Use this when

• The check or run fails with this symptom: a Comet API rate limit hits mid experiment export and log downloads fail.
• You are working on an agent workflow exporting Comet experiments and their logs for analysis or migration and the failure repeats on the same data or configuration.
• You need the failing run, call, or stage isolated before changing code or thresholds.

Not for this skill when

• A limit that persists beyond its stated window, which indicates an account level issue to raise with the provider.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Note which experiment and which chunk the export reached. Verify: the export position is recorded
2. Split the export into per-experiment or per-page requests and store each response as it arrives. Verify: stored chunks accumulate without rework
3. On a limit response, pause for the window the API states, then continue from the next chunk. Verify: the export resumes instead of restarting
4. Assemble the export from the stored chunks and verify experiment counts against the workspace. Verify: the assembled export is complete
5. Schedule recurring exports incrementally so each run moves only new experiments. Verify: export volume per run stays small

Variant phrasings

comet api rate limit mid experiment export logs failed

The shorter phrasing focused on the named failure.

comet export rate limit agent error

The symptom phrasing a searcher tries next.

comet export pagination resume fix

The variant naming the failing stage or tool directly.

Compatibility: Comet ML API pagination and rate limits, agent export workflows

Why it happens

Agent exports are usually written as one request because the test workspace is small. At real scale the export duration crosses rate limit windows, and without chunk level persistence the only recovery is starting over, which hits the same limit at the same place.

Edge cases / pitfalls

• Verify completeness by counts from the workspace, not by the absence of errors in the export log.
• Log downloads dominate export size; export metadata for everything and logs only for the runs that need them.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstI2xNEHtvdn7ISgaEYIz-A