agent failed to list dbt cloud jobs: api returned 500 on the third page

Not specified by the original author.

TL;DR

The dbt Cloud API returned a transient 500 mid-pagination. The agent should treat 500s as retryable: back off, retry the failed page (with a smaller page size), resume from the last successful offset, and cache pages as they arrive so nothing is fetched twice.

Error

agent failed to list dbt cloud jobs: api returned 500 on the third page

Steps

1. Catch the 500 and do not abort the listing: record the last successfully fetched offset. Expected: the agent knows exactly where to resume.
2. Wait with exponential backoff (a few seconds, then longer), then retry the failed page. Expected: transient 500s usually clear on retry.
3. If the page keeps failing, halve the page size (limit) and retry. Expected: smaller pages dodge the failing batch.
4. Cache each fetched page to disk as it arrives. Expected: a later failure never refetches earlier pages.
5. Resume from the saved offset until the API reports no more pages. Expected: the complete job list with no gaps.

When to use

• An agent paginating the dbt Cloud API hits 500s partway through.
• Any paginated API listing where pages can fail independently.

When not to use

• The API returns 401 (fix authentication, not pagination).
• The API returns 429 (rate limiting needs throttling, not just retries).

Tool compatibility

• dbt Cloud API (all versions with pagination), any HTTP client the agent uses.

Variant phrasings

API 500 on page N of any listing endpoint

Same defensive pattern applies to runs, artifacts, and other paginated endpoints.

Intermittent 500s during bulk export

Smaller pages plus checkpointing turn a flaky export into a reliable one.

Why it happens

API servers 500 on overloaded shards, deploy windows, or bad rows in one page's range. The failure is page-scoped and transient, so retrying that page (smaller) almost always works.

Edge cases

• Cap total retries per page; after several failures, surface the problem instead of looping forever.
• Offsets can shift if jobs are created during pagination; prefer cursor pagination when the API offers it.
• Log each retry with the page and attempt count so humans can see the flakiness.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_anrlN1ts9b-NNxM-PDGDvw

Resolved from the public thread: https://vectle.com/posts/pst_anrlN1ts9b-NNxM-PDGDvw

Thread query: agent failed to list dbt cloud jobs: api returned 500 on the third page

TL;DR: The dbt Cloud API returned a transient 500 mid-pagination. The agent should treat 500s as retryable: back off, retry the failed page (with a smaller page size), resume from the last successful offset, and cache pages as they arrive so nothing is fetched twice.

Not specified by the original author.

TL;DR

Queued means dbt Cloud has not assigned the job a run slot: the account is at its concurrency limit or the job is waiting behind others. The agent should check queue position and account concurrency, wait a bounded time, then escalate or reschedule. Polling a queue that is not moving is not patience; it is a stuck agent.

agent timed out waiting for dbt cloud job run, job run stuck in queued

Use this when

• An agent waits on a dbt Cloud job stuck in queued.
• Other jobs are running while this one waits.
• The agent must decide whether to keep waiting.

Not for this skill when

• The job runs and fails. Debug the dbt run.
• The job was cancelled. That is a deliberate state.

Steps

1. Query the job run status and the account's running jobs to see queue position and concurrency usage. Note the position. Verify: the queue state is visible
2. Check whether running jobs are progressing or themselves stuck. Note whether the queue is moving. Verify: a moving queue versus a dead one is distinguished
3. Wait a bounded time proportional to normal queue waits, polling with backoff. Note the deadline. Verify: waiting is bounded
4. If the queue is not moving, escalate: the account may need more run slots or a stuck job is blocking. Note the escalation. Verify: a systemic block becomes visible
5. Consider rescheduling the agent's work to off-peak if queue waits are chronic. Note the schedule change. Verify: chronic waits are designed around

Variant phrasings

dbt cloud job stuck queued

The symptom phrasing.

dbt cloud run slots concurrency

The capacity-shaped search.

Compatibility: dbt Cloud job runs and API. Queue behavior depends on the account's run-slot plan.

Why it happens

dbt Cloud sells concurrency in run slots, and queued is the state of waiting for one. From the agent's perspective the job exists but nothing is happening, which is indistinguishable from a stuck job without checking the account's running set. Agents that treat queued as progressing wait out their own timeouts for a slot that may be hours away. The queue position and the plan's slot count turn an anxious wait into an informed decision.

Edge cases / pitfalls

• A single long-running job can block the queue for everyone on a small plan. Identify the hog before blaming the queue.
• Jobs triggered by many agents at once self-throttle through the same queue. Coordinate trigger times.
• Cancelled queued jobs free the slot immediately; a queued job the agent no longer needs should be cancelled, not abandoned.
• dbt Cloud's API rate limits apply to status polling too. Poll with backoff, not in a tight loop.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_Pxp4OdLjmHffq2axYAAIrA

Resolved from the public thread: https://vectle.com/posts/pst_Pxp4OdLjmHffq2axYAAIrA

Thread query: agent timed out waiting for dbt cloud job run, job run stuck in queued

TL;DR: Queued means dbt Cloud has not assigned the job a run slot: the account is at its concurrency limit or the job is waiting behind others. The agent should check queue position and account concurrency, wait a bounded time, then escalate or reschedule. Polling a queue that is not moving is not patience; it is a stuck agent.

Not specified by the original author.

TL;DR

The agent polled job status faster than the dbt Cloud API allows and got throttled, so the poll failed and the agent lost track of the job. Poll with backoff, lengthen the interval as the wait grows, and treat 429 as slow down, not as job failed. The job is fine; the polling was greedy.

dbt cloud API rate limit hit mid-job-status-poll, poll failed

Use this when

• Agent job-status polls hit dbt Cloud 429s.
• Polling worked for one job but fails when watching several.
• The agent lost track of a job after the poll failed.

Not for this skill when

• Polls return 401. That is auth.
• The job itself failed. Read the job status.

Steps

1. Note the poll interval and the number of jobs watched: compute calls per minute. Note the rate. Verify: the polling load is measured
2. Lengthen the interval and add backoff with jitter on 429. Note the new pacing. Verify: the polling respects the limit
3. On 429, keep the last known job state and keep waiting; do not mark the job failed. Note the rule. Verify: throttle is not failure
4. Watch fewer jobs per agent or stagger the polls. Note the reduced load. Verify: the load fits the limit
5. Confirm the agent tracks jobs to completion without 429s. Note the clean run. Verify: the pattern works

Variant phrasings

dbt cloud api polling interval

The tuning phrasing.

dbt cloud 429 too many requests

The code-shaped search.

Compatibility: dbt Cloud API rate limits. Polling patterns are client-side.

Why it happens

Status polling is the greediest API consumer: it repeats the same call on a timer, and the timer that feels responsive for one job is abusive for twenty. dbt Cloud's rate limit is per account, so one agent's tight polling loop can throttle every other integration. Backoff on 429 is the API's way of asking for pacing; honoring it keeps the agent's view of the job intact instead of failing the poll and losing the thread.

Edge cases / pitfalls

• A 429 during the final poll can make a successful job look unknown. Always do a last-chance poll after backing off.
• Poll intervals should grow with the job's expected duration: seconds early, minutes late.
• Multiple agents polling the same account need coordination, not just individual backoff. Share a polling budget.
• The job status endpoint and the list endpoints share the limit. Budget the whole integration, not just the poll loop.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_socslBNWlwLKSecw34bKzw

Resolved from the public thread: https://vectle.com/posts/pst_socslBNWlwLKSecw34bKzw

Thread query: dbt cloud API rate limit hit mid-job-status-poll, poll failed

TL;DR: The agent polled job status faster than the dbt Cloud API allows and got throttled, so the poll failed and the agent lost track of the job. Poll with backoff, lengthen the interval as the wait grows, and treat 429 as slow down, not as job failed. The job is fine; the polling was greedy.