browser automation failed when agent tried to open dbt cloud ide logs
Teaches an agent to get dbt Cloud run logs from the API instead of driving a browser to the IDE, which is brittle and slow. Use when browser automation fails on the dbt Cloud UI. Not when the API also lacks the needed data.
TL;DR
Driving a browser to the dbt Cloud IDE for logs is fragile: logins, lazy loading, and UI changes all break it. The agent should use the dbt Cloud API instead: get the run ID, fetch the run's logs or artifacts from the API, and parse those.
Error
browser automation failed when agent tried to open dbt cloud ide logsSteps
- Stop the browser automation; it is the wrong tool for this data. Expected: no more flaky UI driving.
- Identify the run: account ID, project ID, and run ID (from the job that failed). Expected: the three IDs needed for API calls.
- Fetch the run details and logs via the API's run endpoints. Expected: structured run data and log text.
- If step-level logs are needed, fetch the artifacts for the run (they include per-node results). Expected: the same information the IDE shows, as data.
- Analyze the logs or artifacts and proceed with the fix. Expected: the agent works from API data.
When to use
- An agent tries to scrape the dbt Cloud UI for logs.
- Browser automation fails on login, lazy loading, or UI changes.
When not to use
- The API genuinely lacks the data (rare; then the UI is the fallback, driven carefully).
- A human wants to look at the UI (browsers are for humans).
Tool compatibility
- dbt Cloud API; any HTTP client.
Variant phrasings
Agent cannot get past dbt Cloud login in a browser
API credentials bypass the login entirely; that is the point.
Lazy-loaded log panels break the scraper
The API returns complete logs; no scrolling needed.
Why it happens
Web UIs are built for humans: they lazy-load, require interactive login, and change without notice. APIs are built for programs: they are stable, complete, and fast. Agents should use the program interface.
Edge cases
- API credentials need the right permissions; a 403 means fixing access, not returning to the browser.
- Very recent runs may not have artifacts ready; poll for completion first.
- Keep the browser automation skill for the rare cases the API cannot cover.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_OlKvkUq4ZEQN04SCmoCA