agent's dashboard scraper failed on the ramp portal lazy-load error
Fixes Ramp portal scrapers broken by lazy-loading with scroll-and-wait patterns or the Ramp API. Use when a dashboard scraper misses lazily loaded Ramp data. Not for login failures.
TL;DR
The scraper read the page before lazy-loaded rows rendered, so it captured a partial dashboard. Scroll through the data region waiting for new rows to settle, or better, replace the scraper with Ramp's API for the same data.
Error
agent's dashboard scraper failed on the ramp portal lazy-loadSteps
- Confirm the miss: compare the scraped row count to the dashboard's total count. Expected: the scraper captured only the initially visible rows.
- Add scroll-and-settle: scroll the data container to the bottom, wait for the row count to stop growing, repeat. Expected: all rows load before extraction.
- Extract only after the count stabilizes across two consecutive checks. Expected: a complete dataset.
- Prefer the Ramp API: transaction and expense endpoints return the same data without rendering. Expected: no scraper maintenance going forward.
- If scraping remains necessary, re-validate the row count on every run and alert on shortfalls. Expected: silent partial captures become loud failures.
When to use
- A Ramp dashboard scraper returns partial data.
- The portal lazy-loads rows on scroll.
When not to use
- The scraper cannot log in (an authentication problem).
- The data is wrong, not incomplete (check filters and dates).
Tool compatibility
- Ramp web portal; Playwright or Selenium scrapers; Ramp API as replacement.
Variant phrasings
Scraper missed lazy-loaded rows
Same failure, plain wording.
Partial dashboard capture
The symptom phrasing.
Why it happens
Lazy-loading renders rows on demand as the user scrolls. A scraper that reads the DOM once sees only the first viewport of data; everything below the fold is missing.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_Sp-79zx-9vE7w9YC7iZufQ