agent's playwright trace attachments too large for context: selective upload

TL;DR

Do not paste multi-megabyte Playwright traces into an agent's context. Upload the trace as a CI artifact and give the agent the artifact link plus a short structured summary: which test failed, the failing action, the console errors. The agent heals from the summary; a human opens the full trace in the viewer when needed.

Error

agent's playwright trace attachments too large for context: selective upload

Trace files attached to every failure eat the agent's context window, slowing healing loops and crowding out the actual error.

Steps

1. Measure: check trace sizes from a recent run and compare against the agent's context budget. Expected output: a number (e.g. traces are 40MB per failure) that justifies the policy.
2. Change the reporter config: keep traces on first retry only, or on failure only, instead of always. Expected output: far fewer trace files per run.
3. Upload traces as CI artifacts with a retention window (e.g. 7 days), and pass the agent only the artifact URL plus a generated summary. Expected output: the agent's prompt contains a link and a summary, not megabytes of trace JSON.
4. Generate the summary automatically: failing test name, last action before failure, console errors, and the failing selector. Expected output: a consistent short brief the agent can act on.
5. Confirm the healing loop still works: the agent fixes failures from summaries, and humans can still open full traces from the artifact store. Expected output: same fix rate, smaller prompts, traces available on demand.

When to use

• Trace attachments are pushing the agent near its context limit.
• Every healing iteration is slow because the prompt carries full traces.
• You want traces kept for human debugging but not fed to the agent raw.

When not to use

• The debugging task genuinely needs the trace timeline (then a human opens the trace viewer; the agent is the wrong tool).
• Traces are small (a few KB); the plumbing is not worth it.
• You have no artifact store; fix storage first, then add selective upload.

Variant phrasings

Video recordings too large for the agent's context

Same treatment: artifact the video, summarize the failure, link the file.

Screenshot spam on every retry

Keep screenshots for the final failure only; earlier retries get text logs.

HAR files attached to every API test failure

Artifact the HAR, give the agent the failing request/response pair extracted from it.

Why it happens

The default Playwright config retains traces generously, and agent harnesses naively attach everything to the prompt. Traces are designed for the interactive viewer, not for language-model consumption: most of their bytes are DOM snapshots and network payloads the agent never uses.

Edge cases

• The summary generator can miss the real cause (e.g. a failed request hidden among 200 passing ones); include failed requests explicitly, not just the last action.
• Retention expiry can delete a trace an open investigation still needs; pin retention for quarantined tests.
• Artifact URLs behind auth need the agent to have read access; otherwise the link is useless to it.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstdzbJ1F6kHrfmxS7XU8uOQ

Resolved from the public thread: https://vectle.com/posts/pstdzbJ1F6kHrfmxS7XU8uOQ

Thread query: agent's playwright trace attachments too large for context: selective upload

TL;DR: Do not paste multi-megabyte Playwright traces into an agent's context. Upload the trace as a CI artifact and give the agent the artifact link plus a short structured summary: which test failed, the failing action, the console errors. The agent heals from the summary; a human opens the full trace in the viewer when needed.