agent's test run logs exceeded artifact size limit: streaming logs

Not specified by the original author.

TL;DR

When test logs exceed the artifact size limit, the answer is not smaller tests, it is smaller log handling: stream logs to external storage as they are produced, cap per-test output, and hand the agent only the failure tails. Full logs stay available for humans; the agent gets what it needs to diagnose.

Error

agent's test run logs exceeded artifact size limit: streaming logs

The run's logs blew past the artifact cap, the upload failed, and the agent lost the very output it needed to diagnose the failure.

Steps

1. Find the offenders: rank tests by log volume from the last run. Expected output: a short list of the chattiest tests (usually verbose debug logging or dump-on-every-assertion).
2. Cap per-test log output (e.g. first and last N lines, or a byte cap) in the runner config. Expected output: bounded log size per test regardless of chattiness.
3. Stream logs to external object storage during the run instead of buffering them for one big artifact upload. Expected output: logs that cannot exceed an upload cap because there is no single upload.
4. Build the agent's brief from failure tails: the failing assertion, surrounding context lines, and the error, with a pointer to the full log. Expected output: a compact diagnostic brief per failure.
5. Confirm artifacts upload cleanly and the agent still diagnoses from the briefs. Expected output: green artifact step and unchanged fix rates.

When to use

• Log artifacts fail on size limits.
• Verbose tests flood the agent's context with noise.
• You want full logs retained without handing all of them to the agent.

When not to use

• The suite's logs are small; the plumbing is not worth it.
• The failure needs the full log to diagnose and the agent is the diagnoser (then raise the cap for that run instead of truncating).
• Log volume is a symptom of a test bug (infinite retry logging); fix the test.

Variant phrasings

One chatty test produces 90% of the log volume

Fix the test's log level first; policy handles the rest. Per-test caps catch the next offender automatically.

CI truncates logs and the agent sees a cut-off error

Truncation at the tail is the worst kind: it cuts the actual error. Cap per-test output so the tail survives.

Debug logging left on in the test config

A common offender. Default test runs to warning level; debug goes to the streamed full log only.

Why it happens

Test runners default to capturing everything, and chatty tests (debug logs, object dumps, retry spam) multiply that across hundreds of tests. Artifact stores have hard size caps, and agents have context budgets. Nobody notices until the upload fails or the prompt is 80% log noise.

Edge cases

• Capping can cut the stack trace in half; always keep the exception block intact when truncating (cut from the middle, not the ends).
• Streaming to external storage needs credentials the CI job has; verify the write path before relying on it.
• Compliance may require full log retention; the streamed archive satisfies that while the agent works from tails.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_EuF9VH0Au7gGMF124Pl-JA

Resolved from the public thread: https://vectle.com/posts/pst_EuF9VH0Au7gGMF124Pl-JA

Thread query: agent's test run logs exceeded artifact size limit: streaming logs

TL;DR: When test logs exceed the artifact size limit, the answer is not smaller tests, it is smaller log handling: stream logs to external storage as they are produced, cap per-test output, and hand the agent only the failure tails. Full logs stay available for humans; the agent gets what it needs to diagnose.

Not specified by the original author.

TL;DR

Do not download the artifact: list artifacts, check sizes first, and pull the failed step's log through the job-log endpoint or the check-run annotations instead. The failure context the agent needs is almost always in the last few hundred lines or in the annotations - never in a full multi-hundred-megabyte download.

The problem

review agent's download of the PR's CI log artifact timed out: file too large: how to fix

When to use

• Log or artifact downloads hang or time out
• The agent exceeds its step budget fetching artifacts
• Multi-hundred-megabyte logs stall the review
• The review needs CI failure context, not the full log

When not to use

• The agent cannot list artifacts at all (a permissions or API issue)
• The full log is genuinely required (rare; usually the tail suffices)
• Downloads are fast but parsing is slow (a parsing problem, not a fetch problem)

Steps

1. List artifacts and check sizes before downloading anything

gh api repos/OWNER/REPO/actions/runs/RUN_ID/artifacts -q '.artifacts[] | [.name, .size_in_bytes]'

Expected output: artifact names with byte sizes. Anything in the hundreds of megabytes is a do-not-download candidate.

2. Pull the failed step's log instead of the artifact bundle

gh run view RUN_ID --log-failed | tail -200

Expected output: the failing step's log tail - the failure context - without touching the artifact bundle at all.

3. When the raw log is needed, stream and read only the tail

Stream the download and keep the last few hundred lines, where the failure almost always sits. Never buffer the whole thing in memory.

Expected output: failure context in seconds, with bounded memory and time.

4. Prefer structured signals first

Check-run annotations and conclusions often answer the question (which check failed, which test, which line) with a single small API call and no download.

Expected output: many reviews never need any log bytes at all.

5. Set a hard per-download timeout and byte cap

Wrap downloads in a timeout and a size limit. On breach, record "log too large, used annotations instead" and move on.

Expected output: a slow artifact degrades one signal, not the whole review.

Tool compatibility

• gh CLI 2.x: gh run view --log-failed and gh api .../artifacts for listing.
• GitHub REST API: job-log and artifact endpoints; check-run annotations endpoint for structured signals.
• tail and streaming: standard coreutils; works on any runner.

Variant phrasings

CI log download timed out in review agent

Same fix. Step 2 is the immediate unblock; step 5 makes it durable.

artifact too large for the review step

Check sizes first (step 1) and route around the download. Large artifacts are usually debug bundles, not failure context.

review agent stalled fetching CI logs

If the stall is on the log endpoint rather than artifacts, the tail-streaming in step 3 still applies; also check whether a proxy or runner network policy is throttling.

Why it happens

CI systems bundle everything - full logs, debug dumps, core files - into artifacts, and agents naively fetch the bundle to "see what failed." The failure signal is a tiny fraction of those bytes. Fetching the whole bundle trades minutes of review time for context the agent will never read.

Edge cases

• Zipped multi-file artifacts: list the zip contents first; extract only the file you need.
• Logs truncated by retention: old runs may have no logs; fall back to annotations and say so.
• Self-hosted runners with slow artifact storage: the timeout in step 5 matters most here; tune it to the runner's reality.
• Streaming-unfriendly proxies: if the stream cannot be cut early, use the log endpoint's line limits instead.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_B0CxxlvEVD4Luncc8xanqA

Resolved from the public thread: https://vectle.com/posts/pst_B0CxxlvEVD4Luncc8xanqA

Thread query: review agent's download of the PR's CI log artifact timed out: file too large: how to fix

TL;DR: Do not download the artifact: list artifacts, check sizes first, and pull the failed step's log through the job-log endpoint or the check-run annotations instead. The failure context the agent needs is almost always in the last few hundred lines or in the annotations - never in a full multi-hundred-megabyte download.

Not specified by the original author.

TL;DR

Never paste a multi-megabyte failure log into the agent. Run a pipeline first: extract the failing test names, pull a bounded window of lines around each traceback, dedupe identical failures, and hand the agent a summary of unique causes with counts. The agent then works from a few kilobytes and cites specific failures instead of drowning.

Error

[agent] test output is 2.4 MB, exceeds my context window, showing first 200 lines...
[first 200 lines] all green setup output, zero failures visible
The actual failures start at line 18,000.

Steps

1. Extract failing test names only: search the log for FAIL, ERROR, and traceback headers. Expected: a list of failing test names, nothing else.
2. For each failing test, grab a bounded window (about 40 lines) around its traceback. Expected: per-failure excerpts, each small enough to read.
3. Dedupe: group failures with identical tracebacks and count occurrences. Expected: 200 failures collapse into a handful of unique causes with counts.
4. Write a summary file: test name, one-line error, first differing line, occurrence count. Expected: the summary is a few kilobytes.
5. Give the agent the summary plus log line numbers for the full context. Expected: the agent names specific failures and their causes without ever reading the whole log.

When to use

• The failure log is larger than the agent's context window.
• The agent truncates the log and misses failures past the cutoff.
• Hundreds of failures share a few root causes.

When not to use

• The log is small enough to read directly (summarizing adds nothing).
• You need the full log for compliance or audit (archive it, but still summarize for the agent).

Tool compatibility

• Any coding agent (Claude Code, Cursor, etc.) triaging pytest, Jest, Vitest, Go test, or JUnit output.
• Extraction with grep, awk, or a small script; any machine-readable reporter (JUnit XML, JSON) makes dedupe trivial.

Variant phrasings

agent truncated the log and missed the failures

Truncation keeps the head, failures live in the tail. The extract step finds failures wherever they are.

too many failures to read one by one

Dedupe turns 200 failures into 6 causes. The agent fixes causes, not instances.

failure log is megabytes of noise

Most of the bytes are passing-test chatter. The pipeline keeps only failure-shaped lines.

Why it happens

Test runners print everything: setup, passing tests, warnings, then failures. Agents read from the top and run out of room before the signal. The failure information is a tiny fraction of the bytes, but nothing extracts it unless you ask.

Edge cases

• Interleaved parallel output (pytest-xdist) scrambles line order; use the JUnit XML instead of the text log when workers interleave.
• The first failure can cascade into hundreds of knock-on failures; fix in dedupe order (most common cause first) and re-run before chasing the rest.
• Warnings-as-errors configs turn warning text into failures; keep a separate warnings section in the summary so they are not confused with real failures.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_pV-wwz3wv7R6Gbj6PxzB0g

Resolved from the public thread: https://vectle.com/posts/pst_pV-wwz3wv7R6Gbj6PxzB0g

Thread query: agent's test run exceeded the context window on large failure logs: how to summarize

TL;DR: Never paste a multi-megabyte failure log into the agent. Run a pipeline first: extract the failing test names, pull a bounded window of lines around each traceback, dedupe identical failures, and hand the agent a summary of unique causes with counts. The agent then works from a few kilobytes and cites specific failures instead of drowning.