agent's test run logs exceeded artifact size limit: streaming logs

TL;DR

When test logs exceed the artifact size limit, the answer is not smaller tests, it is smaller log handling: stream logs to external storage as they are produced, cap per-test output, and hand the agent only the failure tails. Full logs stay available for humans; the agent gets what it needs to diagnose.

Error

agent's test run logs exceeded artifact size limit: streaming logs

The run's logs blew past the artifact cap, the upload failed, and the agent lost the very output it needed to diagnose the failure.

Steps

1. Find the offenders: rank tests by log volume from the last run. Expected output: a short list of the chattiest tests (usually verbose debug logging or dump-on-every-assertion).
2. Cap per-test log output (e.g. first and last N lines, or a byte cap) in the runner config. Expected output: bounded log size per test regardless of chattiness.
3. Stream logs to external object storage during the run instead of buffering them for one big artifact upload. Expected output: logs that cannot exceed an upload cap because there is no single upload.
4. Build the agent's brief from failure tails: the failing assertion, surrounding context lines, and the error, with a pointer to the full log. Expected output: a compact diagnostic brief per failure.
5. Confirm artifacts upload cleanly and the agent still diagnoses from the briefs. Expected output: green artifact step and unchanged fix rates.

When to use

• Log artifacts fail on size limits.
• Verbose tests flood the agent's context with noise.
• You want full logs retained without handing all of them to the agent.

When not to use

• The suite's logs are small; the plumbing is not worth it.
• The failure needs the full log to diagnose and the agent is the diagnoser (then raise the cap for that run instead of truncating).
• Log volume is a symptom of a test bug (infinite retry logging); fix the test.

Variant phrasings

One chatty test produces 90% of the log volume

Fix the test's log level first; policy handles the rest. Per-test caps catch the next offender automatically.

CI truncates logs and the agent sees a cut-off error

Truncation at the tail is the worst kind: it cuts the actual error. Cap per-test output so the tail survives.

Debug logging left on in the test config

A common offender. Default test runs to warning level; debug goes to the streamed full log only.

Why it happens

Test runners default to capturing everything, and chatty tests (debug logs, object dumps, retry spam) multiply that across hundreds of tests. Artifact stores have hard size caps, and agents have context budgets. Nobody notices until the upload fails or the prompt is 80% log noise.

Edge cases

• Capping can cut the stack trace in half; always keep the exception block intact when truncating (cut from the middle, not the ends).
• Streaming to external storage needs credentials the CI job has; verify the write path before relying on it.
• Compliance may require full log retention; the streamed archive satisfies that while the agent works from tails.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_EuF9VH0Au7gGMF124Pl-JA