agent timed out downloading the 2TB test image and verification failed
Fixes verification failing when an agent times out downloading a 2TB test image: stream or mount instead of downloading, resume partial transfers, and place tests near the data. Use when huge test images cannot download inside agent timeouts. Not for small image transfer failures.
TL;DR
Downloading a 2TB image through an agent with a job timeout is an architecture problem, not a patience problem. Stop moving the image to the test: mount or stream it in place, run the verification where the data lives, or use resumable chunked transfer with progress checkpoints, and size any unavoidable transfer from measured throughput with the timeout set well above it.
agent timed out downloading the 2TB test image and verification failedUse this when
- Verification of multi-terabyte images times out during download while smaller images pass.
- The transfer restarts from zero after any interruption.
- Throughput tests show the timeout is mathematically impossible for the image size.
Not for this skill when
- The download fails instantly from permissions or missing files. Access problem.
- The image is corrupt once downloaded. Integrity problem after transfer.
Steps
- Calculate the minimum transfer time from measured link throughput and compare it with the agent timeout. Verify: impossibility or feasibility is proven with numbers
- Change the verification to mount the image read-only from its storage or run the checker on the storage host. Verify: a verification runs without a full download
- If transfer is unavoidable, switch to a resumable tool with checkpoint restart and part verification. Verify: interrupting a test transfer resumes instead of restarting
- Place the test execution environment on the same network segment or cloud region as the image store. Verify: throughput improves measurably
- Run the full verification on the 2TB image. Verify: it completes within the window using the new path
Variant phrasings
large image verification without download mount
The mount phrasing.
2tb download timeout resumable transfer test
The resumable variant.
agent test huge file transfer too slow colocate
The colocation wording.
Compatibility: Backup verification workflows with VM images and archives in the terabyte range on LAN, WAN and cloud storage, with resumable transfer tools.
Why it happens
Agent timeouts are set for API calls and scripts, not bulk data movement, but naive verification designs route bulk data through the agent anyway. At terabyte scale even healthy links need hours, so any design requiring a complete download inside a script timeout fails deterministically. Data-local verification removes the transfer from the critical path.
Edge cases / pitfalls
- Mounting backup images read-only is a native feature of major backup tools. Prefer it over exports.
- Resumable transfer still needs the window sized honestly. Resume fixes interruptions, not arithmetic.
- Verify checksums incrementally per part so a late failure does not invalidate the whole transfer.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_6idaJSE5Nh-wscgz6pm47Q