agent's test matrix hung and timed out after queueing 400 restore combos, run failed

TL;DR

A 400-combination restore matrix hangs because the queue is treated as a promise instead of a plan: combos multiply silently, some wait on shared resources, and nothing reports progress. Prune combinations by risk and coverage value, execute as a stream with per-combo timeouts and visible progress, and give the run a global time budget that ends with a partial-coverage report instead of a hang.

agent's test matrix hung and timed out after queueing 400 restore combos, run failed

Use this when

• The run queues hundreds of combos and then makes no visible progress.
• Most combos duplicate the same risk with trivial variations.
• Killed runs produce no usable coverage information.

Not for this skill when

• The sandbox crashes under parallel load. Capacity problem with a different signature.
• One combo deterministically hangs a restore. That combo needs isolation and its own diagnosis.

Steps

1. Export the queued matrix and group combos by what risk each actually covers. Verify: redundant and unique combos are counted
2. Prune to a risk-based set: every distinct platform, storage type and failure mode, sampled variations beyond that. Verify: the reduced matrix still covers all distinct risks
3. Execute combos as a stream with individual timeouts and live progress reporting. Verify: progress is visible per combo
4. Set a global run budget that finalises with a coverage report of done, failed and not-run combos. Verify: a forced budget stop produces the report
5. Run the pruned matrix. Verify: it completes inside the budget with full distinct-risk coverage

Variant phrasings

test matrix explosion combinations prune risk

The pruning phrasing.

agent test run hangs no progress timeout

The hang variant.

400 restore combos coverage sampling

The scale wording.

Compatibility: Agent-driven test matrices across backup products, platforms and storage targets, with CI-style runners and coverage reporting.

Why it happens

Matrices explode combinatorially while risk grows only with distinct dimensions, so most queued combos test the same failure repeatedly. Without streaming progress and budgets, the first slow combo looks identical to a total hang, and operators kill runs that were merely oversized. Risk-based pruning is a testing discipline fix, not an agent tuning tweak.

Edge cases / pitfalls

• Keep a slow full-matrix run on a rare cadence if compliance wants it, separate from the routine pruned run.
• A combo that times out individually should be reported, not allowed to consume the run.
• Coverage reports should name uncovered risks explicitly. Silence reads as coverage.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstoOYExhjUpKzr8w38pHKVQ