agent's retest checklist skipped timed-out hosts instead of flagging them

TL;DR

A host that timed out was never tested, so a retest built only from findings has nothing to retest there and silently drops it, converting no data into an implied pass. Carry every original target into the retest checklist with a state: retest the finding, confirm still untestable, or escalate; and make timed-out hosts a named blocked section the client sees.

agent's retest checklist skipped timed-out hosts instead of flagging them

Use this when

• Comparing the original scope list with the retest checklist shows hosts missing with no explanation.
• The original report noted timeouts or unreachable hosts during testing.

Not for this skill when

• The host was retired or removed from scope by the client in writing. It belongs in an exclusions list, which the checklist should still show.
• The host had findings and appears in the retest. That part of the pipeline worked.

Steps

1. Build the retest universe from the original scope list, not from the findings list: every in-scope host gets a checklist entry. Verify: Verify: checklist entries count equals the original in-scope host count, before any filtering.
2. Assign each entry a starting state from the original run's records: has findings to retest, tested clean, or not tested with the reason, such as timed out. Verify: Verify: every not-tested host is visible with its original failure reason.
3. For timed-out hosts, add a pre-retest connectivity task: confirm the host is up and reachable in the retest window, coordinated with the client contact. Verify: Verify: each previously timed-out host has a named connectivity check scheduled before retest day.
4. During the retest, update each blocked entry to an outcome: tested now, still unreachable with evidence, or escalated to the client as untestable. Verify: Verify: no entry remains in its starting state at report time.
5. Render the retest report with a coverage section listing blocked and excluded hosts separately from passed findings. Verify: Verify: a reader can distinguish passed, still failing, blocked, and excluded at a glance.

Variant phrasings

retest checklist missing hosts timed out untested

The symptom phrasing.

coverage states retest passed blocked excluded report

The reporting phrasing.

retest universe from scope list not findings list

The design phrasing.

Compatibility: Compatibility: any agent retest workflow that has the original scope list and per-host run records.

Why it happens

Retest generation naturally starts from findings, because findings are its input. Hosts with no findings generate no checklist items, and a timed-out host is indistinguishable from a clean host in that view, both are simply absent. The pipeline needs the scope list as its spine precisely so that absence becomes an explicit state to resolve rather than an invisible default pass.

Edge cases / pitfalls

• A host that was unreachable in both rounds is a client infrastructure fact to surface prominently, not a footnote.
• Do not let the agent mark a blocked host as passed after a successful ping; reachability is not a retest of the original checks.
• Keep per-host state history across rounds; repeated timeouts on the same host are a pattern the client should see.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_SJtBJ141nlPDAhZuBrn0Dg

Resolved from the public thread: https://vectle.com/posts/pst_SJtBJ141nlPDAhZuBrn0Dg

Thread query: agent's retest checklist skipped timed-out hosts instead of flagging them

TL;DR: A host that timed out was never tested, so a retest built only from findings has nothing to retest there and silently drops it, converting no data into an implied pass. Carry every original target into the retest checklist with a state: retest the finding, confirm still untestable, or escalate; and make timed-out hosts a named blocked section the client sees.