agent failed the restore check after its ssh session dropped mid-run

TL;DR

An SSH session is a fragile thread to hang a long restore check on: networks reset, laptops sleep, and servers reap idle sessions. Move the check into a persistent terminal session or a remote job the agent starts and polls, make every step safe to re-run, and have the agent reconnect and read the remote state instead of assuming the check died with the connection.

agent failed the restore check after its ssh session dropped mid-run

Use this when

• Restore checks fail when the SSH connection drops but succeed on stable networks.
• The remote restore may have completed while the agent reported failure.
• Long checks fail more often than short ones on the same host.

Not for this skill when

• SSH login is refused from the start. Authentication or firewall setup.
• The remote host itself rebooted mid-check. Host event, different handling.

Steps

1. Review how the agent runs remote steps today and whether any remote state survives a disconnect. Verify: the current fragility is mapped
2. Wrap remote checks in a persistent session tool or a nohup-style job writing progress and results to files on the remote host. Verify: a deliberate disconnect during a test leaves the remote job running
3. Make each check step idempotent so re-running after reconnect cannot corrupt the test. Verify: re-running a completed step reports already done safely
4. Implement agent reconnect logic that reads the remote progress file and resumes reporting. Verify: after a forced drop, the agent recovers the true outcome
5. Run a full restore check with a forced mid-run disconnect. Verify: the check completes and the agent reports the real result

Variant phrasings

agent ssh disconnect long job tmux resume

The persistent-session phrasing.

restore check lost connection ssh session dropped

The symptom variant.

agent remote job survive disconnect state file

The state-file wording.

Compatibility: Agents driving Linux and network device checks over SSH with tmux, screen, or remote job files, OpenSSH clients and servers.

Why it happens

SSH is a transport, not a job manager: when it dies, attached processes get hangup signals and orchestrators lose both control and knowledge. Long verification jobs need the remote side to own execution and state, with the agent as a reconnecting observer. That inversion also fixes laptops closing and VPN drops, not just network resets.

Edge cases / pitfalls

• Server-side session timeouts kill idle-looking sessions during long silent commands. Persistent sessions avoid the idle classification.
• Polling a progress file too aggressively wastes the connection you are protecting. Poll at human timescales.
• Always record the remote job identifier in the agent run log. It is the only handle after a drop.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_DRwds4AzL5X654m-nyscqA