agent's daily scheduled run failed to archive outputs, data missing

TL;DR

A scheduled run that executes but does not archive its outputs produces results nobody can find later: the output notebook is overwritten next run or lives on an ephemeral disk. Add archiving to the run itself: after execution, copy the output notebook (and key artifacts) to durable storage with a timestamped name, and verify the copy. A run without an archived output did not happen, as far as anyone auditing later is concerned.

agent's daily scheduled run failed to archive outputs, data missing

Use this when

• Scheduled runs execute but their outputs cannot be found later.
• The output path is ephemeral or overwritten each run.
• Audits or humans ask 'what did yesterday's run produce?'

Not for this skill when

• The run itself fails. Fix the run first.
• Outputs exist but are wrong. That is a logic bug.

Steps

1. Define the archive destination (object storage, shared dir) with timestamped names. Verify: the destination is writable from the run context.
2. Add the archive step to the run after successful execution. Verify: outputs land in the archive.
3. Verify the copy (size check or checksum). Verify: the archived file is complete.
4. Add a post-run check: the agent confirms the archive exists and alerts if not. Verify: a failed archive pages, not silently passes.
5. Backfill policy: decide whether missing history matters and backfill if so. Verify: the decision is recorded.

Variant phrasings

scheduled run outputs not archived

Short form. Archive is part of done.

papermill output missing daily run

The stack phrasing. Timestamp and store.

agent run no output saved

The agent phrasing. Verify the copy.

Compatibility: Any scheduler and storage. The done-definition is process, not tooling.

Why it happens

Execution feels like the job, so archiving gets treated as optional cleanup and dropped under time pressure. But a scheduled run's value is its recorded output: without the archive, the compute was spent and nothing durable exists. Making archiving part of the run's success criteria (with verification and alerting) is what turns runs into records.

Edge cases / pitfalls

• Overwriting a single 'latest' file destroys history. Timestamp every archive.
• The archive step needs the same retry discipline as the run. A flaky copy loses the output.
• Archive the parameters and environment info with the output. An output without its inputs is undebuggable.
• Retention policy matters: decide how long archives live before storage bills decide for you.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_sKBTco4EzHcePOhmg0A4Gw