## TL;DR
No space left is a filesystem fact, not a training bug: check free space and inode counts on the checkpoint filesystem, then clear the real consumer, which is usually old checkpoints kept by an unbounded keep-all setting, event files, or other jobs sharing the volume. Set a bounded checkpoint retention, such as keeping the last few plus the best, so the run cannot refill the disk.

```text
model checkpoint save failed with OSError error 'No space left on device' during training run
```

## Use this when
- The run or call fails with this symptom: OSError: No space left on device when saving a model checkpoint during training.
- You are working on a long training run saving checkpoints to a local or shared volume and the failure repeats on the same run or configuration.
- You need the failing run, call, or stage isolated before changing code or thresholds.

## Not for this skill when
- A quota on a shared filesystem that is lower than the volume's free space suggests, where the fix is the quota rather than cleanup.
- The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

## Steps
1. Measure free space and inode usage on the exact filesystem the checkpoint path resolves to. Verify: space and inode headroom are known
2. Find the largest consumers on that filesystem, starting with the run's own checkpoint history. Verify: the main consumer is identified
3. Delete or archive old checkpoints and set a bounded keep policy in the training configuration. Verify: retention is capped and space is reclaimed
4. Resume saving and confirm the next checkpoints write fully, checking sizes against earlier ones. Verify: checkpoints complete at expected sizes

## Variant phrasings
### no space left on device checkpoint save training
The verbatim phrasing.
### oserror no space left model checkpoint disk full fix
The disk-full phrasing.
### training run checkpoint retention filled disk
The retention phrasing.

Compatibility: Training checkpoint saving on local or shared filesystems

## Why it happens
Checkpointing writes gigabytes on a schedule, and retention defaults in many frameworks keep everything. The disk fills gradually, then a save fails mid-write, which can also corrupt the checkpoint being written, so the space problem and the restore problem arrive together.

## Edge cases / pitfalls
- Inode exhaustion mimics space exhaustion with free gigabytes reported; check both numbers.
- A failed mid-write checkpoint should be deleted, not retried against; it is partial by construction.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_yNB4j9IrDROAJLavwJt3Wg