test artifacts retention policy: what to keep

TL;DR

Keep artifacts from failing runs much longer than from passing runs, and keep flakiness history the longest of all. Classify artifacts into videos, screenshots, traces, reports, and logs, give each class its own retention window, and compress or downsample the bulky ones. A policy nobody enforces is just a wish, so put the deletion in the pipeline.

The query

test artifacts retention policy: what to keep

Use this when

• Artifact storage costs or quota keep growing
• Debugging an old failure and the video is already gone
• Compliance or security asks how long test data is kept

Not for

• Application logs in production
• Build artifact retention (compiled outputs are separate)
• Backup policy for source code

Steps

1. Inventory what you store. List every artifact type the pipeline uploads: videos, screenshots, traces, HTML reports, raw logs, coverage files.

Expected output: a written inventory with current sizes per type.

1. Set retention by outcome. A common starting point: failing-run artifacts 30 to 90 days, passing-run artifacts 7 to 14 days, quarantine and flake history 90 days.

Expected output: a table mapping artifact type and run outcome to a retention window.

1. Compress the bulky ones. Videos and traces compress well; downsample videos from passing runs or drop them entirely.

Expected output: a storage drop of at least half on the next billing cycle, with failing-run videos untouched.

1. Automate deletion. Add a scheduled job that deletes artifacts older than their window. Manual cleanup never happens.

Expected output: a scheduled cleanup job with logs showing deletions per run.

1. Document the policy where the team can find it. Link it from the CI docs and the on-call runbook.

Expected output: a policy page linked from the pipeline README, with an owner and a review date.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_hNY6PTc4k4kDB6JqyOWVaw