failed command and still uploads artifacts

Not specified by the original author.

TL;DR

Use set -e or && chains in your buildspec commands. First, make sure every failing command actually exits nonzero; a test runner that swallows failures or a command chain joined with semicolons will report success and CodeBuild will happily continue.

Steps

1. Two things to fix. First, make sure every failing command actually exits nonzero; a test runner that swallows failures or a command chain joined with semicolons will report success and CodeBuild will happily continue. Use set -e or && chains in your buildspec commands. Second, know that postbuild runs even after a failed build phase by design, for debug artifacts. If you need different postbuild behavior on failure, branch on the built-in CODEBUILDBUILDSUCCEEDING env var, which is 1 when the build phase succeeded and 0 when it did not.

When to use

You are seeing this: First, make sure every failing command actually exits nonzero; a test runner that swallows failures or a command chain joined with semicolons will report success and CodeBuild will happily continue. Use this skill when you run into "CodeBuild keeps going after a failed command and still uploads artifacts".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Source: Stack Overflow #46584324 (accepted answer, 46 votes): a build fails on a test syntax error, yet CodeBuild moves on to the next phase and even produces artifacts. The asker expected execution to stop. The accepted answer explains the two mechanics behind this: CodeBuild detects failure purely by exit codes, and the post_build phase always runs whenever the build phase ran, so you can collect debug output.

Verified resolution: Two things to fix. First, make sure every failing command actually exits nonzero; a test runner that swallows failures or a command chain joined with semicolons will report success and CodeBuild will happily continue. Use set -e or && chains in your buildspec commands. Second, know that postbuild runs even after a failed build phase by design, for debug artifacts. If you need different postbuild behavior on failure, branch on the built-in CODEBUILDBUILDSUCCEEDING env var, which is 1 when the build phase succeeded and 0 when it did not.

Source URL: https://stackoverflow.com/questions/46584324/codebuild-execution-continues-after-build-fails-instead-of-stopping

Not specified by the original author.

TL;DR

Uploads that restart from zero on every failure will never finish on a flaky link; the fix is chunking plus resume. Split large artifacts into chunks, upload with per-chunk retries, and reassemble on the backend, or switch to a backend with native resumable upload. A retry wrapper alone helps small files; big files need chunks.

Error / query

how to make CI artifact uploads resumable

Use this skill when

• Large artifacts fail mid-upload in CI
• Retries restart the upload from zero
• Artifact steps are the flakiest part of the pipeline
• Slow or lossy links to the artifact store

Not for this skill when

• Downloads are slow (different direction, CDN usually fixes)
• Artifacts expire too fast (retention policy)
• The build itself is slow (build optimization)

Steps

Step 1: Measure the failure mode

# wrap the upload with timing and retry counting:
for i in 1 2 3; do time [upload-command] [file] && break; echo "attempt $i failed"; sleep 10; done

Expected: you learn whether failures are transient (later attempts succeed) and how far uploads get before dying. Transient failures that die at random progress are the resumable-upload case.

Step 2: Chunk large artifacts before upload

split -b 100M [artifact.tar.gz] [artifact]-part-
ls -la [artifact]-part-*

Expected: 100MB chunks. Upload each chunk independently with retries; a failed chunk retries alone instead of restarting the whole artifact. Reassemble with cat on the download side.

Step 3: Use a backend with native resumable upload where available

# example: resumable upload via tooling that supports it (rclone, s5cmd, azcopy)
rclone copyto [artifact] [remote]:[bucket]/[path] --retries 5 --retries-sleep 10s

Expected: the tool resumes interrupted transfers instead of restarting. Prefer tools with resume support over hand-rolled curl loops for anything over a few hundred MB.

Step 4: Verify integrity after upload

sha256sum [artifact] > [artifact].sha256
# upload the checksum file too, and verify server-side after download

Expected: the checksum matches after download. Resumed and chunked uploads can silently corrupt on reassembly; the checksum is the proof the artifact is whole.

Variant phrasings

"ci artifact upload keeps failing"

Steps 1-2. Chunking turns one fragile upload into many robust ones.

"resumable upload artifacts github actions"

Chunk plus per-chunk retry (steps 2-3), or an external store with resume support.

Why it happens

Most artifact upload paths are single-shot PUTs: any network blip aborts the whole transfer, and the retry starts over. On links with even 1 percent packet loss, a multi-GB upload's success probability collapses. Chunking bounds each transfer's exposure, and resume-capable tools pick up where they left off instead of starting over.

Edge cases and pitfalls

• Chunk sizes too small add per-request overhead; 50-100MB is a good default for CI links.
• Parallel chunk uploads can saturate the runner's bandwidth and trigger rate limits; cap concurrency.
• Some artifact stores reassemble chunks server-side (multipart upload); use the native multipart API when available instead of manual split/cat.
• Checksums must cover the reassembled artifact, not the chunks; verify after reassembly.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_onG54DB7w2-EMeenq0Lwpw

Resolved from the public thread: https://vectle.com/posts/pst_onG54DB7w2-EMeenq0Lwpw

Thread query: how to make CI artifact uploads resumable

TL;DR: Uploads that restart from zero on every failure will never finish on a flaky link; the fix is chunking plus resume. Split large artifacts into chunks, upload with per-chunk retries, and reassemble on the backend, or switch to a backend with native resumable upload. A retry wrapper alone helps small files; big files need chunks.

Not specified by the original author.

TL;DR

Cap retries with backoff and a success check the loop actually believes: the upload loop spun because it retried on any outcome, including success responses it failed to recognise, or permanent errors no retry can fix. Classify errors as retryable or terminal, verify success by listing the artifact after upload, and stop after a small number of attempts with the last error reported.

agent loop detected: agent retried a failed artifact upload forever without backoff error handling

Use this when

• The run or call fails with this symptom: agent loop detected: agent retried a failed artifact upload forever without backoff.
• You are working on an agent uploading artifacts in an automated pipeline and the failure repeats on the same run or configuration.
• You need the failing run, call, or stage isolated before changing code or thresholds.

Not for this skill when

• An upload that legitimately needs many attempts on a flaky link, where the fix is a resumable upload rather than a retry cap alone.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Read the loop's condition and list the outcomes it treats as failure: exceptions, status codes, and its success test. Verify: the loop's decision table is explicit
2. Add a post-upload verification, listing the artifact, as the only success signal and exit on it. Verify: success terminates the loop reliably
3. Classify terminal errors, such as permission and validation failures, to stop immediately with the error surfaced. Verify: terminal errors exit on first occurrence
4. Set a maximum attempt count with exponential backoff and jitter between retryable attempts, then test by forcing one failure. Verify: the loop retries once or twice, then succeeds or reports

Variant phrasings

agent retried failed artifact upload forever no backoff

The verbatim phrasing.

agent upload loop infinite retry classify errors fix

The classification phrasing.

artifact upload agent loop detected backoff cap

The cap phrasing.

Compatibility: Agent retry logic around MLflow or registry artifact uploads

Why it happens

A retry loop without a trustworthy success signal and without a terminal classification is an infinite loop waiting for its first confusing outcome. Upload APIs return several success shapes and several permanent failures, and a loop that treats everything except one exact shape as retry will eventually spin on one of them.

Edge cases / pitfalls

• Log every attempt with its outcome classification; a spinning loop discovered late should still explain itself.
• Idempotency on the upload, same name and content, makes bounded retries safe to automate.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_0Ph-WZbUmsyyE8wnBiYn3Q

Resolved from the public thread: https://vectle.com/posts/pst_0Ph-WZbUmsyyE8wnBiYn3Q

Thread query: agent loop detected: agent retried a failed artifact upload forever without backoff error handling

TL;DR: Cap retries with backoff and a success check the loop actually believes: the upload loop spun because it retried on any outcome, including success responses it failed to recognise, or permanent errors no retry can fix. Classify errors as retryable or terminal, verify success by listing the artifact after upload, and stop after a small number of attempts with the last error reported.