Serverless job stuck in IN_PROGRESS after calling progress_update

What was going on Calling runpod.serverless.progress_update(job, ...) near the end of a handler left the job stuck in IN_PROGRESS even after the handler returned. The reporter confirmed it was a race condition by adding a one-second sleep before the return, which let the async progress update finish first. Maintainer justinmerrell dug in, deployed a production-side fix, and confirmed that once jobs are marked done they can no longer revert to in-progress status.

What fixed it If your job stays INPROGRESS after the handler returns, the async `progressupdate` call is racing with the completion signal. The platform-side fix is deployed, so jobs marked done now stay done, but if you are on an older flow the one-second sleep before returning from your handler is the confirmed workaround. Keep your progress updates away from the very end of the handler where possible. The maintainer confirmed the production fix after reproducing the race.

In the thread, the question was: Calling runpod.serverless.progress_update(job, ...) near the end of a handler left the job stuck in INPROGRESS even after the handler returned. The reporter confirmed it was a race condition by adding a one-second sleep before the return, which let the async progress update finish first. Maintainer justinmerrell dug in, deployed a production-side fix, and confirmed that once jobs are marked done they can no longer revert to in-progress status. The verified resolution: If your job stays INPROGRESS after the handler returns, the async progress_update call is racing with the completion signal. The platform-side fix is deployed, so jobs marked done now stay done, but if you are on an older flow the one-second sleep before returning from your handler is the confirmed workaround. Keep your progress updates away from the very end of the handler where possible. The maintainer confirmed the production fix after reproducing the race. Thread: https://vectle.com/threads/thruTovVcVVTA8fLGru5u9mg