buildkit scheduler error: return leaving incoming open

The problem

We've been seeing the following error mostly in CI, however, we've also seen it on a developer's machine as well. The issue reproduced on the developer machine several times and only went away after changing the base image of the target (seems sort of random, but perhaps the team can make more sense of it). On CI, the job often fails multiple times in a row if you just re-run it as is.

The fix

[alexcb (contributor)]: The intermittent hang on COPY plus wait-end was a buildkit race condition, fixed upstream and released in Earthly v0.7.21. Upgrade (Satellites auto-update overnight if enabled). If a buildkitd gets stuck in the bad state, restarting it with docker stop earthly-buildkitd recovers.

In gh:earthly/earthly#2957 (https://github.com/earthly/earthly/issues/2957), the reporter ran into this: We've been seeing the following error mostly in CI, however, we've also seen it on a developer's machine as well. The issue reproduced on the developer machine several times and only went away after changing the base image of the target (seems sort of random, but perhaps the team can make more sense of it). On CI, the job often fails multiple times in a row if you just re-run it as is. The verified resolution: [alexcb (contributor)]: The intermittent hang on COPY plus wait-end was a buildkit race condition, fixed upstream and released in Earthly v0.7.21. Upgrade (Satellites auto-update overnight if enabled). If a buildkitd gets stuck in the bad state, restarting it with docker stop earthly-buildkitd recovers.