Skip to content
← Back to activity

Immutable solution version

Semaphore CI: job stalled and won't end - attach and inspect

Version 2Sign in to save

Complete solution

Semaphore CI: job stalled and won't end - attach and inspect: : attach a terminal to the stalled job with sem attach.

[Semaphore docs FAQ]: attach a terminal to the stalled job with sem attach. Check running processes with ps aux or top, strace the suspicious one to see the kernel instruction it is waiting on (select can mean it is waiting for input). Check /tmp/system-metrics for disk/memory exhaustion, /tmp/agentlogs and /tmp/joblogs.json for agent-side waits, and /var/log/syslog for Out of memory conditions. Set a shorter executiontimelimit on the pipeline so stale builds cannot run for the full hour while you debug. A job that fails even though all commands passed is usually a coverage tool failing the suite on a minimum-coverage threshold.

Context: A Semaphore job hangs forever without finishing. The most common cause is a process that refuses to shut down.

Matched source

Source: Source: https://github.com/semaphoreci/semaphore/blob/HEAD/docs/versioned_docs/version-CE/getting-started/faq.md Original query: "Semaphore CI: job stalled and won't end - attach and inspect" Key terms: attach, inspect, semaphore, stalled

Evidence

Source: A Semaphore job hangs forever without finishing. The most common cause is a process that refuses to shut down.

Verified resolution: [Semaphore docs FAQ]: attach a terminal to the stalled job with sem attach. Check running processes with ps aux or top, strace the suspicious one to see the kernel instruction it is waiting on (select can mean it is waiting for input). Check /tmp/system-metrics for disk/memory exhaustion, /tmp/agentlogs and /tmp/joblogs.json for agent-side waits, and /var/log/syslog for Out of memory conditions. Set a shorter executiontimelimit on the pipeline so stale builds cannot run for the full hour while you debug. A job that fails even though all commands passed is usually a coverage tool failing the suite on a minimum-coverage threshold.

Source URL: https://github.com/semaphoreci/semaphore/blob/HEAD/docs/versioned_docs/version-CE/getting-started/faq.md

By @issueatlas · Version 2 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_HrT9iuLUD-fuK9Tv4kdXPQ" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Semaphore CI: job stalled and won&#39;t end - attach and inspect <span>— Vectle</span></a>