GitLab CI pipeline stuck in pending: runner debugging

TL;DR

A pipeline stuck in pending means no runner claimed the job: check the job's tags against registered runners, verify a runner is online and not paused, and look at concurrency limits. Most of the time its a tag mismatch or every runner being busy or offline. Match the job to a live runner and it starts.

The error

This job is stuck because of one of the following reasons:
- There are no active runners online
- No active runners can run this job (tags)

Job shows a gray pending icon indefinitely, never goes to running.

Use this when

• jobs sit in pending and never start
• the stuck message mentions runners or tags
• a new runner was just registered
• pipelines worked and stopped with no config change

Not for

• jobs that start and fail (thats the script, not the runner)
• pipeline YAML syntax errors (the pipeline wont even create)
• GitLab.com side outages (check the status page)

Steps

1. Read the stuck reason on the job page, then check runner status:

Go to Settings, CI/CD, Runners in the project (or Admin, Runners for instance runners).

Expected: each runner shows online or offline, paused or active. An offline or paused runner explains everything.

1. Compare the job's tags with the runner's tags:

job:
  tags:
    - docker
    - linux

Expected: at least one online, active runner has ALL the tags the job asks for. Tags are AND, not OR: one missing tag means no match. The classic break is adding a tag to the job and forgetting the runner.

1. Check the runner isn't starved by concurrency:

On the runner host, check config.toml concurrent and limit, and look at what the runner is already doing:

gitlab-runner verify

Expected: verify passes and the runner has free slots. concurrent = 1 with a long job ahead of yours looks exactly like stuck.

1. For specific runners, check the host is healthy:

systemctl status gitlab-runner
journalctl -u gitlab-runner --since "30 min ago" | tail -20

Expected: service active, logs show it polling for jobs. Errors about tokens mean the runner was reset or deleted server-side; re-register it.

1. If using autoscaling runners (docker-machine or custom executor), check the scaler:

Expected: the scaler has capacity and cloud quota left. Pending with autoscale usually means the scaler cant create machines: quota exhausted, bad credentials, or the machine image gone.

1. Fix and watch the job pick up:

# re-register if needed
gitlab-runner register

Expected: the pending job starts within a polling interval (default seconds). If it stays pending, re-read the stuck message; it updates with the current blocker.

Variant: pending only for protected branches

The job needs a protected runner and none is marked protected, or the runner's protected flag doesnt match. Check runner settings, not tags.

Variant: worked yesterday, pending today, nothing changed

Runner host rebooted, disk filled, or the registration token was rotated. Check the host first; runners die quietly.

Variant: shared runners disabled on the project

Project settings, CI/CD, Runners: shared runners can be turned off per project. If your jobs expect shared runners and theyre off, everything pends.

Why it happens

GitLab matches each job to a runner by tags, protected status, and availability. Pending is the scheduler saying no runner qualifies right now. Its usually not GitLab being broken, its the match failing: wrong tags, paused runner, full concurrency, or dead autoscaler.

Edge cases

• Untagged jobs run on any runner that allows untagged jobs; a runner with run_untagged=false ignores them silently.
• Runner managers vs runners: on newer versions the runner record and the worker are separate; both must be healthy.
• Pipeline-level resource_group can serialize jobs and look like stuck when an earlier job holds the lock.
• After GitLab upgrades, old runner versions may be rejected; keep runners within a supported version of the server.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst3asJKgerqf5NbDF9CyNrg