# kafka error: consumer lag growing on the group - how to find the slow consumer

## TL;DR

Lag grows when consumption is slower than production. Find which partitions lag (kafka-consumer-groups --describe), then find which consumer instance owns them and why it is slow: processing time per record, downstream calls, or too few instances for the partition count.

```text
kafka consumer lag growing on the group (LAG column climbing in consumer-group describe)
```

## Steps

1. Describe the group: kafka-consumer-groups --bootstrap-server [host] --describe --group [group].
   Expected: The output shows per-partition CURRENT-OFFSET, LOG-END-OFFSET, and LAG.

2. Identify the lagging partitions and which consumer instance owns them (CONSUMER-ID column).
   Expected: One or two instances own all the lagging partitions.

3. Check that instance's processing: per-record processing time, downstream latency, and error rates. A slow downstream call per record is the usual cause.
   Expected: You see the bottleneck in the consumer's own metrics.

4. Check the partition-to-instance ratio: if partitions far outnumber instances, add instances up to the partition count (more instances than partitions adds nothing).
   Expected: Lag drains after scaling.

5. If processing is inherently slow, reduce per-record work: batch downstream calls, add caching, or move heavy work to a follow-up queue.
   Expected: Per-record time drops and the consumer keeps up.

## When to use

- consumer group lag grows over time
- you need to name the slow consumer instance
- lag is concentrated on some partitions

## When not to use

- lag spikes then drains (that is a burst, not a slow consumer)
- all partitions lag equally and production spiked (the producer outran a healthy consumer)
- the consumer is not running at all (lag grows because nothing consumes)

## Tool compatibility

- Kafka 2.x/3.x
- kafka-consumer-groups.sh

## Variant phrasings

- "kafka consumer lag growing find slow consumer"
- "kafka consumer group lag increasing"
- "kafka per partition lag describe"

## Why it happens

Lag is the distance between the newest produced offset and the consumer's committed offset. It grows whenever the consumer's sustained throughput is below the produce rate. Per-partition lag pinpoints where, the consumer-id mapping pinpoints who, and the instance's own metrics explain why.

## Edge cases

- rebalances pause consumption and spike lag; distinguish rebalance storms from genuine slowness
- a poison-pill record that always fails and retries looks like a slow consumer; check the error rate per partition
- lag on __consumer_offsets itself can make the describe output stale; that is meta-lag, not consumer lag

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_gz-bZJ_JhIlOuO9GOGURFg