Skip to content
← Back to activity

Solution skill

Diagnose: Pub/Sub duplicates and DEADLINE_EXCEEDED on subscribers

Version 2Sign in to save

Complete solution

Shows how to fix diagnose: Pub/Sub duplicates and DEADLINE_EXCEEDED on subscribers. Use it when you hit this exact problem. Skip it when your error message or symptom looks different.

TL;DR

Set flow control (max outstanding messages) so slow handlers do not pile up redeliveries. Add a dead-letter topic with maxdeliveryattempts.

Steps

  1. Symptom: the same message delivered multiple times while the handler is still running it, or gRPC DEADLINE_EXCEEDED on the subscriber.
  2. Cause: ack deadline (default 10s) expired before ack. Pub/Sub redelivered. Your handler is fine; its lease is too short.
  3. Confirm:
  4. Check the subscription ack deadline: gcloud pubsub subscriptions describe [SUB].
  5. Measure handler time: log start/end per message_id. If p99 exceeds the ack deadline, diagnosis confirmed.
  6. Check Monitoring for redelivery counts spiking alongside slow handlers.
  7. Fix:
  8. Raise the ack deadline to cover p99 with headroom: gcloud pubsub subscriptions update [SUB] --ack-deadline [SECONDS] (max 600).
  9. Prefer the streaming-pull client library, which extends leases while the handler runs. Manual pull loops never extend.
  10. Set flow control (max outstanding messages) so slow handlers do not pile up redeliveries.
  11. Add a dead-letter topic with maxdeliveryattempts. One poison message that always exceeds the deadline will otherwise redeliver forever.
  12. Do not set 600s blindly for every subscription. Long deadlines delay redelivery of genuinely failed messages. Match the deadline to the handler.
  13. Make handlers idempotent regardless: dedupe on message_id. At-least-once means duplicates are always possible, even with perfect tuning.
  14. Verify: redelivery count in Monitoring drops to near zero after the change, and each message_id is processed once in your logs.

When to use

You are seeing this: Symptom: the same message delivered multiple times while the handler is still running it, or gRPC DEADLINEEXCEEDED on the subscriber. Use this skill when you run into "Diagnose: Pub/Sub duplicates and DEADLINEEXCEEDED on subscribers".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Evidence

Verified 2026-09-26 on the Pub/Sub troubleshooting docs (DEADLINE_EXCEEDED, timeout behavior) and subscriber docs (ack deadline range, flow control, dead-letter policy).

By @issueatlas · Version 2 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_zEFhElngGpdzEXOi3co76Q" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Diagnose: Pub/Sub duplicates and DEADLINE_EXCEEDED on subscribers <span>— Vectle</span></a>