kubernetes error: ndots:5 causing slow DNS lookups - how to fix

kubernetes error: ndots:5 causing slow DNS lookups - how to fix

TL;DR

Every pod's resolv.conf sets ndots:5, so any name with fewer than 5 dots gets tried against every search domain first. A single lookup can become a dozen queries. Fix it: set ndots lower, use fully-qualified names with a trailing dot, or add stub domains for the names you actually use.

kubernetes: DNS lookups slow or timing out (ndots:5 search-domain expansion)

Steps

1. Confirm the symptom: run a DNS lookup inside the pod with timing (dig or getent) and watch it take seconds for short names. Expected: Short names are slow, fully-qualified names are fast.

2. Check resolv.conf in the pod: you will see ndots:5 and several search domains. Expected: The expansion behavior is confirmed.

3. Fix per workload: set dnsConfig with ndots: 2 (or 1) in the pod spec for apps that use short names. Expected: Lookups stop expanding through every search domain.

4. Alternatively, use fully-qualified names with a trailing dot (myservice.myns.svc.cluster.local.) in the app config. Expected: The trailing dot skips search expansion entirely.

5. For external names you query often, add stubDomains or upstreamNameservers in the cluster DNS config. Expected: Those names resolve without the search-domain walk.

When to use

• DNS lookups inside pods are slow
• short hostnames take seconds, FQDNs are fast
• you see DNS timeouts under load

When not to use

• DNS is fast but the app cannot resolve at all (that is CoreDNS health, not ndots)
• the slowness is outside the cluster (check the upstream resolver)
• you already use trailing-dot FQDNs everywhere (ndots is not your problem)

Tool compatibility

• Kubernetes 1.24+
• CoreDNS

Variant phrasings

• "kubernetes ndots 5 slow dns"
• "ndots:5 dns lookup timeout pods"
• "kubernetes dns slow short names"

Why it happens

ndots:5 means any name with fewer than 5 dots is first tried with each search domain appended: name.default.svc.cluster.local, name.svc.cluster.local, and so on. A name like 'database' becomes 4-6 sequential queries before the bare name is ever tried, and each NXDOMAIN costs a round trip.

Edge cases

• lowering ndots changes resolution for every name the app uses; test short-name behavior after the change
• some apps cache DNS aggressively and will not pick up the fix until restarted
• CoreDNS autopath can mitigate this for cluster names; it does not help external names

Provenance

Resolved from the public thread: https://vectle.com/posts/pstHPeJHAO0GJ0O1dNXWuueQ