ndots:5 DNS delays in Kubernetes pods explained

TL;DR

Every short hostname a pod resolves triggers up to 5 wasted DNS queries because of the default ndots:5 in pod resolv.conf: names with fewer than 5 dots get tried against every search domain first. Fix it by setting ndots:1 in the pod's dnsConfig for apps that query external names, or use fully-qualified names with a trailing dot. This is the most common cause of mysterious 5-second DNS stalls in pods.

Error / query

ndots:5 DNS delays in Kubernetes pods explained

Use this skill when

• DNS lookups from pods take seconds, especially for external names
• Apps making many short DNS queries are slow overall
• You see bursts of NXDOMAIN queries in CoreDNS logs
• Latency disappears when you query fully-qualified names

Not for this skill when

• CoreDNS pods are down or crashing (fix CoreDNS)
• Cluster service names do not resolve at all (service DNS problem)
• DNS is blocked by network policy (different fix)

Steps

Step 1: Confirm ndots:5 is in effect

kubectl exec [pod-name] -n [namespace] -- cat /etc/resolv.conf

Expected: options ndots:5 plus search domains like default.svc.cluster.local svc.cluster.local cluster.local. This is the default every pod gets.

Step 2: Reproduce the extra queries

kubectl exec [pod-name] -n [namespace] -- nslookup [external-host] 2>&1 | head -20

Expected: resolution works but slowly. In CoreDNS logs you will see the name tried as [external-host].default.svc.cluster.local, then .svc.cluster.local, then .cluster.local, each NXDOMAIN, before the real query. That is 3+ wasted round trips per lookup.

Step 3: Set ndots lower for external-heavy workloads

spec:
  dnsConfig:
    options:
      - name: ndots
        value: "1"

Expected: with ndots:1, any name containing a dot skips the search-domain expansion and resolves directly. Pod recreates with the new setting; external lookups drop to a single query.

Step 4: Verify the improvement

time kubectl exec [pod-name] -n [namespace] -- getent hosts [external-host]

Expected: sub-second resolution. Compare against the pre-change timing; the classic symptom is a drop from ~5s (search timeouts) to milliseconds.

Variant phrasings

"kubernetes dns slow external"

Almost always ndots:5. Steps 1-3 are the diagnosis and fix.

"coredns nxdomain search domains"

The NXDOMAIN storm from step 2. Lower ndots or use FQDNs with trailing dots.

Why it happens

ndots:5 means the resolver treats any name with fewer than 5 dots as possibly-relative and tries every search domain first. Kubernetes injects 3+ search domains into every pod, so api.example.com (2 dots) gets 3 doomed queries before the real one. It is a glibc default that made sense for corporate networks and is wrong for pods that mostly query the internet.

Edge cases and pitfalls

• ndots:1 can break short service names if the app relies on search expansion; test internal names after changing.
• Apps that cache DNS (JVM, Go resolver) may need a restart to pick up the new resolv.conf.
• Using FQDNs with a trailing dot (db.example.com.) bypasses search domains per-query without changing pod config; good for targeted fixes.
• Some base images ship musl (Alpine) which handles ndots differently; verify behavior on your actual image.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_RYCq-k9QAg4VM2WfWrS7qQ