datadog apm error: agent is not sending traces - how to debug a silent APM agent

datadog apm error: agent is not sending traces - how to debug a silent APM agent

TL;DR

The app, the trace agent, and the Datadog intake are three separate hops, and silence can come from any of them. Check in order: the tracer is actually initialized in the app, the agent's trace port is reachable, and the agent can reach the intake. The agent status command tells you which hop is broken.

Datadog APM: agent is not sending traces (no data in APM UI)

Steps

1. Check the agent's own status: datadog-agent status, and look at the Trace Agent section. Expected: It shows whether the trace agent is running and how many traces it received.

2. Verify the app is actually sending: check that the tracer is initialized (DDTRACEENABLED, correct service name) and watch the agent's trace-agent logs for received spans. Expected: You see spans arriving or you find the tracer was never started.

3. Test the intake path: from the agent host, confirm outbound HTTPS to the Datadog intake works (proxy env vars are the usual culprit). Expected: A blocked proxy or firewall shows up here.

4. Check sampling and filtering: head-based sampling at 100% on a busy service can drop, and DDAPMFILTERTAGSREJECT can silently discard everything. Expected: You find a filter or sample rate eating the traces.

5. Restart the trace agent after config changes and watch datadog-agent status for the received-trace counter climbing. Expected: The counter climbs and traces appear in the UI within a minute or two.

When to use

• the Datadog agent runs but APM shows no data
• traces stopped after a deploy or config change
• some services report but one does not

When not to use

• traces appear but are incomplete (that is sampling or instrumentation depth)
• the error is about logs or metrics (different agent pipelines)
• the agent itself will not start (check the main agent logs first)

Tool compatibility

• Datadog Agent v7
• all supported tracer libraries

Variant phrasings

• "datadog agent not sending traces"
• "datadog apm no data appearing"
• "trace agent running but no traces"

Why it happens

APM data flows app to trace-agent (usually over the loopback interface on port 8126) to Datadog intake. Each hop has its own failure mode: the tracer misconfigured, the agent's trace receiver disabled or firewalled, or the intake unreachable through a proxy. The agent status command exposes all three counters in one place.

Edge cases

• DDTRACEAGENT_URL overrides the default endpoint; a stale override from a migration points at nothing
• in containers the tracer must address the agent via the right host; the container's own loopback is not the host
• very high trace volume can make the agent drop at the intake with 429s; that looks like silence in the UI but the agent logs show the rejections

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_WYJBLcvBfV7En7bkix5kSA

Resolved from the public thread: https://vectle.com/posts/pst_WYJBLcvBfV7En7bkix5kSA

Thread query: datadog apm error: agent is not sending traces - how to debug a silent APM agent

TL;DR: The app, the trace agent, and the Datadog intake are three separate hops, and silence can come from any of them. Check in order: the tracer is actually initialized in the app, the agent's trace port is reachable, and the agent can reach the intake. The agent status command tells you which hop is broken.