# agent's eBPF profiler dropped events under load and reported no hotspots

## TL;DR

eBPF's perf buffers are finite; under load the kernel drops events it cannot buffer, and the profiler silently reports a calm system. Check the lost-event counters: if they are nonzero, the profile is incomplete. Raise buffer sizes, lower the sampling rate, or filter in-kernel so the buffer keeps up.

```text
agent's eBPF profiler dropped events under load and reported no hotspots
```

## Steps

1. Check for loss: most eBPF profilers expose dropped/lost event counters; read them for the capture window.
   Expected: Nonzero lost events confirm the profile is incomplete.

2. Reduce the event rate: lower the sampling frequency or filter to the processes of interest in-kernel.
   Expected: Fewer events fit the buffer.

3. Raise the perf buffer size (per-CPU buffers) to absorb bursts.
   Expected: Loss counters drop to zero.

4. Re-capture under the same load and verify zero loss before trusting the hotspots.
   Expected: The profile now shows the real hot paths.

5. Make loss-checking part of the capture workflow: refuse to analyze a profile with nonzero drops.
   Expected: Incomplete profiles never become decisions.

## When to use

- an eBPF profile shows no hotspots on a busy system
- lost-event counters are nonzero
- profiling high-throughput workloads with eBPF

## When not to use

- loss counters are zero (the quiet profile is honest)
- the workload is genuinely idle (no events is correct)
- using a profiler that backpressures instead of dropping (check the tool's design)

## Tool compatibility

- eBPF-based profilers
- Linux

## Variant phrasings

- "ebpf profiler dropped events"
- "ebpf lost events no hotspots"
- "ebpf perf buffer overflow profile"

## Why it happens

eBPF programs push events into per-CPU perf buffers that user space drains. When events arrive faster than user space drains them, the kernel drops the overflow and increments a counter. The profile then under-represents exactly the busiest moments, which are the ones you needed to see.

## Edge cases

- raising buffer sizes costs memory per CPU; size it against the event rate, not hope
- in-kernel filtering (by PID, by cgroup) is cheaper than bigger buffers
- lost counters are per-buffer; check all CPUs, not just the total

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_2ras0YfMT0HBPnWwo056Og