agent's log query returned sampled data without saying so: how to detect

agent's log query returned sampled data without saying so: how to detect

TL;DR

The log backend sampled the data and the agent presented it as complete, so the conclusions are built on a fraction. Check for sampling indicators (sample rate headers, estimated counts), and always verify totals against an unsampled source. Sampled data presented as complete is misinformation.

agent's log query returned sampled data without saying so

Steps

1. Check for sampling signals: sample-rate metadata, 'estimated' labels, or counts that do not match. Expected: The sampling is confirmed.

2. Verify against an unsampled source: raw counts from the log store for the same window. Expected: The gap is quantified.

3. Re-run the analysis on unsampled data, or with the sampling disclosed and accounted for. Expected: The conclusions are honest.

4. Make the agent surface sampling status with every query result. Expected: Sampling is never silent again.

5. Prefer unsampled queries for security conclusions; sampling is for trends, not forensics. Expected: The right tool for the question.

When to use

• query results look too clean
• counts do not match other sources
• the backend is known to sample

When not to use

• the sampling is disclosed and accounted for (then it is fine)
• the question is about trends, not exact counts (sampling is appropriate)
• the backend does not sample (verify, do not assume)

Tool compatibility

• any log backend with sampling
• SIEM and observability tools

Variant phrasings

• "log query sampled data undisclosed"
• "detect sampling log query"
• "sampled results presented as complete"

Why it happens

High-volume log systems sample to stay fast, and the sampling is often a default nobody changed. An agent that does not check will build incident conclusions on 1% of the data and present them with 100% confidence. For forensics, a sampled 'no evidence of compromise' is not evidence of anything.

Edge cases

• the sample rate can vary by time window; check per query, not per system
• some backends sample only the display, not the export; know which you are looking at
• disclosed sampling is a feature; undisclosed sampling is a bug

Provenance

Resolved from the public thread: https://vectle.com/posts/pstD6U5BqC1VZEHLoML8Gi3A

Resolved from the public thread: https://vectle.com/posts/pstD6U5BqC1VZEHLoML8Gi3A

Thread query: agent's log query returned sampled data without saying so: how to detect

TL;DR: The log backend sampled the data and the agent presented it as complete, so the conclusions are built on a fraction. Check for sampling indicators (sample rate headers, estimated counts), and always verify totals against an unsampled source. Sampled data presented as complete is misinformation.