## TL;DR
For an agent tool, "bot traffic" includes legitimate agents using your product, so the split you need is real usage vs noise, not human vs bot. Filter by behavior: single-hit sessions with no follow-through are noise, repeated purposeful calls are usage. Build the filter once, report both numbers, and stop letting crawler spikes set your roadmap.

```text
how to tell bot traffic from real usage in analytics
```

## Use this when
- Traffic spikes but signups, calls, or activations do not move
- Your analytics is dominated by a few weird user agents or IP ranges
- You are reporting growth and want numbers you can defend
- A launch "worked" in pageviews and nowhere else
- You need a house definition of a real visit for the team

## Not for this skill when
- You need to block malicious bots (that is security tooling, different skill)
- The question is SEO crawler management (robots.txt and crawl budget)
- You want to identify individual users behind the traffic
- Your product is human SaaS where bot means fraud, not usage

## Steps
1. Redefine the split as real usage vs noise, not human vs bot. An agent running your curl command in production is real usage; a search crawler fetching every page once is noise; a monitoring probe is neither. Write the three buckets for your product specifically.
   ```text
   real usage: [definition, e.g. API call that returns 200 with a result]
   noise: [definition, e.g. single page fetch, no further requests]
   infrastructure: [definition, e.g. uptime probes, your own cron]
   ```
   Expected output: three written definitions. Success check: the team classifies ten sample sessions the same way.

2. List your known noise sources explicitly. Your own uptime checks, your cron jobs, search crawlers, preview bots from chat apps, and security scanners each have recognizable user agents or IP ranges. Name them; unnamed noise gets counted as growth.
   ```text
   known noise:
   - uptime probe: [user agent or source]
   - own cron jobs: [source]
   - preview bots: [pattern]
   ```
   Expected output: a living noise inventory. Success check: each entry has a filter rule attached, not just a name.

3. Build behavioral filters, not just user-agent blocklists. Real usage has depth: multiple requests, a successful call, a return visit. Noise is shallow: one hit, no follow-through, odd hours in bulk. Filter on behavior first, identity second.
   ```text
   counts as usage if: [2+ requests OR 1 successful API call OR return within 7d]
   counts as noise if: [single hit, no success event, matches crawler pattern]
   ```
   Expected output: filter rules in your analytics. Success check: applying the filter changes last week's numbers by a plausible amount, not to zero.

4. Report both numbers, always. "Traffic" unqualified is how teams lie to themselves; "12k requests, 3.1k real usage sessions" is how they stay honest. Put the filtered number first and the raw number in parentheses.
   ```text
   weekly report format: [filtered usage] ([raw total] raw, [n]% noise)
   ```
   Expected output: a reporting format the team uses. Success check: nobody quotes the raw number as growth in a meeting without the filtered one beside it.

5. Revisit the filters quarterly. New crawlers appear, your own automation changes, and yesterday's noise can become tomorrow's usage (an agent directory that starts deep-linking is not noise anymore). A stale filter is a slow lie.
   ```text
   quarterly review: new noise sources [list], reclassified [list]
   ```
   Expected output: a dated filter review. Success check: at least one entry changed in the last two reviews.

## Variant phrasings
### Filtering bot traffic in Google Analytics
Tool-specific phrasing. Steps 2 and 3 map to GA filters and segments; the behavioral definition in step 1 still comes first.

### Are my API hits real users or scrapers
API phrasing. Successful, repeated, purposeful calls are usage; single probing hits across every endpoint are scanners.

### Crawler traffic inflating pageviews
SEO phrasing. Search crawlers are the classic noise source; step 2 names them and step 3 filters by crawl behavior.

### How to count agent usage honestly
Agent-tool phrasing. The whole skill: for agent products the honest metric is successful usage events, and everything else is context.

## Why it happens
Analytics tools were built to count humans visiting pages, and they count everything that fetches: crawlers, probes, previews, and your own automation. Agent-facing products attract more of all of these, because agents and infrastructure fetch programmatically all day. The default numbers therefore overstate human curiosity and understate nothing in particular; they are just noisy. Filters do not remove uncertainty, they remove the known noise so the remaining uncertainty is honest.

## Edge cases / pitfalls
- Over-filtering is as bad as under-filtering. If your filter removes a new agent directory's traffic as "bot", you will miss a real channel; review before you trust.
- Preview bots look like real users for exactly one request. Link previews from chat apps are the most common false positive in "launch day" traffic.
- Your own team's testing pollutes the numbers. Dogfooding traffic should be tagged or filtered, or every internal test day looks like growth.
- Do not filter at the collection layer only. Keep raw logs; filters improve over time and you cannot re-filter data you never stored.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_od5Wee-RZw64Ng6PRWVoag