## TL;DR

Pulling 100k records to fuzzy-match locally is slow, memory-hungry, and rate-limit bait. Instead, use HubSpot search with tight filters to pull small candidate sets per record, then fuzzy-match only within each set. The search API does the coarse filtering; your code does the fine matching.

## Error

```text
Agent failed: search requests throttled and the local match job ran out of memory at 62,000 records.
```

## Steps

1. For each source record, build a HubSpot search with strong filters (email domain, phone digits, name token). Expected: tens of candidates per record, not thousands.
2. Page the candidates (they will be few) and fuzzy-match locally within that set. Expected: fast, low-memory matching.
3. Cache candidate sets for repeated runs. Expected: reruns skip the search phase.
4. Pace the searches to stay under the search rate limits. Expected: no 429s.
5. Log match scores and the winning candidate for audit. Expected: every merge decision is explainable.

## When to use

- Fuzzy matching jobs fail on large record sets.
- An agent downloads everything to match locally.
- Search throttling kills the match run.

## When not to use

- Exact email or id matching (no fuzzy logic needed).
- Small sets where local matching is fine.

## Tool compatibility

- HubSpot CRM search API v3; local fuzzy matching libraries.
- Blocking keys to narrow candidates.

## Variant phrasings

### search throttled during matching

Narrow candidates per record and pace.

### out of memory fuzzy matching

Do the coarse filter server-side.

## Why it happens

Fuzzy matching is O(n*m) without blocking. At 100k records that is billions of comparisons, plus the API cost of fetching everything. Server-side search cuts the problem by orders of magnitude.

## Edge cases

- Overly tight filters miss true matches; validate recall on a labeled sample.
- Search indexes lag real-time writes by a bit; very fresh records may not appear.
- Score thresholds need tuning per field; names need looser thresholds than emails.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_wgSaCG_UG6Ewl73kWMukJw