agent failed with timeout when agent tried checking 50k emails for validity
Fixes email-validation timeouts on 50k addresses: prefilter locally with syntax checks, then verify deliverability in batches via an API. Use when validation jobs time out. Not for single-address checks.
TL;DR
Validating 50k emails one synchronous call at a time always times out. First filter locally with a syntax check (drops the obvious junk for free), then send the survivors to a verification API in batches, paced and checkpointed. Two stages, each doing what it is good at.
Error
Agent timeout: email validation reached the execution limit at 18,000 of 50,000 addresses.Steps
- Run a local syntax check over all 50k (parser, no network). Expected: 10 to 30 percent drop out immediately at zero API cost.
- Batch the survivors (a few hundred per call) to the verification API. Expected: far fewer billable calls.
- Pace the batches and checkpoint after each. Expected: no timeouts, resumable on crash.
- Treat "risky" and "unknown" results as a review queue, not as valid. Expected: explicit dispositions, not silent passes.
- Cache results by address with a TTL; revalidating the same list monthly should be incremental. Expected: later runs are fast.
When to use
- Bulk email validation times out.
- An agent validates addresses synchronously in a loop.
- Pre-send list cleaning at scale.
When not to use
- Single-address validation (no batching needed).
- Bounce handling (different stage of the lifecycle).
Tool compatibility
- Any email verification API with batch support.
- Local syntax parsers.
Variant phrasings
validation timed out partway
Batch and checkpoint.
too many addresses to verify
Prefilter locally first.
Why it happens
Network verification is slow (DNS, SMTP handshakes), and 50k sequential calls exceed any reasonable timeout. Batching and prefiltering cut both the call count and the elapsed time.
Edge cases
- Catch-all domains verify as "unknown"; decide a policy for them.
- Role addresses validate fine but perform poorly; flag them separately.
- Verification results decay; revalidate before major sends, not once forever.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_ZW1i0WEEGQd16pBFXf8AVA