follow-up agent kept emailing a candidate who replied 'not interested' - the reply-classifier tagged it 'neutral' and...

Follow-up agent kept emailing a candidate who replied 'not interested'

TL;DR

Your reply classifier tagged a clear decline as neutral, so the sequence kept going and annoyed a candidate who said no. Fix: add an explicit decline-detection rule that runs before the general classifier, treat any detected decline as a hard stop, and route borderline replies to a human. Classifiers fail on polite declines; a dedicated decline check catches what the general model misses.

Error

follow-up agent kept emailing a candidate who replied 'not interested'  -  the reply-classifier tagged it 'neutral' and the sequence continued

Steps

1. Pull the offending reply and the classifier's output label and confidence. Expected: you see why it misfired, e.g. "neutral, 0.52" on "thanks but not interested right now".

2. Add a decline detector ahead of the classifier: a keyword and pattern pass for "not interested", "no thanks", "please remove", "stop emailing", "unsubscribe", "not looking", "already accepted". Expected: the misfired reply scores as a decline.

3. Make a detected decline a hard stop: remove the candidate from the sequence, add to suppression, never re-add automatically. Expected: no further emails to that candidate from any sequence.

4. Build a fallback: classifier confidence below 0.7, or neutral labels with decline-ish words, go to a human review queue; default action is stop. Expected: borderline replies pause instead of continuing.

5. Replay the last 500 classified replies through the new detector and count how many would now stop. Expected: a small number of newly caught declines, zero new false stops on clearly positive replies.

Use this when

• candidates who declined keep getting follow-ups
• your classifier labels polite declines as neutral
• reply handling is fully automated with no human review
• you run multi-touch sequences

Not for this skill when

• the candidate never replied and the sequence just ran too long (that is sequence-length policy)
• the decline went to a suppression list that was never loaded (that is the spreadsheet problem)
• the agent emailed after an unsubscribe via a different channel (check webhook wiring)

Variant phrasings

• "reply classifier missed a decline"
• "candidate said no but sequence continued"
• "polite rejection classified as neutral"
• "opt-out reply not detected"

Why it happens

General sentiment classifiers are trained on clear-cut text, but real declines are polite and hedged ("thanks for reaching out, not the right time"). The classifier averages the polite words and lands on neutral. A sequence treats neutral as "keep going." Without a dedicated decline path, the safest-sounding rejections are exactly the ones that slip through.

Edge cases

• Sarcasm and non-English replies need their own patterns; do not over-fit on English keywords alone.
• "Not interested in X role but open to Y" is a decline for this sequence but not for all outreach; stop this sequence, keep the candidate warm elsewhere.
• Out-of-office replies often contain "not available" wording; classify OOO separately and pause rather than stop.
• Review the human queue daily or it becomes a second inbox nobody reads.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_mqQBuynTkEdk1-PIde9PhQ