# screening agent scored every candidate above 90 out of 100 - the rubric had no negative examples so everything looked strong

## TL;DR

The rubric described only what good looks like, so the agent had no way to recognize bad. Add negative examples (what a 40 looks like), anchor the scale with reference candidates, and require the agent to cite specific evidence per score. A rubric without failure modes is a compliment generator.

```text
screening agent scored every candidate above 90 out of 100 - the rubric had no negative examples
```

## Steps

1. Plot the score distribution over the last few hundred screenings.
   Expected: Everything clusters at 85-100 with no spread.

2. Rewrite the rubric with anchored levels: describe a 30, a 60, and a 90 with concrete resume examples, including what disqualifies.
   Expected: Each level names specific evidence.

3. Require evidence citations: every score must quote the resume lines that justify it.
   Expected: Scores without citations are rejected by the harness.

4. Calibrate on a labeled set: run the new rubric over 50 past candidates with known outcomes and check the spread.
   Expected: Scores now span the range and correlate with outcomes.

5. Monitor the distribution weekly; rubrics drift as the agent learns to be generous again.
   Expected: The spread stays healthy over time.

## When to use

- all candidates score 85+
- the rubric has only positive descriptions
- screening passes everyone

## When not to use

- scores spread normally (the rubric is fine)
- the candidate pool is genuinely strong (check the source before blaming the rubric)
- the issue is ranking, not scoring (different fix)

## Tool compatibility

- any LLM-based screening agent
- rubric-prompted evaluation

## Variant phrasings

- "screening agent scores everyone high"
- "rubric no negative examples"
- "candidate scores collapsed at top"

## Why it happens

An LLM asked to evaluate against a positive-only rubric has no concept of the low end of the scale; everything matches 'good' at least a little. Without negative anchors and required evidence, generosity is the default and the scores carry no information.

## Edge cases

- reference candidates go stale as the role evolves; re-anchor quarterly
- evidence citations can be gamed by quoting generously; spot-check the quotes
- score inflation also comes from temperature; keep scoring at temperature 0

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_2ld1MgajMRBgJxtvkIBqNg