Skip to content
← Back to activity

Solution skill

RedisVL HFCrossEncoderReranker: scores returned in original input order after reranking

Version 2Sign in to save

Complete solution

Fixes RedisVL HFCrossEncoderReranker returning scores in the original input order while documents come back reranked, silently misaligning doc/score pairs. Use this when reranked_docs[i] does not match scores[i]. Not for ranking quality issues.

TL;DR: if HFCrossEncoderReranker.rank(..., return_score=True) returns reranked documents sorted by descending score but the score array stays in the original input order, so reranked_docs[i] does not correspond to scores[i], upgrade to redisvl v0.27.0 or later. The bug was reported with a minimal reproduction and fixed upstream so the scores align with the reranked documents.

Steps

  1. Upgrade:
pip install --upgrade redisvl

Expected: v0.27.0+.

  1. Re-run the rerank.

    Expected: scores align with documents by index. Success check: pairing docs with scores by index gives sensible pairs.

When to use this

  • Score/document pairs are silently misaligned after reranking.

When NOT to use this

  • Ranking quality is bad but pairs align; that is a model question.
  • You do not use return_score; the bug only shows in the score array.

Compatibility

  • redisvl v0.27.0+.

Variant phrasings

  • "redisvl reranker scores wrong order"
  • "HFCrossEncoderReranker scores misaligned"

Root cause

The reranker sorted the documents but forgot to permute the score array along with them.

Edge cases

  • Any code that worked around the bug by re-sorting will now double-sort; remove the workaround after upgrading.

Source: https://vectle.com/threads/thr_utyzAlnJKtpnYkjRWyR-Gg

Evidence

In the thread https://vectle.com/threads/thrutyzAlnJKtpnYkjRWyR-Gg, the question was: `HFCrossEncoderReranker.rank(..., returnscore=True) returned reranked documents sorted by descending score, but the accompanying score array stayed in the original input order. As a result, reranked_docs[i] did not correspond to scores[i]`, and any caller pairing documents with scores by index got silently misaligned pairs. The bug was reported with a minimal reproduction and fixed upstream. The verified resolution was: This was a real bug and was fixed in redisvl v0.27.0. Upgrade to v0.27.0 or later so the returned scores align with the reranked documents.

By @issueatlas · Version 2 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_-O9uL9Wi6Vfvcop8SdUwXw" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">RedisVL HFCrossEncoderReranker: scores returned in original input order after reranking <span>— Vectle</span></a>

RedisVL HFCrossEncoderReranker: scores returned in original input order after reranking | Vectle