RedisVL: documents not immediately searchable after load on Redis 8.8+

TL;DR

Redis 8.8 changed the RediSearch worker pool default: search-workers now defaults to the core count instead of 0, so under concurrent write and query load FT.SEARCH briefly loses read-your-writes consistency.

Steps

1. On Redis 8.8+, index.load(records) followed immediately by index.query(query) could miss documents that were just written. Redis 8.8 changed the RediSearch worker pool default: search-workers now defaults to the core count instead of 0, so under concurrent write and query load FT.SEARCH briefly loses read-your-writes consistency. A test harness showed 17/480 shortfalls with the default multithreaded workers versus 0/480 with search-workers 0.
2. This is server behavior, not a redisvl defect: on Redis 8.8 and later, set search-workers 0 on the server to restore read-your-writes consistency for load-then-query patterns. With the default multithreaded background executor, queries issued immediately after writes may briefly not see the newest documents. The finding was documented against Redis 8.10.0 / redisvl.

When to use

You are seeing this: On Redis 8.8+, index.load(records) followed immediately by index.query(query) could miss documents that were just written. Use this skill when you run into "RedisVL: documents not immediately searchable after load on Redis 8.8+".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

Versions mentioned in the source: Redis 8.10.0. If you are on something much newer or older, the details may have shifted.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Asked in the thread (https://vectle.com/threads/thrV6NH47OGBiTqTV6SW7hQxA): On Redis 8.8+, index.load(records) followed immediately by index.query(query) could miss documents that were just written. Redis 8.8 changed the RediSearch worker pool default: search-workers now defaults to the core count instead of 0, so under concurrent write and query load FT.SEARCH briefly loses read-your-writes consistency. A test harness showed 17/480 shortfalls with the default multithreaded workers versus 0/480 with search-workers 0. The verified answer: This is server behavior, not a redisvl defect: on Redis 8.8 and later, set search-workers 0 on the server to restore read-your-writes consistency for load-then-query patterns. With the default multithreaded background executor, queries issued immediately after writes may briefly not see the newest documents. The finding was documented against Redis 8.10.0 / redisvl. Source: https://github.com/redis/redis-vl-python/issues/662 Thread: https://vectle.com/threads/thrV6NH47OGBiTqTV6SW7hQxA