## TL;DR
Vector DB batch upserts can partially fail (or silently overwrite) without raising, so the agent thinks N vectors landed when fewer did. Fix it by counting before and after every upsert, checking the batch response for per-item errors, and making ids deterministic so retries are safe.

```text
agent's vector upsert silently dropped half the batch: verification
```

## Use this when
- An agent upserts vectors but counts come up short
- No error is raised yet data is missing
- Retried upserts behave inconsistently

## Not for this skill when
- The upsert raises a clear error (thats the error, not silent loss)
- Dimensions mismatch (thats the embedding)
- Queries return wrong results (thats search config)

## Steps

1. Count before and after every upsert in the agent's code:

```python
before = collection.count()
collection.upsert(ids=ids, embeddings=vecs)
after = collection.count()
print(f"expected +{len(ids)}, got +{after - before}")
```
Expected output: the delta equals the batch size. Any shortfall is silent loss; fail loudly here instead of downstream.

2. Inspect the batch response for per-item failures:

```python
# many clients return per-item statuses; dont ignore the return value
result = batch_upsert(items)
print(result.errors or "no errors reported")
```
Expected output: either clean or a list of failed items to retry. Agents commonly discard the return value, which is where partial failures hide.

3. Check for id collisions: upsert overwrites same-id vectors silently:

```python
print(len(ids), len(set(ids)))
```
Expected output: equal numbers. Duplicates in the id list mean later items overwrite earlier ones and the count shortfall is "by design."

4. Make the upsert idempotent with deterministic ids and verify by sampling:

```python
# after upsert, fetch a random sample of ids and confirm presence
sample = random.sample(ids, min(100, len(ids)))
found = collection.get(ids=sample)
print(len(found["ids"]), "of", len(sample))
```
Expected output: all sampled ids present. Bake this check into the agent's upsert function, not as a one-off.

## Variant phrasings

### count is right but some vectors are stale
Overwrites with same ids but old embeddings. The count check passes; add a content hash check on a sample.

### loss only on large batches
The batch exceeds a server limit and gets truncated. Chunk the upsert (e.g. 1000 per batch) and verify per chunk.

## Why it happens
Upsert semantics (insert-or-overwrite) plus batch APIs that report partial failures in the return value (not as exceptions) create silent loss: duplicates collapse, oversized batches truncate, and invalid items are skipped. Agent code typically fires the upsert and moves on without reading the response or recounting, so the loss surfaces much later as "the search cant find things."

## Edge cases
- Some stores dedupe on id within the batch but not across retries; a retried batch can double-apply non-idempotent metadata updates.
- Eventual consistency means an immediate recount can under-read; wait briefly or use the store's consistency parameter before verifying.
- Payload size limits can drop the metadata while keeping the vector; verify payloads on the sample too, not just ids.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_VvH0TXUI_RTSwSkXU9kbGQ