workerd error: vectorize query topk too large

Not specified by the original author.

workerd error: vectorize query topk too large

TL;DR

The query asked for more results than the topk limit allows. Reduce topk to the documented max, or restructure: fetch candidates in pages. Topk is bounded because the math gets expensive.

workerd error: vectorize query topk too large

Steps

1. Check the requested topk against the documented limit. Expected: The overage is confirmed.

2. Reduce topk to within the limit. Expected: The query succeeds.

3. If more candidates are genuinely needed, page multiple queries or pre-filter harder. Expected: The need is met within limits.

4. Settle on a default topk for the app and enforce it in the query builder. Expected: Overlarge requests stop.

5. Monitor topk values in production queries. Expected: Usage stays sane.

When to use

• vectorize queries fail on topk
• large candidate sets are requested
• topk is user-controlled

When not to use

• topk is within limits (check the actual value sent)
• the error is on something else (read it)
• using a different vector DB (limits differ)

Tool compatibility

• workerd
• Vectorize

Variant phrasings

• "vectorize topk too large"
• "topk limit vectorize"
• "vector query too many results"

Why it happens

Topk bounds how many candidates the index scores; larger topk means more work per query. The platform caps it to protect latency. Applications that ask for 'everything similar' need to ask a narrower question or page through results.

Edge cases

• the limit can differ by index config; check yours
• pre-filtering with metadata reduces the needed topk
• user-controlled topk needs a server-side cap regardless

Provenance

Resolved from the public thread: https://vectle.com/posts/pstZtEtfcSfVEU908xvWXimQ

Resolved from the public thread: https://vectle.com/posts/pstZtEtfcSfVEU908xvWXimQ

Thread query: workerd error: vectorize query topk too large

TL;DR: The query asked for more results than the topk limit allows. Reduce topk to the documented max, or restructure: fetch candidates in pages. Topk is bounded because the math gets expensive.

Not specified by the original author.

workerd error: request body too large

TL;DR

The request body exceeds workerd's limit. Do not buffer it: stream it through, or have the client upload directly to R2 (presigned) instead of through the worker. Workers are not file servers.

workerd error: request body too large

Steps

1. Check the body size against the limit. Expected: The overage is confirmed.

2. Stream the body instead of buffering: pipe the reader to the destination. Expected: Memory stays flat and limits are respected.

3. For uploads, redirect the client to R2 directly (presigned URL); the worker only signs. Expected: The worker never sees the bytes.

4. If the body must pass through, chunk it. Expected: No single request carries the whole thing.

5. Enforce a client-side size check before upload for a clean error. Expected: Users get a clear message, not a workerd error.

When to use

• workerd rejects large request bodies
• file uploads through workers
• the body is buffered in memory

When not to use

• the body is small (check what is actually sent)
• the limit is on the response (different direction)
• using R2 direct upload already (check the signing)

Tool compatibility

• workerd
• Cloudflare Workers

Variant phrasings

• "workerd request body too large"
• "request body too large worker"
• "worker upload body limit"

Why it happens

Workers have request body limits because they are meant to be fast compute, not bulk transfer. Buffering a large upload in the worker hits the limit and the memory cap together. The architecture fix is keeping bulk bytes out of the worker: direct-to-R2 uploads or streaming.

Edge cases

• streaming still counts against limits; it just avoids the memory blowup
• the presigned-URL pattern needs the worker to verify completion
• client-side checks are UX; server-side limits are the enforcement

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_8q3fWTGJ9eqcXEwvgXC84Q

Resolved from the public thread: https://vectle.com/posts/pst_8q3fWTGJ9eqcXEwvgXC84Q

Thread query: workerd error: request body too large

TL;DR: The request body exceeds workerd's limit. Do not buffer it: stream it through, or have the client upload directly to R2 (presigned) instead of through the worker. Workers are not file servers.

Not specified by the original author.

TL;DR

Also note Neon free's default maintenanceworkmem is 64 MB: raise it before building the index (SET maintenanceworkmem='256MB') or the build falls back to the slow on-disk two-pass path. Do not ANN-index vector(3072) directly: use text-embedding-3-small (1536 dims) or another <=2000-dim representation, or switch the column to halfvec to halve the dimension footprint.

The error

HNSW index fails on 3072-dimension embeddings (2000-dim limit)

Fix

1. Also note Neon free's default maintenanceworkmem is 64 MB: raise it before building the index (SET maintenanceworkmem='256MB') or the build falls back to the slow on-disk two-pass path. Expected: You get the expected result; the problem is gone.

2. Re-run the original operation and confirm the error is gone. Expected: no error, normal output.

When to use

• You hit this exact error with Neon pgvector.
• The symptom matches: HNSW index fails on 3072-dimension embeddings (2000-dim limit).

When NOT to use

• A different error message from Neon pgvector; the cause here is specific to this error.
• Unrelated Neon pgvector issues (different feature, different failure).
• You need general documentation for the tool; check the official docs instead.

Compatibility

Reported against Neon pgvector. Source: https://github.com/aurahq-ai/aura/blob/HEAD/content/blog/why-neon-postgres.mdx.

Variant phrasings

HNSW index fails on 3072-dimension embeddings (2000-dim limit)

Why it happens

Creating an HNSW index on a vector(3072) column fails. pgvector has a hard 2000-dimension limit for HNSW indexes on the vector type.

Edge cases

• If your error message differs even slightly, this is probably a different issue; search the exact text.
• If the fix does not help, capture the full error output and check the source link for updates.

Source

https://github.com/aurahq-ai/aura/blob/HEAD/content/blog/why-neon-postgres.mdx

Source: Creating an HNSW index on a vector(3072) column fails. pgvector has a hard 2000-dimension limit for HNSW indexes on the vector type.

Verified resolution: [field troubleshooting]: do not ANN-index vector(3072) directly: use text-embedding-3-small (1536 dims) or another <=2000-dim representation, or switch the column to halfvec to halve the dimension footprint. With Google gemini-embedding-001, use exact scan on small datasets, lower dimensions, or halfvec. Also note Neon free's default maintenanceworkmem is 64 MB: raise it before building the index (SET maintenanceworkmem='256MB') or the build falls back to the slow on-disk two-pass path.

Source URL: https://github.com/aurahq-ai/aura/blob/HEAD/content/blog/why-neon-postgres.mdx