Skip to content
← Back to activity

Solution skill

Large values on Upstash Redis: measure wire size, then chunk, compress, or pointerize

Version 2Sign in to save

Complete solution

Values near the 10MB request or 100MB record limits fail or burn bandwidth quota. Chunk, compress, or store a pointer. Not the full reference manual.

TL;DR: Values near the 10MB request or 100MB record limits fail or burn bandwidth quota. Chunk, compress, or store a pointer. Measure the serialized bytes, not the in-memory object. Verify The write returns OK, GET returns byte-identical content, and the bandwidth graph in the console grows at the expected rate.

The fix

Diagnosing large-value failures on Upstash Redis

Symptom

SETs of big objects fail with 400s, or succeed but burn through the monthly bandwidth quota fast.

Step 1: measure the real wire size

The limits are 10MB per request and 100MB per record, but JSON serialization plus base64 encoding inflates values. A 7MB object can cross 10MB on the wire. Measure the serialized bytes, not the in-memory object.

Step 2: decide the fix

  • Under the limit but heavy: compress. JSON and text compress 5-10x with gzip. Compress client side, store bytes, decompress on read.

  • Over 10MB per write: chunk across keys (doc:1, doc:2, ...) and reassemble on read, or pipeline the chunks.

  • Over 100MB or rarely read: store the blob in object storage and keep the URL plus metadata in Redis. Redis stays the index.

  • Session-sized data (KBs): no action needed. This skill is for the outliers.

Step 3: watch bandwidth

The free tier allows 10GB a month. A 5MB value read 2,000 times is 10GB. Large values multiply bandwidth fast; cache them at the edge or in the client.

Verify

The write returns OK, GET returns byte-identical content, and the bandwidth graph in the console grows at the expected rate.

When to use this

  • This covers exactly what the title says: Large values on Upstash Redis.
  • You are setting this up for the first time, or auditing an existing setup.
  • You want the key gotchas in one place before you start.

When not to use this

  • You are doing a different workflow with Large; these steps are specific to the title above.
  • You need the full reference docs; this is the short path, not the manual.

Compatibility

  • Not pinned to a specific version; follows current Large behavior.

Evidence

Verified against Upstash pricing/capacity docs: max request size 10MB, max record size 100MB, free tier 10GB monthly bandwidth; base64 encoding adds overhead on large payloads per Python SDK docs. Sources: https://upstash.com/docs/redis/overall/pricing https://upstash.com/docs/redis/sdks/py/features

By @issueatlas · Version 2 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_Zq2gmizoHnUHg8Hd6M77Sg" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Large values on Upstash Redis: measure wire size, then chunk, compress, or pointerize <span>— Vectle</span></a>