RangeError: Map maximum size exceeded when uploading large file to Apify key-value store

TL;DR: if Actor.setValue() throws RangeError: Map maximum size exceeded on a large file, do not pass a Buffer or string. Convert the file into a ReadableStream and hand that to setValue() instead. The reporter confirmed this works for almost any file size.

The error, verbatim:

RangeError: Map maximum size exceeded

Steps

1. Confirm the pattern: small files save fine, files over roughly 9 MB throw. If small files also fail, this is not your problem.
2. Replace the materialized payload with a stream:

const stream = fs.createReadStream('archive.zip');
await Actor.setValue('my-file', stream, { contentType: 'application/zip' });

Expected: setValue accepts the stream without throwing.

3. Success check: the upload completes and the key reads back the full file. Building the archive itself was never the slow part; the crash happened while saving, so a fast archive build plus this crash is the signature.

When to use this

• Actor.setValue() throws RangeError: Map maximum size exceeded on large payloads (seen above ~9 MB).
• You are writing to an Apify Actor key-value store.

When NOT to use this

• Dataset pushes or request-queue limits; different storage, different errors.
• Small payloads failing; look at serialization or auth instead.

Compatibility

• crawlee 3.1.3 / apify 3.1.1, Actor key-value store setValue. The crash is size-dependent, not version-dependent beyond that.

Variant phrasings

• apify setValue large file RangeError
• crawlee key value store Map maximum size exceeded
• upload zip to apify kv store fails over 9mb

Root cause

The store materializes the whole payload in a Map-backed structure before writing, and V8's Map size limit trips on large buffers. A ReadableStream never materializes the full payload in one structure, so the limit is never hit.

Edge cases

• Content type still matters; pass it explicitly so the stored key is readable as the right type.
• If you need progress reporting, wrap the stream; setValue itself gives you no progress callback either way.

Source: https://github.com/apify/crawlee/issues/1732

In the thread, the asker reported: Uploading a zip file larger than about 9 MB to an Apify Actor's key-value store with Actor.setValue() (crawlee 3.1.3 / apify 3.1.1) throws RangeError: Map maximum size exceeded. Building the archive itself takes only ~5 seconds; the crash happens while saving to the key-value store. Smaller files save without any issue, so the failure is size-dependent. The verified answer that resolved it: Pass the data as a ReadableStream instead of a fully materialized Buffer/string. Convert the file into a ReadableStream and hand that to Actor.setValue(); the reporter confirmed this works for almost any file size. The crash comes from the store materializing the whole payload in a Map-backed structure, which a stream avoids. Source: https://github.com/apify/crawlee/issues/1732 Thread: https://vectle.com/threads/thr__7MxyYvptVhbwxuoVacimg