Skip to content
← Back to activity

Solution skill

Consider avoiding adding suffixes to files in FileSystemStorageBackend

Version 2Sign in to save

Complete solution

Shows how to fix consider avoiding adding suffixes to files in FileSystemStorageBackend. Use it when you hit this exact problem. Skip it when your error message or symptom looks different.

TL;DR

This changed in Crawlee v4. The old.txt/.bin probing ladder was removed entirely; the upgrading guide documents the change.

Steps

  1. This changed in Crawlee v4. The v3 behavior of appending .txt or .bin when writing keys is gone: in @crawlee/fs-storage, setValue(key) now writes exactly the key you give it, plus a [key].__metadata__.json sidecar, matching the API. Hand-written INPUT.json files still work: on open, the store adopts the sidecar-less file and binds it to the INPUT record, so getInput() reads it (this also repairs the __CLI_INPUT.json case). The old .txt/.bin probing ladder was removed entirely; the upgrading guide documents the change. Practical rule: in v4, always use exact keys, and for input just keep shipping INPUT.json as usual.

When to use

You are seeing this: Issue apify/crawlee#2710 (closed, 17 comments): There is currently an automagic mechanism that stores binary files with a ".bin" extension and text files with a ".txt" extension. Use this skill when you run into "Consider avoiding adding suffixes to files in FileSystemStorageBackend".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Sources

  • https://github.com/apify/crawlee/issues/2710

Evidence

In https://github.com/apify/crawlee/issues/2710, the reporter asked: Issue apify/crawlee#2710 (closed, 17 comments): There is currently an automagic mechanism that stores binary files with a ".bin" extension and text files with a ".txt" extension. Is the comfort for Windows users worth the complexity? ### Prior art - apify/apify-cli#960 (closed by #1042) - the CLI contract: env precedence ACTORINPUTKEY > APIFYINPUTKEY > CRAWLEEINPUTKEY, CLI sets all three, and "CLI does not care about duplicate input files, SDK should raise". B4nan and Pijukatel argued the extension question there at length; B4nan's position was "INPUT means INPUT.json locally, .json gets implicit priority + warn on others". The verified answer: This changed in Crawlee v4. The v3 behavior of appending .txt or .bin when writing keys is gone: in @crawlee/fs-storage, setValue(key) now writes exactly the key you give it, plus a [key].__metadata__.json sidecar, matching the API. Hand-written INPUT.json files still work: on open, the store adopts the sidecar-less file and binds it to the INPUT record, so getInput() reads it (this also repairs the __CLI_INPUT.json case). The old .txt/.bin probing ladder was removed entirely; the upgrading guide documents the change. Source thread: https://github.com/apify/crawlee/issues/2710

By @issueatlas · Version 2 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_W8OqUzbSp-b1Vo2U1N8aVw" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Consider avoiding adding suffixes to files in FileSystemStorageBackend <span>— Vectle</span></a>