Solution skill
Consider avoiding adding suffixes to files in FileSystemStorageBackend
Complete solution
Shows how to fix consider avoiding adding suffixes to files in FileSystemStorageBackend. Use it when you hit this exact problem. Skip it when your error message or symptom looks different.
TL;DR
This changed in Crawlee v4. The old.txt/.bin probing ladder was removed entirely; the upgrading guide documents the change.
Steps
- This changed in Crawlee v4. The v3 behavior of appending
.txtor.binwhen writing keys is gone: in@crawlee/fs-storage,setValue(key)now writes exactly the key you give it, plus a[key].__metadata__.jsonsidecar, matching the API. Hand-writtenINPUT.jsonfiles still work: on open, the store adopts the sidecar-less file and binds it to theINPUTrecord, sogetInput()reads it (this also repairs the__CLI_INPUT.jsoncase). The old.txt/.binprobing ladder was removed entirely; the upgrading guide documents the change. Practical rule: in v4, always use exact keys, and for input just keep shippingINPUT.jsonas usual.
When to use
You are seeing this: Issue apify/crawlee#2710 (closed, 17 comments): There is currently an automagic mechanism that stores binary files with a ".bin" extension and text files with a ".txt" extension. Use this skill when you run into "Consider avoiding adding suffixes to files in FileSystemStorageBackend".
When not to use
If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.
Versions
No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.
Why this happens
The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.
Sources
- https://github.com/apify/crawlee/issues/2710
Evidence
In https://github.com/apify/crawlee/issues/2710, the reporter asked: Issue apify/crawlee#2710 (closed, 17 comments): There is currently an automagic mechanism that stores binary files with a ".bin" extension and text files with a ".txt" extension. Is the comfort for Windows users worth the complexity? ### Prior art - apify/apify-cli#960 (closed by #1042) - the CLI contract: env precedence ACTORINPUTKEY > APIFYINPUTKEY > CRAWLEEINPUTKEY, CLI sets all three, and "CLI does not care about duplicate input files, SDK should raise". B4nan and Pijukatel argued the extension question there at length; B4nan's position was "INPUT means INPUT.json locally, .json gets implicit priority + warn on others". The verified answer: This changed in Crawlee v4. The v3 behavior of appending .txt or .bin when writing keys is gone: in @crawlee/fs-storage, setValue(key) now writes exactly the key you give it, plus a [key].__metadata__.json sidecar, matching the API. Hand-written INPUT.json files still work: on open, the store adopts the sidecar-less file and binds it to the INPUT record, so getInput() reads it (this also repairs the __CLI_INPUT.json case). The old .txt/.bin probing ladder was removed entirely; the upgrading guide documents the change. Source thread: https://github.com/apify/crawlee/issues/2710
By @issueatlas · Version 2 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_W8OqUzbSp-b1Vo2U1N8aVw" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Consider avoiding adding suffixes to files in FileSystemStorageBackend <span>— Vectle</span></a>