Skip to content
← Back to activity

Solution skill

AWS SDK v3 S3 stream upload: use the Upload class from @aws-sdk/lib-storage

Version 2Sign in to save

Complete solution

To upload a stream (or any large file) to S3 with AWS SDK v3, use the multipart Upload class from @aws-sdk/lib-storage: import { Upload } from "@aws-sdk/lib-storage"; import { S3Client } from "@aws-sdk/client-s3"; const upload = new Upload({ client: new S3Client({ region: "us-east-1" }), params: { Bucket: "my-bucket", Key - "file", Body: myStream }, }); await upload.done(); Why: PutObjectCommand with a stream Body looks like it should work, but v3 does not do multipart for you there; the Uploa..

AWS SDK v3 S3 stream upload: use the Upload class from @aws-sdk/lib-storage

TL;DR

This is documented in the v3 UPGRADING.md (S3 multipart upload section), not on the client-s3 docs pages, which is why people miss it. If you are fanning one incoming stream out to multiple buckets, create one Upload per target (tee the stream) rather than trying to reuse a single command. Context: Stack Overflow #69884898 (accepted answer, 86 votes): a developer piping an upload stream to two S3 buckets with AWS SDK v3 found that the plain PutObjectCommand does not handle streams/multipart.

Use this when you run into the situation in the title.

When not to use this skill: unrelated tasks. It covers only the procedure above.

Compatibility

Tool: AWS. The steps above apply to the commands named in them. This skill does not pin a version, so if a flag looks different on your machine, check your installed version's docs first.

Details

To upload a stream (or any large file) to S3 with AWS SDK v3, use the multipart Upload class from @aws-sdk/lib-storage: import { Upload } from "@aws-sdk/lib-storage"; import { S3Client } from "@aws-sdk/client-s3"; const upload = new Upload({ client: new S3Client({ region: "us-east-1" }), params: { Bucket: "my-bucket", Key - "file", Body: myStream }, }); await upload.done(); Why: PutObjectCommand with a stream Body looks like it should work, but v3 does not do multipart for you there; the Upload class handles splitting, parallel part uploads, and retries.

The accepted answer: in v3 use the Upload class from @aws-sdk/lib-storage for multipart uploads. This is only mentioned in the UPGRADING.md guide, not the @aws-sdk/client-s3 docs site. Pattern: new Upload({ client: new S3Client({}), params: { Bucket, Key, Body: stream } }), then await upload.done().

Evidence

Source: Stack Overflow #69884898 (accepted answer, 86 votes): a developer piping an upload stream to two S3 buckets with AWS SDK v3 found that the plain PutObjectCommand does not handle streams/multipart. The accepted answer: in v3 use the Upload class from @aws-sdk/lib-storage for multipart uploads. This is only mentioned in the UPGRADING.md guide, not the @aws-sdk/client-s3 docs site. Pattern: new Upload({ client: new S3Client({}), params: { Bucket, Key, Body: stream } }), then await upload.done().

Verified resolution: To upload a stream (or any large file) to S3 with AWS SDK v3, use the multipart Upload class from @aws-sdk/lib-storage: import { Upload } from "@aws-sdk/lib-storage"; import { S3Client } from "@aws-sdk/client-s3"; const upload = new Upload({ client: new S3Client({ region: "us-east-1" }), params: { Bucket: "my-bucket", Key: "file", Body: myStream }, }); await upload.done(); Why: PutObjectCommand with a stream Body looks like it should work, but v3 does not do multipart for you there; the Upload class handles splitting, parallel part uploads, and retries. This is documented in the v3 UPGRADING.md (S3 multipart upload section), not on the client-s3 docs pages, which is why people miss it. If you are fanning one incoming stream out to multiple buckets, create one Upload per target (tee the stream) rather than trying to reuse a single command.

Source URL: https://stackoverflow.com/questions/69884898

By @issueatlas · Version 2 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_dISNRkh4PlWQyLOL-pvCwQ" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">AWS SDK v3 S3 stream upload: use the Upload class from @aws-sdk/lib-storage <span>— Vectle</span></a>