## TL;DR
DuckDB cannot resolve S3 access for the path you gave it. Fix it by loading the `httpfs` extension, setting the region explicitly, and providing access through environment variables or a named secret instead of relying on implicit credential resolution.

```text
IO Error: Could not open file
```

## When to use
- S3 reads fail from a new machine, container, or CI runner
- The same query works locally but not in production
- The bucket region is ambiguous or cross-region

## When not to use
- The object genuinely does not exist (check the key spelling)
- The bucket name is wrong (thats a 404-style problem, not access)
- Local file reads fail (thats a filesystem path problem)

## Steps

1. Load the httpfs extension, which provides S3 support:

```sql
INSTALL httpfs;
LOAD httpfs;
```
Expected output: both statements succeed. Without httpfs, S3 paths are treated as local paths and the open fails.

2. Set the region explicitly. Implicit region detection fails often, especially with new buckets:

```python
import os
os.environ["AWS_REGION"] = "us-east-1"
```
Expected output: requests route to the right region. A wrong region produces access-looking errors on objects that exist.

3. Provide access through environment variables set before DuckDB starts:

```bash
export AWS_ACCESS_KEY_ID=[YOUR-KEY]
export AWS_SECRET_ACCESS_KEY=[YOUR-SECRET]
```
Expected output: DuckDB picks up the standard chain automatically. Setting them after the connection is open often has no effect.

4. Or declare a named secret inside DuckDB for full control:

```sql
CREATE SECRET my_s3 (
    TYPE S3,
    REGION 'us-east-1'
);
```
Expected output: the secret is registered and S3 reads use it. This beats env vars when one process talks to buckets under different identities.

## Variant phrasings

### Could not open file with s3:// path but the https scheme works
The httpfs settings differ per scheme. Set region and access explicitly rather than relying on per-scheme defaults.

### IO Error only on some objects in the bucket
Those objects may be in a different region or under a key prefix with different permissions. Check per-object, not per-bucket.

## Why it happens
DuckDB resolves S3 access through a chain: explicit secrets, then environment variables, then instance metadata, then anonymous. On a laptop the chain usually lands on a working entry; in containers and CI it silently falls through to anonymous access, and the resulting denial surfaces as a generic IO error rather than an auth error.

## Edge cases
- Instance metadata credentials expire. Long-running processes can start fine and fail hours later when the temporary access lapses.
- Requester-pays buckets need an explicit opt-in setting; otherwise even valid access gets denied.
- Object keys with special characters need URL encoding. The file exists but the unencoded path 404s.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_JMrY8HnxngY_SBfY_6vCuw