## TL;DR
Cut the data before the expensive stage: add partition filters, replace full-table ORDER BY with aggregation first, and swap exact distinct counts for approximate ones. Resources exceeded means one query stage (usually a shuffle or sort) needed more memory than a worker allows, so the fix is always to shrink what flows into that stage.

```text
Query failed: Resources exceeded during query execution
```

## Use this when
- BigQuery fails with "resources exceeded during query execution"
- The error mentions shuffle or sort stages
- A query that worked on small data dies on the full table

## Not for this skill when
- The error is about quotas (rate limits, concurrent queries)
- The query succeeds but is slow or pricey
- You are asking about BigQuery pricing tiers

## Steps

1. Open the query plan and find the stage that blew up:

```sql
-- in the console: click "Execution details" on the failed job
-- look for the stage with the largest "shuffle output" or one that never completed
SELECT * FROM events WHERE _PARTITIONDATE = '2026-10-03' LIMIT 10;
```
Expected output: the plan shows which stage (join, aggregation, sort) consumed the resources. The fix targets that stage, not the whole query.

2. Add partition and cluster filters so the heavy stage reads less data URIs

```sql
-- before: full table scan into a giant shuffle
SELECT user_id, COUNT(*) FROM events GROUP BY user_id;
-- after: partitioned, filtered
SELECT user_id, COUNT(*)
FROM events
WHERE _PARTITIONDATE BETWEEN '2026-10-01' AND '2026-10-03'
GROUP BY user_id;
```
Expected output: bytes processed drops dramatically and the shuffle fits. An unfiltered full-table aggregation is the most common trigger.

3. Aggregate before joining, not after. Joining raw tables then grouping shuffles far more data URIs

```sql
-- before: join billions of rows, then aggregate
-- after: shrink each side first
WITH e AS (
  SELECT user_id, COUNT(*) AS events
  FROM events WHERE _PARTITIONDATE = '2026-10-03' GROUP BY user_id
),
p AS (
  SELECT user_id, plan FROM profiles WHERE _PARTITIONDATE = '2026-10-03'
)
SELECT p.plan, SUM(e.events) AS total_events
FROM e JOIN p USING (user_id)
GROUP BY p.plan;
```
Expected output: the join inputs are already small, so the shuffle stage stays within limits.

4. Replace exact distinct counts over huge cardinality with approximations:

```sql
SELECT APPROX_COUNT_DISTINCT(user_id) AS approx_users
FROM events
WHERE _PARTITIONDATE = '2026-10-03';
```
Expected output: a count accurate to about 1 percent using a fraction of the memory. Exact COUNT(DISTINCT) over hundreds of millions of values is a classic resources-exceeded trigger.

5. Kill the full-result ORDER BY, or push it after a LIMIT:

```sql
-- before: sorts the entire result set
SELECT * FROM events ORDER BY created_at;
-- after: sort only what you need
SELECT * FROM events
WHERE _PARTITIONDATE = '2026-10-03'
ORDER BY created_at
LIMIT 10000;
```
Expected output: the sort stage handles a bounded input. Sorting an unbounded result is pure memory with no benefit when nobody pages past the first chunk.

## Variant phrasings

### bigquery resources exceeded shuffle
The shuffle stage ran out of memory. Steps 2-4 shrink shuffle inputs; pre-aggregation (step 3) is the highest-leverage fix.

### resources exceeded during query execution sort
The ORDER BY stage blew up. Step 5: filter, aggregate, or limit before sorting.

### bigquery query too large to complete
Same family. The query plan will show whether it is shuffle, sort, or a single stage doing too much; split the query into staged temp tables if one stage cant be shrunk.

## Why it happens
BigQuery executes queries as stages with per-worker memory limits. Certain operations, shuffling rows between workers for a join or group-by, sorting an entire result, or tracking hundreds of millions of distinct values, need memory proportional to the data flowing through that stage. When the input is unbounded (no partition filter, no pre-aggregation), the stage exceeds what one worker can hold and the query dies.

## Edge cases
- Cross joins, even accidental ones from a missing join condition, explode shuffle size; check join keys before anything else.
- ORDER BY on a high-cardinality expression without LIMIT is the sort-stage killer; always bound it.
- Very wide rows (big JSON columns) multiply shuffle memory; select only the columns you need.
- If no rewrite shrinks the stage enough, split the query: write intermediate results to temp tables and query those.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_7Qeu8I0h70dr64LZluaOxA