BigQuery load jobs from GCS: schema, partitioning, and location traps

from google.cloud import bigquery
client = bigquery.Client(project="[PROJECT-ID]", location="EU")
job_config = bigquery.LoadJobConfig(
    source_format=bigquery.SourceFormat.PARQUET,
    write_disposition=bigquery.WriteDisposition.WRITE_TRUNCATE,
)
uri = "gs://[BUCKET]/data/*.parquet"
job = client.load_table_from_uri(uri, "[PROJECT].[DATASET].[TABLE]", job_config=job_config)
job.result()
print("loaded", job.output_rows, "rows")

Traps:

1. Location again. Load job location must match the dataset location or the job fails. Same rule as queries.
2. Schema autodetect (autodetect=True) guesses types from a sample. It guesses wrong on edge data (a column that is all nulls, mixed int/string). For anything recurring, declare the schema explicitly.
3. WRITETRUNCATE vs WRITEAPPEND vs WRITEEMPTY. The default is WRITEAPPEND. Agents expecting a fresh table get duplicates. Choose deliberately every time.
4. CSV specifics: set skipleadingrows, allow quoted newlines if your data has them, and null markers. Parquet and Avro avoid most of this; prefer them for machine-generated data.
5. Partitioning: partition by ingestion time or a date column at load. Querying unpartitioned multi-TB tables is the classic surprise bill. Clustering on top of partitioning for high-cardinality filters.
6. Permissions split: the loader needs bigquery.jobs.create on the billing project AND bigquery.dataEditor (or tables update) on the target dataset AND storage.objects.get on the GCS objects. Missing any one of the three fails with a different error.

Verify: check job.output_rows, then query the table metadata for schema and partition spec before downstream jobs run.

Verified 2026-09-26 on the BigQuery loading docs and error docs (location mismatch, permission model). Write dispositions and partitioning guidance are documented load-job properties.