BigQuery IO
Revision GuideActive Topic
Cheatsheet: BigQuery IO
Recommended reading: 4 mins
Core Description
Read and write data high-throughput at scale to Google BigQuery.
WriteToBigQuery()
returns: PDonePurpose
Writes collections directly to BigQuery tables using bulk batch or high-throughput stream APIs.
Syntax Signature
WriteToBigQuery(table, write_disposition, create_disposition, method)Usage Example
import apache_beam as beam
from apache_beam.io.gcp.bigquery import WriteToBigQuery
rows = p | "Create Rows" >> beam.Create([{"name": "Alice", "age": 30}])
rows | "WriteBQ" >> WriteToBigQuery(
"project:dataset.table",
write_disposition=beam.io.BigQueryDisposition.WRITE_APPEND,
create_disposition=beam.io.BigQueryDisposition.CREATE_IF_NEEDED,
method=WriteToBigQuery.Method.STORAGE_WRITE_API
)Used In
Sinking processed data pipeline events to reporting tables.
Remember:
Prefer Method.STORAGE_WRITE_API for production pipelines; it provides transaction-safe stream writes.
More Free Data Engineering Cheatsheets (DataPlayArena)Interactive syntax references