Cheatsheet: Pub/Sub IO

Topic Overview

Integrate with serverless Google Cloud Pub/Sub for messaging.

Syntax Snapshot

import apache_beam as beam

# 1. Read from Pub/Sub topic/subscription
messages = p | "ReadPubSub" >> beam.io.ReadFromPubSub(
    subscription="projects/my-project/subscriptions/my-sub"
)

# 2. Write to Pub/Sub topic
messages | "WritePubSub" >> beam.io.WriteToPubSub(
    topic="projects/my-project/topics/my-topic"
)

Key Points

• Serverless messaging source/sink for streaming pipelines.
• Automatically extracts message publish time as the element event-time.
• Can configure custom timestamp attributes using timestamp_attribute.
• Supports reading attributes and payload byte blobs.

Production Recommendations