Streaming LabHard
Lab: Telecom Monitoring
Estimated time: 45 mins
Who This Lab Is For
Advanced developers looking to learn event-time tracking, custom timestamp assignments, and fixed time windows.
What You Will Learn
- How to extract epoch timestamps and assign event time using TimestampedValue.
- How to partition streaming logs into fixed duration windows.
- How to access window boundaries at runtime using WindowParam.
1. Business Scenario
Process real-time streaming cellular events using Fixed Windows.
2. Input Dataset (\`dataset.csv\`)
Save the following raw rows locally as \`dataset.csv\` to test your pipeline:
text
timestamp,call_id,caller,receiver,duration_sec
1719830400,call1,userA,userB,45
1719830420,call2,userC,userD,120
1719830480,call3,userA,userE,30
1719830540,call4,userF,userB,60
1719830600,call5,userC,userA,903. Starter Code Skeleton
Create a local file named \`starter.py\` and copy the following skeleton. Complete the missing transformations:
python
# starter.py - Telecom Monitoring
import apache_beam as beam
from apache_beam.options.pipeline_options import PipelineOptions
def run_pipeline():
options = PipelineOptions()
with beam.Pipeline(options=options) as p:
# TODO: Assign event-time timestamps
# TODO: Apply Fixed Windows of 2 minutes (120 seconds)
# TODO: Calculate duration sums per window frame
pass
if __name__ == "__main__":
run_pipeline()4. Lab Requirements
- Extract and assign event timestamps to streaming calls.
- Apply 2-minute Fixed Windows.
- Aggregate total call duration per window pane.
5. Step-by-Step Guide & Solution
Advertisement
AdSense Slot #847392Leaderboard Banner (728x90)