Windowing & Streaming⏱️ ~22 mins
25. User Inactivity Session Windows
Enterprise Architecture Context
In production stream-processing architectures (Google Cloud Dataflow / Flink), pipeline stages must handle parallel transformations without data loss, managing schema mutations and aggregations across distributed worker workers.
Problem Statement
### Business Context
Web analytics platforms cluster user click events into browsing sessions that automatically close after 300 seconds (5 minutes) of user inactivity.
### Problem Statement
Write a function `session_windows(input_pcoll)` that groups records into session windows with a gap duration of **300 seconds** using `beam.WindowInto(Sessions(300))`.
Key Learning Objectives
- Understand distributed Apache Beam execution DAG stages and pipeline lifecycle.
- Apply idiomatic functional Python transforms using the pipe operator
|. - Ensure data consistency and idempotency across distributed stream workers.
Sample Data Fixtures
Sample Example 1
Input Stream:
[('user1', 'click1'), ('user1', 'click2')]
Expected Output:
[('user1', 'click1'), ('user1', 'click2')]
Sample Example 2
Input Stream:
[('user2', 'login')]
Expected Output:
[('user2', 'login')]
Topics:#Windowing#Sessions
solution.pyPython 3.11 (Apache Beam)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Input PCollection
[('user1', 'click1'), ('user1', 'click2')]Expected Output PCollection
[('user1', 'click1'), ('user1', 'click2')]Windowing & Streaming⏱️ ~22 mins
25. User Inactivity Session Windows
Enterprise Architecture Context
In production stream-processing architectures (Google Cloud Dataflow / Flink), pipeline stages must handle parallel transformations without data loss, managing schema mutations and aggregations across distributed worker workers.
Problem Statement
### Business Context
Web analytics platforms cluster user click events into browsing sessions that automatically close after 300 seconds (5 minutes) of user inactivity.
### Problem Statement
Write a function `session_windows(input_pcoll)` that groups records into session windows with a gap duration of **300 seconds** using `beam.WindowInto(Sessions(300))`.
Key Learning Objectives
- Understand distributed Apache Beam execution DAG stages and pipeline lifecycle.
- Apply idiomatic functional Python transforms using the pipe operator
|. - Ensure data consistency and idempotency across distributed stream workers.
Sample Data Fixtures
Sample Example 1
Input Stream:
[('user1', 'click1'), ('user1', 'click2')]
Expected Output:
[('user1', 'click1'), ('user1', 'click2')]
Sample Example 2
Input Stream:
[('user2', 'login')]
Expected Output:
[('user2', 'login')]
Topics:#Windowing#Sessions