Session Windows

1. Introduction

2. Why This Concept Exists

Fixed and sliding windows are bound to static calendar intervals (e.g. every hour). However, human behavior is irregular. For example, a user visits an e-commerce store, browses items for 15 minutes, and leaves. You want to analyze their behavior as a single "session." Session windows allow you to capture these irregular bursts of activity, grouping events by inactivity breaks rather than clock boundaries.

3. Key Terminology

• Inactivity Gap: The maximum duration of silence allowed before a session is declared closed (e.g., 30 minutes).
• Dynamic / Key-scoped: Unlike fixed windows, session windows are unique to each key. User-A and User-B can have completely different session start and end times.
• Window Merging: The process where adjacent session windows are combined together if a new event arrives that bridges the gap between them.

4. How It Works

1. Initial assignment: When an event arrives, Beam assigns it to a temporary session window starting at the event's timestamp and ending at timestamp + inactivity_gap.
2. Evaluate overlaps: Beam checks if this new temporary window overlaps with any existing session windows for that specific key.
3. Merge: If they overlap, Beam merges the windows into a single, larger session window.
4. Repeat: The session window expands continuously as long as elements arrive before the inactivity gap expires.

5. Visual Diagram

6. Code Example

Grouping web clicks into session windows with a 30-minute inactivity gap:

pythonCopy
import apache_beam as beam
from apache_beam.transforms.window import Sessions

with beam.Pipeline() as p:
    (p
     | "ReadClicks" >> beam.io.ReadFromPubSub(subscription="projects/my-gcp/subs/clicks")
     | "FormatKV" >> beam.Map(lambda x: (x["user_id"], x["page_url"]))
     | "ApplySessions" >> beam.WindowInto(Sessions(gap_size=30 * 60)) # 30-minute gap
     | "GroupSessionPages" >> beam.GroupByKey()
     | "LogSessions" >> beam.Map(print))

7. Code Explanation

• Sessions(gap_size=30 * 60) defines a 30-minute inactivity gap (1800 seconds).
• GroupByKey() groups the urls per user within each session.
• If User-1 clicks at 12:00 and 12:15, they are merged into one session 12:00 - 12:45.
• If User-1 clicks again at 13:00, it starts a separate second session.

8. Real Production Example

In online gaming analytics, session windows are used to track player matches. You set a session gap of 15 minutes. When a player logs in and plays 5 matches in a row with less than 15 minutes between matches, all game statistics are grouped as a single gaming session for metrics aggregation.

9. Common Mistakes

• Running without keys: Session windows require key-value tuples to work. Merging occurs independently per key. Running sessions on flat collections will throw an execution exception.
• Setting an excessively large gap size: If you set the gap size too large (e.g. 24 hours), sessions will rarely merge or close, causing workers to hold window state in memory indefinitely.

10. Best Practices

• Clean up session resources using a reasonable gap size.
• Set an allowed lateness policy to handle out-of-order logs that might merge past session intervals.

11. Summary

• Session windows are based on periods of activity separated by inactivity gaps.
• Dynamic and scoped per key.
• Merge adjacent windows automatically as elements arrive.

12. Interactive Challenges

13. Related Content

• Fixed Windows
• Sliding Windows