1. Introduction

A Sliding Window (sometimes called a hopping window) is a windowing strategy that partitions a data stream into overlapping, equal-sized time intervals. It is defined by two properties: the Window Size (duration) and the Slide Period (recalculation frequency).

2. Why This Concept Exists

Fixed Windows segment time into strict, non-overlapping blocks. However, many business metrics require a continuous moving average—such as calculating the "average website response time over the last 10 minutes, updated every 1 minute." With Fixed Windows, you only get updates once every 10 minutes. Sliding Windows allow you to overlay multiple time blocks, giving you real-time moving metrics.

3. Key Terminology

Window Size: The duration representing how much history each window contains (e.g., 10 minutes).
Slide Period: The frequency of how often a new window starts (e.g., 1 minute).
Overlapping: Because the slide period is smaller than the window size, a single data point belongs to multiple windows simultaneously.

4. How It Works

Beam generates a new window at every interval defined by the Slide Period.
When an event arrives, Beam inspects its timestamp.
Because windows overlap, the event is assigned to all active windows that cover its timestamp.
For example, if size is 10 minutes and slide is 2 minutes, a single event will be replicated and processed inside 5 separate overlapping windows.

5. Visual Diagram

Event at [12:03] falls into multiple sliding windows:

Window 1 (12:00 - 12:10)

Contains Event [12:03]

Window 2 (12:02 - 12:12)

Contains Event [12:03]

6. Code Example

Calculating a moving average of values over 10-minute windows, recalculating every 1 minute:

python
import apache_beam as beam
from apache_beam.transforms.window import SlidingWindows

with beam.Pipeline() as p:
    (p
     | "ReadStream" >> beam.io.ReadFromPubSub(subscription="projects/my-gcp/subs/metrics")
     | "ParseFloats" >> beam.Map(lambda x: float(x))
     | "ApplySlidingWindows" >> beam.WindowInto(SlidingWindows(size=10 * 60, period=60)) # 10m size, 1m slide
     | "MovingAverage" >> beam.CombineGlobally(beam.combiners.MeanCombineFn()).without_defaults()
     | "Log" >> beam.Map(print))

7. Code Explanation

SlidingWindows(size=10 * 60, period=60) configures the window boundaries (600s size, 60s period).
CombineGlobally(MeanCombineFn()) computes the mean independently for each window.
Outputs are produced every 1 minute, representing the moving average of the last 10 minutes of logs.

8. Real Production Example

In stock trading platforms, you monitor price volatility. You configure a sliding window of 1 hour, sliding every 10 seconds, to compute a rolling standard deviation of stock prices, alerting traders immediately if price variance spikes.

9. Common Mistakes

Creating too many overlapping windows: Setting a very large size and tiny slide (e.g. 24-hour windows sliding every 1 second) forces Beam to replicate every element 86,400 times. This will crash worker memory and result in massive computation delays.
Expecting updates before the slide period: The pipeline will only output results when a slide boundary passes.

10. Best Practices

Ensure the Slide Period is an exact divisor of the Window Size (e.g. size 60s, slide 10s) to keep window bounds clean.
Avoid small slide periods (like 1 second) on high-throughput streams.

11. Summary

Sliding windows group elements into overlapping time blocks.
Configured by Window Size and Slide Period.
Multiplies elements across overlapping intervals to compute rolling statistics.

12. Interactive Challenges

Challenge 1: Moving Window Configuration (Beginner)

Write a windowing transform statement that partitions an incoming stream into sliding windows of 15 minutes, recalculating every 3 minutes.

Challenge 2: Moving Average Volume (Intermediate)

Write a pipeline segment inside a context block that reads element values, windows them into 5-minute sliding windows sliding every 30 seconds, and calculates the mean value per window.

Challenge 3: Hourly Slide with Lateness (Advanced)

Write a windowing segment that applies a 1-hour sliding window, sliding every 5 minutes, while allowing late-arriving data for up to 2 minutes after the watermark passes.

Sliding Windows