Pandas: Window Functions

Q: Does rolling support time-based windows?

Yes — rolling('7D') uses a 7-day calendar window, while rolling(7) uses a 7-row window. Time-based windows are ideal for non-uniformly sampled data (skipping weekends or missing days), while row-based windows suit uniform sampling. Time-based windows require the Index to be a DatetimeIndex, or you can specify a time column with the on parameter.

Q: How is the performance of custom functions?

Custom functions are 10-100x slower than built-in aggregations — a Python function call is made for every window. Optimization tips: use raw=True to pass a NumPy array (5-10x faster), and replace Python loops with NumPy operations. Built-in functions are implemented in C, while custom functions go through the Python interpreter. If a built-in function does the job, avoid writing a custom one.

Q: What is the relationship between ewm's span/halflife/com?

All three are mathematically equivalent — just different ways of expressing the same thing. span=N behaves like an N-period SMA but with more weight on recent data; com=span-1 is the center of mass; halflife is the number of periods for the weight to halve. span is the most commonly used (most intuitive). α=2/(span+1) is the smoothing factor. You only need to specify one — there is no need to set multiple parameters at once.

Q: What should I watch out for with groupby+rolling?

Two key points: ① Always sort_values by the group key + time first, otherwise the window may span across groups; ② transform(lambda x: x.rolling(N).mean()) returns a result with the same length as the original DataFrame, while calling groupby+rolling directly returns a MultiIndex. For large datasets, transform is more convenient.

Q: What win_type options are available?

All window functions from scipy.signal.windows: gaussian (Gaussian weighting), triang (triangular), blackman, hamming, kaiser, and more. Requires pip install scipy . win_type applies weights to data within the window (higher weight in the middle, lower at the edges), producing smoother results than an equal-weight window.

Q: What does the closed parameter do?

closed controls which end of the window is inclusive — 'right' (default, includes the right end = current row), 'left' (includes the left end), 'both' (both ends), 'neither' (neither end). In finance, 'right' is commonly used (only historical data), while certain statistical scenarios require 'both' or 'left'.

Last updated: 2026-08-26

Lesson 17 introduced rolling/expanding/ewm. This lesson goes deeper: custom window aggregation, advanced time windows, ewm parameter relationships, and groupby+rolling combinations. Window functions are a core tool for quantitative analysis and sensor data processing — master them, and you master the ability to "see trends through a sliding lens."

⚠️ Note: The code below must be run in a local Python environment.

1. What You Will Learn

❶ Advanced rolling parameters
❷ Custom window aggregation
❸ expanding cumulative windows in depth
❹ ewm parameters explained
❺ groupby + rolling

2. Carol's Exercise Heart Rate Window Analysis

(1) The Problem: A 30-Second Window Mean Is Not Enough

Carol's exercise heart rate data needs richer window statistics — not just the mean, but also the maximum and heart rate variability (standard deviation):

PYTHON

import pandas as pd
import numpy as np

np.random.seed(42)
hr = pd.DataFrame({
    'heart_rate': np.random.normal(75, 10, 60).round(0)
}, index=pd.date_range('2024-01-15 08:00', periods=60, freq='30s'))

TEXT 📖 Display only

> **Output:** Run in your local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

(2) The Solution: rolling with Multiple Aggregations

▶ Example: rolling Multi-Metric Statistics (Difficulty ⭐)

PYTHON

import pandas as pd
import numpy as np

np.random.seed(42)
hr = pd.DataFrame({
    'heart_rate': np.random.normal(75, 10, 60).round(0)
}, index=pd.date_range('2024-01-15 08:00', periods=60, freq='30s'))

# Rolling with multiple aggregations
stats = hr['heart_rate'].rolling(10).agg(['mean', 'std', 'min', 'max'])
stats.columns = ['hr_ma', 'hr_std', 'hr_min', 'hr_max']
print(stats.dropna().head(5).round(1))

TEXT 📖 Display only

> **Output:** Run in your local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

3. Advanced rolling

(1) Key Parameters

▶ Example

TEXT 📖 Display only

> **Output:** Run in your local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

: rolling Parameter Reference (Difficulty ⭐⭐)

PYTHON

import pandas as pd
import numpy as np

np.random.seed(42)
s = pd.Series(np.random.randn(20).cumsum() + 100)

# min_periods: allow partial windows
print(s.rolling(5, min_periods=2).mean().head(5))

# center: centered window (for smoothing)
print(s.rolling(5, center=True).mean().head(5))

# win_type: weighted window (Gaussian, triangular, etc.)
print(s.rolling(5, win_type='gaussian').mean(std=1.5).head(8))

# on: use a column as time (when not the index)
df = pd.DataFrame({
    'timestamp': pd.date_range('2024-01', periods=20, freq='6h'),
    'value': np.random.randn(20).cumsum()
})
print(df.rolling('12h', on='timestamp')['value'].mean().head(5))

TEXT 📖 Display only

> **Output:** Run in your local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

(2) rolling Parameter Quick Reference

Parameter	Default	Description
window	Required	Window size (integer row count or time string)
min_periods	window	Minimum observations required in the window
center	False	Center the window
win_type	None	Weighting type (gaussian/triangular, etc.)
on	None	Use a column as the time axis (instead of Index)
closed	'right'	Which end of the window is inclusive

4. Custom Window Aggregation

▶ Example

TEXT 📖 Display only

> **Output:** Run in your local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

: Custom Aggregation Functions (Difficulty ⭐⭐⭐)

PYTHON

import pandas as pd
import numpy as np

np.random.seed(42)
df = pd.DataFrame({
    'value': np.random.randint(10, 100, 20)
})

# Custom: range (max - min) in window
def window_range(arr):
    return arr.max() - arr.min()

df['rolling_range'] = df['value'].rolling(5).apply(window_range, raw=True)

# Custom: coefficient of variation (std/mean)
def cv(arr):
    return arr.std() / arr.mean() if arr.mean() != 0 else 0

df['rolling_cv'] = df['value'].rolling(5).apply(cv, raw=True)

# Using rolling.apply with raw=True for performance
# raw=True: receives numpy array (fast)
# raw=False: receives Series (slow but flexible)

print(df.head(10))

TEXT 📖 Display only

> **Output:** Run in your local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

💡 Tip: Using raw=True in custom functions is about 10x faster (receives a NumPy array instead of a Series), but you lose access to Index information. Use raw=True for simple numerical computations; use raw=False when you need the Index.

5. expanding In Depth

▶ Example

TEXT 📖 Display only

> **Output:** Run in your local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

: Custom expanding Functions (Difficulty ⭐)

PYTHON

import pandas as pd
import numpy as np

np.random.seed(42)
df = pd.DataFrame({
    'score': np.random.randint(60, 100, 15)
})

# Built-in expanding methods
df['cummean'] = df['score'].expanding().mean()
df['cumstd'] = df['score'].expanding().std()
df['cummax'] = df['score'].expanding().max()

# Expanding with min_periods
df['cummean_5'] = df['score'].expanding(min_periods=5).mean()

# Custom: expanding z-score
def zscore(arr):
    if len(arr) < 2: return np.nan
    return (arr[-1] - arr.mean()) / arr.std()

df['zscore'] = df['score'].expanding().apply(zscore, raw=True)
print(df.round(2).head(10))

TEXT 📖 Display only

> **Output:** Run in your local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

6. ewm Parameters Explained

(1) The Four Parameter Relationships

Parameter	Relationship	Meaning
span	com = span - 1	Similar to an N-period SMA
com	α = 1/(1+com)	Center of mass
halflife	α = 1 - 0.5^(1/halflife)	Number of periods for weight to halve
alpha	Specified directly	Smoothing factor 0<α≤1

▶ Example

TEXT 📖 Display only

> **Output:** Run in your local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

: ewm Parameter Comparison (Difficulty ⭐⭐)

PYTHON

import pandas as pd
import numpy as np

np.random.seed(42)
s = pd.Series(np.random.randn(30).cumsum() + 100)

# These are all equivalent (approximately)
ewm_span5 = s.ewm(span=5).mean()
ewm_com4 = s.ewm(com=4).mean()          # com = span - 1
ewm_hl3 = s.ewm(halflife=3).mean()      # different weighting
ewm_alpha = s.ewm(alpha=0.333).mean()   # alpha = 2/(span+1)

# Compare with SMA
sma5 = s.rolling(5).mean()

print("SMA vs EWM (first 10):")
print(pd.DataFrame({
    'value': s, 'SMA_5': sma5, 'EWM_span5': ewm_span5
}).head(10).round(2))

# ewm std (volatility)
ewm_std = s.ewm(span=10).std()
print(f"\nLatest EWM std: {ewm_std.iloc[-1]:.2f}")

TEXT 📖 Display only

> **Output:** Run in your local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

7. groupby + rolling

▶ Example

TEXT 📖 Display only

> **Output:** Run in your local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

: Rolling Windows Within Groups (Difficulty ⭐⭐⭐)

PYTHON

import pandas as pd
import numpy as np

# Sensor data: multiple sensors, each with its own time series
np.random.seed(42)
df = pd.DataFrame({
    'sensor_id': np.repeat(['T01', 'T02', 'T03'], 20),
    'timestamp': list(pd.date_range('2024-01', periods=20, freq='6h')) * 3,
    'temperature': np.random.normal(20, 3, 60).round(1)
})

# Rolling within each sensor group
df = df.sort_values(['sensor_id', 'timestamp'])
df['temp_ma'] = df.groupby('sensor_id')['temperature'].transform(
    lambda x: x.rolling(4, min_periods=1).mean()
)

# Alternative: groupby + rolling (returns MultiIndex)
result = df.groupby('sensor_id')['temperature'].rolling(4).mean()
print(result.head(12))

# Reset for clean output
df['temp_ma2'] = df.groupby('sensor_id')['temperature'].transform(
    lambda x: x.rolling(4).mean().values
)
print(df[df['sensor_id'] == 'T01'].head(6))

TEXT 📖 Display only

> **Output:** Run in your local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

8. Complete Example: Exercise Heart Rate Data Analysis

(5) ▶ Comparing Three Window Types

graph LR
    A[Window Functions] --> B[rolling Fixed Window]
    A --> C[expanding Cumulative Window]
    A --> D[ewm Exponential Weighting]
    B --> E[Local Trends / Moving Average]
    C --> F[Cumulative Stats / From Start to Now]
    D --> G[Recent Data Weighted More / Smoothing]

TEXT 📖 Display only

> **Output:** Run in your local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

▶ Example

TEXT 📖 Display only

> **Output:** Run in your local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

: Full Window Analysis Pipeline (Difficulty ⭐⭐⭐)

PYTHON

import pandas as pd
import numpy as np

# ============================================
# Comprehensive example: Fitness heart rate
# window analysis pipeline
# ============================================

# 1. Generate 2-hour workout data (1 reading per 30s = 240 rows)
np.random.seed(42)
timestamps = pd.date_range('2024-01-15 08:00', periods=240, freq='30s')
exercise_type = (['Warmup'] * 30 + ['Cardio'] * 90 + ['Weights'] * 60 + ['Cooldown'] * 60)
hr_base = ([70] * 30 + list(np.linspace(70, 140, 90)) + [120] * 60 + list(np.linspace(120, 70, 60)))
heart_rates = [max(50, h + np.random.randn() * 5) for h in hr_base]

df = pd.DataFrame({
    'timestamp': timestamps,
    'exercise': exercise_type,
    'heart_rate': np.round(heart_rates, 0)
})

# 2. Rolling: 1-minute (2 readings) and 5-minute (10 readings) moving avg
df['hr_ma_1min'] = df['heart_rate'].rolling(2).mean()
df['hr_ma_5min'] = df['heart_rate'].rolling(10).mean()

# 3. Rolling: heart rate variability (std in 5-min window)
df['hrv'] = df['heart_rate'].rolling(10).std()

# 4. Expanding: cumulative average
df['cum_avg_hr'] = df['heart_rate'].expanding().mean()

# 5. EWM: smoothed trend
df['hr_ewm'] = df['heart_rate'].ewm(span=10).mean()

# 6. Groupby + rolling: per exercise type
df['hr_ma_per_exercise'] = df.groupby('exercise')['heart_rate'].transform(
    lambda x: x.rolling(5, min_periods=1).mean()
)

# 7. Summary by exercise
print("=== Exercise Summary ===")
summary = df.groupby('exercise')['heart_rate'].agg(['mean', 'max', 'min', 'std']).round(1)
print(summary)

TEXT 📖 Display only

> **Output:** Run in your local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.

❓ FAQ

Q Does rolling support time-based windows?

A Yes — rolling('7D') uses a 7-day calendar window, while rolling(7) uses a 7-row window. Time-based windows are ideal for non-uniformly sampled data (skipping weekends or missing days), while row-based windows suit uniform sampling. Time-based windows require the Index to be a DatetimeIndex, or you can specify a time column with the on parameter.

Q How is the performance of custom functions?

A Custom functions are 10-100x slower than built-in aggregations — a Python function call is made for every window. Optimization tips: use raw=True to pass a NumPy array (5-10x faster), and replace Python loops with NumPy operations. Built-in functions are implemented in C, while custom functions go through the Python interpreter. If a built-in function does the job, avoid writing a custom one.

Q What is the relationship between ewm's span/halflife/com?

A All three are mathematically equivalent — just different ways of expressing the same thing. span=N behaves like an N-period SMA but with more weight on recent data; com=span-1 is the center of mass; halflife is the number of periods for the weight to halve. span is the most commonly used (most intuitive). α=2/(span+1) is the smoothing factor. You only need to specify one — there is no need to set multiple parameters at once.

Q What should I watch out for with groupby+rolling?

A Two key points: ① Always sort_values by the group key + time first, otherwise the window may span across groups; ② transform(lambda x: x.rolling(N).mean()) returns a result with the same length as the original DataFrame, while calling groupby+rolling directly returns a MultiIndex. For large datasets, transform is more convenient.

Q What win_type options are available?

A All window functions from scipy.signal.windows: gaussian (Gaussian weighting), triang (triangular), blackman, hamming, kaiser, and more. Requires pip install scipy. win_type applies weights to data within the window (higher weight in the middle, lower at the edges), producing smoother results than an equal-weight window.

Q What does the closed parameter do?

A closed controls which end of the window is inclusive — 'right' (default, includes the right end = current row), 'left' (includes the left end), 'both' (both ends), 'neither' (neither end). In finance, 'right' is commonly used (only historical data), while certain statistical scenarios require 'both' or 'left'.

📖 Summary

Advanced rolling: min_periods allows partial windows, center centers the window, win_type applies weighting, on specifies a time column
Custom aggregation uses .apply(func, raw=True); raw=True is about 10x faster
expanding accumulates from the beginning to the current position and supports any aggregation function
ewm parameters: span (N periods) / com (N-1) / halflife / alpha — all four are equivalent
groupby+rolling: sort first, then compute; transform returns a same-length result
Choosing a window type: local trends → rolling, cumulative → expanding, smoothing → ewm

📝 Exercises

Basic (Difficulty ⭐): Create 30 days of temperature data. Use rolling(7, min_periods=3) to compute the moving mean and standard deviation, and use ewm(span=7) to compute exponential smoothing.
Intermediate (Difficulty ⭐⭐): Create multi-sensor temperature data (3 sensors, 20 rows each). Use groupby+rolling to compute a 5-period moving average for each sensor.
Challenge (Difficulty ⭐⭐⭐): Simulate 240 rows of exercise heart rate data (with 4 exercise types). Complete the following pipeline: rolling multi-metric (mean/std/max) → expanding cumulative average → ewm smoothing → groupby+rolling by exercise type → summary report.

← Previous Lesson: MultiIndex · Next Lesson: Visualization →

Web-Tutorial Tech Team

A team of developers maintaining programming tutorials. Each tutorial is written and reviewed by developers with expertise in that field. We work to keep our content accurate and reliable — if you spot an issue, please let us know.

About Us

Pandas: Window Functions

1. What You Will Learn

2. Carol's Exercise Heart Rate Window Analysis

(1) The Problem: A 30-Second Window Mean Is Not Enough

(2) The Solution: rolling with Multiple Aggregations

▶ Example: rolling Multi-Metric Statistics (Difficulty ⭐)

3. Advanced rolling

(1) Key Parameters

▶ Example

(2) rolling Parameter Quick Reference

4. Custom Window Aggregation

▶ Example

5. expanding In Depth

▶ Example

6. ewm Parameters Explained

(1) The Four Parameter Relationships

▶ Example

7. groupby + rolling

▶ Example

8. Complete Example: Exercise Heart Rate Data Analysis

(5) ▶ Comparing Three Window Types

▶ Example

❓ FAQ

📖 Summary

📝 Exercises

🙏 帮我们做得更好