Pandas: Date and Time
Last updated: 2026-08-26
The first step in time series analysis is converting string dates into Pandas datetime types. DatetimeIndex unlocks aggregation by week, month, or quarter; the dt accessor lets you decompose dates just like splitting strings; and Timedelta makes date arithmetic straightforward. This section covers the datetime type system, parsing, extraction, and arithmetic.
1. What You Will Learn
- ❶ Timestamp / DatetimeIndex
- ❷ pd.to_datetime parsing
- ❸ .dt accessor
- ❹ Timedelta arithmetic
- ❺ date_range and frequency
2. Charlie's Weather Station Time Dilemma
(1) Pain Point: Dates Are Strings, Making Monthly Aggregation Impossible
Charlie's weather station data stores dates as strings, making monthly statistics impossible:
import pandas as pd
df = pd.DataFrame({
'date_str': ['2024-01-15', '2024-01-20', '2024-02-10', '2024-02-28', '2024-03-15'],
'temperature': [5.2, 3.8, 8.1, 9.5, 12.3]
})
# df.groupby(df['date_str'].str[:7]) — fragile hack!
> **Output:** Run in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
(2) Solution: to_datetime + dt
▶ Example
> **Output:** Run in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
: to_datetime parsing and extraction (Difficulty ⭐)
import pandas as pd
df = pd.DataFrame({
'date_str': ['2024-01-15', '2024-01-20', '2024-02-10', '2024-02-28', '2024-03-15'],
'temperature': [5.2, 3.8, 8.1, 9.5, 12.3]
})
# Convert string to datetime
df['date'] = pd.to_datetime(df['date_str'])
# Extract month for grouping
df['month'] = df['date'].dt.month
# Group by month
monthly = df.groupby('month')['temperature'].mean()
print(monthly)
# month
# 1 4.50
# 2 8.80
# 3 12.30
> **Output:** Run in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
3. Pandas Datetime Type System
(1) Four Datetime Types
| Type | Purpose | Example |
|---|---|---|
| Timestamp | A single point in time | pd.Timestamp('2024-01-15') |
| DatetimeIndex | A sequence of timestamps | pd.date_range('2024-01', periods=3, freq='M') |
| Timedelta | A duration between two points | pd.Timedelta(days=7) |
| Period | A time span | pd.Period('2024-01', freq='M') |
▶ Example
> **Output:** Run in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
: Creating datetime types (Difficulty ⭐)
import pandas as pd
# Timestamp — single point in time
ts = pd.Timestamp('2024-01-15 10:30:00')
print(ts.year, ts.month, ts.day, ts.hour)
# 2024 1 15 10
# DatetimeIndex — sequence of timestamps
idx = pd.date_range('2024-01-01', periods=5, freq='D')
print(idx)
# DatetimeIndex(['2024-01-01', '2024-01-02', ...], dtype='datetime64[ns]')
# Timedelta — duration
delta = pd.Timedelta(days=7, hours=3)
print(delta) # 7 days 03:00:00
print(ts + delta) # 2024-01-22 13:30:00
# Period — time span (month/quarter/year)
p = pd.Period('2024-Q1', freq='Q')
print(p.start_time, p.end_time)
> **Output:** Run in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
4. pd.to_datetime
(1) Parsing Multiple Formats
▶ Example
> **Output:** Run in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
: to_datetime multi-format parsing (Difficulty ⭐⭐)
import pandas as pd
# ISO format (auto-detected)
dates1 = pd.to_datetime(['2024-01-15', '2024-02-20', '2024-03-10'])
print(dates1)
# Mixed formats
dates2 = pd.to_datetime(['01/15/2024', 'Feb 20, 2024', '20240310'])
print(dates2)
# Specify format explicitly (faster for large data)
dates3 = pd.to_datetime(['15-01-2024', '20-02-2024'], format='%d-%m-%Y')
print(dates3)
# Handle errors
bad_dates = pd.to_datetime(
['2024-01-15', 'not-a-date', '2024-03-10'],
errors='coerce' # invalid → NaT (not errors='raise')
)
print(bad_dates)
# DatetimeIndex(['2024-01-15', 'NaT', '2024-03-10'])
# Unix timestamp
unix_ts = pd.to_datetime([1705276800, 1707974400], unit='s')
print(unix_ts)
> **Output:** Run in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
(2) Common format Codes
| Code | Meaning | Example |
|---|---|---|
| %Y | 4-digit year | 2024 |
| %m | 2-digit month | 01 |
| %d | 2-digit day | 15 |
| %H | 24-hour clock | 14 |
| %M | Minute | 30 |
| %S | Second | 00 |
| %b | Abbreviated month name | Jan |
| %B | Full month name | January |
| %a | Abbreviated weekday | Mon |
5. .dt Accessor
(1) Extracting Datetime Properties
▶ Example
> **Output:** Run in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
: dt accessor properties (Difficulty ⭐⭐)
import pandas as pd
df = pd.DataFrame({
'timestamp': pd.date_range('2024-01-15 08:30:00', periods=5, freq='12h')
})
# Extract date components
df['year'] = df['timestamp'].dt.year
df['month'] = df['timestamp'].dt.month
df['day'] = df['timestamp'].dt.day
df['hour'] = df['timestamp'].dt.hour
df['day_of_week'] = df['timestamp'].dt.dayofweek # 0=Mon, 6=Sun
df['day_name'] = df['timestamp'].dt.day_name() # Monday, Tuesday...
df['is_weekend'] = df['timestamp'].dt.dayofweek >= 5
df['quarter'] = df['timestamp'].dt.quarter
print(df[['timestamp', 'month', 'day_name', 'is_weekend', 'quarter']])
> **Output:** Run in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
(2) dt Methods
▶ Example
> **Output:** Run in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
: dt method operations (Difficulty ⭐)
import pandas as pd
dates = pd.Series(pd.date_range('2024-01-15', periods=5, freq='2D'))
# Convert to period
print(dates.dt.to_period('M')) # monthly periods
# Normalize to midnight (remove time component)
print(dates.dt.normalize())
# Round to nearest hour
timestamps = pd.Series(pd.date_range('2024-01-15 08:35:00', periods=3, freq='45min'))
print(timestamps.dt.round('h')) # round to nearest hour
# Floor / Ceil
print(timestamps.dt.floor('h')) # round down
> **Output:** Run in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
6. Timedelta Arithmetic
(1) Date Addition and Subtraction
▶ Example
> **Output:** Run in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
: Timedelta arithmetic (Difficulty ⭐⭐)
import pandas as pd
# Create timestamps
start = pd.Timestamp('2024-01-15')
deadline = start + pd.Timedelta(days=30)
print(f"Deadline: {deadline}")
# Difference between dates
delivery = pd.Timestamp('2024-02-10')
delay = delivery - deadline
print(f"Delay: {delay.days} days") # -4 days (early)
# Timedelta from string
td = pd.to_timedelta('5 days 3 hours')
print(td)
# Operations on Series
df = pd.DataFrame({
'order_date': pd.date_range('2024-01-01', periods=5),
'delivery_days': [3, 5, 2, 7, 4]
})
df['delivery_date'] = df['order_date'] + pd.to_timedelta(df['delivery_days'], unit='D')
df['is_late'] = df['delivery_days'] > 5
print(df)
> **Output:** Run in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
7. date_range and Frequency
(1) Generating Date Sequences
▶ Example
> **Output:** Run in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
: date_range date generation (Difficulty ⭐⭐)
import pandas as pd
# Fixed number of periods
print(pd.date_range('2024-01-01', periods=5, freq='D'))
# 2024-01-01 to 2024-01-05
# Start + End
print(pd.date_range('2024-01-01', '2024-01-10', freq='2D'))
# 2024-01-01, 2024-01-03, 2024-01-05, 2024-01-07, 2024-01-09
# Business days only
bdays = pd.bdate_range('2024-01-01', periods=5) # skip weekends
print(bdays)
# Monthly frequency
months = pd.date_range('2024-01', periods=4, freq='MS') # Month Start
print(months)
# Hourly
hours = pd.date_range('2024-01-15 09:00', periods=4, freq='h')
print(hours)
> **Output:** Run in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
(2) Frequency Alias Quick Reference
| Alias | Meaning | Example |
|---|---|---|
| D | Calendar day | Every day |
| B | Business day | Skips weekends |
| W | Weekly | Every Sunday |
| MS | Month start | 1st of each month |
| M | Month end | Last day of each month |
| QS | Quarter start | First day of each quarter |
| Q | Quarter end | Last day of each quarter |
| H | Hourly | Every hour |
| T/min | Minutely | Every minute |
8. Complete Example: Coffee Shop Time Series Analysis
(5) ▶ Datetime Type System
graph LR
A[Pandas Datetime Types] --> B[Timestamp Point in Time]
A --> C[DatetimeIndex Time Series]
A --> D[Timedelta Duration]
A --> E[Period Time Span]
B --> F[Single moment]
C --> G[Aggregate by frequency]
D --> H[Date arithmetic]
E --> I[Monthly/quarterly aggregation]
> **Output:** Run in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
▶ Example
> **Output:** Run in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
: Full time series analysis workflow (Difficulty ⭐⭐⭐)
import pandas as pd
import numpy as np
# ============================================
# Comprehensive example: Coffee shop time
# series analysis with datetime operations
# ============================================
# 1. Create daily sales data for 90 days
np.random.seed(42)
df = pd.DataFrame({
'date': pd.date_range('2024-01-01', periods=90, freq='D'),
'sales': np.random.poisson(50, 90) * 10,
'customers': np.random.poisson(30, 90)
})
df = df.set_index('date')
# 2. Extract time features
df['month'] = df.index.month
df['day_of_week'] = df.index.dayofweek
df['day_name'] = df.index.day_name()
df['is_weekend'] = df.index.dayofweek >= 5
df['week_number'] = df.index.isocalendar().week.astype(int)
# 3. Monthly summary
monthly = df.groupby('month').agg(
total_sales=('sales', 'sum'),
avg_customers=('customers', 'mean'),
days=('sales', 'count')
).round(0)
print("=== Monthly Summary ===")
print(monthly)
# 4. Weekday vs Weekend
wknd = df.groupby('is_weekend')['sales'].mean().round(0)
print(f"\n=== Weekday vs Weekend ===\n{wknd}")
# 5. Best day of week
best_day = df.groupby('day_name')['sales'].mean().sort_values(ascending=False)
print(f"\n=== Best Day ===\n{best_day.head(3)}")
# 6. Month-over-month growth
monthly_sales = df.resample('M')['sales'].sum()
mom_growth = monthly_sales.pct_change().round(3) * 100
print(f"\n=== MoM Growth % ===\n{mom_growth}")
> **Output:** Run in a local Python environment (pandas 2.x). The Piston server does not have pandas pre-installed. Please install it locally (`pip install pandas`) and follow along. Actual values may vary slightly depending on the pandas version.
❓ FAQ
pd.to_datetime(s, format='%d/%m/%Y'). Python strftime format codes: %Y for year, %m for month, %d for day, %H for hour, %M for minute, %S for second. Specifying format is 5-10x faster than auto-detection.date_range(freq='D') generates every day, while bdate_range() or freq='B' generates only business days. For monthly frequency, MS = month start, M = month end. Choosing the wrong frequency can lead to very different resample results.df.index.tz_localize('UTC').tz_convert('Asia/Shanghai'). Pandas uses the pytz/dateutil time zone libraries. A common workflow: read UTC timestamps → convert to local time zone → aggregate by local date.📖 Summary
- Pandas has four datetime types: Timestamp (point in time) / DatetimeIndex (time series) / Timedelta (duration) / Period (time span)
- pd.to_datetime parses string dates; the format parameter specifies the format; errors='coerce' handles invalid values gracefully
- The .dt accessor extracts datetime properties (year/month/day/hour/dayofweek/quarter, etc.)
- Timedelta supports date addition and subtraction; pd.to_timedelta creates durations from strings or numbers
- date_range generates date sequences with frequency aliases D/B/W/MS/M/QS/Q/H
- DatetimeIndex as an index enables time series analysis (resample/shift/rolling)
- Use bdate_range or freq='B' for business days; use tz_localize/tz_convert for time zones
📝 Exercises
- Basic (Difficulty ⭐): Create a Series containing date strings, parse it with to_datetime, extract month and dayofweek, and count the number of records per month.
- Intermediate (Difficulty ⭐⭐): Create a 30-day daily sales DataFrame (with DatetimeIndex), use dt to extract is_weekend, compare average sales between weekdays and weekends, and calculate a Timedelta to find how long ago the highest-sales day was.
- Challenge (Difficulty ⭐⭐⭐): Simulate 90 days of coffee shop sales data and complete the following: to_datetime parsing → set_index → dt extraction of month/quarter/weekday → groupby monthly summary → calculate month-over-month growth rate → find the best-performing weekday.
← Previous: String Operations · Next: Advanced Time Series →