Capstone Exercise
You've spent five lessons learning the tools. Now it's time to put them all together and complete a real ML project.
1. What You'll Learn
- End-to-end workflow: Data Loading → EDA → Feature Selection → Baseline Model → Evaluation
- EDA in practice: Distribution, correlation, and trend analysis on SalesPredict e-commerce data
- Baseline model construction: LinearRegression as a sales prediction benchmark
- Interpreting results: Computing R², MAE, and RMSE — and what they mean for the business
- Project retrospective: Key takeaways from the beginner stage and directions for growth
2. A Real Story from an ML Newbie
(1) The Pain Point: Knowing All the Tools but Not How to Connect Them
Bob finished learning NumPy, Pandas, Seaborn, and Sklearn. But when he faced the real SalesPredict dataset, he had no idea where to start — clean first or visualize first? Which model to use? How do you know if a model is any good? He knew the tools, but the project workflow was a mystery.
(2) The Solution: An End-to-End Workflow
Every ML project follows a fixed pipeline: Load → Explore → Clean → Feature → Baseline → Evaluate → Iterate. Get the baseline running first, then optimize step by step.
PYTHON
# End-to-end ML workflow template
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, r2_score
# Step 1: Load & Explore
df = pd.read_csv("sales_data.csv")
print(df.describe())
# Step 2: Clean & Prepare features
features = ["ad_spend", "traffic", "last_month_sales"]
target = "monthly_revenue"
df = df.dropna(subset=features + [target])
# Step 3: Train baseline
X_train, X_test, y_train, y_test = train_test_split(df[features], df[target], test_size=0.2)
model = LinearRegression().fit(X_train, y_train)
pred = model.predict(X_test)
# Step 4: Evaluate
print(f"R²: {r2_score(y_test, pred):.3f}, MAE: {mean_absolute_error(y_test, pred):.0f} USD")
(3) The Payoff: First End-to-End Model in 30 Minutes
Following the standard workflow, Bob had his first baseline model running in just 30 minutes. The R² was only 0.72, but with a baseline in place, every subsequent improvement has a reference point.
3. The End-to-End ML Workflow
(1) Full Project Pipeline
graph LR
A[1. Load Data] --> B[2. EDA]
B --> C[3. Clean Data]
C --> D[4. Feature Selection]
D --> E[5. Train Baseline]
E --> F[6. Evaluate]
F --> G{Good Enough?}
G -->|No| H[7. Iterate<br/>Better Features / Model]
H --> D
G -->|Yes| I[8. Deploy]
▶ Example: Generating Simulated SalesPredict Data
PYTHON
import pandas as pd
import numpy as np
rng = np.random.default_rng(42)
n = 1000
df = pd.DataFrame({
"date": pd.date_range("2023-01-01", periods=n, freq="D"),
"ad_spend_k": rng.uniform(5, 100, n),
"traffic_k": rng.uniform(10, 500, n),
"category": rng.choice(["Electronics", "Clothing", "Food", "Books", "Home"], n),
"is_promotion": rng.choice([0, 1], n, p=[0.8, 0.2]),
"customer_count": rng.integers(50, 500, n),
})
# Revenue formula with realistic relationships
df["revenue_k"] = (
20
+ 0.6 * df["ad_spend_k"]
+ 0.08 * df["traffic_k"]
+ 0.5 * df["customer_count"]
+ 50 * df["is_promotion"]
+ rng.normal(0, 15, n)
)
df["revenue_k"] = df["revenue_k"].clip(lower=0)
# Inject some missing values and outliers
missing_idx = rng.choice(n, size=50, replace=False)
df.loc[missing_idx[:25], "ad_spend_k"] = np.nan
df.loc[missing_idx[25:], "traffic_k"] = np.nan
outlier_idx = rng.choice(n, size=10, replace=False)
df.loc[outlier_idx, "revenue_k"] *= 5
print(f"Dataset shape: {df.shape}")
print(f"Missing values:\n{df.isnull().sum()}")
df.to_csv("salespredict_data.csv", index=False)
Output:
TEXT
# Executed successfully
4. Exploratory Data Analysis (EDA)
(1) Data Overview
▶ Example: Comprehensive EDA
PYTHON
import pandas as pd
import numpy as np
df = pd.read_csv("salespredict_data.csv", parse_dates=["date"])
# Basic overview
print("=" * 50)
print("DATA OVERVIEW")
print("=" * 50)
print(f"Shape: {df.shape}")
print(f"\nDtypes:\n{df.dtypes}")
print(f"\nBasic Stats:\n{df.describe().round(2)}")
print(f"\nMissing Values:\n{df.isnull().sum()}")
print(f"\nCategory Distribution:\n{df['category'].value_counts()}")
Output:
TEXT
=
DATA OVERVIEW
=
(2) Distribution and Trend Analysis
▶ Example: Visual EDA
PYTHON
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
df = pd.read_csv("salespredict_data.csv", parse_dates=["date"])
sns.set_theme(style="whitegrid")
fig, axes = plt.subplots(2, 3, figsize=(18, 10))
# (1) Revenue distribution
sns.histplot(df["revenue_k"], bins=40, kde=True, ax=axes[0, 0], color="#2196F3")
axes[0, 0].set_title("Revenue Distribution (thousand USD)")
# (2) Revenue by category
sns.boxplot(data=df, x="category", y="revenue_k", ax=axes[0, 1], palette="Set2")
axes[0, 1].set_title("Revenue by Category")
# (3) Promotion effect
sns.barplot(data=df, x="is_promotion", y="revenue_k", ax=axes[0, 2], palette="Pastel1")
axes[0, 2].set_title("Promotion Effect on Revenue")
# (4) Ad spend vs Revenue scatter
axes[1, 0].scatter(df["ad_spend_k"], df["revenue_k"], alpha=0.3, s=10)
axes[1, 0].set_xlabel("Ad Spend (thousand USD)")
axes[1, 0].set_ylabel("Revenue (thousand USD)")
axes[1, 0].set_title("Ad Spend vs Revenue")
# (5) Traffic vs Revenue scatter
axes[1, 1].scatter(df["traffic_k"], df["revenue_k"], alpha=0.3, s=10, color="#4CAF50")
axes[1, 1].set_xlabel("Traffic (thousand)")
axes[1, 1].set_ylabel("Revenue (thousand USD)")
axes[1, 1].set_title("Traffic vs Revenue")
# (6) Correlation heatmap
num_cols = ["ad_spend_k", "traffic_k", "customer_count", "is_promotion", "revenue_k"]
corr = df[num_cols].corr()
sns.heatmap(corr, annot=True, fmt=".2f", cmap="RdBu_r", ax=axes[1, 2], vmin=-1, vmax=1)
axes[1, 2].set_title("Feature Correlations")
plt.tight_layout()
plt.savefig("eda_overview.png", dpi=150)
Output:
TEXT
# Executed successfully
(3) EDA Key Findings Template
| Finding # | Finding | Business Impact | Next Action |
|---|---|---|---|
| 1 | Promotion days average 50k USD higher revenue | Promotions are highly effective | Keep is_promotion feature |
| 2 | ad_spend and revenue correlation = 0.72 | Ads drive sales | Core feature |
| 3 | 10 extreme high-value outliers | Likely data entry errors | Needs cleaning |
| 4 | 5% of data has missing values | Affects model training | Impute or drop |
5. Data Cleaning and Feature Preparation
▶ Example: Data Cleaning Pipeline
PYTHON
import pandas as pd
import numpy as np
df = pd.read_csv("salespredict_data.csv", parse_dates=["date"])
# Step 1: Handle missing values
print(f"Before cleaning: {len(df)} rows")
df = df.dropna(subset=["revenue_k"]) # Drop if target is missing
df["ad_spend_k"] = df["ad_spend_k"].fillna(df["ad_spend_k"].median())
df["traffic_k"] = df["traffic_k"].fillna(df["traffic_k"].median())
# Step 2: Remove outliers using IQR
def remove_outliers(df, col, factor=1.5):
q1, q3 = df[col].quantile([0.25, 0.75])
iqr = q3 - q1
return df[(df[col] >= q1 - factor * iqr) & (df[col] <= q3 + factor * iqr)]
df = remove_outliers(df, "revenue_k")
print(f"After cleaning: {len(df)} rows")
# Step 3: Feature engineering
df["month"] = df["date"].dt.month
df["day_of_week"] = df["date"].dt.dayofweek
df["is_weekend"] = (df["day_of_week"] >= 5).astype(int)
# Step 4: Encode categorical
df_encoded = pd.get_dummies(df, columns=["category"], drop_first=True)
# Step 5: Select features
feature_cols = [c for c in df_encoded.columns if c not in ["date", "revenue_k"]]
X = df_encoded[feature_cols]
y = df_encoded["revenue_k"]
print(f"Features: {feature_cols}")
print(f"X shape: {X.shape}, y shape: {y.shape}")
Output:
TEXT
# Functions defined successfully
6. Baseline Model Construction and Evaluation
▶ Example: Training a Baseline LinearRegression
PYTHON
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
import numpy as np
# Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Baseline pipeline
baseline = Pipeline([
("scaler", StandardScaler()),
("model", LinearRegression()),
])
baseline.fit(X_train, y_train)
y_pred = baseline.predict(X_test)
# Comprehensive evaluation
mae = mean_absolute_error(y_test, y_pred)
rmse = np.sqrt(mean_squared_error(y_test, y_pred))
r2 = r2_score(y_test, y_pred)
mape = np.mean(np.abs((y_test - y_pred) / y_test)) * 100
print("=" * 50)
print("BASELINE MODEL EVALUATION")
print("=" * 50)
print(f"MAE: {mae:.2f} thousand USD")
print(f"RMSE: {rmse:.2f} thousand USD")
print(f"R²: {r2:.4f}")
print(f"MAPE: {mape:.1f}%")
Output:
TEXT
=
BASELINE MODEL EVALUATION
=
▶ Example: Visualizing Prediction Results
PYTHON
import matplotlib.pyplot as plt
import numpy as np
fig, axes = plt.subplots(1, 2, figsize=(14, 5))
# (1) Actual vs Predicted scatter
axes[0].scatter(y_test, y_pred, alpha=0.5, s=20)
axes[0].plot([y_test.min(), y_test.max()], [y_test.min(), y_test.max()], "r--", lw=2)
axes[0].set_xlabel("Actual Revenue (thousand USD)")
axes[0].set_ylabel("Predicted Revenue (thousand USD)")
axes[0].set_title("Actual vs Predicted")
# (2) Residual distribution
residuals = y_test - y_pred
axes[1].hist(residuals, bins=30, edgecolor="white", color="#2196F3")
axes[1].axvline(x=0, color="red", linestyle="--")
axes[1].set_xlabel("Residual (thousand USD)")
axes[1].set_title("Residual Distribution")
plt.tight_layout()
plt.savefig("baseline_evaluation.png", dpi=150)
Output:
TEXT
# Executed successfully
(1) Interpreting Business Metrics
| Metric | Baseline Value | Business Meaning | Target |
|---|---|---|---|
| MAE | ~12k USD | Average prediction off by $12k | < 10k |
| RMSE | ~15k USD | Large errors are penalized more | < 12k |
| R² | ~0.72 | Explains 72% of variance | > 0.85 |
| MAPE | ~15% | Average percentage error | < 10% |
📌 Key Point: A MAPE of 15% means that when Bob predicts $1M in monthly sales, the error could be as high as $150k — directly leading to overstock or stockouts. The advanced models in Phases 2–3 will bring this error below 8%.
❓ FAQ
Q Why use LinearRegression for the baseline instead of a more complex model?
A The purpose of a baseline is to establish a performance floor and understand the inherent predictability of the data. Complex models can overfit and mask the true signal. Start simple to set a benchmark, then improve incrementally.
Q Should I look at MAE or RMSE?
A Both. MAE is robust to outliers and reflects average performance; RMSE penalizes large errors more heavily and reflects worst-case scenarios. If RMSE is much larger than MAE, it means there are a few large individual errors.
Q Is a MAPE of 15% good or bad?
A It depends on the domain. For e-commerce sales forecasting, MAPE < 10% is production-grade; 15% is baseline-level. But for financial time-series, MAPE < 2% is considered good. The key is to tie it back to business cost.
Q How much time should I spend on EDA?
A For a new dataset, EDA should take 20–30% of total project time. Don't skip EDA and jump straight to modeling — you might miss critical features or data issues.
Q Should I impute missing values with the median or the mean?
A Use the median when outliers are present (it's robust); use the mean when the distribution is approximately normal. You can also use more advanced methods like KNNImputer or IterativeImputer.
Q Should I remove outliers?
A Distinguish between "true anomalies" and "false anomalies." Data entry errors → remove. Real extreme events (like Black Friday) → keep them and flag with a special feature. Blindly removing outliers loses information.
📖 Summary
- Standard ML project pipeline: Load → EDA → Clean → Feature → Baseline → Evaluate → Iterate
- Core goals of EDA: Understand distributions, spot anomalies, find strongly correlated features, validate business assumptions
- Four steps of data cleaning: Handle missing values → Handle outliers → Type conversion → Feature encoding
- Use LinearRegression + Pipeline for the baseline to establish a performance benchmark
- Tie evaluation metrics to business meaning: MAPE of 15% means a $150k error on a million-dollar forecast
- Beginner-stage takeaways: Toolchain connected, project workflow established, baseline built — now iterate and improve
📝 Exercises
- Basic (Difficulty ⭐): Use your own data (or a built-in sklearn dataset like housing) to complete the full pipeline: Load → EDA → Baseline Model → Evaluation. Hint: Refer to the code templates in Sections 4–6.
- Intermediate (Difficulty ⭐⭐): On top of the baseline model, add 2 new features (e.g., an
ad_spend * is_promotioninteraction term) and compare whether R² and MAPE improve. Hint: Usedf.assign()to add new columns. - Challenge (Difficulty ⭐⭐⭐): Implement a simple model comparison framework — train LinearRegression, DecisionTreeRegressor, and RandomForestRegressor simultaneously, and output a comparison table of MAE/RMSE/R²/MAPE for each model. Hint: Loop over models using a dictionary and collect results into a DataFrame.
← Previous: Getting Started with Scikit-learn | Next: Linear Regression →
