Getting Started with Scikit-learn
Scikit-learn's unified API means every algorithm is called the same way — learn one, and you know them all.
1. What You'll Learn
- Scikit-learn's design philosophy and unified API: fit → predict → score
- The three core interfaces: Estimator, Transformer, and Predictor
- Your first model: train a decision tree classifier on the Iris dataset, walking through the full train-test-split → fit → predict → evaluate workflow
- Data preprocessing Pipelines: chaining StandardScaler + model in a single object
- Bob's first attempt: using sklearn to make simple predictions on the SalesPredict small-sample dataset
2. A Real Story from an ML Beginner
(1) The Pain Point: Every Library Has a Different API
Bob tried three different ML libraries, and each one had its own calling convention — one used train(), another used fit_model(), and yet another used learn(). He spent two days reading documentation just to figure out how to call them, and switching to a different algorithm meant starting over from scratch. Inconsistent APIs are the biggest barrier to getting started with ML.
(2) Scikit-learn's Solution
Scikit-learn defines a unified API design: every algorithm follows the fit → predict → score pattern. Switching algorithms only requires changing a single line.
PYTHON
from sklearn.linear_model import LinearRegression
from sklearn.ensemble import RandomForestRegressor
# Same API, different algorithm - just change the model class
models = {
"LinearRegression": LinearRegression(),
"RandomForest": RandomForestRegressor(n_estimators=100),
}
for name, model in models.items():
model.fit(X_train, y_train)
score = model.score(X_test, y_test)
print(f"{name}: R² = {score:.3f}")
(3) The Payoff: One API Covers 30+ Algorithms
Once Bob mastered sklearn's unified API, the calling convention for 30+ algorithms became identical — from LinearRegression to XGBoost, you just swap out the class name.
3. Scikit-learn Design Philosophy and the Unified API
(1) The Unified API Pattern
graph LR
A[Estimator<br/>Base Class] --> B[Transformer<br/>fit + transform]
A --> C[Predictor<br/>fit + predict]
A --> D[Model<br/>fit + predict + score]
B --> E[StandardScaler<br/>PCA<br/>OneHotEncoder]
C --> F[KMeans<br/>DBSCAN]
D --> G[LinearRegression<br/>RandomForest<br/>SVM]
(2) The Three Core Interfaces
| Interface | Core Methods | Purpose | Examples |
|---|---|---|---|
| Estimator | fit(X, y) |
Learn parameters from data | Base class for all models |
| Transformer | fit() + transform() + fit_transform() |
Data preprocessing | StandardScaler, PCA |
| Predictor | fit() + predict() |
Generate predictions | LinearRegression, SVM |
▶ Example: Estimator/Transformer/Predictor Usage Patterns
PYTHON
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
import numpy as np
# Generate sample data
rng = np.random.default_rng(42)
X = rng.uniform(0, 100, (200, 3))
y = 50 + 0.8 * X[:, 0] + 1.2 * X[:, 1] - 0.5 * X[:, 2] + rng.normal(0, 5, 200)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Transformer: StandardScaler
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train) # fit + transform in one step
X_test_scaled = scaler.transform(X_test) # only transform (use training stats)
# Predictor: LinearRegression
model = LinearRegression()
model.fit(X_train_scaled, y_train)
predictions = model.predict(X_test_scaled)
score = model.score(X_test_scaled, y_test)
print(f"R² score: {score:.4f}")
print(f"Coefficients: {model.coef_}")
print(f"Intercept: {model.intercept_:.2f}")
Output:
TEXT
# Executed successfully
4. Your First ML Model: Iris Classification
(1) The Complete ML Workflow
▶ Example: Full Iris Decision Tree Classification Pipeline
PYTHON
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix
# Step 1: Load data
iris = load_iris()
X, y = iris.data, iris.target
print(f"Features: {iris.feature_names}")
print(f"Classes: {iris.target_names}")
print(f"Shape: {X.shape}")
# Step 2: Split train/test
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.3, random_state=42, stratify=y
)
print(f"Train: {X_train.shape[0]}, Test: {X_test.shape[0]}")
# Step 3: Train model
model = DecisionTreeClassifier(max_depth=3, random_state=42)
model.fit(X_train, y_train)
# Step 4: Predict
y_pred = model.predict(X_test)
# Step 5: Evaluate
acc = accuracy_score(y_test, y_pred)
print(f"\nAccuracy: {acc:.4f}")
print(f"\nClassification Report:\n{classification_report(y_test, y_pred, target_names=iris.target_names)}")
print(f"Confusion Matrix:\n{confusion_matrix(y_test, y_pred)}")
Output:
TEXT
Features: ['sepal length (cm)', 'sepal width (cm)', ...]
Classes: ['setosa' 'versicolor' 'virginica']
Shape: (150, 4)
Train: 105, Test: 45
Accuracy: 1.0000
▶ Example: Visualizing the Decision Tree
PYTHON
from sklearn.tree import plot_tree
import matplotlib.pyplot as plt
fig, ax = plt.subplots(figsize=(12, 8))
plot_tree(model, feature_names=iris.feature_names,
class_names=iris.target_names, filled=True, ax=ax)
plt.title("Iris Decision Tree (max_depth=3)")
plt.tight_layout()
plt.savefig("iris_tree.png", dpi=150)
Output:
TEXT
# Executed successfully
(2) Understanding train_test_split in Detail
| Parameter | Meaning | Recommended Value |
|---|---|---|
| test_size | Proportion for the test set | 0.2–0.3 |
| random_state | Random seed | 42 (for reproducibility) |
| stratify | Stratified sampling | Required for classification tasks |
| shuffle | Whether to shuffle data | True by default |
5. Data Preprocessing Pipelines
(1) Why You Need a Pipeline
A Pipeline binds preprocessing and the model together as a single unit, preventing data leakage (where test set information leaks into the training process).
▶ Example: A Pipeline with StandardScaler + Model
PYTHON
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.datasets import load_iris
iris = load_iris()
X, y = iris.data, iris.target
# Build pipeline
pipe = Pipeline([
("scaler", StandardScaler()),
("classifier", LogisticRegression(max_iter=200)),
])
# Cross-validation with pipeline (no data leakage!)
scores = cross_val_score(pipe, X, y, cv=5, scoring="accuracy")
print(f"CV Accuracy: {scores.mean():.4f} +/- {scores.std():.4f}")
# Fit on full data and predict
pipe.fit(X, y)
new_sample = [[5.1, 3.5, 1.4, 0.2]]
prediction = pipe.predict(new_sample)
print(f"Prediction: {iris.target_names[prediction[0]]}")
Output:
TEXT
# Executed successfully
(2) Pipeline vs. Manual Preprocessing
| Aspect | Manual Preprocessing | Pipeline |
|---|---|---|
| Data leakage risk | High (easy to accidentally fit the scaler on test data) | Low (Pipeline isolates automatically) |
| Code readability | Scattered across multiple steps | Consolidated in one object |
| Cross-validation | Requires manual looping | cross_val_score(pipe) |
| Deployment convenience | Must save scaler and model separately | Save a single pipeline object |
▶ Example: SalesPredict Baseline Pipeline
PYTHON
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
import pandas as pd
import numpy as np
# Simulate SalesPredict data
rng = np.random.default_rng(42)
n = 500
df = pd.DataFrame({
"ad_spend": rng.uniform(10, 100, n),
"traffic": rng.uniform(100, 1000, n),
"category": rng.choice(["Elec", "Cloth", "Food"], n),
"is_weekend": rng.choice([0, 1], n),
})
df["revenue"] = 50 + 0.8 * df["ad_spend"] + 0.1 * df["traffic"] + rng.normal(0, 10, n)
# Define feature groups
num_features = ["ad_spend", "traffic"]
cat_features = ["category", "is_weekend"]
# ColumnTransformer for mixed preprocessing
preprocessor = ColumnTransformer([
("num", StandardScaler(), num_features),
("cat", OneHotEncoder(drop="first"), cat_features),
])
# Full pipeline
pipe = Pipeline([
("preprocessor", preprocessor),
("model", LinearRegression()),
])
X = df.drop(columns=["revenue"])
y = df["revenue"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
pipe.fit(X_train, y_train)
score = pipe.score(X_test, y_test)
print(f"SalesPredict Baseline R²: {score:.4f}")
# Predict
sample = pd.DataFrame({"ad_spend": [50], "traffic": [500], "category": ["Elec"], "is_weekend": [1]})
pred = pipe.predict(sample)
print(f"Predicted revenue: {pred[0]:.1f} thousand USD")
Output:
TEXT
# Executed successfully
6. Model Evaluation Fundamentals
(1) Classification vs. Regression Metrics
| Task | Metric | Meaning | sklearn Function |
|---|---|---|---|
| Classification | Accuracy | Proportion of correct predictions | accuracy_score |
| Classification | Precision | Proportion of true positives among predicted positives | precision_score |
| Classification | Recall | Proportion of true positives among actual positives | recall_score |
| Classification | F1 | Harmonic mean of precision and recall | f1_score |
| Regression | MAE | Mean Absolute Error | mean_absolute_error |
| Regression | MSE | Mean Squared Error | mean_squared_error |
| Regression | R² | Coefficient of determination | r2_score |
▶ Example: Comprehensive Regression Model Evaluation
PYTHON
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
import numpy as np
y_true = np.array([100, 150, 200, 250, 300])
y_pred = np.array([95, 160, 190, 260, 310])
mae = mean_absolute_error(y_true, y_pred)
mse = mean_squared_error(y_true, y_pred)
rmse = np.sqrt(mse)
r2 = r2_score(y_true, y_pred)
mape = np.mean(np.abs((y_true - y_pred) / y_true)) * 100
print(f"MAE: {mae:.2f} thousand USD")
print(f"RMSE: {rmse:.2f} thousand USD")
print(f"R²: {r2:.4f}")
print(f"MAPE: {mape:.1f}%")
Output:
TEXT
# Executed successfully
❓ FAQ
Q What's the difference between fit_transform and transform?
A fit_transform = fit + transform. Use it on the training set (learns parameters and applies the transformation). transform only applies already-learned parameters and should be used on the test set. Never call fit on the test set — that causes data leakage.
Q What are the benefits of using a Pipeline?
A Three key benefits — 1) Prevents data leakage (automatically isolates fit/transform during cross-validation); 2) Cleaner code (one object contains all steps); 3) Easier deployment (save a single pickle file).
Q What does random_state=42 mean?
A It sets the random seed to ensure reproducibility. 42 is a programming culture reference to "the answer" from The Hitchhiker's Guide to the Galaxy — it has no effect on result quality.
Q When is the stratify parameter required?
A It's required for classification tasks. stratify=y ensures that the class distribution in both the training and test sets matches the original data, preventing minority classes from ending up entirely in the test set.
Q What does cv=5 mean in cross_val_score?
A It means 5-fold cross-validation — the data is split into 5 folds, and the model is trained on 4 folds and validated on the remaining fold in rotation. The final score is the average across all 5 folds. This is more reliable than a single train_test_split.
Q What's the difference between ColumnTransformer and Pipeline?
A Pipeline chains steps sequentially (the output of one step becomes the input of the next), while ColumnTransformer applies different transformers to different columns in parallel and then concatenates the results. They are typically used together.
📖 Summary
- sklearn's unified API: Estimator(fit) → Transformer(fit+transform) → Predictor(fit+predict+score)
- The complete ML workflow: load data → split into train/test → fit to train → predict → evaluate with score
- Pipelines bind preprocessing + model into a single unit, preventing data leakage and simplifying deployment
- ColumnTransformer handles mixed-type features (StandardScaler for numeric columns, OneHotEncoder for categorical columns)
- Classification evaluation uses accuracy/precision/recall/F1; regression evaluation uses MAE/RMSE/R²/MAPE
- cross_val_score performs cross-validation, which is more reliable than a single train/test split
📝 Exercises
- Basic (Difficulty ⭐): Use sklearn to load the Iris dataset, train a LogisticRegression classifier, and print the accuracy. Hint: follow the complete workflow from Section 4.
- Intermediate (Difficulty ⭐⭐): Build a Pipeline (StandardScaler + LogisticRegression) and use 5-fold cross-validation to compare accuracy with and without the scaler. Hint: run cross_val_score on the Pipeline vs. the standalone model.
- Challenge (Difficulty ⭐⭐⭐): Use ColumnTransformer to build a mixed-type Pipeline for the SalesPredict data (standardize numeric columns + one-hot encode categorical columns), then train a LinearRegression model and compute R² and MAE. Hint: refer to the SalesPredict Pipeline example in Section 5.
← Previous: Data Visualization with Matplotlib and Seaborn | Next: Comprehensive Exercise — Beginner Project →
