Convolutional Neural Networks
CNNs give computers "eyes" — convolution kernels act like visual receptors, recognizing images layer by layer, from edges to objects.
1. What You'll Learn
- Convolution operations: kernels/filters, stride, padding, multi-channel convolution
- Pooling layers: MaxPool/AvgPool, spatial downsampling and translation invariance
- Classic architectures: the evolution from LeNet → AlexNet → VGG → ResNet and residual connections
- Transfer learning: pre-trained models (ImageNet) + fine-tuning for high accuracy with limited data
- SalesPredict product image classification: automatic product category recognition
2. A Real-World E-Commerce Story
(1) The Pain Point: Manual Product Categorization Is Slow and Inconsistent
Bob's platform lists 1 thousand new products every day, and the operations team must manually sort product images into the correct categories. Manual classification is slow (5 seconds per image), inconsistent (different people assign the same product to different categories), and expensive ($30 thousand USD per month in labor costs). Image classification is the bottleneck in e-commerce operations.
(2) The CNN Solution
CNNs automatically learn visual features from product images to perform classification — millisecond-level speed, strong consistency, and zero labor cost.
PYTHON
import torchvision.models as models
# Transfer learning: use pre-trained ResNet
model = models.resnet18(pretrained=True)
model.fc = nn.Linear(model.fc.in_features, num_classes) # Replace last layer
(3) The Results: 95% Classification Accuracy, $30K Saved Monthly
Bob used CNN + transfer learning to automate product classification, achieving over 95% accuracy. Processing speed improved from 5 seconds per image to 0.01 seconds per image, saving $30 thousand USD in monthly labor costs.
3. How Convolution Works
(1) Kernels and Feature Extraction
graph TB
INPUT[Input Image<br/>H x W x C] --> CONV1[Conv Layer 1<br/>Edges & Lines]
CONV1 --> POOL1[Pool Layer 1<br/>Reduce Spatial]
POOL1 --> CONV2[Conv Layer 2<br/>Textures & Parts]
CONV2 --> POOL2[Pool Layer 2]
POOL2 --> CONV3[Conv Layer 3<br/>Objects & Shapes]
CONV3 --> FLATTEN[Flatten]
FLATTEN --> FC[Fully Connected]
FC --> OUTPUT[Class Probabilities]
▶ Example: Implementing Convolution Manually
PYTHON
import torch
import torch.nn as nn
# Input: 1 image, 1 channel, 5x5
input_img = torch.tensor([[
[1, 2, 0, 1, 3],
[0, 1, 2, 3, 1],
[1, 3, 1, 0, 2],
[2, 0, 3, 1, 1],
[0, 1, 2, 0, 1],
]], dtype=torch.float32).unsqueeze(0) # shape: (1, 1, 5, 5)
# Conv2d: 1 input channel, 1 output channel, 3x3 kernel
conv = nn.Conv2d(in_channels=1, out_channels=1, kernel_size=3, stride=1, padding=0)
print(f"Kernel weights:\n{conv.weight.data.squeeze()}")
print(f"Bias: {conv.bias.data.item():.4f}")
output = conv(input_img)
print(f"\nInput shape: {input_img.shape}")
print(f"Output shape: {output.shape}")
print(f"Output:\n{output.squeeze().detach()}")
Output:
TEXT
# Execution successful
(2) Convolution Parameters
| Parameter | Meaning | Effect |
|---|---|---|
| kernel_size | Kernel size | 3x3 (common) / 5x5 / 7x7 |
| stride | Step size | 1 (default) / 2 (downsampling) |
| padding | Zero-padding | 0 (shrinks) / 1 (preserves size) |
| in_channels | Number of input channels | RGB=3, grayscale=1 |
| out_channels | Number of output channels | Equals the number of kernels |
▶ Example: Comparing Different Convolution Configurations
PYTHON
import torch.nn as nn
x = torch.randn(1, 3, 32, 32) # 1 batch, 3 channels, 32x32
configs = {
"3x3, stride=1, pad=1": nn.Conv2d(3, 16, 3, stride=1, padding=1),
"3x3, stride=2, pad=1": nn.Conv2d(3, 16, 3, stride=2, padding=1),
"5x5, stride=1, pad=2": nn.Conv2d(3, 16, 5, stride=1, padding=2),
"5x5, stride=2, pad=0": nn.Conv2d(3, 16, 5, stride=2, padding=0),
}
for name, layer in configs.items():
out = layer(x)
print(f"{name:25s}: {x.shape} → {out.shape}")
Output:
TEXT
# Execution successful
4. Pooling Layers and CNN Architectures
(1) Pooling Operations
▶ Example: MaxPool vs. AvgPool
PYTHON
import torch
import torch.nn as nn
x = torch.tensor([[
[1, 3, 2, 4],
[5, 6, 7, 8],
[9, 2, 1, 3],
[4, 5, 6, 7],
]], dtype=torch.float32).unsqueeze(0) # (1, 1, 4, 4)
maxpool = nn.MaxPool2d(kernel_size=2, stride=2)
avgpool = nn.AvgPool2d(kernel_size=2, stride=2)
print(f"Input:\n{x.squeeze()}")
print(f"\nMaxPool (2x2):\n{maxpool(x).squeeze()}")
print(f"\nAvgPool (2x2):\n{avgpool(x).squeeze()}")
Output:
TEXT
# Execution successful
(2) Evolution of Classic Architectures
| Architecture | Year | Innovation | Depth | Parameters |
|---|---|---|---|---|
| LeNet | 1998 | First CNN | 5 | 60K |
| AlexNet | 2012 | ReLU + Dropout + GPU | 8 | 60M |
| VGG | 2014 | Stacked small kernels (3x3) | 16-19 | 138M |
| ResNet | 2015 | Residual connections (skip connections) | 18-152 | 11-60M |
▶ Example: Building a Simple CNN with PyTorch
PYTHON
import torch
import torch.nn as nn
class SimpleCNN(nn.Module):
def __init__(self, num_classes=5):
super().__init__()
self.features = nn.Sequential(
nn.Conv2d(3, 32, 3, padding=1), # 3→32 channels, 32x32
nn.ReLU(),
nn.MaxPool2d(2, 2), # 32→16x16
nn.Conv2d(32, 64, 3, padding=1), # 32→64 channels, 16x16
nn.ReLU(),
nn.MaxPool2d(2, 2), # 16→8x8
nn.Conv2d(64, 128, 3, padding=1), # 64→128 channels, 8x8
nn.ReLU(),
nn.MaxPool2d(2, 2), # 8→4x4
)
self.classifier = nn.Sequential(
nn.Flatten(),
nn.Linear(128 * 4 * 4, 256),
nn.ReLU(),
nn.Dropout(0.5),
nn.Linear(256, num_classes),
)
def forward(self, x):
x = self.features(x)
x = self.classifier(x)
return x
model = SimpleCNN(num_classes=5)
print(model)
print(f"Parameters: {sum(p.numel() for p in model.parameters()):,}")
Output:
TEXT
# Function defined successfully
5. Transfer Learning
(1) Why Transfer Learning Works
ImageNet pre-trained models have already learned general visual features (edges → textures → object parts). You only need to replace the final layer to adapt to a new task.
▶ Example: Transfer Learning with ResNet18
PYTHON
import torch
import torch.nn as nn
import torchvision.models as models
import torchvision.transforms as transforms
from torch.utils.data import DataLoader
import numpy as np
# Load pre-trained ResNet18
model = models.resnet18(weights=models.ResNet18_Weights.DEFAULT)
# Freeze all layers (feature extractor mode)
for param in model.parameters():
param.requires_grad = False
# Replace the last fully connected layer
num_classes = 5 # Electronics, Clothing, Food, Books, Home
model.fc = nn.Linear(model.fc.in_features, num_classes)
# Only the new fc layer has trainable parameters
trainable = sum(p.numel() for p in model.parameters() if p.requires_grad)
total = sum(p.numel() for p in model.parameters())
print(f"Trainable: {trainable:,} / Total: {total:,} ({trainable/total*100:.1f}%)")
# Data transforms for pre-trained model
transform = transforms.Compose([
transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
Output:
TEXT
# Execution successful
(2) Fine-Tuning Strategies
| Strategy | Frozen Layers | Training Data Size | Training Time | Best For |
|---|---|---|---|---|
| Feature Extractor | All (train fc only) | Small (<1k) | Fast | Very limited data |
| Partial Fine-tune | Last few layers | Medium (1-10k) | Medium | Moderate data |
| Full Fine-tune | None | Large (>10k) | Slow | Abundant data |
▶ Example: Fine-Tuning the Last Few Layers
PYTHON
import torchvision.models as models
import torch.nn as nn
model = models.resnet18(weights=models.ResNet18_Weights.DEFAULT)
# Freeze early layers, unfreeze layer4
for name, param in model.named_parameters():
if "layer4" in name or "fc" in name:
param.requires_grad = True
else:
param.requires_grad = False
model.fc = nn.Linear(model.fc.in_features, 5)
trainable = sum(p.numel() for p in model.parameters() if p.requires_grad)
print(f"Fine-tune trainable params: {trainable:,}")
Output:
TEXT
# Execution successful
6. SalesPredict Product Image Classification
▶ Example: Complete Transfer Learning Training Pipeline
PYTHON
import torch
import torch.nn as nn
import torchvision.models as models
from torch.utils.data import DataLoader, TensorDataset
from sklearn.model_selection import train_test_split
import numpy as np
# Simulate image data (in practice, load real images with ImageFolder)
rng = np.random.default_rng(42)
n = 2000
X = rng.standard_normal((n, 3, 224, 224)).astype(np.float32) # 3-channel 224x224
y = rng.integers(0, 5, n) # 5 product categories
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
train_ds = TensorDataset(torch.FloatTensor(X_train), torch.LongTensor(y_train))
test_ds = TensorDataset(torch.FloatTensor(X_test), torch.LongTensor(y_test))
train_loader = DataLoader(train_ds, batch_size=32, shuffle=True)
test_loader = DataLoader(test_ds, batch_size=64)
# Pre-trained ResNet18 with custom head
model = models.resnet18(weights=models.ResNet18_Weights.DEFAULT)
for param in model.parameters():
param.requires_grad = False
model.fc = nn.Linear(model.fc.in_features, 5)
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.fc.parameters(), lr=0.001)
# Training
for epoch in range(10):
model.train()
correct, total = 0, 0
for X_batch, y_batch in train_loader:
y_pred = model(X_batch)
loss = criterion(y_pred, y_batch)
optimizer.zero_grad()
loss.backward()
optimizer.step()
correct += (y_pred.argmax(1) == y_batch).sum().item()
total += y_batch.size(0)
if epoch % 2 == 0:
print(f"Epoch {epoch}: loss={loss.item():.4f}, acc={correct/total:.3f}")
# Evaluate
model.eval()
correct, total = 0, 0
with torch.no_grad():
for X_batch, y_batch in test_loader:
y_pred = model(X_batch)
correct += (y_pred.argmax(1) == y_batch).sum().item()
total += y_batch.size(0)
print(f"\nTest Accuracy: {correct/total:.3f}")
Output:
TEXT
# Execution successful
❓ FAQ
Q Why are CNNs better suited for images than MLPs?
A Three reasons — 1) Parameter sharing (the same kernel scans the entire image, reducing parameters); 2) Local connectivity (each neuron only sees a local region); 3) Translation invariance (objects are recognized regardless of position). MLPs flatten images and lose spatial structure.
Q How much data does transfer learning require?
A Feature Extractor mode (training only the fc layer) needs just a few hundred images; fine-tuning the last few layers requires 1-10k images; training from scratch needs 10k+ images. The less data you have, the more layers you should freeze.
Q Why is the 3x3 kernel the most commonly used?
A Two stacked 3x3 convolutions have the same receptive field as one 5x5 kernel, but with fewer parameters (18 vs. 25) and stronger non-linearity (2 ReLU activations). VGG demonstrated that stacking small kernels outperforms large ones.
Q What are residual connections in ResNet?
A y = F(x) + x — the network learns the residual (increment) rather than the full mapping. Benefits: solves the vanishing gradient problem in deep networks and enables training of 100+ layer networks.
Q Why do we need Normalize in image preprocessing?
A Pre-trained models were normalized using ImageNet's mean and standard deviation. New data must use the same normalization; otherwise, feature distributions won't match and transfer learning performance will suffer.
Q How do you handle class imbalance?
A Three approaches — 1) Weighted loss function (nn.CrossEntropyLoss(weight=class_weights)); 2) Oversampling the minority class; 3) Data augmentation (random flips/rotations/crops).
📖 Summary
- Convolution kernels act like visual receptors, extracting features layer by layer: edges → textures → parts → objects
- Pooling layers reduce spatial dimensions, increase translation invariance, and lower computational cost
- CNN architectures have evolved from LeNet to ResNet; residual connections solve the training problem for deep networks
- Transfer learning: freeze pre-trained layers + replace the classification head for high accuracy with limited data
- Three product classification strategies: Feature Extractor (little data), Partial Fine-tune (moderate data), Full Fine-tune (abundant data)
- Image preprocessing must match the pre-trained model (Resize + Normalize)
📝 Exercises
- Basic (Difficulty ⭐): Build a 2-layer CNN using nn.Conv2d + nn.MaxPool2d with input 3x32x32 and output 16x8x8. Print the shape after each layer. Hint: Conv2d(3,8,3,pad=1) → Pool → Conv2d(8,16,3,pad=1) → Pool.
- Intermediate (Difficulty ⭐⭐): Use transfer learning (ResNet18) to classify CIFAR-10 (10 classes). Freeze all layers except fc and train for 5 epochs. Hint: torchvision.datasets.CIFAR10 + transforms.
- Challenge (Difficulty ⭐⭐⭐): Compare Feature Extractor vs. Partial Fine-tune vs. Full Fine-tune on CIFAR-10 in terms of accuracy and training time. Plot accuracy vs. epoch curves for all three strategies. Hint: Freeze different numbers of layers and record test accuracy at each epoch.
← Previous Lesson: Introduction to Deep Learning | Next Lesson: Model Evaluation and Tuning →
