Machine Learning: KNN与聚类 — 最近邻分类与无监督分群指南
物以类聚——KNN找"最像你的邻居"来分类，K-Means找"最紧密的团伙"来分群。
1. 你将学到
- KNN分类：距离度量(Euclidean/Manhattan/Cosine)、K值选择、投票机制
- K-Means聚类：算法流程、K值选择(肘部法则/轮廓系数)、收敛条件
- DBSCAN密度聚类：核心点/边界点/噪声点、eps与min_samples调参
- 层次聚类：AgglomerativeClustering与树状图(Dendrogram)
- Bob的用户画像分群：RFM聚类发现高价值/沉睡/流失用户群体
2. 一个用户运营经理的真实故事
(1) 痛点：50万用户一刀切运营，效率极低
Bob的平台有500 thousand用户，所有用户收到同样的促销邮件。结果是：高价值用户觉得促销太低端，沉睡用户根本不打开邮件，整体转化率只有2%。用户是多样的，一刀切策略浪费80%的营销预算。
(2) 聚类的解法
聚类能自动将用户分为高价值/活跃/沉睡/流失等群体，实现精准运营。
PYTHON
from sklearn.cluster import KMeans
# RFM clustering: group users by Recency, Frequency, Monetary
rfm = df[["recency", "frequency", "monetary"]]
kmeans = KMeans(n_clusters=4, random_state=42)
df["segment"] = kmeans.fit_predict(rfm)
for i in range(4):
segment = df[df["segment"] == i]
print(f"Segment {i}: {len(segment)} users, "
f"Avg Monetary={segment['monetary'].mean():.0f} USD")
(3) 收益：精准运营提升转化率4倍
Bob用聚类将用户分为4群后，高价值群推送高端优惠，沉睡群推送唤醒礼包，整体转化率从2%提升到8%，营销ROI提升3倍。
3. KNN分类
(1) 距离度量与K值选择
▶ 示例：KNN分类 + 不同K值对比
PYTHON
from sklearn.neighbors import KNeighborsClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import cross_val_score
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
import numpy as np
X, y = load_iris(return_X_y=True)
# Compare different K values
for k in [1, 3, 5, 7, 11, 21]:
pipe = Pipeline([
("scaler", StandardScaler()),
("knn", KNeighborsClassifier(n_neighbors=k)),
])
scores = cross_val_score(pipe, X, y, cv=5, scoring="accuracy")
print(f"K={k:2d}: Accuracy={scores.mean():.3f} +/- {scores.std():.3f}")
输出：
TEXT
📖 仅展示
# 执行成功
| 距离度量 | 公式特点 | 适用场景 |
|---|---|---|
| Euclidean | 直线距离 | 连续特征，各维度同等重要 |
| Manhattan | 曼哈顿距离 | 高维数据、异常值多 |
| Cosine | 角度距离 | 文本向量、方向比大小重要 |
| Minkowski | 泛化距离(p=2→Euclidean, p=1→Manhattan) | 需要灵活调整 |
(2) K值的影响
| K值 | 决策边界 | 风险 |
|---|---|---|
| K=1 | 极不规则 | 过拟合(噪声敏感) |
| K=小(3-7) | 适度弯曲 | 通常最佳 |
| K=大(20+) | 近乎线性 | 欠拟合 |
▶ 示例：KNN用户购买预测
PYTHON
from sklearn.neighbors import KNeighborsClassifier
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
import numpy as np
rng = np.random.default_rng(42)
n = 1000
X = np.column_stack([
rng.uniform(0, 100, n), # browsing_time
rng.uniform(0, 50, n), # cart_value_usd
rng.integers(1, 30, n), # pages_viewed
rng.integers(0, 10, n), # previous_purchases
])
y = (X[:, 1] * 0.05 + X[:, 3] * 0.3 + rng.normal(0, 0.5, n) > 2).astype(int)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
pipe = Pipeline([
("scaler", StandardScaler()),
("knn", KNeighborsClassifier(n_neighbors=7, weights="distance")),
])
pipe.fit(X_train, y_train)
print(classification_report(y_test, pipe.predict(X_test)))
输出：
TEXT
📖 仅展示
# 执行成功
4. K-Means聚类
(1) 算法流程
sequenceDiagram
participant Init as Initialize K Centers
participant Assign as Assign Points
participant Update as Update Centers
participant Check as Converged?
Init->>Assign: Random K center positions
loop Until convergence
Assign->>Update: Each point → nearest center
Update->>Check: Centers = mean of assigned points
Check->>Assign: Not converged (centers moved)
end
Check-->>Done: Converged! Return clusters
▶ 示例：K-Means聚类 + 肘部法则
PYTHON
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt
import numpy as np
rng = np.random.default_rng(42)
# Generate 4 natural clusters
X = np.vstack([
rng.normal([20, 500], [3, 50], (200, 2)), # High value
rng.normal([50, 200], [5, 30], (300, 2)), # Medium
rng.normal([80, 50], [8, 20], (350, 2)), # Low
rng.normal([10, 800], [2, 40], (150, 2)), # VIP
])
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
# Elbow method
inertias = []
K_range = range(2, 10)
for k in K_range:
km = KMeans(n_clusters=k, random_state=42, n_init=10)
km.fit(X_scaled)
inertias.append(km.inertia_)
fig, ax = plt.subplots(figsize=(8, 5))
ax.plot(K_range, inertias, "bo-", linewidth=2)
ax.set_xlabel("Number of Clusters (K)")
ax.set_ylabel("Inertia (Within-Cluster Sum of Squares)")
ax.set_title("Elbow Method for Optimal K")
ax.axvline(x=4, color="red", linestyle="--", label="Elbow at K=4")
ax.legend()
plt.tight_layout()
plt.savefig("elbow_method.png", dpi=150)
输出：
TEXT
📖 仅展示
# 执行成功
(2) 轮廓系数
▶ 示例：轮廓系数选择K
PYTHON
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
from sklearn.preprocessing import StandardScaler
import numpy as np
# Using same X_scaled from previous example
for k in range(2, 8):
km = KMeans(n_clusters=k, random_state=42, n_init=10)
labels = km.fit_predict(X_scaled)
score = silhouette_score(X_scaled, labels)
print(f"K={k}: Silhouette Score={score:.3f}")
输出：
TEXT
📖 仅展示
# 执行成功
| 指标 | 含义 | 最优K判断 |
|---|---|---|
| Inertia(肘部法则) | 簇内平方和 | 拐点处 |
| Silhouette Score | 凝聚度vs分离度 | 最大值处 |
| Gap Statistic | 与随机分布对比 | 首次低于上限处 |
5. DBSCAN密度聚类
▶ 示例：DBSCAN发现任意形状的簇
PYTHON
from sklearn.cluster import DBSCAN
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import silhouette_score
import numpy as np
rng = np.random.default_rng(42)
# Create non-spherical clusters (two circles)
from sklearn.datasets import make_circles
X, _ = make_circles(n_samples=500, factor=0.5, noise=0.05, random_state=42)
X_scaled = StandardScaler().fit_transform(X)
# K-Means vs DBSCAN
km = KMeans(n_clusters=2, random_state=42, n_init=10)
km_labels = km.fit_predict(X_scaled)
db = DBSCAN(eps=0.3, min_samples=10)
db_labels = db.fit_predict(X_scaled)
n_clusters = len(set(db_labels)) - (1 if -1 in db_labels else 0)
n_noise = list(db_labels).count(-1)
print(f"K-Means Silhouette: {silhouette_score(X_scaled, km_labels):.3f}")
print(f"DBSCAN Clusters: {n_clusters}, Noise points: {n_noise}")
if n_clusters > 1:
db_valid = db_labels != -1
print(f"DBSCAN Silhouette: {silhouette_score(X_scaled[db_valid], db_labels[db_valid]):.3f}")
输出：
TEXT
📖 仅展示
# 执行成功
| 维度 | K-Means | DBSCAN |
|---|---|---|
| 簇形状 | 球形 | 任意形状 |
| K值 | 必须预设 | 自动发现 |
| 噪声点 | 不处理 | 标记为-1 |
| 密度不均 | 效果差 | 仍可处理 |
| 计算速度 | 快 | 中等 |
6. Bob的RFM用户画像分群
▶ 示例：完整RFM聚类项目
PYTHON
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
import pandas as pd
import numpy as np
rng = np.random.default_rng(42)
n = 5000
df = pd.DataFrame({
"user_id": range(10001, 10001 + n),
"recency_days": rng.integers(1, 365, n),
"frequency": rng.integers(1, 50, n),
"monetary_usd": rng.exponential(500, n),
})
# RFM features
rfm = df[["recency_days", "frequency", "monetary_usd"]]
# Log transform for skewed features
rfm_log = rfm.copy()
rfm_log["frequency"] = np.log1p(rfm_log["frequency"])
rfm_log["monetary_usd"] = np.log1p(rfm_log["monetary_usd"])
# Scale
scaler = StandardScaler()
rfm_scaled = scaler.fit_transform(rfm_log)
# K-Means with K=4
km = KMeans(n_clusters=4, random_state=42, n_init=10)
df["segment"] = km.fit_predict(rfm_scaled)
# Analyze segments
segment_summary = df.groupby("segment").agg({
"recency_days": "mean",
"frequency": "mean",
"monetary_usd": "mean",
"user_id": "count",
}).round(1)
segment_summary.columns = ["avg_recency", "avg_frequency", "avg_monetary", "count"]
segment_summary = segment_summary.sort_values("avg_monetary", ascending=False)
# Label segments
labels = ["Champions", "Loyal", "At Risk", "Lost"]
for idx, (_, row) in enumerate(segment_summary.iterrows()):
print(f"{labels[idx]:10s}: {int(row['count']):5d} users, "
f"Recency={row['avg_recency']:.0f}d, "
f"Freq={row['avg_frequency']:.1f}, "
f"Monetary={row['avg_monetary']:.0f} USD")
输出：
TEXT
📖 仅展示
# 执行成功
| 群体 | Recency | Frequency | Monetary | 运营策略 |
|---|---|---|---|---|
| Champions | 低(近期活跃) | 高 | 高 | VIP服务、高端推荐 |
| Loyal | 中 | 中高 | 中 | 忠诚计划、交叉销售 |
| At Risk | 高(很久未购) | 中 | 中 | 唤醒礼包、限时优惠 |
| Lost | 很高 | 低 | 低 | 低成本触达、问卷调研 |
❓ 常见问题
Q KNN和K-Means的K有什么区别？
A KNN的K是"参考几个邻居投票"，是分类算法；K-Means的K是"分几个群"，是聚类算法。完全不同的概念。
Q KNN需要标准化吗？
A 必须。KNN基于距离计算，未标准化的特征(如收入0-100000 vs 年龄0-100)会完全主导距离。始终配合StandardScaler。
Q K-Means如何选择K值？
A 综合肘部法则和轮廓系数。肘部法则找拐点，轮廓系数找最大值。两者可能不一致，以业务合理性为准。
Q DBSCAN的eps怎么选？
A 画K-distance图(sorted k-nearest neighbor distances)，找拐点。典型eps值在0.1-1.0(标准化后)。min_samples通常设为2*维度。
Q 聚类结果如何评估？
A 无监督聚类没有"正确答案"。用轮廓系数(数学评估) + 业务可解释性(领域评估)双重验证。能对应业务含义的聚类才有价值。
Q RFM聚类前为什么要做对数变换？
A Frequency和Monetary通常右偏分布(skewed)，少数高值用户会拉大簇间距离。log变换让分布更对称，K-Means效果更好。
📖 小节
- KNN分类：距离度量选K个最近邻居投票，K小过拟合/大欠拟合，必须标准化
- K-Means聚类：迭代分配+更新中心，肘部法则和轮廓系数选K，只适合球形簇
- DBSCAN密度聚类：自动发现簇数、标记噪声、处理任意形状，但参数敏感
- RFM用户分群是电商聚类的经典应用：Recency + Frequency + Monetary → 用户画像
- 聚类前做对数变换处理偏态分布，标准化消除量纲差异
- 聚类评估=数学指标(轮廓系数)+业务可解释性，两者缺一不可
📝 作业
- 基础题（难度⭐）：用KNN对Iris分类，对比K=1,5,10,20的交叉验证accuracy，找出最优K。提示：Pipeline(StandardScaler+KNN) + cross_val_score。
- 进阶题（难度⭐⭐）：用make_blobs生成4簇数据，分别用K-Means和DBSCAN聚类，对比轮廓系数和对噪声点的处理。提示：DBSCAN标记-1为噪声。
- 挑战题（难度⭐⭐⭐）：实现Bob的RFM聚类——生成模拟用户数据，做log变换+标准化，用肘部法则选K，K-Means聚类后给每个群起有意义的名字(Champions/Loyal/At Risk/Lost)，输出每个群的平均RFM值。提示：参考第6节完整示例。
