Machine Learning: NumPy速览 — 数组操作与数值计算完全指南
NumPy是Python数据科学的基石——所有的ML库（Pandas、Scikit-learn、PyTorch）都构建在NumPy数组之上。
1. 你将学到
- ndarray创建与形状操作：reshape、transpose、broadcasting机制
- 数组索引与切片：布尔索引、花式索引、多维切片
- 数值计算：向量化运算、矩阵乘法、统计函数
- 随机数生成与线性代数基础：np.random、np.linalg
- 用NumPy处理Bob的SalesPredict销售数据矩阵
2. 一个数据工程师的真实故事
(1) 痛点：Python列表处理百万行数据太慢
Bob需要计算过去12个月、5个品类的月度销售额统计——60 thousand条记录。用Python原生列表循环计算均值和标准差，耗时超过30秒，而Alice的美国数据有500 thousand条，直接卡死。原生Python循环是ML数据处理的瓶颈。
(2) NumPy的解法
NumPy的ndarray使用连续内存块和C级底层运算，向量化计算比Python循环快50-100倍。
PYTHON
import numpy as np
# Python list loop vs NumPy vectorization
sales_list = list(range(1, 60001)) # 60,000 records
sales_arr = np.array(sales_list)
# NumPy vectorized: 100x faster
mean_val = sales_arr.mean()
std_val = sales_arr.std()
print(f"Mean: {mean_val:.2f}, Std: {std_val:.2f}")
(3) 收益：计算时间从30秒降到0.3秒
Bob将数据处理从Python循环迁移到NumPy向量化后，60 thousand条数据的统计计算从30秒降到0.3秒。Alice的500 thousand条数据也能在1秒内完成。
3. ndarray创建与形状操作
ndarray是NumPy的核心数据结构——同类型元素的多维数组，存储在连续内存中。
(1) 创建数组
PYTHON
import numpy as np
# From Python list
a = np.array([1, 2, 3, 4, 5])
# Built-in constructors
zeros = np.zeros((3, 4)) # 3x4 matrix of zeros
ones = np.ones((2, 3)) # 2x3 matrix of ones
range_arr = np.arange(0, 10, 2) # [0, 2, 4, 6, 8]
linspace = np.linspace(0, 1, 5) # [0, 0.25, 0.5, 0.75, 1.0]
print(f"arange: {range_arr}")
print(f"linspace: {linspace}")
▶ 示例：创建SalesPredict销售矩阵
PYTHON
import numpy as np
# Monthly sales for 5 categories over 12 months (in thousand USD)
sales = np.array([
[120, 135, 150, 142, 160, 175, 180, 190, 195, 200, 220, 250], # Electronics
[80, 85, 90, 88, 95, 100, 105, 108, 110, 115, 130, 145], # Clothing
[50, 48, 55, 60, 58, 62, 65, 68, 70, 72, 80, 90], # Food
[30, 35, 40, 38, 42, 45, 48, 50, 52, 55, 60, 70], # Books
[200, 210, 225, 220, 240, 260, 270, 280, 290, 300, 350, 400], # Home
])
print(f"Shape: {sales.shape}")
print(f"Total annual revenue: {sales.sum() / 1000:.1f} million USD")
print(f"Q4 revenue: {sales[:, 9:].sum() / 1000:.1f} million USD")
输出：
TEXT
📖 仅展示
Shape: (5, 12)
Total annual revenue: 6.3 million USD
Q4 revenue: 2.3 million USD
(2) 形状操作
| 操作 | 方法 | 说明 |
|---|---|---|
| 改变形状 | reshape() |
不改变数据，仅改变视图 |
| 转置 | T 或 transpose() |
行列互换 |
| 展平 | flatten() / ravel() |
多维→1D |
| 扩维 | np.newaxis / expand_dims |
增加维度 |
▶ 示例：reshape与transpose
PYTHON
import numpy as np
# Reshape: 1D to 2D
data = np.arange(12)
matrix = data.reshape(3, 4)
print(f"Reshaped to 3x4:\n{matrix}")
# Transpose: swap rows and columns
transposed = matrix.T
print(f"Transposed to 4x3:\n{transposed}")
# -1 means auto-calculate dimension
auto_reshape = data.reshape(2, -1)
print(f"Auto reshape (2, -1): shape={auto_reshape.shape}")
输出：
TEXT
📖 仅展示
# 执行成功
(3) Broadcasting机制
Broadcasting让不同形状的数组进行算术运算时自动扩展，无需手动复制数据。
graph TB
A[Scalar + Array] --> B["Scalar broadcast to array shape"]
C["1D + 2D Array"] --> D["1D broadcast along axis 0"]
E["Shape (3,1) + (1,4)"] --> F["Both broadcast to (3,4)"]
▶ 示例：Broadcasting实战
PYTHON
import numpy as np
# Scalar + array: scalar broadcasts
prices = np.array([100, 200, 300, 400])
discount = 0.9 # 10% off
print(f"Discounted prices: {prices * discount}")
# 1D + 2D: row broadcasts across rows
monthly_sales = np.array([[100, 200], [110, 210], [120, 220]]) # 3 months x 2 categories
growth_rate = np.array([1.05, 1.10]) # different growth per category
print(f"Projected sales:\n{monthly_sales * growth_rate}")
# Column + row broadcast
col = np.array([[1], [2], [3]]) # shape (3, 1)
row = np.array([10, 20, 30, 40]) # shape (4,)
print(f"Col + Row result shape: {(col + row).shape}")
输出：
TEXT
📖 仅展示
# 执行成功
4. 数组索引与切片
(1) 基础索引与切片
PYTHON
import numpy as np
arr = np.arange(10)
print(f"arr[3]: {arr[3]}") # Single element
print(f"arr[2:7]: {arr[2:7]}") # Slice
print(f"arr[::2]: {arr[::2]}") # Step=2
# 2D indexing
matrix = np.arange(12).reshape(3, 4)
print(f"matrix[1, 2]: {matrix[1, 2]}") # Row 1, Col 2
print(f"matrix[0, :]: {matrix[0, :]}") # Entire row 0
print(f"matrix[:, 1]: {matrix[:, 1]}") # Entire col 1
(2) 布尔索引
▶ 示例：筛选高销售额品类
PYTHON
import numpy as np
# Category monthly sales (thousand USD)
sales = np.array([250, 145, 90, 70, 400])
categories = np.array(["Electronics", "Clothing", "Food", "Books", "Home"])
# Boolean indexing: find categories above 100k
high_sales = sales > 100
print(f"High sales mask: {high_sales}")
print(f"High sales categories: {categories[high_sales]}")
print(f"High sales values: {sales[high_sales]}")
# Compound conditions
mid_range = (sales >= 80) & (sales <= 200)
print(f"Mid-range categories: {categories[mid_range]}")
输出：
TEXT
📖 仅展示
# 执行成功
(3) 花式索引（Fancy Indexing）
▶ 示例：选取特定月份和品类
PYTHON
import numpy as np
sales = np.arange(60).reshape(5, 12) # 5 categories x 12 months
# Select specific months: Jan, Apr, Jul, Oct (indices 0,3,6,9)
quarterly = sales[:, [0, 3, 6, 9]]
print(f"Quarterly data shape: {quarterly.shape}")
# Select top 3 categories by total sales
total = sales.sum(axis=1)
top3_idx = np.argsort(total)[-3:]
print(f"Top 3 category indices: {top3_idx}")
print(f"Top 3 total sales: {total[top3_idx]}")
输出：
TEXT
📖 仅展示
# 执行成功
| 索引方式 | 语法 | 返回维度 | 用途 |
|---|---|---|---|
| 基础索引 | arr[2] |
降维 | 取单个元素 |
| 切片 | arr[1:5] |
同维 | 取连续范围 |
| 布尔索引 | arr[mask] |
1D | 条件筛选 |
| 花式索引 | arr[[1,3,5]] |
同维 | 取不连续位置 |
5. 数值计算与统计
(1) 向量化运算
NumPy的向量化运算替代Python循环，是性能的核心。
▶ 示例：向量化 vs 循环性能对比
PYTHON
import numpy as np
import time
size = 1_000_000
a = np.random.rand(size)
b = np.random.rand(size)
# Vectorized computation
start = time.time()
c = a + b
vec_time = time.time() - start
# Python loop (slow!)
start = time.time()
c_list = [a[i] + b[i] for i in range(size)]
loop_time = time.time() - start
print(f"Vectorized: {vec_time:.4f}s")
print(f"Loop: {loop_time:.4f}s")
print(f"Speedup: {loop_time / vec_time:.0f}x")
输出：
TEXT
📖 仅展示
# 执行成功
(2) 统计函数
▶ 示例：SalesPredict销售统计
PYTHON
import numpy as np
# Daily sales data for 30 days (thousand USD)
daily_sales = np.array([
45, 52, 48, 61, 55, 72, 68, 50, 53, 49,
58, 63, 71, 66, 59, 54, 47, 70, 75, 62,
51, 57, 64, 69, 73, 60, 56, 67, 74, 78
])
print(f"Mean: {daily_sales.mean():.1f}k USD")
print(f"Median: {np.median(daily_sales):.1f}k USD")
print(f"Std: {daily_sales.std():.1f}k USD")
print(f"Min: {daily_sales.min()}k USD")
print(f"Max: {daily_sales.max()}k USD")
print(f"Total: {daily_sales.sum()}k USD")
# Cumulative sum for trend
cum_sales = daily_sales.cumsum()
print(f"Day 30 cumulative: {cum_sales[-1]}k USD")
输出：
TEXT
📖 仅展示
# 执行成功
(3) 矩阵运算
▶ 示例：广告投放ROI矩阵计算
PYTHON
import numpy as np
# Ad spend per channel (3 channels) x 4 products
ad_matrix = np.array([
[10, 15, 8, 12], # Google Ads (thousand USD)
[5, 20, 10, 8], # Facebook Ads
[3, 7, 15, 10], # Email Marketing
])
# Conversion rate matrix (channel x product)
conv_rate = np.array([
[0.05, 0.03, 0.08, 0.04],
[0.03, 0.06, 0.04, 0.05],
[0.10, 0.08, 0.12, 0.09],
])
# Revenue per conversion (per product, thousand USD)
revenue_per_conv = np.array([50, 30, 80, 40])
# Matrix multiply: expected conversions
expected_conv = ad_matrix * conv_rate # element-wise
# Total revenue per product
total_revenue = expected_conv.sum(axis=0) * revenue_per_conv
print(f"Revenue by product: {total_revenue} thousand USD")
print(f"Total ROI revenue: {total_revenue.sum():.1f} thousand USD")
输出：
TEXT
📖 仅展示
# 执行成功
6. 随机数与线性代数
(1) 随机数生成
PYTHON
import numpy as np
rng = np.random.default_rng(seed=42) # Reproducible
# Common distributions
uniform = rng.uniform(0, 100, size=5) # Uniform [0, 100)
normal = rng.normal(50, 10, size=5) # Normal(mean=50, std=10)
integers = rng.integers(1, 100, size=5) # Random integers [1, 100)
choice = rng.choice(["A", "B", "C"], size=5) # Random choice
print(f"Uniform: {uniform}")
print(f"Normal: {normal}")
print(f"Integers: {integers}")
(2) 线性代数
▶ 示例：线性回归正规方程
PYTHON
import numpy as np
# Simulate: sales = 50 + 0.8 * ad_spend + noise
rng = np.random.default_rng(42)
n = 100
ad_spend = rng.uniform(10, 100, n)
noise = rng.normal(0, 5, n)
sales = 50 + 0.8 * ad_spend + noise
# Solve with normal equation: w = (X^T X)^-1 X^T y
X = np.column_stack([np.ones(n), ad_spend]) # Add bias column
w = np.linalg.inv(X.T @ X) @ X.T @ y := sales
# Note: fixed syntax below
w = np.linalg.inv(X.T @ X) @ (X.T @ sales)
print(f"Intercept: {w[0]:.2f}")
print(f"Coefficient: {w[1]:.2f}")
print(f"True values: intercept=50, coeff=0.8")
输出：
TEXT
📖 仅展示
# 执行成功
| 函数 | 用途 | 语法 |
|---|---|---|
np.dot / @ |
矩阵乘法 | A @ B |
np.linalg.inv |
矩阵求逆 | inv(A) |
np.linalg.det |
行列式 | det(A) |
np.linalg.norm |
向量/矩阵范数 | norm(v) |
np.linalg.svd |
奇异值分解 | U, S, Vt = svd(A) |
np.linalg.solve |
解线性方程组 | solve(A, b) |
❓ 常见问题
Q ndarray和Python list有什么本质区别？
A ndarray元素类型一致、内存连续、支持向量化运算；list元素类型混合、内存分散、只能循环计算。ndarray在数值计算上快50-100倍。
Q reshape会复制数据吗？
A 不会。reshape返回原数据的视图(view)，不复制内存。但如果形状不兼容(如非连续内存)，会自动复制。用
.base属性检查是否是视图。Q broadcasting的规则是什么？
A 从最右维度开始对齐，维度为1或缺失的自动扩展。两个维度要么相同，要么其中一个为1，否则报错。
Q 什么时候用np.random.seed vs np.random.default_rng？
A 推荐用default_rng(新版API)，seed是旧版API。default_rng使用PCG64算法，统计质量更好，且不会影响全局状态。
Q 如何判断两个数组是否共享内存？
A 用
np.shares_memory(a, b)检查。或检查a.base is b或b.base is a。共享内存时修改一个会影响另一个。Q 为什么说NumPy是ML的基础？
A Pandas的底层是NumPy数组，Scikit-learn的输入输出都是NumPy ndarray，PyTorch的Tensor也与NumPy高度兼容。掌握NumPy就是掌握ML的数据语言。
📖 小节
- ndarray是同类型、连续内存的多维数组，是NumPy的核心数据结构
- Broadcasting让不同形状数组自动扩展进行运算，避免手动复制
- 向量化运算比Python循环快50-100倍，是NumPy性能的核心
- 布尔索引和花式索引提供灵活的数据筛选方式
- np.linalg提供矩阵求逆、SVD等线性代数运算，支撑ML算法底层数学
- 随机数生成用default_rng(seed)确保可复现性
📝 作业
- 基础题（难度⭐）：创建一个5x5的矩阵，值从1到25，然后用切片提取对角线元素。提示：对角线元素的行列索引相同，或使用
np.diag()。 - 进阶题（难度⭐⭐）：生成1000个正态分布N(100, 15)的随机销售数据，用布尔索引筛选出超过130的值，计算其占比。提示：使用
rng.normal()和布尔索引。 - 挑战题（难度⭐⭐⭐）：用NumPy正规方程实现多元线性回归：假设sales = 20 + 3ad_spend + 1.5traffic + noise，生成模拟数据并求解系数，与真实系数对比。提示：构建X矩阵时添加bias列，使用
np.linalg.inv和矩阵乘法。
← 上一课：机器学习简介 | 下一课：Pandas数据处理 →
