NumPy: Performance Optimization
Last updated: 2026-08-26
1. What You'll Learn
- ❶ Vectorization — the #1 performance rule
- ❷ Memory layout: C-contiguous vs Fortran-contiguous
- ❸ In-place operations vs copies
- ❹ Profiling NumPy code
- ❺ Common performance pitfalls
2. Key Concepts
(1) Vectorization
PYTHON
import numpy as np
import time
n = 10_000_000
a = np.random.rand(n)
b = np.random.rand(n)
# Slow: Python loop
t0 = time.time()
c = np.empty(n)
for i in range(n):
c[i] = a[i] + b[i]
t1 = time.time()
# Fast: vectorized
t2 = time.time()
d = a + b
t3 = time.time()
print(f"Loop: {t1-t0:.3f}s, Vectorized: {t3-t2:.6f}s")
print(f"Speedup: {(t1-t0)/(t3-t2):.0f}x")
(2) In-Place Operations
PYTHON
# Avoid copies with out= parameter
a = np.random.rand(1000000)
b = np.random.rand(1000000)
# Copy (allocates new memory)
c = a + b
# In-place (no allocation)
np.add(a, b, out=a)
▶ Example: Vectorization speed comparison (Difficulty ⭐⭐)
PYTHON
import numpy as np
import time
n = 1_000_000
a = np.random.rand(n)
b = np.random.rand(n)
# Slow: Python loop
t0 = time.time()
c = np.empty(n)
for i in range(n):
c[i] = a[i] + b[i]
t1 = time.time()
# Fast: vectorized
t2 = time.time()
d = a + b
t3 = time.time()
loop_time = t1 - t0
vec_time = t3 - t2
print(f"Loop: {loop_time:.3f}s, Vectorized: {vec_time:.6f}s")
print(f"Speedup: {loop_time/vec_time:.0f}x")
Output:
TEXT 📖 Display onlyLoop: 0.312s, Vectorized: 0.002s Speedup: 156x
▶ Example: In-place operations with out= (Difficulty ⭐⭐)
PYTHON
import numpy as np
a = np.array([10, 20, 30, 40, 50])
b = np.array([1, 2, 3, 4, 5])
# Without out: creates new array
c = np.add(a, b)
print("Without out:", c)
# With out: reuses existing memory
result = np.empty(5)
np.add(a, b, out=result)
print("With out:", result)
# Chain operations in-place
np.multiply(result, 2, out=result)
np.add(result, 1, out=result)
print("Chained:", result)
Output:
TEXT 📖 Display onlyWithout out: [11 22 33 44 55] With out: [11 22 33 44 55] Chained: [23 45 67 89 111]
▶ Example: Memory layout impact (Difficulty ⭐)
PYTHON
import numpy as np
# C-contiguous (row-major) — default
c = np.array([[1, 2, 3], [4, 5, 6]])
print("C-contiguous:", c.flags.c_contiguous)
print("Fortran-contiguous:", c.flags.f_contiguous)
# Convert to Fortran-contiguous (column-major)
f = np.asfortranarray(c)
print("Fortran-contiguous:", f.flags.f_contiguous)
# Shape and strides
print("C strides:", c.strides)
print("F strides:", f.strides)
Output:
TEXT 📖 Display onlyC-contiguous: True Fortran-contiguous: False Fortran-contiguous: True C strides: (24, 8) F strides: (8, 24)
❓ FAQ
Q What's the single most important performance rule?
A Avoid Python loops. Every time you write
for i in range(n), ask: "Can I vectorize this?" If yes, NumPy will be 10-100x faster.Q What's the difference between C-order and Fortran-order?
A C-order (row-major) stores rows contiguously. Fortran-order (column-major) stores columns contiguously. Use the order that matches your access pattern for best cache performance.
Q When should I use
out= parameter?A When you're doing many operations on the same large array,
out= avoids allocating new memory for each operation. Use it in loops and pipelines.📖 Summary
- Vectorization is the #1 performance rule — avoid Python loops at all costs
- Use
out=parameter for in-place operations to avoid allocations - C-contiguous (row-major) is the default; match access pattern to memory layout
- Profile with
%timeitin Jupyter ortime.perf_counter()in scripts - Common pitfalls: loops, unnecessary copies, non-contiguous memory after transpose
📝 Exercises
-
Beginner (Difficulty ⭐): Compare the time to compute
a + bfor 10 million elements using a loop vs vectorization. Record the speedup factor. -
Intermediate (Difficulty ⭐⭐): Compare the time of
a + bwith and withoutout=a. Measure the difference in allocation overhead. -
Advanced (Difficulty ⭐⭐⭐): Create a large array, transpose it, then sum along an axis. Compare the performance of the transposed array vs a contiguous copy. Explain the difference.
