Performance Optimization — From 100 to 1 Million QPS

Performance optimization is like building a highway—you start by fixing the most congested sections (the bottlenecks), measuring the results after each section is completed, rather than widening the entire road indiscriminately. Blind optimization is a waste; measurement-driven optimization is the most efficient approach.

1. What You'll Learn

• Troubleshooting Asynchronous Blocking: asyncio Pitfalls and run_in_executor Rewriting Synchronous Code
• Database Optimization: Indexing Strategies, Query Optimization, Connection Pool Tuning
• Concurrency Model: Number of Uvicorn Workers and Gunicorn Configuration
• Response Compression: GZipMiddleware and JSON Serialization Optimization (orjson)
• Alice Scenario: The Optimization Path for PriceTracker from 500 QPS on a Single Machine to 1 Million QPS in a Cluster

2. Alice's True Story

(1) Pain Point: 500 QPS on a single server is not enough

After PriceTracker went live, a single server could only handle 500 QPS, but during major sales events, traffic surged to 5,000 QPS, causing API response timeouts. Charlie added more servers, but the improvement was negligible—the database connection pool was exhausted, the synchronization code was blocking the event loop, and JSON serialization was consuming 40% of the CPU.

(2) A Systematic Approach to Performance Optimization

Performance optimization isn't about guesswork; it's a cycle of measurement → identifying bottlenecks → optimization → validation. We tackle issues layer by layer, from the code level (async/serialization) to the database level (indexes/connection pools) to the architectural level (Workers/load balancing).

(3) Revenue

PriceTracker's QPS has been optimized from 500 to 10,000 (on a single server), and with a load-balancing cluster, it can reach 1 million QPS, while the P99 latency has been reduced from 500 ms to 30 ms.

3. Levels of Performance Optimization

(1) Three-Layer Optimization Model

graph TD
    L3[Infrastructure Layer] --> L2[Database Layer]
    L2 --> L1[Application Layer]
    
    L1 --- A1[async/await - Block detection]
    L1 --- A2[Serialization - orjson]
    L1 --- A3[Compression - GZip]
    
    L2 --- D1[Indexes - Query optimization]
    L2 --- D2[Connection Pool - Size tuning]
    L2 --- D3[Query Patterns - N+1 prevention]
    
    L3 --- I1[Workers - Gunicorn config]
    L3 --- I2[Load Balancer - Nginx]
    L3 --- I3[Auto-scaling - K8s HPA]

graph TD
    L3[Infrastructure Layer] --> L2[Database Layer]
    L2 --> L1[Application Layer]
    
    L1 --- A1[async/await - Block detection]
    L1 --- A2[Serialization - orjson]
    L1 --- A3[Compression - GZip]
    
    L2 --- D1[Indexes - Query optimization]
    L2 --- D2[Connection Pool - Size tuning]
    L2 --- D3[Query Patterns - N+1 prevention]
    
    L3 --- I1[Workers - Gunicorn config]
    L3 --- I2[Load Balancer - Nginx]
    L3 --- I3[Auto-scaling - K8s HPA]

(2) Paths to Improving QPS

flowchart LR
    A[500 QPS\nBaseline] -->|async + orjson| B[2000 QPS]
    B -->|DB indexes + pool| C[10000 QPS]
    C -->|4 Workers| D[40000 QPS]
    D -->|Nginx LB x 5| E[200000 QPS]
    E -->|K8s x 5 pods| F[1000000 QPS]

flowchart LR
    A[500 QPS\nBaseline] -->|async + orjson| B[2000 QPS]
    B -->|DB indexes + pool| C[10000 QPS]
    C -->|4 Workers| D[40000 QPS]
    D -->|Nginx LB x 5| E[200000 QPS]
    E -->|K8s x 5 pods| F[1000000 QPS]

4. Application Layer Optimization

(1) Troubleshooting Asynchronous Blocking

(1) ▶ Example: Synchronous Code That Blocks the Event Loop

import asyncio
import time

# BAD: sync I/O blocks the event loop
async def get_price_bad(product_id: int):
    time.sleep(0.1)  # Blocks ALL other requests for 100ms!
    return {"product_id": product_id, "price": 9.99}

# GOOD: use asyncio for I/O
async def get_price_good(product_id: int):
    await asyncio.sleep(0.1)  # Non-blocking, other requests continue
    return {"product_id": product_id, "price": 9.99}

Output:

# Function defined successfully

(2) ▶ Example: run_in_executor wraps synchronous code

import asyncio
from functools import partial

# Sync function that cannot be made async
def sync_scrape_price(url: str) -> float:
    # Uses requests library (sync only)
    import requests
    response = requests.get(url, timeout=10)
    return parse_price(response.text)

async def get_price_async(product_id: int):
    loop = asyncio.get_event_loop()
    # Run sync function in thread pool - doesn't block event loop
    price = await loop.run_in_executor(
        None,  # Default thread pool
        partial(sync_scrape_price, f"https://api.example.com/prices/{product_id}"),
    )
    return {"product_id": product_id, "price": price}

Output:

# Function defined successfully

(2) JSON Serialization Optimization

(3) ▶ Example: Using orjson instead of json

# Install: uv add orjson
from fastapi import FastAPI
from fastapi.responses import ORJSONResponse

app = FastAPI(default_response_class=ORJSONResponse)

@app.get("/api/v1/products")
async def list_products():
    # orjson is 3-10x faster than stdlib json
    return {"products": [{"id": i, "name": f"Product {i}"} for i in range(100)]}

Output:

# Function defined successfully

(4) ▶ Example: GZipMiddleware Compression

from fastapi import FastAPI
from starlette.middleware.gzip import GZipMiddleware

app = FastAPI()
app.add_middleware(GZipMiddleware, minimum_size=1000)  # Compress responses > 1KB

@app.get("/api/v1/products")
async def list_products():
    # Large JSON responses are automatically compressed
    return {"products": [...]}  # 50KB → ~5KB with gzip

Output:

# Function defined successfully

5. Database Layer Optimization

(1) Indexing Strategy

(1) ▶ Example: PriceTracker Index Design

from sqlalchemy import Index

class Product(Base):
    __tablename__ = "products"
    __table_args__ = (
        # Single-column indexes for common filters
        Index("ix_products_category", "category"),
        Index("ix_products_created_at", "created_at"),
        # Composite index for category + price range queries
        Index("ix_products_category_base_price", "category", "base_price"),
        # Partial index for active products only (PostgreSQL specific)
        Index("ix_products_active", "created_at", postgresql_where=("deleted_at IS NULL")),
    )
    ...

Output:

# Execution Successful

(2) Connection Pool Tuning

(2) ▶ Example: Production-Level Connection Pool Configuration

engine = create_async_engine(
    DATABASE_URL,
    pool_size=25,           # Persistent connections per worker
    max_overflow=10,        # Extra connections when pool exhausted
    pool_timeout=30,        # Wait time for available connection
    pool_recycle=1800,      # Recycle connections after 30 min
    pool_pre_ping=True,     # Test connection before use
    echo=False,             # Disable SQL logging in production
)

Output:

# Execution Successful

6. Concurrency Model Optimization

(1) Workers Configuration

(1) ▶ Example: Gunicorn + Uvicorn Workers

# Production: Gunicorn manages Uvicorn workers
gunicorn app.main:app \
    --workers 9 \
    --worker-class uvicorn.workers.UvicornWorker \
    --bind 0.0.0.0:8000 \
    --timeout 120 \
    --graceful-timeout 30 \
    --access-logfile - \
    --error-logfile -

Output:

# Command executed successfully

# In Dockerfile
CMD ["gunicorn", "app.main:app", \
     "--workers", "9", \
     "--worker-class", "uvicorn.workers.UvicornWorker", \
     "--bind", "0.0.0.0:8000"]

❓ FAQ

📖 Summary

• Three-tier model for performance optimization: Application layer (async/serialization) → Database layer (indexes/connection pool) → Infrastructure layer (Workers/LB)
• Wrap synchronous I/O with run_in_executor to prevent blocking the event loop
• orjson is 3 to 10 times faster than json for serialization, and GZipMiddleware compresses large responses
• Database indexes are designed based on query patterns (single-column, composite, and partial indexes); the connection pool size is coordinated with the number of workers
• Gunicorn + Uvicorn Workers: (2 x CPU) + 1 workers, K8s autoscaling to one million QPS

📝 Exercises

1. Basic Exercise (Difficulty: ⭐): Install orjson, change FastAPI's default response class to ORJSONResponse, and use curl to compare the response times for JSON and orjson. Hint: default_response_class=ORJSONResponse
2. Advanced Exercise (Difficulty ⭐⭐): Add indexes to the products table in PriceTracker (category, created_at, and a composite index of category and base_price), and compare the execution times of the same query before and after adding the indexes. Hint: Index("ix_name", "col1", "col2") + EXPLAIN ANALYZE
3. Challenge (Difficulty: ⭐⭐⭐): Comprehensive performance optimization—orjson serialization + GZipMiddleware + connection pool tuning + Gunicorn with 4 workers. Use Locust to run load tests and compare QPS and P99 latency before and after optimization. Hint: locust -f locustfile.py --host=http://localhost:8000

---|

Web-Tutorial Tech Team

A team of developers maintaining programming tutorials. Each tutorial is written and reviewed by developers with expertise in that field. We work to keep our content accurate and reliable — if you spot an issue, please let us know.

About Us