Docker Deployment — Containerization and Multi-Stage Builds
Docker is like a shipping container—it packages applications and all their dependencies into standard containers that can be unloaded and run directly at any port (server), eliminating the "it works on my machine" problem.
1. What You'll Learn
- Multi-stage Dockerfile: Build stage (installing UV dependencies) vs. Run stage (streamlining the image)
- Docker Compose Orchestration: A Complete Stack of FastAPI + PostgreSQL + Redis + Celery Worker
- Environment Variables and Secrets Management:
.envFiles and Docker Secrets - Health Check:
HEALTHCHECKCommand and FastAPI/healthEndpoint - Alice Scenario: Charlie launches the PriceTracker full-stack service with a single click
docker compose up
2. Alice's True Story
(1) Pain Point: Deployment failures caused by environmental differences
When Charlie deployed PriceTracker—which he had configured locally—to the production server, he encountered issues such as Python version mismatches, a missing libpq library, UV not being installed, and different PostgreSQL connection configurations—forcing him to spend two hours manually troubleshooting each deployment. To make matters worse, the four services—FastAPI, PostgreSQL, Redis, and Celery—had to be started and managed separately, and their startup order was complex and interdependent.
(2) The Docker Compose Solution
Dockerfile defines application containers, while Docker Compose orchestrates all services and their dependencies. A single docker compose up command launches the entire service stack, and environment consistency is guaranteed by the image.
(3) Revenue
Deployment time has gone from a 2-hour manual process to 30 seconds with a single click, and the local development environment is now identical to production, so Charlie will never have to say, "It works on my server," again.
3. Multi-stage Dockerfile
(1) Build Process
flowchart LR
A[Builder Stage] -->|Copy installed deps| B[Runtime Stage]
subgraph Builder
A1[UV install deps] --> A2[Compile wheels]
end
subgraph Runtime
B1[Python slim image] --> B2[Copy app code]
B2 --> B3[Copy deps from builder]
B3 --> B4[Run uvicorn]
end
(1) ▶ Example: Multi-stage Dockerfile
DOCKERFILE
# === Stage 1: Builder ===
FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim AS builder
WORKDIR /app
# Copy dependency files first (cache layer)
COPY pyproject.toml uv.lock ./
# Install dependencies into virtual environment
RUN uv sync --frozen --no-dev --no-install-project
# Copy application code
COPY app/ app/
# === Stage 2: Runtime ===
FROM python:3.12-slim-bookworm AS runtime
WORKDIR /app
# Install runtime system dependencies
RUN apt-get update && \
apt-get install -y --no-install-recommends libpq5 && \
rm -rf /var/lib/apt/lists/*
# Copy virtual environment from builder
COPY --from=builder /app/.venv /app/.venv
# Copy application code
COPY app/ app/
# Set environment variables
ENV PATH="/app/.venv/bin:$PATH" \
PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1
# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"
# Run application
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4"]
Output:
TEXT
// Execution Successful
(2) Comparison of Image Sizes
| Method | Image Size | Description |
|---|---|---|
| Single-stage (Python 3.12) | ~1.2 GB | Includes build tools and cache |
| Multi-stage (Python: 3.12-slim) | ~200 MB | Runtime dependencies only |
| Alpine (python:3.12-alpine) | ~80MB | Smaller but has compatibility issues |
4. Docker Compose Orchestration
(1) Complete Stack Architecture
flowchart TD
Nginx[Nginx Reverse Proxy] --> API1[FastAPI Worker 1]
Nginx --> API2[FastAPI Worker 2]
API1 --> PG[(PostgreSQL)]
API2 --> PG
API1 --> Redis[(Redis)]
API2 --> Redis
API1 --> Broker[Redis Broker]
API2 --> Broker
Broker --> CW1[Celery Worker 1]
Broker --> CW2[Celery Worker 2]
CW1 --> PG
CW2 --> PG
CW1 --> Redis
CW2 --> Redis
Flower[Flower Monitor] --> Broker
(1) ▶ Example: docker-compose.yml
YAML
version: "3.8"
services:
api:
build:
context: .
dockerfile: docker/Dockerfile
ports:
- "8000:8000"
environment:
- DATABASE_URL=postgresql+asyncpg://pricetracker:${DB_PASSWORD}@postgres:5432/pricetracker
- REDIS_URL=redis://redis:6379/0
- CELERY_BROKER_URL=redis://redis:6379/1
- SECRET_KEY=${SECRET_KEY}
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"]
interval: 30s
timeout: 10s
retries: 3
postgres:
image: postgres:16-alpine
environment:
POSTGRES_USER: pricetracker
POSTGRES_PASSWORD: ${DB_PASSWORD}
POSTGRES_DB: pricetracker
volumes:
- postgres_data:/var/lib/postgresql/data
ports:
- "5432:5432"
healthcheck:
test: ["CMD-SHELL", "pg_isready -U pricetracker"]
interval: 10s
timeout: 5s
retries: 5
redis:
image: redis:7-alpine
ports:
- "6379:6379"
volumes:
- redis_data:/data
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
timeout: 5s
retries: 5
celery-worker:
build:
context: .
dockerfile: docker/Dockerfile
command: celery -A app.core.celery_app worker --loglevel=info --concurrency=4
environment:
- DATABASE_URL=postgresql+asyncpg://pricetracker:${DB_PASSWORD}@postgres:5432/pricetracker
- REDIS_URL=redis://redis:6379/0
- CELERY_BROKER_URL=redis://redis:6379/1
depends_on:
- redis
- postgres
flower:
build:
context: .
dockerfile: docker/Dockerfile
command: celery -A app.core.celery_app flower --port=5555
ports:
- "5555:5555"
depends_on:
- redis
volumes:
postgres_data:
redis_data:
Output:
TEXT
CONTAINER ID IMAGE STATUS PORTS
abc123 nginx:latest Up 2 hours 0.0.0.0:80->80/tcp
(2) ▶ Example: .env file
INI
# .env - Docker Compose reads this automatically
DB_PASSWORD=change-me-in-production
SECRET_KEY=your-very-long-random-secret-key-at-least-32-chars
CELERY_BROKER_URL=redis://redis:6379/1
Output:
TEXT
// Execution Successful
5. Health Checks and Boot Sequence
(1) depends_on + healthcheck
The depends_on + condition: service_healthy in Docker Compose ensure the correct service startup order: PostgreSQL and Redis are ready first, and then FastAPI starts.
| Services | Startup Order | Health Checks |
|---|---|---|
| PostgreSQL | 1 (First) | pg_isready -U pricetracker |
| Redis | 1 (First) | redis-cli ping |
| FastAPI | 2 (After PG+Redis is ready) | GET /health |
| Celery Worker | 3 (after Redis and PostgreSQL are ready) | Internal heartbeat |
(1) ▶ Example: FastAPI Health Check Endpoint
PYTHON
@app.get("/health")
async def health_check(db: AsyncSession = Depends(get_db), redis: Redis = Depends(get_redis)):
# Check database connectivity
try:
await db.execute(select(1))
db_status = "healthy"
except Exception:
db_status = "unhealthy"
# Check Redis connectivity
try:
await redis.ping()
redis_status = "healthy"
except Exception:
redis_status = "unhealthy"
overall = "healthy" if db_status == "healthy" and redis_status == "healthy" else "unhealthy"
return {
"status": overall,
"database": db_status,
"redis": redis_status,
}
Output:
TEXT
# Function defined successfully
❓ FAQ
Q Why use a multi-stage build?
A A full Python image (including the compiler) is used during the build stage, while a slim image is used during runtime. The final image does not contain any build tools, is 5 to 10 times smaller, and has a smaller attack surface.
Q Is Docker Compose suitable for production?
A It is suitable for small and medium-sized projects. For large-scale production environments, use Kubernetes or Docker Swarm. Compose is suitable for development, testing, and small-scale deployments.
Q How do I handle database migrations?
A Run the migrations before the API container starts. Add an init container, or in
entrypoint.sh, run alembic upgrade head first before starting uvicorn.Q How are secrets securely transmitted?
A For development, use
.env files (added to .gitignore). For production, use Docker secrets or K8s Secrets; never hard-code them or commit them to Git.Q How do I set the number of workers?
A Formula:
(2 x CPU cores) + 1. For a 4-core server, set 9 workers. However, you should also consider memory limitations; each worker uses 50-100 MB.Q How do I view container logs?
A
docker compose logs api View FastAPI logs, docker compose logs -f monitor in real time. In production environments, use ELK/Loki to aggregate logs.📖 Summary
- Multi-stage Dockerfile: Dependencies are installed during the Builder stage, and only the results are copied during the Runtime stage, reducing the image size from 1.2 GB to 200 MB
- Full-Stack Orchestration with Docker Compose: API + PostgreSQL + Redis + Celery Worker + Flower
depends_on+healthcheckEnsures the following startup sequence: the database becomes ready first, followed by the API.envFile management environment variables; use Docker secrets in the production environment- The health check verifies the connection status of the endpoint database and Redis; the
HEALTHCHECKcommand performs automatic detection
📝 Exercises
- Basic Problem (Difficulty ⭐): Write a single-stage Dockerfile for PriceTracker, build the image, run the container, and verify that the
/healthendpoint is accessible. Hint:FROM python:3.12-slim+CMD ["uvicorn", ...] - Advanced Exercise (Difficulty ⭐⭐): Modify the Dockerfile to use a multi-stage build, and write a
docker-compose.ymlfile that includes three services: API, PostgreSQL, and Redis, so that they can be started with a single command:docker compose up. Hint:AS builder+COPY --from=builder - Challenge (Difficulty ⭐⭐⭐): Complete the Docker Compose orchestration—add Celery Worker and Flower services, add a health check, set a password for the
.envfile, and addentrypoint.shto run before startupalembic upgrade head. Hint:depends_on+condition: service_healthy+ entrypoint script
---|
