Mirror Optimization and Best Practices
Optimizing images isn't just about reducing their size—smaller images mean faster deployments, lower storage costs, and a smaller attack surface.
1. What You'll Learn
- Strategies for Optimizing the Number of Mirroring Layers
- Tips for Combining RUN Commands
- Dependency Cleanup and Cache Management
- Using Dockerfile Lint Tools
- Developing Cache Optimization Methods
2. A True Story of a CI Engineer
(1) Pain Point: CI build time has skyrocketed from 8 minutes to 25 minutes
Bob’s CI build time suddenly skyrocketed from 8 minutes to 25 minutes. Upon investigation, he discovered that every build was reinstalling all dependencies—because COPY . . was placed before RUN pip install, causing the dependency installation cache to expire with every code change. The 20-minute wait significantly reduced development efficiency.
(2) Solutions for Cache Optimization
By rearranging the order of the Dockerfile instructions and moving the rarely changing COPY package.json to the beginning, we were able to use layer caching to reduce the build time back to 3 minutes.
DOCKERFILE
# Before: code change invalidates pip cache
COPY . .
RUN pip install -r requirements.txt
# After: dependency change rate << code change rate
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
(3) Benefits: Build time reduced from 25 minutes to 3 minutes
By simply rearranging the order of two lines of code, the build time dropped from 25 minutes to 3 minutes (when the cache is hit), resulting in an 8-fold increase in CI pipeline efficiency.
3. Key Strategies for Mirror Optimization
(1) Before-and-After Comparison Framework
graph TB
subgraph Before["Before Optimization"]
B1["FROM ubuntu:22.04<br/>77 MB"] --> B2["RUN apt-get install<br/>300 MB"]
B2 --> B3["COPY . .<br/>with node_modules"]
B3 --> B4["RUN npm install<br/>200 MB"]
B4 --> B5["Final: 1.2 GB"]
end
subgraph After["After Optimization"]
A1["FROM node:20-alpine<br/>135 MB"] --> A2["COPY package.json"]
A2 --> A3["RUN npm ci<br/>50 MB"]
A3 --> A4["COPY . .<br/>None node_modules"]
A4 --> A5["Final: 180 MB"]
end
(2) Overview of Optimization Strategies
| Strategy | Effectiveness | Difficulty of Implementation |
|---|---|---|
| Switch to a base image (alpine/slim) | Size ↓50–80% | ⭐ |
| Multi-stage build | Volume ↓60–90% | ⭐⭐ |
| RUN Merge + Clear Cache | Size ↓20–40% | ⭐ |
| Adjust COPY Order | Build Speed ↑3–8x | ⭐ |
| .dockerignore | Context ↓30–90% | ⭐ |
| hadolint check | Potential issues found | ⭐⭐ |
4. Merging RUN Commands
(1) Merger Principle
Each RUN command creates a new layer. Combine related operations into a single RUN command and clean up temporary files within the same layer.
▶ Example: Combining Commands (Difficulty ⭐⭐)
DOCKERFILE
# Bad: 3 separate layers, apt cache stays in image
RUN apt-get update
RUN apt-get install -y curl
RUN apt-get install -y nginx
# Good: 1 layer, cache cleaned within the same layer
RUN apt-get update && \
apt-get install -y --no-install-recommends \
curl \
nginx && \
rm -rf /var/lib/apt/lists/*
(2) The effect of --no-install-recommends
| Option | Installation Content | Typical Space Savings |
|---|---|---|
| Default | Main Packages + Recommended Packages + Suggested Packages | Baseline |
--no-install-recommends |
Main package + dependencies only | ↓30–50% |
5. Optimizing Cache Hit Rate
(1) Layer Caching Mechanism
When Docker builds an image, if the instructions and context of a layer remain unchanged, the cache is reused. Once a layer’s cache expires, all subsequent layers must be rebuilt.
graph TB
L1["FROM python:3.12<br/>✅ Cache Hit"] --> L2["WORKDIR /app<br/>✅ Cache Hit"]
L2 --> L3["COPY requirements.txt<br/>✅ Cache Hit (no change)"]
L3 --> L4["RUN pip install<br/>✅ Cache Hit"]
L4 --> L5["COPY . .<br/>❌ Cache Miss (code changed)"]
L5 --> L6["CMD python app.py<br/>❌ Rebuild"]
▶ Example: Comparing builds with and without --no-cache (Difficulty: ⭐⭐)
BASH
# Normal build: uses cache where possible
docker build -t myapp:1.0 .
# Force rebuild without any cache
docker build --no-cache -t myapp:1.0 .
# Use a previous image as cache source (CI optimization)
docker build --cache-from=myapp:latest -t myapp:1.0 .
(2) Optimize the order of Dockerfile instructions
DOCKERFILE
# ============================================
# Optimized Dockerfile: high cache hit rate
# ============================================
FROM node:20-alpine
WORKDIR /app
# 1. Least frequently changed: package metadata
COPY package*.json ./
# 2. Dependency installation (cached unless package.json changes)
RUN npm ci --only=production
# 3. Most frequently changed: application code
COPY . .
EXPOSE 3000
CMD ["node", "server.js"]
| Scenario Changes | Optimized Version | Unoptimized Version |
|---|---|---|
| Code changes only | ✅ npm ci uses cache (5 seconds) | ❌ npm ci rebuilds (3 minutes) |
| Update dependency version | ✅ Rebuild with npm ci (required) |
❌ Rebuild with npm ci (same as above) |
| Modify Dockerfile | ✅ Rebuild only the affected layers | ❌ Rebuild all |
6. Image Analysis Tools
▶ Example: Analyzing Image Layers with dive (Difficulty: ⭐⭐)
dive is an interactive mirror layer analysis tool that shows which files were added, modified, or deleted in each layer.
BASH
# Install dive (Linux)
wget https://github.com/wagoodman/dive/releases/download/v0.12.0/dive_0.12.0_linux_amd64.deb
sudo dpkg -i dive_0.12.0_linux_amd64.deb
# Analyze an image
dive nginx:latest
💡 Tip: The dive interface displays: ① the size and efficiency score for each layer on the left; ② the file tree added to each layer on the right. Red markings indicate wasted space (files that have been overwritten or deleted by subsequent layers).
▶ Example: Checking a Dockerfile with hadolint (Difficulty: ⭐⭐)
hadolint is a linting tool for Dockerfiles that checks for violations of best practices.
BASH
# Install hadolint (Linux)
wget -O /usr/local/bin/hadolint https://github.com/hadolint/hadolint/releases/download/v2.12.0/hadolint-Linux-x86_64
chmod +x /usr/local/bin/hadolint
# Check a Dockerfile
hadolint Dockerfile
💻 Output:
TEXT
Dockerfile:3 DL3013 warning: Pin versions in pip. Instead of `pip install <package>` use `pip install <package>==<version>`
Dockerfile:5 DL3059 info: Do not use `--no-cache-dir` in pip install. It is redundant when `--cache-from` is used.
Dockerfile:7 DL3008 warning: Pin versions in apt-get install. Instead of `apt-get install <package>` use `apt-get install <package>=<version>`
7. Complete Comparison of the Dockerfile Before and After Optimization
(1) Examples of Node.js Application Optimization
DOCKERFILE
# ============================================
# BEFORE: Unoptimized Dockerfile (1.2 GB)
# ============================================
FROM node:20
WORKDIR /app
COPY . .
RUN npm install
EXPOSE 3000
CMD ["npm", "start"]
DOCKERFILE
# ============================================
# AFTER: Optimized Dockerfile (180 MB)
# ============================================
# 1. Alpine base instead of full Node
FROM node:20-alpine
WORKDIR /app
# 2. Copy dependency metadata first (cache optimization)
COPY package*.json ./
# 3. Production-only dependencies
RUN npm ci --only=production && \
npm cache clean --force
# 4. Copy source code (most frequently changed)
COPY . .
# 5. Run as non-root user
RUN addgroup -S appgroup && \
adduser -S appuser -G appgroup && \
chown -R appuser:appgroup /app
USER appuser
EXPOSE 3000
HEALTHCHECK --interval=30s --timeout=5s \
CMD wget -qO- http://localhost:3000/health || exit 1
CMD ["node", "server.js"]
(2) Comparison of Optimization Results
| Metric | Before Optimization | After Optimization | Improvement |
|---|---|---|---|
| Image Size | 1.2 GB | 180 MB | 6.7x ↓ |
| Build Time (Code Changes) | 3 minutes | 5 seconds | 36x ↓ |
| Build Time (Dependency Changes) | 3 minutes | 45 seconds | 4x ↓ |
| Security Vulnerabilities | 200+ | 30 | 6.7x ↓ |
| CIS Benchmark Score | 45 | 85 | +40 |
8. Language-Specific Installation Policies
| Language | Dependencies | Installation Command | Cleanup Command |
|---|---|---|---|
| Node.js | package*.json |
npm ci --only=production |
npm cache clean --force |
| Python | requirements.txt |
pip install --no-cache-dir |
(Built-in --no-cache-dir) |
| Go | go.mod go.sum |
go mod download |
go clean -modcache |
| Java | pom.xml / build.gradle / mvn dependency:resolve |
Clean .m2 cache | |
| Ruby | Gemfile Gemfile.lock |
bundle install --without dev |
Cleanup .bundle Cache |
9. Complete Example: End-to-End Optimization of a Node.js Application
DOCKERFILE
# ============================================
# Fully optimized Node.js production Dockerfile
# Features: alpine, multi-stage, cache, non-root
# ============================================
# ---------- Stage 1: Build ----------
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build
# ---------- Stage 2: Production ----------
FROM node:20-alpine
WORKDIR /app
# Copy only production dependencies
COPY package*.json ./
RUN npm ci --only=production && \
npm cache clean --force
# Copy built application from builder
COPY --from=builder /app/dist ./dist
# Security: non-root user
RUN addgroup -S appgroup && \
adduser -S appuser -G appgroup && \
chown -R appuser:appgroup /app
USER appuser
EXPOSE 3000
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD wget -qO- http://localhost:3000/health || exit 1
CMD ["node", "dist/server.js"]
BASH
# Build and verify
docker build -t node-app:optimized .
# Compare sizes
docker images node-app
# Analyze with dive
dive node-app:optimized
# Check with hadolint
hadolint Dockerfile
❓ FAQ
Q Does each RUN increase the image size?
A Yes, even if the RUN only deletes files—because a new layer records the "delete operation," while the underlying layer still exists. Therefore, "installation + cleanup" must be completed within the same RUN, so that the temporary files created during installation and the delete operation are contained within the same layer, and the underlying layer cannot see the temporary files.
Q
apt-get clean Why does it reduce the image size?A
apt-get clean alone is almost ineffective (the same applies to rm). It must be used within the same RUN: apt-get update && apt-get install && rm -rf /var/lib/apt/lists/*. --no-install-recommends is more effective than apt-get clean.Q How can I tell if the cache was hit?
A In the
docker build output: Using cache indicates a hit, and Step 3/7 : RUN xxx indicates a rebuild. You can also use --progress=plain to view detailed logs. In CI, docker build --cache-from=app:latest allows you to reuse the cache from the previous build.Q
--no-cache When should it be used?A In two scenarios: ① When the base image requires a security update and needs to be rebuilt; ② When a build fails and you suspect the cache is corrupted. Do not use
--no-cache during daily development, as it will cause every build to start from scratch.Q How do I detect security vulnerabilities in images?
A
docker scout quickview <image> (official Docker tool) or trivy image <image> (open source). Trivy is the most popular open-source image scanning tool, capable of detecting known CVEs in OS packages and language dependencies. We recommend integrating a scanning step into your CI pipeline.📖 Summary
- Core of image optimization: Reduce the number of layers + Clean up within the same layer + Choose a smaller base image
- RUN Merge Principle: Combine related operations into a single RUN; installation and cleanup within the same layer
- Cache optimization: Place instructions that rarely change at the beginning (COPY dependency files), and those that change frequently at the end (COPY source code)
--no-install-recommendsReduces installation volume by 30–50%- Use
diveto analyze image layer efficiency, andhadolintto check for Dockerfile best practices - Complete optimization process: Alpine base image → multi-stage build → cache optimization → non-root → health check
📝 Exercises
- Basic Question (Difficulty ⭐): Use
diveto analyze each layer of an existing image (such asnginx:latest), noting which layer is the largest and what files have been added. - Advanced Exercise (Difficulty ⭐⭐): Use hadolint to check the Dockerfiles from Lessons 7–9 and fix all warning-level suggestions.
- Challenge (Difficulty: ⭐⭐⭐): Compare the image sizes of the same Python application built using the alpine, debian, and slim base images, and analyze the reasons for the differences in size.
