Docker layer caching: cut build times from 5 minutes to 30 seconds
A Docker build that reinstalls all dependencies on every code change is not a Docker build — it is a punishment. Layer caching is the difference between a 30-second build and a 5-minute one. The rules are simple but non-obvious, and most Dockerfiles I see violate at least two of them. Here is the complete guide.
How layer caching works
Every instruction in a Dockerfile creates a layer. Docker caches each layer based on:
- The instruction itself (the exact command string)
- The parent layer's cache key
- For
COPYandADD: the contents of the source files
If any layer is invalidated (changed), all subsequent layers are invalidated too. This is the key insight: put things that change frequently at the bottom, things that change rarely at the top.
The classic mistake: COPY before dependency install
# BAD: copies all source files first
# Any code change invalidates the npm install layer
FROM node:20-slim
WORKDIR /app
COPY . . # ← cache busts on every change
RUN npm ci # ← re-runs on every change (5+ minutes)
# GOOD: copy dependency files first, then install, then copy source
FROM node:20-slim
WORKDIR /app
COPY package.json package-lock.json ./ # ← only changes when deps change
RUN npm ci # ← cached unless package.json changes
COPY . . # ← can change without re-running npm ci
With this ordering: changing any source file only invalidates the final COPY . . layer. The npm ci layer — the slow one — stays cached. Build time drops from 5 minutes to 10 seconds.
Python with pip
# GOOD: install dependencies before copying source
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["python", "main.py"]
Multi-stage builds for smaller images
Multi-stage builds combine layer caching with final image size reduction:
# syntax=docker/dockerfile:1
FROM node:20-slim AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --omit=dev # production deps only
FROM node:20-slim AS builder
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci # include devDeps for build
COPY . .
RUN npm run build
FROM node:20-slim AS runtime
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY --from=builder /app/dist ./dist
COPY package.json .
USER node
CMD ["node", "dist/index.js"]
The runtime stage contains only production code and dependencies — no build tools, no dev dependencies, no source files.
BuildKit cache mounts for package manager caches
Even with perfect layer ordering, npm ci downloads from the internet every time the cache misses. BuildKit cache mounts persist the package manager cache between builds:
# syntax=docker/dockerfile:1
FROM node:20-slim AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN --mount=type=cache,target=/root/.npm npm ci --omit=dev
# Python equivalent
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache/pip pip install -r requirements.txt
COPY . .
The cache mount persists between builds on the same machine. A cold cache still downloads from the internet, but subsequent builds — even with different dependency versions — benefit from the cache for unchanged packages.
Using cache in GitHub Actions
# GitHub Actions with layer caching
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: myimage:latest
cache-from: type=gha
cache-to: type=gha,mode=max
The type=gha cache backend stores layer cache in GitHub Actions Cache. Each build restores the cache from the previous run, making cold CI builds rare.
Investigating cache misses
# Build with detailed output to see which layers hit/miss cache
docker build --progress=plain -t myimage . 2>&1 | grep -E "CACHED|RUN|COPY"
# Example output:
# => CACHED [deps 2/3] COPY package.json package-lock.json ./ 0.0s
# => CACHED [deps 3/3] RUN npm ci --omit=dev 0.0s
# => [builder 3/4] COPY . . 0.3s ← cache miss (source changed)
# => [builder 4/4] RUN npm run build 8.2s ← rebuilds from source
The ordering rules summarised
- Base image and tool installation (
apt-get install) — changes almost never - Dependency manifest files (
package.json,requirements.txt) — changes infrequently - Dependency installation (
npm ci,pip install) — changes when deps change - Application source code — changes constantly
- Build command — changes when source changes
Follow this order and your builds will consistently land in the 15-30 second range for incremental code changes, regardless of how many dependencies you have.