Your Dockerfile Is Slower Than a Manual Setup: Fix the Layering

You just made a one-line change to your app. Then you rebuild the image. Three minutes later you're still waiting on RUN apt-get install. Meanwhile, your teammate who manually runs commands inside a fresh container finishes in thirty seconds. What gives?

The answer is almost always poor layer ordering. Every Dockerfile instruction creates a new layer, and Docker caches layers based on the previous layer's hash. If you put your source code before your dependencies, any tiny edit invalidates the entire cache—including those slow package installs you already ran. Here's how to fix it.

Who Needs This and What Goes Wrong Without It

This is for anyone who writes Dockerfiles regularly, especially if you build images many times a day. It's for the side-project dev who notices their CI takes forever. It's for the team lead who sees developers complaining about slow builds and not connecting the dots to caching.

Without proper layering, you get three common problems. First, you rebuild a mountain of packages every time you edit a Python file. That's wasted time. Second, your image size bloats because you're carrying around build tools and temporary files that should have been stripped. Third—and this one hurts—you accidentally cache secrets like API keys because they got embedded in a layer you didn't mean to keep.

Imagine a typical Node.js Dockerfile. You write:

FROM node:18
COPY . /app
RUN npm install

That puts npm install after the COPY of your entire source tree. Change any file—even a comment—and the next build redoes npm install from scratch. If you have forty dependencies with native bindings, that's a three-minute penalty per rebuild. For a team of ten, each pushing twenty commits a day, that's ten hours of collective waiting. And it's entirely preventable.

The Cost of Bad Layering

It's not just time. Bad layering makes your production images fragile. A package update that breaks your app might sneak in because you rebuild dependencies more often than you should, and you miss the chance to pin versions correctly. Worse, if you ever copy credentials into an intermediate layer and forget to delete them from the final image, they're there forever. Docker layers are immutable—once committed, they persist in the registry until you explicitly remove them.

What You'll Be Able to Do After Reading This

By the time you finish this article, you'll know exactly which instructions should go first (stable dependencies), which go second (configuration and tooling), and which go last (your code). You'll be able to write a Dockerfile that takes advantage of Docker's layer cache so that a one-line code change rebuilds in under ten seconds. You'll also learn when not to follow the advice—cases where chaining everything into a single RUN actually makes more sense.

Prerequisites and Context to Settle First

Before we dive into ordering rules, you need a solid grasp of what a Docker layer actually is. Every FROM, RUN, COPY, and ADD instruction creates a new layer. Each layer is a diff of the filesystem changes made by that instruction. Docker caches these layers locally, and when you build an image it tries to reuse cached layers if the instruction text and the parent layer haven't changed.

The key insight is that Docker caches a layer only if all layers before it are still cached and the instruction itself hasn't changed. So if you change a COPY instruction that adds a file, that layer and every layer after it will be rebuilt. That's why ordering matters so much. Put things that change rarely at the top, and things that change often at the bottom.

What Types of Changes Invalidate Caches

Not all changes are equal. A RUN apt-get update instruction that stays the same text will still be cached as long as the base image hasn't changed. But the contents inside COPY are hashed based on the actual files being copied. So even if the instruction text is identical, changing a byte in a file invalidates that COPY layer.

This is a common trap. You might write a Dockerfile that copies package.json and npm-shrinkwrap.json before running npm install, and that works well—until someone modifies the Dockerfile itself or the build context has extra files that trigger a hash mismatch. Always specify exactly what you're copying; avoid COPY . /app if you can.

One Important Exception: Multi-Stage Builds

Multi-stage builds are a separate pattern that solves a different problem (reducing final image size), but they interact with layering. In a multi-stage build, you can copy artifacts from one stage to another without carrying over the intermediate layers. This means you can have a “builder” stage that installs compilers and dev dependencies, and a “runtime” stage that only gets the compiled output. The layering advice still applies to each stage independently—you want stable things first in both stages.

For example, if your builder stage compiles a Go binary, you'd copy go.mod and go.sum first, run go mod download, then copy the rest of the source and run go build. That way, dependency changes are cached unless you actually modify the module files.

Core Workflow: Order Your Dockerfile in Three Phases

Let's walk through the ordering process step by step, using a Python Flask app as our example. The same principle applies to Node.js, Go, Java, or anything else.

Phase 1: Base Image and System Dependencies

Start with the FROM instruction. Use a specific tag like python:3.11-slim-bookworm rather than python:latest. A pinned base image doesn't change unless you explicitly update the tag, so it will be cached almost always.

Next, install system packages that your app needs. These change maybe once a month when you add a new package. Put them in a single RUN instruction to keep the number of layers low:

FROM python:3.11-slim-bookworm AS builder
RUN apt-get update && apt-get install -y --no-install-recommends \
 build-essential libpq-dev && \
 rm -rf /var/lib/apt/lists/*

The rm -rf at the end cleans up the apt cache, reducing layer size. Never leave that out; it saves tens of megabytes.

Phase 2: Application Dependencies

After system dependencies, copy only the files that declare your application's dependencies—requirements.txt for Python, package.json for Node.js, go.mod for Go. Then run the package manager:

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

This is the trick that saves the most time. As long as requirements.txt stays the same, this layer is cached. When you change your application code, you skip the expensive pip install entirely.

One note: if you have a requirements.txt that changes often, consider splitting it into a "core" dependencies file and a "dev" file. The core file goes into this phase; the dev file can go later if needed.

Phase 3: Application Code

Now copy the rest of your source code. Since this layer changes every time you edit a file, it should be last:

COPY . /app
WORKDIR /app
CMD ["python", "app.py"]

Because everything above this point is cached, rebuilding after a code edit just runs this one COPY step. The whole build takes seconds.

Putting It All Together

Here's the complete Dockerfile with phases clearly separated:

FROM python:3.11-slim-bookworm AS builder
RUN apt-get update && apt-get install -y --no-install-recommends build-essential libpq-dev && rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
EXPOSE 8000
CMD ["gunicorn", "-w", "4", "app:app"]

That's it. Now when you change a Python file, the first three layers (FROM, apt-get, pip install) are cached, and only the final COPY runs.

Tools, Setup, and Environment Realities

Getting this right depends on a few environment factors you should check beforehand.

Docker BuildKit

BuildKit is the modern builder introduced in Docker 18.09. It's now default in Docker Desktop, but if you're on an older server installation, you might still use the legacy builder. BuildKit offers better caching and can skip unused stages. You can explicitly enable it with DOCKER_BUILDKIT=1 docker build ..

One feature of BuildKit that helps with layering is the --mount=type=cache option. You can mount a persistent cache directory for package managers so that even if you rebuild from scratch, you don't download packages again.

.dockerignore Files

If you don't have a .dockerignore file, you're probably copying more than you think. The Docker build context includes everything in the current directory unless you explicitly exclude it. If you have a node_modules or .git folder, those files get hashed and invalidate your COPY layer even though they're not used.

Create a .dockerignore with at least:

node_modules
.git
*.md
__pycache__

This reduces the context size and prevents cache busts from irrelevant files.

CI/CD Build Machines

On a typical CI runner, you get a fresh VM each time. That means the Docker layer cache starts empty. Without caching, proper ordering still helps a little (because unchanged layers are skipped in the build output), but you don't get the full speedup. To fix that, you need to save and restore the cache between builds.

Many CI services (GitHub Actions, GitLab CI, CircleCI) offer Docker layer caching via docker save/docker load or dedicated actions. For example, in GitHub Actions you can use docker/build-push-action@v5 with cache-from and cache-to pointing to a registry or local cache storage. Without this, every CI build starts from scratch regardless of your Dockerfile ordering.

So the advice in this article gives you half the benefit on CI—you still need explicit cache persistence for full speed. But for local development, proper ordering alone can cut build times by 90%.

Variations for Different Constraints

Not every project fits the standard pattern. Here are three common variations.

Monorepo with Multiple Services

If you have a monorepo where multiple Dockerfiles share common base dependencies, consider using a shared base image. Build a common "base" Dockerfile that installs all shared system packages and common libraries, tag it, and then have each service Dockerfile start with FROM myorg/base:latest. This way, the slow system installs are built once for the whole repo.

The downside is that the base image becomes a separate artifact that you need to maintain and push. If the base image changes, all downstream images rebuild their next layers. But the net gain is usually positive for teams with more than three services.

Languages That Compile Natively (Rust, Go, C++)

For compiled languages, the dependency phase is often a download step (e.g., cargo build for Rust) that produces compiled artifacts. The trick is to separate the dependency download from the compilation. For Rust, you can copy Cargo.toml and Cargo.lock first, create a dummy src/main.rs, run cargo build to download and compile dependencies, then overwrite with the real source and do a final build. The second build reuses already-compiled artifacts.

This pattern is sometimes called "dependency caching via a dummy build." It works for Go too: copy go.mod and go.sum, run go mod download, then copy the rest. The key is that go mod download is much faster on subsequent runs when the module files haven't changed.

Field note: containerization plans crack at handoff.

Field note: containerization plans crack at handoff.

When to Ignore This Advice

Sometimes it's better to combine everything into a single RUN instruction. For example, if your base image already carries most dependencies and you only install one or two small packages, the layering overhead might not be worth it. Also, if you build an image only once a week and never change it, there's no real benefit—just write a simple Dockerfile.

Another case: if you're using a tool like sbt (Scala) that caches dependencies internally, you might want to keep the source code in the build context earlier to let sbt resolve incremental changes. But that's a niche scenario—most projects benefit from the standard ordering.

Pitfalls, Debugging, What to Check When It Fails

Even with perfect ordering, things can go wrong. Here's what to look for.

Pitfall 1: COPY Before RUN Causes Cache Miss

The most common mistake is copying .dockerignore-excluded files without realizing it. Say your .dockerignore lists node_modules, but someone removes that line. Suddenly, the COPY layer includes the entire node_modules directory, which changes frequently, and every npm install layer after it's rebuilt. Check your .dockerignore regularly.

Pitfall 2: Clock Skew and Cache Invalidation

If you notice that the same build on different machines produces different layer hashes, check file timestamps. Some build systems (especially inside CI) set file modification times to the current time during checkout, which changes the hash of files even if content is the same. This is rare but has been reported with Git LFS or shallow clones. The fix is to normalize timestamps inside the Dockerfile using touch --reference or a dedicated tool like reproducible-builds-maven-plugin.

Pitfall 3: Secrets Leaking Through Layers

If you ever use COPY to add a file containing secrets (like an SSH key) to install private packages, that secret ends up in a layer. Even if you RUN rm the file in the next instruction, the secret is still in the previous layer and can be extracted with docker history. Use BuildKit's --secret mount instead:

RUN --mount=type=secret,id=ssh_key \
 mkdir -p ~/.ssh && cp /run/secrets/ssh_key ~/.ssh/id_rsa && \
 pip install git+ssh://[email protected]/private/repo.git && \
 rm -rf ~/.ssh

This keeps the secret entirely out of the image layers.

Debugging Cache Misses

If a build seems to ignore a cached layer, check the build output. Docker prints --- Using cache for cached layers. If you see Step X/XX : RUN apt-get update running fresh, look at the message before it—Docker often says which hash changed. You can also use docker build --no-cache-filter= to selectively invalidate layers for debugging.

Another trick: run docker history --no-trunc myimage to see the size of each layer. If a layer that should be small is large, you might have accidentally included build artifacts or temporary files.

FAQ and Checklist in Prose

Let's answer the most common questions with concrete advice, then give you a final checklist you can paste next to your editor.

FAQ

Why does my Dockerfile build the same layer even when nothing changed?
Check if the base image tag is floating (e.g., node:latest). The tag might have updated, invalidating the first layer. Pin to a specific version.

Does the order of ARG and ENV matter for caching?
Yes. ARG and ENV are instructions that create layers only if they change the filesystem? Actually, ENV sets environment variables that persist in the image metadata, but they don't create a new filesystem layer. However, if you reference an ARG in a RUN instruction, the value of that ARG is part of the instruction string, so changing ARG invalidates that RUN layer. Put ARG instructions as early as possible, but be aware that changing them will bust the cache for all following layers that use them.

Should I use a separate Dockerfile for development and production?
Often yes. A development Dockerfile might include hot-reload tools and debuggers, while a production one strips those out. The layering advice applies to both. You can also use multi-stage to keep one Dockerfile that starts with a common base stage and then branches for dev and prod.

Final Checklist

1. Pin your base image to a specific version tag.
2. Add a .dockerignore that excludes unnecessary files.
3. Install system packages in a single RUN with cleanup.
4. Copy dependency manifests first, then run package manager.
5. Copy application code last.
6. Use multi-stage builds when you need to separate build and runtime.
7. Use BuildKit features like --mount=type=cache for persistent caches.
8. Cache secrets with --mount=type=secret instead of copying them.
9. On CI, set up external layer caching (e.g., registry cache).
10. Run docker history to audit layer sizes and detect unwanted files.

That's the full layering fix. You'll still wait for the first build—there's no escaping that—but every subsequent change will be measured in seconds, not minutes. And you'll never have to ask yourself why your Dockerfile is slower than a manual setup again.

Treat the first failed reading as a process signal, not a personal mistake—the fix is usually in the checklist order.

— A biomedical equipment technician, clinical engineering

When specs conflict, default to the manufacturer IFU over tribal knowledge; auditors notice the difference.

— A hospital biomedical supervisor, device maintenance