Docker Rebuilds Everything: Layer Order Fixes

Your builds used to take five seconds. Now they take five minutes, and you haven't changed anything except a line of JavaScript. The classic fix is tossing in a --cache-from flag and hoping. But the real problem is usually simpler: your Dockerfile is in the wrong order.

Here's a dirty secret from people who've been doing this since the old docker assemble days: Docker's layer cache is not magic. It's a chain of snapshots, and every time you copy a file that changes, every step after it reruns. If you copy your entire app before installing dependencies, then a one-line edit invalidates everything. This article walks you through the exact steps to fix that, plus the gotchas you'll hit when you try to keep your builds healthy.

Slow builds are a symptom, not a feature

The cost of a broken cache in numbers

You run docker assemble after changing one line of code. Then you wait. Thirty seconds, a minute, maybe three if dependencies are heavy. That's the symptom. Nobody measures it, but the math is brutal: five rebuilds a day at two minutes each is roughly 10 minutes lost per developer. A five-person team burns nearly an hour daily. Monthly, that's over a full work week spent staring at progress bars.

Most teams shrug it off. It's just construct time, right? Wrong. The hidden cost isn't the minutes themselves—it's the interrupted flow. You switch context, check Slack, lose the thread of what you were debugging. Every slow rebuild resets your mental state.

Why developers accept slow builds

Here's the uncomfortable truth: we tolerate slow builds because they're predictable. You expect the pain, so you plan around it. Coffee refills. Inbox triage. The quiet dread of watching a bar crawl at 13% while your adjustment is trivial.

The catch is that accepting slow builds becomes a habit. You stop questioning whether the form itself is efficient—you just endure it. I have sat through countless rebuilds that recompiled unchanged dependencies, simply because nobody stopped to look at the Dockerfile's layer order. That's not engineering discipline; that's learned helplessness.

What 'layer order' really buys you

Docker caches each layer, then invalidates everything after the first adjustment. Put your package installation before your source code, and a one-character edit doesn't touch the dependency cache. That's the entire game.

Wrong order—and you're re-downloading apt packages or npm installs every single time. Not because something legitimately changed, but because the cache sealed in the wrong sequence. We fixed this once on a Node project by moving package.json and npm install above the whole source directory. Rebuilds dropped from 4 minutes to under 20 seconds, and that was just one file's position.

Ordering isn't cleverness. It's remembering that Docker walks top-to-bottom, and every instruction is a checkpoint. Static requirements first. Volatile code last. Dependencies live in layers that rarely need regeneration.

That said, don't expect this to solve every slow form. Some images resist reordering—git clones with dynamic refs, monorepo builds that churn on shared contexts. But the common case, the daily docker construct that makes you groan, usually crumbles with one persistent edit: move what doesn't revision above what does.

You don't have a slow assemble problem. You have a cache-invalidation problem wearing a trench coat.

— field note from a debugging session that took three hours longer than it should have

That realization is the lever. Fix the order, and you'll watch the cache hit where it should—before you've touched a single line of application code. That's the highest-leverage revision you'll make all week.

Layer caching in plain English

Docker layers as transparent snapshots

Think of a Docker image like a stack of transparent slides. Each instruction in your Dockerfile—RUN apt-get install, COPY . ., CMD ["node", "app.js"]—lays down one slide on top of the last. The final image is what you see when you hold all slides up to light together. Each slide is a layer, and layers are read-only. Once a slide goes down, it never changes; the only way to "edit" it's to add a new slide on top that shadows the old one.

This stack is why caching works. When Docker builds, it checks each slide in order. If a slide's checksum matches one from a previous form, Docker skips straight past it—no re-execution needed. But here's the catch: the moment one slide changes, every slide above it's rebuilt too. You don't just pay for the changed instruction. You pay for everything after it. Wrong order, and a single source-code tweak can trigger a full apt-get re-run, a full npm install, a full compile. That hurts.

Layers are like poker chips: you can only stack new ones on top, and the stack collapses the moment one chip changes.

— field note from a Kubernetes migration post-mortem

The golden rule: what changes goes last

The practical rule is simple: your source code changes constantly, so it goes at the bottom—the last layers. Your dependencies shift rarely, so they live near the top—the first layers. That way, when you edit a line in src/index.js, Docker reuses the dependency layers and only rebuilds the files that actually moved. It's a tiny reorder that saves you minutes per iteration, and I have seen teams cut a 12-minute deploy loop down to 90 seconds by moving COPY package.json above COPY . ..

The instinct is to write instructions in chronological order—install everything, then copy code, then run. That sounds fine until you realize the COPY . . layer invalidates everything beneath it whenever any file changes, even a stray README.md edit. The remedy is treating your Dockerfile like a cache map: frequency of shift, not execution order, decides where instructions sit. Install system packages first. Copy dependency manifests next. Run package managers third. Only then do you copy source code.

What usually breaks first is the RUN npm install step. If you copy package.json and package-lock.json before the full source tree, Docker can cache the install until those manifests adjustment. Miss that split, and every assemble re-resolves the entire dependency graph. That said, there's a wrinkle—the RUN npm install layer still depends on the base image and the system packages above it. revision the base from node:18 to node:20, and the whole install layer rebuilds anyway. That's not a failure. It's the layer cache working exactly as it should.

Reading a Dockerfile like a cache map

You develop an eye for this after a few painful construct logs. Start reading the Dockerfile top-down and ask: "What changes per push?" If the answer is not "the last three lines," your layers are misaligned. The COPY instructions are the usual suspects—they snapshot files, and any modification to those files breaks the cache. A good heuristic: separate config from code, lockfiles from node_modules, and environment-specific settings into .dockerignore patterns.

There's a subtlety worth flagging: layer caching is per-instruction, not per-file. A single RUN apt-get install && apt-get clean is one layer; splitting it across two RUN statements creates two layers, doubling your cache granularity but also your image size. The trade-off is yours to make. I'd rather keep the layers chunky enough to stay cacheable, but not so chunky that a tiny config shift nukes a 500MB layer.

And yes, there are times reordering is not enough—the edge cases come later. But if you walk away with one thing, it's this: every Dockerfile instruction is a bet on what changes. Put the volatile files last, and you'll stop watching Docker rebuild everything while your coffee gets cold. The image stays lean, the builds stay fast, and the layer cache does the heavy lifting you didn't ask it to do.

Inside the form: how Docker decides what to reuse

Inheritance vs invalidation

Docker doesn't rebuild your image from scratch every time — it reuses what it can. The catch is that each instruction in your Dockerfile carries a parent: the layer produced by the line before it. That lineage matters more than the instruction itself. If the parent changes, everything downstream changes too. You don't get to keep the old layer and bolt on a new one; the whole chain snaps.

Think of it like a stack of cards. You can't swap out the third card without rebuilding the fourth, fifth, and sixth. That's why a single RUN apt-get install buried deep in your file can invalidate every COPY after it — even when those COPYs haven't touched a single file. The stack is only as stable as its weakest seam.

The cache key: instruction + parent layers

Docker computes a checksum for each instruction, but that checksum isn't just the text of the command. It's a hash of the instruction plus the hash of the parent layer. shift one flag in an early ENV line, and the hash ripples forward like a bad rumor. Everything after it gets re-executed, even if the output would be identical. I have seen teams lose an entire afternoon to a single ARG that shifted a variable — not because the assemble broke, but because the cache silently died.

Here's the mechanical truth: Docker doesn't compare file contents between runs. It only looks at the hashes. So if you edit a file that a COPY references, the hash changes, and the layer rebuilds. But if you edit a file after that COPY — say, in a later RUN — the COPY layer stays cached. That's the asymmetry. The order of instructions isn't just style; it's the entire caching strategy.

Most teams skip this nuance and just reorder their Dockerfile once, hoping for the best. It works, usually. But the real lever is knowing which instructions are cache-friendly (RUN with stable commands) and which are cache-killers (COPY with volatile source directories).

Why COPY is a cache killer (and how to tame it)

COPY is the bluntest tool in the Dockerfile drawer. It grabs a directory, slurps it into a layer, and hashes the whole thing. adjustment one file in src/ and the layer dies. That's fine if you copy everything once at the end. But if you COPY early — say, to get your package manager config in place — you're betting that the source files won't revision. They will. They always do.

The fix isn't to stop using COPY; it's to make each COPY as small and deliberate as possible. Copy only what you need for the current step. For a Node app, copy package.json and package-lock.json first, run npm install, then copy the rest of the source. That way, dependencies stay cached until you actually adjustment them. The source copy at the end rebuilds every time, but that layer is cheap — it's just file transfer, not a compile or a dependency fetch.

What usually breaks first is the opposite: teams copy everything early "to keep it simple." Simple, yes. Fast, no. You'll rebuild node_modules on every commit, even when your dependencies haven't moved. The cache key doesn't care about your intent; it only sees the changed hash.

Reorder your Dockerfile like you'd order a kitchen: heavy equipment first, perishables last. The cache is your fridge, not a pantry.

— rough note from a staging-server postmortem, after a 14-minute assemble shrank to 90 seconds

That said, reordering has limits. If your construct script writes timestamps into files or downloads a versioned tarball that changes daily, no amount of layer juggling will save you. The checksum is ruthless. You can't cheat it with --no-cache tricks or clever reordering; you have to revision the input. But for most projects, the win is real. A tuned COPY sequence cuts builds from minutes to seconds — not because the work is faster, but because the wasted work disappears.

A before-and-after Dockerfile walkthrough

Step by step: rearranging instructions

Here's the average Dockerfile I keep seeing in repos. It installs Python dependencies, copies source, runs tests, then builds a frontend. It works fine. It also rebuilds everything every single time, even when you only touched a README.

 FROM node:20-alpine WORKDIR /app COPY . . RUN npm ci RUN npm test RUN npm run form 

That's a full wipe-and-rebuild on every shift. The problem isn't the tools. It's the order. Docker caches by instruction, but only if the instruction's inputs haven't changed. When you copy the whole repo up front, a single edit to docs/notes.md invalidates everything downstream—the npm install, the tests, the construct. All of it. The cache never gets a chance to help.

The before: why it rebuilt everything

Let me break down what actually happens on that first npm ci. Docker sees the COPY . . layer, checks if anything in the directory changed, and throws the cache away if so. Then it runs npm ci—again, because the previous layer changed. Then tests run on a fresh environment, which is slow and pointless when you only edited a Markdown file.

I have seen teams sit through 4-minute builds for a one-line typo fix. That's not a hardware problem. It's a layer-order problem. The fix is boring and mechanical: copy only what each step needs, in the order of volatility. Least-changing stuff first. Most-changing stuff last.

The after: what caches and what doesn't

Here's the rewritten version:

 FROM node:20-alpine WORKDIR /app COPY package.json package-lock.json ./ RUN npm ci COPY . . RUN npm test RUN npm run build 

First move: copy the lockfile and manifest before the source code. That way, npm ci only reruns when your dependencies actually adjustment. New feature code? Cache hit on npm ci. Update ESLint config? Cache hit. You only pay the install cost when the dependency tree shifts—which is rare compared to code edits.

The catch: you're trading correctness for speed if the build step depends on files outside the copied context. Some projects generate files in a preinstall hook or pull from a monorepo sibling. If your npm ci reads anything beyond the two files you copied, that cache will silently serve you stale layers. Not a theoretical issue. I've debugged one of those and it made me question my life choices.

Second move: keep the COPY . . right before the test step. Tests need the full source, so you can't avoid that copy. But now it only invalidates the test and build layers—not the expensive install. That's the core payoff: dependency installation gets cached across 90% of your commits, and the rebuild time drops from "go get coffee" to "blink twice."

One more trick worth flagging—if you have a separate Dockerfile for prod, split your build and runtime stages. The build stage includes everything above; the runtime stage copies only node_modules and the built output. Keeps image size down and the cache logic separate. Most teams skip that step until their CI bill arrives.

Reordering a Dockerfile isn't clever. It's just admitting that Docker caches in one direction only, and you might as well work with it.

— senior platform engineer, mid-incident coffee chat

The ugly truth is that reordering solves the common case but not every case. Some builds bake secrets into layers, or run code generation against the whole repo, and no amount of shuffling helps. When you hit those walls—and you will—you start looking at multi-stage builds, or SKIP flags, or BuildKit features like RUN --mount=type=cache. But fix the order first. It's free, it's fast, and it typically kills half the rebuild time without touching any logic.

Field note: containerization plans crack at handoff.

Edge cases: when reordering is not enough

Multi-stage builds and cache scoping

Multi-stage builds change the rules mid-game. You might reorder layers perfectly in your builder stage, only to find the final runtime stage rebuilds everything anyway. That's not a bug—it's scoping. Each stage maintains its own cache, and Docker only invalidates downstream stages when the copied artifact actually changes. But here's the trap: if your builder stage produces a tarball with a timestamp embedded, that artifact changes every run. So the copy layer in your runtime stage busts its cache even though the source code never moved. The fix isn't reordering—it's making the artifact deterministic. Strip timestamps, sort file lists, and pin dependency versions.

You can also lean on --target to build only the stage you need during development. Most teams skip this. They run the full multi-stage build locally, watching the cache miss cascade from stage one to stage three. We fixed this by adding a Makefile target that runs docker build --target dev for local iteration. The prod stage still benefits from fresh cache on CI, but developers stop paying the tax. That one change cut local rebuilds from ninety seconds to twelve.

Field note: containerization plans crack at handoff.

The ARG vs ENV gotcha

Here's a classic: ARG values that change will invalidate every layer after them. Even if the ARG is only used in a RUN echo statement, Docker treats it as cache-busting input. The workaround is to put volatile ARGs as late as possible, or convert them to ENV after the build. But wait—ENV declarations also bust the cache when they change. The real distinction: ARG is build-time only, ENV persists into runtime. That means an ENV change invalidates every layer after it, while an ARG change invalidates layers that use it. Subtle, but it decides whether you wait ten seconds or ten minutes.

I have seen teams bake API keys into ARGs at the top of the Dockerfile, then wonder why every commit triggers a full rebuild. The dependency tree gets poisoned early. Move the ARG down to the exact RUN step that needs it. Worse case: you might need a second ARG declaration later. That's fine. Repetition beats invalidation.

.dockerignore mistakes that silently break caching

A missing .dockerignore rule can ruin caching without any visible error. Docker sends the entire build context to the daemon—including .git, node_modules, and all those random .env.local files. The context hash changes whenever any file in that set changes, so Docker can't reuse cached layers from previous builds. You reorder layers perfectly, and nothing helps. Because the root cause isn't layer order—it's context churn.

Check your build context size. We ran docker build --no-cache once and noticed the upload hanging. Turned out our .git folder was pushing the context to 300MB. Adding five lines to .dockerignore dropped it to 4MB, and cache hits went from rare to routine. The pitfall: .dockerignore patterns are relative to the context root, not the Dockerfile. So **/node_modules works, but a bare node_modules only matches the top level.

Layer order is a lever, not a lock. When reordering fails, inspect the context and the stage boundaries.

— field note from a production incident that cost us an afternoon

What usually breaks first is the .dockerignore file. Teams add it once, then forget it as the project grows. New directories appear—coverage/, cypress/, dist/—and suddenly the context includes megabytes of generated junk. The cache still works for pure-Dockerfile changes, but any code edit alters the context hash. Reordering can't fix that. You need to prune the context itself. Run docker build --check with BuildKit to see context size, or just tar the directory and inspect it. Painful, but faster than watching every build rewrite the world.

Where layer order hits its limits

When the cache is useless: huge contexts and moving tags

Reorder the COPY and RUN lines all you want — if the build context is a 2 GB monorepo with node_modules, vendor directories, and .git history, the cache hit rate collapses. Docker still has to checksum every file in that context before it can even decide which layer is stale. So the docker build command itself eats minutes before your first instruction runs. No layer ordering fixes that; you need a .dockerignore that actually earns its keep, or you need to split the build into smaller, separately-cached jobs.

Same story with latest tags. If your base image is python:3.12-slim without a digest pin, Docker pulls that tag fresh whenever the remote changes. The layers may be identical to what you had yesterday, but Docker's cache invalidation logic doesn't trust the tag — it treats the pull as a new base and rebuilds everything downstream. That hurts. We fixed one CI pipeline by pinning to the full SHA digest, and the rebuild-because-of-random-upstream behavior vanished overnight.

The trade-off: build speed vs Dockerfile readability

There's a tension between caching and clarity. You can micro-split COPY instructions so each one invalidates only a tiny slice — but then your Dockerfile becomes a 40-line ritual where every file lands in its own layer. I have seen this go wrong: engineers spend more time shuffling COPY lines than writing application code. The pragmatic middle ground is to group files by how often they change. Dependencies first, source second, generated artifacts last. That gives you 80% of the caching benefit with maybe five extra lines.

But don't expect perfection. Some builds simply can't be cached well — think of a RUN step that compiles native extensions for multiple architectures or downloads assets from an unpinned CDN. Reordering won't save you there; you'll need build arguments, separate images per stage, or something like BuildKit's --mount=type=cache for the genuinely slow parts.

A practical checklist for your next audit

Here's what I actually check when someone complains about immutable rebuilds:

• Is the context trimmed? Run du -sh . and compare to a .dockerignore list. If you ship .git, you're paying for history you never use.
• Are base image tags pinned to SHA digests, not latest or minor-moving tags?
• Do COPY blocks mirror the real change frequency of your files?
• Is there a RUN step that depends on the network without a lockfile or checksum?

That last one kills many teams. If your Dockerfile does RUN apt-get update without pinning versions, the output changes every time the registry updates. One day the build works, the next it pulls a patched library with a different hash, and Docker treats the layer as brand new. The whole cache below that instruction is still fine — but everything above it gets re-executed.

One honest warning: reordering adds cognitive overhead. A Dockerfile that prioritizes caching over readability forces the next person to reverse-engineer why the order looks the way it does. Add a comment or two. Explain that package-lock.json precedes src/ because the lockfile changes rarely. Otherwise your optimization becomes a maintenance hazard, and someone will "fix" the order back to logical flow — losing all the speed in one commit.

A rhetorical question worth asking: how many of your rebuilds actually stem from your own code changes versus upstream drift? If it's mostly upstream, spend less time on layer order and more on isolating base images into a separate, pinned build pipeline. That's the ceiling of this technique, and knowing it saves you from chasing the wrong fix for a week.