The Off-Key Bard
Technology

Why Do Thousands of Tiny Files Take Longer to Copy Than One Huge File?

Or: apparently 10 GB isn't always 10 GB. When you're copying files, the computer isn't only moving bytes — and the paperwork can take longer than the furniture.

Or: Apparently 10 GB Isn't Always 10 GB

You need to copy 10 GB of data. The first time, it's one large video file — the transfer settles into a rhythm and finishes quickly. The second time, it's a folder containing 80,000 tiny files. Suddenly the speed jumps all over the place, the estimated time remaining seems to have been calculated by someone throwing darts at a clock, and Windows spends what feels like forever actually finishing.

Same amount of data. Completely different experience.

That's because when you copy files, the computer isn't only moving bytes. It's managing files — and one 10 GB file creates considerably less administrative work than 80,000 small ones.

Every File Is More Than Its Contents

A file isn't simply raw data. The file system also stores metadata for each one: the filename, size, timestamps, permissions, and the information the system needs to locate the file's contents on the storage device. When you copy a file, the computer reads the source metadata, creates a new entry at the destination, allocates storage for it, copies the data, updates metadata at both ends, and closes the file properly.

For one large file, that overhead happens once. For 80,000 small files, it happens 80,000 times.

Think of it like a shipping operation. Moving one pallet with a shipping label, a scan at origin, a scan at destination, and one delivery is very different from moving 10,000 individual packages — each one needs its own label, its own tracking, its own handling. The total cargo weight hasn't changed. Everything surrounding the cargo has exploded.

Opening and Closing Files Takes Time

Before the operating system can read or write a file, it has to establish access to it. When it's finished, it closes that access. For a large file, the sequence is roughly: open → copy a lot of data → close. For thousands of files, it's open → copy tiny amount → close, repeated tens of thousands of times.

Each individual operation takes very little time. But tiny amounts multiplied by 100,000 stop being tiny. Computers are extremely fast — they're not infinitely fast.

Sequential vs. Random: The Storage Performance Difference

Modern SSDs can transfer data at remarkable speeds under the right conditions. Reading a large video file stored in a contiguous sequence lets the storage device settle into a sustained stream — like driving down a clear highway at 70 mph. That's an ideal workload for a storage device, and it's where those impressive advertised speeds come from.

Thousands of tiny files create a completely different pattern. The device has to access a new location, handle metadata, start a new operation, access another location, handle more metadata — over and over. It's more like driving through a city: capable of the same speed, but the conditions never allow it.

This is measured by two different storage metrics. Sequential throughput (MB/s) describes how much data can move in a sustained stream. IOPS — Input/Output Operations Per Second — describes how many individual operations the drive can handle. Moving one large file cares mostly about throughput. Moving thousands of small files makes latency and IOPS the relevant numbers. A drive can excel at one and still struggle with the other.

SSDs dramatically outperform traditional hard drives at random access because they have no moving parts. A mechanical hard drive's read/write heads physically move to different locations on spinning platters for each operation. At milliseconds per seek, and thousands of seeks required, the time adds up fast. An SSD can access any location nearly instantly, which is one reason replacing a hard drive with an SSD makes an entire computer feel faster — starting Windows, opening applications, and loading games all involve enormous numbers of small file operations, not just large sequential transfers.

Your Antivirus Is Watching

Security software may examine files as they're opened, created, or modified. Copy one large video and it has one object to inspect. Copy 50,000 files and it may have 50,000 opportunities to pause and ask "what's this?" That's useful behavior — malware frequently arrives as files — but security checks aren't free. They require processing time, and they happen per file, not per byte.

Some file types trigger more inspection than others. A folder full of executables, scripts, or documents with active content may be scrutinized more heavily than a folder of images. Two folders with identical total sizes can copy at different speeds based entirely on what's inside.

Why the Speed Graph and Time Estimate Bounce Around

The speed display during a file transfer shows the result of all these moving parts simultaneously — cache filling, metadata writing, security scanning, file creation, data transfer — and those phases don't stay consistent. Speed spikes when large data blocks are flowing; it drops when the system is doing administrative work between files.

The time estimate faces an even harder problem. Suppose the first part of the folder contains several large files. They copy quickly. Windows estimates two minutes remaining. Then it reaches 47,000 tiny files and the workload changes completely: seventeen minutes remaining. Then more large files: six minutes remaining. Windows isn't bad at arithmetic — it's recalculating based on recent performance, and if the workload keeps changing, so does the estimate. Anyone who has watched the estimate go from five minutes to two hours and back again has experienced this.

Allocation Units: Why Size and Size on Disk Don't Match

There's a related wrinkle worth knowing. Storage devices and file systems allocate space in fixed-size units rather than tracking every individual byte. If the file system uses 4 KB allocation units, a file containing 1 byte still occupies an entire 4 KB block. A directory containing enormous numbers of tiny files can therefore consume significantly more disk space than the sum of their file sizes suggests.

This is why Windows shows both "Size" and "Size on disk" for folders — and why those numbers aren't always close. The gap reflects storage that belongs to small files but can't be used for anything else.

The ZIP Trick

One practical consequence of all this: packaging thousands of small files into a single archive before transferring them can sometimes be significantly faster than copying them individually. The storage or network system sees one file operation instead of tens of thousands, even if the total data is identical.

This is one reason software downloads and game installations often arrive as one large package rather than thousands of loose files. It's also why game developers store assets in large container formats — tens of thousands of individual textures and audio files packaged into a few large archives, which the game accesses efficiently internally, rather than sitting on disk as separate files that would require separate operations to open.

The Bard's Take

Give a modern SSD one giant file and tell it to move from here to there, and it settles into a beautiful sustained stream — gigabytes per second, just like the spec sheet promised. Give it 100,000 tiny files containing the same total data and you've changed the job entirely. Now it has to open files, create files, allocate space, update metadata, perform security checks, close files, and repeat the process tens of thousands of times.

The amount of cargo didn't change. The number of deliveries did.

That's why a 10 GB movie can copy faster than a 2 GB folder full of tiny files. It's why your SSD doesn't transfer anywhere near its rated speed during certain workloads. And it's why "time remaining" occasionally appears to have abandoned mathematics.

Storage performance isn't just about how quickly a device moves bytes. Sometimes the real question is how much work the computer has to do between those bytes — and when you've got 100,000 tiny files, the paperwork can take longer than moving the furniture.

Sources

← Back to the Desk — Technology