The Off-Key Bard
Technology

What Is a Codec, and Why Won't This Video Play?

Or: two MP4 files aren't necessarily the same thing. The file extension is just the box — the codec is what's packed inside, and your device has to speak both languages.

Or: Two MP4 Files Aren't Necessarily the Same Thing

You have two video files. Both end in .mp4. You double-click the first — it plays perfectly. You open the second: "Unsupported video format." They're both MP4 files. Shouldn't that mean they're the same kind of video?

Not necessarily. Because .mp4 doesn't actually tell you how the video was compressed. It describes the container. What's inside that container is encoded by something else entirely: the codec. And codecs are one of the biggest reasons a video can play perfectly on one device while another device looks at the exact same file and has no idea what to do with it.

The Problem Codecs Are Solving

Digital video is enormous before compression gets involved. A single frame of 1080p video contains over two million pixels. At 30 frames per second, that's more than 62 million pixel values every second. At 4K and 60 frames per second, you're approaching half a billion. Storing all of that without compression produces files that are completely impractical to work with. So we compress them — aggressively — and that's the codec's job.

Codec is short for coder/decoder (or compressor/decompressor). When you record or export a video, a codec encodes it — compresses the raw data into a much smaller representation. When you play it back, the codec decodes it — reconstructs something displayable from the compressed data. The clever part is doing that without making the video look terrible.

How Video Compression Actually Works

Video contains a lot of repetition. Imagine two people talking in a room where the camera isn't moving. The wall, the couch, the table — most of the frame barely changes between one second and the next. Storing a completely independent full-resolution image 30 times per second would be extraordinarily wasteful.

Instead, modern video codecs exploit the similarity between frames. Rather than storing a complete picture every time, they can store a reference frame and then describe only what changed. Think of it as giving someone instructions: instead of handing them an entirely new photograph, you say "keep everything the same, but move his hand slightly to the left." The instruction is far smaller than the full image.

This is why scrubbing through a video sometimes causes a brief delay — the player has to locate an appropriate reference frame, then decode forward from there to reconstruct exactly where you jumped to. Modern devices do this nearly instantaneously, but the process is happening regardless.

The Container vs. the Codec

Here's the distinction at the heart of most video compatibility confusion. MP4 is a container — think of it as a standardized box. Inside that box you might find video, audio, subtitles, chapter markers, and metadata. The container format defines how those pieces are packaged together and identified.

What the container doesn't specify is exactly how the video inside was encoded. Two MP4 files can look identical from the outside while containing video compressed by completely different codecs. Same box, completely different contents — and your device needs to understand the contents, not just the box.

This is why "but they're both MP4" doesn't resolve a compatibility problem. The file extension tells you what type of container you have. It doesn't tell you what language the video inside is speaking.

H.264: The Universal Language

For most of the past decade and a half, the dominant video codec has been H.264, also known as AVC (Advanced Video Coding). It became the standard for streaming, web video, phones, cameras, Blu-ray, and video files of all kinds. Its single biggest advantage today isn't technical sophistication — it's that virtually everything understands it. Old televisions, budget phones, ancient laptops, game consoles, streaming boxes. If broad compatibility matters, H.264 remains the safe choice.

H.265: Better Compression, Narrower Support

H.265, also called HEVC (High Efficiency Video Coding), was designed to be roughly twice as efficient as H.264 — the same visual quality at about half the file size, or better quality at the same size. For 4K video in particular, that efficiency matters enormously: smaller files, lower streaming bandwidth, more practical high-resolution content.

The catch is that more sophisticated compression requires more sophisticated processing to decode, and older devices either can't manage it in software or lack the dedicated hardware to handle it efficiently. An older television or streaming box may understand H.264 perfectly while having no idea what to do with H.265. Same container, different codec inside, completely different result.

AV1: The Open Alternative

AV1 is the codec increasingly running your streaming video right now, even if you've never heard of it. Developed by the Alliance for Open Media — a coalition that includes Google, Apple, Microsoft, Netflix, Amazon, and others — it offers roughly 30% better compression efficiency than H.265 and carries a royalty-free license, which makes it attractive to streaming services that distribute enormous volumes of video. YouTube, Netflix, and Twitch are among the services already using it.

AV1 has the same adoption challenge every new codec faces: devices need support before content can use it broadly. Modern hardware increasingly includes dedicated AV1 decoding circuitry. Older devices either can't decode it at all, or have to use the CPU to do it in software — which works, but consumes far more power and processing resources than dedicated hardware would.

Why a Powerful Old Computer Can Struggle With a New Codec

This surprises people. An older high-end machine suddenly stutters and runs hot playing a video that a cheap new laptop handles without effort. The explanation is usually hardware decoding. Modern processors and GPUs contain dedicated circuitry for decoding specific video formats — specialized silicon that handles the job far more efficiently than general-purpose computation. The older machine, regardless of its raw speed, is brute-forcing the decoding in software. The newer machine offloads it to hardware built specifically for the task. Specialized beats fast, at least for this.

This matters most on devices where power consumption is critical — phones, laptops, and streaming boxes where decoding a video shouldn't drain the battery noticeably.

Audio Has Codecs Too

Everything above applies equally to audio. MP3, AAC, FLAC, Opus — these are all audio codecs, each making different tradeoffs between file size, quality, compatibility, and computational requirements. A single MP4 file commonly contains both a video codec and an audio codec. Your device needs to understand both. A television might handle the video codec perfectly and fail on the audio codec, giving you picture with no sound. The container opened fine; one of the things inside didn't.

Transcoding: The Compatibility Workaround

If you run a home media server, you've likely encountered this already. Your server has a video in H.265. The television in the bedroom only supports H.264. Rather than keeping separate versions, the server can transcode on the fly — decode the H.265 video and re-encode it as H.264 in real time before sending it to the television. This is why media servers can suddenly hammer the CPU: transcoding is computationally expensive, especially at higher resolutions.

When the device supports the original file directly — "direct play," in media server terminology — the server does almost nothing. When it doesn't, the server may end up doing substantial work for every device that can't handle the original format. It's one reason people building home media setups think carefully about codec support on their playback devices.

Transcoding also has a quality cost worth knowing about. Converting between lossy codecs means decompressing video that already discarded some information, then compressing it again. Each generation of lossy encoding can introduce additional artifacts. For casual streaming this usually isn't noticeable, but for archival purposes it's a reason to preserve originals.

Why the Same Resolution Can Look Completely Different

Two videos can both say 1080p and look dramatically different. Resolution describes the dimensions of the image — how many pixels. It says nothing about how much compression was applied, what bitrate was allocated, or which codec did the encoding. A heavily compressed 1080p video encoded with an older codec can look noticeably worse than a well-encoded 1080p video from a newer, more efficient one. Resolution is one dimension of quality, not a summary of it.

The Bard's Take

A video file is not just a video file, even when the extension is the same. MP4 is the box. The codec is how the contents were packed — the language the video is written in. Your device has to understand both.

That's why two MP4 files can behave completely differently. One might contain H.264 your ten-year-old television reads without a second thought. The other might contain a newer codec that leaves the same television utterly confused. Newer codecs squeeze increasingly impressive quality into increasingly manageable file sizes, but efficiency comes with a cost: something has to know how to decode it, and not everything does yet.

So when a device tells you "unsupported video format" on a file you expected to work — don't look at the extension. The box may be perfectly familiar. It's what's packed inside that your device doesn't speak.

Sources

← Back to the Desk — Technology