HD Video – A Primer

This weekend I spent an exorbitant amount of time trying to deal with HD video in the same way I manage my digitized DVD collection. By that I mean, attempting to figure out how to get ripped HD movies into a format that I can play over my network, with the XBox 360, the Mac in our room and a PC as the target players. Things have not gone well, and while some people may have worked out a system they can live with, I’ve not found a solution that can be applied efficiently enough to make it worth the effort. Nonetheless, I’ll document what I’ve learned, in case its useful to anyone. There is no new information here — all of it is on the web somewhere — I’m just hoping to pull it into a coherent form…

MPEG4

For starters, high def content is invariably distributed in MPEG4-10 format. The different ways it can be bundled into a file will be discussed later on, but its important to first understand that not all MPEG4 is created equal.
MPEG4-2, or Part 2, is very common, and not really suitable for HD content. DivX and XVid are both implementations of MPEG4-2. DivX is MPEG4-2 with a couple extra features added. XVid is an independent re-implementation of DivX that, while usually compatible, is definitely a different beast. A player with DivX stamped on it can probably play any MPEG4-2 content, and probably play any XVid content. In cannot, however play MPEG4-10 content.
MPEG4-10, or Part 10, is relatively new and most commonly known as H.264. Its based on the same technology as its older cousin, but includes improvements to color handling, and allows higher quality content at lower bit rates.
Bit Rate: Loosely defined, the Bit Rate is the number of bits of data that go into showing 1 second of video. The higher the bit rate, the more information making up the image you see on the screen. Higher bit rates = better quality video.
H.264 can do better picture, with less artifacting, and less “chunkiness” (if you don’t know what I mean by chunkiness, watch a dark scene in any DivX movie) at a lower bit rate. A 2GB MPEG4-10 video will look better than a 4GB MPEG4-2 video.

Containers

All video, regardless of the codec (be it MPEG2, MPEG4-2, H.264, etc…) is typically distributed inside a container, bundling up the video track with an audio track, and possibly other features such as alternate audio tracks (containing a different language) or subtitles. Wikipedia has a good entry comparing the different container formats, but here are the common ones you’ll see for HD Video:
Quicktime (.MOV)
Quicktime is an incredibly flexible container that can hold virtually any collection of files, by using plug-in codec support. A .MOV container can hold an H.264 video track plus a DTS 5.1 surround sound audio track without any problem.
Limitations: There’s not many devices with full Quicktime support. Basically you’re stuck with something from Apple. The XBox does have basic support for .MOV containers, but doesn’t have the ability to plug-in codecs to extend its limited offering.
Windows Media File (.WMV)
Windows Media is actually very fully featured, with support for containing high bit-rate HD video and audio tracks with 5.1 surround sound.
Limitations: Its a fantastic format, despite being from Microsoft, but there aren’t many devices that support it, and I frankly hate Windows Media Player. You’ll rarely find any content distributed in a WMV container.
MP4 (.MP4)
By far the most popular, and infuriatingly limited container for H.264 (MPEG4-10). Do not confuse “MPEG4” for “MP4” — one is the video track, one is the container it travels in. MP4 supports H.264 video at all bit rates (but not all bit rates can be played on all devices, read about levels here) and can hold an AAC-LC audio track (probably more than one.) But herein lies the biggest problem with MP4: AAC-LC means “Low Complexity” or in other words “no digital surround sound.” You can encode Dolby Pro Logic II into the stereo track and thus get surround sound that way, but it won’t be DTS and it’ll basically suck.
Matroska (.MKV)
The Matroska container is the most common one you’ll see — both from ripped optical media, such as Bluray or HD-DVD discs, and online. Its very flexible, and can hold super high bit-rate H.264 video tracks as well as multiple audio tracks, including DTS (Digital Theater Surround).
Limitations: Apparently its a poorly defined standard and there are lots of different interpretations on how to implement it. No OS or media player software supports Matroska out-of-the-box. Few set-top devices can support Matroska. The XBox 360 does not.

Useage

None of these containers formats are really mature enough to be considered ready for consumer use. Different devices can handle different containers — but each with restrictions on what’s inside the container. Here’s a list of what the XBox 360 can handle.
The best standardized container, MP4, is missing a pretty critical piece of the home theater puzzle: digital surround sound. This is not a new feature here, folks. It needs to be supported for MP4 to be useful. The other containers, which do support surround sound, are either too proprietary or too disorganized to be useful yet.
If you were to obtain a MKV file with HD video inside (H.264) and a surround sound audio track, and you wanted to play it on your XBox, you’d basically have two choices on how to proceed:
Extraction and Re-bundling
Using an array of poorly-made tools and hacks available online, which I won’t cover here out of disdain for them and the hours of time I wasted on them, you can extract both the H264 video stream and the AC3 (digital surround sound) audio track into separate files. Then you can modify them to be playable on your target device, and re-contain them in a supported container.
For example, you could drop the bit rate on the H264 stream down to at least level 4.1, outputting it to a new, slightly more compressed, but certainly still “Hi-Def” file. Then you could pare the audio track down to Stereo, with Dolby PL2 intact. Finally you could combine the two files together into an .MP4 container and play them on your XBox 360, PS3, etc…
A friend claims to have this process down to a few clicks done in less than 10 minutes (he says 6 minutes or less, but I’ll chalk that up to bragging.) Using a Mac with Windows running in a VM, then later, after much frustration and waiting, using a well-equipped physical Windows PC after sneaker-netting the files, I was unable to complete all the steps of the process after 5 hours. The “software” is buggy, has poorly documented dependencies and 0 troubleshooting information if anything goes wrong.
My friend calls this operator error and says I’m technologically inept. I call it really bad software being used to do something it wasn’t really intended for. I finally gave up.
Down-sampling
The less attractive, but much easier option, is to simply down-sample the whole contained file into a MPEG4-2/DivX file. If you have a Matroska file, you might still want to do some conversion first — Quicktime Pro can ‘Save As’ a MKV file into a MOV in a couple seconds.
From there, many mature applications, like Handbrake or FFMPEGx on the Mac, will be able to convert the whole thing into something playable for you. It may take some time, and you’ll want to compress at a high bit-rate to keep as much of the video quality as possible (and don’t fool yourself — you will loose quality), but it’ll be a usable, decent-looking file that can play on a wide-range of devices, AND it can contain DTS 5.1 audio.

Conclusions

HD Video is the biggest screw-up in consumer technology history. We’ve been waiting for this for years… Waiting while they sort out 720p, 1080i, or 1080p as the resolution we’re supposed to buy. Waiting while the cable companies struggle to get a trickle of 720p content to our sets (then screw their customers around trying to save money on it.) Waiting while big companies publically fight over which optical format we’re supposed to go out and re-buy all our movies on…
In fact, “HD” video (eg: 1080p) describes only the resolution. You can have a 268kbps bit-rate video in 1920×1080 and call it “HD” if you want. It’ll look like crap, but it’ll be big!
Learning about the different container formats and how they’re still a fractured mess of incompatible technologies only makes me angrier. Here I stand, ready to jump on the Hi-Def band wagon, with my brand-new TV, my brand-new XBox 360, cash in my hand… and the only reasonable and affordable way to get HD content is the now-defunct $50 HD-DVD add-on player I bought on clearance.
Bluray is ridiculously expensive. No one offers 1080p downloads or subscription content. Even the pirate world can’t figure out how to distribute content playable on any common device…
Its a disaster. I spent the better part of a day trying to make it useful, and I’ve decided that, for now, I’ll be storing my content in high bit-rate DivX with my surround sound intact, thank you very much. I’ll re-visit this again sometime in 2009, and hope they’ve got their crap together…
Anyone have any corrections to this, or suggestions to make this space more understandable and usable?

• Update: Gizmodo has a great article with more details on this topic.