his last request

Post-production always seems to be much harder than it should be. Maybe it's just my ignorance, but at every stage of moving digital files from one platform to another I seem to encounter all kinds of incompatibilities and problems.

One problem from the original mastering of His Last Request was the way the codec used by AVID Xpress, AVID Meridien Uncompressed, treats black levels. Pablo Baudet's black and white "Film Noir"-style shooting looked great on the AVID editing system, with rich, deep blacks, but on the final DVD it looked washed out and grey.

Here's the difference:

How it looks on the AVID

How it looks on the DVD

Here's the sequence we originally used to create the DVD, all on PCs:
1) Edit on AVID XPress
2) Export as a Quicktime Reference Movie
3) Import the Quicktime Reference movie into Adobe Encore DVD and master the DVD

Recently we've remastered using a Mac, and the same problem turned up again. Here's the sequence:

1) Edit on AVID XPress
2) Export as a Quicktime Reference Movie
3) Import the Quicktime Reference Movie into iDVD '08 and master the DVD

I eventually found solutions to the problem, once on the PC, and a better, long-term solution on the Mac.

The Problem

There are two main standards for dealing with colours that concern us here. Computers tend to use 8-bit RGB, where each colour is expressed as a set of three numbers from 0 to 255. (0,0,0) is black and (255,255,255) is white. Video has a different 8-bit standard, variously known as ITU 601 or Y'CbCr. Here, black is represented as (16,16,16) and white as (235,235,235). The extra space at the bottom and top of the range appears to be to allow equipment room to capture sub-blacks and super-whites that won't be visibly different. Who knows why? The bottom line is that the standard is different.

Unbeknownst to us, when we exported AVID Meridien Uncompressed, the colourspace used was 601/Y'CbCr. So the blackest black on "His Last Request" was (16,16,16) and the whitest white was (235,235,235). Quicktime, for whatever reason, seems to think that the AVID's colourspace is actually RGB and so interprets (16,16,16) as medium grey and (235,235,235) as light grey. This interpretation is passed on to the mastering program (Adobe Encore or iDVD) and the result is a DVD with a washed-out look.

That's the problem, and it took quite a while to figure out. If it hadn't been a black and white film I might never have spotted what was going on. The give away was the title cards. It was clear that the black on the cards should have been (0,0,0) as they were prepared in Photoshop, but they consistently came out as (16,16,16). On a standard colour movie I might never have figured it out.

Solution (1)

For the original master, after a lot of trial and error, we came up with the following solution:

1) Encode the exported Quicktime Reference Movie using TMPGEnc from Pegasys. This includes an option that forces the encoder to interpret the incoming data as ITU 601, i.e. Y'CbCr.
2) Master using Adobe Encore, telling it to use already encoded MPEG-2 files.

This was a while ago (2005) and the versions of both Encore and TMPGEnc will have changed, but I'm pretty sure this will still work.

Solution (2)

iDVD on the Mac is not so simple as it doesn't allow you to use externally encoded MPEG-2 files. It insists on doing the encoding itself.

After a great deal of trial and error, I stumbled across Bitjazz's Sheer Video Codecs. These are uncompressed codecs intended for archival purposes. They convert reversibly between colourspaces, always preserving the original pixel information. In particular, they allow you convert between RGB 8-bit and ITU 601/Y'CbCr colourspaces. As it happened, they also suffered from the problem of misinterpreting what AVID Meridien Uncompressed was feeding them, but the founder of the company, Andreas Wittenstein, spent some weekends tracking down the problem and producing me a new build of the product. As Andreas explains it,

When I export your sample to Sheer Y'CbCr[A] 8bv 4:2:2:[4] with QuickTime Player, QuickTime Player feeds the images to the Sheer Y'CbCr[A] 8bv 4:2:2[:4] encoder in the RGB 8bf 'ARGB' pixel format. I find this odd, since the Avid Meridien Uncompressed (AVUI) codec purportedly stores the data in Y'CbCr[A] 8bv 4:2:2[:4] format. On inspecting the AvidAVUICodec, I see that ARGB is the only pixel format that the AVUI codec supports for input and output; it lacks the 'cpix' resource used to list other supported pixel formats. 

For the last few frames in the movie (Silicon Artists presenta El Último Deseo), the "black" ARGB pixels passed to the Sheer Y'CbCr[A] 8bv 4:2:2[:4] encoder have the value: 

ARGB = {0xFF,0x10,0x10,0x10} (hexadecimal)
ARGB = {255,16,16,16} (decimal)

When encoding to the Avid Meridien Uncompressed (AVUI) codec the settings dialog ('Avid Meridien Uncompressed Codec Configuration - v1.8.0') lets you specify the input color range as either ITU-R 601 video-range [16..235] or full-range [0..255]. However, QuickTime does not provide any mechanism to specify settings for a decoder. QuickTime unequivocally defines the ARGB pixel format as having full-range components, so the fact that AVID's AVUI decompressor outputs video-range ARGB is just wrong. 

Got that? In other words, let's all kill AVID.

The new version of Sheer 2.6.6.9 contains a workaround that detects the presence of AVID Meridien and compensates for it. Here's the new solution:

1) Open the Quicktime Reference Movie exported from AVID in Quicktime Pro
2) Export a self-contained Quicktime movie using the Sheer 8-bit RGB codec and selecting "video range" as the input setting. For "His Last Request", this created a 30Gb file.
3) Use this self-contained movie in iDVD.

Voila! The Sheer Encoded version has expanded (16,16,16)-(235,235,235) to the full RGB range (0,0,0)-(255,255,255). The Sheer player codecs are free on both PC and Mac, so you don't have to worry about the company going under - your files will always be playable. The encoders are cheap and well worth it.

Thanks to Andreas and Bitjazz for taking the time to help out and modify their product!

The new DVD of "His Last Request" (complete with rich blacks!) should be available on Amazon from May 2008.