After Effects

Thoughts on keying part 1- Technology

For a long time I’ve subscribed to the Media Motion After Effects email list, an invaluable resource for all things After Effects as well as being a friendly online community.  The other day I found an email I posted there about 2 years ago, in which I listed some of the changes I’ve seen in keying throughout my working career.  I’d written the email to kill time while I was rendering, and I’d forgotten I wrote it until the other day.  It’s always weird to read things you can’t remember writing, but it did make me think about how technology changes some things and not others.  So here I’ve taken my original vague ramblings from the email and fleshed them out a bit… this is part 1, the things that have changed.

It’s often said that you learn from your mistakes, but when it comes to keying you’re often learning from other people’s mistakes.  If you’re given “perfect” footage and all you have to do is click the eyedropper in Keylight to get a perfect result, then you haven’t learnt much.  It’s when the footage is poor, the lighting is wrong, and everything falls apart and you’re expected to rescue the shot that you start to learn how it’s all working “under the hood”.

When I started my professional career in 1997 everything we shot was on Betacam SP.  The company I worked for owned a mid-level 3-chip camera with the cheapest Fuji lens available, and I was working with a Media 100 editing system.  Beta-SP is an analogue format that has a lot of noise, the occasional dropout, and the Media 100 system was doing the analogue to digital conversion to an 8-bit lossy compression format.  But this wasn’t too bad – our setup was pretty typical for corporate work produced in Melbourne at that time, and occasionally the projects I worked on had bigger budgets than usual- which meant digital betacam or even film, or smaller than usual- which meant DV.  So typically there were many factors that contributed to the quality of the footage I’d be working with, and they can be summarised here:

– Footage shot on an industrial camera with a low-quality lens

– Recorded onto Beta-SP, a noisy analogue format

– Everything is interlaced, standard definition

– Digitising via analogue component signal

– Media 100 used a lossy 8-bit compression system

– All video software was 8-bit

– After Effects only had basic keyers unless you purchased the “Production Bundle”, which gave you the Difference Keyer and the Linear Colour Keyer.  But no Keylight…

– If you wanted to purchase a 3rd party keyer then the only option was Ultimatte, which was a few thousand dollars…

I’ve mentioned the camera lens because I’ll always remember when a good friend of mine was acting as our DOP on a project, and he turned up with a beautiful Canon lens that was worth more than the rest of our camera kit combined.  With all other factors being equal, the lens made an astonishing difference to the quality of the footage- it was a shame we only had it for a day.

Looking back I’m surprised I was able to key anything at all, but keying became a routine part of my job and one which was made a lot easier when I won a copy of “Composite Wizard“.  Not only did this set of plugins save my footage on many occasions, but the manual was hugely informative and I learned a lot from reading it cover-to-cover.  So keying in After Effects was something I approached with confidence, but it was always an involved multi-stage process:

– Digitising interlaced analogue 8-bit footage using Media 100

– Use the Composite Wizard “denoise” filter to smooth out the video

– Use the Composite Wizard “smooth screen” filter to even out the background lighting

– Boost the screen colour saturation using the AE Hue/Saturation filter

– Key with the AE Linear Key filter set to “chroma”

– Clean the matte with the Composite Wizard “Miracle Alpha Cleaner”

– Adjust the edges with either the Composite Wizard “Matte Feather” or the AE “Matte Choker” filters

– Use the AE Spill Supressor to clean up colour spill

Because these steps were slow, and computers back then were much slower, I would render the foreground footage out with an alpha and do the compositing as a separate step.  And that’s how I worked for over 5 years.

Looking at these steps it’s interesting to note how much “fudging” is going on.  Because the footage I’m beginning with isn’t perfect almost every stage of the process is dealing the problems that begin there – too much noise, uneven lighting, low-resolution formats.  The denoise filter is a fudge, the smooth screen filter is a fudge, boosting the saturation of the screen is a fudge, using the miracle alpha cleaner is a fudge, but all neccessary processes to get an acceptable result – and all standard practises for the technology of the time.  I haven’t had to key interlaced standard definition footage for many years now, and I hope I’ll never have to again.

So what’s different now?

Technology has advanced in all areas relating to all stages above, making it easier to produce higher quality results in less time.

The single biggest change is the availability of higher quality acquisition formats that capture progressive images.  Even if you have to work with Digital Betacam a progressive camera can make a huge difference.  But HD is much better- so many more pixels to work with- and RED is better still.  The easiest footage to key, and the highest quality results I’ve obtained have come from RED footage.  Even if your final delivery format is standard definition, shooting on HD or RED will make keying much much easier.

The post production chain is now entirely digital, which removes analogue conversions from the process along with the video noise inherent with older analoge formats like Beta-SP.  If you’re working with RED or even DSLRs then there isn’t even tape to worry about.

Software now works with higher bit-depths… editing systems can work with 10-bit uncompressed footage and After Effects now works in 16 or 32 bit modes.    Even when working with standard definition digital betacam footage you can obtain better results by working with uncompressed 10-bit footage than you can with 8-bit, with After Effects in 16-bit rather than 8-bit mode.

Keying software has become more advanced, and Keylight is wonderful when dealing with well lit blue or green screens.  It’s a big step up from the After Effects linear colour keyer.  Keylight is great, but it’s only one improvement in the chain.  It would still struggle with the noisy, interlaced, compressed analogue Beta-SP footage from my 1997 days.

I often still use the bundled Colour Finesse plugin to tweak the colour of the screen before I apply Keylight, but more often to adjust the hue than to boost the saturation.  Boosting the saturation is pretty much the same as the “screen gain” control in Keylight, so I may as well do it there if neccessary.  But I find that the blue and green in the video footage can usually be hue-shifted towards a more pure “digital blue” or “digital green” and this can improve the result.

Likewise, in some cases I will obtain significantly better results by de-noising the footage first, but these days the After Effects “remove grain” plugin is much cleverer and gentler than the Composite Wizard equivalent.  However, the AE remove grain plugin is really slow, so even though we have faster computers than we did 10 years ago, we’re working with higher resolution files and more intensive algorithms.  So degraining 3K RED footage is still an overnight render…

For me the single biggest improvement over the past 13 years of my career has been the move from interlaced to progressive acquisition formats.  The post-production chain is now entirely digital, which eliminates conversion artefacts from the process as well.  The software side of things has improved fidelity by offering increased bit-depths and more pixels to work with, and the underlying algorithms that power the plugins we use have increased in their sophistication and speed.  But I don’t think these improvements are as significant as getting rid of fields!

To get a good key 13 years ago required a combination of fudges to overcome the limitations of the technology at the time- beginning with the interlaced nature of the acquisition format and the analogue conversion required just to get it into the computer.  Today we have better tools and we can produce better results more easily, and the processes we go through are no longer fudges designed to overcome technical shortcomings, but tools that improve and enhance the final result.

But while technology has changed and made some things easier, some things have stayed the same… and that’s part 2!