Thoughts on keying part 2- The Myth of the Single Click
Many people I work with are surprised to discover that I have previously worked as a writer, producer and director (not at the same time) and it’s the producing part especially which relates to keying. And while technology has advanced in leaps and bounds over the course of my career, some other things haven’t changed at all- which brings us to part 2.
In part 1 I talked about the technology that was in common use when I began my career, and I said that many of the steps I used to obtain an acceptable key in After Effects were essentially “fudges”. I call them fudges because – at the time – I always felt as though I was compensating for mistakes. There’s a saying “the grass is always greener on the other side” but for me it was greenscreens, not grass. I always felt as though I was dealing with inferior footage and that the guys working for big-name post-houses on expensive TVCs would always have perfect footage that would key with one click of their SGI mouse.
I was certainly not alone in this thought. Most of my friends with similar corporate production jobs had the same “greenscreen is always greener at the other facilities” inferiority complexes. We all had a basic assumption that if the footage was shot properly and if our software was good enough then we would be able to key with one click. When our footage didn’t key well we would blame everything in turn. We would blame the DOP for not lighting the screen properly, or the screen for not being painted with the right paint. We would blame the camera for being too noisy; the lens for being too blurry; the recording formats for lacking fidelity; the capture hardware and codecs for their compression; the software for lacking precision; our accounting departments for not buying us the right plugins and so on… We were using 8-bit After Effects on desktop Macs and we all had Flame/Inferno envy (back then there was also Henry envy, but that sounds like a cabaret singer). Surely the lucky few who worked in the million-dollar Flame/Quantel rooms could key with one-click… why else would you have a computer the size of a fridge?
The fancy ads in CineFx for the Ultimatte keying software worked- convinced that this plug-in would solve all problems one of my friends had his company buy it despite the huge cost. It was better than the AE keyers but the improvements were not miraculous and it was very slow. Meanwhile a neighbouring company contracted a programmer to write their own custom keyer (cheaper than multiple Ultimatte licences) but I don’t think any of us worked on a project where the footage keyed perfectly with one click. So we’d drink another coffee and complain more about how the lighting was rubbish, and then I’d reach for another Composite Wizard plug-in, or mask out another shadow, or roto out some footprints, and basically pile on the fudge.
Looking back with the benefits of 13 years experience, I now realise that I was/ we were wrong. Since then I’ve produced a few projects with 6-figure budgets and if I dig back into my past and pull on my producer hat again, I can do some basic sums and expose the “myth of the single click”.
Professional film and video productions cost a lot of money. If you add up the costs of all the crew and equipment that are hired for the shoot, and then divide it by the number of hours you shoot for then you have a basic figure of how much your production costs every hour. Using this method I can look back at a mid-level TVC I produced and calculate that we spent around $8,000 for every hour in the studio. A high-end corporate video cost $3,000 per hour in a similar studio. These prices are just for crew and equipment for the shoot. If I wanted to exaggerate the point I could also include the fees of the actors and the director and maybe even pre-production costs. In this way I could easily claim that every hour in the studio was costing $10,000 or more.
If you’re wondering what this has to do with keying then imagine yourself on set, trying to tell the Producer that the DOP hasn’t lit the greenscreen very well and that he really should set up a monitor with vectorscope to get the best possible result, and that the grips/gaffer need to adjust the lighting arrangement so the greenscreen is evenly lit. This is a very good way to learn new swear words.
There are many great articles on the internet about keying, and how to light for keying and how to shoot for keying in order to get good results. I think I’ve read them all (I wrote one of them) and I’ve learnt a lot from them. They’re great when you have a lot of time, or when you’re working for a company that has its own studio and equipment. But when you’ve hired lots of expensive people and you’re burning $3000-$10,000 per hour and you know that overtime kicks in after 8 hours then suddenly your priorities change and you decide that there are better things to do than to spend time adjusting the lights. Sure the crew could help you out and adjust everything so the screen is perfectly lit, but it could end up costing thousands and thousands of dollars. Yes- if the footage doesn’t key perfectly then you will need to spend time rotoscoping it, but roto is cheaper than a studio full of equipment and crew. And the footage will probably need some roto anyway.
I spent several years in front of After Effects moaning about poor DOPs and producers who would decide to “fix it in post” before becoming one of them myself. Because ultimately- when the clock is ticking and money is flowing and everyone has less time on set than they need anyway- lighting the greenscreen perfectly isn’t that important. You can fix it in post. Roto is cheaper than having the gaffer pull more lights out of the truck and spend an hour setting them up.
Of course this argument is only valid if you have a producer who understands the maths, and allows for roto in the budget. This is where the failing usually is. It’s when there is no allowance – in time or in money – to fix difficult keys with roto that compositors get frustrated, and we all start moaning again.
In this respect, it’s the producers who expect footage to be keyed with a single click that are really the ones to blame- its their belief in the myth of the single click that can cause grief in post-production. I mentioned earlier that you can find many great articles on the internet to do with keying and shooting greenscreens, and it’s certainly easy to find technical tips and techniques to ensure that greenscreen footage is shot well. However there are two very valuable non-technical lessons to learn as well. Firstly, no matter how well the footage is lit there will almost certainly be some level of rotoscoping required, even if it’s just garbage mattes. And secondly- rotoscoping is not easy and is not something that can be palmed off to inexperienced and cheaper workers. The majority of producers I’ve worked with have assumed that rotoscoping is a “fix” that is only required when something goes wrong, and because they haven’t budgeted for it their first reaction is to try and have it done as cheaply as possible. But cheap roto is not always good roto.
So while technology has changed, budgets and basic accounting haven’t. If anything – as software has increased in sophistication and footage is better to begin with – roto will become easier and cheaper too, so overall standards will rise. Even when you have footage that does key well there will always be a rough spot here or a shadow there, that can only be fixed manually.
I recently updated my showreel and the new version opens with a project I worked on for Nokia, produced by Kemistry in London. This is an example of a project that was well produced and a pleasure to work on. Once the edit was locked-off and the footage was ready for compositing, the producer (Andrew Pearce) had myself and another compositor begin work on keying with 2 dedicated rotoscopers to help us out. The roto guys had not been called in after we found keying difficult- Andrew had simply recognised from the beginning that there would need to be roto work at some stage, and so they were included in the budget. In actual fact the footage (shot on RED) keyed very well but there were still clips that needed extensive roto and the guys would wait to be allocated clips by either myself or the other compositor as we needed their help. Additionally – Andrew had hired genuine roto specialists who knew what they were doing, and not cheap & eager youngsters who lacked experience. It all worked well.
In part 1 I listed the stages that I used to go through in order to key footage. This time I’ll list my thoughts as a producer on shooting footage for keying:
1) Shooting greenscreen footage perfectly is expensive
2) It is cheaper to shoot imperfect footage and fix it in post
3) All greenscreen footage will probably need some degree of rotoscoping
4) Rotoscoping is not easy, nor is it an entry-level skill
5) Keying will always take more than one-click of a mouse
My figures above ($3000 – $10,000 per hour in the studio) were based on my experiences on low to mid level projects. But it’s pretty easy to imagine that with typical Hollywood budgets- compounded by limited availability of main actors – the figures are massively higher. So the time spent setting up a greenscreen and adjusting the lighting for a feature film will be even more expensive, so there’s even less likelihood of it being done “perfectly”. Judging from DVD special features and behind-the-scenes specials that discuss visual-fx it sometimes looks as though the greenscreen is simply a visual aide for rotoscopers rather than something that can be feasibly keyed with any software.
Many years ago I read an article/blog called something like “the myth of the blue screen”. I’ve tried to find it again with google but I’ve failed. The gist of the article is that many big Hollywood visual-fx facilities don’t even try to use keying plugins, keying is generally done entirely by rotoscope artists instead. The author of the article had worked on the film “Forest Gump” and claimed that although the film is extensively referenced for the use of bluescreen/greenscreen to integrate Tom Hanks into archival footage and to digitally remove the legs of actor Gary Sinise, all of those effects were achieved by manual rotoscoping and no keying plugins were used at all. Basically- the footage is shot under such expensive and stressful circumstances that there is no way the key colours will ever be lit perfectly. So some roto will always be needed- and if some roto is needed then it’s simply easier to do the entire shot with roto. I can understand why. If anyone knows the article then please email me the link- it would be a great reference.
So by thinking about the costs involved in a film/video shoot, as well as the stressful nature of filming in general, it’s easy to see why footage never keys with a single click.
As for the greenscreen envy I used to have when I started- not only was I wrong about the myth of the single click, but there’s every chance that the guys with the Flames and Henrys were having to deal with worse footage than I was.
Although I’ve just written a lot about how it’s not realistic to key footage perfectly with a single click, I have had many projects that have come close. Perhaps all those articles on the internet about lighting greenscreens have soaked into the collective unconscious, as the quality of greenscreen footage I’ve had to work with in recent years has been really good. However- as the quality of the footage increases, so does the quality of the compositing. Even if the footage itself does key with a single click, there are always going to be several more steps involved to improve the quality of the final composite. So perhaps I shouldn’t talk about the myth of the single-click key, but rather the myth of the single-click composite.
In part 3 I’ll discuss compositing in general and show some examples of keying difficult footage, and also reveal the most difficult footage I’ve ever had to key…