ScreenTech: Disappearing acts 

Whatever you call them - virtual actors, cyber stars, synthespians - digital doubles are becoming a common sight on the big screen. Or at least they would be a common sight if advances hadn’t made them so lifelike that they are now very difficult to spot. “The thing with digital doubles is they tend to disappear,” says Joe Letteri, senior visual effects supervisor at Weta Digital, one of the most in-demand companies in the field. “We’ve done them where no-one knows what they’re seeing.”

Over the past decade, photo-real CG versions of actors have appeared in The Curious Case Of Benjamin Button, Avatar and Tron Legacy, but it was on last year’s Furious 7 that Weta Digital took the process to a new level so the film could be completed after the death of star Paul Walker (see below). It is now routine for actors to be scanned so that VFX artists can create CG doubles for use in dangerous or physically impossible action sequences, to age or give characters a more youthful look, to tone up an actor, or simply to change shots after live-action filming has been completed.

“We do it quite commonly,” says Victoria Alonso, executive vice president, physical production at Marvel Studios. “But only to enhance and not to take over a performance.”

While the technology has advanced dramatically, building a digital double, particularly one good enough for full-face close-ups, is still “incredibly expensive, so it’s not a trivial option”, says The Moving Picture Company’s Adam Valdez, VFX supervisor on The Jungle Book. “As you get closer and closer to perfection, it becomes harder and harder and you can quickly, for very minor reasons, fall into the ‘uncanny valley’ [the effect whereby photo-real characters somehow feel ‘off’].”

In addition to the technology, digital doubles make big demands on VFX wizards. “It’s mostly the artistry,” insists Letteri. “Learning to understand what is essential about a performance and what differs from one actor to another. A lot of work goes into creating the character and a lot goes into getting the character to perform properly on screen.”

VFX experts suggest that with increases in computing power and the arrival of virtual reality pushing the envelope of film technology, far more sophisticated digital doubles are on the horizon. “In the next five years, you’re going to see [digital doubles] working in real time, which will give directors the ability to film things right on set,” says Andrew Egiziano, co-founder of Los Angeles-based CounterPunch Studios, whose proprietary technology has been used to build digital doubles for US TV series and Bollywood features such as action thriller Ek Tha Tiger. “The algorithms will be able to calculate fast enough that someone could be ‘conducting’ a performance backstage that would be emulated perfectly on a set.”

“A lot goes into getting the character to perform properly on screen,” Joe Letteri, Weta Digital

Further out, the experience gained by effects artists could be incorporated into the advancing technology. “We’ve just hit the point where anything is possible with time and money,” says Paul Debevec, chief visual officer at the University of Southern California (USC) Institute for Creative Technologies, who developed the Light Stage system now commonly used to scan actors for digital double creation. But, Debevec adds that “over the next 10 years there is a lot of research and technical engineering to be done in trying to take the lessons that the artists have learnt and building those more into the tools that we’re using”.

Where the rise of digital doubles will leave real-life actors is an intriguing question, one the industry is yet to address fully. Though some US states give actors and their heirs protection, and union agreements have provisions regarding nudity and simulated sex, most actors have little legal control over digital clones. Outside of the top echelon of stars, digital doubles are not yet an issue dealt with in actors’ contracts, says Jonathan Handel, an entertainment and IP lawyer at TroyGould Attorneys and an adjunct professor at the USC Gould School of Law.

“There isn’t any explicit approval right or prohibition on scanning someone’s face and then using it elsewhere,” Handel says. “People are reluctant to bring this up as a negotiating point because if you’re not sure you have the leverage to get the prohibition, it’s better to let the sleeping dog lie and have the contract be silent on the issue.”

Case studies

Batman v Superman: Dawn Of Justice 
This spring’s superhero showdown was the first production to use a portable version of the Light Stage scanning system to take high-res scans of its actors on set. The resulting data was used to make digital doubles that could be combined with live-action footage of the actors in scenes such as the title characters’ brutal punch-up.

The Jungle Book
In Disney’s part live-action, part CG version of the Rudyard Kipling tale, a full digital double version of Mowgli (played for most of the film by Neel Sethi) appears in scenes including the buffalo stampede. Other scenes — such as the river ride with bear Baloo — use partial digital doubles to make physical interaction between Mowgli and the film’s CG animals more convincing.

Captain America: Civil War
Marvel Studios’ latest superhero outing uses digital doubles for its Captain America, Bucky Barnes and Vision characters at various points during the big airport fight sequence. And the scene with Iron Man Tony Stark watching a hologram of his younger self includes a digital double based on footage from actor Robert Downey Jr’s early films.

Furious 7
When Paul Walker died in a car accident midway through the shoot for the Fast & Furious franchise’s seventh instalment, Weta Digital created digital doubles — made using outtakes from the series, scans of Walker’s brothers Cody and Caleb and motion-capture work — that allowed the film to be completed. One such scene has Vin Diesel’s Dominic Toretto bid his friend farewell. Below, bottom, from left to right: a live shot of Paul Walker’s brother Caleb shooting the scene for the physical performance, then compositing archival footage of Walker from a previous film, and then digitally enhancing this to match the lighting.