The Academy of Motion Picture Arts and Sciences announced that Paul Debevec, associate director, graphics research at the USC Institute for Creative Technologies, Tim Hawkins of LightStage LLC, John Monos of Sony Pictures Imageworks, and Mark Sagar of WETA Digital, will be honored with a Scientific and Engineering Academy Award® for “the design and engineering of the Light Stage capture devices and the image-based facial rendering system developed for character relighting in motion pictures.”
According to the Academy’s citation, “the combination of these systems, with their ability to capture high fidelity reflectance data of human subjects, allows for the creation of photorealistic digital faces as they would appear in any lighting condition.” The award recognizes over ten years of research, development and application of technologies designed to help achieve the goal of photoreal digital actors.
Based on original research led by Debevec at the University of California at Berkeley and published at the 2000 SIGGRAPH conference, the Light Stage systems efficiently capture how an actor’s face appears when lit from every possible lighting direction. From this captured imagery, specialized algorithms create realistic virtual renditions of the actor in the illumination of any location or set, faithfully reproducing the color, texture, shine, shading, and translucency of the actor’s skin.
While the first Light Stage had just one spotlight which spiraled around on a wooden gantry, Light Stage 2 built at USC’s Institute for Creative Technologies featured thirty bright strobe lights on a ten foot semicircular arm which rotated to capture detailed facial reflectance in just eight seconds. In 2002, this process attracted the attention of visual effects supervisor Scott Stokdyk of Sony Pictures Imageworks, who chose it for creating photoreal computer-generated stunt doubles of actors Alfred Molina (“Doc Ock”) and Tobey Maguire (“Spider-Man™”) for the movie Spider-Man™ 2. Mark Sagar, a collaborator on the original research, led the effort to adapt the process for film production. He was soon joined by computer graphics supervisor John Monos on Imageworks’ look development team. The technology was used in nearly 40 shots for the 2004 film which earned an Academy Award® for Best Achievement in Visual Effects.
After Spider-Man™ 2, Mark Sagar transitioned to Peter Jackson’s visual effects company WETA Digital in New Zealand where he oversaw the use of USC’s Light Stage 2 system to record the facial reflectance of actress Naomi Watts for her digital stunt double in Peter Jackson’s King Kong in 2005. Continuing at Sony Imageworks, John Monos led an effort which used Light Stage 2 scans of actor Brandon Routh to create a digital Superman character for the 2006 movie Superman Returns. The film achieved a new high water mark in the realism of virtual actors, with the digital Superman being successfully employed in both action sequences and extended close-up shots. The seamless digital character work helped earn Superman Returns an Academy Award® nomination for Best Visual Effects.
Sony Imageworks subsequently used Light Stage 2, as well as its full-sphere LED-based successors Light Stage 3 and Light Stage 5, to create digital versions of principal actors for Spider-Man™ 3 in 2007 and Hancock in 2008.
In 2008, visual effects company Digital Domain used detailed reflectance information captured with ICT’s Light Stage 5 system to help create a computer-generated version of Brad Pitt as an old man for David Fincher’s The Curious Case of Benjamin Button. The film, which featured the first extended performance of a digitally rendered actor in a feature film, won last year’s Academy Award for Best Visual Effects.
USC ICT’s Light Stage 5 system was most recently employed in the extensive visual effects in James Cameron’s worldwide hit Avatar. Working closely with the visual effects team at WETA Digital, ICT’s Graphics Laboratory digitized the faces of most of the film’s principal cast using a new high-resolution version of their geometry and appearance capture techniques. This innovative technology, housed at ICT’s Marina del Rey campus, precisely captures the shape, shine, color and texture of an actor’s face down to the level of each skin pore, crease, and wrinkle. These detailed scans were used by WETA Digital in their process of creating the film’s photorealistic digital humans and humanoid aliens, which have been lauded as a groundbreaking achievement in the evolution of digital filmmaking.
Through USC’s Stevens Institute for Innovation, the Light Stage technologies have been licensed to LightStage LLC, a Burbank-based company which offers commercial scanning services to the motion picture and interactive entertainment industries. LightStage LLC’s Chief Technology Officer Tim Hawkins was involved in the development of the Light Stage technology beginning with the original research at UC Berkeley and throughout its application in motion pictures as a researcher in the Graphics Laboratory at USC ICT.
Paul Debevec, who is also a research associate professor in the computer science department of USC’s Viterbi School of Engineering, continues to lead ICT’s graphics research program, which has published over 20 peer-reviewed publications involving the Light Stage systems to date.
The Academy’s Scientific and Technical Awards honor the men, women and companies whose discoveries and innovations have contributed in significant, outstanding and lasting ways to motion pictures. The Scientific and Engineering Award will be presented to Debevec, Hawkins, Monos and Sagar at the Academy’s Scientific and Technical Awards Ceremony in Beverly Hills on February 20th, 2010.
Links:
Academy Scientific and Technical Awards Press Release
The Academy Scientific and Technical Awards
The Scientific and Engineering Award (Academy Plaque)
The Light Stages at USC ICT
USC Institute for Creative Technologies
ICT Graphics Laboratory
LightStage LLC
Sony Pictures Imageworks
WETA Digital
