We describe a process for compositing a live performance of an actor into a virtual set wherein the actor is consistently illuminated by the virtual environment. The Light Stage used in this work is a two-meter sphere of inward-pointing RGB light emitting diodes focused on the actor, where each light can be set to an arbitrary color and intensity to replicate a real-world or virtual lighting environment. We implement a digital two-camera infrared matting system to composite the actor into the background plate of the environment without affecting the visible-spectrum illumination on the actor. The color reponse of the system is calibrated to produce correct color renditions of the actor as illuminated by the environment. We demonstrate moving-camera composites of actors into real-world environments and virtual sets such that the actor is properly illuminated by the environment into which they are composited.
Paul Debevec explains Light Stage 3 by showing some images taken in the graphics laboratory at the USC Institute for Creative Technologies of LS3's first subject, film director Randal Kleiser.
Volunteer Catherine Zandonella steps into the stage and is illuminated by the blue test pattern.
Andy Wenger tests illuminating Catherine by just one of the light sources, and her image from the digital video camera is projected onto the screen.
Andy loads the Grace Cathedral lighting environment into the Light Stage 3 progam and it converts it into its representation as forty-one colored light sources. Catherine is illuminated by the light captured in Grace Cathedral in 1998.
Jamie Waese adjusts the exposure on the camera as Chris Tchou turns on the back light in order for Andy to obtain an alpha matte for Catherine.
Using the matte, Catherine is composited live onto a spinning Grace cathedral background as she is illuminated by the light that was actually there.
The video screen shows the Grace Cathedral light probe image resampled to the 41 light sources and an image of the composite.