Cosine Lobe Based Relighting from Gradient Illumination Photographs
SIGGRAPH 2009 Poster
Graham Fyffe   
USC Institute for Creative Technologies
Introduction

We present an image-based method for relighting a scene by analytically fitting a cosine lobe to the reflectance function at each pixel, based on gradient illumination photographs. Relit images are computed simply by sampling the cosine lobe reflectance functions for each light source in the novel illumination condition. Realistic results for many materials are obtained using a single per-pixel cosine lobe obtained from just two color photographs: one under uniform white illumination and the other under colored gradient illumination. See figure 1. For materials with wavelength-dependent scattering, a better fit can be obtained using independent cosine lobes for the red, green, and blue channels, obtained from three monochromatic gradient illumination conditions instead of the colored gradient condition. See figure 2 on poster.

Method

We photograph the subject under gradient illumination conditions much like Ma et al. [2007], but instead of computing normals, we analytically fit a cosine lobe reflectance model to the photographs. We explore two cosine lobe reflectance functions. Hemispherical cosine lobes of the form work well for diffuse and specular materials, but fail for broadly scattering materials such as fur. Spherical cosine lobes of the form work well for fur and still produce visually plausible results for diffuse and specular materials. In both models, refers to the axis of the lobe, n and k are constants, and Θ refers to the angle of incident light. Figure 3 compares the two models.

Analysis of Fitted Exponents

Ghosh et al. [2009] directly measure reflectance lobe widths. We obtain the widths from fewer photographs by imposing the cosine lobe model. Discounting occlusion and interreflection, the hemispherical cosine lobe exponent n will be 1 for Lambertian diffuse materials, and higher for specular materials. Inspection of the fitted exponents shows strong agreement with the materials of the actual subject, suggesting that the model is a good fit. Refer to figure 4. For example, the golden body of the cat has exponents ranging from 1.2 up to 6.0 at grazing angles, indicating a specular material with dependency on angle, in this case a gold paint. The red collar, ears, and nose have exponents of 1.1 in the red channel, and higher in the green and blue channels, indicating a dominant diffuse red component with a small specular component in all channels, in this case a semi-gloss red paint. The green bib has exponents of 0.7 in the green channel, and somewhat higher in the red and blue channels, in this case a very soft diffuse green paint with a faint gloss.

Analysis of Relighting Results

Figure 5 shows two novel illumination conditions with ground truth photographs, relighting results using the hemispherical cosine lobe model, and relighting results using the four input photographs as a linear basis for comparison. The linear basis results fail to reproduce the appearance of the different materials of the cat subject. The cosine lobe model, however, produces visually plausible results over a wide variety of materials and illumination conditions, using very few input observations.

Downloads

CVMP 2009 Paper
CosineLobeBaseRelighting_cvmp2009.pdf, (49.7MB)

SIGGRAPH 2009 Poster
CosineLobeBaseRelighting _4ftx3ft_Poster.png, (5.89MB)
CosineLobeBaseRelighting_SIG2009.pdf, (5.27MB)
Footer With Address And Phones