We present a practical method for modeling layered facial reflectance consisting of specular reflectance, single scattering, and shallow and deep subsurface scattering. We estimate parameters of appropriate reflectance models for each of these layers from just 20 photographs recorded in a few seconds from a single viewpoint. We extract spatially-varying specular reflectance and singlescattering parameters from polarization-difference images under spherical and point source illumination. Next, we employ direct-indirect separation to decompose the remaining multiple scattering observed under cross-polarization into shallow and deep scattering components to model the light transport through multiple layers of skin. Finally, we match appropriate diffusion models to the extracted shallow and deep scattering components for different regions on the face. We validate our technique by comparing renderings of subjects to reference photographs recorded from novel viewpoints and under novel illumination conditions.
This paper aims to develop a practical appearance model which is in addition easy
to incorporate in existing rendering systems. The detail in the facial appearance
model should be such that full-screen close-ups can be faithfully reproduced.
Additionally, working with live subjects requires fast acquisition to avoid
registration problems, temporal changes in the appearance (e.g., due to sweat or
blood flow), and to enable capture of facial appearance of natural expressions
which are difficult to hold for more than a few seconds.
To achieve these goals, we model facial skin reflectance as a combination of
different layers: specular reflectance, single scattering, and shallow and deep
multiple scattering (Fig. 1). A suitable reflectance or scattering model is
selected for each layer, and parameters are obtained using a single high-resolution
still camera to capture a small set of 20 photographs under environmental and
projected lighting conditions. For each reflectance component, we estimate or
infer high-frequency details such as albedo and normals per pixel based on the
environmental illumination patterns, while modeling lower-frequency BRDF and
scattering behavior per region based on the projected patterns. This allows
for fast acquisition and straightforward processing, while achieving a high
level of realism in the resulting models.
Figure 2: Modeled skin reflectance components.
Figure 3:(a) Polarization difference image under spherical illumination, used for
estimating specular albedo. (b) Cross-polarized image under spherical illumination, used
to measure total scattered albedo. (c) Polarization difference image under directional
illumination, used for estimating the specular lobe shape per region. The image also
includes some polarization preserving non-specular backscattering (which we model
as mostly single-scattering), which can be seen to pick up color from the melanin
in the epidermis. (d) Cross-polarized image under directional illumination, showing
multiple scattering. (e) "direct" component of (d), showing shallow scattering.
(f) "indirect" component of (d), showing deeply scattered light. Note that (d) =
(e) + (f) and that (c) + (d) produces a typical front-lit photograph.