Learning Formation of Physically-Based Face Attributes

Ruilong Li^1,2 Karl Bladin¹ Yajie Zhao*¹ Chinmay Chinara¹ Owen Ingraham¹
Pengda Xiang^1,2 Xinglei Ren¹ Pratusha Prasad¹ Bipin Kishore¹ Jun Xing¹ Hao Li^1,2,3

Institute for Creative Technologies¹ University of Southern California² Pinscreen³ *Equal contribution

We introduce a comprehensive framework for learning physically based face models from highly constrained facial scan data. Our deep learning based approach for 3D morphable face modeling seizes the fidelity of nearly 4000 high resolution face scans encompassing expression and identity separation (a). The model (b) combines a multitude of anatomical and physically based face attributes to generate an infinite number of digitized faces (c). Our model generates faces at pore level geometry resolution (d).

Abstract

Based on a combined data set of 4000 high resolution facial scans, we introduce a non-linear morphable face model, capable of producing multifarious face geometry of pore-level resolution, coupled with material attributes for use in physically-based rendering. We aim to maximize the variety of the participant’s face identities, while increasing the robustness of correspondence between unique components, including middle-frequency geometry, albedo maps, specular intensity maps and high-frequency displacement details. Our deep learning based generative model learns to correlate albedo and geometry, which ensures the anatomical correctness of the generated assets. We demonstrate potential use of our generative model for novel identity generation, model fitting, interpolation, animation, high fidelity data visualization, and low-to-high resolution data domain transferring. We hope the release of this generative model will encourage further cooperation between all graphics, vision, and data focused professionals, while demonstrating the cumulative value of every individuals’ complete biometric profile.