Single/Multi-View 3-D Face Reconstruction for Pose-Robust Face Recognition

One challenge in automated face recognition is matching facial images acquired from different views. The pose variation causes not only misalignment between images but also inconsistency in facial appearance. With extensive study on these two issues, several contributions in single/multi-view 3D face reconstruction for facial pose normalization and pose-robust facial signature generation for pose invariant face matching have been made in this dissertation. The first contribution is a method for single-view 3D face reconstruction. An algorithm based on subspace learning and partial least-squares regression is proposed for reconstructing a sparse 3D facial shape from 2D facial landmarks and a 3D super-resolution algorithm is proposed for recovering a dense 3D facial shape from the estimated sparse shape. The second contribution is an end-to-end method for single-view 3D face reconstruction based on deep neural networks. A 3D facial shape subspace model is utilized for parametric 3D face representation and a deep neural network model is proposed for model parameter estimation from a 2D image. The third contribution is a method for extracting and matching pose-robust facial signatures for pose invariant face recognition. Discriminative feature learning and part-based face representation are combined to enhance the extracted facial features with estimated self-occlusion encodings and create pose-aware facial signatures. The fourth contribution is a method for multi-view 3D face reconstruction based on recurrent neural networks. A neural network model with two long short-term memory (LSTM) layers is proposed for aggregating the contextual identity signal from a facial image set and estimating the parameters for 3D face reconstruction. The fifth contribution is an study on the impact of single/multi-view 3D face reconstruction on the performance of 3D-aided face recognition. With extensive experiments, the performances of the proposed single/multi-view 3D face reconstruction methods are demonstrated to have outperformed the state-of-the-art. In average, the reconstruction error of the proposed methods is 2.6%-31.5% lower than existing methods on three public benchmarks. For 3D-aided face recognition, the proposed facial signature achieves significant improvement over the state-of-the-art facial matchers. In average, the face identification Rank-1 rates are improved by 2 to 19.5 percentage points on two challenging public benchmarks.