Face Recognition in the presence of variance in Pose, Expression, and Occlusions

Face recognition is a technology in which a computing device either classifies a human identity based on a facial image or verifies whether two images belong to the same subject. The recent advances have achieved remarkable performance when comparing images that are both frontal and non-occluded. However, significant challenges remain in the presence of variations in pose, expression, and occlusions. The goal of this dissertation is to achieve statistically significant improvement in the performance of face recognition systems using 2D images that depict individuals with facial expressions and accessories. Four contributions made in this dissertation can be summarized as follows: (i) a 3D-aided 2D face recognition system with additional evaluation package that is modular, easy to use, and easy to install was designed, implemented, and evaluated. This proposed system can work with the facial images that have variations in head pose as large as 90 degree and improved the face recognition performance by 9% on average when compared with FaceNet on UHDB31 dataset. (ii) two landmark detectors were developed and evaluated on 2D images that are fast and accurate; (iii) feature aggregation learning was proposed for face reconstruction from a single image, which achieved 16% and 10% improvement when compared with the current state-of-the-art on the BU-3DFE and JNU-3D datasets, respectively, and (iv) an occlusion-aware face recognition approach was proposed that improved the generalizability of the facial embedding generator and a graph neural network was designed in an unsupervised manner to adapt the knowledge learned in the image-based scenario to mixed-media set scenario.