Illumination-invariant Face Recognition



Journal Title

Journal ISSN

Volume Title



Face recognition algorithms have been applied to many smart products, such as smartphones and doorbell cameras with face unlocking. However, face recognition algorithms remain challenges when matching face images with large variations in facial appearance due to poor lighting conditions. The goal of this dissertation is to achieve statistically significant improvement to the performance of face recognition systems using 2D face images that exhibit variations in illumination. Three primary objectives are proposed to achieve this goal, including (i) collect and annotate facial data from various lighting conditions, (ii) develop and evaluate a face recognition algorithm to overcome appearance change due to illumination variations, and (iii) develop and evaluate a face recognition algorithm to overcome perceptual change due to extreme illumination conditions. First, the dissertation presents two face datasets that facilitate the evaluation of face recognition algorithms on illumination variations. Second, a face relighting based data augmentation method is introduced to enrich face datasets with illumination variations. Face recognition algorithms trained with the proposed data augmentation method gain 1.77% improvement in terms of rank-1 identification rate. Third, a low light face enhancement method is presented to tackle the challenge of face recognition under insufficient lighting conditions. By applying the proposed method, the gap of verification rate between extreme low light and neutral light face images is reduced from approximately 3% to 0.5%. Lastly, a cross-spectral NIR-VIS face recognition method is proposed to address the challenge of face recognition in a completely dark environment. Experiments confirmed that the proposed method significantly improves the rank-1 identification rate by 6.7% on the acquired dataset and achieves state-of-the-art performance on popular benchmarks.



illumination, face recognition


Portions of this document appear in: H. Leand I. A. Kakadiaris. “UHDB31: A dataset for better understanding face recognitionacross pose and illumination variation,” In Proc. IEEE International Conference on ComputerVision Workshops, Venice, Italy, Oct. 2017.; H. Le, C. Smailis, L. Shi, and I. A. Kakadiaris, “EDGE20: A cross spectral evaluation datasetfor multiple surveillance problems,” In Proc. IEEE Winter Conference on Applications ofComputer Vision, Snowmass Village, CO, Mar. 2020.; H. Leand I. A. Kakadiaris. “Illumination-invariant face recognition with deep relit faceimages,” In Proc. Winter Conference on Applications of Computer Vision, Waikoloa Village,Hawaii, Jan. 2019.; X. Xu,H. Le, and I. A. Kakadiaris. “On the importance of feature aggregation for facereconstruction,” In Proc. Winter Conference on Applications of Computer Vision, WaikoloaVillage, Hawaii, Jan. 2019.; H. Leand I. A. Kakadiaris. “Semi-supervised low light face enhancement for mobile faceunlock,” In Proc. International Conference On Biometrics, Crete, Greece, Jun. 2019.; X. Xu,H. Le, P. Dou, Y. Wu and I.A. Kakadiaris. “Evaluation of a 3D-aided pose invariant2D face recognition system,” In Proc. IEEE International Joint Conference on Biometrics,Denver, Colorado, Oct 1-4, 2017.