Kakadiaris, Ioannis A.2020-01-07May 20192019-05May 2019Portions of this document appear in: Leng, Mengjun, Panagiotis Moutafis, and Ioannis A. Kakadiaris. "Joint prototype and metric learning for image set classification: Application to video face identification." Image and Vision Computing 58 (2017): 204-213. And in: Leng, Mengjun, and Ioannis A. Kakadiaris. "Confidence-Driven Network for Point-to-Set Matching." In 2018 24th International Conference on Pattern Recognition (ICPR), pp. 3414-3420. IEEE, 2018. And in: Leng, Mengjun, and Ioannis A. Kakadiaris. "Recursive Binary Template Embedding for Face Image Sets." In 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS), pp. 1-8. IEEE, 2018.https://hdl.handle.net/10657/5804A face image set is a group of face images from the same person. In set-based face recognition systems, face image sets are employed either in the gallery, probe, or both for comparisons. Compared with a single face image, an image set provides more information; hence, better performance is expected. However, it also brings a lot of challenges and remains an open problem in real-life scenarios. First, there are large variations within an image set (e.g., poses, expressions, and occlusions). Second, the number of images varies for different image sets. Third, there may be outliers in a set due to misdetection or mistracking. Fourth, the computation and storage costs are very high, especially for large-scale image sets. The goal of this dissertation is to design effective and efficient algorithms in template generation and matching that can represent identity information and take advantage of the within-set variations. The first contribution is a set-based prototype and metric learning algorithm (SPML) that generates compact templates and robust similarity measurements for set-to-set matching. The second contribution is a confidence-driven network (CDN) to quantify the confidence level of images in a set and enhance the point-to-set matching. The third contribution is a confidence prediction network (CPN) that can serve as an add-on module to enhance the performance of a sample-based face recognition system for set-based face recognition tasks. The fourth contribution is an attention-based recursive binary embedding (ARBE) algorithm to extract binary templates for face image sets. The proposed algorithms achieved significant improvements when compared with previous advances.application/pdfengThe author of this work is the copyright owner. UH Libraries and the Texas Digital Library have their permission to store and provide access to this work. UH Libraries has secured permission to reproduce any and all previously published materials contained in the work. Further transmission, reproduction, or presentation of this work is prohibited except with permission of the author(s).Image set classificationFace recognitionPerformance predictionAttention mechanism2020-01-07Thesisborn digital