Improving Set-Based Face Recognition

A face image set is a group of face images from the same person. In set-based face recognition systems, face image sets are employed either in the gallery, probe, or both for comparisons. Compared with a single face image, an image set provides more information; hence, better performance is expected. However, it also brings a lot of challenges and remains an open problem in real-life scenarios. First, there are large variations within an image set (e.g., poses, expressions, and occlusions). Second, the number of images varies for different image sets. Third, there may be outliers in a set due to misdetection or mistracking. Fourth, the computation and storage costs are very high, especially for large-scale image sets. The goal of this dissertation is to design effective and efficient algorithms in template generation and matching that can represent identity information and take advantage of the within-set variations. The first contribution is a set-based prototype and metric learning algorithm (SPML) that generates compact templates and robust similarity measurements for set-to-set matching. The second contribution is a confidence-driven network (CDN) to quantify the confidence level of images in a set and enhance the point-to-set matching. The third contribution is a confidence prediction network (CPN) that can serve as an add-on module to enhance the performance of a sample-based face recognition system for set-based face recognition tasks. The fourth contribution is an attention-based recursive binary embedding (ARBE) algorithm to extract binary templates for face image sets. The proposed algorithms achieved significant improvements when compared with previous advances.