Person Search Using Identity Attributes

Date

2019-05

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

The goal of this dissertation is to develop and evaluate algorithms and a prototype system to retrieve frames depicting humans with specific identity attributes obtained from a textual description. Solving this problem requires addressing three separate subproblems, namely (i) defining the ontology of the identity and the identity-related attributes, (ii) developing and evaluating algorithms for extracting identity attributes from images, and (iii) developing and evaluating an algorithm for attribute-based person search in databases of image frames. This dissertation presents a list of methods on visual attribute classification and person search that significantly improve the accuracy over previous work. The methods presented tackle key limitations of previous work such as the class imbalance of visual attributes, or the challenge of learning discriminative representations from the textual input. By learning to retrieve the most relevant images of individuals based on textual descriptions, such techniques can have a broader impact in cases of missing children or in surveillance applications. The works introduced in this dissertation are capable of successfully identifying which images contain humans with such characteristics which could reduce dramatically the effort and the time required to identify such information. In each method a detailed overview of the benefits and limitations of each approach is introduced, extensive experimental evaluation and ablation studies are provided to analyze the impact of different modules, and further limitations have been identified that need to be addressed by future work.

Description

Keywords

Visual attributes, Person search

Citation

Portions of this document appear in: Sarafianos, Nikolaos, Xiang Xu, and Ioannis A. Kakadiaris. "Deep imbalanced attribute classification using visual attention aggregation." In Proceedings of the European Conference on Computer Vision (ECCV), pp. 680-697. 2018. And in: Sarafianos, Nikolaos, Theodore Giannakopoulos, Christophoros Nikou, and Ioannis A. Kakadiaris. "Curriculum learning for multi-task classification of visual attributes." In Proceedings of the IEEE International Conference on Computer Vision, pp. 2608-2615. 2017. And in: Sarafianos, Nikolaos, Michalis Vrigkas, and Ioannis A. Kakadiaris. "Adaptive svm+: Learning with privileged information for domain adaptation." In Proceedings of the IEEE International Conference on Computer Vision, pp. 2637-2644. 2017. And in: Sarafianos, Nikolaos, Christophoros Nikou, and Ioannis A. Kakadiaris. "Predicting privileged information for height estimation." In 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 3115-3120. IEEE, 2016. And in: Kakadiaris, Ioannis A., Nikolaos Sarafianos, and Christophoros Nikou. "Show me your body: Gender classification from still images." In 2016 IEEE International Conference on Image Processing (ICIP), pp. 3156-3160. IEEE, 2016. And in: Sarafianos, Nikolaos, Theodoros Giannakopoulos, Christophoros Nikou, and Ioannis A. Kakadiaris. "Curriculum learning of visual attribute clusters for multi-task classification." Pattern Recognition 80 (2018): 94-108. And in: Sarafianos, Nikolaos, Bogdan Boteanu, Bogdan Ionescu, and Ioannis A. Kakadiaris. "3d human pose estimation: A review of the literature and analysis of covariates." Computer Vision and Image Understanding 152 (2016): 1-20.