Design of an Automated System for the Retrieval of Emotional Content in Natural Images

Date

2018-05

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

Emotions often drive us and our behaviors. An ability to reliably and automatically estimate the degree and kind of emotion aroused in a typical individual has widespread applicability. Two dimensions of emotion are key: arousal, or the extent to which a scene excites/calms and its valence, i.e. the extent to which it is pleasant/unpleasant. Past attempts to automate emotion detection in images have failed. Here, we adopt a hybrid, integrated approach that broadly consists of two components: a front-end consisting of a bank of classifiers that recognize the presence of specific semantic content (e.g. person/animal/beach) in the image; a back-end that weights the importance of each semantic category to generate a discrete output of image valence and arousal. Model performance was compared with ground truth, i.e. ratings provided by human subjects: test accuracy was 96.0% on valence (chance=50%) and 92.0% on arousal classification (chance=33%) across the externally validated image set.

Description

Keywords

Emotion capture, Arousal, Valence

Citation