Automatic Content Understanding for Safe and Positive Media Experiences

In this dissertation, we introduce the task of automatic content understanding for media. Digital media products, such as movies, songs, and books, serve not only as sources of entertainment but also as vehicles for knowledge dissemination. While ubiquitous in people's daily lives, media content may not always be suitable for users of all ages. For instance, movies with sexually suggestive language or violent scenes might be inappropriate for children. Policymakers have developed systems like the Motion Picture Association (MPA) film rating system for movies and the Parental Advisory Label (PAL) for music to classify age suitability for media products. Likewise, service providers such as streaming websites invest effort in offering suitability suggestions to their customers. However, current rating systems depend on expert classification, resulting in an expensive and inefficient process. Furthermore, these ratings provide only general age-based suitability labels, offering limited information for users seeking deeper content insights. While media can have an educational impact, leveraging it as a resource for teaching morals and building character, especially for children, can be challenging. For instance, parents may need an appropriate story to teach their children about honesty. Manually sifting through every product to identify those with the desired educational value is impractical, even for experts in the field. We address these shortcomings by automating the media content understanding process through various machine learning (ML)-based formulations using natural language processing (NLP) techniques. We have two standing points in this research: One is protective action for safe experiences: first, we investigate rating movie severity in different age-restricted aspects (such as violence and sex) to provide perceptible level information as a compliment for the general suitability category. Then we expand the scope from movies to music products to assess the song lyrics for not only the risky aspects but also the positive messages. The other is proactive action for positive experiences: we go one step further to study how and to what extent the NLP model can interpret themes and educational values from the literature. Then we study how a story reaches a positive or negative narrative outcome with the moral it intends to convey. All of our efforts will go towards one goal: providing comprehensive information for media products. In line with these research topics, we first formulate media content understanding tasks as machine learning problems and then create benchmark datasets for movies, music, and literature. Based on these benchmarks, we successfully designed, implemented, evaluated, and analyzed novel methods to address the research problems. Our proposed techniques demonstrate superior performance compared to existing methods and pave the way for future methodological exploration. These research outcomes will significantly contribute to the goal of providing people with safe and positive media experiences.