A New Method for Measuring Infant's Multimodal Experiences: Combining Head-Mounted Eye-Tracking and Electroencephalogram (EEG) Techniques

Eye-tracking and EEG methods are both widely used methods to understand human behaviors. While eye-tracking studies have provided precise information on the distribution of everyday attention behaviors [1], little is known about the cognitive significance of these behaviors [2]. Though EEG studies offer a glimpse into brain activity, the presence of ocular artifacts has limited the interpretability of relevant neural activities [3]. The combined method allows researchers to obtain a more comprehensive understanding of early attention behaviors and their significance for learning experiences by obtaining increased temporal resolution data. The use of eye-tracking data can provide more precise attention behaviors, such as blinks and gaze shifts, that are associated with brain activity. The use of EEG data can help identify artifacts and noise out of the relevant brain activities. We expect that children’s brain waves will differ when focusing their gaze and maintaining attention on people and objects in the play context. Parent-infant play with six toys for a six-minute play-session with a head-mounted eye-tracker and EEG cap together. Then for data process, we use the Yarbus software to process the parent and child mobile eye-tracking data from the play session. Annotated videos frame-by-frame for child attention on four target regions of interests including objects, parent's hands, child's hand, and parent's face. Finally, we use EEGLAB and MATLAB to process the EEG data. The present study is still in its pilot phase and focuses on characterizing changes in brain activity associated with maintaining meaningful attention.