Assessment of Mental Workload by EEG+fNIRS

Due to the high cognitive demands of modern technology on operators, recent studies have focused on quantitative assessment of mental workload (MWL) in order to enhance the performance, safety, and productivity in working environments. MWL monitoring systems have been used broadly in industry, military, and academia. Moreover, they can be very useful in healthcare as well. For instance, by the measurement of the cognitive load of the anesthetist or surgeons during laparoscopic and robot-assisted surgeries, the patient’s safety and the operation efficiency can be effectively improved. Brain activity measurements are highly capable of differentiation of mental states in comparison to other physiological measurements. We studied the capability of a Hybrid functional neuroimaging technique to quantify human MWL. We have used electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) as the imaging modalities with seventeen healthy subjects performing the letter n-back task, a standard experimental paradigm related to working memory. For simultaneous EEG+fNIRS data acquisition, we introduced a state of the art whole-head EEG+fNIRS set up in a single headpiece with 19 EEG and 19 fNIRS channels. We quantified the detection ability of MWL by a fully automated algorithm and compared the characteristics of each uni-modal system vs. Hybrid system. In addition, we obtained a systematic perspective about the effectiveness of features’ sub-types in the quantification of MWL. We also analyzed the spatiotemporal behavior of the brain during changes in the level of MWL. We found a specific temporal pattern for HbO concentration on the forehead. Moreover, we localized the spatial significance of the well-known EEG rhythms on the whole-head by means of multivariate statistical analysis. The Hybrid system had an accuracy that was significantly higher than that of either EEG or fNIRS. Our results suggest that EEG+fNIRS features combined with a classifier are capable of robustly discriminating among various levels of MWL. In addition, Hybrid system outperforms the unimodal systems for each subject, every classification problem, every number of features, and every window size. Based on our findings, EEG+fNIRS should be preferred to only EEG or fNIRS in developing passive BCIs and other applications, which need to monitor users’ MWL.