Emotion Recognition Using Continuous Wavelet Transform and Ensemble of Convolutional Neural Networks through Transfer Learning from Electroencephalogram Signal

Tehran University of Medical Sciences Frontiers in Biomedical Technologies 2345-5837 10 1 2022 10 01 Emotion Recognition Using Continuous Wavelet Transform and Ensemble of Convolutional Neural Networks through Transfer Learning from Electroencephalogram Signal 47 56 Sara Bagherzadeh Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran Keivan Maghooli Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran Ahmad Shalbaf Department of Biomedical Engineering and Medical Physics, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran Arash Maghsoudi Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran 2022 01 18 2022 04 04 Purpose: Emotions are integral brain states that can influence our behavior, decision-making, and functions. Electroencephalogram (EEG) is an appropriate modality for emotion recognition since it has high temporal resolution and is a noninvasive and cheap technique. Materials and Methods:.A novel approach based on Ensemble pre-trained Convolutional Neural Networks (ECNNs) is proposed to recognize four emotional classes from EEG channels of individuals watching music video clips. First, scalograms are built from one-dimensional EEG signals by applying the Continuous Wavelet Transform (CWT) method. Then, these images are used to re-train five CNNs: AlexNet, VGG-19, Inception-v1, ResNet-18, and Inception-v3. Then, the majority voting method is applied to make the final decision about emotional classes. The 10-fold cross-validation method is used to evaluate the performance of the proposed method on EEG signals of 32 subjects from the DEAP database. Results:.The experiments showed that applying the proposed ensemble approach in combinations of scalograms of frontal and parietal regions improved results. The best accuracy, sensitivity, precision, and F-score to recognize four emotional states achieved 96.90%±0.52, 97.30±0.55, 96.97±0.62, and 96.74±0.56, respectively. Conclusion: So, the newly proposed model from EEG signals improves recognition of the four emotional states in the DEAP database. https://fbt.tums.ac.ir/index.php/fbt/article/view/453 https://fbt.tums.ac.ir/index.php/fbt/article/download/453/277